Design Node Launch Failure

ARATA
ARATA Partner, Registered Posts: 12 Partner

I have set up a Dataiku on AWS and configured it to automatically start/stop the Fleet Manager and design nodes using AWS functionality.

Yesterday, both the EC2 instances and DSS started successfully.

However, today, while the EC2 instances started successfully, the DSS failed to start.

For reference, the version of Dataiku being used is 14.0.0.

Issue

First, upon checking the Fleet Manager, the design node’s status was marked as InErr Looking into the Events logs, the following message was recorded:

pi-initialization-failed - 2025-07-08 09:40:17  
{  
  "physicalInstanceId": "pi-xxx",  
  "ec2InstanceId": "i-xxx",  
  "error": {  
    "errorType": "<class 'dataikufmagent.exception.FmAgentException'>",  
    "message": "Location /data is mounted on unexpected device /dev/nvme0n1 instead of requested device /dev/nvme1n1",  
    "detailedMessage": "<class 'dataikufmagent.exception.FmAgentException'>: Location /data is mounted on unexpected device /dev/nvme0n1 instead of requested device /dev/nvme1n1",  
    "stackTrace": []  
  }  
}

Next, I attempted to stop the design node and re-provision it.

The re-provisioning failed, and the following message was logged in the Events:

pi-initialization-failed - 2025-07-08 10:02:42  
{  
  "physicalInstanceId": "pi-xxx",  
  "ec2InstanceId": "i-xxx",  
  "error": {  
    "errorType": "<class 'dataikufmagent.exception.FmAgentException'>",  
    "message": "Device has unexpected filesystem",  
    "detailedMessage": "<class 'dataikufmagent.exception.FmAgentException'>: Device has unexpected filesystem",  
    "stackTrace": []  
  }  
}

After repeating the stop and re-provisioning actions several times, the DSS finally started successfully.

Questions

  1. Was my response and action in handling the error messages described above appropriate?
  2. What could be the possible root causes of this issue?

Additional Information

For reference, the design node in question has previously had its data volume replaced on AWS (detaching and attaching a different data volume).

Thank you in advance for your assistance.

Operating system used: almalinux (9.6)

Tagged:

Answers

Setup Info
    Tags
      Help me…