How to shrink an over-extended disk by Fleet manager
Hello,
We manage a Fleet Manager and Design Node deployed using "Cloud Stacks for AWS".
As a result of generating a large number of container images such as code envs and code studio templates, the disk size of Design (EBS gp2) became enormous. By deleting old DSS versions and unused images, it was able to free up a significant amount of disk space. Fleet manager could extend the disk size, but cannot shrink them. The size of EBS in AWS has a significant impact on costs, so we want to take immediate action.
Does anyone tried to shrink disk size managed by FM? If yes, tell me how to do it?
Operating system used: AlmaLinux (8.10)
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,467 NeuronNote this an AWS issue not a Dataiku issue:
-
裕也 Partner, PartnerApplicant, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 17 Partner
No, this is also an issue with Dataiku, I guess.
The reason why we cannot determine whether a generally, well-known method, which contain reduction the disk, will work without causing any problemn in DSS process.
In concrete, we don't know which path and files should sync to new disk. (only dss_data/ path?) -
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,467 Neuron
I mean that the problem is that the OS doesn’t support shrinking volumes so it’s not Dataiku who prevents you from doing it. Ideally you should have a separate volume for DSS. That way you can easily resize the volume as needed without affecting the OS. All of the files of the volume should be copied. DSS should be stopped before copying the files.
-
裕也 Partner, PartnerApplicant, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 17 Partner
By attaching a new smaller disk and synchronizing the data, we were able to reduce the disk size.
However, when we re-provisioned from Fleet Manager, it attempted to reattach the old large disk instead of the smaller disk. We believe this is because Fleet Manager manages which EBS to attach.
How can we change this? -
fleet manager에서 관리하는 postgre database에 해당 ebs정보가 들어있습니다 해당 값을 작은 디스크의 ebs id로 변경하면 작은 디스크를 바라 볼 것으로 예상됩니다.
해당 작업은 dataiku측에 문의해서 진행하시면 좋을 것같습니다.
fleetmanger상에서 볼륨 사이즈를 조정하는 ui가 있는것으로 아는데 줄이는 것은 안되나요???
