Automate Cgroup Limits - Resource Control
Hi Team,
I have setup cgroup limits (Path Template and limits ) in resource control on one of our instance and I want to replicate this to all our existing instance and the new installations.
Is there a way to automate this ?
Thanks in advance!!
Operating system used: Centos 7.9, RHEL 8, ROcky Linux 8.8
Best Answer
-
Hello,
For such automation tasks, I suggest using the Python API. You would use it to manipulate the general settings and enter your cgroup-specific configuration.
You can find more information on doing this in the developer guide . It's the get_general_settings entry point you are interested in.
It might not be obvious from the doc but the general scheme to update an instance setting is the following (here for flow limits):
# Update the Flow limits client = dataikuapi.DSSClient(dku_url, api_key) settings_obj = client.get_general_settings() settings = settings_obj.get_raw() settings['maxRunningActivities'] = new_limit settings_obj.save()
For cgroups, it's a nested dictionary under the general settings 'cgroupSettings' key. You'll have to experiment a bit to set it to what you want. I suggest setting the right value in the UI and then using the Python code to dump the configuration before trying to use Python to set it programmatically.
HTH
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,043 Neuron
To automate Dataiku deployment and configuration you can use Cloud Stacks:
https://knowledge.dataiku.com/latest/admin-deploying/cloud-stacks/index.html
If you are not on Cloud Stacks you have to do it yourself.
-
Thanks for the response!
I am not using cloud, Is configuring cgroups limits manually the only