Automate Cgroup Limits - Resource Control

mohdwaqar

Hi Team,

I have setup cgroup limits (Path Template and limits ) in resource control on one of our instance and I want to replicate this to all our existing instance and the new installations.

Is there a way to automate this ?

Thanks in advance!!

---

Operating system used: Centos 7.9, RHEL 8, ROcky Linux 8.8

VincentRD

Hello,

For such automation tasks, I suggest using the Python API. You would use it to manipulate the general settings and enter your cgroup-specific configuration.

You can find more information on doing this in the developer guide . It's the get_general_settings entry point you are interested in.

It might not be obvious from the doc but the general scheme to update an instance setting is the following (here for flow limits):

# Update the Flow limits
client = dataikuapi.DSSClient(dku_url, api_key)
settings_obj = client.get_general_settings()
settings = settings_obj.get_raw()
settings['maxRunningActivities'] = new_limit
settings_obj.save()

For cgroups, it's a nested dictionary under the general settings 'cgroupSettings' key. You'll have to experiment a bit to set it to what you want. I suggest setting the right value in the UI and then using the Python code to dump the configuration before trying to use Python to set it programmatically.

HTH

Turribeach

To automate Dataiku deployment and configuration you can use Cloud Stacks:

https://knowledge.dataiku.com/latest/admin-deploying/cloud-stacks/index.html

If you are not on Cloud Stacks you have to do it yourself.

mohdwaqar

Thanks for the response!

I am not using cloud, Is configuring cgroups limits manually the only