Automate Cgroup Limits - Resource Control

[mohdwaqar]

Hi Team, 

I have setup cgroup limits (Path Template and limits ) in resource control on one of our instance and I want to replicate this to all our existing instance and the new installations.

Is there a way to automate this ?

Thanks in advance!!

---

Operating system used: Centos 7.9, RHEL 8, ROcky Linux 8.8

[Turribeach]

To automate Dataiku deployment and configuration you can use Cloud Stacks:

https://knowledge.dataiku.com/latest/admin-deploying/cloud-stacks/index.html

If you are not on Cloud Stacks you have to do it yourself. 

[mohdwaqar]

Thanks for the response!

I am not using cloud, Is configuring cgroups limits manually the only

[VincentRD · Accepted solution]

Hello,

For such automation tasks, I suggest using the Python API. You would use it to manipulate the general settings and enter your cgroup-specific configuration.

You can find more information on doing this in the developer guide . It's the get_general_settings  entry point you are interested in.

It might not be obvious from the doc but the general scheme to update an instance setting is the following (here for flow limits):

# Update the Flow limits
client = dataikuapi.DSSClient(dku_url, api_key)
settings_obj = client.get_general_settings()
settings = settings_obj.get_raw()
settings['maxRunningActivities'] = new_limit
settings_obj.save()

 For cgroups, it's a nested dictionary under the general settings 'cgroupSettings' key. You'll have to experiment a bit to set it to what you want. I suggest setting the right value in the UI and then using the Python code to dump the configuration before trying to use Python to set it programmatically.

HTH