Disable selection of backend in recipe compute configuration
How to set containerized environment as the only option available for the user?
We wanted to disable the selection of backend from the options so that none of the jobs submitted will go to server backed.
Operating system used: Linux RHEL
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptionsI don’t think restricting this is currently possible but I will be glad to be proved wrong as we may want to do this in the very near future. The best alternatives I can think of are:
- Use the Python API to programmatically and regularly move recipes to execute in containers
- Implement cgroups with a very low CPU allowance so users are forced to select containers as cgroups won’t allow them to run any recipe locally
Makes sense?
-
We already kept a lower configuration for cgroups to control the backend server access. But this was needed when environment is getting more and more users added. Eg : offload every ones compute to AKS so that backend servers & application performance/availability not impacted.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptions
Well as I said I don't you can restrict the list. However you are able to set a default cluster and execution config in Settings > Compute & Scaling > Containerized Execution. This will basically be applied by default to all projects and recipes unless the user customises the settings since the default project setting for cluster selection is "Use DSS global cluster settings" and for exec config "Inherit instance default". Feel free to raise a Product Idea to be able to restrict the execution of recipes in the DSS backend server.