Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
How to create containerized execution using dsscli or python API
Added on February 27, 2025 2:06PM
Likes: 0
Replies: 3
DrissiReda
Registered Posts: 67 ✭✭✭✭✭
Hello,
I'm trying to automate the deployment of Dataiku so I need to configure a few things to be set up automatically.
I figured out how to do that for users and groups, for fine grained permissions I use the python API.
I still need to figure out how to do this for creating containerized executions and code envs I didn't find much in the documentation.
Dataiku version used is 13 but I cannot set that in the community forum.
Operating system used: ubuntu
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,396 NeuronThere are a number of things that you can't do using the official Python APIs among them creating or updating containerized execution configurations or creating Code Studio Templates as two examples. While unsupported and against Dataiku advice we have for long time modified Dataiku configuration files ourselves using Ansible and Python. Once you go this road you need to acknowledge you will need to retest your configuration code for every Dataiku upgrade as there is no guarantee the config files won't be changed breaking your code. Containerized execution configurations are stored in the main configuration file: DATADIR/config/general-settings.json. It is recommended that you don't change this file with DSS running but if you do you will need to clear the config cache with this command:
dsscli config-cache-invalidate --path /
So your call really, it can be done and it works well but up to you if you want to go this path.
-
A while ago that's what I used to do, modify files like general-settings.json connections_conf.json kernel_conf.json etc
I wanted to do things a bit more properly now. I am well aware of the robustness limitation and that retesting after every upgrade is mandatory. Thanks for confirming I'll just have to modify the general-settings.json file for the containerized exec. -
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,396 Neuron
Yeah the options are limited in this case. It's either custom manipulation of config files or no automation. I side with custom manipulation as well. Developers tend to do hacky things when there isn't a proper way of doing things…