Who should be able to deploy bundles into Production Automation node?
A user must be Admin on the Automation node project in order to be able to deploy a bundle there. But Admin permissions in Production seems like too much for an ordinary user to have given the risk of impacts to production assets.
Should deployment then be restricted to a smaller set of users, such as project leads or a dedicated ML support team?
Interested to hear what deployment processes and governance people are using in practice.
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,090 Neuron"A user must be Admin on the Automation node project in order to be able to deploy a bundle there"
Not really correct. Not sure where you got this from. You can certainly have non-admin users doing deployments. You just need to permission all the relevant areas properly. The first project deployment into Automation node always needs a bit more work as you need to set the project permissions in the Automation node. This is because Dataiku does not move the project permissions from the Design node as this will usually be different in the Automation node.
-
Your question should be answered by your Operating Model. Different organizations have different needs on how to use and manage the platform. But maybe there are some useful golden nuggets you can get out of our story.
We have three types of users that have permission to deploy something to the next MLOps stage: a platform admin with God-mode power, a less previleged admin from the business department and a Team Lead.We have two types of workloads, this is how the process goes:
- batch: create a flow in design —> bundle pushed to preprod automation node —> bundle deployed in prod automation node (last step to prod will need approval in govern node)
- real-time: design api in design —> publish api service on deployer —> deploy api to one of the three nodes/stages we have (defined as infrastructures on deployer):
- development api node
- acceptance api node
- production api node
We don't give the admin form the business department global admin permissions, but project level admin permissions. We use a script in the project creation macro to give them project admin rights on every newly created project.
-
This has been my experience over the past few years and I also just set up a new test project to validate it (see below); Dataiku Support also verified that "a user must be a project admin or project owner in order to deploy a bundle to the automation node".
To be clear, here is the scenario I have just tested:
- A new project is created where there will be User A (project lead) and User B (contributor) involved
- User A is the project owner on Design node
- A group is created on Design node to represent the team and both users are members; this group has all project permissions except Admin
- Otherwise, both users just have the typical base user privileges across all nodes
- I have configured the Local Deployer to propagate permissions across the nodes
- User A does the first deployment to an Automation node
- The Local Deployer project is set up with User A as Admin and the group having all but Admin
- Due to permissions propagations, User A is automatically the project owner of the project on the Automation node, and the group has the same project permissions as Design node (i.e. everything but Admin)
- User B can create a new bundle and publish it to the Local Deployer, but they cannot deploy the bundle on the Automation node, they get an Action Forbidden error.
- The only solution I have found (and what Dataiku Support recommends) is for all users who need to deploy on the Automation node, need to have Admin permission on the project on the Automation node
Not sure how you have managed to achieve this - what do you mean by "you need to set the project permissions in the Automation node"? Are you actually giving users Admin permission here in order to be able to deploy, when they're not the project owner?
-
Thanks Jonathyan, that model makes a lot of sense. This does seem ideal, as an "ordinary" user who is just contributing to the project probably shouldn't have the ability to deploy to production due to the risk. We do plan to think through our ML operating model and find something that works for us given our use cases and resourcing.
