Extend the "Rebuild Code Studio templates" option to non-admins when updating a code environment

Tanguy

I was pleasantly surprised to discover the "Rebuild Code Studio templates" option in the "Containerized Execution" settings of a code environment. This feature enables the rebuilding of Code Studio templates that rely on a given code environment, effectively killing two birds with one stone.

However, after investigating with Dataiku Support, I learned that this option is currently restricted to admins. As a result, even if a non-admin user has the necessary permissions to update a Code Studio template and its associated code environment, they cannot use this feature. Instead, they must update the code environment first and then rebuild the Code Studio templates separately, requiring two steps.

According to Dataiku Support, this limitation exists because non-admin users cannot view Code Studio template usage via the "Compute Usage" section in the code environment.

I struggle to understand why this functionality is restricted to admins. It seems logical to extend this capability to non-admin users who have the appropriate permissions to update both a code environment and a Code Studio template linked to it.

Turribeach

I would certainly agree with you. This limitation seems silly. We have not enabled automatic rebuilds of Code Studio Templates when Code Envs they depend on change because of concerns around the space these templates rebuilds will take in the local file system. Some of our templates are very large (20 GB) and a user making changes in a code environment could easily end up filling the local disk. Dataiku does not clean up Code Studio Templates images from the local docker cache after they are uploaded to the container repository which I think it's not really good practice since these images have no use in the local file system. We have therefore disabled the automatic rebuild for now as we build tooling to purge the old Code Studio Templates images to avoid risking filling up the local disk. Having said it was in our plans to use this functionality to allow users to self-serve updating Code Studio Templates.

One option to work around this limitation would be to have a process that runs regularly and checks the last modified time of a code environment and triggers the impacted Code Studio Template rebuilds with an admin API. You can retrieve the code envs attached to a Code Studio Template via the Python API. But obviously you will need to run this process frequently to allow for a more seemless experience. An alternative could be to develop a pipeline in another tool (we use Jenkins) and ask the user to run it after updating a code environment. This removes the need of running the process regularly to check what code envs have been changed and would allow the pipeline to notify the user when the template rebuild is completed which the GUI is not good at. But I guess that will be almost the same as rebuilding the template via the GUI aside from the fact that an automated process would be able to figure out the impacted templates and build them programmatically.

Tanguy

In our case I think the admins have scheduled a batch that pushes the images to AWS ECR and removes them locally.