Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Our team is currently using both a CI/CD tool for automating deployments through Project Deployer API's and a remote git repository for version control. We would like to go one step further and implement automated deployments that are triggered when a merge to the master branch in the remote git repository occurs. The immediate issue is that when a merge occurs in the master branch of the remote git repository (not in the Dataiku project on the local Dataiku server), the CI/CD tool is triggered to run a Dataiku deployment on that project, but there is no apparent way to first automatically pull the master branch of the remote git repository into the project via API's or some other method.
Current state, in order for someone to deploy the newest change to master, a user would need to go into the Dataiku UI for that project and make a pull on the master branch of the remote git repository into their project before either manually or by some other means triggering the CI/CD tool's deployment process. Ideally there would be some API call in Dataiku for us to pull the remote git repository on the project's branch before carrying out the deployment via the Project Deployer API. Any recommendations on how to perform this with the current state of Dataiku DSS 9.0? Is there an API available for this? Should this be a feature request?
One idea we had was setting up the CI/CD tool to monitor the Dataiku server's local git repository. This is however a rather invasive approach that requires modification to the Dataiku server that would enable it to 'serve up' its local git repo. Ideally there would be a built-in feature of Dataiku to handle this kind of automation.
Ideal Process
Hi,
There is indeed at the moment no API for triggering a pull of the remote branch, and this is indeed a feature request that is in our backlog.
However, the Git repository for each project is a normal Git repository, so you can go into the "DATADIR/config/projects/YOUR_PROJECT" and run normal git commands, including "git pull". While this requires a bit more plumbing (because you need to run a shell command), this should do what you need.
Hope this helps,
Hi,
There is indeed at the moment no API for triggering a pull of the remote branch, and this is indeed a feature request that is in our backlog.
However, the Git repository for each project is a normal Git repository, so you can go into the "DATADIR/config/projects/YOUR_PROJECT" and run normal git commands, including "git pull". While this requires a bit more plumbing (because you need to run a shell command), this should do what you need.
Hope this helps,
Hi Clรฉment,
Thanks for your reply! We did not consider simply pulling the repository on the backend. That may be our best option.
If this is in fact a feature request on your backlog, is there a place where I can promote it?
Thank you,
Jace
Hi all,
How would you go about implementing this?
We're having a similar issue where we want to automatically create a bundle after the completion of a pull request (into main). Somehow you want to kick of a series of commands that perform GIT actions on the Design Node. A part of this script would be checking out to the main branch, thereby changing the referenced code on the Design Node. So, I'm assuming that including this series of commands into a scenario is a no go (because you're switching the branch in the scenario itself). However, how then do I run a series of commands from an external source (say a pipeline) on the Design Node? Maybe I'm missing something :).
Best,
Jasper
Hey Did you find the solution for this problem?