Sample Python code to access GIT in Dataiku
Hi Team,
Suppose I am using Jenkins to Create my Bundle of DESIGN Node but my code is available on the GIT repository.
So while creating the Bundle how can I pass the GIT repository link as a parameter to my Job.
Thanks in Advance
Answers
-
CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭
Hi, @sj0071992
, thank you for posting! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if you’ve tried any fixes already?This should lead to a quicker response from the community.In terms of your specific ask, here are a number of resources that you may find helpful:
- Building a Jenkins pipeline for API services in Dataiku DSS
- Building a Jenkins pipeline for Dataiku DSS with Project Deployer
- Building a Jenkins pipeline for Dataiku DSS without Project Deployer
- Building CI/CD pipelines for Dataiku DSS
I hope this helps!
-
sj0071992 Partner, Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2022, Neuron 2023 Posts: 131 Neuron
Hi Corey,
Thanks for the link. Actually, I am following the same link but something is missing in this. The approach is mostly covers the Visual Recipe but my concern is regarding the Code recipes as well.
Let me Explain
Suppose I build one flow on DESIGN Node and I want to publish it in PREPROD/PROD with the help of Jenkins pipeline. My Workflow consist of both Visual and Code Recipes.
The code recipes are having code with is in GIT repository.
In Common scenarios we made a connection with GIT using "Libraries" section under Code under "</>" section
But how we are going to pass this in Jenkins Job?
for example, creating Bundles requires HOST and APIKEY(password) to connect with Dataiku. How we can pass GitHub link so that it will connect with Repository to fetch all the Code used in Code Recipe
I hope you got my concern.
-
CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭
Bundles are consistent packaging of a project. So if you are using project libraries, be it managed locally in DSS or cloned from GitHub, those project libraries will be included in the bundle and restored on the Automation node once the bundle is deployed.
You can test it manually if you want to see that. Create a project library in DESIGN, create a new bundle, deploy this bundle and you will see that, on the Automation node, the library is there.
I hope this helps!
-
sj0071992 Partner, Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2022, Neuron 2023 Posts: 131 Neuron
Hi,
This Would be helpful.
Just a small question "How to parameterize the git URL?" What if my PREPROD/PROD GitHub URL is different than what to do?
I want to migrate my Codebase from DEV to TEST/PROD then how can i use this codebase in PREPROD/PROD
Any snippit?
Thanks,
-
sj0071992 Partner, Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2022, Neuron 2023 Posts: 131 Neuron
Any Update on this?
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,212 Dataiker
Hi,
Setting remote git URL is not possible via the Python APIs, neither is using project/global variables within the URL. You will need to change this within Version Control in the UI for each environment.
The actual remote URL is stored in dss_home/config/projects/PROJECT_ID/.git/config
url = https://github.com/..../.....git
If you do want to programmatically change you do edit the above file directly but generally would not be advised.