Sample Python code to access GIT in Dataiku

sj0071992
Sample Python code to access GIT in Dataiku

Hi Team,

 

Suppose I am using Jenkins to Create my Bundle of DESIGN Node but my code is available on the GIT repository.

So while creating the Bundle how can I pass the GIT repository link as a parameter to my Job.

 

Thanks in Advance

0 Kudos
6 Replies
CoreyS
Dataiker Alumni

Hi, @sj0071992, thank you for posting! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if youโ€™ve tried any fixes already?This should lead to a quicker response from the community.

In terms of your specific ask, here are a number of resources that you may find helpful:

I hope this helps!

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as โ€˜Accepted Solutionโ€™ to help others like you!
0 Kudos
sj0071992
Author

Hi Corey,

 

Thanks for the link. Actually, I am following the same link but something is missing in this. The approach is mostly covers the Visual Recipe but my concern is regarding the Code recipes as well. 

Let me Explain

 

Suppose I build one flow on DESIGN Node and I want to publish it in PREPROD/PROD with the help of Jenkins pipeline. My Workflow consist of both Visual and Code Recipes.

The code recipes are having code with is in GIT repository.

In Common scenarios we made a connection with GIT using "Libraries" section under Code under "</>" section

 

But how we are going to pass this in Jenkins Job?

for example, creating Bundles requires HOST and APIKEY(password) to connect with Dataiku. How we can pass GitHub link so that it will connect with Repository to fetch all the Code used in Code Recipe

 

I hope you got my concern.

0 Kudos
CoreyS
Dataiker Alumni

Bundles are consistent packaging of a project. So if you are using project libraries, be it managed locally in DSS or cloned from GitHub, those project libraries will be included in the bundle and restored on the Automation node once the bundle is deployed.

You can test it manually if you want to see that. Create a project library in DESIGN, create a new bundle, deploy this bundle and you will see that, on the Automation node, the library is there.

I hope this helps!

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as โ€˜Accepted Solutionโ€™ to help others like you!
0 Kudos
sj0071992
Author

Hi, 

 

This Would be helpful.

 

Just a small question "How to parameterize the git URL?" What if my PREPROD/PROD GitHub URL is different than what to do?

I want to migrate my Codebase from DEV to TEST/PROD then how can i use this codebase in PREPROD/PROD

Any snippit?

Thanks,

0 Kudos
sj0071992
Author

Any Update on this?

0 Kudos
AlexT
Dataiker

Hi,

Setting remote git URL is not possible via the Python APIs, neither is using project/global variables within the URL. You will need to change this within Version Control in the UI for each environment. 

The actual remote URL is stored in  dss_home/config/projects/PROJECT_ID/.git/config

url = https://github.com/..../.....git

If you do want to programmatically change you do edit the above file directly but generally would not be advised.  

 

 

0 Kudos