Sample Python code to access GIT in Dataiku

sj0071992
sj0071992 Partner, Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2022, Neuron 2023 Posts: 131 Neuron

Hi Team,

Suppose I am using Jenkins to Create my Bundle of DESIGN Node but my code is available on the GIT repository.

So while creating the Bundle how can I pass the GIT repository link as a parameter to my Job.

Thanks in Advance

Answers

  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    Hi, @sj0071992
    , thank you for posting! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if you’ve tried any fixes already?This should lead to a quicker response from the community.

    In terms of your specific ask, here are a number of resources that you may find helpful:

    I hope this helps!

  • sj0071992
    sj0071992 Partner, Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2022, Neuron 2023 Posts: 131 Neuron

    Hi Corey,

    Thanks for the link. Actually, I am following the same link but something is missing in this. The approach is mostly covers the Visual Recipe but my concern is regarding the Code recipes as well.

    Let me Explain

    Suppose I build one flow on DESIGN Node and I want to publish it in PREPROD/PROD with the help of Jenkins pipeline. My Workflow consist of both Visual and Code Recipes.

    The code recipes are having code with is in GIT repository.

    In Common scenarios we made a connection with GIT using "Libraries" section under Code under "</>" section

    But how we are going to pass this in Jenkins Job?

    for example, creating Bundles requires HOST and APIKEY(password) to connect with Dataiku. How we can pass GitHub link so that it will connect with Repository to fetch all the Code used in Code Recipe

    I hope you got my concern.

  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    Bundles are consistent packaging of a project. So if you are using project libraries, be it managed locally in DSS or cloned from GitHub, those project libraries will be included in the bundle and restored on the Automation node once the bundle is deployed.

    You can test it manually if you want to see that. Create a project library in DESIGN, create a new bundle, deploy this bundle and you will see that, on the Automation node, the library is there.

    I hope this helps!

  • sj0071992
    sj0071992 Partner, Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2022, Neuron 2023 Posts: 131 Neuron

    Hi,

    This Would be helpful.

    Just a small question "How to parameterize the git URL?" What if my PREPROD/PROD GitHub URL is different than what to do?

    I want to migrate my Codebase from DEV to TEST/PROD then how can i use this codebase in PREPROD/PROD

    Any snippit?

    Thanks,

  • sj0071992
    sj0071992 Partner, Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2022, Neuron 2023 Posts: 131 Neuron

    Any Update on this?

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker

    Hi,

    Setting remote git URL is not possible via the Python APIs, neither is using project/global variables within the URL. You will need to change this within Version Control in the UI for each environment.

    The actual remote URL is stored in dss_home/config/projects/PROJECT_ID/.git/config

    url = https://github.com/..../.....git

    If you do want to programmatically change you do edit the above file directly but generally would not be advised.

Setup Info
    Tags
      Help me…