Want to Stop Rebuilding "Expensive" Parts of your Flow? Explicit Builds are the Answer!READ MORE

"Update from GIT" via scenario ?

Fred_Ruelle
Level 1
"Update from GIT" via scenario ?

Hello DataIKU community,

I wonder if it is possible to update library from GIT via a scenario?

I tried a step "Update GIT Reference", but this do not seem to pull my external GIT library.

 

Any tip?

 

Thanks a lot 

0 Kudos
2 Replies
SarinaS
Dataiker
Dataiker

Hi @Fred_Ruelle,

It indeed is possible to update your project library git references from a scenario. Is it possible that your git branch selected in DSS is not the branch that was updated outside of DSS? That could cause a discrepancy in what you think might get updated in DSS, if you are looking at a different git branch. 

I'll outline what I see when I make an update to a project library git reference outside of DSS, and then run the scenario step "Update Git reference". If you are taking any different steps, please let me know the steps you end up taking. 

Here is a simple project library that I imported from git. Note that I selected the branch main when I set up the git reference, so the only time I'll get an update is if the branch main is updated. 

Screen Shot 2022-05-04 at 4.09.09 PM.png

Outside of DSS, I make a modification to my library file to add a function. Note that I am also on branch main, which is the branch that contains the update and is also the branch selected in DSS. 

Screen Shot 2022-05-04 at 4.09.51 PM.png
Then I run a scenario with the step you reference:

Screen Shot 2022-05-04 at 4.16.16 PM.png

Once the step finishes, if I check out my Last runs > View step log, we can see the git references getting updated: 

[2022/05/04-16:10:20.266] [FT-PullGitReferencesFutureThread-EF771FQx-1032] [DEBUG] [com.dataiku.dip.transactions.git.CommitQueue]  - Executing Git commit:
[PROJECT] [SARINATESTFOLDER] [admin] Git references update for SARINATESTFOLDER : 
 - /projects/SARINATESTFOLDER/lib/sarinagit
 - /projects/SARINATESTFOLDER/lib/external-libraries.json
 - /projects/SARINATESTFOLDER/lib/sarinagit/__init__.py
 - /projects/SARINATESTFOLDER/lib/sarinagit/sarina_python_lib.py


And then if I navigate back to my project library, I can see the latest code pulled into DSS:

Screen Shot 2022-05-04 at 4.10.50 PM.png

If you want to double check your selected branch, you can click on the `...` next to your git library reference, and click on "Edit git reference". This will show the selected branch (and allow you to switch branches) under the "Checkout" field. 

I hope that information is helpful. Please let me know if the steps above are not working as expected for you, or if you are referring to a different workflow!

Thanks,
Sarina

0 Kudos
Fred_Ruelle
Level 1
Author

Hello @SarinaS ,

 

Thanks a lot to have taken time to answer to my question and to do this quick test for me !

Apparently something was wrong in my setup, indeed. Now it is working like a charm !

 

Have a nice day !