How to connect to remote private GIT repository

Solved!
epsi95
Level 3
How to connect to remote private GIT repository

I am trying to import code from my private git repository in `Library Editor` (I can't make it public since the information is sensitive). As a result it is throwing me error

 

 Remote references can't be fetched
Branches and tags could not be fetched from remote Git, caused by: IOException: Process failure, caused by: IOException: Process execution failed (return code 128) (error=remote: Support for password authentication was removed on August 13, 2021. Please use a personal access token instead. remote: Please see https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information. fatal: Authentication failed for 'https://github.com/xxxx/' )
Please check the flow for more details

More information on this error in DSS documentation

Additional technical details: HTTP code: 500, Code: ERR_GIT_LIST_REFS_FAILED, type: com.dataiku.dip.exceptions.CodedException

 

How can I solve this issue, I have the access token with me. How to tell dataiku to use it?

0 Kudos
1 Solution
AlexT
Dataiker

Hi @epsi95 ,

To authenticate with your private git repo you can use preferably use an SSH key.  https://doc.dataiku.com/dss/latest/collaboration/git.html

For this to work you will need to:

1) Add the DSS user  SSH key to the github account that will be used. 

2) Configure Group rules under Settings - Git in DSS , for multiple ssh key see this section

Once that is configured you can simply use the ssh git path e.g git@github.com:user/remote-git-test.git and pull the respective files. 

Just to mention this is a one-time operation for project libraries so if you need to update these it has to be done with another pull.  

If the SSH key is not an option you can try caching credentials in custom  .gitconfig and specify this in the Group Rules for GIT in DSS. 

 https://stackoverflow.com/questions/46645843/where-to-store-the-personal-access-token-from-github 

 

View solution in original post

5 Replies
CoreyS
Dataiker Alumni

Hi @epsi95 while you wait for a more complete response, if you haven't already accessed them, here are some resources you may find helpful:

I hope this helps!

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as โ€˜Accepted Solutionโ€™ to help others like you!
AlexT
Dataiker

Hi @epsi95 ,

To authenticate with your private git repo you can use preferably use an SSH key.  https://doc.dataiku.com/dss/latest/collaboration/git.html

For this to work you will need to:

1) Add the DSS user  SSH key to the github account that will be used. 

2) Configure Group rules under Settings - Git in DSS , for multiple ssh key see this section

Once that is configured you can simply use the ssh git path e.g git@github.com:user/remote-git-test.git and pull the respective files. 

Just to mention this is a one-time operation for project libraries so if you need to update these it has to be done with another pull.  

If the SSH key is not an option you can try caching credentials in custom  .gitconfig and specify this in the Group Rules for GIT in DSS. 

 https://stackoverflow.com/questions/46645843/where-to-store-the-personal-access-token-from-github 

 

Marlan

Hi @AlexT,

I am working on pretty much the same task, that is connecting to an Azure DevOps git repo from DSS hosted on Azure. Seems like an SSH connection is a good approach and so working on getting that set up.

A question about your note though. Is your step 2 required? I thought that was optional and only necessary if we wanted to to limit access to repos by group.

Thanks!

Marlan

P.S., the link for step 1 as some extra text (%C2%A0) that make it not work.

AlexT
Dataiker

Hi,

Indeed similar steps would apply for Azure Devops

For step 2 you will need to at least use .* or the regex suggested and allow your groups git access.

Otherwise, you will get an error like "You are not allowed to use remote Git commands". If Allow git for the groups is not enabled. Refer to screenshot below.

Screenshot 2021-10-22 at 22.18.59.png

Thanks,

VitaliyD
Dataiker

Hi,

In addition to what already said, if you didn't come across it yet, you can find detailed usage guidance on cloning a Library from a Remote git repository in our knowledge base: https://knowledge.dataiku.com/latest/courses/advanced-code/work-environment/library-pull.html#clonin...

Best,

Vitaliy