Encountering a problem, trying to create a new branch

Harald
Harald Registered Posts: 3 ✭✭

In Version Control, I tried to create a branch while duplicating the project.

However i encounter the following problem:

There seems to be a reference to an old code env, that doesn't exist anymore. However, I cannot find where the reference is. I already tried to manually set all notebook envs, as well as python-recipe envs to proper, existing environments.

Now I don't really know anymore, where to look to remove the reference.

I would kindly ask for some help :)

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,162 Neuron

    Create the code env again, then go to the Code Env, Usages and click on Compute Usage:

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,162 Neuron

    Next time before deleting a code env do check the usages so you can prevent this situation but in reality Dataiku should be warning users of this really.

    Finally not sure why you want to do some branching since Dataiku's Git integration is very weak and once you branch you can't really merge back to master. So it's much better to duplicate a project or to work on a separate flow zone.

    Git documentation:

    https://doc.dataiku.com/dss/latest/collaboration/version-control.html#working-with-branches

  • Harald
    Harald Registered Posts: 3 ✭✭

    It works fine. You can branch+duplicate, push the branch to remote, merge branch to master in remote and finally pull in master project, after which branched, duplicated project can be deleted.

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,162 Neuron

    Right I didn't mean to say it doesn't work but that it's highly impractical and effectively useless. First you need to do your merge outside of Dataiku which means your users need to know how to use another tool, understand how Git work, etc. But secondly and more importantly you will not be able to see changes properly since all Dataiku objects are stored as JSON files. This means for instance that code recipes are very hard to merge as you are not looking at code you are looking at a JSON file. Even for visual recipes the JSON structure can easily become very complex. But the most important issue is that you will easily end up with a corrupted/broken objects/project doing a merge and dealing with conflicts. This is because your Git conflict resolution will not have any view on leaving you with an inconsistent flow or invalid recipes. And since you don't really know the correct structure of each JSON file you have no way of knowing how to merge conflicts for them.

    So yeah it "works" in a small test but in real life it doesn't. You have been warned.

Setup Info
    Tags
      Help me…