Add a recipe in the middle of the flow

Thomas24
Thomas24 Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 3

Dear Dataiku users,

I am having trouble finding an answer to my question.

I am building a dataiku flow with multiple recipes. Is it possible to add a recipe (let's say a prepare recipe) in the middle of the flow ? I just want to reorganize the order of my columns for the last datasets created in my flow.

The only solution I found was to create a prepare recipe and copy paste the existing recipes following this new prepare recipe before deleting the original ones but I don't find it very convenient...

Thanks for your help

Thomas


Operating system used: MS 11

Best Answer

  • MayeulR
    MayeulR Dataiker Posts: 11 Dataiker
    Answer ✓

    Hello Thomas,

    Please note that we are working on a way to insert a recipe before another one, I'll keep you updated once this feature is available.

    As a workaround, here is what you could do:

    - Let say you have Dataset A -> Recipe B -> Dataset C and you want to insert Recipe D before Recipe B.

    - Create Recipe D with Dataset A as input and Dataset E as output.

    - Edit Recipe B, go to Input/Output and change input from Dataset A to Dataset E

    - You now have: Dataset A -> Recipe D -> Dataset E -> Recipe B -> Dataset C.

    Best,

    Mayeul

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,024 Neuron

    I heard Dataiku is working in a better solution for this since the current way takes a few extra steps. To replace an existing recipe first select the input dataset, then select the new recipe type, then in the Output dataset section you create a new dataset output. Now you have a new branch on your flow which you need to re-plumb to the existing flow. Go to the next recipe you want to join the new branch with and look for the recipe Input/Outputs, this is usually in the Input/Output tab but can be in the Advanced tab depending on the recipe type. On the input section click on Change to point to the new output dataset you created. Finally go and delete the older recipe or left over flow branch. You can re-plumb any inputs or outputs this way. If you re-plumb an output make sure to use the "Use Existing Dataset" option to avoid creating a new dataset. This will also work with Folders where applicable.

  • Thomas24
    Thomas24 Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 3

    Thank you both for your answers. I'll keep an eye out to check when Dataiku releases this feature

    Best

    Thomas

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,024 Neuron
  • apichery
    apichery Dataiker, Alpha Tester, Registered, Product Ideas Manager, Moderator Posts: 63 Dataiker

    and it's now available out of the box on DSS 12.4.0 !

    insert-recipe.png

Setup Info
    Tags
      Help me…