Easier recipe replacement

Sometimes, I need to change out the recipe I've used in a flow. For example, I may have used a sync before, but now I need a filter or a sort recipe instead. Or I may have used groups, but now need a pivot, etc. Many of the parameters of the recipes will be preserved. For example, computed columns, input and output datasets, aggregations, and filters are all common to multiple types of recipes. When a recipe is deleted and re-added, it totally breaks the flow, sending the right side of the broken link all the way to the left of the flow. That makes it pretty difficult to find the target dataset again. But if the link isn't broken first, and instead a new recipe is added, the output dataset cannot be reused. The solution I tend to use is opening a second browser tab while I'm on the settings screen of my new recipe, then delete the old recipe after I've typed the output dataset's name into the first recipe, the copy it, refresh, and paste it to connect the new recipe to the old output.

But this process is pretty cumbersome, and on big, complex projects, if the flow is broken, basically the only consistent way to find out how things were meant to connect is to go back into the git logs and see what dataset was disconnected.

It would be a huge timesaver for this workflow if when selecting a recipe in the flow, I could simply click a button to replace the recipe with a different type of recipe. Which parameters are preserved for the new recipe would depend on the type of recipe being selected, but even if the implementation only preserved input and output datasets, that would be a huge time saver.

5 Comments
ElisaS
Dataiker

Hi @natejgardner ! 

 

Thanks for this suggestion. I have added this to our backlog. We can't provide a timeline at this point, but be sure to check back for updates.

Best,
Elisa

Status changed to: In Backlog

Hi @natejgardner ! 

 

Thanks for this suggestion. I have added this to our backlog. We can't provide a timeline at this point, but be sure to check back for updates.

Best,
Elisa

So this is an interesting one and while I think it's a very narrow use case it's certainly a pain to change recipes in the flow. It is already possible to grab all the existing inputs for an existing recipe by multi-selecting them and clicking on the desired new recipe. So in reality the pain exists since you can't assign the output dataset until it's released from the "old" recipe. Maybe this idea will be much easier to implement if Dataiku allowed you to select the existing output dataset and move it to the new recipe rather than simply having all in-use datasets greyed out as "already used in output". It will still be cumbersome to find the existing dataset though...

And here is the curve ball. Since you can create new recipes programatically I am pretty sure it will be possible to have a Macro that allows you to "replace" an existing recipe keeping the inputs and outputs of the old recipe. So if this really bothers you there might be a solution...

So this is an interesting one and while I think it's a very narrow use case it's certainly a pain to change recipes in the flow. It is already possible to grab all the existing inputs for an existing recipe by multi-selecting them and clicking on the desired new recipe. So in reality the pain exists since you can't assign the output dataset until it's released from the "old" recipe. Maybe this idea will be much easier to implement if Dataiku allowed you to select the existing output dataset and move it to the new recipe rather than simply having all in-use datasets greyed out as "already used in output". It will still be cumbersome to find the existing dataset though...

And here is the curve ball. Since you can create new recipes programatically I am pretty sure it will be possible to have a Macro that allows you to "replace" an existing recipe keeping the inputs and outputs of the old recipe. So if this really bothers you there might be a solution...

MichaelG
Community Manager
Community Manager
 
I hope I helped! Do you Know that if I was Useful to you or Did something Outstanding you can Show your appreciation by giving me a KUDOS?

Looking for more resources to help you use DSS effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!
Status changed to: In the Backlog
 
AshleyW
Dataiker

Hi @natejgardner

We did it! With the release of 12.4, you can now more easily insert a new recipe into an existing Flow segment. Select a dataset and use the 'Insert recipe after this dataset' action; no more Flow breakage, broken links, and orphaned datasets.

Its opposite, 'snip', is also in the works for a future release.

Take care,
Ashley

Status changed to: Released

Hi @natejgardner

We did it! With the release of 12.4, you can now more easily insert a new recipe into an existing Flow segment. Select a dataset and use the 'Insert recipe after this dataset' action; no more Flow breakage, broken links, and orphaned datasets.

Its opposite, 'snip', is also in the works for a future release.

Take care,
Ashley

Awesome!

Awesome!