Reusing flows

Solved!
yahia22
Level 2
Reusing flows

Hello I have a question,

We have a set of steps that needs to be performed cross regions. We want to automate everything and have a central point where we can change something and the changes need to be reflected everywhere at once.

I have two questions:

1) Is it possible to link all sub flows together( same flow) and have a change in one reflected in others

2) If I use python spark scripts recipes to do this , are changes in the master recipe reflected in all instances of this recipe across flows.

Best 

0 Kudos
1 Solution
fchataigner2
Dataiker

There are no option to "link" recipes together and have changes applied to one transferred to the other. Short of writing the copying logic manually with the public API of DSS, custom recipes (in plugins) or app-as-recipes are the closest to this use case.

View solution in original post

4 Replies
fchataigner2
Dataiker

Hi,

if you want to design a reusable flow, in the sense of a sequence of recipes applied to a given set of inputs, then you should check https://doc.dataiku.com/dss/latest/applications/index.html#application-as-recipe (there are some limitations, notably around partitioned datasets)

Other solutions would require manually synchronizing recipes between subflow (or automating it with the public API of DSS)

yahia22
Level 2
Author

Hello,

Thanks for your response, do you have  any idea if I were to update the recipe(code or configuration) , then that change would be reflected in places where I used it. 

0 Kudos
fchataigner2
Dataiker

There are no option to "link" recipes together and have changes applied to one transferred to the other. Short of writing the copying logic manually with the public API of DSS, custom recipes (in plugins) or app-as-recipes are the closest to this use case.

yahia22
Level 2
Author

Thank you so much. That is exactly what I am looking for.

0 Kudos