For / while loop in Dataiku with visual recipes

nkt

Hi all,

Is it possible to achieve for / while loops in Dataiku with only visual recipes?

Example for loop:
Determine for a list of ZIP codes the potential sales by ZIP code based on 20+ visual recipes and 5 external data sources. The flow of the potential sales calculation for each ZIP code is the same, however it is not possible to do the calculation for all ZIP codes at once (due to various reasons). So I like to batch process the flow with a single ZIP code as input for every batch. How do I do this? I looked at the Application-As-Recipe, however it doesn't allow iterations.

Example while loop:
Determine the optimal locations for new stores in a step-wise manner (running 20+ recipes per step), incorporating the impact of the previously calculated stores in the determination of the next optimal store. This iteration will go on, until X store locations are defined. How do I do this?

Hopefully, you can give me direction how to achieve this. I'm not looking for advice how to achieve the results differently and I can't give you much more details about the project. I'm simply looking for a conclusion whether for / while loops are possible (with visual recipes) and how to achieve them.

Best,

Nick

Miguel Angel

Hi nkt,

In general, loops are used with code recipes.

In Prepare Recipe we have the formula processor where you can use 'forEeach', 'forEachIndex', 'forNonBlank' and 'forRange' as the only visual way of doing loops. The caveat is that the values we want to loop through need to be in the same row. You could do an upstream aggregation to achieve that. Another option to loop through rows could consist on exploring the custom aggregations section on Group or Window recipes, though not as visual since you have to write in SQL.

But before considering that, the business problem of your for loop is a great use case for working with partitions: https://doc.dataiku.com/dss/latest/partitions/index.html

nkt

Thank you for your quick reply. It is clear to me, I marked your answer as solution