For / while loop in Dataiku with visual recipes

nkt Registered Posts: 4 ✭✭✭

Hi all,

Is it possible to achieve for / while loops in Dataiku with only visual recipes?

Example for loop:
Determine for a list of ZIP codes the potential sales by ZIP code based on 20+ visual recipes and 5 external data sources. The flow of the potential sales calculation for each ZIP code is the same, however it is not possible to do the calculation for all ZIP codes at once (due to various reasons). So I like to batch process the flow with a single ZIP code as input for every batch. How do I do this? I looked at the Application-As-Recipe, however it doesn't allow iterations.

Example while loop:
Determine the optimal locations for new stores in a step-wise manner (running 20+ recipes per step), incorporating the impact of the previously calculated stores in the determination of the next optimal store. This iteration will go on, until X store locations are defined. How do I do this?

Hopefully, you can give me direction how to achieve this. I'm not looking for advice how to achieve the results differently and I can't give you much more details about the project. I'm simply looking for a conclusion whether for / while loops are possible (with visual recipes) and how to achieve them.




Best Answer

  • Miguel Angel
    Miguel Angel Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 118 Dataiker
    Answer ✓

    Hi nkt,

    In general, loops are used with code recipes.

    In Prepare Recipe we have the formula processor where you can use 'forEeach', 'forEachIndex', 'forNonBlank' and 'forRange' as the only visual way of doing loops. The caveat is that the values we want to loop through need to be in the same row. You could do an upstream aggregation to achieve that. Another option to loop through rows could consist on exploring the custom aggregations section on Group or Window recipes, though not as visual since you have to write in SQL.

    But before considering that, the business problem of your for loop is a great use case for working with partitions:


Setup Info
      Help me…