Run all dataset
Hello
I have a different files that have been consolidated through Stack recipe. Also, I have done different transformations, cleaning and filters with different recipes to get the final version of what I want to achieve. So, the flow is now complete.
Now that I have everything and that the test, based on limited number of rows, is ok, I would like to run all the different steps with the full data set and get the final result. This perhaps is done through a particular process, but I do not know which one.
Is there any tuto in the academy that explains how to do this ? I could not find it.
Thanks and kindest regards
Best Answer
-
Hi Laurent,
What you are likely looking for is a recursive build of your final output dataset, which you can read more about here: https://doc.dataiku.com/dss/latest/flow/building-datasets.html
Also, in case it was a point of confusion, it is worth noting that the Explore tab in the UI is simply a sample but the recipes will use the full underlying datasets when they are being executed. Therefore, by doing a full recursive build of your Flow, the final output dataset will contain the full data (assuming that your initial input datasets contain the full data).
I hope that this clarifies your question!
Best,
Andrew
Answers
-
LaurentS Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 21 ✭✭✭✭
Hi and thanks a lot for your reply. Clear. Kindest regards