Run all dataset

Options
LaurentS
LaurentS Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 21 ✭✭✭✭

Hello

I have a different files that have been consolidated through Stack recipe. Also, I have done different transformations, cleaning and filters with different recipes to get the final version of what I want to achieve. So, the flow is now complete.

Now that I have everything and that the test, based on limited number of rows, is ok, I would like to run all the different steps with the full data set and get the final result. This perhaps is done through a particular process, but I do not know which one.

Is there any tuto in the academy that explains how to do this ? I could not find it.

Thanks and kindest regards

Best Answer

  • ATsao
    ATsao Dataiker Alumni, Registered Posts: 139 ✭✭✭✭✭✭✭✭
    Answer ✓
    Options

    Hi Laurent,

    What you are likely looking for is a recursive build of your final output dataset, which you can read more about here: https://doc.dataiku.com/dss/latest/flow/building-datasets.html

    Also, in case it was a point of confusion, it is worth noting that the Explore tab in the UI is simply a sample but the recipes will use the full underlying datasets when they are being executed. Therefore, by doing a full recursive build of your Flow, the final output dataset will contain the full data (assuming that your initial input datasets contain the full data).

    I hope that this clarifies your question!

    Best,

    Andrew

Answers

  • LaurentS
    LaurentS Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 21 ✭✭✭✭
    Options

    Hi and thanks a lot for your reply. Clear. Kindest regards

Setup Info
    Tags
      Help me…