How does Dataiku run a recipe from a scenario if the recipe has multiple outputs?

pnkj
pnkj Registered Posts: 10 ✭✭

Let us say there is a python recipe with input as A and outputs B, C, D.

I want to build the datasets B, C and D from a scenario. So, should I specify datasets B, C and D in the build step in the scenario?

Will it run the recipe thrice to create each of the datasets? If so, is there a way to run the whole recipe from the scenario so that it updates the datasets by itself?

Best Answer

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,160 Neuron
    Answer ✓

    All the datasets will be built even if you specify only one. If the recipe runs it will build all 3 outputs, assuming there is no conditional code to selectively build outputs depending on parameters.

Answers

  • Konstantina
    Konstantina Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 27 ✭✭✭✭✭

    Hi @pnkj!

    If the outputs are all defined in the same Python recipe, which from my understanding are, then, my guess is that by specifying only one of the B, C or D datasets in the scenario build step, all of these outputs datasets will be built. This build step will actually trigger the corresponding Python recipe (which builds all three datasets through code), so if it runs once, all the output datasets will be built.

    Hope this helps! 😊

    -Konstantina

Setup Info
    Tags
      Help me…