How to create a conditional table in dataiku after flow is successful

AGS
AGS Registered Posts: 2
edited March 13 in Using Dataiku

Once the load of table is done and flow is successful add "1" to table value indicating load complete.

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,321 Neuron

    Can you please explain your requirement in detail not how you think you can implement it? There might be a better way of achieving what you want.

  • AGS
    AGS Registered Posts: 2
    edited March 13

    Meaning once the flow is completed , then in a ref table a true/false needs to be written.   Based on that we will be updating the data in the final table if true. So basically what script can be used to find out flow completion status

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,321 Neuron

    This is the means of achieving a goal. What is your goal? Why do you need a table with a true false?

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,321 Neuron

    So to give you an idea of a bad pattern would that you want your table to have a True column/row so that you can refresh a report in your reporting layer once the flow is completed. This is a bad pattern since it uses a "poll" mechanism, the reporting layer is constantly polling the table for a true value after which it refreshes. This a bad pattern since it wastes a lot of compute resources to do the polling and does not refresh immediately (depending on the polling frequency). A better design will be to have Dataiku trigger the reporting layer refresh as part of the scenario flow. This is a "push" mechanism and much more efficient and able to handle things like reruns.

Setup Info
    Tags
      Help me…