Concept of update query

MRvLuijpen
MRvLuijpen Partner, L2 Admin, L2 Designer, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Frontrunner 2022 Participant, Neuron 2023 Posts: 107 Neuron

Hi community,

I was wondering if and how the concept of updating a data set is implemented in Dataiku.

Formed in a question: Is it possible to update a dataset, without creating a new dataset.

Thus a recipe having the input and output directed to the same dataset.

Thanks in advance

Best Answer

  • Alex_Combessie
    Alex_Combessie Alpha Tester, Dataiker Alumni Posts: 539 ✭✭✭✭✭✭✭✭✭
    Answer ✓

    Hi,

    The Flow in Dataiku is a "Directed Acyclic Graph" https://en.wikipedia.org/wiki/Directed_acyclic_graph so a recipe can not have the same input and output dataset.

    I recommend performing the "update dataset" as a code (Python or SQL) step in a Scenario. In a scenario, you can read/write datasets regardless of their position in the flow.

    [ADDITION] Another option would be to not "declare" the dataset as input to your recipe, but only as output.

    Hope it helps,

    Alex

Answers

  • MRvLuijpen
    MRvLuijpen Partner, L2 Admin, L2 Designer, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Frontrunner 2022 Participant, Neuron 2023 Posts: 107 Neuron

    Hello Alex,

    Thank you for your response, it sure helps.

    I was already thinking something similar was the case.

    Thanks again.

    Marc Robert

Setup Info
    Tags
      Help me…