Concept of update query
Hi community,
I was wondering if and how the concept of updating a data set is implemented in Dataiku.
Formed in a question: Is it possible to update a dataset, without creating a new dataset.
Thus a recipe having the input and output directed to the same dataset.
Thanks in advance
Best Answer
-
Hi,
The Flow in Dataiku is a "Directed Acyclic Graph" https://en.wikipedia.org/wiki/Directed_acyclic_graph so a recipe can not have the same input and output dataset.
I recommend performing the "update dataset" as a code (Python or SQL) step in a Scenario. In a scenario, you can read/write datasets regardless of their position in the flow.
[ADDITION] Another option would be to not "declare" the dataset as input to your recipe, but only as output.
Hope it helps,
Alex
Answers
-
MRvLuijpen Partner, L2 Admin, L2 Designer, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Frontrunner 2022 Participant, Neuron 2023 Posts: 107 Neuron
Hello Alex,
Thank you for your response, it sure helps.
I was already thinking something similar was the case.
Thanks again.
Marc Robert