Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Concept of update query

Solved!
MRvLuijpen
Neuron
Neuron
Concept of update query

Hi community, 

I was wondering if and how the concept of updating a data set is implemented in Dataiku.

Formed in a question: Is it possible to update a dataset, without creating a new dataset.

Thus a recipe having the input and output directed to the same dataset.

Thanks in advance

0 Kudos
1 Solution
Alex_Combessie
Dataiker Alumni

Hi,

The Flow in Dataiku is a "Directed Acyclic Graph" https://en.wikipedia.org/wiki/Directed_acyclic_graph so a recipe can not have the same input and output dataset.

I recommend performing the "update dataset" as a code (Python or SQL) step in a Scenario. In a scenario, you can read/write datasets regardless of their position in the flow.

[ADDITION] Another option would be to not "declare" the dataset as input to your recipe, but only as output.

Hope it helps,

Alex

View solution in original post

0 Kudos
2 Replies
Alex_Combessie
Dataiker Alumni

Hi,

The Flow in Dataiku is a "Directed Acyclic Graph" https://en.wikipedia.org/wiki/Directed_acyclic_graph so a recipe can not have the same input and output dataset.

I recommend performing the "update dataset" as a code (Python or SQL) step in a Scenario. In a scenario, you can read/write datasets regardless of their position in the flow.

[ADDITION] Another option would be to not "declare" the dataset as input to your recipe, but only as output.

Hope it helps,

Alex

0 Kudos
MRvLuijpen
Neuron
Neuron
Author

Hello Alex, 

Thank you for your response, it sure helps. 

I was already thinking something similar was the case.

Thanks again.

Marc Robert

 

0 Kudos