Concept of update query

Solved!
MRvLuijpen
Concept of update query

Hi community, 

I was wondering if and how the concept of updating a data set is implemented in Dataiku.

Formed in a question: Is it possible to update a dataset, without creating a new dataset.

Thus a recipe having the input and output directed to the same dataset.

Thanks in advance

0 Kudos
1 Solution
Alex_Combessie
Dataiker Alumni

Hi,

The Flow in Dataiku is a "Directed Acyclic Graph" https://en.wikipedia.org/wiki/Directed_acyclic_graph so a recipe can not have the same input and output dataset.

I recommend performing the "update dataset" as a code (Python or SQL) step in a Scenario. In a scenario, you can read/write datasets regardless of their position in the flow.

[ADDITION] Another option would be to not "declare" the dataset as input to your recipe, but only as output.

Hope it helps,

Alex

View solution in original post

0 Kudos
2 Replies
Alex_Combessie
Dataiker Alumni

Hi,

The Flow in Dataiku is a "Directed Acyclic Graph" https://en.wikipedia.org/wiki/Directed_acyclic_graph so a recipe can not have the same input and output dataset.

I recommend performing the "update dataset" as a code (Python or SQL) step in a Scenario. In a scenario, you can read/write datasets regardless of their position in the flow.

[ADDITION] Another option would be to not "declare" the dataset as input to your recipe, but only as output.

Hope it helps,

Alex

0 Kudos
MRvLuijpen
Author

Hello Alex, 

Thank you for your response, it sure helps. 

I was already thinking something similar was the case.

Thanks again.

Marc Robert

 

0 Kudos