Concept of update query

[MRvLuijpen]

Hi community, 

I was wondering if and how the concept of updating a data set is implemented in Dataiku.

Formed in a question: Is it possible to update a dataset, without creating a new dataset.

Thus a recipe having the input and output directed to the same dataset.

Thanks in advance

[Alex_Combessie · Accepted solution]

Hi,

The Flow in Dataiku is a "Directed Acyclic Graph" https://en.wikipedia.org/wiki/Directed_acyclic_graph so a recipe can not have the same input and output dataset.

I recommend performing the "update dataset" as a code (Python or SQL) step in a Scenario. In a scenario, you can read/write datasets regardless of their position in the flow.

[ADDITION] Another option would be to not "declare" the dataset as input to your recipe, but only as output.

Hope it helps,

Alex

[MRvLuijpen]

Hello Alex, 

Thank you for your response, it sure helps. 

I was already thinking something similar was the case.

Thanks again.

Marc Robert