Concept of update query

MRvLuijpen

Hi community,

I was wondering if and how the concept of updating a data set is implemented in Dataiku.

Formed in a question: Is it possible to update a dataset, without creating a new dataset.

Thus a recipe having the input and output directed to the same dataset.

Thanks in advance

Alex_Combessie

Hi,

The Flow in Dataiku is a "Directed Acyclic Graph" https://en.wikipedia.org/wiki/Directed_acyclic_graph so a recipe can not have the same input and output dataset.

I recommend performing the "update dataset" as a code (Python or SQL) step in a Scenario. In a scenario, you can read/write datasets regardless of their position in the flow.

[ADDITION] Another option would be to not "declare" the dataset as input to your recipe, but only as output.

Hope it helps,

Alex

MRvLuijpen

Hello Alex,

Thank you for your response, it sure helps.

I was already thinking something similar was the case.

Thanks again.

Marc Robert