How to choose between updating and appending an output table
I have an connection / output recipe that is outputting to a table in snowflake, I don't see any option in here to choose between outputting and appending data, where can I set that up? does the option pop up after I press run? or is that something I can set up before hand?
Operating system used: windows
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,160 NeuronYou will find this setting on the recipe itself. Where it will be will depend on the recipe type but it is always where you define the inputs and outputs below the output name you will find the "Append instead of overwrite" option. Be careful though, this options works but it's not a guarantee Dataiku will not drop the output table. In particular every time there is a schema change Dataiku will drop the output table and recreate it with the new schema even if the "Append instead of overwrite" option is enabled. So DO NOT use this option for historical tables you need to keep the data for ever.