How to choose between updating and appending an output table

Options
abalo006
abalo006 Registered Posts: 29

I have an connection / output recipe that is outputting to a table in snowflake, I don't see any option in here to choose between outputting and appending data, where can I set that up? does the option pop up after I press run? or is that something I can set up before hand?


Operating system used: windows

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,728 Neuron
    Options

    You will find this setting on the recipe itself. Where it will be will depend on the recipe type but it is always where you define the inputs and outputs below the output name you will find the "Append instead of overwrite" option. Be careful though, this options works but it's not a guarantee Dataiku will not drop the output table. In particular every time there is a schema change Dataiku will drop the output table and recreate it with the new schema even if the "Append instead of overwrite" option is enabled. So DO NOT use this option for historical tables you need to keep the data for ever.

Setup Info
    Tags
      Help me…