append table

Oh_Lily

Hello,
I have an output table in my dataiky flow where results are appended after each build.
We had a data problem and we would like to relaunch the last run while removing the results of the previous one from the table and replacing it by the new one?
How can we do this without revert to a previous version because we have other changes we would like to keep.

Thank you in advance for your help

---

Operating system used: windows

Turribeach

There is no built-in functionality for what you want to do. Furthermore I would warn you that append recipes are risky thing to use in Dataiku if you depend on your historical data. The default functionality in Dataiku is that when there are schema changes and changes are propagated is to drop and recreate the table, irrespective of you having set the "Append instead of override" check box in any recipe.

Oh_Lily

Thank you for your response !

Turribeach

A better way of doing this will be to take the output of the Dataiku flow and do deltas outside of Dataiku so that you can manage your historical table outside of Dataiku. If you need to read your historical table in Dataiku you can add it to the flow as "unmanaged/external" dataset which means Dataiku will not be responsible for managing the schema of that table and you will need to manage it yourself (ie adding/deleting columns, etc).