Avoid to modify recipies in case of new data in input file
Hello,
I would like to limit the time consuming when a new column is added in the input file which need to be displayed in the output.
In a standard project i need to modified each step (recipies) but i don't know if something exist to avoid this.
Thank you for your help.
Kind regards,
Pierre-Yves
Operating system used: Windows 10 Enterprise
Answers
-
Miguel Angel Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 118 DataikerHello,
DSS comes with a tool to avoid manually updating downstream object schemas after an input dataset schema has changed. It is called the 'Schema Propagation Tool' and it can be accessed from the Flow view. Right click the flow object and select 'Propagate schema across Flow from here'.
There is a more detailed explanation in this help article: https://knowledge.dataiku.com/latest/courses/flow-views-and-actions/schema-propagation-concept-summary.html#schema-propagation-tool
Please note that this tool has some limitations. It cannot verify the output schema of recipes without running them first, e.g. code recipes.