Avoid to modify recipies in case of new data in input file

PYB
Level 1
Avoid to modify recipies in case of new data in input file

Hello,

I would like to limit the time consuming when a new column is added in the input file which need to be displayed in the output.

In a standard project i need to modified each step (recipies) but i don't know if something exist to avoid this.

Thank you for your help.

Kind regards,

Pierre-Yves


Operating system used: Windows 10 Enterprise

0 Kudos
1 Reply
MiguelangelC
Dataiker

Hello,

DSS comes with a tool to avoid manually updating downstream object schemas after an input dataset schema has changed. It is called the 'Schema Propagation Tool' and it can be accessed from the Flow view. Right click the flow object and select 'Propagate schema across Flow from here'.

There is a more detailed explanation in this help article: https://knowledge.dataiku.com/latest/courses/flow-views-and-actions/schema-propagation-concept-summa...

Please note that this tool has some limitations. It cannot verify the output schema of recipes without running them first, e.g. code recipes.

0 Kudos