Source input excel schema changes

IrinaS

Hi,

I'm loading excel files from FTP to SQL db, everything works fine but I want the build to be stopped if the excel structure changes (missing a column, adding a new column ) and the new schema not to be loaded in the db.

I've tried with sync recipe, but If a column is missing the data is loaded automatically from the next columns.

Is there a way to obtain this?

Thanks,

Irina

Alexandru

Hi @IrinaS
,

There 2 ways to handle schema for sync recipes, using "Free output schema" then it will perform named-based. That may help to some extent in your case depending on the type of changes.

If that doesn't help, you can use Scenario to build these datasets and use the "check flow consistency step" and not proceed to build but instead alert in case the schema is different from the data.

https://knowledge.dataiku.com/latest/courses/flow-views-and-actions/schema-propagation-concept-summary.html

Let me know if that helps!