Source input excel schema changes
Hi,
I'm loading excel files from FTP to SQL db, everything works fine but I want the build to be stopped if the excel structure changes (missing a column, adding a new column ) and the new schema not to be loaded in the db.
I've tried with sync recipe, but If a column is missing the data is loaded automatically from the next columns.
Is there a way to obtain this?
Thanks,
Irina
Best Answer
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptionsHi @IrinaS
,There 2 ways to handle schema for sync recipes, using "Free output schema" then it will perform named-based. That may help to some extent in your case depending on the type of changes.
If that doesn't help, you can use Scenario to build these datasets and use the "check flow consistency step" and not proceed to build but instead alert in case the schema is different from the data.
Let me know if that helps!