Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Whenever there is a change in a source table(SQL Server) is there a way to find the schema changes before the project fails? Its a daily run project. I am new to Dataiku so. still weak at understanding the software.
Hello,
Do you want to check that automatically, on a daily basis? Or is it when designing your flow that you want to check schema changes?
Im up for anything. I would like to include this step in already built projects/flows.
I would like to add this to future flows as well.. Need not be a daily check but whenever there is a change...
In your case, I think that the best way to automate that would be to use a SQL Trigger in a scenario. You can read more there about scenario triggers here: https://doc.dataiku.com/dss/latest/scenarios/triggers.html.
When you create a scenario, you can choose the trigger called "SQL query change". You then have to add in the code area "SQL Script" something like :
select column_name, data_type from INFORMATION_SCHEMA.COLUMNS where table_name = '<name of table>';
Where you just have to replace <name of table> by your table name.
Then you can add any step you need, like "Send message" 😉
Is that what you need?