Join us on Wednesday, June 3rd for a deep dive into Customer Predictive Analytics Learn more

Does dataiku has a recipe or something that checks for the changes in the source tables ?

Left The Nest
Closed Account
Does dataiku has a recipe or something that checks for the changes in the source tables ?

Whenever there is a change in a source table(SQL Server) is there a way to find the schema changes before the project fails? Its a daily run project. I am new to Dataiku so. still weak at understanding the software.

0 Kudos
3 Replies
Dataiker
Dataiker

Hello,

Do you want to check that automatically, on a daily basis? Or is it when designing your flow that you want to check schema changes?

Estelle
0 Kudos
Left The Nest
Closed Account
Author

Im up for anything. I would like to include this step in already built projects/flows. 

I would like to add this to future flows as well.. Need not be a daily check but whenever there is a change...

0 Kudos
Dataiker
Dataiker

In your case, I think that the best way to automate that would be to use a SQL Trigger in a scenario. You can read more there about scenario triggers here: https://doc.dataiku.com/dss/latest/scenarios/triggers.html

When you create a scenario, you can choose the trigger called "SQL query change". You then have to add in the code area "SQL Script" something like : 

select column_name, data_type
from INFORMATION_SCHEMA.COLUMNS where table_name = '<name of table>';

Where you just have to replace <name of table> by your table name.

Then you can add any step you need, like "Send message" 😉

Is that what you need?

Estelle
0 Kudos