Does dataiku has a recipe or something that checks for the changes in the source tables ?

Unknown

Whenever there is a change in a source table(SQL Server) is there a way to find the schema changes before the project fails? Its a daily run project. I am new to Dataiku so. still weak at understanding the software.

EstelleB

Hello,

Do you want to check that automatically, on a daily basis? Or is it when designing your flow that you want to check schema changes?

Unknown

Im up for anything. I would like to include this step in already built projects/flows.

I would like to add this to future flows as well.. Need not be a daily check but whenever there is a change...

EstelleB

In your case, I think that the best way to automate that would be to use a SQL Trigger in a scenario. You can read more there about scenario triggers here: https://doc.dataiku.com/dss/latest/scenarios/triggers.html.

When you create a scenario, you can choose the trigger called "SQL query change". You then have to add in the code area "SQL Script" something like :

select column_name, data_type
from INFORMATION_SCHEMA.COLUMNS where table_name = '<name of table>';

Where you just have to replace <name of table> by your table name.

Then you can add any step you need, like "Send message"

Is that what you need?