Rebuild/Refresh Snowflake Database in Scenario

DaeWangIma

Hi

I use Dataiku with Snowflake.

My Project Flow is the same as the picture below. (Sample)

Next is Scenario Python Script.

When Scenario works, I want to refresh/rebuild 'TRAFFIC' data in Snowflake Database.

How do I make Script??

Thank you.

Alexandru

Hi @DaeWangIma
,
Since the traffic dataset is a input dataset this is "refreshed" everytime you read it again from a dependent recipe.

What you can do here is do a recursive build via the scenario.
You can start the scenario based on SQL trigger for example to identify if the underlying data changed then re-run the scenario
https://doc.dataiku.com/dss/latest/scenarios/triggers.html#sql-triggers

You don't need to include custom python step a simple dataset build recursively should do the trick of any of the dependent datasets in the right of your flow.

https://knowledge.dataiku.com/latest/data-preparation/pipelines/concept-dataset-build-modes.html

Hope that helps!