Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I use Dataiku with Snowflake.
My Project Flow is the same as the picture below. (Sample)
Next is Scenario Python Script.
When Scenario works, I want to refresh/rebuild 'TRAFFIC' data in Snowflake Database.
How do I make Script??
Hi @DaeWangIma ,Since the traffic dataset is a input dataset this is "refreshed" everytime you read it again from a dependent recipe.
What you can do here is do a recursive build via the scenario. You can start the scenario based on SQL trigger for example to identify if the underlying data changed then re-run the scenario https://doc.dataiku.com/dss/latest/scenarios/triggers.html#sql-triggers
You don't need to include custom python step a simple dataset build recursively should do the trick of any of the dependent datasets in the right of your flow.
Hope that helps!
Post a Question