Survey banner
Switching to Dataiku - a new area to help users who are transitioning from other tools and diving into Dataiku! CHECK IT OUT

SQL Change Trigger in a Scenario is running the steps twice

Level 3
SQL Change Trigger in a Scenario is running the steps twice

Hi All

I have a FLOW which is automated thru a SCENARIO1 using SQL Change Trigger to run a python code STEP to RECURSIVE FORCE BUILD     TARGET1



SQL Change Trigger = select max(rec_created_dt) from SOURCE1

STEP= scenario.build_dataset(TARGET1, RECURSIVE_FORCE_BUILD)

in above STEP im extract job details(job_name,execution_time) from list_jobs and Target metrics(ex record_count) and inserting them into a REDSHIFT table .

Here i am seeing whenever SOURCE1 is loaded with new data SCENARIO1 is triggring twice and job is executing twice.What could be the reason ? any help would be much appreciated.


In first attempt i see 0 records being calcualted on TARGET1 but in second run actual record count is being calculated.


0 Kudos
2 Replies
Dataiker Alumni

Hi @rsingamsetty 

It's a little hard to figure out what might be causing this behaviour without specifics of your tables, scenarios and other processes settings. When you say the scenario runs twice, what's the time difference in between the two runs? Could it be that your data is written into the database in two batches? 


0 Kudos
Level 3

Hi @Liev ,

I have figured out the issue. Actually from source side the tables are getting truncate and load

well truncate table is also activating trigger so i have to modify the sql statement to avoid activation with 0 counts