About how to trigger a scenario on workdays only
Hi so I would like to set up a trigger for scenario that only fires on workdays, I have set up some global variables and I have set my custom trigger like this:
from dataiku.scenario import Trigger
t = Trigger()
if variables['standard']['LastCalendarDay'] == variables['standard']['LastChinaWorkingDay']
or variables['standard']['NextCalendarDay'] == variables['standard']['NextChinaWorkingDay']:
t.fire()
And I set the code to run once 86400 secs(1day). Will this trigger do the work? Or are there any better ways to do it?
Thanks in advance!
Best Answer
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,611 NeuronHi, we can't really know what's on those custom variables but I presume your logic is sound so this should work. There are however several reasons as to why using custom Python triggers is not a good idea at all:
- Custom Python triggers need to be executed to be evaluated, so they are a bit inneficient in that need constant execution if the condition for execution can't be set in time. In your case it's fine since you will be executing it once a day but keep it in mind for other use cases where you need to evaluate your condition multiple times a day
- Custom Python triggers do not scale well. If every user created similar triggers which need to be evaluated very frequently soon a lot of CPU cycles will wasted evaluating these triggers. Often they will be trying to achieve the same thing (ie work out if it is a working day)
- It's currently imposible to combine multiple triggers logically so this could result in Custom Python triggers executing at unwanted times. For instance your trigger will be executed when the Dataiku instance is restarted and every day at that time going forward. Is that the time you really want your scenario to execute?
- There is no built-in mechanism at all to indicate when triggers fail. If one of your variables got deleted, renamed or had a Null value your trigger will constantly fail to be evaluated but there will be nothing for you to see anywhere in the GUI nor any automated alerts. Your scenario will never run again unless it has other triggers. Trigger failures are therefore silent and only show in the backend.log which is only available to Administrators (and most of them don't look for errors in the logs)
- Even if your Custom Python trigger do run they are very hard to debug because there is no trace of the execution values
The above should be enough to discourage you to use Custom Python triggers. So what's the alternative given that Dataiku doesn't provide an advanced scheduler which handles things like "run only on working days in China"?
The easiest way to achieve your requirement is to create a Time-based trigger which only runs Mon-Fri. You will have full control on the time the trigger will fire and you can set the days to exclude running on Sat and Sun. But obviously this doesn't cover national bank holidays so let's see how you can achieve that. The best flow /scenario is one that can run on bank holidays which means that it doesn't really matter if it is national bank holiday, it can handle these cases normally. This means your scenario/flow is bank holiday agnostic.
If you can't do that the you could add some national bank holiday logic to the scenario itself. I covered this on this post where I show to conditionally execute a step in a scenario based on the value of a variable that is calculated as part of the scenario run. The beauty of using scenario variables and conditional step execution is that should anything fail the scenario will fail and you can set a mail reporter to know. Even without a mail reporter there will be a scenario failure in the GUI. And all the variable evaluations will be present in the scenario logs for you to understand what happened.
Answers
-
Thank you for the detailed explanation! TIL how scheduling and customized scheduling works! I actually found that my previous solution would not work because for some reason it will not run everyday and evaluate the criterias. However, the link that you provided where getting this done by step condition worked very well!