Triggering custom python - how to properly set run every (seconds) and grace delay (seconds)

Options
jrmathieu63
jrmathieu63 Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 26 ✭✭✭✭✭
edited July 16 in Using Dataiku

Have a custom python code that works fine when testing for 1 min and 1 hours time frames.

The code is expected to run on a specific business day.

When I set the code to run once every day - 86400 seconds, it does not appear to run.
Not sure why this is happening.

Example for a run to be executed today and will run if I change the run seconds to 1 hour.
Was set yesterday to be for today but never triggered as expected this morning.

import pandas as pd
from datetime import datetime
import calendar

# get today's date in YYYY-MM-DD format
todaystr = datetime.today().strftime("%Y-%m-%d")
print("Today is: " f'{todaystr}')
# get the year and month of today's date
year = datetime.today().year
month = datetime.today().month
# need to determine max days for current month
_, num_days = calendar.monthrange(year, month)
# create a date range with only business days for the current month
s = pd.bdate_range(start=f"{year}-{month}-01", end=f"{year}-{month}-{num_days}")
# check for business day of the current month as a string in YYYY-MM-DD format
chkBDCMstr = s[2].strftime("%Y-%m-%d")
print("Check Business Date is: " f'{chkBDCMstr}')
# check if either of them matches today's date
if chkBDCMstr == todaystr:
    from dataiku.scenario import Trigger
    t = Trigger()
    t.fire()

Not sure why this cannot be schedule to run once a day.


Operating system used: AL@

Best Answer

  • jrmathieu63
    jrmathieu63 Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 26 ✭✭✭✭✭
    edited 4:20PM Answer ✓
    Options

    Here is the complete requirement for a trigger for a specific business run date.
    Turns out that setting Run every (seconds) value should not be set for a day since it will never get triggered if environment is scheduled to be shutdown daily for financial reasons.

    Here is update using an hour setting in addition to the date.
    Set Run every (seconds) value to 3600 for every hour.

    import pandas as pd
    from datetime import datetime
    import calendar
    
    # get today's date in YYYY-MM-DD format
    todaystr = datetime.today().strftime("%Y-%m-%d")
    print("Today is: " f'{todaystr}')
    # get the year and month of today's date
    year = datetime.today().year
    month = datetime.today().month
    # need to determine max days for current month
    _, num_days = calendar.monthrange(year, month)
    # create a date range with only business days for the current month
    s = pd.bdate_range(start=f"{year}-{month}-01", end=f"{year}-{month}-{num_days}")
    # Get hour to run at
    hour = datetime.now().hour
    # check for business day of the current month as a string in YYYY-MM-DD format
    chkBDCMstr = s[5].strftime("%Y-%m-%d")
    print("Check Business Date is: " f'{chkBDCMstr}')
    # check if either of them matches today's date
    if chkBDCMstr == todaystr and hour == 6:
        print("Scenario will trigger and run now")
        from dataiku.scenario import Trigger
        t = Trigger()
        t.fire()

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,717 Neuron
    Options

    I changed your code to have todaystr match chkBDCMstr and it works fine for me. Are you sure the trigger and Scenario are active (ie ON). Also have a read at this thread:

    https://community.dataiku.com/t5/Using-Dataiku/Scenario-Custom-Trigger-Tips/m-p/5561

    If you still have problems post the backend log events from your trigger.

  • jrmathieu63
    jrmathieu63 Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 26 ✭✭✭✭✭
    Options

    Have already read the link provided.

    Also do not see any changes for the todaystr and chkBDCMstr and should not be required since it does run for me, if the Run every (second) is set to be for 1 min, 1 hour but not for 1 day.

    It appears the Run every should be set for every hour to trigger and we should add a check for a specific hour to run.

Setup Info
    Tags
      Help me…