trigger scenario on file arrives in a subfolder

HeteshPatel

hi, wondering if it is possible to trigger a scenario when a file arrives to a a subfolder. I have a network shared folder mounted to a managed folder, the mounted folder is a large network path with multiple subfolders. how do I only point the trigger to a specific subfolder to monitor as opposed the the parent folder?

JordanB

Hi @HeteshPatel
,

I would suggest pointing a new managed folder to the specific subfolder that you would like to watch and use the "trigger on dataset change" trigger with that managed folder selected.

Thanks!

Jordan

HeteshPatel

I managed to get this working via a custom code as a trigger, posting here so it may help others.

from dataiku.scenario import Trigger
from datetime import datetime
import dataiku
import glob
import os
from datetime import timedelta

t = Trigger()

Folder = dataiku.Folder("<FOLDER ID>")
Folder_path = Folder.get_path()

#Assuming you have a managed folder and within that folder is your mounted folder
input_file = glob.glob(Folder_path + "/" +"MOUNTED_FOLDER/SUBFOLDER" + "/TriggerFile.txt")[0]

#Function to check if your file is younger than the specified delta (ideally should match your trigger refresh timing)
def is_file_younger_than(file, delta):
    cutoff = datetime.utcnow() - delta
    mtime = datetime.utcfromtimestamp(os.path.getmtime(file))
    if mtime > cutoff:
        return True
    return False

if is_file_younger_than(input_file, timedelta(minutes=15)):
    t.fire()