Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
hi, wondering if it is possible to trigger a scenario when a file arrives to a a subfolder. I have a network shared folder mounted to a managed folder, the mounted folder is a large network path with multiple subfolders. how do I only point the trigger to a specific subfolder to monitor as opposed the the parent folder?
Hi @HeteshPatel,
I would suggest pointing a new managed folder to the specific subfolder that you would like to watch and use the "trigger on dataset change" trigger with that managed folder selected.
Thanks!
Jordan
Hi @HeteshPatel,
I would suggest pointing a new managed folder to the specific subfolder that you would like to watch and use the "trigger on dataset change" trigger with that managed folder selected.
Thanks!
Jordan
I managed to get this working via a custom code as a trigger, posting here so it may help others.
from dataiku.scenario import Trigger
from datetime import datetime
import dataiku
import glob
import os
from datetime import timedelta
t = Trigger()
Folder = dataiku.Folder("<FOLDER ID>")
Folder_path = Folder.get_path()
#Assuming you have a managed folder and within that folder is your mounted folder
input_file = glob.glob(Folder_path + "/" +"MOUNTED_FOLDER/SUBFOLDER" + "/TriggerFile.txt")[0]
#Function to check if your file is younger than the specified delta (ideally should match your trigger refresh timing)
def is_file_younger_than(file, delta):
cutoff = datetime.utcnow() - delta
mtime = datetime.utcfromtimestamp(os.path.getmtime(file))
if mtime > cutoff:
return True
return False
if is_file_younger_than(input_file, timedelta(minutes=15)):
t.fire()