Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi,
I was wondering if it is possible in Dataiku to get notified or emailed when a job exceeds a pre-determined duration.
For example I am a user administrator and I want to receive an email if there are jobs running for more than 1 hour.
Regards,
Weng Kin
Hi,
Not possible at the job level, but you can leverage the APIs and run a scenario to find long-running jobs and send emails.
https://doc.dataiku.com/dss/latest/python-api/jobs.html Here is an example you can use and modify as needed :
import dataiku
import datetime
from dataiku.core.message_sender import MessageSender
client = dataiku.api_client()
client = dataiku.api_client()
projects = client.list_projects()
# e.g adjust here to find job in this example 4 hours or older and state still running
four_hour_ago = datetime.datetime.now() - datetime.timedelta(hours=4)
timestamp = int(four_hour_ago.timestamp() * 1000)
job_info_str = ""
for p in projects:
pk = p["projectKey"]
project = client.get_project(pk)
jobs = project.list_jobs()
running_job_list = [job for job in jobs if job['state'] == 'RUNNING' and job['def']['initiationTimestamp'] >= timestamp]
if len(running_job_list) >=1:
job_info_str += p["projectKey"] + "\n" + "\n".join([str(job) for job in running_job_list]) + "\n"
print(job_info_str)
# Now send either one email for each long-running or create a list with the project key + job info from above.
s = MessageSender(channel_id='my-email-channel', type='mail-scenario', configuration={})
s.send(message=str(final_list), subject="test", recipient='recipient@domain.com', sender="abc@abc.com")
Hi,
Not possible at the job level, but you can leverage the APIs and run a scenario to find long-running jobs and send emails.
https://doc.dataiku.com/dss/latest/python-api/jobs.html Here is an example you can use and modify as needed :
import dataiku
import datetime
from dataiku.core.message_sender import MessageSender
client = dataiku.api_client()
client = dataiku.api_client()
projects = client.list_projects()
# e.g adjust here to find job in this example 4 hours or older and state still running
four_hour_ago = datetime.datetime.now() - datetime.timedelta(hours=4)
timestamp = int(four_hour_ago.timestamp() * 1000)
job_info_str = ""
for p in projects:
pk = p["projectKey"]
project = client.get_project(pk)
jobs = project.list_jobs()
running_job_list = [job for job in jobs if job['state'] == 'RUNNING' and job['def']['initiationTimestamp'] >= timestamp]
if len(running_job_list) >=1:
job_info_str += p["projectKey"] + "\n" + "\n".join([str(job) for job in running_job_list]) + "\n"
print(job_info_str)
# Now send either one email for each long-running or create a list with the project key + job info from above.
s = MessageSender(channel_id='my-email-channel', type='mail-scenario', configuration={})
s.send(message=str(final_list), subject="test", recipient='recipient@domain.com', sender="abc@abc.com")
Hi,
Thanks for prompt reply.
Much appreciated.
Regards,
Weng Kin