Send condition-based emails
I'm trying to build a flow that monitors data quality for a table.
How can I build a flow that sends emails, based on a condition in a column.
Example: If a date field comes with null values, send me an email.
Operating system used: Win
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,981 Neuron
In your Dataset status tab create a Metric (you can use Column Statistics -> Empty Value Count or if this doesn't suit your needs create a custom Python prove at the bottom). Then use a Check to validate the Metric round thresholds. In your case anything above zero will be bad. Finally add steps to your Scenario to Compute Metrics and Run Checks. Checks can result in Warning (soft) or Failed (hard) so then run a Mail reporter on condition "outcome != 'SUCCESS'".
If you prefer to have the scenario not end on a failure when DQ issues happen you can use this trick to evaluate the metric in the scenario and conditionally execute a scenario step (like a mail reporter): -
nicols13 Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 2 ✭
Hey Turribeach Thanks so much for the answer.
If it's not too much to ask, can you give me a more detailed step-by-step guide?
I tried to do it but I still haven't figered out how to do it, I'm still starting to use dss.
Thank so much! -
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,981 Neuron
Let me twist your request around. Write down all the steps you taken, explain how far you got and where you are stuck or where you get an error and we can try to help you out.