Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I'm trying to build a flow that monitors data quality for a table.
How can I build a flow that sends emails, based on a condition in a column.
Example: If a date field comes with null values, send me an email.
Operating system used: Win
In your Dataset status tab create a Metric (you can use Column Statistics -> Empty Value Count or if this doesn't suit your needs create a custom Python prove at the bottom). Then use a Check to validate the Metric round thresholds. In your case anything above zero will be bad. Finally add steps to your Scenario to Compute Metrics and Run Checks. Checks can result in Warning (soft) or Failed (hard) so then run a Mail reporter on condition "outcome != 'SUCCESS'".
If you prefer to have the scenario not end on a failure when DQ issues happen you can use this trick to evaluate the metric in the scenario and conditionally execute a scenario step (like a mail reporter):
Hey Turribeach Thanks so much for the answer.
If it's not too much to ask, can you give me a more detailed step-by-step guide?
I tried to do it but I still haven't figered out how to do it, I'm still starting to use dss.
Thank so much!
Let me twist your request around. Write down all the steps you taken, explain how far you got and where you are stuck or where you get an error and we can try to help you out.