Checks with custom error/warning messages
Is it possible to create a custom error message for a check? I've created some checks for important datasets but find the default error messaging of "Checks on the output produced 1 error" a bit lacking as this does not give any information on what's wrong to users who are not as versed in Dataiku or programming in general.
I would love to be able to create dynamic messages, using variables to describe potential issues as clear as possible but even a note as to which check failed would be great!
Operating system used: Windows
Operating system used: Windows
Best Answer
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,925 NeuronWell I think this is more to do training users how to use Dataiku to be honest. In the job screen if you scroll down the Activity log the error is highlighted in red and shows what the rule is:
Then if you click on the Output Dataset button and navigate to the Data Quality tab you clearly see the rules failing there:
So I am not sure what else could Dataiku do. You need to consider that a dataset could fail many data quality rules so displaying them all in the job screen may not be ideal.
PS: Checks are called Data Quality in v12.6.x and above.
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,925 Neuron
Where exactly do you see this error and want to see a custom message?
-
In the job overview (for the job that builds the dataset with the check) I get an error like this:
The check itself in this case is a minimum number of records that is not being met. I think I would like to see the custom message in this same activity log if possible.
-
Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 319 NeuronI agree. If the message can say checks have failed, it should be able to say what checks in which datasets have failed. That would be a lot more helpful.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,925 Neuron
PS2: Nothing stops you from building your own WebApp to show your jobs and data quality checks in whichever way you want.
-
Thank you for the answer, I did not know it also displayed the specific check in the activity logs. Though I'm still a bit unsatisfied by the clarity of the message as it gives no usable information on how to find the issue or remedy it to users who are not as familiar with the technical site of Dataiku.
For our case, we have a team of users who supply the data and process it via scenarios only. They are not familiar with the technical side of Dataiku as much and would not know what to do here, meaning they will have to request help from developers. These users would be able to find out and remedy the issue themselves in most cases if the error displayed what is wrong exactly. This error message could be custom of course, generated by the developers in a python-script check, but should be given in the error message of the job directly, in my opinion.
