raise error when a check does not work

[tanguy]

Dataiku offers the ability to add custom metrics and checks which is very useful for testing the flow.

However, some checks can be improperly defined or break for whatever reason. In this case, dataiku ignores the check, and the flow can be built without the user being notified of a problem. This is a silent error: the check fails but no error is raised.

Here is a demo showing this problem:

• the user has a custom python check containing an error (the check fails)

• the check is supposed to run at each build on the table (e.g. the following option is checked) 

• and yet, when the table is built, no errors is raised: the user is not notified of the problem on the check potentially (and more importantly) on the table

It would be appreciated if dataiku could raise an error in this case (which, from what I know, is the default behaviour of common testing frameworks).

[Turribeach]

Wow! How on earth this is not considered a bug? ("It's not a bug it's a feature"). This is a bug 100% to me since there is no way that you would expect that a custom piece of code that fails  will do so silently. This bug is extremely dangerous. We rely a lot on metrics and checks to assure our data meets our data quality requirements. In particular machine learning models are extremelly susceptible to values out of range, nulls, etc. Gets my vote 110% and I will ask others on my team to vote for this. 

[ktgross15]

Hi @tanguy ,

Thank you very much for the feedback! We are exploring general improvements here and will let you know if we have any updates.

Best,

Katie