Checks help to make sure that the flow’s pipeline is behaving like expected. They also accelerate the development process by:
However, I noticed that these two features do not always work when implementing a check at the entire dataset level (not at the partition level) of a partitioned table. In particular:
Here is an illustration of this issue:
This improvement could be added to this one.
I've raised the issue of the check failing message not being informative recently in direct feedback to someone at Dataiku. This happens on all datasets, not just partitioned ones.
In general, metrics and checks are a really nice feature but there are some improvements that could be made both in how they operate and in the UI (too many clicks).
Thanks for the info @Marlan. For NP datasets, I haven't encountered any problems. Do you have a simple example to reproduce?
Thanks for your idea @tanguy Your idea meets the criteria for submission, we'll reach out should we require more information.
If you’re reading this and think this would be a great capability to add to DSS, be sure to kudos the original post!
Only members of the Community can comment.