Data audit on Dataiku Datasets.
Hi @AlexT
,
I need some help in doing data quality checks on DSS datasets columns. I know there is one plugin available for data quality checks on the source data which is in some SQL databases.
Can I get the code that will do some data quality operation on columns of DSS datasets.
Regards,
Ankur.
Best Answer
-
Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭OptionsYes, all the metrics will be available in the internal datasets. To add these to the flow do: +dataset, Internal, Metrics, Single Object, Dataset.
I hope this helps.
Best regards
Answers
-
Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭Options
Hi,
Before you use plugins or code, have you tried the metrics and checks functionality? One of its purposes is the automation of data quality and perhaps it is good enough for your needs.
I recommend that you complete this tutorial, which gives a good overview of the functionality: https://knowledge.dataiku.com/latest/courses/automation/metrics-checks-hands-on.html
This document also introduces the concept, https://knowledge.dataiku.com/latest/courses/automation/metrics-checks-summary.html
I hope this helps.
Best regards
-
Ankur30 Partner, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer Posts: 40 PartnerOptions
Hi @Manuel
,The below links you shared are very informative but I don't want to display these metrics output rather store it in DSS datasets.
Is there any way to store all the metrics that we computed in some DSS datasets?
Regards.
Ankur.