Copying / saving Checks and metrics
Hello,
Is there any way to copy checks and metrics from one data set to another?
Is it possible to save a custom code check as a plugin ?
Operating system used: Centos 7
Best Answer
-
Miguel Angel Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 118 Dataiker
Hi,
Using the Python API you can copy the metrics/checks metadata from a dataset:
import dataiku
client=dataiku.api_client()
project=client.get_project(dataiku.default_project_key())
sourcedataset=project.get_dataset("<source dataset name>")
wantedmetrics=sourcedataset.get_settings().get_raw()["metrics"]
wantedchecks=sourcedataset.get_settings().get_raw()["metricsChecks"]
And apply them to any other dataset in the node. For example, to apply them to a dataset in the same project:
targetdataset=project.get_dataset("<target dataset name>")
datasetsettings=targetdataset.get_definition()
datasetsettings["metrics"]=wantedmetrics
datasetsettings["metricsChecks"]=wantedchecks
targetdataset.set_definition(datasetsettings)
Regadring plugins, in the plugin components you can see two components to develop metrics and checks probes:
Answers
-
Maryam Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 7 Partner
Thank you Miguel !
-
Hello, just as an update here,
Now with data quality rules replacing checks on datasets, as of version 12.6.6, you can easily copy and paste DQ rules from one dataset from another, and in 13.1.0, you can create data quality templates for even more streamlined reuse.
Let me know if you have questions or other feebdack!
Katie