Logging metrics

ben_p
ben_p Neuron 2020, Registered, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant Posts: 143 ✭✭✭✭✭✭✭

Hi everyone,

I am running SQL recipes that create datasets in DataIKU, on occasion I need to troubleshoot the output of these and I am interested in seeing metrics such as row count, trended over time.

I could write another SQL recipe that looks out the output dataset and creates another, but I wondered if there was a way to leverage the inbuilt in Status > Metrics to store them over time and present them in a dashboard?

Ben

Best Answer

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,226 Dataiker
    Answer ✓

    Hi @ben_p
    ,

    You can use the history of the dataset metrics in DSS. Metrics are added to the history when computed.

    You can retrieve these :

    - in the Flow +Dataset > Internal > Metrics , choose the Dataset/s.

    - Python APIs

    - Dashboards, You can directly publish metrics history to a dashboard as well + Metrics , choose the dataset and the metrics and you can display the history or last value

    Screenshot 2022-03-23 at 12.40.38.png

    Hope that helps!

Setup Info
    Tags
      Help me…