How can I retrieve the metric values logged in experiment tracking using Dataiku APIs in Python?

[Suhail]

Hello everyone,

I am currently working on a project in Dataiku in which i have run several experiments and logged various metrics in experiment tracking. Now I want to retrieve these metric values programmatically using Python.

Can anyone help me with this? Any guidance or insights would be much appreciated.

Thank you in advance!

[ZachM]

Hi @Suhail,

You can retrieve the metrics from an Experiment Tracking run by using the MLflow run ID:

import dataiku
import mlflow

project = dataiku.api_client().get_default_project()
managed_folder = project.get_managed_folder("MY_FOLDER_ID")

with project.setup_mlflow(managed_folder=managed_folder) as mlflow:
    run = mlflow.get_run("MY_RUN_ID")
    print(run.data.metrics)

You can get the run ID from the Experiment Tracking page in the GUI, or you can save it when you start the run.

Screenshot of the run ID in the GUI:

Example code that prints the run ID when starting an MLflow run:

import dataiku
import mlflow

project = dataiku.api_client().get_default_project()
managed_folder = project.get_managed_folder("MY_FOLDER_ID")

with project.setup_mlflow(managed_folder=managed_folder) as mlflow:
    # Note: if you don't call this (i.e. when no experiment is specified), the default one is used
    mlflow.set_experiment("My experiment")

    with mlflow.start_run(run_name="my_run") as run:
        print("Run ID:", run.info.run_id)
        mlflow.log_metric("foo", 8)
        mlflow.log_metric("bar", 10)

You can find more information about the mlflow module in the MLflow documentation.

Thanks,

Zach

[Suhail]

Hi @ZachM ,

Thanks for the help 👍