How can I retrieve the metric values logged in experiment tracking using Dataiku APIs in Python?

Options
Suhail
Suhail Registered Posts: 18 ✭✭✭✭

Hello everyone,

I am currently working on a project in Dataiku in which i have run several experiments and logged various metrics in experiment tracking. Now I want to retrieve these metric values programmatically using Python.

Can anyone help me with this? Any guidance or insights would be much appreciated.

Thank you in advance!

Best Answer

  • Zach
    Zach Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 153 Dataiker
    edited July 17 Answer ✓
    Options

    Hi @Suhail
    ,

    You can retrieve the metrics from an Experiment Tracking run by using the MLflow run ID:

    import dataiku
    import mlflow
    
    project = dataiku.api_client().get_default_project()
    managed_folder = project.get_managed_folder("MY_FOLDER_ID")
    
    with project.setup_mlflow(managed_folder=managed_folder) as mlflow:
        run = mlflow.get_run("MY_RUN_ID")
        print(run.data.metrics)

    You can get the run ID from the Experiment Tracking page in the GUI, or you can save it when you start the run.

    Screenshot of the run ID in the GUI:

    4A084ED2-E548-48ED-A297-7324E7B709EF_1_201_a.jpeg

    Example code that prints the run ID when starting an MLflow run:

    import dataiku
    import mlflow
    
    project = dataiku.api_client().get_default_project()
    managed_folder = project.get_managed_folder("MY_FOLDER_ID")
    
    with project.setup_mlflow(managed_folder=managed_folder) as mlflow:
        # Note: if you don't call this (i.e. when no experiment is specified), the default one is used
        mlflow.set_experiment("My experiment")
    
        with mlflow.start_run(run_name="my_run") as run:
            print("Run ID:", run.info.run_id)
            mlflow.log_metric("foo", 8)
            mlflow.log_metric("bar", 10)

    You can find more information about the mlflow module in the MLflow documentation.

    Thanks,

    Zach

Answers

Setup Info
    Tags
      Help me…