Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

Write mlflow logs to a managed folder

skandagn
Level 2
Write mlflow logs to a managed folder

I am trying to use mlflow with dataiku and trying to write all the experiment tracking logs and metrics to a specific folder. 

I can see the folder in the flow, but the logs and metrics dont appear there but it appears in the dataiku's ExperimentTracking UI. What's wrong in the code doing this?

 

 

project = dataiku.api_client().get_default_project()
managed_folder = project.get_managed_folder('YvNMItjP')

mlflow_extension = project.get_mlflow_extension()

with project.setup_mlflow(managed_folder=managed_folder) as mlflow_handle:
    print(mlflow_handle.get_tracking_uri())
  
    experiment_id = str(int(time.time()))
    mlflow_handle.create_experiment(experiment_id)
    mlflow_handle.set_experiment(experiment_id)

    with mlflow_handle.start_run(run_name="21-12-2023") as run:
        print(run.info.run_id)
        mlflow_handle.tracking.MlflowClient().set_experiment_tag(
            experiment_id, "library", "Keras")

        mlflow_handle.tracking.MlflowClient().set_experiment_tag(
            experiment_id, "predictionType", "BINARY_CLASSIFICATION")

 


Operating system used: Linux

 

0 Kudos
2 Replies
Turribeach

Why do you want the logs and metrics written to the folder? What are you trying to achieve?

0 Kudos
skandagn
Level 2
Author

I want to view the files and logs in that folder / kinda data governance thing. We dont want to log to a folder that is invisible to users in the flow. So basically instead of logging to default mlflow folder, I want to log it to a specific managed folder in the flow 

 

0 Kudos