Write mlflow logs to a managed folder

Skanda Gurunathan

I am trying to use mlflow with dataiku and trying to write all the experiment tracking logs and metrics to a specific folder.

I can see the folder in the flow, but the logs and metrics dont appear there but it appears in the dataiku's ExperimentTracking UI. What's wrong in the code doing this?

project = dataiku.api_client().get_default_project()
managed_folder = project.get_managed_folder('YvNMItjP')

mlflow_extension = project.get_mlflow_extension()

with project.setup_mlflow(managed_folder=managed_folder) as mlflow_handle:
    print(mlflow_handle.get_tracking_uri())
  
    experiment_id = str(int(time.time()))
    mlflow_handle.create_experiment(experiment_id)
    mlflow_handle.set_experiment(experiment_id)

    with mlflow_handle.start_run(run_name="21-12-2023") as run:
        print(run.info.run_id)
        mlflow_handle.tracking.MlflowClient().set_experiment_tag(
            experiment_id, "library", "Keras")

        mlflow_handle.tracking.MlflowClient().set_experiment_tag(
            experiment_id, "predictionType", "BINARY_CLASSIFICATION")

---

Operating system used: Linux

Turribeach

Why do you want the logs and metrics written to the folder? What are you trying to achieve?

Skanda Gurunathan

I want to view the files and logs in that folder / kinda data governance thing. We dont want to log to a folder that is invisible to users in the flow. So basically instead of logging to default mlflow folder, I want to log it to a specific managed folder in the flow