Turn a custom model in the flow into a model object

Tanguy
Tanguy Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2023 Posts: 120 Neuron

I was told that it was possible to turn a custom trained model, typically stored in a managed folder, into a visual model object in the flow.

Currently our flow looks like this:

model-folder_object.JPG

but we would like to see something like this in the flow:

model-flow_object.JPG

I couldn’t find any documentation on how to do this, so I’m turning to the Dataiku community for help

Best Answer

  • JordanB
    JordanB Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 296 Dataiker
    edited July 17 Answer ✓

    Hi @tanguy
    ,

    There is example code in the link I provided above (see here).

    I've added the code below for reference:

    import dataiku
    
     # if using API from inside DSS
    client = dataiku.api_client()
    
    project = client.get_project("PROJECT_ID")
    
    # 1. Create DSS Saved Model
    saved_model = project.create_mlflow_pyfunc_model(name, prediction_type)
    
    # 2. Load the MLflow Model as a new version of DSS Saved Model
    ## either from DSS host local filesystem:
    mlflow_version = saved_model.import_mlflow_version_from_path("version_id", model_directory, 'code-environment-to-use')
    ## or from a DSS managed folder:
    mlflow_version = saved_model.import_mlflow_version_from_managed_folder('version_id', 'managed_folder_id', path_of_model, 'code-environment-to-use')
    
    # 3. Evaluate the saved model version
    # (Optional, only for regression or classification models with tabular input data, mandatory to have access to the saved model performance tab)
    mlflow_version.set_core_metadata(target_column, classes, evaluation_dataset_name)
    mlflow_version.evaluate(evaluation_dataset_name)

    Thanks,

    Jordan

Answers

  • JordanB
    JordanB Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 296 Dataiker

    Hi @tanguy
    ,

    You can import MLFlow models as saved models to be deployed to your flow: https://doc.dataiku.com/dss/latest/mlops/mlflow-models/importing.html

    If your model is not currently saved as an MLFlow model, you can do so within your python recipe and load it into a managed folder. Then, use the DSS Python APIs to create your DSS saved model and deploy it.

    Thanks,

    Jordan

  • Tanguy
    Tanguy Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2023 Posts: 120 Neuron

    Thanks Jordan. Is there any documentation or example regarding this step?

    use the DSS Python APIs to create your DSS saved model and deploy it.

Setup Info
    Tags
      Help me…