Turn a custom model in the flow into a model object

Options
Tanguy
Tanguy Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2023 Posts: 112 Neuron

I was told that it was possible to turn a custom trained model, typically stored in a managed folder, into a visual model object in the flow.

Currently our flow looks like this:

model-folder_object.JPG

but we would like to see something like this in the flow:

model-flow_object.JPG

I couldn’t find any documentation on how to do this, so I’m turning to the Dataiku community for help

Answers

  • JordanB
    JordanB Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 293 Dataiker
    Options

    Hi @tanguy
    ,

    You can import MLFlow models as saved models to be deployed to your flow: https://doc.dataiku.com/dss/latest/mlops/mlflow-models/importing.html

    If your model is not currently saved as an MLFlow model, you can do so within your python recipe and load it into a managed folder. Then, use the DSS Python APIs to create your DSS saved model and deploy it.

    Thanks,

    Jordan

  • Tanguy
    Tanguy Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Neuron 2023 Posts: 112 Neuron
    Options

    Thanks Jordan. Is there any documentation or example regarding this step?

    use the DSS Python APIs to create your DSS saved model and deploy it.

  • JordanB
    JordanB Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 293 Dataiker
    edited July 17
    Options

    Hi @tanguy
    ,

    There is example code in the link I provided above (see here).

    I've added the code below for reference:

    import dataiku
    
     # if using API from inside DSS
    client = dataiku.api_client()
    
    project = client.get_project("PROJECT_ID")
    
    # 1. Create DSS Saved Model
    saved_model = project.create_mlflow_pyfunc_model(name, prediction_type)
    
    # 2. Load the MLflow Model as a new version of DSS Saved Model
    ## either from DSS host local filesystem:
    mlflow_version = saved_model.import_mlflow_version_from_path("version_id", model_directory, 'code-environment-to-use')
    ## or from a DSS managed folder:
    mlflow_version = saved_model.import_mlflow_version_from_managed_folder('version_id', 'managed_folder_id', path_of_model, 'code-environment-to-use')
    
    # 3. Evaluate the saved model version
    # (Optional, only for regression or classification models with tabular input data, mandatory to have access to the saved model performance tab)
    mlflow_version.set_core_metadata(target_column, classes, evaluation_dataset_name)
    mlflow_version.evaluate(evaluation_dataset_name)

    Thanks,

    Jordan

Setup Info
    Tags
      Help me…