How to access the model metrics and features from a deployed model?

MNOP
Level 3
How to access the model metrics and features from a deployed model?

Hi,
I have deployed a model in Dataiku flow. I'm trying to access the model from a notebook and get some model details, as shown below. 

 

import dataiku
# Connect to the Dataiku instance
client = dataiku.api_client()
project = client.get_project(PROJECT_KEY)  
 
# Access the saved model
model_id = 'MODEL_ID'
saved_model = project.get_saved_model(model_id)
 
active_version_id=saved_model.get_active_version()['id']
saved_model.get_metric_values(active_version_id)

 

But instead of returning a list of metrics, it returns the following. 

 

<dataikuapi.dss.metrics.ComputedMetrics at 0x7ff5ddf5eee0>

 


How do I access the model metrics? Similarly, how do I access the features used in the model?


Operating system used: Windows


Operating system used: Windows

0 Kudos
3 Replies
AdrienL
Dataiker

If you have a ComputedMetrics object, you can for instance call its get_all_ids method to list the metrics ids, then its get_global_value method to get the value for a given metric.

However, for a Saved Model's active version, it may be simpler, for what it seems you're trying to achieve, to get the metrics via get_version_details that returns a DSSTrainedPredictionModelDetails object, on which you can call get_performance_metrics for instance.

0 Kudos
MNOP
Level 3
Author

What about the features? How do I get the features used in a model that is present in the lab (not saved) ?

 

0 Kudos
AdrienL
Dataiker

For the feature, the train model details can also give the preprocessing settings containing those.

features_handling = details.get_preprocessing_settings()['per_feature']
features_used = {name: (features_handling[name]['role']=='INPUT') for name in features_handling}

features_used will have feature names as keys and a boolean as value, with True when it was used as an input to the model's training and False if it wasn't.

0 Kudos