How to access the model metrics and features from a deployed model?
Hi,
I have deployed a model in Dataiku flow. I'm trying to access the model from a notebook and get some model details, as shown below.
<dataikuapi.dss.metrics.ComputedMetrics at 0x7ff5ddf5eee0>
But instead of returning a list of metrics, it returns the following.
import dataiku # Connect to the Dataiku instance client = dataiku.api_client() project = client.get_project(PROJECT_KEY) # Access the saved model model_id = 'MODEL_ID' saved_model = project.get_saved_model(model_id) active_version_id=saved_model.get_active_version()['id'] saved_model.get_metric_values(active_version_id)
How do I access the model metrics? Similarly, how do I access the features used in the model?
Operating system used: Windows
Operating system used: Windows
Answers
-
If you have a ComputedMetrics object, you can for instance call its get_all_ids method to list the metrics ids, then its get_global_value method to get the value for a given metric.
However, for a Saved Model's active version, it may be simpler, for what it seems you're trying to achieve, to get the metrics via get_version_details that returns a DSSTrainedPredictionModelDetails object, on which you can call get_performance_metrics for instance.
-
What about the features? How do I get the features used in a model that is present in the lab (not saved) ?
-
For the feature, the train model details can also give the preprocessing settings containing those.
features_handling = details.get_preprocessing_settings()['per_feature'] features_used = {name: (features_handling[name]['role']=='INPUT') for name in features_handling}features_used will have feature names as keys and a boolean as value, with True when it was used as an input to the model's training and False if it wasn't.
