How to add a custom calculated model metric into the evaluation store in Dataiku?

Suhail
Level 3
How to add a custom calculated model metric into the evaluation store in Dataiku?

Hello everyone,

I am currently working on a project in Dataiku and trying to log a custom model metric into the evaluation store. The model is not a visual ml model its a custom model that is logged in Dataiku as a saved model using the mlflow integration that Dataiku offers.
However, I am not sure how to add a custom calculated model metric into the evaluation store output.

Can anyone help me with this? Any guidance or insights would be much appreciated.

Thank you in advance!

2 Replies
Pierre-MaëlM
Dataiker

Hello @Suhail 

Thank you for your message. At the moment it is not possible to use a custom metric in an evaluation with a MLflow model. 

We have forwarded your inquiry to our product team.

Best regards,
Pierre-Maël

dadbuz
Level 3

Is this still the case? i think this capability has been introduced in 12.3. Can you provide an example of how one might add custom metrics to the model evaluation store for mlflow models. thank you!

0 Kudos