Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I am currently working on a project in Dataiku and trying to log a custom model metric into the evaluation store. The model is not a visual ml model its a custom model that is logged in Dataiku as a saved model using the mlflow integration that Dataiku offers.
However, I am not sure how to add a custom calculated model metric into the evaluation store output.
Can anyone help me with this? Any guidance or insights would be much appreciated.
Thank you in advance!
Thank you for your message. At the moment it is not possible to use a custom metric in an evaluation with a MLflow model.
We have forwarded your inquiry to our product team.
Is this still the case? i think this capability has been introduced in 12.3. Can you provide an example of how one might add custom metrics to the model evaluation store for mlflow models. thank you!