How to add a custom calculated model metric into the evaluation store in Dataiku?

Options
Suhail
Suhail Registered Posts: 18 ✭✭✭✭

Hello everyone,

I am currently working on a project in Dataiku and trying to log a custom model metric into the evaluation store. The model is not a visual ml model its a custom model that is logged in Dataiku as a saved model using the mlflow integration that Dataiku offers.
However, I am not sure how to add a custom calculated model metric into the evaluation store output.

Can anyone help me with this? Any guidance or insights would be much appreciated.

Thank you in advance!

Answers

  • Pierre-MaëlM
    Pierre-MaëlM Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 3 Dataiker
    Options

    Hello @Suhail

    Thank you for your message. At the moment it is not possible to use a custom metric in an evaluation with a MLflow model.

    We have forwarded your inquiry to our product team.

    Best regards,
    Pierre-Maël

  • dadbuz
    dadbuz Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer Posts: 16 ✭✭✭✭
    Options

    Is this still the case? i think this capability has been introduced in 12.3. Can you provide an example of how one might add custom metrics to the model evaluation store for mlflow models. thank you!

Setup Info
    Tags
      Help me…