COMMAND EVALUATE CUSTOM MODEL

marirapellini

Hi

I have created a custom model for a multiclass prediction tasks and I want to use the command EVALUATE

My model returns a list for the predictions and I get this error:

Could not run command EVALUATE: : <class 'Exception'> : Can't handle model output: <class 'list'>

which classes are supported as output for the model?
Thank you

marirapellini

I also get this error: : Can't handle MLFlow model output: <class 'list'>

what could I do?

AurelienL

Hello,

Can you please make sure that your MLflow model returns a pandas dataframe or a numpy array? If you don't have control over that, you can wrap the model you are using in a subclass of PythonModel, as showcased here: https://developer.dataiku.com/latest/tutorials/machine-learning/experiment-tracking/xgboost-pyfunc/index.html#wrapping-an-xgboost-classifier-alongside-a-scikit-learn-pre-processing-layer

marirapellini

Hi

thank you

can a multiclass prediction model support a dataframe as output?

thank you

AurelienL

Yes, for a multiclass model, you can return either a dataframe having only one column (the prediction), or a dataframe with one column per class, in which case each column must contain the probability for this class. For the latter, please make sure that your column order match the labels order you used in the "class_labels" parameter of set_core_metadata.

In both cases the number of rows of your dataframe must match the number of rows of the input dataframe ("model_input" in the dev guide example).

marirapellini

Thanks

i wanted to ask, referred to the link you posted yesterday, is there a way to be able to train a model and then associated to it different Wrappers?

thank you

AurelienL

I'm not sure I understand your question, you would like to wrap your model in different wrappers? I'm curious about what your use case would be for that?

Anyway, I don't see why it would not work, you can simply write your different wrappers and make them all load and call the same underlying model.