save custom model in plugin and reuse it

og_dz_disrupt

Hi DSS community,

I am trying to create an optimization (not an ML model using sklearn and other libraries) plugin for a specific project. The model I created has .fit() and .predict() functions.

When training the model, I used the .fit() function. How can I use this saved model later with its .predict() function, in the same plugin ?

Small example:

1) Training my model using a first python recipe recipe.py:

my_model=model.fit()

2) Use it with a second recipe using the same plugin in the prediction_recipe.py:

my_prediction=my_model.predict()

Thanks in advance

pmasiphelps

Hi,

I'd recommend having two "recipe" components as part of your plugin - one for training, one for scoring.

Here's an example of how the deep learning image plugin has separate train/score recipes:

The train recipe has your train dataset as an input, and it outputs a serialized model object (saved as ONNX, h5, pickle, etc.) saved to a managed folder. The score recipe then takes the model folder and your dataset-to-be-scored as inputs, and outputs predictions to a dataset.

(Note that the deep learning image plugin is now deprecated in favor of deep learning integrated in the regular visual ML interface - but this is still a good example of how you can structure it)

Here's the github repo for this plugin: https://github.com/dataiku/dss-plugin-deeplearning-image/tree/master/custom-recipes

In each of the sub-folders here^, there's a separate recipe (for train, score, etc.).

Best,

Pat