Manipulation and documentation of custom model

Antal Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 85 Neuron
edited July 16 in General Discussion

Hi there,

I made a model using VisualML and utilizing a self-developed custom model plugin (CatBoost).

I also deployed the resulting model to the flow successfully.

I would love to be able to do 2 things with this model:

1. Export model documentation from the VisualML model summary "Export model summary". This throws errors with the custom plugin. However, all the information that would end up in the document seems to me to be available in the model summary page. Is there any way to get this to work?

2. Pick up the model object in a python notebook using the python API. That way I can use the predictor for other tasks (for example calculate permutation importance).

Normally I'd do it like this

# Retrieve trained model object
model = dataiku.Model("qC0ANLpX")
predictor = model.get_predictor()
clf = predictor._clf

But that throws an error, because the model is not a known model type in dataiku's inner workings, I guess. Is there another way to get the predictor/sklearn model object from the saved flow model?

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-3-c3a6a531446f> in <module>
      1 # Retrieve trained model object
      2 model = dataiku.Model("qC0ANLpX")
----> 3 predictor = model.get_predictor()
      4 clf = predictor._clf

/opt/dss/dataiku-dss-10.0.4/python/dataiku/core/ in get_predictor(self, version_id)
    206                 model_folder = target_model_folder
--> 208             self._predictors[version_id] = build_predictor_for_saved_model(model_folder, self.get_type(), sm.get("conditionalOutputs", []))
    209         return self._predictors[version_id]

/opt/dss/dataiku-dss-10.0.4/python/dataiku/core/ in build_predictor_for_saved_model(model_folder, model_type, conditional_outputs)
    331     from import get_saved_model_resolved_split_desc
    332     split_desc = get_saved_model_resolved_split_desc(model_folder)
--> 333     return build_predictor(model_type, model_folder, model_folder, conditional_outputs, core_params, split_desc)

/opt/dss/dataiku-dss-10.0.4/python/dataiku/core/ in build_predictor(model_type, model_folder, preprocessing_folder, conditional_outputs, core_params, split_desc, train_split_desc)
    396             pkl_path = osp.join(model_folder, "clf.pkl" if is_prediction else "clusterer.pkl")
    397             with open(pkl_path, "rb") as f:
--> 398                 clf = pickle.load(f)
    399                 try:
    400           "Post-processing model")

ModuleNotFoundError: No module named 'modelcatboost'

Best Answer

  • Antal
    Antal Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 85 Neuron
    Answer ✓

    Thanks for the tip. That wasn't the issue...

    I've managed to figure out a workaround, actually.

    Both functionalities weren't able to find the custom model code, because they're not executed from within the plugin's environment, but from the dataiku global environment.

    I've added the custom catboost code to the Global code library. That way the custom model code is available globally and the functionalities are able to find them. This way, I was able to make model object manipulation, model documentation generation and model views (error analysis, fairness report) work with models using the custom model plugin.

    It's not exactly pretty, but it does work!


Setup Info
      Help me…