Saved model cannot be used in Python

Jennnnnny
Jennnnnny Registered Posts: 9
edited July 16 in Using Dataiku

Hello,

I've got this error message when using a custom python recipe

Code:

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu
from sklearn.metrics import silhouette_samples



# Read recipe inputs
EBS = dataiku.Dataset("EBS")

df = EBS_.get_dataframe()

model = dataiku.Model("NNNN") # replace by your model id

predictor = model.get_predictor() # get model
preprocessing = predictor.get_preprocessing() # get preprocessor

df_transformed = predictor.preprocess(df)[0] # transform data

scores = silhouette_samples(df_transformed, df.cluster_labels) # compute silhouette scores

df['score'] = scores

# Compute recipe outputs from inputs
# TODO: Replace this part by your actual code that computes the output, as a Pandas dataframe
# NB: DSS also supports other kinds of APIs for reading and writing data. Please see doc.


# Write recipe outputs

EBSn = dataiku.Dataset("score") # replace by your output dataset
EBSn.write_with_schema(df)

Error:

Job failed: Error in Python process: At line 20: <class 'Exception'>: Saved model NNNN cannot be used : declare it as input or output of your recipe More info about this error

Tagged:

Best Answer

  • FlorentD
    FlorentD Dataiker, Dataiku DSS Core Designer, Registered Posts: 25 Dataiker
    edited July 17 Answer ✓

    Hi,

    if you did not declare your save model as an input, you won't be able to use it.

    In any case, you should see your saved model in the input/output tabs (of the code editor of your recipe). If it is not the case, please use the "+Add" button to add your saved model to the recipe.

    If your custom python recipe is a plugin recipe, you will also have to add the good input parameter in yous JSON file:

    "inputRoles": [
      ...
      {
                "name": ...,
                "label": ...,
                "description": ...,
                "required": ...,
                "arity": ...,
                "acceptsDataset": false,
                "acceptsSavedModel": true
      },
    ...]

    And the in the code retrieve it :

    from dataiku.customrecipe import get_input_names_for_role
    
    saved_model_id = get_input_names_for_role(YOUR_INPUT_NAME)[0]

    Then you should be able to use it.

    Hope this helps.

    Best

Answers

  • Jennnnnny
    Jennnnnny Registered Posts: 9
    edited July 17

    Thank you this problem is solved!

    But i got another error:

    Job failed: Error in Python process: At line 26: <class 'TypeError'>: 'NoneType' object is not subscriptable More info about this error
    View job details

    Can you help me with this please?

  • Jennnnnny
    Jennnnnny Registered Posts: 9
    edited July 17

    Line 26 :

    predictor = model.get_predictor() # get model
  • FlorentD
    FlorentD Dataiker, Dataiku DSS Core Designer, Registered Posts: 25 Dataiker
    edited July 17

    Very hard to debug without the code...

    But I would suspect, if your code is the "same" that the one you shared before, this line

    df_transformed = predictor.preprocess(df)[0]

    could be the root of your problem.

    You should have more details in the job log, and also do not hesitate to log if you really have something when you run

    predictor.preprocess(df)
  • Jennnnnny
    Jennnnnny Registered Posts: 9

    Yes the code is the same as before

  • FlorentD
    FlorentD Dataiker, Dataiku DSS Core Designer, Registered Posts: 25 Dataiker

    So it seems your model is empty. You should check the model Id, if it the good one. You should also pay attention that your model could have been trained with a different version of (let's suppose) scikit-learn than the one you use in your custom recipe, which can lead to incompatibilities.

    But I don't get the point where you want to go... If you are trying to re-code an evaluation recipe, you should do rather go to the associated recipe, as replicating the preprocessing manually is a bit of a doomed endeavour (some preprocess are "simple", but many are not)

  • Jennnnnny
    Jennnnnny Registered Posts: 9

    Hello,

    The Model ID it's ok so i really don't understand why...

    Why i'm doing this, it's because in my previous receipe we don't know how to calculate the silhouette score, so i'm trying to do it with a python code.

    Br

Setup Info
    Tags
      Help me…