Issue calling deployed Dataiku model into Python code running in a container

Options
suhail_shaik
suhail_shaik Registered Posts: 4 ✭✭✭✭
edited July 16 in Using Dataiku

Hello,

I'm facing an issue trying to read a Dataiku Visual Model into a Python code which is a part of a webapp running in a container. I think the model call from dataiku.Model is making the correct call but it's not able to process the get_predictor() command correctly and I'm seeing a file not found error.

if i am running the webapp using local execution then it seems to work just fine the issue raises only when the webapp runs on a container

Attaching the log of the error for refrence

File "/opt/dataiku/python/dataiku/core/saved_model.py", line 202, in get_predictor
self._predictors[version_id] = build_predictor_for_saved_model(model_folder, self.get_type(), sm.get("conditionalOutputs", []))
File "/opt/dataiku/python/dataiku/core/saved_model.py", line 268, in build_predictor_for_saved_model
core_params = dkujson.load_from_filepath(osp.join(model_folder, "core_params.json"))
File "/opt/dataiku/python/dataiku/core/dkujson.py", line 36, in load_from_filepath
with open(filepath) as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/app/dataiku/DSS_DATA_DIR/saved_models/<PROJECTKEY>/<modelid>/versions/initial/core_params.json'

Answers

  • MehdiH
    MehdiH Dataiker, Dataiku DSS Core Designer, Dataiku DSS Core Concepts Posts: 21 Dataiker
    Options

    Hello @suhail_shaik

    Thank you for reaching out.

    On the attached log, the project key and model ids seem to be missing (look at the `<PROJECTKEY>` and `<modelid>` in the filepath)

    The issue must probably lie in the definition of your `filepath`, could you attach the code that performs this operation ?

    Cheers

  • suhail_shaik
    suhail_shaik Registered Posts: 4 ✭✭✭✭
    Options

    Hi @MehdiH

    The `<PROJECTKEY>` and `<modelid>` in the file path were deliberately added by me while posting the question here, in the actual error log the path is correct.

    The code that i have used is referenced from the below documentation. and you can see that the file path is not manually given.
    Interaction with saved models — Dataiku DSS 9.0 documentation

    However the above code is running on a containerized execution.

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker
    Options

    Hi,

    Unfortunately, it is not currently possible to use these APIs when the webapp is running in a container. You'll need to run this particular webapp locally.

  • Tsfan_22
    Tsfan_22 Registered Posts: 5
    Options

    Hello Clement,

    Would this limitation also apply to a notebook run in a container?

    Thanks!

  • MehdiH
    MehdiH Dataiker, Dataiku DSS Core Designer, Dataiku DSS Core Concepts Posts: 21 Dataiker
    Options

    Yes it also applies in containerized notebooks

  • Sreekrishnan
    Sreekrishnan Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 2
    Options

    hi is this fixed on Dataiku 12?

  • Sreekrishnan
    Sreekrishnan Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 2
    Options

    I am trying to get the dataiku trained model into my python recipe on same project and trying to get the predictor

    client = dataiku.api_client()
    project = client.get_default_project()
    #print(project.list_saved_models())
    sm_id = 'Anomaly detection Model'
    m = dataiku.Model(sm_id, ignore_flow = True)
    predictor = m.get_predictor()

    This works on notebook , but not on containerized execution. Getting error

    Job failed: Error in python process: At line 33: <class 'Exception'>: Error fetching dir /opt/dataiku/data/saved_models/

  • MehdiH
    MehdiH Dataiker, Dataiku DSS Core Designer, Dataiku DSS Core Concepts Posts: 21 Dataiker
    Options

    Hello Sreekrishnan,

    This should be fixed starting from Dataiku 12.3

    Best,

Setup Info
    Tags
      Help me…