Folder access error in API endpoint

sdp

In the API endpoint script, I get an error when trying to access a folder which is placed in the resources of my environment, it's a sentence transformer model which I need to access.

Can someone please suggest?

sdp

In the API code I'm accessing this env in this way

sentence_transformer_home = os.getenv('SENTENCE_TRANSFORMERS_HOME')
model_path = os.path.join(sentence_transformer_home, 'sentence-transformers_all-MiniLM-L6-v2')
model = SentenceTransformer(model_path)

Turribeach

Check the permissions of that folder / file and if exists in your API node. It looks like you may need to download the transformers model first before you can invoke it in your function. The API node is a different machine so it won’t have the same thing your Designer node has.

Turribeach

What is

SENTENCE_TRANSFORMERS_HOME

set to? And have you installed the transformers model in this folder?

sdp

It's

also yes, I have downloaded the model

Turribeach

As you can see by the error the code is looking for the model in a tmp directory Dataiku uses to deploy the service. This is not desirable. You should download the model in a generic location in your API node machine and then use an absolute path to reference it in your API service code. That way any service using the transformers model can use the same downloaded model.

sdp

That makes sense, thank you @Turribeach
, I'll try this approach.