Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Use file from code env resources directory with API Test queries
Added on January 11, 2023 2:19PM
Likes: 1
Replies: 4
Antal
Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 93 Neuron
I've built a python API endpoint that uses a certain code environment.
In the resources folder of that code environment is a file.
If I run the API code in a notebook using that code env, I'm able to access that file. But when I run Test queries in the API Designer, the file cannot be found. The code used is the same in both instances.
It seems when you run Test Queries, a temporary API node is created which the queries are then run against. However the file from the code env resource directory seems to not be copied to that temp instance.
What am I doing wrong?
I store the path to the resource directory in an environment variable in the code env using a resource script, which stores the relative path. I have "Run resources initialization script on API node." checked.
/var/dss_data/code-envs/resources/python/Python37_nn_api/certificate/
When I read that variable from a notebook using the code env by running os.getenv("CERT_CACHE") it reads
# Set data cache directory set_env_path("CERT_CACHE", "certificate") # Grant everyone read access to files in folder grant_permissions(os.getenv("CERT_CACHE"))
and when I run Test Queries in the API Designer it reads as
/var/dss_data/tmp/apinode-devserver/services/FRMAPI/gens/FRMAPI5250601358624916284/endpoint-frm_api/python-env-live/resources/certificate/
Operating system used: Windows
Answers
-
Antal Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 93 Neuron
I managed to find a workaround.
Still interested to know if it's possible for APIs to "inherit" files in the code env resource directory if they were manually added.
-
JordanB Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 297 DataikerHi @Antal
,API services will include project libraries for project library files included within the same project as the API service. If a package exists in another project and not the one you are currently developing your API service, the project libraries must be shared with the project you are using to develop your API service by going to the project libraries for the "target" project, and entering in the "source" project within the external-libraries.json file under "importLibrariesFromProjects":
Note that your user must also have "read project content" access to the "source" project. Please see the following on this as well: https://doc.dataiku.com/dss/latest/python/reusing-code.html#importing-libraries-from-other-projectsPlease let us know if this helps.Thanks,Jordan -
What was the workaround? Can you please let us know, I have the same issue
https://community.dataiku.com/t5/Using-Dataiku/Folder-access-error-in-API-endpoint/m-p/42478#M16199 -
Antal Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 93 Neuron
In my case I needed to add SSL certificates to the cert file.
I put code that defines the certificates and appends them to the cert file in the Resources tab of the code env used by the API as well, so that the code is run when the code env is built.
So, a workaround for my specific case.
