Submit your innovative use case or inspiring success story to the 2023 Dataiku Frontrunner Awards! LET'S GO

The specified Code Environment is not used when running tests for Python Function API Endpoint

Solved!
Wolfram36
Level 2
The specified Code Environment is not used when running tests for Python Function API Endpoint

I have setup a Conda environment which I am successfully using for two Webapps and some recipes.

However, when trying to create a Python Function API Endpoint, and using the "RUN TEST QUERIES" function, I get errors such as "ModuleNotFoundError: No module named 'transformers'" indicating that the prepared Code Environment is not used. Also, when observing the list of virtual environments in Anaconda, I can see that new VMs are created every time I run test queries, with the general name "env" and not using the dedicated "api_p38_env". The environments called "env" have completely different packages than the environment I have prepared.

I have specified theCode Environment settings as can be seen in the attached screenshot.

Also, I tried using two different Code Environments, so that the one used for the two Webapps is separate, but to no avail. 

The log file for the test, including errors, has been attached. I would appreciate any help.

I noticed that my issue looks similar to this one: https://community.dataiku.com/t5/Using-Dataiku/Use-file-from-code-env-resources-directory-with-API-T...

setting.png

code_env.png

 

 


Operating system used: MacOS 13.3.1 (a), Arm64


Operating system used: MacOS 13.3.1 (a), Arm64

0 Kudos
1 Solution
JordanB
Dataiker

Hi @Wolfram36,

I see, real-time API's are not listed under the free license version. Sorry about that. You may want to explore the Dataiku Online 14 day free trial: https://www.dataiku.com/product/get-started/

Thanks!
Jordan

View solution in original post

0 Kudos
7 Replies
JordanB
Dataiker

Hi @Wolfram36,

The transformers package was likely not there at the time that the dev server started and installed your code env. To make sure that the transformers package gets installed, please complete the following steps:

Double-check that transformers is listed in the "packages to install" tab under "requested packages".  Then, please rebuild your code environment from the Administration > Code envs with the "Rebuild env" checkbox checked:

Screen Shot 2023-05-17 at 5.22.00 PM.png

Next, please restart your API dev server from Administration > Settings > Deployer:

Screen Shot 2023-05-17 at 5.43.30 PM.png

And, test your api test query once more.

Please send us an updated log if you receive an error.

Thanks!

Jordan

 

Wolfram36
Level 2
Author

Thanks @JordanB for your fast assistance.

I carried out the steps you suggested. (Initially, I used Terminal and pip install to setup the virtual environment on Anaconda, but this time I added all packages on Dataiku instead.)

Still, when starting the test, I can see that a new virtual environment called "env" is launched in Anaconda. But, it seems that this new virtual environment has the packages as declared in Dataiku.

The new error is:

com.dataiku.dip.io.SocketBlockLinkKernelException: Failed to initiate function server : <class 'Exception'> : Default project key is not specified (no DKU_CURRENT_PROJECT_KEY in env)

See attached log file and screenshot.

anaconda_newly_created_env.png

I referred to the following instructions, but could not figure out how to specify the DKU_CURRENT_PROJECT_KEY and how to use it in my API:

0 Kudos
JordanB
Dataiker

Hi @Wolfram36,

The error message is telling us that it does not know to which project should it connect.
When running the same code from a notebook you'd not hit this error because 'DKU_CURRENT_PROJECT_KEY' corresponds to the project in which the notebook exists. 
The code the API designer is running has no notion by default of what project is supposed to be talking to. Furthermore, the idea is that ultimately the code will be run from the API node, which is outside of the Design node. 
 
You will need to install the dataikuapi version of the dataiku package and set_remote_dss(). You will have to connect to DSS instance by host and api-key, then connect to the project by the project key and then you can run the rest of your code. 
import dataikuapi
# Connect to DSS and get a client
dataiku.set_remote_dss("http://localhost:port", "insert-api-key")
client = dataiku.api_client()
 
 
Please let us know if you run into any further issues.
 
Thanks!
Jordan
0 Kudos
Wolfram36
Level 2
Author

Hi @JordanB 

I followed the following steps:

  1. Added 'dataiku-api-client' to the list of packages to install for my selected virtual environment
  2. Next, I wanted to create an API key, so I navigated to 'http://localhost:11200/projects/myprojectname/security/api' but I was met with the text "The public API is not available in this version of DSS."
  3. Next attempt was to navigate to 'http://localhost:11200/projects/myprojectname/api-designer/my-api/config/'. This screen allows creation of API Keys, but after refreshing the page, the key is gone, so I am not sure how to revoke an issued key in the future.
  4. Anyways, now I had an API key, so I added the following code on top of the code for my API:

 

import dataiku
import dataikuapi
import logging
host = "http://localhost:11200"
apiKey = "my--newly--created--key"
client = dataikuapi.DSSClient(host, apiKey)
logging.info(client.list_project_keys())

 

But, now I get the error "Failed to initiate function server : <class 'dataikuapi.utils.DataikuException'> : com.dataiku.dip.exceptions.NotAuthenticatedException: Unknown API Key"

 

Are my difficulties related to my license (i.e. the Free version)? I am still evaluating your product and could potentially switch to sign up for a limited trial, if that helps.

0 Kudos
JordanB
Dataiker

Hi @Wolfram36,

You need a global (instance) API key for this, not a project level api key. What happens when you select your icon at the top right > profile and settings > API Keys > +New API Keys.

Screen Shot 2023-05-19 at 10.46.40 AM.png

Screen Shot 2023-05-19 at 10.47.30 AM.png

Can you please give that a try?

Thanks!

Jordan

0 Kudos
Wolfram36
Level 2
Author

Hi @JordanB 

I have attached a screenshot for what I see after clicking on that link.

It seems that I cannot create an API for a Python function with the FREE license. 

 

0 Kudos
JordanB
Dataiker

Hi @Wolfram36,

I see, real-time API's are not listed under the free license version. Sorry about that. You may want to explore the Dataiku Online 14 day free trial: https://www.dataiku.com/product/get-started/

Thanks!
Jordan

0 Kudos