How to reference global variables in API

NikMishin
NikMishin Registered, Frontrunner 2022 Participant Posts: 8 ✭✭✭✭
edited July 16 in Using Dataiku

Hi,

I have a set of Global variables (the ones you set in Administration->Settings->Variables) and would like to reference these in Python in an API. The example code is:

import dataiku
def api_py_function(param1):
    return param1 + dataiku.get_custom_variables()["environment"]

It works absolutely fine in a Notebook, but gives the following error if I try to use it in an API.

Failed: Failed to run function : <class 'Exception'> : Default project key is not specified (no DKU_CURRENT_PROJECT_KEY in env)

Please help:)

Thank you!

Best Answer

  • dimitri
    dimitri Dataiker, Product Ideas Manager Posts: 33 Dataiker
    edited July 17 Answer ✓

    Hi @NikMishin
    ,

    API endpoints are meant to be deployed on external API nodes. Therefore, custom python functions need to be written as if they were used outside of DSS, since there is no context inherited from the DSS node on which they are developed and tested.

    Thus, if you want to access global instance-level variables (which are defined in Administration->Settings->Variables), then you need to go through the public API.

    The code sample below should help you to do that.

    import dataiku
    
    dataiku.set_remote_dss("http(s)://DSS_HOST:DSS_PORT/", "Your API Key secret")
    
    def api_py_function(param1):
        return param1 + dataiku.get_custom_variables(project_key="PROJECT KEY")["environment"]

    Have a great day!

Answers

  • NikMishin
    NikMishin Registered, Frontrunner 2022 Participant Posts: 8 ✭✭✭✭

    Thank you Dimitri! I guess I was working with the assumption that you can set global variables on API nodes as well that the endpoint code could reference. What you said makes perfect sense to me. Thank you again.

  • dimitri
    dimitri Dataiker, Product Ideas Manager Posts: 33 Dataiker
    edited July 17

    Hi @NikMishin
    ,

    If you want to set variables that have a specific value for a single API Node, the recommended way is to use environment variables.

    You can export them in the profile of the user, or in the DATA_DIR/bin/​env-site.sh file of your API Node as described below. (You have to restart the instance, the entire procedure is detailed in the Advanced customization part of our documentation).

    echo 'export MY_CUSTOM_ENV_VARIABLE="custom_value"' >> bin/env-site.sh

    Then, you can access it from the python function of you API endpoint.

    import os
    
    def api_py_function(param1):
        return param1 + os.environ['MY_CUSTOM_ENV_VARIABLE']

    Have a great day!

  • importthepandas
    importthepandas Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 115 Neuron

    Hi @dimitri
    - bumping this for a slightly related topic.

    With endpoints such as lookups and for enrichments on prediction endpoints, is it expected behavior for DSS to not recognize a variable set in a connection used for the enrichment? I'm assuming this is expected based on your previous answer about python endpoints, but just want to confirm. We've a setup where we set variables to flip from "dev" to "prod" (design to auto) upon deployment for database schemas, so we use variables in the connections. We tried to use these same connections for lookup endpoints but receive errors due to the variables. We ended up working around by setting hard coded connections with an API_ prefix.

    edit to note: we deploy API services on EKS and set connections in the infrastructures on an independent deployer node.

    Thank you!

Setup Info
    Tags
      Help me…