Reference global variables in python recipe

Options
yesitsmeoffical
yesitsmeoffical Registered Posts: 15
edited July 16 in Using Dataiku

In Global Variables section, I created a numeric global variable -> "global_var": 1

I would like to reference this variable in a python recipe:

import dataiku
from dataiku import pandasutils as pdu

project = dataiku.api_client().get_project("my_project")
projectVars = project.get_variables()
global_var = projectVars["global_var"]

dataset = dataiku.Dataset("my_dataset")
df = dataset.get_dataframe()

if global_var == 1:
   df['col_A'] = 'Y'
else:
   df['col_A'] = 'N'

I am getting the error message below:

Job failed: Error in Python process: At line 13: <class 'KeyError'>: global_var

Tagged:

Answers

  • Catalina
    Catalina Dataiker, Dataiku DSS Core Designer, Registered Posts: 135 Dataiker
    edited July 17
    Options

    This code works:

    import dataiku
    from dataiku import pandasutils as pdu
    
    project = dataiku.api_client().get_project("PROJECT_KEY")
    projectVars = project.get_variables()
    global_var = projectVars["standard"]["global_var"]
    
    print(global_var)
    
    dataset = dataiku.Dataset("my_dataset")
    df = dataset.get_dataframe()
    
    
    if global_var == 1:
       df['column1'] = 'Y'
    else:
       df['column1'] = 'N'
     

  • yesitsmeoffical
    yesitsmeoffical Registered Posts: 15
    Options

    does ["standard"] imply that global variables?

    I ended up using a different approach and it worked:

    global_var = int(dataiku.get_custom_variables()["globar_var"])

    in this approach you don't have to specify the project name, but need to force convert the variable back to integer.

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,720 Neuron
    Options

    The code Catalina produced is for project global variables, not instance global variables. Project variables are defined at project level (Project => ... => Variables), instance global variables at instance level (Admin => Settings => Global Variables). Project variables only exist in the context of a project hence why you need a project ID and a project handle to manipulate them. Note that the dataiku.get_custom_variables() method will retrieve all available variables, project and instance globals.

    Within a project you can define Global variables and Local variables at project level. Local project variables are there to override Instance or Project Global variables. They are called "Local" because they won't be included in the project bundle so won't be migrated to other envs. Project Global variables can override Instance Global variables or simply define project level variables. The project.get_variables() method returns projectVars["standard"] for Project Global variables and projectVars["local"] for Project Local variables. Which is why your code was failing. Using dataiku.get_custom_variables() you get all variables in a single dictionary but obviously only 1 value per variable name if the variable is overriden somewhere.

    So for any future posts please make sure you clarify if you are using Instance or Project Global variables.

    With regards to your int() point you should consider that dataiku.get_custom_variables() returns all variables as a disctionary of strings.

Setup Info
    Tags
      Help me…