Env variable cannot be set

rump
rump Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 7
edited July 16 in Using Dataiku

I'm trying to load a transformer model through the env Resources but it does not work as it does not set the variables

I tried the following as a test:

*********************************************************
{'value': 'This is a test', 'type': 'RAW_STRING'}
********************************************************* 

In the log I can see the variable test is kind of created and accessed.

import os
from dataiku.code_env_resources import clear_all_env_vars
from dataiku.code_env_resources import set_env_var
from dataiku.code_env_resources import get_env_var

# Clears all environment variables defined by previously run script
clear_all_env_vars()

# Sets a test variable in the environment
set_env_var('test', 'This is a test')

# Prints the variable to test the access
print(get_env_var('test'))

But if I try to access the variable in my notebook (with the correct environment selected), it's like it doesn't exist:

import os

os.environ['test']
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_3089981/4148336718.py in <cell line: 0>()
      1 import os
      2 
----> 3 os.environ['test']

/usr/lib64/python3.11/os.py in __getitem__(self, key)
    677         except KeyError:
    678             # raise KeyError with the original key value
--> 679             raise KeyError(key) from None
    680         return self.decodevalue(value)
    681 

KeyError: 'test'

Does anyone know what I am doing wrong?

Thanks in advance

Best Answer

  • rump
    rump Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 7
    Answer ✓

    Finally it was related to the jupyter notebook suppport in the tab of Packages to install of the environment configuration. The thing is that if this box is ticked, after the resources environment init script finishes, the environment set the variables through. It can be seen in the log "updateEnvAccordingToSpec.log" (Log tab) the following confirmation:

    Adding resources environment variables to Jupyter kernel.json ...

    If the box jupyter notebook support was not ticked, this message wouldn't appear and then, the variables weren't set.

Answers

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker

    Hi @rump
    ,
    Have you tried to "force reload" from the top right of the notebook after updating the code environment?
    If you have and still don't see the variables, please share the exact DSS version you are using.

    Screenshot 2024-05-08 at 3.01.54 PM.png

Setup Info
    Tags
      Help me…