Is there a list of all the builtin variables like ${projectKey}, ${dssUserLogin}, etc
I noticed the user of some builtin variables in the documentation like:
- projectKey jobProjectKey
- dssUserLogin
- jobProjectKey
- jobId
- activityId
but it's there a list somewhere with all the possibilities, and where they can be used?
For example is not clear to me if I can use ${jobProjectKey} in the Administration > Settings > Compute & Scaling > Conteinerized execution > Containerized execution configs > Kubernetes namespace. it just say that "Variable expansion is supported" but what variables can be used there exactly?
I know I can use dsUserLogin since it's mention explicitly in https://doc.dataiku.com/dss/latest/containers/namespaces.html but what other variables are available?
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker
Hi @ecerulm
,
There isn't a single centralized list of which built-in variables will depend on the context ( job, scenario, webapp, notebook, etc.)
You can see variables available in specific recipes/scenarios e.g:Priority of variables is explained in more detail here when variables expansion is supported: https://doc.dataiku.com/dss/latest/connecting/sql/snowflake.html#how-to-set-it-up
Recipe > Scenario > Project > User > Global ( instance)You can always print all other variables available. You can use the datiaku API to see the difference from a notebook/recipe/scenario.
# get client import dataiku client = dataiku.api_client() #When running a job print available variables( excludes recipe variables) print(dataiku.get_custom_variables()) # print instance global variables global_variables = client.get_global_variables() print(global_variables) #project variables project = client.get_project("PROJECT_KEY") #there are standard + local project variables project_variables = project.get_variables() print(project_variables) #user properties user = client.get_user("user_id") print(user.get_settings().get_raw()['userProperties'])
For a scenario, you can use the following to print all variables:
from dataiku.scenario import Scenario import json print(json.dumps(Scenario().get_all_variables(), indent=2))
For the case of the namespace, while all variables are expanded the built-in variables available will depend on the context.You should opt for variables available in all contexts, including notebooks and recipes. Otherwise, the containerized execution could not be both recipes and notebooks. The built-in variables jobProjectKey / jobId are NOT available in notebooks for example
The built-in variables you can use safely are projectKey and dssUserLogin. You can also use global variables user properties or project variables.
Thanks, -
so running your script in a notebook it only shows the following variables
- dku.install.dir
- projectKey
- dip.home
I don't see dssUserLogin there.
So I would really like to see a list of contexts and the variables available in each of those contexts. LIke
Notebooks
- ${dku.install.dir}
- ${dip.home}
- ${projectKey}
- dssUserLogin not available here ??
- project variables, user variables and global variables
Jobs
- projectKey (??)
- dssUserLogin
- jobProjectKey (how is this different from projectKey?)
- jobId
- activityId ??
- project variables, user variables and global variables
- other ??
the other contexts
I think having such reference would be a good addition to the dataiku documentation. I'm not particularly keen on having to run code to find out what variables exists in each context, I think is way better to have them documented.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,043 Neuron
@info-rchitect
This could be a very clunky way to figure out the context where the code is running as it seems there are different variables exposed at each context.
https://community.dataiku.com/t5/Using-Dataiku/Understanding-if-Python-code-is-running-in-a-recipe-scenario-web/m-p/37493#M13842 -
@Turribeach
I will code something up and share it with the community if it works out. -
I agree with this, having the variables documented is preferable to having to run code to see what variables are available.