Community Conundrum 25:Feature Visualization is now live! Read More

Python: Getting global variables from inside and outside DSS

Level 2
Python: Getting global variables from inside and outside DSS

When I am running a script inside DSS, I can do this:

from dataiku.scenario import Scenario
s = Scenario()
vars = s.get_all_variables()
print(vars)

When I am using an external client, I can get variables like this:

import dataikuapi
client = dataikuapi.DSSClient("myhost", "mykey")
vars = client.get_variables()
priint(vars)

My questions are:

  • is there a more common way to achieve this whether inside or outside?
  • why do I need to be an admin user to run "client.get_variables()" when outside DSS when this isn't required in a recipe?
0 Kudos
8 Replies
Dataiker
Dataiker

Hi John,

 

You can use the dataiku internal python api to get access to project variables without starting a DSS client. 

Something like this should help you 

import dataiku

proj_var = dataiku.get_custom_variables()

0 Kudos
Level 2
Author

Hi Alex,

Thanks for the response.

My requirement is to access global and project variables. For testing with an external client I won't necessarily have a project context so my choice seems to be between "get_all_variables()" internally and "get_variables()" externally. For the latter, it seems I need admin privileges; this is not a problem long-term but it adds a bit of inconsistency in testing code developed on a PC.

John

0 Kudos
Level 2
Author

So, I have opted to use a custom scenario to get both global and project variables which works well enough in scenarios for non-admins but now I have discovered it doesn't work in notebooks. Is there any generic method that can view global and project variables as a non-admin that works in both scenarios and notebooks?

Error msg:

from dataiku.scenario import Scenario
print (Scenario().get_all_variables())

The API ticket used is not associated with a scenario run

 

0 Kudos
Level 2
Author

Sorry. Have I misunderstood get_custom_variables()?

I thought this only returned project variables.

0 Kudos
Dataiker
Dataiker

Hi @JohnB 

you could do this 

# get client
import dataiku
client = dataiku.api_client() 

# this gets you all variables
variables = client.get_variables()

# this gets you project variables
project = client.get_project(projectKey)
project_variables = project.get_variables()

 

Here the client is the same public client you use but when called within DSS, so the equivalent of 

import dataikuapi
client = dataikuapi.DSSClient("myhost", "mykey")

 

Regarding your question about needing to be an admin in order to retrieve the instance variables, indeed this is stated in the API  HTTP API  and in the Python version of the public API 

 

0 Kudos
Dataiker
Dataiker

Hi @JohnB if you need to access the variables, you can do so indeed from within and outside DSS. The only difference is how to initialize the client.

Within DSS you don't need to use the syntax that uses apiKey, as you're already authenticated.

import dataiku
client = dataiku.api_client()

instance_vars = client.get_variables()
project_vars = client.get_project(projectKey).get_variables()

Now, regarding permissions for the instance variables, indeed you need to be admin as outlined in the Python API and this is the case both from within and outside DSS.

0 Kudos
Level 2
Author

Hi Liev,

Thanks for the link to the dssclient.py, that helps a lot.

From some tests yesterday it seems that dataiku.get_custom_variables() gets both project and global variables without the need for admin privileges. Can you confirm that?

The only issue there is that get_custom_variables() requires a project key, which for an external client may not be the case.

0 Kudos
Dataiker
Dataiker

yes indeed, using

dataiku.get_custom_variables()

 will retrieve a combined dictionary. The disadvantage is that this is the internal API and therefore it will not be accessible from outside your instance.

 

0 Kudos