Python: Getting global variables from inside and outside DSS
When I am running a script inside DSS, I can do this:
from dataiku.scenario import Scenario
s = Scenario()
vars = s.get_all_variables()
print(vars)
When I am using an external client, I can get variables like this:
import dataikuapi
client = dataikuapi.DSSClient("myhost", "mykey")
vars = client.get_variables()
priint(vars)
My questions are:
- is there a more common way to achieve this whether inside or outside?
- why do I need to be an admin user to run "client.get_variables()" when outside DSS when this isn't required in a recipe?
Answers
-
Hi John,
You can use the dataiku internal python api to get access to project variables without starting a DSS client.
Something like this should help you
import dataiku
proj_var = dataiku.get_custom_variables()
-
Hi Alex,
Thanks for the response.
My requirement is to access global and project variables. For testing with an external client I won't necessarily have a project context so my choice seems to be between "get_all_variables()" internally and "get_variables()" externally. For the latter, it seems I need admin privileges; this is not a problem long-term but it adds a bit of inconsistency in testing code developed on a PC.
John
-
So, I have opted to use a custom scenario to get both global and project variables which works well enough in scenarios for non-admins but now I have discovered it doesn't work in notebooks. Is there any generic method that can view global and project variables as a non-admin that works in both scenarios and notebooks?
Error msg:
from dataiku.scenario import Scenario
print (Scenario().get_all_variables())The API ticket used is not associated with a scenario run
-
Sorry. Have I misunderstood get_custom_variables()?
I thought this only returned project variables.
-
Hi @JohnB
you could do this
# get client import dataiku client = dataiku.api_client() # this gets you all variables variables = client.get_variables() # this gets you project variables project = client.get_project(projectKey) project_variables = project.get_variables()
Here the client is the same public client you use but when called within DSS, so the equivalent of
import dataikuapi client = dataikuapi.DSSClient("myhost", "mykey")Regarding your question about needing to be an admin in order to retrieve the instance variables, indeed this is stated in the API HTTP API and in the Python version of the public API
-
Hi @JohnB
if you need to access the variables, you can do so indeed from within and outside DSS. The only difference is how to initialize the client.Within DSS you don't need to use the syntax that uses apiKey, as you're already authenticated.
import dataiku client = dataiku.api_client() instance_vars = client.get_variables() project_vars = client.get_project(projectKey).get_variables()
Now, regarding permissions for the instance variables, indeed you need to be admin as outlined in the Python API and this is the case both from within and outside DSS.
-
Hi Liev,
Thanks for the link to the dssclient.py, that helps a lot.
From some tests yesterday it seems that dataiku.get_custom_variables() gets both project and global variables without the need for admin privileges. Can you confirm that?
The only issue there is that get_custom_variables() requires a project key, which for an external client may not be the case.
-
yes indeed, using
dataiku.get_custom_variables()
will retrieve a combined dictionary. The disadvantage is that this is the internal API and therefore it will not be accessible from outside your instance.
