Python: Getting global variables from inside and outside DSS

JohnB
JohnB Registered Posts: 32 ✭✭✭✭✭

When I am running a script inside DSS, I can do this:

from dataiku.scenario import Scenario
s = Scenario()
vars = s.get_all_variables()
print(vars)

When I am using an external client, I can get variables like this:

import dataikuapi
client = dataikuapi.DSSClient("myhost", "mykey")
vars = client.get_variables()
priint(vars)

My questions are:

  • is there a more common way to achieve this whether inside or outside?
  • why do I need to be an admin user to run "client.get_variables()" when outside DSS when this isn't required in a recipe?

Answers

  • AlexH
    AlexH Dataiker, PartnerAdmin Posts: 1 Dataiker

    Hi John,

    You can use the dataiku internal python api to get access to project variables without starting a DSS client.

    Something like this should help you

    import dataiku

    proj_var = dataiku.get_custom_variables()

  • JohnB
    JohnB Registered Posts: 32 ✭✭✭✭✭

    Hi Alex,

    Thanks for the response.

    My requirement is to access global and project variables. For testing with an external client I won't necessarily have a project context so my choice seems to be between "get_all_variables()" internally and "get_variables()" externally. For the latter, it seems I need admin privileges; this is not a problem long-term but it adds a bit of inconsistency in testing code developed on a PC.

    John

  • JohnB
    JohnB Registered Posts: 32 ✭✭✭✭✭
    edited July 17

    So, I have opted to use a custom scenario to get both global and project variables which works well enough in scenarios for non-admins but now I have discovered it doesn't work in notebooks. Is there any generic method that can view global and project variables as a non-admin that works in both scenarios and notebooks?

    Error msg:

    from dataiku.scenario import Scenario
    print (Scenario().get_all_variables())

    The API ticket used is not associated with a scenario run

  • JohnB
    JohnB Registered Posts: 32 ✭✭✭✭✭

    Sorry. Have I misunderstood get_custom_variables()?

    I thought this only returned project variables.

  • Liev
    Liev Dataiker Alumni Posts: 176 ✭✭✭✭✭✭✭✭
    edited July 17

    Hi @JohnB

    you could do this

    # get client
    import dataiku
    client = dataiku.api_client() 
    
    # this gets you all variables
    variables = client.get_variables()
    
    # this gets you project variables
    project = client.get_project(projectKey)
    project_variables = project.get_variables()

    Here the client is the same public client you use but when called within DSS, so the equivalent of

    import dataikuapi
    client = dataikuapi.DSSClient("myhost", "mykey")

    Regarding your question about needing to be an admin in order to retrieve the instance variables, indeed this is stated in the API HTTP API and in the Python version of the public API

  • Liev
    Liev Dataiker Alumni Posts: 176 ✭✭✭✭✭✭✭✭
    edited July 17

    Hi @JohnB
    if you need to access the variables, you can do so indeed from within and outside DSS. The only difference is how to initialize the client.

    Within DSS you don't need to use the syntax that uses apiKey, as you're already authenticated.

    import dataiku
    client = dataiku.api_client()
    
    instance_vars = client.get_variables()
    project_vars = client.get_project(projectKey).get_variables()

    Now, regarding permissions for the instance variables, indeed you need to be admin as outlined in the Python API and this is the case both from within and outside DSS.

  • JohnB
    JohnB Registered Posts: 32 ✭✭✭✭✭

    Hi Liev,

    Thanks for the link to the dssclient.py, that helps a lot.

    From some tests yesterday it seems that dataiku.get_custom_variables() gets both project and global variables without the need for admin privileges. Can you confirm that?

    The only issue there is that get_custom_variables() requires a project key, which for an external client may not be the case.

  • Liev
    Liev Dataiker Alumni Posts: 176 ✭✭✭✭✭✭✭✭
    edited July 17

    yes indeed, using

    dataiku.get_custom_variables()

    will retrieve a combined dictionary. The disadvantage is that this is the internal API and therefore it will not be accessible from outside your instance.

Setup Info
    Tags
      Help me…