How to write and read from a Python recipe override variables?

info-rchitect
info-rchitect Registered Posts: 180 ✭✭✭✭✭✭

Hi,

I want to write/read these variables using Python DSS API:

dataiku_python_recipe_variables.png

thx much


Operating system used: Windows 10

Best Answer

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,000 Neuron
    edited July 17 Answer ✓

    My bad, I was on my phone and I didn't see the full image on my phone and I thought you were after project variables. Here you go. Given these recipe variables:

    {
       "var1": 1,
       "var2": "Some New Text",
       "var3": true
    }

    import dataiku
    client_handle = dataiku.api_client()
    project_handle = client_handle.get_project('PROJECT_KEY')
    recipe_handle = project_handle.get_recipe('recipe_name')
    recipe_settings = recipe_handle.get_settings()
    recipe_definition = recipe_settings.get_recipe_raw_definition()
    for variable in recipe_definition['variables']:
        print(variable + ' Value: ' + str(recipe_definition['variables'][variable]))

    Gives me:

    var1 Value: 1
    var2 Value: Some Text
    var3 Value: True

    Then changing it:

    recipe_handle = project_handle.get_recipe('compute_customers_out')
    recipe_settings = recipe_handle.get_settings()
    recipe_definition = recipe_settings.get_recipe_raw_definition()
    recipe_definition['variables']['var2'] = "Some New Text"
    recipe_settings.save()
    for variable in recipe_definition['variables']:
        print(variable + ' Value: ' + str(recipe_definition['variables'][variable]))

    Output:

    var1 Value: 1
    var2 Value: Some New Text
    var3 Value: True

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,000 Neuron
  • info-rchitect
    info-rchitect Registered Posts: 180 ✭✭✭✭✭✭

    Hi @Turribeach
    ,

    You cannot use `get_custom_variables`, did you mean something else in the link you sent?

    dataiku_python_recipe_variables_2.png

  • info-rchitect
    info-rchitect Registered Posts: 180 ✭✭✭✭✭✭
    edited July 17

    Very close, but on DSS 11 I get this error:

    [13:19:55] [INFO] [dku.utils]  -   File "/data/dataiku/data_dir/code-envs/python/pyoss/lib/python3.8/site-packages/dku/dataiku.py", line 104, in set_recipe_variables
    [13:19:55] [INFO] [dku.utils]  -     recipe_settings = recipe_handle.get_settings()
    [13:19:55] [INFO] [dku.utils]  -   File "/data/dataiku/dataiku-dss-11.3.2/python/dataikuapi/dss/recipe.py", line 194, in get_settings
    [13:19:55] [INFO] [dku.utils]  -     type = data["recipe"]["type"]
    [13:19:55] [INFO] [dku.utils]  - KeyError: 'recipe'
  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,000 Neuron
    edited July 17

    @info-rchitect
    wrote:

    Very close, but on DSS 11 I get this error:

    [13:19:55] [INFO] [dku.utils]  -   File "/data/dataiku/data_dir/code-envs/python/pyoss/lib/python3.8/site-packages/dku/dataiku.py", line 104, in set_recipe_variables
    [13:19:55] [INFO] [dku.utils]  -     recipe_settings = recipe_handle.get_settings()
    [13:19:55] [INFO] [dku.utils]  -   File "/data/dataiku/dataiku-dss-11.3.2/python/dataikuapi/dss/recipe.py", line 194, in get_settings
    [13:19:55] [INFO] [dku.utils]  -     type = data["recipe"]["type"]
    [13:19:55] [INFO] [dku.utils]  - KeyError: 'recipe'

    You forgot to change the recipe name on the second recipe_handle assignment. The first 3 lines on the save part are redundant if you are already reading the values. But I included them in case you just wanted to save a variable.

    Personally I think it's a bug since the Dataiku API is not returning any errors on "project_handle.get_recipe('unknown_recipe')". But if you run "recipe_handle.get_status()" you will get the expected exception:

    DataikuException: com.dataiku.dip.server.controllers.NotFoundException: recipe does not exist: TEST_PROJECT.dummy_recipe

Setup Info
    Tags
      Help me…