Use parameter set for dynamic select in a macro

[Moaï]

Hi,

I am trying to implement a dynamic MULTISELECT (i.e. getChoicesFromPython:true) in a macro, using parameter set in order to provide credentials to contact an external service.

I understand presets are reachable inside the body of the runnable, but I would like a confirmation / infirmation that this is indeed impossible to get these fields in the body of the 'do' function.

Best regards,
Moaï

[ZachM]

Hi @Moaï,

Yes, it is possible to access a parameter set from the "do()" function when using "getChoicesFromPython".

Here's an example "do()" function that accesses the "test_preset" preset, which is defined in the runnable.json params:

# Located in the resource/ directory
def do(payload, config, plugin_config, inputs):
    username = config["test_preset"]["username"]
    password = config["test_preset"]["password"]
    
    choices = [
        { "value": username, "label": f"Username: {username}"},
        { "value": password, "label": f"Password: {password}"}
    ]
    return {"choices": choices}

Thanks,

Zach

[Moaï]

Hi @ZachM ,
thank you for your answer ! However, I'm affraid I can't reproduce your results : it works as expected in the Runnable, not in the "do" function.  Here is what I tried :

# PRESET :

/* This file is the descriptor for the parameter set tokens */
{
"meta" : {
// label: name of the parameter set as displayed, should be short
"label": "Gitlab Access",

// description: longer string to help end users understand what these parameter correspond to
"description": "This parameter set hides the Gitlab token.",

// icon: must be one of the FontAwesome 3.2.1 icons, complete list here at https://fontawesome.com/v3.2.1/icons/
"icon": "icon-puzzle-piece"
},

"defaultDefinableInline": false,

"defaultDefinableAtProjectLevel": false,

"pluginParams": [],

"params":[
{
"name": "gitlab_url",
"label": "Gitlab url",
"type": "STRING",
"description": "URL to contact Gitlab",
"mandatory": true
},
{
"name": "gitlab_token",
"label": "Gitlab token",
"type": "PASSWORD",
"description": "Token to interact with Gitlab",
"mandatory": true
}
]
}

# MACRO SETTINGS :

/* This file is the descriptor for the python runnable update-plugin */
{
"meta": {
// label: name of the runnable as displayed, should be short
"label": "Create/Update plugin",

// description: longer string to help end users understand what this runnable does
"description": "This macro is intended to help updating plugins inside a scenario.",

// icon: must be one of the FontAwesome 3.2.1 icons, complete list here at https://fontawesome.com/v3.2.1/icons/
"icon": "icon-cogs"
},

/* whether the runnable's code is untrusted */
"impersonate": false,
"paramsPythonSetup": "get_choices_create_update_plugin.py",
"params": [
{
"name": "gitlab_access",
"label": "Gitlab access",
"type": "PRESET",
"parameterSetId": "gitlab-access",
"mandatory": true
},
{
"name": "dss_role",
"label": "DSS role",
"type": "SELECT",
"selectChoices":[{"value":"admin", "label":"admin"},
{"value":"users", "label":"users"}],
"defaultValue":"admin",
"mandatory": true
},
{
"name": "selection_mode",
"label": "Selection",
"type": "SELECT",
"selectChoices":[{"value":"all", "label":"all"},
{"value":"by hand", "label":"by hand"},
{"value":"regex", "label":"regex"}],
"defaultValue":"by hand",
"mandatory": true
},
...

# DO FUNCTION :

import dataiku
import gitlab
import jstyleson

def do(payload, config, plugin_config, inputs):
choices = []
try :
choices.append({ "value" : config["gitlab_access"], "label" : str(config["gitlab_access"])})
gitlab_url = config["gitlab_access"]["gitlab_url"]
gitlab_token = config["gitlab_access"]["gitlab_token"]

with gitlab.Gitlab(gitlab_url, gitlab_token) as gl:
gl.ssl_verify = False
git_projects = {}
for p in gl.projects.list():
print(p.web_url)
if "dss-codebase/{}/plugins".format(config.get("dss_role")) in p.web_url:
git_projects[p.name] = p.id

for project_name in sorted(git_projects.keys()):
choices.append({ "value" : git_projects[project_name], "label" : project_name})
except Exception as e:
choices.append({ "value" : str(e), "label" : str(e)})
return {"choices": choices}

The result is attached (key error exception explains the second choice). I am kind of clueless as of what I could have missed...

Regards,
Moaï

[ZachM]

I tried using your code, but I'm not able to reproduce the issue. This is what the selection looks like for me (I removed the web_url if-statement, so it's adding every project):

In your screenshot, it appears to be loading the wrong parameter set. It's loading a parameter set with the keys "mode" and "name", but it's supposed to have the params "gitlab_url" and "gitlab_token".

Do you have another parameter set that you're using by mistake? Or maybe you forgot to save the parameter-set.json after updating it? If this is an old version of the gitlab-access parameter set, you might need to tell DSS to reload it by reloading the plugin or restarting DSS.

[Moaï · Accepted solution]

Thanks again for your time @ZachM !

I've given it another try, starting from scratch, but got the same results... Before I found occurrences when it suddenly worked. After some toying with the plugin, I found the solution : don't insert trigger params exept If you are absolutely certain that they will be activated. Indeed, the following sections of my macro were:

        {
            "name": "selection_mode",
            "label": "Selection",
            "type": "SELECT",
            "selectChoices":[{"value":"all", "label":"all"},
                             {"value":"by hand", "label":"by hand"},
                             {"value":"regex", "label":"regex"}],
            "defaultValue":"by hand",
            "mandatory": true
        },
        {
            "name": "regex",
            "label": "REGEX",
            "type": "STRING",
            "defaultValue":".*",
            "mandatory": true,
            "visibilityCondition":"model.selection_mode=='regex'"
        },
        {
            "name": "plugins",
            "label": "Plugins",
            "type": "MULTISELECT",
            "getChoicesFromPython":true,
            "mandatory": true,
            "triggerParams":["gitlab_access", "dss_role"],
            "visibilityCondition":"model.selection_mode=='by hand'"
        }

 Therefore, having dss_role with a predefined value, and gitlab_access retrieving the first preset available made the whole mechanism inoperant.

I ended up removing the trigger parameters, and it's all fine now !

Thanks again ZachM !