Use parameter set for dynamic select in a macro

Aymeric
Aymeric Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 15 Partner

Hi,

I am trying to implement a dynamic MULTISELECT (i.e. getChoicesFromPython:true) in a macro, using parameter set in order to provide credentials to contact an external service.

I understand presets are reachable inside the body of the runnable, but I would like a confirmation / infirmation that this is indeed impossible to get these fields in the body of the 'do' function.

Best regards,
Moaï

Best Answer

  • Aymeric
    Aymeric Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 15 Partner
    edited July 17 Answer ✓

    Thanks again for your time @ZachM
    !

    I've given it another try, starting from scratch, but got the same results... Before I found occurrences when it suddenly worked. After some toying with the plugin, I found the solution : don't insert trigger params exept If you are absolutely certain that they will be activated. Indeed, the following sections of my macro were:

            {
                "name": "selection_mode",
                "label": "Selection",
                "type": "SELECT",
                "selectChoices":[{"value":"all", "label":"all"},
                                 {"value":"by hand", "label":"by hand"},
                                 {"value":"regex", "label":"regex"}],
                "defaultValue":"by hand",
                "mandatory": true
            },
            {
                "name": "regex",
                "label": "REGEX",
                "type": "STRING",
                "defaultValue":".*",
                "mandatory": true,
                "visibilityCondition":"model.selection_mode=='regex'"
            },
            {
                "name": "plugins",
                "label": "Plugins",
                "type": "MULTISELECT",
                "getChoicesFromPython":true,
                "mandatory": true,
                "triggerParams":["gitlab_access", "dss_role"],
                "visibilityCondition":"model.selection_mode=='by hand'"
            }

    Therefore, having dss_role with a predefined value, and gitlab_access retrieving the first preset available made the whole mechanism inoperant.

    I ended up removing the trigger parameters, and it's all fine now !

    Thanks again ZachM !

Answers

  • Zach
    Zach Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 153 Dataiker
    edited July 17

    Hi @Moaï
    ,

    Yes, it is possible to access a parameter set from the "do()" function when using "getChoicesFromPython".

    Here's an example "do()" function that accesses the "test_preset" preset, which is defined in the runnable.json params:

    # Located in the resource/ directory
    def do(payload, config, plugin_config, inputs):
        username = config["test_preset"]["username"]
        password = config["test_preset"]["password"]
        
        choices = [
            { "value": username, "label": f"Username: {username}"},
            { "value": password, "label": f"Password: {password}"}
        ]
        return {"choices": choices}

    Thanks,

    Zach

  • Aymeric
    Aymeric Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 15 Partner

    Hi @ZachM
    ,
    thank you for your answer ! However, I'm affraid I can't reproduce your results : it works as expected in the Runnable, not in the "do" function. Here is what I tried :

    # PRESET :

    /* This file is the descriptor for the parameter set tokens */
    {
    "meta" : {
    // label: name of the parameter set as displayed, should be short
    "label": "Gitlab Access",

    // description: longer string to help end users understand what these parameter correspond to
    "description": "This parameter set hides the Gitlab token.",

    // icon: must be one of the FontAwesome 3.2.1 icons, complete list here at https://fontawesome.com/v3.2.1/icons/
    "icon": "icon-puzzle-piece"
    },

    "defaultDefinableInline": false,

    "defaultDefinableAtProjectLevel": false,


    "pluginParams": [],

    "params":[
    {
    "name": "gitlab_url",
    "label": "Gitlab url",
    "type": "STRING",
    "description": "URL to contact Gitlab",
    "mandatory": true
    },
    {
    "name": "gitlab_token",
    "label": "Gitlab token",
    "type": "PASSWORD",
    "description": "Token to interact with Gitlab",
    "mandatory": true
    }
    ]
    }

    # MACRO SETTINGS :

    /* This file is the descriptor for the python runnable update-plugin */
    {
    "meta": {
    // label: name of the runnable as displayed, should be short
    "label": "Create/Update plugin",

    // description: longer string to help end users understand what this runnable does
    "description": "This macro is intended to help updating plugins inside a scenario.",

    // icon: must be one of the FontAwesome 3.2.1 icons, complete list here at https://fontawesome.com/v3.2.1/icons/
    "icon": "icon-cogs"
    },

    /* whether the runnable's code is untrusted */
    "impersonate": false,
    "paramsPythonSetup": "get_choices_create_update_plugin.py",
    "params": [
    {
    "name": "gitlab_access",
    "label": "Gitlab access",
    "type": "PRESET",
    "parameterSetId": "gitlab-access",
    "mandatory": true
    },
    {
    "name": "dss_role",
    "label": "DSS role",
    "type": "SELECT",
    "selectChoices":[{"value":"admin", "label":"admin"},
    {"value":"users", "label":"users"}],
    "defaultValue":"admin",
    "mandatory": true
    },
    {
    "name": "selection_mode",
    "label": "Selection",
    "type": "SELECT",
    "selectChoices":[{"value":"all", "label":"all"},
    {"value":"by hand", "label":"by hand"},
    {"value":"regex", "label":"regex"}],
    "defaultValue":"by hand",
    "mandatory": true
    },
    ...


    # DO FUNCTION :

    import dataiku
    import gitlab
    import jstyleson

    def do(payload, config, plugin_config, inputs):
    choices = []
    try :
    choices.append({ "value" : config["gitlab_access"], "label" : str(config["gitlab_access"])})
    gitlab_url = config["gitlab_access"]["gitlab_url"]
    gitlab_token = config["gitlab_access"]["gitlab_token"]

    with gitlab.Gitlab(gitlab_url, gitlab_token) as gl:
    gl.ssl_verify = False
    git_projects = {}
    for p in gl.projects.list():
    print(p.web_url)
    if "dss-codebase/{}/plugins".format(config.get("dss_role")) in p.web_url:
    git_projects[p.name] = p.id

    for project_name in sorted(git_projects.keys()):
    choices.append({ "value" : git_projects[project_name], "label" : project_name})
    except Exception as e:
    choices.append({ "value" : str(e), "label" : str(e)})
    return {"choices": choices}

    The result is attached (key error exception explains the second choice). I am kind of clueless as of what I could have missed...

    Regards,
    Moaï

  • Zach
    Zach Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 153 Dataiker

    I tried using your code, but I'm not able to reproduce the issue. This is what the selection looks like for me (I removed the web_url if-statement, so it's adding every project):

    5AE549DF-CF4E-4BED-8DC3-251D05E1547D_1_201_a.jpeg

    In your screenshot, it appears to be loading the wrong parameter set. It's loading a parameter set with the keys "mode" and "name", but it's supposed to have the params "gitlab_url" and "gitlab_token".

    Do you have another parameter set that you're using by mistake? Or maybe you forgot to save the parameter-set.json after updating it? If this is an old version of the gitlab-access parameter set, you might need to tell DSS to reload it by reloading the plugin or restarting DSS.

Setup Info
    Tags
      Help me…