API call for Scenarios

Options
ArvinUbhi
ArvinUbhi Dataiku DSS Core Designer, Registered Posts: 16 ✭✭✭✭
edited July 16 in Using Dataiku

I am trying to gather infomation on scenarios within projects within dss.

Effectively, I want t be able to see what projects have what scenarios and what code envs those scenarios are using.

The code below shows how I am currently gathering scenario infomation, however, this doesnt return infomation regarding the code envs that the scenario is using.

import dataiku
from dataiku import pandasutils as pdu
import pandas as pd

client = dataiku.api_client()

dss_projects = client.list_project_keys()
#iterate over the keys
for pk in dss_projects:
    project = client.get_project(pk)
    for scenario in project.list_scenarios(as_type="objects"):
        print(vars(scenario.get_settings()))

Answers

  • Marlan
    Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 317 Neuron
    edited July 17
    Options

    Hi @ArvinUbhi
    ,

    I don't know if the following will return any additional information than vars() but worth checking I'd think:

    settings = scenario.get_settings()
    scenario_info = settings.get_raw()

    If you are checking step-based scenarios, step info is in scenario_info['params']['steps']. I haven't checked myself but possible that the code environment is included in the step info.

    Marlan

  • ArvinUbhi
    ArvinUbhi Dataiku DSS Core Designer, Registered Posts: 16 ✭✭✭✭
    Options

    I have changed it to get_raw() however aI am recieving the same thing as vars. I still cannot retrieve the code env being used in the python scenario (not step based).

  • Marlan
    Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 317 Neuron
    Options

    Hi @ArvinUbhi
    ,

    Unfortunately, I don't think the code environment is available via the API. I checked the code for the scenario API on Dataiku's github site and it doesn't look like the code environment is returned anywhere.

    You could add a request that this be added to the API in the Product Ideas section of this site but that of course doesn't help with the current need.

    Sorry I wasn't more help.

    Marlan

  • AlexandreL
    AlexandreL Dataiker Posts: 36 Dataiker
    edited July 17
    Options

    Hi,

    Indeed you can use the python API to do so. Keep in mind that code environments can be used in various parts of a scenario independently: a python step (if the scenario is step-based), the script code env (if the scenario is a python script), and both scenario types can also have a custom trigger, coded in python.

    This way, you will need to parse environments from different sections of the settings. The following code will let you retrieve all environments used in all of a project's scenarios. You might additionally need to get the project's default environment if those scenarios use it:

    scenario_envs = {}
    
    for scenario in project.list_scenarios():
        scenario_id = scenario["id"]
        scenario_envs[scenario_id] = []
        scenario_handle = project.get_scenario(scenario_id)
        settings = scenario_handle.get_settings()
        raw_settings = settings.get_raw()
        
        if raw_settings.get("type") == "step_based":
            trigger_envs = [trigger.get("params").get("envSelection") for trigger in settings.raw_triggers if trigger.get("type") == "custom_python"]
            step_envs = [step.get("params").get("envSelection") for step in settings.raw_steps if step.get("type") == "custom_python"]
            scenario_envs[scenario_id] = trigger_envs + step_envs
        elif raw_settings.get("type") == "custom_python":
            scenario_env = [settings.get_raw().get("params").get("envSelection")]
            trigger_envs = [trigger.get("params").get("envSelection") for trigger in settings.raw_triggers if trigger.get("type") == "custom_python"]        
            scenario_envs[scenario_id] =  trigger_envs + scenario_env
            
            
Setup Info
    Tags
      Help me…