API call for Scenarios

ArvinUbhi

I am trying to gather infomation on scenarios within projects within dss.

Effectively, I want t be able to see what projects have what scenarios and what code envs those scenarios are using.

The code below shows how I am currently gathering scenario infomation, however, this doesnt return infomation regarding the code envs that the scenario is using.

import dataiku
from dataiku import pandasutils as pdu
import pandas as pd

client = dataiku.api_client()

dss_projects = client.list_project_keys()
#iterate over the keys
for pk in dss_projects:
    project = client.get_project(pk)
    for scenario in project.list_scenarios(as_type="objects"):
        print(vars(scenario.get_settings()))

Marlan

Hi @ArvinUbhi
,

I don't know if the following will return any additional information than vars() but worth checking I'd think:

settings = scenario.get_settings()
scenario_info = settings.get_raw()

If you are checking step-based scenarios, step info is in scenario_info['params']['steps']. I haven't checked myself but possible that the code environment is included in the step info.

Marlan

ArvinUbhi

I have changed it to get_raw() however aI am recieving the same thing as vars. I still cannot retrieve the code env being used in the python scenario (not step based).

Marlan

Hi @ArvinUbhi
,

Unfortunately, I don't think the code environment is available via the API. I checked the code for the scenario API on Dataiku's github site and it doesn't look like the code environment is returned anywhere.

You could add a request that this be added to the API in the Product Ideas section of this site but that of course doesn't help with the current need.

Sorry I wasn't more help.

Marlan

AlexandreL

Hi,

Indeed you can use the python API to do so. Keep in mind that code environments can be used in various parts of a scenario independently: a python step (if the scenario is step-based), the script code env (if the scenario is a python script), and both scenario types can also have a custom trigger, coded in python.

This way, you will need to parse environments from different sections of the settings. The following code will let you retrieve all environments used in all of a project's scenarios. You might additionally need to get the project's default environment if those scenarios use it:

scenario_envs = {}

for scenario in project.list_scenarios():
    scenario_id = scenario["id"]
    scenario_envs[scenario_id] = []
    scenario_handle = project.get_scenario(scenario_id)
    settings = scenario_handle.get_settings()
    raw_settings = settings.get_raw()
    
    if raw_settings.get("type") == "step_based":
        trigger_envs = [trigger.get("params").get("envSelection") for trigger in settings.raw_triggers if trigger.get("type") == "custom_python"]
        step_envs = [step.get("params").get("envSelection") for step in settings.raw_steps if step.get("type") == "custom_python"]
        scenario_envs[scenario_id] = trigger_envs + step_envs
    elif raw_settings.get("type") == "custom_python":
        scenario_env = [settings.get_raw().get("params").get("envSelection")]
        trigger_envs = [trigger.get("params").get("envSelection") for trigger in settings.raw_triggers if trigger.get("type") == "custom_python"]        
        scenario_envs[scenario_id] =  trigger_envs + scenario_env