for monitoring is possible show all info project/scenario/trigger on python

AGdB

code python :

# this_project = this_client.get_default_project()
#print("projectKey, Nome_Scenario, Active, Type-Trigger , Name-trigger, Delay")
import pandas as pd
import json

for this_project in dss_projects:
curr_project = this_client.get_project(this_project)
for scenario in curr_project.list_scenarios(as_type="objects"):
curr_settings = scenario.get_settings()
if curr_settings.active:
print (curr_settings.get_raw())

ex. output :

{'projectKey': 'AAAAAAAA', 'id': 'unloadnotebooks', 'type': 'step_based', 'name': 'unloadnotebooks', 'active': true, 'versionTag': {'versionNumber': 2, 'lastModifiedBy': {'login': 'admin'}, 'lastModifiedOn': 1608300844622}, 'checklists': {'checklists': ' '}, 'delayedTriggersBehavior': {'delayWhileRunning': true, 'squashDelayedTriggers': true, 'suppressTriggersWhileRunning': true}, 'tags': , 'triggers': [{'id': 'IFACIlOZ', 'type': 'temporal', 'name': 'UNLOADNOTEBOOKS', 'delay': 5, 'active': true, 'params': {'frequency': 'Daily', 'daysOfWeek': ['Saturday'], 'dayOfMonth': 1, 'minute': 0, 'hour': 2, 'count': 5}}], 'reporters': , 'params': {'steps': [{'id': 'runmacro_pyrunnable_builtin-macros_kill-jupyter-sessions', 'type': 'runnable', 'name': 'Step #1', 'runConditionType': 'RUN_IF_STATUS_MATCH', 'runConditionStatuses': ['SUCCESS', 'WARNING'], 'runConditionExpression': '', 'resetScenarioStatus': false, 'delayBetweenRetries': 10, 'maxRetriesOnFail': 0, 'params': {'runnableType': 'pyrunnable_builtin-macros_kill-jupyter-sessions', 'config': {'maxIdleTimeHours': 8, 'maxSessionAgeHours': 24, 'dontKillBusyKernels': true, 'dontKillConnectedKernels': true, 'simulate': false}, 'adminConfig': {}, 'proceedOnFailure': false}}]}, 'automationLocal': true, 'customFields': {}}

is possible covert from json dataiku to file csv ?

Please Help Me !

Sarina

Hi @AGdB
,

I think it might work well for you to use the following method, writing all scenario setting objects to a folder, and then creating a dataset out of the folder. Here's an example:

I create a folder to hold all of my scenario settings output:

This is my modified version of your Python code to now write to the Folder I just created:

folder = dataiku.Folder('scenario_jsons')
import dataiku 

client = dataiku.api_client()
for this_project in client.list_project_keys():
    project = client.get_project(this_project)
    scenarios = project.list_scenarios(as_type='objects')
    for curr_scenario in scenarios:
        settings = curr_scenario.get_settings()
        if settings.active:
            # write file in PROJECT_SCENARIO format
            filename = this_project + '_' + curr_scenario.id
            folder.write_json(filename, settings.get_raw())

Now my folder contains a bunch of json files:

I can select "Create dataset" from this folder to now create a single dataset from my JSON files:

And a snippet of my final dataset: 

In addition to this method, I wanted to mention that there is an internal stats dataset for scenarios that will contain scenario run information, just in case it is sufficient for your use case. You can create and view this by going to + Dataset > Internal > Internal Stats > select "Scenario runs" as your "type" and leave the project field blank for all projects. Here's an example of what it looks like:

 

In addition to both of the above methods, you could also parse the scenario settings dictionary in Python and create a Pandas dataframe for the specific columns you care about, and then write that dataframe as your dataset. The folder approach simply allows you to skip the parsing yourself. I can paste an example of this if you would prefer that method though.

I hope that's helpful.

Thanks,
Sarina 

 

AGdB

Thx perfect