Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Scenario run time logging

Pkhande11
Level 2
Scenario run time logging

Hello Dataiku Community,

I would like to know how to extract scenario completion run time using custom python code or API.

I can see the runtime from Run logs. Is there a better way to extract those runtime for multiple scenarios?

Thanks!!

0 Kudos
4 Replies
AlexT
Dataiker
Dataiker

Hi @Pkhande11,

You can see scenario runtime for all scenario accorss all projects by 

adding an Internal dataset ( + Dataset - Internal Stats - Scenario Runs - Preview( make sure to click preview before saving))

Save to a dataset and use a prepare recipe to compute the duration based on start/end times.  e.g Formula 

diff(time_end,time_start, 'seconds')

 

Screenshot 2022-04-28 at 13.07.37.pngScreenshot 2022-04-28 at 13.10.44.png

Let me know if this works in your case. 

Pkhande11
Level 2
Author

Hi @AlexT ,

Thanks for sharing the solution. 

My question is more around the runtime shared under "Last runs" tab as highlighted in the image below.

Once the scenario is completed, we could see the List of Run logs (Success Run) and the total runtime for the scenario. 

dataiku_scenario_last_runs.PNG

 

runtime_logs.PNG

Can we extract those time duration?  (highlighted ones)

Thanks!!

0 Kudos
Ignacio_Toledo

Hi @Pkhande11,

I think the times you get with @AlexT instructions are the ones highlighted in your screenshots, is just that they are in seconds, and not formatted as a human-readable time range.

For example, given this scenario log, with the highlighted scenario id:

scenario02.png

You'll find that the time matches the ones you can find using @AlexT instructions:

scenario01.png

I only added a new column with a formula, so the seconds were transformed to a human-readable format.

Hope this helps!

PS: By the way, the formula I used for the format, assuming the column with the time in seconds is called scenario_duration, is:

'%02d hour(s), %02d minute(s), %02d seconds(s)'.format(scenario_duration//3600, scenario_duration % 3600 // 60, scenario_duration % 3600 % 60)

 

Pkhande11
Level 2
Author

Thanks @Ignacio_Toledo will check it out.

0 Kudos