Scenario Status (warnings)

s-cordo

Hi,

I'm facing trouble with scenarios status.

1/ I dont understand why when some steps inside a steps end with a warning status, the step is considered as 'OK' and also the scenario.

Here is a quick example below :

In this example, i was expected the scenario status to be 'WARNING'

2/ Moreover, is it possible to get access to previous steps status inside a custom python code ?
Example below :

There, inside the custom python code step 'step3', I would like to get the output status of the previous steps : 'step1' and 'step2'. Is that possible ?

3/ Lastly, is that possible to 'force' the output of a custom python code step ? Like something below :

Thanks a lot for your help

MarcH

Hello,

1. As you noticed, unless the scenario encounters an error or is aborted, its outcome will always be "Success", even if one of the steps ended with a "Warning". It does make sense that a step ending in a "Warning" could impact the overall scenario run status, so we'll take a look at improving that.

2. The "get_previous_steps_outputs()" method only contains data for a few of the steps, for example the "Execute SQL" step, which is why the result can appear empty if you only use Python steps for example. However, there is still a way to get details about the previously run steps. You can use the Dataiku API client from within the Python code to get more info about the currently running scenario. You would need to do something like the following in your Python step (replacing the project key and scenario id):

import dataiku

api_client = dataiku.api_client()

project = api_client.get_project('your project key')
scenario = project.get_scenario('your scenario id')
current_run = scenario.get_current_run()
steps = current_run.get_details().steps

"steps" will contain all the steps up to and including the current step. Each object in "steps" is actually a pretty large object as each step contains info about the entire scenario, but you will probably be most interested in the "step" field within the step object as it contains the step's name, and the "result" field as it contains the outcome of that step. One thing to remember is that the current step will be included in these results, but with no "result" field.

3. Besides manually raising an Exception in your Python code (which will cause the step to end in a "Failed" state), there's no way to more finely control the outcome, like setting the outcome to "Warning" in your example. We can also look into improving this to give users some more control of their scenario steps.