If condition satisfied scenarios

pmunoz

Hi!!

I am trying to add certain conditions in the executions of scenarios, but it seems that I am making some mistakes in the format or how I should do it correctly.

If I may detail, the main idea is.

1. Execution Step1.
2. (parallel execution of step 2 and step3).
   1. Execute Step2 if Step1 finishes OK.
   2. Execution Step3 if Step1 finishes OK.
3. Execute Step4 only if Step2 and Step3 finish OK.

These conditions should be placed in the execution on each step (2,3 and 4) like that

So the questions are,

1. What should be in the "if" section in order to recover the OK or Success value of a previous step?
2. If the step 2 and step 3, are at the same time execution of other scenarios (from the same project), how should be written the "IF" condition as recovering scenarioX.outcome?

Thanks!!!

Ignacio_Toledo

Hi @pmunoz
. The documentation on how to fill the 'if condition satisfied' (your "a" questions) can be found at:

https://doc.dataiku.com/dss/latest/scenarios/step_flow_control.html#if-condition-satisfied

As for "b", well, I wouldn't know. From the variables available in the previous documentation, apparently there are no variables associated to the outcome of another scenario, but maybe some dataiker can get back to you with that. Also, I'm no sure if it is possible to run two Steps in parallel.

Cheers.

Liev

Hi @pmunoz

Another way of tackling this is by putting steps 2 and 3 in a separate scenario.

In this case, in your first scenario your second step would be to run this secondary scenario, and upon successful completion, run step 4.

Hope this helps!

tim-wright

@pmunoz
First off thanks for the good question. I'll admit that I also have not done this before so did a little testing on my own. I stumbled upon this link:

https://doc.dataiku.com/dss/latest/scenarios/step_flow_control.html

It appears that in the conditional you put:

stepOutcome_<insert buildstep name (1)> == 'SUCCESS' (see my example below)

where you replace the portion in between <> with the name of your step. It appeared to work for me in my brief testing. Note: It is not very clear how to handle whitespace in your step name, but I *think* you can replace it with an underscore character.

b) I think you could create the same conditional for two separate steps to force them to be in "parallel". I suspect that you can add a final (4th) step to run if no other steps have failed OR to explicitly run when 2 and 3 were successful.

Let us know how it turns out

sonal_18

Hi tim,

This is very helpful, can we also get the error message once the condition is failed?

Actually im creating scenarios for creating logs of the job build and in the table i want to insert the error message as well. Is there a way to do it?

elnurmdov

@tim-wright
hello. I have a question regarding this case. I want my step to run if the previous custom python step returns or prints certain message. How can I do it? I did not get the difference between stepResult_stepName and stepOutput_stepName

Turribeach

As the documentation says there are 3 three different kinds of outputs for each scenario step:

stepOutcome: This is handled by Dataiku for you. Whatever the step does it will have a status, possible values are ‘SUCCESS’, ‘WARNING’, ‘FAILED’, ‘ABORTED’

stepOutput: Some steps can produce output data. For instance you can have a step run a macro that lists datasets. The stepOutput will contain the output of the macro.

stepResult: Is a JSON structure with basic information about the step and the outcome. See sample below:

{"start":1693233602210,"end":1693233603166,"type":"STEP_DONE","outcome":"SUCCESS","target":{"projectKey":"CT_TEST","stepId":"runmacro_pyrunnable_builtin-macros_list-datasets-using-connection","type":"SCENARIO_STEP","scenarioId":"TESTSCENARIO"}}

The easiest way to do your requirement is to have a Custom Python step define a scenario variable:

from dataiku.scenario import Scenario
Scenario().set_scenario_variables(VariableName="some_value")

And then have a step that runs conditionally (if condition satisfied):

VariableName == "some_value"