Self-abort scenario
I would like to create a step in a scenario that will abort the scenario itself (not to abort another scenario) when a certain condition is met
My thought is to use the Custom Python and use the following code:
```import dataiku
project = dataiku.api_client().get_project(dataiku.default_project_key())
scenario = project.get_scenario("scenario_name")
scenario.abort()```
The step is then will be set as 'If condition satisfied'
Please help to advise if this would work fine and not cause any issue
If there is a better alternative, I am open to new suggestions
Apparently, the 'Kill scenario' step is used to abort ANOTHER scenario but not design to self-kill.
Operating system used: Windows 10
Best Answer
-
the kill scenario isn't really a recommended step, and it's better to invert the flow of control, that is, to have a scenario start another scenario run and control it. But you can use the Kill scenario step, or the python method (and indeed it's the same code, sorry about that). Both approaches are equivalent.
The DB corruption of the internal databases should be addressed by moving them to Postgres, which is much more resilient than the default H2 (see https://doc.dataiku.com/dss/latest/operations/runtime-databases.html#externally-hosting-runtime-databases )
Answers
-
Hi,
you can use a "kill a scenario" step applied to the scenario itself, or make a simple python step with just 'raise Exception("...")'.
If you need an abort, you'll have to use the public API, not the internal API, like:
import dataiku c = dataiku.api_client() p = c.get_default_project() s = p.get_scenario("S") s.abort() -
Hi @fchataigner2
I am aware of the Kill scenario.
We are currently using Kill Scenario and it was working for few months but suddenly we faced a DB corruption issue (Missing, locked, unreachable or corrupted internal database).
Our team has reached out to Dataiku vendor and they advised against using Kill scenario.
So should we not use Kill scenario for self-abort?
Dataiku suggest to use raise exception which will lead to failure but I dont want to fail scenario. I want to abort the scenario.I have shared below in my original post. Is this not the same with what you just shared?
import dataiku project = dataiku.api_client().get_project(dataiku.default_project_key()) scenario = project.get_scenario("scenario_name") scenario.abort()Will above not caused any DB corruption like the Kill Scenario step?
-
Since you mention both approaches are equivalent, so if I just replace the kill scenario step with the python code, do you foresee that this corrupted DB issue can still occur?
-
the main cause of corruption are DSS crashes while operations are underway on the internal DBs. Killing a scenario is one of many such operations, so unless you use it very intensively the odds of causing corruption should be low, or at least it shouldn't be a major contributor to these odds.
Again, migrating the internal DBs to Postgres is the best answer you can bring to internal DB corruption.
-
thanks @fchataigner2
for the help!