Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Self-abort scenario

Solved!
farhanromli
Level 3
Self-abort scenario

I would like to create a step in a scenario that will abort the scenario itself (not to abort another scenario) when a certain condition is met

My thought is to use the Custom Python and use the following code:


```import dataiku

project = dataiku.api_client().get_project(dataiku.default_project_key())
scenario = project.get_scenario("scenario_name")
scenario.abort()```

The step is then will be set as 'If condition satisfied'

Please help to advise if this would work fine and not cause any issue
If there is a better alternative, I am open to new suggestions

Apparently, the 'Kill scenario' step is used to abort ANOTHER scenario but not design to self-kill.


Operating system used: Windows 10

0 Kudos
1 Solution
fchataigner2
Dataiker
Dataiker

the kill scenario isn't really a recommended step, and it's better to invert the flow of control, that is, to have a scenario start another scenario run and control it. But you can use the Kill scenario step, or the python method (and indeed it's the same code, sorry about that). Both approaches are equivalent.

The DB corruption of the internal databases should be addressed by moving them to Postgres, which is much more resilient than the default H2 (see https://doc.dataiku.com/dss/latest/operations/runtime-databases.html#externally-hosting-runtime-data... )

View solution in original post

0 Kudos
6 Replies
fchataigner2
Dataiker
Dataiker

Hi,

you can use a "kill a scenario" step applied to the scenario itself, or make a simple python step with just 'raise Exception("...")'.

If you need an abort, you'll have to use the public API, not the internal API, like:

import dataiku
c = dataiku.api_client()
p = c.get_default_project()
s = p.get_scenario("S")
s.abort()
0 Kudos
farhanromli
Level 3
Author

Hi @fchataigner2 

I am aware of the Kill scenario.
We are currently using Kill Scenario and it was working for few months but suddenly we faced a DB corruption issue (Missing, locked, unreachable or corrupted internal database).

Our team has reached out to Dataiku vendor and they advised against using Kill scenario. 
So should we not use Kill scenario for self-abort?

Dataiku suggest to use raise exception which will lead to failure but I dont want to fail scenario. I want to abort the scenario.

I have shared below in my original post. Is this not the same with what you just shared?

import dataiku

project = dataiku.api_client().get_project(dataiku.default_project_key())
scenario = project.get_scenario("scenario_name")
scenario.abort()

Will above not caused any DB corruption like the Kill Scenario step?

0 Kudos
fchataigner2
Dataiker
Dataiker

the kill scenario isn't really a recommended step, and it's better to invert the flow of control, that is, to have a scenario start another scenario run and control it. But you can use the Kill scenario step, or the python method (and indeed it's the same code, sorry about that). Both approaches are equivalent.

The DB corruption of the internal databases should be addressed by moving them to Postgres, which is much more resilient than the default H2 (see https://doc.dataiku.com/dss/latest/operations/runtime-databases.html#externally-hosting-runtime-data... )

0 Kudos
farhanromli
Level 3
Author

Since you mention both approaches are equivalent, so if I just replace the kill scenario step with the python code, do you foresee that this corrupted DB issue can still occur?


0 Kudos
fchataigner2
Dataiker
Dataiker

the main cause of corruption are DSS crashes while operations are underway on the internal DBs. Killing a scenario is one of many such operations, so unless you use it very intensively the odds of causing corruption should be low, or at least it shouldn't be a major contributor to these odds. 

Again, migrating the internal DBs to Postgres is the best answer you can bring to internal DB corruption.

farhanromli
Level 3
Author

thanks  @fchataigner2  for the help!

0 Kudos

Labels

?
Labels (1)

Setup info

?
A banner prompting to get Dataiku