Self-abort scenario

farhanromli
farhanromli Registered Posts: 25 ✭✭✭✭

I would like to create a step in a scenario that will abort the scenario itself (not to abort another scenario) when a certain condition is met

My thought is to use the Custom Python and use the following code:


```import dataiku

project = dataiku.api_client().get_project(dataiku.default_project_key())
scenario = project.get_scenario("scenario_name")
scenario.abort()```

The step is then will be set as 'If condition satisfied'

Please help to advise if this would work fine and not cause any issue
If there is a better alternative, I am open to new suggestions

Apparently, the 'Kill scenario' step is used to abort ANOTHER scenario but not design to self-kill.


Operating system used: Windows 10

Tagged:

Best Answer

Answers

  • fchataigner2
    fchataigner2 Dataiker Posts: 355 Dataiker
    edited July 17

    Hi,

    you can use a "kill a scenario" step applied to the scenario itself, or make a simple python step with just 'raise Exception("...")'.

    If you need an abort, you'll have to use the public API, not the internal API, like:

    import dataiku
    c = dataiku.api_client()
    p = c.get_default_project()
    s = p.get_scenario("S")
    s.abort()
  • farhanromli
    farhanromli Registered Posts: 25 ✭✭✭✭
    edited July 17

    Hi @fchataigner2


    I am aware of the Kill scenario.
    We are currently using Kill Scenario and it was working for few months but suddenly we faced a DB corruption issue (Missing, locked, unreachable or corrupted internal database).

    Our team has reached out to Dataiku vendor and they advised against using Kill scenario.
    So should we not use Kill scenario for self-abort?

    Dataiku suggest to use raise exception which will lead to failure but I dont want to fail scenario. I want to abort the scenario.

    I have shared below in my original post. Is this not the same with what you just shared?

    import dataiku
    
    project = dataiku.api_client().get_project(dataiku.default_project_key())
    scenario = project.get_scenario("scenario_name")
    scenario.abort()

    Will above not caused any DB corruption like the Kill Scenario step?

  • farhanromli
    farhanromli Registered Posts: 25 ✭✭✭✭

    Since you mention both approaches are equivalent, so if I just replace the kill scenario step with the python code, do you foresee that this corrupted DB issue can still occur?


  • fchataigner2
    fchataigner2 Dataiker Posts: 355 Dataiker

    the main cause of corruption are DSS crashes while operations are underway on the internal DBs. Killing a scenario is one of many such operations, so unless you use it very intensively the odds of causing corruption should be low, or at least it shouldn't be a major contributor to these odds.

    Again, migrating the internal DBs to Postgres is the best answer you can bring to internal DB corruption.

  • farhanromli
    farhanromli Registered Posts: 25 ✭✭✭✭
Setup Info
    Tags
      Help me…