Custom trigger to run itself?

safi

Hi everyone,

I want to execute scenario again if its fail, so it can try 3 times, sometimes kubernetes fails or, spark fails could be fixed after run again so, i dont miss time range between fail and fix manually.

This is the code created with LLM, it used python for that. I changed project name to variable "project name", normally it written project name already, dont worry.

from dataiku import api_client

import time


def create_retry_trigger(scenario_name, project_key=None, max_retries=3, wait_time=300):

    """

    Creates a custom trigger to retry failed scenarios in Dataiku using scenario name (or ID).


    Parameters:

    scenario_name (str): Scenario ID (or name, if same) to retry

    project_key (str): Project key where the scenario is located. If None, uses the current project.

    max_retries (int): Maximum number of retry attempts.

    wait_time (int): Time to wait between retries in seconds.


    Returns:

    None

    """

    client = api_client()


    if project_key is None:

        project = client.get_default_project()

    else:

        project = client.get_project(project_key)


    try:

        # Directly get scenario by ID (if name matches ID)

        scenario = project.get_scenario(scenario_name)


        latest_runs = scenario.get_last_runs(1)

        if not latest_runs:

            print(f"No previous runs found for scenario '{scenario_name}'.")

            return


        latest_run = latest_runs[0]


        if latest_run['result']['outcome'] == 'FAILED':

            metadata = scenario.get_custom_metadata()

            retry_count = int(metadata.get('retry_count', 0))


            if retry_count < max_retries:

                scenario.set_custom_metadata({'retry_count': str(retry_count + 1)})


                print(f"Retrying scenario '{scenario_name}'. Attempt {retry_count + 1} of {max_retries}")


                time.sleep(wait_time)

                scenario.run()

            else:

                print(f"Maximum retry attempts ({max_retries}) reached for scenario '{scenario_name}'")

                scenario.set_custom_metadata({'retry_count': "0"})

        else:

            scenario.set_custom_metadata({'retry_count': "0"})


    except Exception as e:

        print(f"Error while processing scenario '{scenario_name}': {str(e)}")


# Example usage

create_retry_trigger(

    scenario_name="Project_name",

    project_key=None,  # Use None for the current project or specify a project key

    max_retries=3,

    wait_time=300  # 5 minutes

)

Operating system used: Windows

Turribeach

So what’s the problem here?

safi

https://community.dataiku.com/discussion/comment/45368#Comment_45368

It didnt work.:) I just sent solution of llm to explain exactly what i want.

Turribeach

It didnt work ⇒ Not really something I can work with. What's the error? What part does it fail on? What steps have you tried to fix it?

In my experience asking LLMs to write code for complex actions like this can easily lead to hallucinations. It's best to go step by step with simple actions and as you get the code you test it and fix each step as needed.

In any case this path seems wrong to me. Scenarios already have retry capabilities. You can do this in each scenario step. So set your retries there and you won't have to use API code. Finally if you want to write API code I suggest you look at the Dataiku Developer site. It's full of great and working examples and should have most of what you need. Here is the section for the Scenarios:

https://developer.dataiku.com/latest/concepts-and-examples/scenarios.html

safi

https://community.dataiku.com/discussion/comment/45370#Comment_45370

No error cause this code snippet didnt work at all(didnt triggered). I just created this to use quickly and provide it here to explain people, much simple way.

Thanks for your direction to link and guidance.

Turribeach

https://community.dataiku.com/discussion/comment/45372#Comment_45372

Python code in a Scenario can be hard to debug. It's recommended you first test the code in a Jupyter Notebook, if possible.