Limiting per project jobs for all projects

sylvyr3
sylvyr3 Registered Posts: 21 ✭✭✭✭

Hi,

Is it possible to limit the number of jobs to say 5 for each project on a DSS instance? The documentation indicates I can do it for specific projects but doesn't make it clear if it's possible for all projects.

I'd like to set the global limit to 20 and limit every project so no project exceeds 5 jobs. It's too cumbersome to list out every project and would need to be updated as users create new projects.

Best Answer

  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 415 Neuron
    Answer ✓

    Hi @sylvyr3

    I think what you are looking for is in Administration > Settings > Flow build:

    flow build.png

     What can be limited is not directly the jobs, but the number of activities that can run simultaneously: if each job you run has one activity, then with a limit of 5 only 5 jobs will run concurrently; if some jobs have 2 or more activities, the number of jobs running will be less.

    From here you can set more limits using keys and values: documentation is at https://doc.dataiku.com/dss/latest/flow/limits.html

    Hope this helps!

Answers

Setup Info
    Tags
      Help me…