Limiting per project jobs for all projects

[sylvyr3]

Hi,

Is it possible to limit the number of jobs to say 5 for each project on a DSS instance?  The documentation indicates I can do it for specific projects but doesn't make it clear if it's possible for all projects.  

I'd like to set the global limit to 20 and limit every project so no project exceeds 5 jobs.  It's too cumbersome to list out every project and would need to be updated as users create new projects.

[Ignacio_Toledo · Accepted solution]

Hi @sylvyr3 

I think what you are looking for is in Administration > Settings > Flow build:

 What can be limited is not directly the jobs, but the number of activities that can run simultaneously: if each job you run has one activity, then with a limit of 5 only 5 jobs will run concurrently; if some jobs have 2 or more activities, the number of jobs running will be less.

From here you can set more limits using keys and values: documentation is at https://doc.dataiku.com/dss/latest/flow/limits.html

Hope this helps!

[ecerulm]

@Ignacio_Toledo , but can you limit all project to 5 concurrent activities like @sylvyr3 asks? 

I mean for a specific single project you  can put in "Additional Limits" 

• Limit key: project/projectkey1
• Limit Value: 5

But I think @sylvyr3 (and me too) wants a way to say "every project get a limit of 5". To me  it seems that wildcards like  "project/*" or "user/*"  are not supported. 

I submitted a project idea Limiting concurrent executions (activities) / default limit for all users / projects  https://community.dataiku.com/t5/Product-Ideas/Limiting-concurrent-executions-activities-default-lim...

[Ignacio_Toledo]

I guess you are right @ecerulm, it looks like I misunderstood the original question, but since it was marked as "solved" I never went back to it.

Creating a project idea for this was the right path!