We're excited to announce that we're launching the second installment of Dataiku Product Days Register Now

Define global job limit on recipe engine

Solved!
tomas
Level 5
Define global job limit on recipe engine

The DSS allows to define a recipe type or tag based limits globally https://doc.dataiku.com/dss/latest/flow/limits.html.



However I dont know how to set it per engine, so for example I would like to set a limit to run at most 10 impala queries and 10 spark jobs at the time, in total at most 20 jobs. But if the scenario is building spark only flow, I want to keep the 10 limit, and vice versa if the flow is pure Impala, I want to keep 10 limit for Impala.



Impala - total 10



Spark - total 10



Global - 20 (or unlimited?)



Is it possible to define it in "Additional limits" section?

0 Kudos
1 Solution
Clément_Stenac
Dataiker
Dataiker
Hi,

DSS does not have feature to limit activities per engine type.

View solution in original post

2 Replies
Clément_Stenac
Dataiker
Dataiker
Hi,

DSS does not have feature to limit activities per engine type.

View solution in original post

tomas
Level 5
Author
If it is not very complicated to implement, then +1 for this feature. Thanks
0 Kudos

Labels

?
Labels (2)
A banner prompting to get Dataiku DSS