Discover the winners & finalists of the 2022 Dataiku Frontrunner Awards!READ THEIR USE CASES

Define global job limit on recipe engine

Solved!
tomas
Define global job limit on recipe engine

The DSS allows to define a recipe type or tag based limits globally https://doc.dataiku.com/dss/latest/flow/limits.html.



However I dont know how to set it per engine, so for example I would like to set a limit to run at most 10 impala queries and 10 spark jobs at the time, in total at most 20 jobs. But if the scenario is building spark only flow, I want to keep the 10 limit, and vice versa if the flow is pure Impala, I want to keep 10 limit for Impala.



Impala - total 10



Spark - total 10



Global - 20 (or unlimited?)



Is it possible to define it in "Additional limits" section?

0 Kudos
1 Solution
Clément_Stenac
Dataiker
Hi,

DSS does not have feature to limit activities per engine type.

View solution in original post

2 Replies
Clément_Stenac
Dataiker
Hi,

DSS does not have feature to limit activities per engine type.
tomas
Author
If it is not very complicated to implement, then +1 for this feature. Thanks
0 Kudos

Labels

?
Labels (2)
A banner prompting to get Dataiku