Define global job limit on recipe engine

tomas Registered, Neuron 2022 Posts: 120 ✭✭✭✭✭

The DSS allows to define a recipe type or tag based limits globally

However I dont know how to set it per engine, so for example I would like to set a limit to run at most 10 impala queries and 10 spark jobs at the time, in total at most 20 jobs. But if the scenario is building spark only flow, I want to keep the 10 limit, and vice versa if the flow is pure Impala, I want to keep 10 limit for Impala.

Impala - total 10

Spark - total 10

Global - 20 (or unlimited?)

Is it possible to define it in "Additional limits" section?

Best Answer


Setup Info
      Help me…