Queued Activities and Job Prioritization

somepunter
Level 3
Queued Activities and Job Prioritization

Is there a way to

  1. query the number of waiting activities due to insufficient slots
    i. at any given time
    ii. Historically?
  2. differentiate an activity in waiting status because itโ€™s awaiting an upstream dependent activity vs a job slot vs global slot.
  3. prioritize jobs over others? E.g. a priority param per job?
  4. If not, can you share the scheduling algorithm? E.g. Is it round robin, FIFO per job or per activity?
0 Kudos
2 Replies
Clรฉment_Stenac

Hi,

1) No

2) If it's waiting for a job slot, it will be in NOT_STARTED state. If it's waiting for a global slot, it will be in RUNNING state

3) No

4) Simple fair queue per limit (so at least, one per job, one global - plus additional one if you setup additional custom limits)

0 Kudos
somepunter
Level 3
Author

Thanks,

2a) in the case of awaiting global slot, how would I differentiate that against an actual running job?

b) will the recorded start time be the time it actually runs or when it was ready to run?

c) apologies, I can't find the enumeration of all statuses in your docs, could you point me to it please?

0 Kudos

Labels

?
Labels (2)
A banner prompting to get Dataiku