Join us at the Everyday AI Conference in London, New York & Bengaluru! REGISTER NOW

Preempt the job queue

0 Kudos

I often run into this message when building datasets by hand:

 

[2022/05/13-11:08:07.403] [ActivityExecutor-32] [INFO] [dku.flow.jobrunner] running compute_allocated_liners_NP - Waiting for a free slot before executing this activity...
[2022/05/13-11:08:07.410] [ActivityExecutor-32] [INFO] [dku.flow.jobrunner] running compute_allocated_liners_NP - Reason: The global limit of 12 running activities has been reached.

 

 

Without lifting the global limit, I'd like to be able to put the job I'm working on at the top of the queue, preempting whatever's already running, sending it back to the queue to be restarted after the job I selected is completed. This will speed up the workflow in big projects that have a lot of data moving all the time.