Preempt the job queue
natejgardner
Neuron, Registered, Neuron 2022, Neuron 2023 Posts: 151 Neuron
I often run into this message when building datasets by hand:
[2022/05/13-11:08:07.403] [ActivityExecutor-32] [INFO] [dku.flow.jobrunner] running compute_allocated_liners_NP - Waiting for a free slot before executing this activity... [2022/05/13-11:08:07.410] [ActivityExecutor-32] [INFO] [dku.flow.jobrunner] running compute_allocated_liners_NP - Reason: The global limit of 12 running activities has been reached.
Without lifting the global limit, I'd like to be able to put the job I'm working on at the top of the queue, preempting whatever's already running, sending it back to the queue to be restarted after the job I selected is completed. This will speed up the workflow in big projects that have a lot of data moving all the time.