Linux kernel: io_uring race can hang a task that is exiting

In the Linux kernel, the following vulnerability has been resolved:

io_uring/io-wq: don't hide a link from cancellations

A task that exits cancels its tracked requests and waits until they are gone. io_uring_cancel_generic() looks everywhere a request can be, and goes to sleep if it found nothing to cancel. It is woken up each time one of the requests of the task is completed by task work or freed.

When io-wq is done with a request that has a link, there is a time in which the next request of the link can't be found:

io_worker_handle_work() io_assign_current_work(worker, NULL) io_wq_free_work(work) io_req_find_next() // takes the link off the request io_free_req() // task work for the task io_assign_current_work(worker, linked) io_run_task_work() cond_resched() worker->cur_work = linked

It is not pending, it is not the current work of a worker, and it is not yet on the list that it is about to be parked on. If the pass that the exiting task makes because of io_free_req() falls into that time, it finds nothing and the task goes to sleep. The worker then runs the link, which IO_WQ_BIT_EXIT doesn't prevent for bounded work. If that parks the request nothing is completed or freed, and nobody wakes up the task. If it is the process that exits it can't be killed any more either.

NOP, IOSQE_ASYNC | IOSQE_IO_LINK FUTEX_WAIT, FUTEX2_PRIVATE exit of the thread, or of the process, up to 63us later

With four of these per round 5 to 8 of 10,000 rounds hang (4 CPUs, KVM):

io_uring_cancel_generic do_exit

Commit f8b632e89a10 ("io_uring: tighten task exit cancellations") is about the same gap, and makes the task look again for whatever happens before io_free_req().

Free the request once the link is the current work of the worker, or pending. Whoever is woken up by that can find the link, until it is parked where cancellations look as well. To do so split io_wq_free_work() in two.