Linux kernel: io_uring ring pollers can sleep through pending completions

In the Linux kernel, the following vulnerability has been resolved:

io_uring: wake ring pollers for local task_work

With IORING_SETUP_DEFER_TASKRUN completions are queued as local work, and only become CQEs when the submitter enters the kernel to wait for events. For an application that finds out via poll() or epoll on the ring fd when it is time to do that, io_uring_poll() reports EPOLLIN if local work is pending.

That only helps if the work is there at the time of the call. When DEFER_TASKRUN was added, io_req_local_work_add() did io_cqring_wake(), which took care of tasks sleeping in poll() as well. Then polling got a wait queue of its own, ->poll_wq, and the wake up in io_req_local_work_add() was replaced by waking the submitter directly. Since then ->poll_wq is only woken when CQEs are committed, which for such a ring is done by the very task that is waiting to be told to:

submitter completion poll(ring_fd) io_uring_poll() // nothing yet schedule() io_req_local_work_add() // ->cq_wait_nr: nobody waits // sleeps on

Wake ->poll_wq when the first entry is put on the work list. Like for the eventfd, that must not be done right away if the work is queued from a wake up handler, as the chain of wake ups could get back to the wait queue whose lock is held. Use an RCU callback in that case, with a reference to the ring.