diff options
| author | Jens Axboe <axboe@kernel.dk> | 2021-09-13 09:20:44 -0600 |
|---|---|---|
| committer | Greg Kroah-Hartman <gregkh@linuxfoundation.org> | 2021-09-18 13:40:06 +0200 |
| commit | 9ac218642dfc1309682baa708e81935e5cec5ff3 (patch) | |
| tree | bc4fae2e41efcec28acab9c2dfaff4df4d1a356a | |
| parent | 548ee201fb4a3c372a332284df3ecc21749918e1 (diff) | |
io-wq: fix wakeup race when adding new work
commit 87df7fb922d18e96992aa5e824aa34b2065fef59 upstream.
When new work is added, io_wqe_enqueue() checks if we need to wake or
create a new worker. But that check is done outside the lock that
otherwise synchronizes us with a worker going to sleep, so we can end
up in the following situation:
CPU0 CPU1
lock
insert work
unlock
atomic_read(nr_running) != 0
lock
atomic_dec(nr_running)
no wakeup needed
Hold the wqe lock around the "need to wakeup" check. Then we can also get
rid of the temporary work_flags variable, as we know the work will remain
valid as long as we hold the lock.
Cc: stable@vger.kernel.org
Reported-by: Andres Freund <andres@anarazel.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
| -rw-r--r-- | fs/io-wq.c | 8 |
1 files changed, 4 insertions, 4 deletions
diff --git a/fs/io-wq.c b/fs/io-wq.c index 8bb17b6d4de3c..3d5fc76b92d01 100644 --- a/fs/io-wq.c +++ b/fs/io-wq.c @@ -895,7 +895,7 @@ append: static void io_wqe_enqueue(struct io_wqe *wqe, struct io_wq_work *work) { struct io_wqe_acct *acct = io_work_get_acct(wqe, work); - int work_flags; + bool do_wake; unsigned long flags; /* @@ -909,14 +909,14 @@ static void io_wqe_enqueue(struct io_wqe *wqe, struct io_wq_work *work) return; } - work_flags = work->flags; raw_spin_lock_irqsave(&wqe->lock, flags); io_wqe_insert_work(wqe, work); wqe->flags &= ~IO_WQE_FLAG_STALLED; + do_wake = (work->flags & IO_WQ_WORK_CONCURRENT) || + !atomic_read(&acct->nr_running); raw_spin_unlock_irqrestore(&wqe->lock, flags); - if ((work_flags & IO_WQ_WORK_CONCURRENT) || - !atomic_read(&acct->nr_running)) + if (do_wake) io_wqe_wake_worker(wqe, acct); } |