diff options
| author | Caleb Sander Mateos <csander@purestorage.com> | 2025-06-20 09:10:06 -0600 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2025-06-30 20:13:42 -0600 |
| commit | 1ceeedb5974937a6d881fd0d92f143ca16f55577 (patch) | |
| tree | 1ef427642c64de78e0eb628a8c911702e55f348b | |
| parent | 8a8fe42d765bb22cc0e91b053bfebf0fbdec3b5a (diff) | |
ublk: optimize UBLK_IO_UNREGISTER_IO_BUF on daemon task
ublk_io_release() performs an expensive atomic refcount decrement. This
atomic operation is unnecessary in the common case where the request's
buffer is registered and unregistered on the daemon task before handling
UBLK_IO_COMMIT_AND_FETCH_REQ for the I/O. So if ublk_io_release() is
called on the daemon task and task_registered_buffers is positive, just
decrement task_registered_buffers (nonatomically). ublk_sub_req_ref()
will apply this decrement when it atomically subtracts from io->ref.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Link: https://lore.kernel.org/r/20250620151008.3976463-13-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
| -rw-r--r-- | drivers/block/ublk_drv.c | 9 |
1 files changed, 8 insertions, 1 deletions
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index e38d4fe5e833..c3c0601db487 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -2030,7 +2030,14 @@ static void ublk_io_release(void *priv) struct ublk_queue *ubq = rq->mq_hctx->driver_data; struct ublk_io *io = &ubq->ios[rq->tag]; - ublk_put_req_ref(ubq, io, rq); + /* + * task_registered_buffers may be 0 if buffers were registered off task + * but unregistered on task. Or after UBLK_IO_COMMIT_AND_FETCH_REQ. + */ + if (current == io->task && io->task_registered_buffers) + io->task_registered_buffers--; + else + ublk_put_req_ref(ubq, io, rq); } static int ublk_register_io_buf(struct io_uring_cmd *cmd, |