diff options
| author | Caleb Sander Mateos <csander@purestorage.com> | 2025-06-19 08:34:34 -0600 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2025-07-02 17:11:23 -0600 |
| commit | daa01d954b13a178c216b6a91f8451a7b83b3bf6 (patch) | |
| tree | 89cf9e2ee9c1b4f26425319cd7b91692178485fb | |
| parent | e448d578264a9512d38deb8c418954d5f3e20712 (diff) | |
io_uring/rsrc: skip atomic refcount for uncloned buffers
io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's
reference count in case it has been cloned into another io_ring_ctx's
registered buffer table. This is an expensive operation and unnecessary
in the common case that the io_mapped_ubuf is only registered once.
Load the reference count first and check whether it's 1. In that case,
skip the atomic decrement and immediately free the io_mapped_ubuf.
Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://lore.kernel.org/r/20250619143435.3474028-1-csander@purestorage.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
| -rw-r--r-- | io_uring/rsrc.c | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index d724602697e7..fc51ca7de733 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -135,8 +135,10 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu) static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu) { - if (!refcount_dec_and_test(&imu->refs)) - return; + if (unlikely(refcount_read(&imu->refs) > 1)) { + if (!refcount_dec_and_test(&imu->refs)) + return; + } if (imu->acct_pages) io_unaccount_mem(ctx, imu->acct_pages); |