From: Kefu Chai Date: Fri, 10 Jul 2026 14:30:32 +0000 (+0800) Subject: crimson/os/seastore/journal: dispatch records greedily while io slots are free X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=0ada4fba7f8202deb3ca1ab132a6487c1a8a4d4b;p=ceph.git crimson/os/seastore/journal: dispatch records greedily while io slots are free `RecordSubmitter` dispatches a small record (or the pending batch) only once the journal is fully drained (`state == IDLE`): `submit()` takes the direct-write path only at IDLE, and `decrement_io_with_flush()` flushes the pending batch only after *all* outstanding io has completed. The remaining dispatch triggers never fire for 4K-class records: `preferred_fullness` measures the padding ratio of a single block, and small records do not reach the batch capacity. At low-to-mid client queue depth this degenerates into strict alternation: one journal io in flight, each batch carrying only the records that arrived during the previous device write. Measured with 4K randwrite QD20 against a single crimson OSD (circular-bounded journal): io_depth 1.07, 1.9 records/io, ~2.2ms `submit_record` latency -- 1.5x the raw device write+flush latency, since each record first waits ~0.5x for the in-flight io to drain. `io_depth_limit` (default 5) is never approached. Dispatch whenever the device has a free io slot instead: - `submit()`: take the direct-write path whenever `state != FULL`; records are written greedily while slots are free, and batching kicks in once `io_depth_limit` is reached. - `decrement_io_with_flush()`: flush the pending batch into the freed slot instead of waiting for IDLE. Ordering is unaffected: a batch's `write_base` is assigned at dispatch from `journal_allocator.get_written_to()`, so the on-disk layout follows submission order; completions are reported in order via the `device_submission` exit turnstile and the exclusive `finalize` phase, so a record is never acked before all earlier records are durable. Replay stops at the first invalid record, and anything beyond such a hole was never acked. The same concurrency already occurs through the `SUBMIT_FULL` path under high load; this extends it to the low-fullness regime. ## Benchmark Ceph 21.3.0 dev (RelWithDebInfo), single crimson OSD, 4 reactors pinned to 4 cores, fio rbd engine on 4 disjoint cores, size-1 pool, 24 GiB prefilled image, consumer NVMe without PLP (~0.7ms fdatasync). 4K randwrite QD20, 180s, `RANDOM_BLOCK_SSD` (circular-bounded journal): | metric | before | after | |-------------------------|-------:|--------------:| | IOPS | 5,800 | 10,918 (+88%) | | avg client latency | 3.4ms | 1.83ms | | avg journal io_depth | 1.07 | 3.18 | | records per io | 1.9 | 1.01 | | `submit_record` latency | 2.2ms | 1.05ms | 4K randwrite, 45s per queue depth, `RANDOM_BLOCK_SSD`: | iodepth | 1 | 4 | 16 | 64 | |---------|----:|------:|------:|--------:| | before | 452 | 1,270 | 3,470 | ~10,300 | | after | 536 | 1,908 | 9,337 | 15,333 | 4K randwrite QD20, 180s, `SEGMENTED` (segmented journal and ool writer, both built on `RecordSubmitter`): | metric | before | after | |----------------------|-------:|--------------:| | IOPS | 2,728 | 6,123 (+124%) | | avg client latency | 7.3ms | 3.26ms | | avg journal io_depth | ~1 | 2.26 | A classic BlueStore OSD on the same 4-core/QD20 harness does 10,243 IOPS; crimson previously trailed it by 1.8x and now matches it. Signed-off-by: Kefu Chai --- diff --git a/src/crimson/os/seastore/journal/record_submitter.cc b/src/crimson/os/seastore/journal/record_submitter.cc index b6d22de97e6..7fdac752b45 100644 --- a/src/crimson/os/seastore/journal/record_submitter.cc +++ b/src/crimson/os/seastore/journal/record_submitter.cc @@ -308,7 +308,9 @@ RecordSubmitter::submit( auto eval = p_current_batch->evaluate_submit( record.size, journal_allocator.get_block_size()); bool needs_flush = ( - state == state_t::IDLE || + // dispatch greedily while the device has free io slots; batching + // kicks in only once io_depth_limit is reached + state != state_t::FULL || eval.submit_size.get_fullness() > preferred_fullness || // RecordBatch::needs_flush() eval.is_full || @@ -507,12 +509,11 @@ void RecordSubmitter::decrement_io_with_flush() ceph_assert(!wait_unfull_flush_promise.has_value()); } + // dispatch the pending batch into the freed io slot instead of + // waiting for the journal to fully drain (state == IDLE) auto needs_flush = ( - !p_current_batch->is_empty() && ( - state == state_t::IDLE || - p_current_batch->get_submit_size().get_fullness() > preferred_fullness || - p_current_batch->needs_flush() - )); + !p_current_batch->is_empty() && + state != state_t::FULL); if (needs_flush) { DEBUG("{} flush", get_name()); flush_current_batch();