Xinyu Huang [Wed, 17 Aug 2022 04:19:24 +0000 (04:19 +0000)]
crimson/os/seastore/omap: optimize merge policy
in some corner cases on single key erase sence,
for e.g. we have three nodes which are brothers:
1 2 3 4 5 | 6 7 8 | 9 10 11, node min size is 3.
When erasing 11 will trigger balance instead of
merge, the middle nodes will adjust to 6 7, this
will not ensure the all nodes above min size.
This corner case might not be harmful but if we
want to erase 10,11 at one time (range remove wip
need this), this will trigger balance in past
policy which will make both two nodes below min
size and we might need at least one more merge.
This is actually because we cannot ensure a node
to merge in tree is above min size.
Ilya Dryomov [Mon, 15 Aug 2022 07:44:09 +0000 (09:44 +0200)]
test/librbd: various fixups for compare-and-write test cases
- missing EILSEQ asserts in IO and IOWithIOHint
- AioCompletion leak in CompareTooSmall and WriteTooSmall
- skip StripeUnitUnaligned and TooLarge on the PWL cache as it wasn't
limited by the 512-byte sector limit before and isn't limited by the
stripe unit limit now
cmp_bl can no longer be shorter than read_bl (i.e. image extent).
This is caught very early at the API level, see commit af96e6dae3f4
("librbd: make C++ cmp&write semantics equal to C API").
However, cmp_bl can still be longer than read_bl and in that case
it should be capped by the image extent length.
Ilya Dryomov [Mon, 15 Aug 2022 07:44:09 +0000 (09:44 +0200)]
librbd: discard cache state if compare-and-write writes to disk
Otherwise this is obviously broken with rbd_cache_policy = writeback or
rbd_cache_policy = writethrough as the write is done on the OSD, beneath
ObjectCacher. This went unnoticed because the original (and so far the
only known) compare-and-write user, tcmu-runner rbd driver, disables the
cache altogether.
Zac Dover [Sat, 13 Aug 2022 23:14:56 +0000 (09:14 +1000)]
doc/rados/operations: add prompts to operating.rst
This commit adds ".. prompt:: bash $"-style prompts to operating.rst.
This brings this file up to the standard established in 2020 when
Kefu added support for the ".. prompt::" directive.
This commit is a part of an initiative to modernize the presentation
of all BASH commands in the RADOS documentation.
The progress of this project can be tracked here:
https://tracker.ceph.com/issues/57108
Jonas Pfefferle [Fri, 5 Aug 2022 17:37:55 +0000 (19:37 +0200)]
librbd: make C++ cmp&write semantics equal to C API
The compare and write C++ API (both sync and async) does not check
compare bufferlist length and executes compare ops of bufferlist length
size, even if (write) "len" is smaller than bufferlist length.
This patch changes this behavior by always issuing compares and
writes of "len" size to match the C API. The bufferlist length
can be greater than "len" for both compare and write but only
"len" bytes are compared and written. If the bufferlist length
is smaller than "len" the call will fail.
Jonas Pfefferle [Fri, 5 Aug 2022 17:36:36 +0000 (19:36 +0200)]
librbd: remove cmp&write sector size restriction
This patch removes the compare and write max sector size len
restriction. We can allow up to stripe unit size accesses if
the access is aligned properly.
To allow larger size compare and write requests in the
journal we split the buffers like we do for writes now.
Ilya Dryomov [Fri, 12 Aug 2022 09:10:45 +0000 (11:10 +0200)]
rbd: remove incorrect use of std::includes()
- std::includes() requires sorted ranges but command specs aren't
sorted
- std::includes() purpose is to check whether the second range is
a subsequence of the first range but here the size of the second
range is always equal to the size of the first range, which means
that, had the ranges been sorted, std::includes() would have checked
straight equality
Ilya Dryomov [Fri, 12 Aug 2022 09:10:45 +0000 (11:10 +0200)]
rbd: find_action() should sort actions first
The order in which objects with static storage duration in
different TUs are initialized is undefined. If the compiler
chooses to initialize Shell::Action objects in action/Trash.cc
before Shell::Action objects in action/TrashPurgeSchedule.cc,
all "rbd trash purge schedule ..." commands get shadowed by
"rbd trash purge" command:
$ rbd trash purge schedule list
rbd: too many arguments
The confusing error arises because "rbd trash purge" takes a single
positional argument. "schedule" gets interpreted as <pool-spec> and
"list" generates an error.
Kefu Chai [Fri, 12 Aug 2022 05:06:25 +0000 (13:06 +0800)]
mgr/dashboard: bump up more-itertools
before this change, more-itertools tries to import Sequence from
collections, this leads us to failures like:
```
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
__import__(pkg_name)
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/__init__.py",
line 9, in <module>
import cherrypy
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cherrypy/__init__.py",
line 76, in <module>
from . import _cprequest, _cpserver, _cptree, _cplogging, _cpconfig
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cherrypy/_cprequest.py",
line 11, in <module>
from cherrypy import _cpreqbody
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cherrypy/_cpreqbody.py",
line 135, in <module>
import cheroot.server
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cheroot/server.py",
line 96, in <module>
from .workers import threadpool
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/cheroot/workers/threadpool.py",
line 20, in <module>
from jaraco.functools import pass_none
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/jaraco/functools.py",
line 8, in <module>
import more_itertools
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/more_itertools/__init__.py",
line 1, in <module>
from more_itertools.more import * # noqa
File
"/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/lib/python3.10/site-packages/more_itertools/more.py",
line 3, in <module>
from collections import Counter, defaultdict, deque, Sequence
ImportError: cannot import name 'Sequence' from 'collections'
(/usr/lib/python3.10/collections/__init__.py)
ERROR: InvocationError for command
/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/bin/python3
-m dashboard.controllers.docs
/home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/.tox/openapi-check/tmp/openapi.yaml
(exited with code 1)
```
after this change, more-itertools is pin'ed at the latest stable
at the time of writing, which includes the fixes including
https://github.com/more-itertools/more-itertools/commit/30a861bc5a4f53a9ba73923c9048a3632a0f9d18
.
please note, more-itertools dropped python3.3 support. but neither
do us support this python version, so we should be safe.
In the middle of test_random_write_concurrrent(), there is a chance
that consume(laddr A), regarding transaction 1, in try_submit_transaction
can be called after transaction 2's consume(laddr A) is invoked (note that
seastore handles both transaction 1 and 2 in order, and each other's laddr
are the same).
To avoid this, this commit adds a write seqeunce to prevent overwriting
old checksum associated with laddr---old checksum will not be updated
after write sequence comparison.
projects / ceph.git / log