myoungwon oh [Tue, 17 Aug 2021 12:51:29 +0000 (21:51 +0900)]
test: allowing >= the real refcount to avoid false alarm
Current dedup allow to contain multiple same sources using
multiset, which results in inconsistent situation as follow
(during set_chunk, but not confined in set_chunk).
1. User issues set_chunk
2. OSD receives the set_chunk, and sends increment message
to an object in the low tier (INPROGRESS).
3. OSD map is changed (841 → 843)
3.5. on_change() is called
4. the set_chunk op is reenqueued by requeue_op()
5. OSD handles the duplicated set_chunk, but it is not able to
know the set_chunk is duplicated because it does not log on the disk yet.
6. OSD issues increment message again to the object
in the low tier. (increment operation is executed twice)
To fix this, this commit allows >= the real refcount in test cases
Kefu Chai [Wed, 16 Jun 2021 11:58:42 +0000 (19:58 +0800)]
crimson/os: use reference for loop variable
for better performance, also silences the warning like:
../src/crimson/os/seastore/random_block_manager/nvme_manager.cc:444:23: warning: loop variable ‘b’ creates a copy from type ‘const crimson::os::seastore::rbm_alloc_delta_t’ [-Wrange-loop-construct]
444 | for (const auto b : alloc_blocks) {
| ^
Kefu Chai [Wed, 16 Jun 2021 11:05:53 +0000 (19:05 +0800)]
crimson/os: return seastar::now() in "finally()" block
so finally() is able to identify the return is a future, and discard it
manually.
otherwise the return value will be discarded even the future is marked
[[nodiscard]], hence the C++ compiler warns.
Kefu Chai [Wed, 16 Jun 2021 09:11:50 +0000 (17:11 +0800)]
crimson: adapt to the new alien API
since seastar now deprecates the old alien::submit_to() API, and
replaced it with a variant which requires a new parameter of "alien"
instance, let's pick it up, to amortize the cost to catch up with
the seastar master HEAD.
Patrick Donnelly [Tue, 15 Jun 2021 17:32:49 +0000 (10:32 -0700)]
Merge PR #41773 into master
* refs/pull/41773/head:
test/pybind/test_ceph_argparse: add a test where args contains comma
cmake: run nose test using "python3 -m nose"
test/pybind/test_ceph_argparse: check for expected argument
test/pybind/test_ceph_argparse: decorate helper method with @nottest
test/pybind: do not test obsoleted command
test/pybind: drop forward and readforward from test
test/pybind: do not use tab for indent
pybind/ceph_argparse: validate csv if desc.N
test/pybind: pep8 cleanups
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com> Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
crimson/osd: fix missed OpSequencer::abort() on primary change.
Commit b5efdc6f1c9563357d7dfd33a8f379053592a215 has unified
the interruption handling among `InternalClientRequest` and
`ClientRequest`. Unfortunately, a call to `abort()` of
`OpSequencer` has been overlooked and dropped. This was causing
crashes at Sepia after after fixing the ordering issues in
`ClientRequest`.
Sebastian Wagner [Tue, 15 Jun 2021 09:24:34 +0000 (11:24 +0200)]
pyhton-common: fix mypy errors
Fixes:
```
py3 run-test: commands[2] | mypy --config-file=../mypy.ini -p ceph
ceph/deployment/service_spec.py: note: In member "yaml_representer" of class "ServiceSpec":
ceph/deployment/service_spec.py:659: error: Argument 1 to "represent_dict" of "SafeRepresenter" has incompatible type "_OrderedDictItemsView[str, Any]"; expected "Mapping[Any, Any]"
```
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
Sebastian Wagner [Tue, 15 Jun 2021 08:19:40 +0000 (10:19 +0200)]
mgr/orch: fix mypy errors
Fixes:
```
orchestrator/__init__.py:6: note: In module imported here:
orchestrator/_interface.py: note: In member "yaml_representer" of class "DaemonDescription":
orchestrator/_interface.py:1039: error: Argument 1 to "represent_dict" of "SafeRepresenter" has incompatible type "ItemsView[Any, Any]"; expected "Mapping[Any, Any]"
orchestrator/_interface.py: note: In member "yaml_representer" of class "ServiceDescription":
orchestrator/_interface.py:1178: error: Argument 1 to "represent_dict" of "SafeRepresenter" has incompatible type "ItemsView[Any, Any]"; expected "Mapping[Any, Any]"
orchestrator/_interface.py: note: At top level:
orchestrator/_interface.py:1181: error: Argument 2 to "add_representer" has incompatible type "Callable[[SafeDumper, DaemonDescription], Any]"; expected "Callable[[SafeDumper, ServiceDescription], Node]"
Found 3 errors in 1 file (checked 29 source files)
```
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
myoungwon oh [Fri, 21 May 2021 05:00:02 +0000 (14:00 +0900)]
seastore: use deltas to deliver allocated info
alloc_extent() will allocates continous blocks (e.g., block 1 ~ 4).
To deliver such informations to journal efficiently,
this commit adds a member variable regarding only modified range in the deltas
Jinyong Ha [Fri, 16 Apr 2021 08:12:55 +0000 (17:12 +0900)]
seastore: add NormalNBD
NormalNBD - NVMe Block device with seastar file
1. Support preffered write granularity/alignment which is reported
from NVMe SSD. User should follow this write guidance for enhancing write
performance.
2. Support multi-stream IO in NVMe SSD.
one transaction got an extent whose state is MUTATION_PENDINGat that time.
but another transaction do split and set the extent state to RETIRED.
when the first transaction resume and do continuation, the state of the extent
has been changed to RETIRED. So need eagain to try again.
Laura Flores [Mon, 14 Jun 2021 17:03:01 +0000 (12:03 -0500)]
script/build-integration-branch: update print statement based on update to documentation
The print statement originally suggested to the user to run "make && ctest -j12", but that is outdated. The user should execute "./run-make-check.sh" instead.
Kefu Chai [Sun, 13 Jun 2021 11:56:26 +0000 (19:56 +0800)]
test/pybind/test_ceph_argparse: add a test where args contains comma
to ensure the support for the new syntax of "prefix --name bon,jour"
does not break existing behavior of "prefix hello cruel,world" where value2
contains reads "hello,world", and the parsed result should be
prefix="prefix"
value=["hello", "cruel,world"]
instead of something like
prefix="prefix"
value=["hello", "cruel", "world"]
or
prefix="prefix"
value=["cruel", "world"]
the above test only applies to the test where "value" is a CephString.
if "value" is a CephChoices, the parsed argument should be
Kefu Chai [Wed, 9 Jun 2021 03:11:40 +0000 (11:11 +0800)]
test/pybind: do not test obsoleted command
"scrub" command was marked obsoleted in e9a5ce0897efc6126caeebea9900bf05ec3d2174, so the test_ceph_argparse
cannot retrieve its command description using "get_command_descriptions"
cli tool anymore, let's drop the related test accordingly.
projects / ceph.git / log