Jonas Pfefferle [Thu, 29 Sep 2022 20:02:56 +0000 (22:02 +0200)]
librbd: remove unnecessary write in cmp&write test
Remove write in compare and write test that tests
successful completion when bufferlist is greater then len.
The write was redundant and overwritten directly after.
duplicates some of the logic from member functions of
RGWRealm/Period/ZoneGroup/ZoneParams, but as free functions in terms
of the sal::ConfigStore APIs
[ 0%] Building CXX object src/compressor/CMakeFiles/compressor_objs.dir/Compressor.cc.o
In file included from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/config_values.h:59,
from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/config.h:27,
from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/config_proxy.h:6,
from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/ceph_context.h:41,
from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/compressor/Compressor.cc:23:
/d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/options/legacy_config_opts.h:1:10: fatal error: global_legacy_options.h: No such file or directory
1 | #include "global_legacy_options.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [src/compressor/CMakeFiles/compressor_objs.dir/build.make:76: src/compressor/CMakeFiles/compressor_objs.dir/Compressor.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:7964: src/compressor/CMakeFiles/compressor_objs.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.1CBUsC (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.1CBUsC (%build)
Solution: add legacy-option-headers dependency to compressor_objs.
Laura Flores [Wed, 28 Sep 2022 17:17:17 +0000 (17:17 +0000)]
mgr/telemetry: handle daemons with complex ids
Treating daemons as `<daemon_type>.x` caused a crash
in the Telemetry module since the current method does not cover a case
where a daemon id is more complex, i.e. `<daemon_type>.x.y`.
When we parse the daemon type and daemon id, we should
split it into a maximum of two pieces rather than splitting
it by every `.` character. Specifying `1` in the Python
.split() function will limit the split to a maximum of two items.
Fixes: https://tracker.ceph.com/issues/57700 Signed-off-by: Laura Flores <lflores@redhat.com>
John Mulligan [Tue, 27 Sep 2022 18:50:44 +0000 (14:50 -0400)]
cephadm: rename import from cd to _cephadm
The `cd` name is an abbreviation of an obsolete name for cephadm.
As this is not widely known it could be confusing or conflict with
other common terms like `cd` for "change dir" or "compact disc".
Therefore we rename it to something much clearer.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 27 Sep 2022 18:50:24 +0000 (14:50 -0400)]
cephadm: rename import from cd to _cephadm
The `cd` name is an abbreviation of an obsolete name for cephadm.
As this is not widely known it could be confusing or conflict with
other common terms like `cd` for "change dir" or "compact disc".
Therefore we rename it to something much clearer.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 27 Sep 2022 18:49:39 +0000 (14:49 -0400)]
cephadm: rename import from cd to _cephadm
The `cd` name is an abbreviation of an obsolete name for cephadm.
As this is not widely known it could be confusing or conflict with
other common terms like `cd` for "change dir" or "compact disc".
Therefore we rename it to something much clearer.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 26 Sep 2022 19:30:25 +0000 (15:30 -0400)]
cephadm: lazily import cephadm in fixtures.py
We can avoid creating side-effects in the cephadm module by only importing
the module in the functions that are meant to generate side effects.
Simply importing fixtures and not calling any of the functions now can
not change cephadm by accident.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 26 Sep 2022 19:20:25 +0000 (15:20 -0400)]
cephadm: remove container_engine argument from test helper function
The with_cephadm_ctx test helper function had an argument that *called*
mock_podman function, thus calling the function at import-time. This can
lead to confusing and unexpected side effects. Because the
container_engine argument is not used by any callers of with_cephadm_ctx
we remove the argument.
All other callers of with_cephadm_ctx that make use of container engine
are already setting `ctx.container_engine = <xyz>` so removing the
argument makes it harder to make the same mistake again in the future.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 26 Sep 2022 19:14:20 +0000 (15:14 -0400)]
cephadm: fix iscsi test_unit_run relying on how cephadm was being imported
The iscsi test_unit_run test case was relying upon a strange side-effect
of how the cephadm module was being imported in fixtures.py. In short,
it expected the podman container_engine not to be a subclass of Podman.
```
(Pdb) p ctx.container_engine
<Mock spec='Podman' id='140676342644256'>
(Pdb) p ctx.container_engine.__class__
<class 'cephadm.Podman'>
(Pdb) p isinstance(ctx.container_engine, Podman)
False
(Pdb) p id(ctx.container_engine.__class__) 94372514199712
(Pdb) p id(Podman) 94372516682112
```
In order to clean up fixtures.py and other imports of cephadm as a
module, we fix the test to expect a mock docker. We choose docker
because it only requires changes the path of the container engine. If we
changed it to a real podman subclass we'd have to change many arguments
in the expected CLI text.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Samuel Just [Thu, 8 Sep 2022 23:12:45 +0000 (23:12 +0000)]
crimson/net: Message::conn needs to be a foreign_ptr
There are two main consequences of this:
1. Messages can't be default copy constructed in crimson. MMonCommand
seems to be the only user, and we simply add a copy constructor that
duplicates data portions of the message.
2. We can't casually copy-construct the conn into other structures.
The main user here is watch/notify. We use copy() explicitely
to populate the object_context structures and avoid passing
ConnectionFRef by value.
Samuel Just [Fri, 2 Sep 2022 21:10:40 +0000 (14:10 -0700)]
crimson/common/operation: release pipeline stages on the core they are on
Otherwise, tasks waiting on the stage will wake up on the wrong core.
Later, we may choose to statically enable this behavior only for stages
that can actually span cores, but this is ok for now.
Samuel Just [Tue, 30 Aug 2022 22:00:37 +0000 (15:00 -0700)]
crimson/osd: replace global tid with unique core-local tids
We don't really want a global counter here if we can avoid it. Instead,
assign tids with core-local counters prefixed with the core id. We
continue to ensure that tids are unique within an osd boot, but lose
the property that sucessive tids on different cores are ordered. I
don't see anything relying on that property, however, so this should be
fine.
Samuel Just [Tue, 30 Aug 2022 21:57:59 +0000 (14:57 -0700)]
crimson/osd: remove explicit pg counter
The main user of num_pgs was get_target_pg_log_entries(), but we
don't really want to be accessing a global counter there if we
can avoid it. Instead, compute a core-local target and use the
core-local pg count.
For now, FuturizedStore implementations assume that methods are invoked
on core 0. Later, we'll adapt each implementation to intelligently
support invocation on any pg core. Until then, this wrapper converts
the existing implementations to a safe, if not particuarly performant,
proxy behavior.
Samuel Just [Thu, 1 Sep 2022 23:22:59 +0000 (23:22 +0000)]
crimson/osd: refactor pg management for multicore
OSDSingletonState will now only be responsible for the spg_t->core
mapping for pgs, the individual PerShardState's will hold local
PGMap's. PG management operations are now proxied from PGShardManager
to PerShardState. Subsequent patches will shard PerShardState.
crimson: create buffer from temporary_buffer with foreign-ptr by default
temporary_buffer is internally shareable with a thread-unsafe
ref-counter, we need to make sure it is released in the same core where
it is constructed.
Users that need the extra efficiency can swap to create_local as needed.
Samuel Just [Thu, 4 Aug 2022 07:17:50 +0000 (00:17 -0700)]
crimson/osd: rework OSDMap handling for multicore
OSDMaps can occupy significant space in memory. Duplicating OSDMaps
across cores would multiply that memory usage as well as complicate
the internal structure sharing we do when applying OSDMap incrementals.
Because PeeringState and other interfaces expect efficient copying of
OSDMapRef objects we don't want to use foreign_ptr directly.
Instead, maintain a single cache and distribute local_shared_foreign_ptrs
to other cores. ShardServices becomes the only OSDMapService.
Adam King [Thu, 22 Sep 2022 13:54:06 +0000 (09:54 -0400)]
Merge pull request #47756 from dparmar18/wip-dparmar-cephadm-after-revert
pybind/mgr/cephadm/upgrade: allow upgrades without reducing max_mds
Reviewed-by: Adam King <adking@redhat.com> Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
projects / ceph.git / log