Kamoltat [Wed, 14 Dec 2022 19:54:00 +0000 (19:54 +0000)]
mon/Monitor.cc: notify_new_monmap() skips removal of non-exist rank
Problem:
In RHCS the user can choose to manually remove a monitor rank
before shutting the monitor down. Causing inconsistency in monmap.
for example we remove mon.a from the monmap, there is a short period
where mon.a is still operational and will try to remove itself from
monmap but we will run into an assertion in
ConnectionTracker::notify_ranks_removed().
Solution:
In Monitor::notify_new_monmap() we prevent the func
from going into removing our own rank, or
ranks that doesn't exists in monmap.
FYI: this is an RHCS problem only, in ODF,
we never remove a monitor from monmap
before shutting it down.
Casey Bodley [Wed, 14 Dec 2022 19:59:58 +0000 (14:59 -0500)]
Merge pull request #49141 from cbodley/wip-58111
rgw: rgw_parse_url_bucket() rejects empty bucket names after 'tenant:'
Reviewed-by: Daniel Gryniewicz <dang@redhat.com> Reviewed-by: Adam C. Emerson <aemerson@redhat.com> Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@cern.ch>
When a physical volume isn't a member of any volume group,
the function `get_all_devices_vgs()` in `ceph_volume.api.lvm`
shouldn't try to return an `VolumeGroup()` object against this
device.
Kefu Chai [Wed, 14 Dec 2022 11:10:54 +0000 (19:10 +0800)]
crimson/osd: include fmt/ranges.h
so we are able to format ss.clone_snaps, which is a
std::map<snapid_t, std::vector<snapid_t>>. before fmtlib v9,
fmtlib is able to fall back to the operator<<() to format the the types
with the operator<<() defined. but after fmtlib v9, we need to
explicitly define them for accessing the specialized formatter.
in this change, fmt/ranges.h is included so we can access the formatter
for std::map and std::vector.
Kefu Chai [Wed, 14 Dec 2022 11:09:49 +0000 (19:09 +0800)]
crimson/os: specialize fmt::formater<> for lba_pin_list_t
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<lba_pin_list_t> is defined so
the tree can compile with fmt v9.
Kefu Chai [Wed, 14 Dec 2022 07:17:33 +0000 (15:17 +0800)]
crimson/os: specialize fmt::formater<> for types
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<> is specialized for tree_stats_t and
test_item_t so the tree can compile with fmt v9.
Kefu Chai [Wed, 14 Dec 2022 00:32:44 +0000 (08:32 +0800)]
seastar: bump up seastar submodule to recent master
to address the fmtlib v9 compatibility issues.
please note, the submodule is bump up to a commit which includes
* a revert commit on top of upstream master HEAD. the revert is to
work around the a recent change which adds a constraint on the parameter
type of handle_exception(). we are using errorator along with
handle_exception(), but errorator cannot fulfill the requirement of
seastar::Future(), so we need to drop the change before we have a
better solution.
* a commit which defines __NR_pidfd_open if it is not defined.
this should address the FTBFS on CentOS8 where glibc does not have
this macro defined.
Kefu Chai [Wed, 14 Dec 2022 00:27:13 +0000 (08:27 +0800)]
test/cls_fio: specialize fmt::formater<> for bpo::options_description
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<bpo::options_description> is defined so
the tree can compile with fmt v9.
Kefu Chai [Wed, 14 Dec 2022 00:20:42 +0000 (08:20 +0800)]
cls/fifo: specialize fmt::formater<> for rados::cls::fifo::info
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<rados::cls::fifo::info> is defined so
the tree can compile with fmt v9.
Marcus Watts [Wed, 30 Nov 2022 23:22:30 +0000 (18:22 -0500)]
rgw: added rgw-policy-check
rgw-policy-check - a program to do syntax checking on bucket policy.
This program just reads the policy into memory, so it is not
checking anything except syntax.
Signed-off-by: Marcus Watts <mwatts@redhat.com>
rgw: Fix return value of `rgw-policy-check`
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
rgw: Use ceph initialization in `rgw-policy-check`
Specifically so we can pull in the options from `ceph.conf` and similar.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Kefu Chai [Tue, 13 Dec 2022 10:23:43 +0000 (18:23 +0800)]
crimson/os: define device_spec_t as packed
per C++ standard 20, alignas cannot be applied to scoped enum. despite
that there is a resolution to address this, see
https://cplusplus.github.io/CWG/issues/2354.html, it's not included in
C++20. so we have to use a different way to address
> runtime error: reference binding to misaligned address 0x610000008395
> for type 'device_type_t', which requires 4 byte alignment
otherwise the code would fail to compile with Clang-15, like:
> <source>:3:13: error: 'alignas' attribute cannot be applied to an enumeration
Add inclusive behavior for both first and last in omap_list_config_t,
and optimize the usage.
Set first closed last open for range for omap range remove.
Xiubo Li [Wed, 23 Nov 2022 05:24:38 +0000 (13:24 +0800)]
qa: switch to https protocol for repos' server
Since the git:// is not reachable any more and have switch to
https://.
The git archive does not support the https protocol, so we couldn't
user the git archive to retrieve the tar ball any more, will split
this into 3 steps:
1, clone the whole ceph repo
2, checkout the commit/tag/branch
3, then change directory to qa/workunits/.
qa: Allow tests to override recovery configs with mClock scheduler enabled
Set osd_mclock_override_recovery_settings option to true for tests that
modify recovery/backfill configuration options. This prevents logging of
the cluster warning when modifying recovery/backfill limits.
Cory Snyder [Wed, 2 Nov 2022 20:06:23 +0000 (20:06 +0000)]
rgw: add 'inline_data' zone placement info option
Adds a new RGW zone placement info option to control whether
an object's first data chunk is placed in the head object.
This allows admins to make a tradeoff between optimizing for
PUT/GET performance vs. DELETE performance for some cluster
configurations.
qa/standalone: Add/Modify tests to verify mclock recovery/backfill limits
- Consolidate all mclock standalone tests under
qa/standalone/misc/mclock-config.sh.
- Revert existing tests in ceph-helpers.sh that verified the earlier hard
override of recovery/backfill limits.
- Add new tests to verify the procedure to change the recovery/backfill
limits with mclock scheduler.
doc: Update mClock config reference doc to reflect new max recovery limits
Document the following:
- New max backfill/recovery defaults for mClock.
- Steps to modify the backfill/recovery defaults.
- Modify defaults using new osd_mclock_override_recovery_settings option
- Steps to mitigate unrealistic OSD bench results to set OSD capacity.
- New capacity threshold options for ssd/hdd
With the earlier cost per io and cost per byte parameters, the
client ops performance was good but the recovery ops were taking
much longer to complete. The new cost parameters optimizes this
further so that the recovery ops progress faster while
maintaining similar client throughput.
This optimization is tested on both a small cluster and a larger
cluster in a scaled environment.
projects / ceph.git / log