rgw: Break out RGWDataIncSyncShardCR

This was formerly the function
RGWDataSyncShardCR::incremental_sync. As with full_sync, we transfer
responsibility for acquiring the lease to the top level
RGWDataSyncShardCR coroutine.

Fixes: https://tracker.ceph.com/issues/57063
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Break out RGWDataFullSyncShardCR

This function was formerly RGWDataSyncShardCR::full_sync. The only
functional difference is that we leave acquiring the lease to the top
level RGWDataSyncShardCR coroutine class, since the lease should be
held on the transition from full to incremental sync.

Fixes: https://tracker.ceph.com/issues/57063
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Create base class for broken out SyncShard coroutines

The problem:

RGWDataSyncShardCR combines full and incremental sync in such a way
that yielding in full sync after the transition to incremental (but
before all the finishing up is done) can jump straight into
incremental.

To deal with this break full and incremental sync into their own
coroutine classes. For the shared functionality (but /not/ shared
data) between them, we have a base class into which parts of the
original class are moved.

Data that are shared between the two (shard cache, lease) are owned
and managed by RGWDataSyncShardCR, with the full and incremental base
class getting references.

Fixes: https://tracker.ceph.com/issues/57063
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw/qa: re-enable bucket_sync_disable and bucket_reshard while testing

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: remove redundant drain_all_cb() call

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: wait for outstanding data_sync_single_entry()s to complete
before spawning shards of next generation

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: move read bilog error injection into RGWDataFullSyncSingleEntryCR

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: RGWDataFullSyncSingleEntryCR: pass the right timestamp and
construct error repo using datalog_oid_for_error_repo()

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

test/rgw: test 'data sync init' over several bucket sync states

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b3662ccde54eee7de3ce3b680d314b8f049cc369)

rgw/multisite: code for error injection and extra logging

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

test/rgw: test 'data sync init' over several bucket sync states

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b3662ccde54eee7de3ce3b680d314b8f049cc369)

qa/multisite: add rgw_sync_data_inject_err_probability to test

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: handle RGWDataFullSyncSingleEntryCR() algorithm gracefully.

contributed by cbodley
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: handle marker_tracker updates.
handle retcodes correctly.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: changes to error handling.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: move out full sync trigger handling block to a new cr class.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: move sync_single_entry() cr function out of RGWDataSyncShardCR

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: full_sync changes.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: use configurable option for bucket sync concurrency window

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: Store generation and its corresponding num shards in a struct.
Call remove_cr() to remove full sync obligations after writing all
shard entries in to error repo. Replace call() with spawn() and yield_spawn_window()

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: build a coroutine class to handle error repo logic

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw/multisite: on reaching incremental sync, write rest of the shards of a
bucket to error repo instead of triggering a bucket sync on them.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #47449 from tchaikov/rpm-gts-11

ceph.spec.in: build with gcc-toolset-11

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #46876 from neesingh-rh/feature_55821

mgr/volumes: add interface to check the presence of subvolumegroups/subvolumes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

cmake: disable LTO when building pmdk

See-also: https://tracker.ceph.com/issues/54473
Signed-off-by: Kefu Chai <tchaikov@gmail.com>

debian: use dpkg/default.mk to init popular make variables

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #47263 from rhcs-dashboard/fix-host-service-instances

mgr/dashboard: host list tables doesn't show all services deployed

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

Merge pull request #46213 from ceph/feature-exporter

exporter: per node metric exporter

Reviewed-by: MrFreezeex <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

ceph.spec.in: BuildRequires gcc-toolset-11-libatomic-devel for aarch64

to address following failure when generating the building system
using CMake:

```
-- Performing Test HAVE_LIBATOMIC
-- Performing Test HAVE_LIBATOMIC - Failed
CMake Error at cmake/modules/CheckCxxAtomic.cmake:66 (message):
  Host compiler /opt/rh/gcc-toolset-11/root/usr/bin/c++ requires libatomic,
but it is not found
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

ceph.spec.in: call %enable_devtoolset11 on centos8

before this change %enable_devtoolset11 is called only when building
with crimson on centos8.

after this change %enable_devtoolset11 is called when building on
centos8. because we've started using gcc-toolset-11 for building
rpm packages on centos8 after the C++20 migration. so, to build
with gcc-11, we need to enable it.

also, because gcc-toolset-11 is used, we have to disable
annotated_build.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #47299 from hookak/loki-dependency

mgr/cephadm: Add loki dependency to grafana, promtail

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47165 from NitzanMordhai/wip-nitzan-librados-ioctx-destroy-Segmentation-fault

librados: rados_ioctx_destroy check for initialized ioctx

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #47138 from kamoltat/wip-ksirivad-fix-test-pool-min-size

qa/tasks/ceph_manager.py: increase test_pool_min_size timeout

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #47065 from ifed01/wip-ifed-fix-blufs-unlink-fsync

os/bluestore: do not signal deleted dirty file to bluefs log

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #46838 from pdvian/wip-version-mgrupdate

mgr: Remove service_daemon handling from MMgrUpdate

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #46507 from aclamk/wip-memstore-leak

os/memstore: Fix memory leak

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44774 from rishabh-d-dave/qa-cleanup

qa/ceph_manager: clean up CephManager.do_rados()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #47327 from rosinL/fix-op-release

crimson/osd/osd_operation: fix abort caused by op be deleted ahead

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #47026 from lxbsz/wip-56517

fuse_ll: fix compile error when libfuse version is larger than 3.0

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Luis Henriques <lhenriques@suse.de>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #47412 from rzarzynski/wip-crimson-monc-nothrow-on-renewal

crimson/net: don't throw on failure when renewing tickets and keyrings

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #47434 from guits/cv-fix-test-get-file-content

ceph-volume/tests: fix test_exception_returns_default

Merge pull request #47177 from yuvalif/wip-yuval-backport-doc-fix

doc: change "master" to "main" in backport doc

reviewed-by: anthonyeleven, ljflores, zdover23

exporter: per node metric exporter

Fixes: https://tracker.ceph.com/issues/55046
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #47428 from tchaikov/wip-crimson-strong-ordering

crimson: use std::strong_ordering mechinary

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #46863 from bryanmontalvan/dashboard-matcher

mgr/dashboard: Simplified silence-form matchers list layout

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: add cephadm e2e tests for checking count for services instances

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

crimson/os: do not use WRITE_*_OPERATORS_{3,4}()

the default-generated comparison operator is able to fulfill our needs
already.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

ceph-volume/tests: fix test_exception_returns_default

There are cases, like running tests as root user, where this test fails
because root user can always read the file content.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

doc: change "master" to "main" in backport doc

also fix the alias setting example

Signed-off-by: yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #47418 from rhcs-dashboard/token-logs-no-good

mgr/dashboard: remove token logging

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #47429 from SUSE/wip-suse-gcc11-or-greater

ceph.spec.in: set BuildRequires: gcc-c++ >= 11 for SUSE distros

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #47426 from cyx1231st/wip-seastore-silence-this

crimson: capture this explicitly to silence warnings

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

ceph.spec.in: set BuildRequires: gcc-c++ >= 11 for SUSE distros

This is much simpler than explicitly requiring gcc 11, and
anyway, openSUSE Tumbleweed is up to gcc 12 now.

Signed-off-by: Tim Serong <tserong@suse.com>

crimson: do not use WRITE_{EQ,CMP}_OPERATORS_2()

the default-generated comparison operators can fulfill our needs.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

crimson: capture this explicitly

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Kefu Chai <tchaikov@gmail.com>

mgr/dashboard: remove token logging

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/dashboard: Simplified silence-form matchers list

This commit removes unmeaning icons on the matchers-list component, and
now only displays the information/content needed when viewing and editing
matchers.

Fixes: https://tracker.ceph.com/issues/42306
Signed-off-by: Bryan Montalvan <bmontalv@redhat.com>

crimson/net: don't throw on failure when renewing tickets and keyrings

In the classical `MonClient` renewal of tickets and rotating
secrets, generally speaking, doesn't verify the `MAuthReply`.
The only difference is breaking the renewal loop:

```cpp
void MonClient::_finish_auth(int auth_err)
{
  // ...
  if (!auth_err && active_con) {
    ceph_assert(auth);
    _check_auth_tickets();
  }
  // ...
}
```

However, `crimson::mon::Client` throws `error::negotiation_failure`
which is finally handled in `Gated::dispatch()`. As the handler
there doesn't expect this particular error, it aborts the entire
process. This was the reason for failing one of the jobs at Sepia:

```
rzarzynski@teuthology:/a/rzarzynski-2022-08-01_11:01:29-crimson-rados-main-distro-default-smithi/6954468$ less ./remote/smithi174/log/ceph-osd.1.log.gz
...
DEBUG 2022-08-01 11:26:13,772 [shard 0] monc - renew_rotating_keyring secrets are up-to-date (they expire after 1659353143.772506)
INFO  2022-08-01 11:26:13,772 [shard 0] monc - renew_tickets: retrieving new tickets
INFO  2022-08-01 11:26:13,772 [shard 0] monc - sending auth(proto 2 132 bytes epoch 0) v1
INFO  2022-08-01 11:26:13,772 [shard 0] monc - waiting
INFO  2022-08-01 11:26:13,773 [shard 0] monc - handle_auth_reply mon v2:172.21.15.174:6802/101543 => v2:172.21.15.174:3300/0 returns auth_reply(proto 2 0 (0) Success) v1: 0
INFO  2022-08-01 11:26:13,773 [shard 0] monc - handle_auth_reply
INFO  2022-08-01 11:26:13,773 [shard 0] monc - renew_tickets: retrieving new tickets
INFO  2022-08-01 11:26:13,773 [shard 0] monc - sending auth(proto 2 132 bytes epoch 0) v1
DEBUG 2022-08-01 11:26:13,773 [shard 0] monc - renew_rotating_keyring secrets are up-to-date (they expire after 1659353143.7732666)
INFO  2022-08-01 11:26:13,773 [shard 0] monc - waiting
INFO  2022-08-01 11:26:13,773 [shard 0] monc - do_auth_single: mon v2:172.21.15.174:6802/101543 => v2:172.21.15.174:3300/0 returns auth_reply(proto 2 0 (0) Success) v1: 0
INFO  2022-08-01 11:26:13,773 [shard 0] monc - handle_auth_reply mon v2:172.21.15.174:6802/101543 => v2:172.21.15.174:3300/0 returns auth_reply(proto 2 0 (0) Success) v1: 0
INFO  2022-08-01 11:26:13,773 [shard 0] monc - handle_auth_reply
INFO  2022-08-01 11:26:13,773 [shard 0] monc - do_auth_single: mon v2:172.21.15.174:6802/101543 => v2:172.21.15.174:3300/0 returns auth_reply(proto 2 0 (0) Success) v1: 0
WARN  2022-08-01 11:26:13,773 [shard 0] auth - cephx client: could not verify service_ticket reply
ERROR 2022-08-01 11:26:13,773 [shard 0] monc - do_auth_single: got error -13 on mon v2:172.21.15.174:3300/0
DEBUG 2022-08-01 11:26:13,773 [shard 0] monc - renew_tickets: don't need new tickets
DEBUG 2022-08-01 11:26:13,774 [shard 0] monc - renew_rotating_keyring secrets are up-to-date (they expire after 1659353143.773988)
ERROR 2022-08-01 11:26:13,774 [shard 0] osd - mon.osd.1 dispatch() ms_dispatch caught exception: std::system_error (error crimson::net:3, negotiation failure)
```

```
2022-08-01T11:26:13.798 INFO:tasks.ceph.osd.1.smithi174.stderr:ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/giga
ntic/release/17.0.0-13947-g4f4eedf5/rpm/el8/BUILD/ceph-17.0.0-13947-g4f4eedf5/src/crimson/common/gated.h:36: crimson::common::Gated::dispatch<crimson::mon::Client::ms_dispatch(crimson::net::ConnectionRef, MessageRef)::<lambda()>&, crimson::mon::Client>(const char*, crimson::mon::Client&, crimson::mon::Client::ms_dispatch(crimson::net::ConnectionRef, MessageRef)::<lambda()>&)::<lambda(std::__exception_ptr::exception_ptr)>: Assertion `*eptr.__cxa_exception_type() == typeid(seastar::gate_closed_exception)' failed.
2022-08-01T11:26:13.811 INFO:tasks.ceph.osd.1.smithi174.stderr:Aborting on shard 0.
```

```cpp
class Gated {
  // ...
  template <typename Func, typename T>
  inline seastar::future<> dispatch(const char* what, T& who, Func&& func) {
    return seastar::with_gate(pending_dispatch, std::forward<Func>(func)
    ).handle_exception([what, &who] (std::exception_ptr eptr) {
      if (*eptr.__cxa_exception_type() == typeid(system_shutdown_exception)) {
        gated_logger().debug(
            "{}, {} skipped, system shutdown", who, what);
        return;
      }
      gated_logger().error(
          "{} dispatch() {} caught exception: {}", who, what, eptr);
      assert(*eptr.__cxa_exception_type()
        == typeid(seastar::gate_closed_exception));
    });
  }
```

See: http://pulpito.front.sepia.ceph.com/rzarzynski-2022-08-01_11:01:29-crimson-rados-main-distro-default-smithi/6954468/

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #47261 from rhcs-dashboard/fix-duplicate-service-creation

mgr/dashboard: Show error on creating service with duplicate service id

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: Show error on creating service with duplicate service id

Fixes: https://tracker.ceph.com/issues/56689
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #47398 from tchaikov/wip-crimson-capture-this

crimson: capture "this" explicitly

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson: capture "this" explicitly

to silence warnings like:

```
alien_store.cc:577:64: warning: implicit capture of ‘this’ via ‘[=]’ is
deprecated in C++20 [-Wdeprecated]
  577 |     return tp->submit(ch->get_cid().hash_to_shard(tp->size()),
      [=, &bl] {
      |                                                                ^
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #45133 from cbodley/wip-cxx20-fixes

enable c++20

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #47197 from ceph/boost-docs

doc/dev: Elaborate on boost .deb creation

Merge pull request #47262 from shriya-deshmukh/fix-delete-bucket-tagging-issue

rgw: Return correct status code in case of DeleteBucketTagging API.

Reviewed-by: Ali Maredia <amaredia@redhat.com>

mgr/dashboard: cluster > hosts: host list tables doesn't show all services deployed

Fixes: https://tracker.ceph.com/issues/53210
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Service instances was displaying only the ceph services, but with these changes it'll display instances
of cephadm services as well.

Merge pull request #47385 from rhcs-dashboard/fix-flake8

mgr: fix flake8:missing whitespace after keyword for assert() & yield()

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

mgr/cephadm: fix flake8

Signed-off-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #47334 from s0nea/wip-osd-objectstore-types-fix

monitoring/ceph-mixin: OSD overview typo fix

Reviewed-by: MrFreezeex <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/cephadm: Add loki dependency to grafana, promtail

- Add loki dependency in order to be consistent with _calc_daemon_deps()
- Add dependency for all loki daemon

Signed-off-by: jinhong.kim <jinhong.kim0@navercorp.com>

Merge pull request #45602 from soumyakoduri/wip-skoduri-dbstore-obj-versioning

rgw/dbstore: Object versioning feature support

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #47345 from cyx1231st/wip-seastore-log-improvements

crimson/os/seastore: misc fixes and log improvements

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #47280 from adk3798/remove-error-upgrade-unpause

mgr/cephadm: clear error message when resuming upgrade

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #47318 from adk3798/rescan-parser-name

cephadm: give disk rescan its own parser object

Reviewed-by: Paul Cuzner <pcuzner@redhat.com>

Merge pull request #47338 from ksperis/master

cephadm: Fix repo_gpgkey should return 2 vars

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #47361 from guits/fix-simple-scan

ceph-volume: fix simple scan

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #47367 from tchaikov/wip-osd-capture-this

osd/PG: capture this explicitly

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

osd/PG: capture variables explicitly

to silence warning if compiled with -std=c++20:

src/osd/PG.h:571:7: warning: implicit capture of ‘this’ via ‘[=]’ is
deprecated in C++20 [-Wdeprecated]
  571 |       [=](auto &history, auto &stats) {
      |       ^

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

osd/PG: capture variables explicitly

to silence warning if compiled with -std=c++20:

src/osd/PG.h:571:7: warning: implicit capture of ‘this’ via ‘[=]’ is
deprecated in C++20 [-Wdeprecated]
  571 |       [=](auto &history, auto &stats) {
      |       ^

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #47300 from kotreshhr/fips-md5-issue

mgr/volumes: Fix subvolume creation in FIPS enabled system.

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #47202 from SUSE/wip-fix-56658

cephfs-shell: move source to separate subdirectory

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #47246 from liu-chunmei/crimson-test-case

crimson/test: comment out EC test cases and some cases not supported now

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

test/common: disable tests for commutativity of operator==()

older C++ compiler like GCC-9 does not rewrite operator==() so that
`a == b` implies `b == a`, in other words:
operator==(const LHS& lhs, const RHS& rhs) is equivalent to
operator==(const RHS& rhs, const LHS& lhs). see
section 1.2 in https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0515r3.pdf

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

common/static_ptr: make free operator==() a member

since operator==() is now considered commutative in C++20, we can
make it a member function.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

debian: pin gcc-11

Signed-off-by: Casey Bodley <cbodley@redhat.com>

ceph.spec.in: add gcc-toolset-11-annobin-plugin-gcc

gcc-toolset-11-annobin is already installed, but ceph.spec.in adds
"-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" which needs the gcc
plugin too

resolves this failure during the cmake configure step:

-- Check for working CXX compiler: /opt/rh/gcc-toolset-11/root/usr/bin/c++
-- Check for working CXX compiler: /opt/rh/gcc-toolset-11/root/usr/bin/c++ - broken
CMake Error at /usr/share/cmake/Modules/CMakeTestCXXCompiler.cmake:59 (message):
  The C++ compiler

    "/opt/rh/gcc-toolset-11/root/usr/bin/c++"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11497-gfae772b8/rpm/el8/BUILD/ceph-17.0.0-11497-gfae772b8/x86_64-redhat-linux-gnu/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/gmake -f Makefile cmTC_09421/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_09421.dir/build.make CMakeFiles/cmTC_09421.dir/build
    gmake[1]: Entering directory '/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11497-gfae772b8/rpm/el8/BUILD/ceph-17.0.0-11497-gfae772b8/x86_64-redhat-linux-gnu/CMakeFiles/CMakeTmp'
    Building CXX object CMakeFiles/cmTC_09421.dir/testCXXCompiler.cxx.o
    /opt/rh/gcc-toolset-11/root/usr/bin/c++   -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection  -o CMakeFiles/cmTC_09421.dir/testCXXCompiler.cxx.o -c /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11497-gfae772b8/rpm/el8/BUILD/ceph-17.0.0-11497-gfae772b8/x86_64-redhat-linux-gnu/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
    cc1plus: fatal error: inaccessible plugin file plugin/annobin.so expanded from short plugin name annobin: No such file or directory
    compilation terminated.

Signed-off-by: Casey Bodley <cbodley@redhat.com>

install-deps: remove use of update-alternatives

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test: Fix implicit-this lambda capture warning

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Silence deprecation warnings from s3select

Since it's a submodule.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Fix implicit capture of this warnings

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

osd: Fix implicit capture of this warnings

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

common/static_ptr: drop ambiguous operator==()

because in C++20, comparison operator is considered reversed also. so
drop the reversed operators.

otherwise we'd have following FTBFS:

../src/test/common/test_static_ptr.cc:64:20: error: use of overloaded operator '==' is ambiguous (with operand types 'static_ptr<base, sizeof(grandchild)>' and 'nullptr_t')
    EXPECT_FALSE(p == nullptr);
                 ~ ^  ~~~~~~~
../src/googletest/googletest/include/gtest/gtest.h:1984:25: note: expanded from macro 'EXPECT_FALSE'
  GTEST_TEST_BOOLEAN_(!(condition), #condition, true, false, \
                        ^~~~~~~~~

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

test/common/test_allocate_unique: drop hint parameter from allocate()

the "hint" parameter was dropped from C++20.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

crimson/os/seastore: drop != operator and use friend operator==

C++20 is able to create != from == operator. so drop the != overload.
this helps to address following FTBFS:

/var/ssd/ceph/src/crimson/os/seastore/omap_manager/btree/omap_btree_node_impl.cc:141:19: error: use of overloaded operator '!=' is ambiguous (with operand types 'crimson::os::seastore::omap_manager::StringKVInnerN$
  assert(child_pt != iter_end());
         ~~~~~~~~ ^  ~~~~~~~~~~
/usr/include/assert.h:93:27: note: expanded from macro 'assert'
     (static_cast <bool> (expr)                                         \
                          ^~~~
/var/ssd/ceph/src/crimson/os/seastore/omap_manager/btree/string_kv_node_layout.h:380:10: note: candidate function
    bool operator==(const iter_t &rhs) const {
         ^

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

crimson/common: add alternative overload for FixedKVNodeLayout<>::iter_t

otherwise we'd have following FTBFS with C++20 and clang13:

In file included from /var/ssd/ceph/src/crimson/os/seastore/lba_manager.cc:7:
In file included from /var/ssd/ceph/src/crimson/os/seastore/lba_manager/btree/btree_lba_manager.h:24:
/var/ssd/ceph/src/crimson/os/seastore/lba_manager/btree/lba_btree.h:474:26: error: use of overloaded operator '==' is ambiguous (with operand types 'crimson::common::FixedKVNodeLayout<254, crimson::os::seastore::l$
    auto end = next_iter == parent->end()
               ~~~~~~~~~ ^  ~~~~~~~~~~~~~
/var/ssd/ceph/src/crimson/common/fixed_kv_node_layout.h:127:10: note: candidate function
    bool operator==(const iter_t &rhs) const {
         ^
/var/ssd/ceph/src/crimson/common/fixed_kv_node_layout.h:127:10: note: candidate function (with reversed parameter order)

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

rpm: BuildRequires gcc-toolset-10-gcc-c++ >= 10.2

Signed-off-by: Casey Bodley <cbodley@redhat.com>

crimson/test: skip EC class SetUp

otherwise crimson will hang

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

install-deps.sh: centos/rhel require gcc 10.2 and devtoolset-10

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Revert "install-deps.sh: drop devtoolset(DTS) check for centos"

This reverts commit 66e81b8b08b1a48a48c16b5b89450aeced9de5f9.

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Conflicts:
install-deps.sh for centos/rhel 8 instead of 7

cmake: raise CMAKE_CXX_STANDARD from 17 to 20

Signed-off-by: Casey Bodley <cbodley@redhat.com>