qa/suites/upgrade/pacific-x: do distro before tasks

Otherwise, we isntall new podman at the end, and the
container-selinux-policy package install triggers a bunch of selinux
errors.

Fixes: https://tracker.ceph.com/issues/50151
Signed-off-by: Sage Weil <sage@newdream.net>

qa/suites/orch/cephadm/smoke: centos 8.2 -> 8.3

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #41984 from tchaikov/wip-crimson-repeat

crimson: s/crimson::do_until/crimson::repeat/

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #41974 from rhcs-dashboard/fix-POST-alerts

mgr/dashboard: Alertmanager fails to POST alerts

crimson: s/crimson::do_until/crimson::repeat/

seastar::do_until() takes a predicate functor and a continuation. while
seastar::repeat() takes a single continuation which returns
stop_iteration::yes or stop_iteration::no. in general, we want to mirror
and extend the facilities offered by seastar instead of changing it in
an unexpected way. while crimson::do_until() only take a single
continuation which returns a bool. in hope to be more consistent, in
this change, all occurances of crimson::do_until are replaced with
crimson::repeat.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41986 from tchaikov/wip-wip-crimson-alienstore-lockless-spin-off

crimson/os/alienstore: cleanups of ThreadPool

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: replace private: section at end of class

to comply to the style guide at https://google.github.io/styleguide/cppguide.html#Declaration_Order

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: replace tab with spaces

to be consistent with the editor variables defined in the header.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: reduce the scope of lock

we just need to synchronize accesses to `stopping`, so only need to
guard the mutation of it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41956 from yuvalif/wip-yuval-fix-51293

rgw/notification: make notifications agnostic of bucket reshard

Merge pull request #41955 from yuvalif/wip-yuval-fix-51305

rgw/notification: send correct size in COPY events

crimson/common/errorator: consider return value as ready future in maybe_handle_error_t

this behavior mirrors seasetar::futurize::apply(), where non-future and
non-void return values are converted to future<>, and returned instead.
this change could simplify some use cases where we always return an
immediately available future.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41831 from yuvalif/wip-yuval-fix-51114

rgw/amqp: remove the explicit "disconnect()" interface

Merge pull request #41902 from idryomov/wip-rbd-thick-conf

rbd: don't disable rbd_discard_on_zeroed_write_same

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #41940 from liu-chunmei/crimson-fix-reactor-stall

crimson: fix reactor stalled and rbd_open failed.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #41964 into master

* refs/pull/41964/head:
doc/cephfs/nfs: update recommendation for versions

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41908 into master

* refs/pull/41908/head:
qa: always format the pgid in hex

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41875 into master

* refs/pull/41875/head:
client: check if a mds rank is `up` before fetching connection addr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41385 into master

* refs/pull/41385/head:
mon/FSCommands: add command to rename a file system
qa/cephfs: split test_admin.TestAdminCommands
mds: remove 'fs_name' from MDSRank

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/seastore: fix reactor stalled and rbd_open failed

omap manger use keys' reference, so the keys' string set should
be kept alive during omap operation. use value capture instead of
reference capture.

otherwise, during omap operation, the keys contents will be changed
and cause logger insert_iterator very long time to stall reactor.
omap_get_value also failed, rbd_open failed.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

Merge pull request #41967 from sandrobonazzola/patch-1

doc/install/get-packages: point to current stable release

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/install/get-packages: point to current stable release

Point to pacific for downloading the cephadm script

Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Signed-off-by: Sandro Bonazzola <sbonazzo@redhat.com>

Merge pull request #41960 from rzarzynski/wip-crimson-alienstore-sync-shardedwq

crimson/os: synchronize producers with consumers in AlienStore's queues.

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Alertmanager fails to POST alerts

Fixes: https://tracker.ceph.com/issues/51312
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #41957 from liewegas/fix-vstart-registry-url

vstart.sh: fix docker url

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41961 from rzarzynski/wip-crimson-intcltreq-fix-assert

crimson/osd: fix construction of InternalClientRequest in DEBUG builds.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41962 from rzarzynski/wip-crimson-intcltreq-more-asserts

crimson/osd: introduce more asserts to the Watch timeout handling.

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/cephfs/nfs: update recommendation for versions

... of Ceph and NFS-Ganesha packages.

Signed-off-by: Ramana Raja <rraja@redhat.com>

crimson/osd: introduce more asserts to the Watch timeout handling.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #41860 into master

* refs/pull/41860/head:
qa: log messages when falling back to force/lazy umount

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

crimson/osd: fix construction of InternalClientRequest in DEBUG builds.

The assert in the ctor of `InternalClientRequest` actually operates on
the ctor's argument we `std::moved` from, not on the class' member.
When a debug build is used, this translates into failures like the one
below:

```
2021-06-16T22:53:03.410 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:02 smithi170 conmon[43770]: ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-
build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4987-gec8844b6/rpm/el8/BUILD/ceph-17.0.0-4987-gec8844b6
/src/crimson/osd/osd_operations/internal_client_request.cc:19: crimson::osd::InternalClientRequest::InternalClientRequest(Ref<crimson::osd::PG>): Assertion `bool(pg)' f
ailed.
2021-06-16T22:53:05.363 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  0# 0x0000558BE7BBF68F in /usr/bin/ceph-osd
2021-06-16T22:53:05.363 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  1# FatalSignal::signaled(int, siginfo_t const*) in /usr/bi
n/ceph-osd
2021-06-16T22:53:05.363 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  2# FatalSignal::install_oneshot_signal_handler<6>()::{lamb
da(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in /usr/bin/ceph-osd
2021-06-16T22:53:05.364 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  3# 0x00007F8AD7535B20 in /lib64/libpthread.so.0
2021-06-16T22:53:05.364 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  4# gsignal in /lib64/libc.so.6
2021-06-16T22:53:05.364 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  5# abort in /lib64/libc.so.6
2021-06-16T22:53:05.364 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  6# 0x00007F8AD5B2EC89 in /lib64/libc.so.6
2021-06-16T22:53:05.365 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  7# 0x00007F8AD5B3CA76 in /lib64/libc.so.6
2021-06-16T22:53:05.365 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  8# crimson::osd::InternalClientRequest::InternalClientRequ
est(boost::intrusive_ptr<crimson::osd::PG>) in /usr/bin/ceph-osd
2021-06-16T22:53:05.365 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]:  9# crimson::osd::Watch::do_watch_timeout(boost::intrusive_ptr<crimson::osd::PG>) in /usr/bin/ceph-osd
2021-06-16T22:53:05.365 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 10# seastar::noncopyable_function<void ()>::direct_vtable_for<crimson::osd::Watch::Watch(crimson::osd::Watch::private_ctag_t, boost::intrusive_ptr<crimson::osd::ObjectContext>, watch_info_t const&, entity_name_t const&, boost::intrusive_ptr<crimson::osd::PG>)::{lambda()#1}>::call(seastar::noncopyable_function<void ()> const*) in /usr/bin/ceph-osd
2021-06-16T22:53:05.366 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 11# 0x0000558BED653759 in /usr/bin/ceph-osd
2021-06-16T22:53:05.366 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 12# 0x0000558BED61B148 in /usr/bin/ceph-osd
2021-06-16T22:53:05.366 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 13# 0x0000558BED61B576 in /usr/bin/ceph-osd
2021-06-16T22:53:05.366 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 14# 0x0000558BED7C93C9 in /usr/bin/ceph-osd
2021-06-16T22:53:05.367 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 15# 0x0000558BED326D5A in /usr/bin/ceph-osd
2021-06-16T22:53:05.367 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 16# 0x0000558BED330E7E in /usr/bin/ceph-osd
2021-06-16T22:53:05.367 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 17# main in /usr/bin/ceph-osd
2021-06-16T22:53:05.367 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 18# __libc_start_main in /lib64/libc.so.6
2021-06-16T22:53:05.368 INFO:journalctl@ceph.osd.6.smithi170.stdout:Jun 16 22:53:05 smithi170 conmon[43770]: 19# _start in /usr/bin/ceph-osd
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: synchronize producers with consumers in AlienStore's queues.

Some time ago we replaced the single, `boost::lockfree`-based queue
in `ThreadPool` with the in-house, lockish `ShardedWorkQueue` vector.
Unfortunately, pushing into such queue isn't synchronized with
consuming from it -- the former happens without locking the `mutex`.
As the underlying primitive behind `ShardedWorkQueue::pending` is
plain `std::deque`, it's unsafe to operate that way in multi-thread
environment. Indeed, weirdly looking crashes have been spotted at Sepia:

```
(virtualenv) rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-06-21_14:49:36-rados-master-distro-basic-smithi/6182668$ less ./remote/smithi196/log/ceph-osd.7.log.gz
...
 0# 0x000055862FD67ADF in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007FB22CF36B20 in /lib64/libpthread.so.0
 4# 0x00005586357540E4 in ceph-osd
 5# 0x00007FB22CF36B20 in /lib64/libpthread.so.0
 6# pthread_cond_timedwait in /lib64/libpthread.so.0
 7# crimson::os::ThreadPool::loop(std::chrono::duration<long, std::ratio<1l, 1000l> >, unsigned long) in ceph-osd
 8# 0x00005586313E303B in ceph-osd
 9# 0x00007FB22CC51BA3 in /lib64/libstdc++.so.6
10# 0x00007FB22CF2C14A in /lib64/libpthread.so.0
11# clone in /lib64/libc.so.6
Fault at location: 0x18
daemon-helper: command crashed with signal 11
```

This fix introduces the synchronization to the `push_back()` method of
`ShardedWorkQueue`. The side effect is that it may stall the reactor.
Therefore, a follow-up change that switches to e.g. `boost::lockfree`
is expected.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

vstart.sh: fix docker url

Signed-off-by: Sage Weil <sage@newdream.net>

rgw/notification: make notifications agnostic of bucket reshard

Fixes: https://tracker.ceph.com/issues/51293
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

rgw/notification: send correct size in COPY events

Fixes: https://tracker.ceph.com/issues/51305
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #41941 from tchaikov/wip-crimson-errorator-loop

crimson/common: extract parallel_for_each_state out

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #41949 from tchaikov/wip-crimson-prometheus

crimson/osd: expose metrics using http server

Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/osd: expose metrics using http server

so, we can query the metrics using HTTP API, like

http://localhost:9180/metrics?name=io*

or

http://192.168.2.8:9180/metrics?name=io_queue_delay

or

http://localhost:9180/metrics

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41934 from cyx1231st/wip-seastore-onode-logs

crimson/onode-staged-tree: improve logs to understand inconsistent load from seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/onode-staged-tree: print NodeExtent with the header

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/onode-staged-tree: validate node header when load

Add logs to detect corruptions when load nodes. assert() is not
informative enough to understand the context.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/onode-staged-tree: delete copy constructor of DummyNodeExtent

Dummy backend is used for unit tests without transactions, so there
should be no copy-on-write behavior.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/onode-staged-tree: add trace logs when start to load nodes

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #41861 from AmnonHanuhov/wip-Refactor_crimson_internals

crimson/net: Complete the refactor to std::unique_ptr inside Messenger

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41921 from gregsfortytwo/wip-mon-stretch-crush-rule

mon: Sanely set the default CRUSH rule when creating pools in stretch…

Reviewed-by: Samuel Just <sjust@redhat.com>

tools/crimson: Use crimson::make_message() in perf_crimson_msgr

Instead of ceph::make_message() because conn::send() in crimson expects
a std::unique_ptr and not boost::intrusive_ptr

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

Merge pull request #41845 from agayev/zoned-revise-per-zone-naming-scheme

os/bluestore: Revise the naming scheme for per-zone cleaning informat…

Reviewed-by: Igor Fedotov <ifedotov@suse,com>

test/crimson: Use crimson::make_message() in test_alien_echo

Instead of ceph::make_message() because conn::send() in crimson expects
a std::unique_ptr and not boost::intrusive_ptr

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

Merge pull request #41830 from tchaikov/wip-ceph-argparse-cleanup

pybind/ceph_argparse: cleanups preparing for type annotations

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

crimson/net: Use MessageURef in messenger internals

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

crimson/osd: Get rid of send_to_osd() overloading

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

osd: Overload send_osd_message() in PeeringState

To allow passing MessageURef from crimson-osd and MessageRef from
ceph-osd

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

crimson/osd: Move message to send_to_osd() in ShardServices

To avoid refcounting the underlying RefCountedObject

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

Merge pull request #41923 from liewegas/fix-51234

ceph_test_librados_service: wait longer for servicemap to update

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41914 from lxbsz/wip-51092

os/memstore: make the used_bytes to atomic

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41896 from ifed01/wip-ifed-verbose-kernel-read

blk/KernelDevice: be more verbose on read errors.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/common: specialize errorator<> for future<>

otherwise it always needs a return value.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: extract parallel_for_each_state out

if `parallel_for_each_state` is defined as a nested class in errorator,
clang fails to compile it:

../src/crimson/common/errorator.h:716:47: error: no class named 'parallel_for_each_state' in 'errorator<AllowedErrors...>'
    friend class errorator<AllowedErrors...>::parallel_for_each_state;
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

and the forward declaration does not help. so we have to extract it
out of the errorator. to speed up the compilation, it is moved into
errorator-loop.h. its name mirrors `include/seastar/core/loop.h`.

we could extract the `errorator<>::parallel_for_each()` out as well,
as its return type can be deduced from the type of Iterator and Func.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41920 from ljflores/patch-1

doc: fixed a small typo in Perf Counters documentation

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41925 from tchaikov/wip-fmtlib

fmt: pickup fix of link failure with clang

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge PR #41900 into master

* refs/pull/41900/head:
qa: use centos latest for fs:upgrade

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #41899 into master

* refs/pull/41899/head:
mon/MDSMonitor: check fscid exists for legacy case

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #41898 into master

* refs/pull/41898/head:
mon/MDSMonitor: fix whitespace in debug message

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #41892 into master

* refs/pull/41892/head:
client: remove unused include from barrier.cc

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41833 into master

* refs/pull/41833/head:
cephfs-mirror: silence warnings when connecting via mon host

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41723 into master

* refs/pull/41723/head:
mds: to print the unknow type value

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #40997 into master

* refs/pull/40997/head:
test: add test to verify adding an active peer back to source
pybind/mirroring: disallow adding a active peer back to source
pybind/cephfs: interface to fetch file system id

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36823 into master

* refs/pull/36823/head:
qa : add a test for the cmd, dump cache
mds : add timeout to the command, dump cache, to prevent it from running too long and affecting the service

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc: fixed a small typo in Perf Counters documentation

There is a small typo in the Perf Counters documentation. Gauge was spelled incorrectly.

Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #40506 from p-se/pse-update-grafana-deprecated-variables

mgr/dashboard: deprecated variable usage in Grafana dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: p-se <NOT@FOUND>

Merge pull request #41808 from rhcs-dashboard/51164-show-only-days-in-bucket-details

mgr/dashboard: bucket details: show lock retention period only in days

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41758 from rhcs-dashboard/support-multiple-crush-trees

mgr/dashboard: crushmap tree doesn't display crush type other than root

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #35903 from agayev/fix-deployment-guide

doc: Add a missing instruction to manual deployment guide.

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: Add a missing instruction to manual deployment guide.

Following the instructions as is results in the following error at step 15:

$ sudo -u ceph ceph-mon --mkfs -i node1 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring
global_init: error reading config file.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>

ceph.spec.in: bump up the required version of fmt-devel to 6.2.1

6.2.1 is the version packaged by EPEL8, in other words, this is the
version we've been testing. so to be more consistent with the
known-to-be-good version, let's bump up the required version.

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian/control: add libfmt-dev for "make check"

so, on debian derivatives, we can use the libfmt-dev package for
building Ceph. this change is created in hope to reduce the compile
time.

>= 6.1.2 is specified, as it is the version packaged by ubuntu focal,
which is used for running "make check" and intergration tests.

find_package(fmt 6.0.0 QUIET)

Signed-off-by: Kefu Chai <kchai@redhat.com>

fmt: pickup fix of link failure with clang

fmtlib v7.1.3 contains the fix of https://github.com/fmtlib/fmt/issues/1753

so let's bump up the submodule to the latest master HEAD of fmtlib
for more fixes.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41911 from tchaikov/wip-crimson-nbd-cleanup

crimson/tools/store_nbd: better cleanup

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/tools/store_nbd: replace wait_pending with seastar::gate

the inc_pending + promise<> solution is pratically identical to
seastar::gate, so let's use the prepackaged solution instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/tools/store_nbd: drop unnecessary seastar::now()

the body of handle_exception() is a synchronous operation, there is no
need to return seastar::now() here.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/tools/store_nbd: better cleanup

* remove unix domain socket file when cleanup
  so we don't need to remove it manually after each run.
* shutdown input and output streams when cleanup
  so reactor does not watch them anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/tools/store_nbd: s/socket/server_socket/

to prepare for the next commit, which will keep track of the
connected_socket as another member variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41916 from AmnonHanuhov/wip-Refactor_test_messenger

test/crimson: Use crimson's make_message in test_messenger

ceph_test_librados_service: wait longer for servicemap to update

mon thrashing may make this take a long time

Fixes: https://tracker.ceph.com/issues/51234
Signed-off-by: Sage Weil <sage@newdream.net>

mon: Sanely set the default CRUSH rule when creating pools in stretch mode

If we get a pool create request while in stretch mode that does not explicitly
specify a crush rule, look at the stretch-mode pools and their rules, and
select the most common one.

Also update set_up_stretch_mode.sh to add a few more rules that let me test
this locally.

Fixes: https://tracker.ceph.com/issues/51270
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

test/crimson: Use crimson's make_message in test_messenger

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>

Merge pull request #41905 from ivancich/wip-improve-cls-rgw-tracing

rgw: clean-up logging of function entering to make thorough and consistent

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

os/bluestore: Revise the naming scheme for per-zone cleaning information.

Use a single letter (G) for the namespace, and use zone_num+oid as the key.

Signed-off-by: Abutalib Aghayev <agayev@psu.edu>

mon/FSCommands: add command to rename a file system

The fs_name of the relevant MDSMap is set to the new name. Also,
the application tags of the data pools and the meta data pool of
the file system is set to the new name.

Fixes: https://tracker.ceph.com/issues/47276
Signed-off-by: Ramana Raja <rraja@redhat.com>

qa/cephfs: split test_admin.TestAdminCommands

Splitting TestAdminCommands makes it easier to exercise a particular
"ceph fs" subcommand. Also, rename class TestSubCmdFsAuthorize to
TestFsAuthorize.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

rgw: clean-up logging of function entering to make thorough and consistent

This provides more thorough and consistent function tracing in CLS/RGW
when logging is set to 10 or higher.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

mds: remove 'fs_name' from MDSRank

There isn't a need to store a file system's name in a MDSRank
object. The MDSRank has a pointer to a MDSMap object, which already
stores the name.

Also, there isn't a need to pass the file system name in the MMDSMap
message. It should be sufficient to pass the MDSMap in the MMDSMap
message as the the file system name is stored in the MDSMap. Pass a
empty string as map_fs_name in the MMDSMap message. It is  simpler than
removing map_fs_name from the message payload altogether.

Fixes: https://tracker.ceph.com/issues/50852
Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge pull request #41903 from liewegas/update-rook-client

rook-client-python: update to update-june-21

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #41835 from TRYTOBE8TME/wip-rgw-keycloak-failure-fix

qa/tasks: Keycloak failure fix

Reviewed-by: Ali Maredia <amaredia@redhat.com>

mgr/rook: comment out osd creation functions

This commit comments out the OSD creation functions in rook_cluster.py
and module.py, since the submodule update has broken them.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>

mgr/rook: fix some mypy typing errors in rook_cluster.py

This commit fixes some errors caught by mypy in rook_cluster.py. Most of the
errors were caused by the update of the rook-client-python submodule in a previous
commit.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>

mgr/rook: pass zone attribute to CephObjectStore CR when creating rgw

This commit passes the zone attribute to the CephObjectStore CR when
creating a RGW instance using the Rook Orchestrator backend:
`ceph orch apply rgw <rgw-name> --realm=<realm-name> --zone=<zone-name>`

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>

os/memstore: make the used_bytes to atomic

Fixes: https://tracker.ceph.com/issues/51092
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #41856 from rhcs-dashboard/maintenance-bug-fix

mgr/dashboard: Fix 500 error while exiting out of maintenance

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>

Merge pull request #41694 from jmolmo/kcli_cephadm_doc

doc: Add kcli utilization for development environments

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>