crimson/os/seastore: do not capture unused variable

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: do not capture unused variable

this change silences the warning from clang, like:

 ../src/crimson/common/interruptible_future.h:276:8: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
       [this, func=std::move(func),
        ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore: mark final class "final"

this change silences warning like:

 ../src/crimson/os/seastore/onode_manager/staged-fltree/fltree_onode_manager.h:80:18: warning: class with destructor marked 'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
   ~FLTreeOnode() final {}
                  ^
 ../src/crimson/os/seastore/onode_manager/staged-fltree/fltree_onode_manager.h:13:8: note: mark 'crimson::os::seastore::onode::FLTreeOnode' as 'final' to silence this warning
 struct FLTreeOnode : Onode, Value {
        ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/crimson/seastore: do not put braces around scalar initializer

this change silences the warning from clang like:

src/test/crimson/seastore/onode_tree/test_value.h:152:12: warning: braces around scalar initializer [-Wbraced-scalar-init]
     return {static_cast<value_size_t>(size - sizeof(value_header_t))};
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore: do not reference non-template with template

this change addresses the build failure with clang:

src/crimson/os/seastore/onode_manager/staged-fltree/node_extent_accessor.h:352:29: error: 'encode_split' following the 'template' keyword does not refer to a template
    test_recorder->template encode_split(split_at, read().p_start());
                   ~~~~~~~~ ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore: do not capture label using &

structured binding does not define variables. unlike GCC, Clang does
not allow this. this change address following build failures with clang:

../src/crimson/os/seastore/omap_manager/btree/omap_btree_node_impl.cc:210:24:
note: 'result' declared here
      auto &[complete, result] = ret;
                       ^
../src/crimson/os/seastore/omap_manager/btree/omap_btree_node_impl.cc:214:6:
error: reference to local binding 'complete' declared in enclosing
lambda expression
            complete = biter == eiter;
            ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41046 from rzarzynski/wip-crimson-monc-auth_reply

crimson/monc: fix use-after-free around Connection::do_auth_single().

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40936 from rhcs-dashboard/support-ingress-service-creation-UI

mgr/dashboard: fix HAProxy (now called ingress)

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40426 from rhcs-dashboard/host-maintenance-followups

mgr/dashboard: Host Maintenance Follow ups

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

crimson/monc: fix use-after-free around Connection::do_auth_single().

This bug caused following failure in teuthology testing [1]:

```
WARN  2021-04-23 07:08:29,449 [shard 0] osd - ms_handle_reset
WARN  2021-04-23 07:08:29,449 [shard 0] monc - active conn reset v2:172.21.15.100:3300/0
INFO  2021-04-23 07:08:29,449 [shard 0] monc - reopen_session to mon.-1
INFO  2021-04-23 07:08:29,449 [shard 0] monc - close
INFO  2021-04-23 07:08:29,449 [shard 0] monc - connecting to mon.0
...
ERROR 2021-04-23 07:08:29,450 [shard 0] none - /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-3454-gddedc2e1/rpm/el8/BUILD/ceph-17.0.0-3454-gddedc2e1/src/crimson/mon/MonClient.cc:228 : In function 'crimson::mon::Connection::do_auth_single(crimson::mon::Connection::request_t)::<lambda(Ref<MAuthReply>)>', ceph_assert(%s)
closed
```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-04-22_00:20:19-rados-master-distro-basic-smithi/6063316/

The root cause is in freeing the `active_con` in `Client::reopen_session()`
while there still could be the second, ongoing part of `do_auth_single()`.
This fix rectifies the issue by switching `std::unique_ptr` to `seastar::shared_ptr`
and extending the life-time with the help of `seastar::enable_shared_from_this<T>`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #40810 into master

* refs/pull/40810/head:
test: tests for *at() libcephfs APIs
test: cleanup files/dirs before finishing test
client: non-at APIs call at-APIs with CEPHFS_AT_FDCWD as file descriptor
libcephfs: introduce basic *at() calls
client: remove redundant caps_issued_mask() call in Client::fstatx()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/monc: assert Connection::handle_auth_reply() is truly provided with a msg.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/monc: rename Connection::reply to auth_reply.

Easier to distinguish from local variables with the same name
we have a few in `MonClient.cc`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #40157 from leseb/do-not-fail-when-nosnap

mgr/pybind/snap_schedule: do not fail when no fs snapshots are available

Reviewed-by: Venky Shankar <vshankar@redhat.com>

mgr/dashboard: Host Maintenance Follow ups

Added the ability to create a host in maintenance mode

Fixes: https://tracker.ceph.com/issues/49998
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41035 from ktdreyer/rbd-cache-err-spelling

librbd: fix spelling in immutable-object-cache conn error

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #41037 from tchaikov/wip-npm-audit

mgr/dashboard: upgrade package-lock.json

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40933 from vshankar/wip-50266

cephfs-mirror: sanitize `daemons status` JSON

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #41013 from tchaikov/wip-doc-mds-confval

doc/cephfs/mds-config-ref: render options using confval directive

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #41022 from tchaikov/wip-cmake-findpmem

cmake/modules/Findpmem: always set pmem_VERSION_STRING

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

mgr/dashboard: upgrade package-lock.json

diminish "npm run fix:audit" divergences

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #41010 from rzarzynski/wip-crimson-backtrace-fix-assert

crimson/common: fix broken assertion on FatalSignal setup.

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: fix spelling in immutable-object-cache conn error

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #41030 from zdover23/wip-doc-adding-one-word-to-cephadm-nfs-2021-Apr-26

doc/cephadm: add a single word

Reviewed-by: Sage Weil <sweil@redhat.com>

Merge pull request #38368 from rishabh-d-dave/cv-list-by-osd-id

ceph-volume: allow listing devices by OSD ID

mgr/dashboard: fix HAProxy (now called ingress)

Fixes: https://tracker.ceph.com/issues/50319
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Support from Dashboard UI to create Ingress service type.

Merge pull request #40992 from zdover23/wip-doc-cephadm-osd-advanced-adding-a-word-2021-apr-22

doc/cephadm: adding "device" to a sentence

Reviewed-by: Sebastian Wagner <swagner@suse.com>

doc/cephadm: add a single word

s/If the nfs-ganesha doesn't/If the
nfs-ganesha pool doesn't/

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #40724 from rhcs-dashboard/upgrade-caniuse-lite

mgr/dashboard: upgrade Browserslist: caniuse-lite

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #41029 from zdover23/wip-doc-cephadm-nfs-service-corrections-2021-Apr-26

doc/cephadm: rewrite "nfs.rst"

Reviewed-by: Sage Weil <sweil@redhat.com>

Merge pull request #40594 from mkogan1/wip-rgw-multipart-complete

rgw: return OK on consecutive complete-multipart reqs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

test: tests for *at() libcephfs APIs

Fixes: http://tracker.ceph.com/issues/50298
Signed-off-by: Venky Shankar <vshankar@redhat.com>

test: cleanup files/dirs before finishing test

Other tests might use the same file names and expect libcephfs
calls to succeed.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

client: non-at APIs call at-APIs with CEPHFS_AT_FDCWD as file descriptor

Signed-off-by: Venky Shankar <vshankar@redhat.com>

libcephfs: introduce basic *at() calls

Signed-off-by: Venky Shankar <vshankar@redhat.com>

client: remove redundant caps_issued_mask() call in Client::fstatx()

No need to include this check since _getattr() checks this anyway.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

doc/cephadm: rewrite "nfs.rst"

This PR makes a few (very small) stylistic
corrections, mostly typographical, to the text.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/dev/osd_internals/mclock_wpq_cmp_study: fix a typo

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/cephfs/mds-config-ref: render options using confval directive

less repeating, better maintanability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/operations/pools: define pool settings using "describe"

better readability and more structured this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration: reference options using :confval:

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake/modules/Findpmem: always set pmem_VERSION_STRING

before this change, `pmem_VERSION_STRING` is not set if it is not able
to fulfill the specified version requirement. the intention was to check
if the version is able to satisfy the requirement. but actually, passing
an empty `pmem_VERSION_STRING` to `find_package_handle_standard_args()`
as the option of `VERSION_VAR` does not fail this check. on the
contrary, it prints

-- Found pmem: pmem_pmemobj_INCLUDE_DIR;pmem_pmem_INCLUDE_DIR (Required
is at least version "1.17")

if we requires pmem 1.17, while the found version is, for instance,
1.10.

if the required version is 1.7, and the found version is 1.10, the
output from cmake is:

-- Found pmem: pmem_pmemobj_INCLUDE_DIR;pmem_pmem_INCLUDE_DIR (found
suitable version "1.10", minimum required is "1.7")

in this change, the version spec is not specified when calling
`pkg_check_modules()`. so, `PKG_${component}_VERSION` is always set.
and we can always delegate the version checking to
`find_package_handle_standard_args()`. please note, we use the lower
version returned by pkg-config if multiple components are required and
both pkg-config settings return their versions.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.spec.in: do not build with system pmdk by default

we need to use libpmem 1.10 in #40493.

without enabling the module stream offering libpmem 1.9.2, we can only
have access to libpmem 1.6.1. and fedora 33 only has libpmem 1.9
packaged. the same applies to openSUSE Tumbleweed and openSUSE Leap. so
let's stop using libpmem packaged by distro by default, until these
distros include libpmem 1.10.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39929 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/elliptic-6.5.4

build(deps): bump elliptic from 6.5.3 to 6.5.4 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/pybind/snap_schedule: return valid json for 'status' command

The status should return a valid list, prior to this path the code was
only appending json blob next to each other resulting in an invalid
json. Now the blobs will be enclosed properly so a valid list is
returned.

Signed-off-by: Sébastien Han <seb@redhat.com>

mgr/pybind/snap_schedule: do not fail when no fs snapshot schedules are available

When listing for available snapshot schedules, we should not an error in case
there is none. We should just return 0 with an empty dict.

Fixes: https://tracker.ceph.com/issues/49837
Signed-off-by: Sébastien Han <seb@redhat.com>

Merge pull request #41015 from inspur-wyq/patch-doc

doc/radosgw/role.rst: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39605 from inspur-wyq/patch-1

doc/radosgw/cloud-sync-module: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/radosgw/role.rst: fix typos

Signed-off-by: wangyunqing <wangyunqing@inspur.com>

Merge pull request #41011 from tchaikov/wip-cephadm-test

mgr/cephadm: s/_hosts_with_daemon_inventory/_schedulable_hosts/

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>

mgr/cephadm: s/_hosts_with_daemon_inventory/_schedulable_hosts/

`_hosts_with_daemon_inventory` was renamed to `_schedulable_hosts`
in f260c48f723641a5f82c64c0e31451199aa191f8 which was merged before
eebb842d0487660c93baf9eafda28a2f87e482f3 got merged, hence the
regression.

this change just updates the test with the new method name.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: fix broken assertion on FatalSignal setup.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #34109 from CongMinYin/optimizing-bluestore-lock

os/bluestore: Optimizing the lock of bluestore writing process

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40863 from adk3798/maint-fix

mgr/cephadm: don't remove daemons on hosts in maintenance or offline mode

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #38641 from agayev/zoned-cleaner

os/bluestore: Some more plumbing for zone cleaning (WIP)

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #40978 from idryomov/wip-reset-authenticate-err

mon/MonClient: reset authenticate_err in _reopen_session()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #41006 from yuriw/wip-yuriw-simlinks-master

qa/tests: changed simlink to upgrade/parallel only

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40994 from neha-ojha/wip-50388

qa/suites/upgrade/pacific-x/rgw-multisite: fix install version

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/tests: changed simlink to upgrade/parallel only

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #40986 from rhcs-dashboard/50484-update-run-api-tests

mgr/dashboard: set required env. variables in run-backend-api-tests.sh

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #40664 from xxhdx1985126/wip-seastar-missing-apis

crimson/os/seastore: add set/get attrs and write/read meta methods for seastore

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40991 from rzarzynski/wip-crimson-backtrace-si_addr

crimson/common: print the address that caused the fault on SIGSEGV.

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore: limit the max size of objects' ns and oid

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

test/crimson/seastore: add set/get attrs test

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

test/crimson/seastore: add write/read_meta tests

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/os/seastore: add write/read_meta methods for seastore

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

qa/suites/upgrade/pacific-x/rgw-multisite: fix install version

Before:

2021-04-22T06:20:58.260 INFO:teuthology.task.install:Ceph rpm upgrade from
16.2.1-103.gb54cefd0.el8 to 16.2.1-103.gb54cefd0

After:

2021-04-22T21:08:16.565 INFO:teuthology.task.install:Ceph rpm upgrade from
16.2.1-105.g540c2494.el8 to 17.0.0-3515.ga5b16446

Fixes: https://tracker.ceph.com/issues/50388
Signed-off-by: Neha Ojha <nojha@redhat.com>

doc/cephadm: adding "device" to a sentence

This tiny PR fixes an issue that I thought
I had squashed into PR#40914: a missing
word, "device", which was caught by Josh
Durgin during review.

Alas. I had not then caught it.

Here then it is, corrected at last.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

rgw: return OK on consecutive complete-multipart reqs

Fixes: https://tracker.ceph.com/issues/50141
Signed-off-by: Mark Kogan <mkogan@redhat.com>
fixup! rgw: return OK on consecutive complete-multipart reqs

crimson/common: print the address that caused the fault on SIGSEGV.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #40526 into master

* refs/pull/40526/head:
spec: add nfs to spec file
mgr/nfs: Don't enable nfs module by default
mgr/nfs: check for invalid chars in cluster id
mgr/nfs: Use CLICommand wrapper
mgr/nfs: reorg nfs files
mgr/nfs: Check if transport or protocol are list instance
mgr/nfs: reorg cluster class and common helper methods
mgr/nfs: move common export helper methods to ExportMgr class
mgr/nfs: move validate methods into new ValidateExport class
mgr/nfs: add custom exception module
mgr/nfs: create new module for export utils
mgr/nfs: rename fs dir to export
mgr/volumes/nfs: Move nfs code out of volumes plugin

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mon/MonClient: reset authenticate_err in _reopen_session()

Otherwise, if "mon host" list has at least one unqualified IP address
without a port and both msgr1 and msgr2 are turned on, there is a race
affecting MonClient::authenticate().

For backwards compatibility reasons such an address is expanded into
two entries, each being treated as a separate monitor.  For example,
"mon host = 1.2.3.4" generates the following initial monmap:

  0: v1:1.2.3.4:6789/0
  1: v2:1.2.3.4:3300/0

See MonMap::_add_ambiguous_addr() for details.

Then, the following can happen:

1. we connect to both endpoints and attempt to authenticate
2. authenticate() sets authenticate_err to 1 and sleeps on auth_cond
3. msgr1 authenticates first (i.e. it gets the final MAuth message
   before msgr2 gets the monmap)
4. active_con is set to msgr1 connection, msgr2 connection is closed
   as redundant
5. _finish_auth() sets authenticate_err to 0 and signals auth_cond,
   but before either the monmap is received or authenticate() wakes
   up, msgr1 connection is closed due to a network hiccup
6. ms_handle_reset() calls _reopen_session() which clears active_con
   and again connects to both endpoints and attempts to authenticate
7. authenticate() wakes up, sees that there is no active_con and goes
   back to sleep, but this time with authenticate_err == 0
8. msgr2 authenticates first but doesn't call _finish_auth() because
   it is called only if authenticate_err == 1
9. active_con is set to msgr2 connection, msgr1 connection is closed
   as redundant
10. authenticate() hangs on auth_cond until timeout defaulting to 5
    minutes

The discrepancy between msgr1 and msgr2 plays a key role.  For msgr1,
authentication is considered to be complete as soon as the final MAuth
message is received -- the monmap is not waited for.  For msgr2,
authentication is considered to be complete only after the monmap is
received.

Avoid the race by setting authenticate_err to 1 in _reopen_session(),
so that _finish_auth() is called on/after every authentication attempt
instead of just the first one.

Fixes: https://tracker.ceph.com/issues/50477
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

crimson/common: fix the messed labels for backtrace printing.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/common: rename print_with_backtrace in fatal_signal.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #40940 from rhcs-dashboard/50440-fix-nfs-duplicated-rows

mgr/dashboard: fix duplicated rows when creating NFS export.

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

crimson/common: drop a non-existing method from the FatalSignal's declaration.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: set required env. variables in run-backend-api-tests.sh

Fixes: https://tracker.ceph.com/issues/50484
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #40979 from leseb/raw-list-fix-dup

ceph-volume: fix raw listing when finding OSDs from different clusters

Merge PR #40913 into master

* refs/pull/40913/head:
qa/tasks/cephadm: fix ctx archive check for teuthology

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #40912 from liewegas/fix-bug-49237

msgr/async: fix unsafe access in unregister_conn()

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40916 from trociny/wip-50395

os/FileStore: don't propagate split/merge error to "create"/"remove"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40900 from myoungwon/wip-fix-50352

test: extend retry timeout from 150s to 300s

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #40944 from Daniel-Pivonka/cephadmhostlabelsorder

python-common: use OrderedDict instead of Set to remove duplicates from host labels list

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sweil@redhat.com>

mgr/dashboard: update documentation about creating NFS export.

- You learn first that orchestrator-managed clusters are detected automatically (therefore the documentation that follows is exclusively for user-defined clusters).
- Include nfs-ganesha in the security scope list.

Fixes: https://tracker.ceph.com/issues/50440
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

mgr/dashboard: fix duplicated rows when creating NFS export.

- Show an error message if the same pool & namespace is used by more than 1 cluster.
- Fix error handling when no rgw daemons found.

Fixes: https://tracker.ceph.com/issues/50440
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

ceph-volume: fix raw listing when finding OSDs from different clusters

When listing OSDs on host with 2 OSDs with the same ID, the output gets
overwritten with the last listed device. So a single OSD will show up.
See the ceph-volume.log which correctly parsed both disks:

```
[2021-04-22 09:44:21,391][ceph_volume.devices.raw.list][DEBUG ] Examining /dev/sda1
[2021-04-22 09:44:21,391][ceph_volume.process][INFO  ] Running command: /usr/bin/ceph-bluestore-tool show-label --dev /dev/sda1
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout {
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "/dev/sda1": {
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "osd_uuid": "423bf64d-f241-4f4b-a589-25a66fc836d1",
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "size": 6442450944,
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "btime": "2021-04-22T09:32:55.894961+0000",
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "description": "main",
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "bfm_blocks": "1572864",
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "bfm_blocks_per_key": "128",
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "bfm_bytes_per_block": "4096",
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "bfm_size": "6442450944",
[2021-04-22 09:44:21,418][ceph_volume.process][INFO  ] stdout "bluefs": "1",
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout "ceph_fsid": "d3cd4b72-5342-4fd3-96ec-a6e581261eab",
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout "kv_backend": "rocksdb",
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout "magic": "ceph osd volume v026",
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout "mkfs_done": "yes",
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout "osd_key": "AQDGQoFg+XHqJBAAw9ZQmtrnotHCLI0Nc2to6A==",
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout "ready": "ready",
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout "whoami": "0"
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout }
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] stdout }
[2021-04-22 09:44:21,419][ceph_volume.devices.raw.list][DEBUG ] Examining /dev/sda2
[2021-04-22 09:44:21,419][ceph_volume.process][INFO  ] Running command: /usr/bin/ceph-bluestore-tool show-label --dev /dev/sda2
[2021-04-22 09:44:21,445][ceph_volume.process][INFO  ] stdout {
[2021-04-22 09:44:21,445][ceph_volume.process][INFO  ] stdout "/dev/sda2": {
[2021-04-22 09:44:21,445][ceph_volume.process][INFO  ] stdout "osd_uuid": "c7c66bbd-7b38-4dcd-ad6d-3769c516f2fe",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "size": 6442450944,
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "btime": "2021-04-22T09:32:21.814768+0000",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "description": "main",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "bfm_blocks": "1572864",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "bfm_blocks_per_key": "128",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "bfm_bytes_per_block": "4096",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "bfm_size": "6442450944",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "bluefs": "1",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "ceph_fsid": "69c40cb1-22af-42e4-9d59-4a4468a2f58f",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "kv_backend": "rocksdb",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "magic": "ceph osd volume v026",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "mkfs_done": "yes",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "osd_key": "AQCkQoFgre9SKBAANgHH6scIb+IiyKxh6MhY0A==",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "ready": "ready",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "require_osd_release": "16",
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout "whoami": "0"
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout }
[2021-04-22 09:44:21,446][ceph_volume.process][INFO  ] stdout }
```

However, a single OSD gets listed by `ceph-volume raw list`:

```
[root@2b5a3b8bf31c /]# ceph-volume raw list
{
    "0": {
        "ceph_fsid": "69c40cb1-22af-42e4-9d59-4a4468a2f58f",
        "device": "/dev/sda2",
        "osd_id": 0,
        "osd_uuid": "c7c66bbd-7b38-4dcd-ad6d-3769c516f2fe",
        "type": "bluestore"
    }
}
```

We now use the osd_uuid so the output will never conflict:

```
[root@2b5a3b8bf31c /]# ceph-volume raw list
{
    "423bf64d-f241-4f4b-a589-25a66fc836d1": {
        "ceph_fsid": "d3cd4b72-5342-4fd3-96ec-a6e581261eab",
        "dev": "/dev/sda1",
        "osd_id": 0,
        "osd_uuid": "423bf64d-f241-4f4b-a589-25a66fc836d1",
        "type": "bluestore"
    },
    "c7c66bbd-7b38-4dcd-ad6d-3769c516f2fe": {
        "ceph_fsid": "69c40cb1-22af-42e4-9d59-4a4468a2f58f",
        "dev": "/dev/sda2",
        "osd_id": 0,
        "osd_uuid": "c7c66bbd-7b38-4dcd-ad6d-3769c516f2fe",
        "type": "bluestore"
    }
}
```

Fixes: https://tracker.ceph.com/issues/50478
Signed-off-by: Sébastien Han <seb@redhat.com>

mon/MonClient: remove reopen_session() callback mechanism

It's been unused for over 5 years, since commit 17d24292b812 ("osd:
remove old stats backoff mechanism").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #40954 from rhcs-dashboard/password-in-dev-tools

mgr/dashboard: Remove username and password from request body

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #40909 from tchaikov/wip-osd-cast

osd/ReplicatedBackend: cast variable to unsigned before comparing

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

crimson/os/seastore: add get/set attr methods

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #40942 from athanatos/sjust/wip-retired-placeholder

crimson/os/seastore: track retired extents until in-progress transactions complete

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40956 from rzarzynski/wip-crimson-depends-on-binutils

ceph.spec.in: crimson depends on binutils b/c of addr2line.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40970 from neha-ojha/wip-delete-some

osd: remove references to _delete_some()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40971 from tchaikov/wip-common-option-osd_erasure_code_plugins

common/options: set default variable of osd_erasure_code_plugins

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40964 from tchaikov/wip-disable-rook-mgr-module

vstart.sh: disable "rook" mgr module by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #40580 from anthonyeleven/anthonyeleven/clean-up-cephfs-standby

doc/cephfs: refine cephfs/standby

Reviewed-by: Zac Dover <zac.dover@gmail.com>

crimson/os/seastore/cached_extent: fix maybe_generate_relative

The only case we really care about here is that record relative addresses
for initial_pending extents need to be turned into block relative extents.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: retain placeholders for retired, uncached extents

We need to track extents retired without first being in cache.  Create
RetiredExtentPlaceholder extent type for conflict detection on those
cases.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: speed up mount with debugging enabled

Logging allocation information really slows down initial mount.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/cache: check cached extents for is_retired

Signed-off-by: Samuel Just <sjust@redhat.com>