crimson: formalize the interface between errors and errorators.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the read path, part 5.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: rework discarding errors in errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: introduce syntactic sugar for visiting errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: migrate make_error<T> to unthrowable_wrapper.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: abstract the error type in unthrowable_wrapper.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: decouple errorator from unthrowable_wrapper.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: decouple errorator from unthrowable_wrapper::instance.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: add more static asserts to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: dissect maybe_handler_error_t from errorator template.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the read path, part 4.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: let an errorator to be converted into more errorated one.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the read path, part 3.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: facilitate errorator extending with new errors.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the read path, part 2.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os, crimson/osd: apply errorator along the read path, part 1.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: bring discard_all{} to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: bring compile time-verified chaining of errorators.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: dissect crc checking on reads into separated procedure.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: alias errorator for CyanStore::get_attrs().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: drop the old FuturizedStore::Exception class.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: bring crimson::errorator with its first appliance.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: Support RBD namespaces (#30935)

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #31357 from votdev/improve_timediff_unittest

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #30534 from jiahuizeng/osd_dev_node

mgr/prometheus: assign a value to osd_dev_node when obj_store is not filestore or bluestore

Merge pull request #31431 from jschmid1/fix_cherrypy_imports

mgr/dashboard: cheroot moved into a separate project

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr: Run python unit tests with tox in the mgr (#30364)

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/dashboard: Remove compression mode unset in pool from (#31376)

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29944 from xxhdx1985126/wip-fix-blustore-fadvice_flags-leak

osd: copy ObjectOperation::BufferUpdate::Write::fadvise_flag to ceph::os::Transaction

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31333 from adamemerson/wip-who-warns-the-warnings

src: use un-deprecated version of aio_create_completion

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31416 from sseshasa/wip-41666-replicaSizeWarn

osd/OSDMap: Show health warning if a pool is configured with size 1

Reviewed-by: Sage Weil <sweil@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31475 from zdover23/wip-doc-community-calendar

doc: Added a link to Ceph Community Calendar.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

doc: Added a link to Ceph Community Calendar.

Co-Authored by: Neha Ojha <nojha@redhat.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge PR #31501 into master

* refs/pull/31501/head:
ceph-mon: bind to public_addrv if set (before public_addr or public_network)
common/options: add public_addrv
common/options: fix ADDRVEC parsing
ceph-daemon: support --mon-addrv for deploy subcommand

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #30914 into master

* refs/pull/30914/head:
ceph: Add doc for deploying cephfs-nfs cluster using rook

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #31540 into master

* refs/pull/31540/head:
ceph-daemon: make ps1 a raw string

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-daemon: make ps1 a raw string

python3 interprets '\u' as a unicode escape

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #31024 into master

* refs/pull/31024/head:
doc: improve in mount.ceph man page
doc: describe mds_namespace option in mount.ceph man page

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30911 into master

* refs/pull/30911/head:
doc/ceph-fuse: describe -n option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #30657 into master

* refs/pull/30657/head:
test_cephfs_shell: pass arg to "help" in test_help
cephfs-shell: set exit_code when an error occurs
cephfs-shell: print traceback only when debug switch is on
cephfs-shell: catch libcephfs.Error only in onecmd()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31452 into master

* refs/pull/31452/head:
Add chown ceph:ceph for /tmp/ceph.mon.keyring

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31538 into master

* refs/pull/31538/head:
ceph-daemon: do not fetch daemon detail when removing a cluster
ceph-daemon: ls: add --no-detail

Reviewed-by: Sebastian Wagner <swagner@suse.com>

ceph-daemon: do not fetch daemon detail when removing a cluster

It's slow and not needed.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: ls: add --no-detail

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31387 into master

* refs/pull/31387/head:
src/CMakeLists: remove leading 'v' from 'git describe' version

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #31524 into master

* refs/pull/31524/head:
mgr/ssh: Fix various Python issues.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

mgr/dashboard: Improve position of MDS chart tooltip (#31368)

mgr/dashboard: Improve position of MDS chart tooltip

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #31512 into master

* refs/pull/31512/head:
mgr/ssh: allow passing LV to 'orchestrator osd create'

Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/dashboard: Support RBD namespaces

Fixes: https://tracker.ceph.com/issues/25125
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Fix data point alignment in MDS counters chart (#31288)

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31519 from votdev/fix_ssh_vagrant

mgr/ssh: Install SSH public key in Vagrantfile box fails

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/ssh: Fix various Python issues.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: RBD tests must use pools with power-of-two pg_n… (#31490)

mgr/dashboard: RBD tests must use pools with power-of-two pg_num

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/ssh: Install SSH public key in Vagrantfile box fails

The current implementation to copy the host users SSH public key into the Vagrant box does not work with Vagrant 2.2.6.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #31419 from smithfarm/wip-cbs-5

ceph-backport.sh: guess component with --existing-pr

Reviewed-by: Laura Paduano <lpaduano@suse.com>

osd/OSDMap: Show health warning if a pool is configured with size 1

Introduce a config option called 'mon_warn_on_pool_no_redundancy' that is
used to show a health warning if any pool in the ceph cluster is
configured with a size of 1. The user can mute/unmute the warning using
'ceph health mute/unmute POOL_NO_REDUNDANCY'.

Add standalone test to verify warning on setting pool size=1. Set the
associated warning to 'false' in ceph.conf.template under qa/tasks so
that existing tests do not break.

Fixes: https://tracker.ceph.com/issues/41666
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

Add chown ceph:ceph for /tmp/ceph.mon.keyring

The keyring will be 600 by default with the owner of the current user who issues the command, which will make command "sudo -u ceph ceph-mon --mkfs" fail since the user "ceph" don't have access to the keyring file.
Because of that, the mon data directory won't have keyring and when "sudo systemctl start ceph-mon" it will show error "unable to read magic from mon data"

Signed-off-by: Jeffrey Chu <peihuachu1112@gmail.com>

Merge pull request #31271 from ivancich/wip-fix-cancel-bucket-reshard

rgw: allow reshard log entries for non-existent buckets to be cancelled

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #31492 from jan--f/c-v-add-part-size

ceph-volume: add proper size attribute to partitions

mgr/ssh: allow passing LV to 'orchestrator osd create'

In this case,

 ceph orchestrator osd create smithi009:/dev/vg_nvme/lv_1

produced

    "0": [
        {
            "devices": [
                "/dev/nvme0n1"
            ],
            "lv_name": "lv_1",
            "lv_path": "/dev/vg_nvme/lv_1",
            "lv_size": "89.40g",
            "lv_tags": "ceph.block_device=/dev/vg_nvme/lv_1,ceph.block_uuid=a1vfDi-0jor-VEPK-DZVd-K90y-cqG1-QoXr9l,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=5d6d2c1c-0362-11ea-825a-001a4aab830c,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.encrypted=0,ceph.osd_fsid=04175a6e-55e7-4244-b683-79da76f723af,ceph.osd_id=0,ceph.type=block,ceph.vdo=0",
            "lv_uuid": "a1vfDi-0jor-VEPK-DZVd-K90y-cqG1-QoXr9l",
            "name": "lv_1",
            "path": "/dev/vg_nvme/lv_1",
            "tags": {
                "ceph.block_device": "/dev/vg_nvme/lv_1",
                "ceph.block_uuid": "a1vfDi-0jor-VEPK-DZVd-K90y-cqG1-QoXr9l",
                "ceph.cephx_lockbox_secret": "",
                "ceph.cluster_fsid": "5d6d2c1c-0362-11ea-825a-001a4aab830c",
                "ceph.cluster_name": "ceph",
                "ceph.crush_device_class": "None",
                "ceph.encrypted": "0",
                "ceph.osd_fsid": "04175a6e-55e7-4244-b683-79da76f723af",
                "ceph.osd_id": "0",
                "ceph.type": "block",
                "ceph.vdo": "0"
            },
            "type": "block",
            "vg_name": "vg_nvme"
        }
    ],

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #30853 from ivancich/wip-tune-sharded-bucket-listing

rgw: reduce per-shard entry count during ordered bucket listing

Merge pull request #31323 from ivancich/wip-autoclean-reshard-long-entries

rgw: auto-clean reshard queue entries for non-existent buckets

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge PR #31505 into master

* refs/pull/31505/head:
mgr/ssh: take IP, CIDR, or addrvec for new mon(s)

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #31507 into master

* refs/pull/31507/head:
ceph-daemon: ceph/daemon-base:latest-master-devel

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

ceph-mon: bind to public_addrv if set (before public_addr or public_network)

This lets you specify a precise addrvec for the monitor to bind to,
instead of inferring a v1 or v2 address from the port number, or assuming
both with default ports, or v2 only with an unrecognized port.

I unfortunately don't quite remember why I didn't make public_addr a
ADDRVEC option way back in nautilus--I only remmeber that it was going to
cause problems.  Adding a separate addrv option here is safe.

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: add public_addrv

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31474 into master

* refs/pull/31474/head:
mgr/devicehealth: ensure we don't store empty objects

Reviewed-by: Neha Ojha <nojha@redhat.com>

ceph-daemon: ceph/daemon-base:latest-master-devel

The -devel one rebuilds daily, while latest-master rebuilds when
ceph-container.git changes.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: reduce per-shard entry count during ordered bucket listing

Currently, if a client requests the 1000 next entries from a bucket,
each bucket index shard will receive a request for the 1000 next
entries. When there are hundreds, thousands, or tens of thousands of
bucket index shards, this results in a huge amplification of the
request, even though only 1000 entries will be returned.

These changes reduce the per-bucket index shard requests. These also
allow re-requests in edge cases where all of one shard's returned
entries are consumed. Finally these changes improve the determination
of whether the resulting list is truncated.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge PR #31498 into master

* refs/pull/31498/head:
ceph-daemon: customize the bash prompt for shell + enter

Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/ssh: take IP, CIDR, or addrvec for new mon(s)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31499 into master

* refs/pull/31499/head:
ceph-daemon: the 'ceph' command may not be installed

Reviewed-by: Michael Fritch <mfritch@suse.com>

ceph-daemon: the 'ceph' command may not be installed

...even though there are legacy data dirs in /var/lib/ceph.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31500 into master

* refs/pull/31500/head:
ceph-daemon: set ssh public identity

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #31472 into master

* refs/pull/31472/head:
mon/MonMap: encode (more) valid compat monmap when we have v2-only addrs

Reviewed-by: Xiaoxi Chen <superdebuger@gmail.com>

Merge PR #31471 into master

* refs/pull/31471/head:
mgr/DaemonServer: warn when we reject reports

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

common/options: fix ADDRVEC parsing

We never actually had any of these yet.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: support --mon-addrv for deploy subcommand

This maps directly to the public_addrv option.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: set ssh public identity

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #31421 from alfredodeza/wip-bz1768523

ceph-volume: allow to skip restorecon calls

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Sebastien Han <shan@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

ceph-daemon: customize the bash prompt for shell + enter

[ceph: root@gnit /]#

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31493 into master

* refs/pull/31493/head:
ceph-daemon: 'profile ...' not 'allow profile ...'
mgr/ssh: 'profile ...' not 'allow profile ...'
mgr/orchestrator_cli: rearrange things a bit
doc/mgr/orchestrator_cli: remove irrelevant line
mgr/ssh: learn to deploy rbd-mirror daemons
mgr/orchestrator: add rbd-mirror commands and hooks
ceph-daemon: learn to deploy rbd-mirror daemon
mgr/ssh: handle lack of node hints more gracefully
mgr/ssh: factor out update_{rgw,mds} into common helper
mgr/ssh: fix update_rgw, update_mgr

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #30890 into master

* refs/pull/30890/head:
mgr: invoke plugin shutdown on SIG{TERM,INT} signals.
mgr/volumes: guard volume delete by waiting for pending ops
mgr/volumes: cleanup libcephfs handles when stopping
Revert "qa/suites/rados/mgr/tasks/module_selftest: whitelist mgr client getting backlisted"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31115 into master

* refs/pull/31115/head:
mds: Reorganize class members in MDSTableClient header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31400 into master

* refs/pull/31400/head:
mds: establish session with mgr only after added to FSMap
mds: do not register as a service daemon
mds: do not try to diagnose cause of MDSMap removal
mds: fix handling of initial MDS states
mds: remove unnecessary const qualifier
mds: cleanup type decl and map iteration
mds: define stream operator for mds_info_t

Reviewed-by: Sage Weil <sage@redhat.com>

mds: establish session with mgr only after added to FSMap

Note that we now sub to the mgrmap after init because the MgrClient
connection to the mgr is driven by receipt of the MgrMap.

This is important so that the MDS does not have metadata with the mgr
when the mons are ignoring the MDS otherwise due to CompatSet
incompatibilities.

Fixes: https://tracker.ceph.com/issues/41538
Fixes: https://tracker.ceph.com/issues/42635
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: do not register as a service daemon

This commit undoes the service daemon registration for the MDS. It doesn't look
absolutely necessary and it causes the MDS to be listed twice in the `ceph
versions` output:

    $ ceph versions
        ...
        "mds": {
            "ceph version v15.0.0-6915-g0143b904676 (0143b9046763ea1801efa8358a0c033ec862cea9) octopus (dev)": 3
        },
        "mds": {
            "unknown": 3
        },
        "overall": {
            "ceph version v15.0.0-6915-g0143b904676 (0143b9046763ea1801efa8358a0c033ec862cea9) octopus (dev)": 10,
            "unknown": 3
        }

Fixing that requires looking for duplicates or ignoring MDSs in the
service daemons when the mon processes `ceph versions`. I have a feeling
that it wasn't actually designed to be used by the MDS this way however.
Additionally, the reason for "unknown" version is because the metadata
sent to the mgr does not include "ceph_version".

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: do not try to diagnose cause of MDSMap removal

The Monitors send an empty MDSMap to an MDS it is removing. The MDS
can't diagnose the cause. Instead suggest looking at the cluster/monitor
logs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: fix handling of initial MDS states

Few things here:

- Make explicit the check for getting removed from the MDSMap. This was
  only done before by checking if MDS held a rank which does not check the
  case where a standby is removed from the FSMap.

- Use mds_info_t::dump to simplify various debug output.

- Add a few sanity asserts for invalid state transitions.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: remove unnecessary const qualifier

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: cleanup type decl and map iteration

This is a trivial refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: define stream operator for mds_info_t

This also cleans up the output to be more readable/useful in debug
output.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29437 into master

* refs/pull/29437/head:
mgr/diskprediction_local: Reverted dependencies, added HGST models
mgr/diskprediction_local: Updated dependencies in ceph.spec.in, debian/control to match requirements.txt
mgr/diskprediction_local: Updated Red Hat developed prediction model. Updated module options to choose between Red Hat and ProphetStor models.
mgr/diskprediction_local: Updated prediction models to use only supported python packages.
mgr/diskprediction_local: Replaced old models and updated predictor.

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-daemon: 'profile ...' not 'allow profile ...'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: 'profile ...' not 'allow profile ...'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orchestrator_cli: rearrange things a bit

Group services together

Signed-off-by: Sage Weil <sage@redhat.com>

doc/mgr/orchestrator_cli: remove irrelevant line

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: learn to deploy rbd-mirror daemons

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orchestrator: add rbd-mirror commands and hooks

This is somewhat different from the other services in that the name is
basically unused: we have a single pool of rbd-mirror daemons for the
whole cluster.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-volume: add proper size attribute to partitions

Fixes: https://tracker.ceph.com/issues/42710
Signed-off-by: Jan Fajerski <jfajerski@suse.com>