mgr/cephadm: _remove_daemon -> _remove_daemons

So we can remove a single daemon synchronously, if we choose.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33539 from cbodley/wip-rgw-doc-reshard

doc/rgw: add design doc for multisite resharding

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #33698 from rhcs-dashboard/vstart-set-prometheus-port-for-each-mgr

vstart.sh: set prometheus port for each mgr.

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33047 into master

* refs/pull/33047/head:
qa/rgw: exercise DeleteRange in test_bucket_index_log_trim

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>

Merge PR #33287 into master

* refs/pull/33287/head:
rgw: move frontends initial init to after global_init()
rgw: ssl: don't try to init certificate if not needed
rgw: frontend: add rgw_frontend_defaults configurable
rgw: beast ssl: enable use of meta variable for cert config
rgw: beast ssl: improve output
rgw: beast frontend: handle default ssl configurables
rgw: update docs about ssl config through config-key
rgw: allow beast ssl frontend cert config via mon config-key
rgw: create config-key svc
rgw: svc/rados: new mon_command call

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33612 from votdev/issue_44322_rgw_user_max_buckets

mgr/dashboard: Not able to restrict bucket creation for new user

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #33721 from smithfarm/wip-nautilus_14_2_8

doc/releases: add 14.2.8 to release timeline

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

doc/rgw: add design doc for multisite resharding

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #33705 into master

* refs/pull/33705/head:
qa/suites/upgrade/nautilus-x/parallel: restart mgr.x before mons

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #33723 from tchaikov/wip-seastar-on-demand

run-make-check.sh: WITH_SEASTAR on demand

Reviewed-by: Nathan Cutler <ncutler@suse.com>

run-make-check.sh: WITH_SEASTAR on demand

don't compile WITH_SEASTAR=ON unless required

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/releases: add 14.2.8 to release timeline

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #33670 from theanalyst/doc/releases/v14.2.8

doc: releases: v14.2.8 release notes

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
Reviewed-By: Patrick Donelly <pdonell@redhat.com>

Merge pull request #33709 from tchaikov/wip-ceph-py-py3-rados

qa/tasks: py3 compat (tasks exercised by rados suites)

Reviewed-by: Kefu Chai <kchai@redhat.com>

vstart.sh: set prometheus port for each mgr.

Fixes: https://tracker.ceph.com/issues/44417
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #33690 from rhcs-dashboard/44237-remove-config-opt-read-from-system-roles

mgr/dashboard: remove 'config-opt: read' perm. from system roles.

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #33699 from liewegas/fix-44078

common/ceph_time: tolerate mono time going backwards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #33701 from tchaikov/wip-mgr-orch-exception

mgr/orch: try harder when pickle fails to marshal an exception

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #33682 from thinkercui/bugfix_monsession_map_lock

mon: Get session_map_lock before remove_session

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33703 from smithfarm/wip-doc-mon_status

doc: adjust for mon_status changes in octopus

Reviewed-by: Sage Weil <sage@redhat.com>

doc: releases: v14.2.8 release notes

Co-Authored-By: Neha Ojha <nojha@redhat.com>
Co-Authored-By: Patrick Donelly <pdonell@redhat.com>
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mon: Get session_map_lock before remove_session

Get session_map_lock to prevent race.

Fixes: https://tracker.ceph.com/issues/44407
Signed-off-by: Xiaofei Cui <cuixiaofei@sangfor.com.cn>

Merge pull request #33482 from rosinL/wip-fix-unitest-seastar-socket

test/test_socket: fix dispatch_sockets() unexpected exception

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

qa/tasks/ceph_deploy: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/manypools: get rid of itervalues for py3 compat

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/vstart_runner: use io.BytesIO for py3 compat

Use io.BytesIO instead of StringIO for py3 compatibility.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/ceph_manager: ensure str for py3 compat

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/ceph_objectstore_tool: get rid of cStringIO for py3

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/divergent_priors2: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/radosbenchsweep: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/reg11184: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/netem: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/scrub_test: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/systemd: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/admin_socket: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/cephadm: get rid of cStringIO for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/omapbench: get rid of itervalues for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/mgr: get rid of itervalues for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/watch_notify_stress: get rid of itervalues for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/osd_max_pg_per_osd: get rid of itervalues for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/radosbench.py: get rid of itervalues for py3 compat

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/rados: get rid of itervalues for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/daemonwatchdog: py3 compat

Address error:
        RuntimeError: dictionary changed size during iteration

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/workunit: py3 compat

Use six.ensure_str for `get_file()` output in order to
make code py3 compatible

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks: fix import module path for py3 compat

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/thrashosds: fix imports for py3

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/ceph: py3 compatibility

Addresses:
  TypeError: 'dict_values' object is not subscriptable

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks: fix imports for py3 compatibility

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/ceph_manager: get rid of CStringIO for py3

Use io.BytesIO instead cStringIO.StringIO for py3 compatibility

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/tasks/ceph: get rid of cStringIO for py3 compat

Use io.BytesIO instead of cStringIO.StringIO
Use six.ensure_str whenever it needs to convert binary to str.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

Merge pull request #33584 from liewegas/fix-39525

compressor/lz4: work around bug in liblz4 versions <1.8.2

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32444 from majianpeng/simple-debug-message

osd/PrimaryLogPG: simple debug message.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33659 from rzarzynski/wip-osd-ps-plco-via-setter

osd: peering updates peer_last_complete_ondisk via setter.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33660 from rzarzynski/wip-osd-clean-pgtrim-includes

osd: drop unnecessary includes of messages/MOSDPGTrim.h.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33673 from cbodley/wip-denc-enum

denc: support enums wider than 8 bits

Reviewed-By: Adam Emersen <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33689 from Songweibin/wip-fix-typos

doc/rbd: fix small typos

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #33667 into master

* refs/pull/33667/head:
mgr/orch: show placement in 'orch ls'
mgr/orch: fix SPEC alignment in 'orch ls'
mgr/orch: include spec ref in ServiceDescription

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33669 into master

* refs/pull/33669/head:
mgr/cephadm: reduce log level for NodeAssignment
mgr/cephadm: make NodeAssignment return a simple host list

Reviewed-by: Joshua Schmid <jschmid@suse.de>

doc: adjust for mon_status changes in octopus

In octopus, we no longer have "ceph mon_status", but we do have
"ceph tell mon.<id> mon_status", so reflect this in the docs.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge PR #33678 into master

* refs/pull/33678/head:
mgr/cephadm: fix upgrade when daemon is stopped

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33671 into master

* refs/pull/33671/head:
mgr/cephadm: refresh configs when dependencies change
mgr/cephadm: track last config time and dependencies

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33695 into master

* refs/pull/33695/head:
mgr/orch: add apply alertmanager to CLI
mgr/orch: add apply grafana to CLI

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33672 into master

* refs/pull/33672/head:
Refactor, prepare for other adoptions

Reviewed-by: Sage Weil <sage@redhat.com>

qa/suites/upgrade/nautilus-x/parallel: restart mgr.x before mons

Fixes: https://tracker.ceph.com/issues/44387
Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/tasks/test_orchestrator_cli: enable test_osd_create exception check

this reverts 9639acfefe09c87adc821bb5c5cc41974685331d, as the test does
make sense. what fails this test is the machinary to marshal/unmarshal
exception fails to handle un-picklable exceptions. the previous commit
is supposed to use a fallback to handle them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/orch: try harder when pickle fails to marshal an exception

pickle cannot marshal instances of class not defined in the top level of
a module. for instance, `DriveGroupValidationError` is defined in a
submodule of `ceph` python module, that's why we cannot capture it.

this prevent the `ceph` command line from getting a proper error when
the command fails if the command is implemented using cross python
module call(s).

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #33664 into master

* refs/pull/33664/head:
doc: change case from `apis` to `APIs`

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33655 from p-se/pse-fix-prom-alert-pool-full

monitoring: restore lost fix for `pool full` alert

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge PR #33636 into master

* refs/pull/33636/head:
qa: add upgrade test for volume upgrade from legacy

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

common/ceph_time: tolerate mono time going backwards

Some kernels (and possibly some hardware?) can trigger a monotonic clock
that goes back in time.  That, in turn, can lead to a negative monotonic
time span.  This would trigger an assert.

This this problem seems to be widespread, tolerate the case and interpret
it as a 0-length interval (vs something negative).

Fixes: https://tracker.ceph.com/issues/44078
Fixes: https://tracker.ceph.com/issues/43365
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33694 into master

* refs/pull/33694/head:
cephadm: fix `alertmanager not implemented yet`

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33679 into master

* refs/pull/33679/head:
cephadm: fix failure when getting keyring for deploying daemons

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/orch: add apply alertmanager to CLI

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: add apply grafana to CLI

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: refresh configs when dependencies change

If a daemon config (e.g., prometheus) depends on other daemons'
existence, refresh the config if that list changes (e.g., new
node-exporter, mgr removed, etc.).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33588 from dillaman/wip-rbd-snapshot-mirror

rbd-mirror: initial end-to-end test and associated bug fixes

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/orch: show placement in 'orch ls'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: fix upgrade when daemon is stopped

Tried to fix this in 5649d5571c14 but didn't remove the early image_id
check.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: fix `alertmanager not implemented yet`

```
Traceback (most recent call last):
  File "/home/user/src/ceph/src/pybind/mgr/cephadm/module.py", line 391, in do_work
    res = self._on_complete_(*args, **kwargs)
  File "/home/user/src/ceph/src/pybind/mgr/cephadm/module.py", line 455, in call_self
    return f(self, *inner_args)
  File "/home/user/src/ceph/src/pybind/mgr/cephadm/module.py", line 2576, in _create_alertmanager
    custom_config=self._generate_alertmanager_config)
  File "/home/user/src/ceph/src/pybind/mgr/cephadm/module.py", line 2051, in _create_daemon
    stdin=json.dumps(cephadm_config))
  File "/home/user/src/ceph/src/pybind/mgr/cephadm/module.py", line 1459, in _run_cephadm
    code, '\n'.join(err)))
RuntimeError: cephadm exited with an error code: 1, stderr:ERROR: alertmanager not implemented yet
```

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

mgr/dashboard: remove 'config-opt: read' perm. from system roles.

Fixes: https://tracker.ceph.com/issues/44237
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

doc/rbd: fix small typos

Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #33120 from vshankar/wip-mgr-volumes-clone-sync-attrs

mgr/volumes: sync inode attributes for cloned subvolumes

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32423 from bk201/wip-43420

mgr/dashboard: allow refreshing inventory page

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

doc: change case from `apis` to `APIs`

use APIs instead of apis to be consistent throughout.

fixes: https://tracker.ceph.com/issues/44374
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #33680 from tchaikov/wip-crimson-cleanups

crimson: clang related cleanups

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #31910 from rosinL/wip-fix-dpdk-test

test/msgr: fix ComplexTest fail when using DPDK protocal stack

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33661 from bk203/patch-1

doc: update Zabbix template reference

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa: add upgrade test for volume upgrade from legacy

This tests that volumes created using the ceph_volume_client.py library
continue to be accessible/function via the Nautilus/Octopus ceph-mgr
volumes plugin.

Fixes: https://tracker.ceph.com/issues/42723
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

cmake: build used source

to address the link failures when compiling using clang

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: do not capture unused variable

this silences the warning of:
```
../src/crimson/osd/osdmap_gate.cc:48:38: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
  std::for_each(first, last, [epoch, this](auto& blocked_requests) {
                                   ~~^~~~
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

cephadm: fix failure when getting keyring for deploying daemons

Fixes: https://tracker.ceph.com/issues/44390
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

doc: update Zabbix template reference

The old link references a 2017 version of the template, I experienced problems using this version of the template with the latest version of Ceph. Ceph would report "Failed to send data to Zabbix", by importing the newer 2019 version of the template within Zabbix Ceph could again send data (due to changed Zabbix Trapper item keys). Propose to replace the link for a link referencing the master branch of the template so the newest version is always referenced in the docs.

Signed-off-by: Mathijs Smit <m.smit@goldenvalue.nl>

Merge pull request #33621 from aclamk/fix_mempool_btree

test/test_mempool: test accounting for btree_map

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33363 into master

* refs/pull/33363/head:
mds: add perf counters for openfiletable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #33491 into master

* refs/pull/33491/head:
mount.ceph: add "fs=<fs_name>" mount options support

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #33506 into master

* refs/pull/33506/head:
client: add client_fs mount option support

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #33538 into master

* refs/pull/33538/head:
mds: just delete MDSIOContextBase during shutdown
mds: add comment explaining unusual locking for MDSIOContextBase

Reviewed-by: Zheng Yan <zyan@redhat.com>

mgr/orch: fix SPEC alignment in 'orch ls'

Signed-off-by: Sage Weil <sage@redhat.com>