ceph-volume: lvm.activate: Return an error if WAL/DB devices absent

Fixes: https://tracker.ceph.com/issues/40100
Signed-off-by: David Casier <david.casier@aevoo.fr>
(cherry picked from commit de439a0a236e4752e64a0cd63b7ce1ad3278eff3)

Merge pull request #28400 from xiexingguo/wip-pr-27735-for-n

nautilus: osd: revamp {noup,nodown,noin,noout} related commands

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

PendingReleaseNotes: refresh the no{up,down,in,out} flags change

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27742 from s0nea/wip-39346-nautilus

nautilus: mgr/dashboard: Manager should complain about wrong dashboard certificate

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #28175 from tspmelo/wip-39961-nautilus

nautilus: mgr/dashboard: Unify the look of dashboard charts

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #28191 from ricardoasmarques/wip-39988-nautilus

nautilus: mgr/dashboard: Unable to see tcmu-runner perf counters

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by:  Tiago Melo <tmelo@suse.com>

Merge pull request #28273 from pdvian/wip-39675-nautilus

nautilus: qa/workunits/rbd: wait for rbd-nbd unmap to complete

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28398 from tchaikov/nautilus-pr-28268

nautilus: mon: fix off-by-one rendering progress bar

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28458 from yuriw/wip-yuriw-40208-nautilus

qa/tests: removed `1node` and `systemd` tests as ceph-deploy is not a…

Reviewed-by: Neha Ojha <nojha@redhat.com>

osd: unify sources of no{up,down,in,out} flags into singleton helpers

Currently there are various ways to set no{up,down,in,out} flags,
by osd, by crush node, or by device class.

Use traditional is_no{up,down,in,out} helper to catch-all,
and add special wrappers for certain internal consumers.

This way osd won't ignore the noup flag set by the new crush node,
device class, or any future sources.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 9d11a3a1c96edd4f38f4584ce76febf920763883)

Conflicts:
        slight conflict from the "Remove dependence on 'using namespac'" change,
        see https://github.com/ceph/ceph/pull/27255

Merge pull request #27677 from smithfarm/wip-39197-nautilus

nautilus: cephfs-shell: ls command produces error: no colorize attribute found error

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27714 from pdvian/wip-39192-nautilus

nautilus: mds: drop reconnect message from non-existent session

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27736 from pdvian/wip-39199-nautilus

nautilus: mds/server: check directory split after rename.

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27842 from pdvian/wip-39209-nautilus

nautilus: mds: initialize cap_revoke_eviction_timeout with conf

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27846 from ashishkumsingh/wip-39470-nautilus

nautilus: mds: better output of 'ceph health detail'

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27849 from Vicente-Cheng/wip-39473-nautilus

nautilus: mds: fail to resolve snapshot name contains '_'

Reviewed-by: Venky Shankar <vshankar@redhat.com>

qa/tests: removed `1node` and `systemd` tests as ceph-deploy is not actively developed

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #27851 from Vicente-Cheng/wip-39232-nautilus

nautilus: kclient: nofail option not supported

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27852 from Vicente-Cheng/wip-39214-nautilus

nautilus: mds: there is an assertion when calling Beacon::shutdown()

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27853 from Vicente-Cheng/wip-39211-nautilus

nautilus: mds: MDSTableServer.cc: 83: FAILED assert(version == tid)

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27879 from pdvian/wip-39222-nautilus

nautilus: mds: behind on trimming and "[dentry] was purgeable but no longer is!"

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27892 from Vicente-Cheng/wip-38876-nautilus

nautilus: mds: high debug logging with many subtrees is slow

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #27901 from Vicente-Cheng/wip-39471-nautilus

nautilus: mds: Expose CephFS snapshot creation time to clients

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #28040 from sebastian-philipp/wip-26654-nautilus

nautilus: mgr/orchestrator: add progress events to all orchestrators

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27625 from liewegas/wip-min-crush-hammer-nautilus

nautilus: common/options: update mon_crush_min_required_version=hammer

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27824 from liewegas/wip-abort-msg-nautilus

nautilus: common/assert: include ceph_abort_msg(arg) arg in log output

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27893 from Vicente-Cheng/wip-39050-nautilus

nautilus: ceph_volume_client: Too many arguments for "WriteOpCtx"

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27919 from sebastian-philipp/wip-39559-nautilus

nautilus: mgr/ansible: Host ls implementation

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #28178 from tspmelo/wip-39658-nautilus

nautilus: mgr/dashboard: Avoid merge conflicts in messages.xlf by auto-generating it at build time?

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>

Merge pull request #28261 from ricardoasmarques/wip-40031-nautilus

nautilus: mgr/dashboard: local variable 'cluster_id' referenced before assignment error when trying to list NFS Ganesha daemons

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #28382 from liewegas/wip-40044-nautilus

nautilus: common: fix parse_env nullptr deref

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>

Merge pull request #27772 from smithfarm/wip-39430-nautilus

nautilus: qa: test_sessionmap assumes simple messenger

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28141 from pdvian/wip-39476-nautilus

nautilus: common/util: handle long lines in /proc/cpuinfo

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28248 from pdvian/wip-39671-nautilus

nautilus: commom: make cluster_network work.

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27818 from liewegas/wip-runtime-configs-nautilus

nautilus: common/options: annotate some options; enable some runtime updates

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28299 from smithfarm/wip-40067-nautilus

nautilus: build/ops: Ceph RPM build fails on openSUSE Tumbleweed with GCC 9

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #28325 from cbodley/wip-40087

nautilus: common: parse ISO 8601 datetime format

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28347 from smithfarm/wip-40090-nautilus

nautilus: mgr: log an error if we can't find any modules to load

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

qa: tolerate wildcard position

It does not make much sense to add this kind of restrictions
as long as user is aware of what is going on.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 199d92aec09f756522bb61ada7951d852c5e3640)

doc: refresh {noup,nodown,noin,noout} changes

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit a3b0dc29b915f84545ac5bd954b6c68b8dde1c69)

mon/OSDMonitor: migrate 'osd {add,rm}-{noup,nodown,noin,noout}' command to new implementation

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 20d05a13b33d7655709d173c43518f26d22a507b)

mon/MonCommands: mark 'osd {add,rm}-{noup,nodown,noin,noout}' as deprecated

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 06e2255f90d178d308c1eaa83c4c591756585bc0)

osd: add no{out,down,in,out} flags by device class

This works as a good supplement of https://github.com/ceph/ceph/pull/27563.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 5d695267ef03e56bf8b28853554d74e4b94b03b7)

Conflicts:
slight conflict from the "Remove dependence on 'using namespac'" change,
        see https://github.com/ceph/ceph/pull/27255

osd,mon: add 'ceph osd {set,unset}-group <flags> <who>' command

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 9a05a065b936130219dfb7d11282abd80d13de85)

mon: fix off-by-one rendering progress bar

Ensure that if progress is < 1.0, we always have a '.' at the end of the
bar.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0e49ad74f9ebbefb418f9eb1c8842dbe8f16fa18)

Merge pull request #28028 from ifed01/wip-ifed-dump-before-nospanid-nautilus

nautilus: os/bluestore: dump before "no-spanning blob id" abort

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27525 from ifed01/wip-ifed-fix-21312-nautilus

nautilus: os/bluestore: call fault_range prior to looking for blob to reuse

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27645 from ifed01/wip-ifed-legacy-statfs-alert-nautilus

nautilus: os/bluestore: backport more bluestore alerts

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27819 from liewegas/wip-bluestore-deferred-metric-fix-nautilus

nautilus: os/bluestore: correctly measure deferred writes into new blobs

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #28007 from smithfarm/wip-39612-nautilus

nautilus: os/bluestore: fix for FreeBSD iocb structure

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

common: fix parse_env nullptr deref

Fixes: https://tracker.ceph.com/issues/39599
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit caba2e25ce9c615b41d229c1d583609ced06b891)

test: add test for parse_env

Test for https://tracker.ceph.com/issues/39599

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ccaf2ccc13374825b6e68277a542edf6ec8e92cf)

Merge pull request #28369 from batrick/i40116

nautilus: qa: cannot schedule kcephfs/multimds

qa: kcephfs/multimds test latest rhel

Fixes: http://tracker.ceph.com/issues/40116
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: fix missing yaml fragment

This is preventing kcephfs/multimds from being scheduled.

Introduced-by: 2526cc6e649cded87113beae745e7c7984569c0d
Partial-revert: 2526cc6e649cded87113beae745e7c7984569c0d
Fixes: http://tracker.ceph.com/issues/40116
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28218 from bk201/wip-39993-nautilus

nautilus: mgr/dashboard: inconsistent result when editing a RBD image's features

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #28022 from Devp00l/wip-39560-nautilus

 nautilus:  mgr/dashboard: Queue notifications as default

Reviewed-by:  Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #27884 from smithfarm/wip-39530-nautilus

nautilus: tests: mgr: tox failures when running make check

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #27998 from smithfarm/wip-39540-nautilus

nautilus: monitoring: Provide a base set of Prometheus alert manager rules that notify the user about common Ceph error conditions

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #28098 from pdvian/wip-39425-nautilus

nautilus: mgr: deadlock

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28189 from votdev/fix_dialog_title

nautilus: mgr/dashboard: Display correct dialog title

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #28262 from jecluis/nautilus-fix-backport-mon-min-release

mon/Monitor: allow probe if MMonProbe::mon_release == 0

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28101 from jan--f/wip-39932-nautilus

nautilus: monitoring: update Grafana dashboards

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by:  Kanika Murarka <kmurarka@redhat.com>

mgr/PyModuleRegistry: log error if we can't find any modules to load

Fixes: https://tracker.ceph.com/issues/39662
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 67a47951053bbca0d5951ac3f49fd99d100a9d14)

Merge pull request #27864 from sebastian-philipp/wip-39312-nautilus

nautilus: mgr/rook: Added missing rgw daemons in service ls

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27863 from sebastian-philipp/wip-39313-nautilus

nautilus: mgr/rook: Fix RGW creation

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27862 from sebastian-philipp/wip-39356-nautilus

nautilus: mgr/rook: Remove support for Rook older than v0.9

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27918 from pdvian/wip-39273-nautilus

nautilus: rgw: Evaluating bucket policies also while reading permissions for an…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27921 from cbodley/wip-38918

nautilus: rgw multisite: add perf counters to data sync

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27924 from Vicente-Cheng/wip-39410-nautilus

nautilus: rgw: inefficient unordered bucket listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27944 from smithfarm/wip-39577-nautilus

nautilus: build/rgw: unittest_rgw_dmclock_scheduler does not need Boost_LIBRARIES #26799

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #27952 from Vicente-Cheng/wip-39414-nautilus

nautilus: multisite: period pusher gets 403 Forbidden against other zonegroups

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27953 from Vicente-Cheng/wip-39503-nautilus

nautilus: rgw: clean up some logging

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27954 from Vicente-Cheng/wip-39574-nautilus

nautilus: rgw: cloud sync module logs attrs in the log

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #27955 from Vicente-Cheng/wip-39575-nautilus

nautilus: librgw: unexpected crash when creating bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28064 from theanalyst/nautilus-cloud-sync-fixes

nautilus: rgw: cloud sync module fails to sync multipart objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27630 from pdvian/wip-39018-nautilus

nautilus: rgw admin: add tenant argument to reshard cancel

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27660 from pdvian/wip-39048-nautilus

nautilus: rgw: beast: set a default port for endpoints

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27664 from pdvian/wip-39178-nautilus

nautilus: rgw: limit entries in remove_olh_pending_entries()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27779 from s0nea/wip-39479-nautilus

nautilus: test/rgw: fix race in test_rgw_reshard_wait and test_rgw_reshard_wait uses same clock for timing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27800 from pdvian/wip-39202-nautilus

nautilus: rgw: race condition between resharding and ops waiting on resharding

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27814 from ashishkumsingh/wip-39417-nautilus

nautilus: rgw: multisite log trimming only checks peers that sync from us

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27826 from ashishkumsingh/wip-39502-nautilus

nautilus: rgw: bucket stats report mtime in UTC

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27827 from ashishkumsingh/wip-39496-nautilus

nautilus: rgw: admin: handle delete_at attr in object stat output

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27904 from pdvian/wip-39228-nautilus

nautilus: rgw_file: save etag and acl info in setattr

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #27890 from Devp00l/wip-39534-nautilus

 nautilus: mgr/dashboard: RBD snapshot name suggestion with local time suffix

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #28021 from Devp00l/wip-39558-nautilus

 nautilus: mgr/dashboard: KV-table transforms dates through pipe

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

common/utime: don't pass %z to utime if there is a 'Z'

This fails on centos 7 for some reason.

If it's UTC, we don't need to parse the tz portion, so avoid the issue by
leaving the %z out of the fmt string.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1d1c4782eea1fac0e9230ed5ac32b19517fdd616)

common/utime: make parse() handle (our) ISO 8601 output

- optional subsecond values
- optional timezone offset

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 668d6e7f97bd37f1896611249cfae85bced4b096)

Merge pull request #28091 from pdvian/wip-39423-nautilus

nautilus: test/librbd: drop 'ceph_test_librbd_api' target

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28120 from dillaman/wip-38869-nautilus

nautilus: librbd: make flush be queued by QOS throttler

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28121 from dillaman/wip-39428-nautilus

nautilus: librbd: async open/close should free ImageCtx before issuing callback

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28122 from dillaman/wip-39462-nautilus

nautilus: rbd-mirror: clear out bufferlist prior to listing mirror images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28203 from dzafman/wip-39699

nautilus: osd: Don't include user changeable flag in snaptrim related assert

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #28258 from pdvian/wip-39672-nautilus

nautilus: os/bluestore: fix missing discard in BlueStore::_kv_sync_thread

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27746 from LenzGr/wip-39345-nautilus

nautilus: mgr/dashboard: code documentation

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #27760 from ricardoasmarques/wip-39454-nautilus

nautilus: mgr/dashboard: Validate if any client belongs to more than one group

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #27868 from LenzGr/wip-39467-nautilus

nautilus: mgr/dashboard: Admin resource not honored

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Wido den Hollander <wido@widodh.nl>

Merge pull request #28011 from ricardoasmarques/wip-39616-nautilus

nautilus: mgr/dashboard: iSCSI should allow exporting an RBD image with Journaling enabled

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #28026 from ricardoasmarques/wip-39631-nautilus

nautilus: mgr/dashboard: iSCSI form does not support IPv6

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>