ceph-volume: add db and wal support to raw mode

Using the raw mode, the OSD preparation can now be called with extra
arguments to specify a block device for either rocksdb db or wal.

The code expects the device to be available and no validation will be
performed. Also, the entire block device will be consumed.

The goal is run this on Kubernetes with Rook and OSD are running on PVC.
Users will request PVC from a certain size, the request will be
acknowledged and a block will be created, then attached to the machine.
Later on consumed by Rook's preparation job.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit c8f23c7182e2681f2bc756776a890995aca1efd2)

ceph-volume: fix raw list

There is no json anymore, just plain output.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit e99eb8cc95c5f50ec13170f202226c3581e3f5e2)

ceph-volume: remove unnecessary comment

This comment is not relevant since we will never bring systemd support
to the raw mode.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit af3f5fded95e4dcc0dfa2a0b2eec5fb2c4252b6c)

ceph-volume: remove osd-id and osd-fsid flag

These are not used since we only rely on the device path so removing
them to avid confusion.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit 525957be98a0a6ff742f6bae4450f291591f516f)

ceph-volume: remove "create" call from raw mode

This mode is intented to work when the cluster is running on Kubernetes
and OSDs are running on PVC.
So we always have a "prepare" pod and an "activate" pod.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit c7a2ecc59394a9eb323a19ff233a2e3dfe04a018)

Merge pull request #32835 from smithfarm/wip-43812-nautilus

nautilus: rgw: support radosgw-admin zone/zonegroup placement get command

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32834 from smithfarm/wip-43789-nautilus

nautilus: rgw: fix opslog operation field as per Amazon s3

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32833 from smithfarm/wip-43786-nautilus

nautilus: rgw: the http response code of delete bucket should not be 204-no-content

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32832 from smithfarm/wip-43782-nautilus

nautilus: rgw: Incorrectly calling ceph::buffer::list::decode_base64 in bucket policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32504 from smithfarm/wip-43477-nautilus

nautilus: rgw: Select the std::bitset to resolv ambiguity

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #32437 from pritha-srivastava/wip-42385-nautilus

nautilus: rgw: Adding 'iam' namespace for Role and User Policy related REST APIs.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32239 from smithfarm/wip-43300-nautilus

nautilus: rgw: fix bugs in listobjectsv1

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32825 from smithfarm/wip-43779-nautilus

nautilus: rgw: return error if lock log shard fails

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32824 from smithfarm/wip-43728-nautilus

nautilus: rgw: move forward marker even in case of many rgw.none indexes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32822 from smithfarm/wip-43576-nautilus

nautilus: rgw: fix a bug that bucket instance obj can't be removed after resharding completed

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32821 from smithfarm/wip-43574-nautilus

nautilus: rgw: fix data consistency error casued by rgw sent timeout

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32820 from smithfarm/wip-43571-nautilus

nautilus: rgw: build_linked_oids_for_bucket and build_buckets_instance_index should return negative value if it fails

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32819 from smithfarm/wip-43373-nautilus

nautilus: rgw: data sync markers include timestamp from datalog entry

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32818 from smithfarm/wip-43372-nautilus

nautilus: rgw: radosgw-admin: sync status displays id of shard furthest behind

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32759 from smithfarm/wip-43203-nautilus

nautilus: rgw: adding mfa code validation when bucket versioning status is changed

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32489 from bzed/clang-build-fix

nautilus: rgw/rgw_rest_conn.h: fix build with clang

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32057 from smithfarm/wip-43010-nautilus

nautilus: rgw multisite: fixes for concurrent version creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #32900 from smithfarm/wip-43830-nautilus

nautilus: rbd: librbd: don't call refresh from mirror::GetInfoRequest state machine

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #32842 from smithfarm/wip-43735-nautilus

nautilus: librbd: support compression allocation hints to the OSD

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32841 from smithfarm/wip-43511-nautilus

nautilus:  librbd: skip stale child with non-existent pool for list descendants

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32840 from smithfarm/wip-43508-nautilus

nautilus: rbd:  creating thick-provision image progress percent info exceeds 100%

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32839 from smithfarm/wip-43501-nautilus

nautilus: librbd: remove pool objects when removing a namespace

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32837 from smithfarm/wip-43237-nautilus

nautilus: librbd: fix rbd_open_by_id, rbd_open_by_id_read_only

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32815 from tchaikov/nautilus-pr-30284

nautilus: tests: add missing header cmath to test/mon/test_mon_memory_target.cc

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #32441 from tchaikov/nautilus-boost-1.72

nautilus: switch to boost 1.72

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

librbd: don't call refresh from mirror::GetInfoRequest state machine

Fixes: https://tracker.ceph.com/issues/43589
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit da46798ab3f56a639cc7a0b885778e8f75505b53)

Conflicts:
src/librbd/api/Mirror.cc
- C_ImageGetInfo ctor takes only two arguments in nautilus
- nautilus does not have LambdaContext as a class; use FunctionContext
  instead

nautilus: monitoring: add details to Prometheus' alerts (#32892)

nautilus: monitoring: add details to Prometheus' alerts

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

Merge pull request #32838 from smithfarm/wip-43241-nautilus

nautilus: qa: krbd_exclusive_option.sh: update for recent kernel changes

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

monitoring: add details to Prometheus' alerts

Fixes: https://tracker.ceph.com/issues/43764
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit fb51c589b5b5cd6a05cbfb08c41ca46a8941b269)

rgw: Adding 'iam' namespace for Role and User Policy related REST APIs.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 121a2695544bd56973459fad9b4f545b4ee412f8)

Conflicts:
src/common/options.cc
src/rgw/CMakeLists.txt
src/rgw/rgw_main.cc
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_rest_s3.h

Merge pull request #32781 from idryomov/wip-doc-fs-authorize-fix-nautilus

nautilus: doc/cephfs/client-auth: description and example are inconsistent

Reviewed-by: Jeff Layton <jlayton@redhat.com>

librbd: allow remove snapshot with child from non-existent pool

Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 50f3cc199e5a4ace0ccf1d755ac682aeb489a454)

Conflicts:
src/test/librbd/test_librbd.cc
- nautilus does not have test "DISABLED_TestSeqWriteAIOPP"
        test/librbd/operation/test_mock_SnapshotRemoveRequest.cc
- nautilus uses "RWLock::RLocker", instead of "std::shared_lock", with
  owner_locker

rgw: fix radosgw-admin zone/zonegroup placement get command

Signed-off-by: jiahuizeng <jhzeng93@foxmail.com>
(cherry picked from commit 9f0e461ce33808496e1df5c0706ed8b4403b7109)

Conflicts:
    src/rgw/rgw_admin.cc
- nautilus uses "store->svc.sysobj"

Merge pull request #32208 from smithfarm/wip-43280-nautilus

nautilus: rpm: add rpm-build to SUSE-specific make check deps

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #32079 from smithfarm/wip-43143-nautilus

nautilus: mds: tolerate no snaprealm encoded in on-disk root inode

Reviewed-by: Ramana Raja <rraja@redhat.com>

doc/rbd: documented 'rbd compression hint' config option

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d611a9860a5cbbdb0bf8f8d958b86fef5ea74fb2)

librbd: support compression allocation hints to the OSD

A new "rbd_compression_hint" configuration option can be applied
globally, at the pool level, or to individual images to send
the associated compression allocation hint to the OSD on write
operations.

Fixes: https://tracker.ceph.com/issues/42097
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 200fc5b2f24859109a79002e30927d457971b939)

librbd: skip stale child with non-existent pool for list_descendants

Fixes: https://tracker.ceph.com/issues/41241
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit a485eb827009b629c4c7804ec1b7191cfe6b6dd7)

librbd: fix error param of ioctx when passing to `list_descendants`

Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit b7ac8a17ef0b0602301eebea0e2e3c8032889174)

rbd:the progress percent info exceeds 100%

Signed-off-by: Xiangdong Mu <muxiangdong@inspur.com>
(cherry picked from commit 4877c8a12181ec9cbc1c1e59bea3a615e0367588)

librbd: remove pool objects when removing a namespace

Avoid leaving RBD pool objects within a pool namespace that is
being deleted.

Fixes: https://tracker.ceph.com/issues/43378
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9e29914a6a9581ea8b41f3d3efd61f254fafeeff)

Conflicts:
src/librbd/api/Namespace.cc
- "r = Mirror<I>::mode_set(ns_ctx, RBD_MIRROR_MODE_DISABLED)"
  block is missing in nautilus

qa: kernel.sh: unlock before rolling back

"rbd snap rollback" expects an unlocked image, but we may get there
locked if object map is enabled (or if lock_on_read is specified in
rbd_default_map_options).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 3aae140353a4b27dfe6e2d2ac7a49cdcdbbe5ffe)

qa: krbd_exclusive_option.sh: update for recent kernel changes

Since 5.3:
- a plain "rbd map" acquires the lock, so it's not different from
  "rbd map -o exclusive" in this regard
- if the lock is held by the exclusive peer, I/O is failed right away
  instead of blocking
- lock_timeout option is respected only by "rbd map" and not by I/O

Since 5.5:
- if the mapping is read-only, the lock isn't acquired

Added blacklisting test case, dropped lock_timeout test case.

Fixes: https://tracker.ceph.com/issues/43127
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 50933b863a1d239bbca0a25663c0dbe7ab8963ce)

librbd: fix rbd_open_by_id, rbd_open_by_id_read_only

These methods incorrectly delete ImageCtx on error, resulting
in double-free heap corruption.

Fixes: https://tracker.ceph.com/issues/43178
Signed-off-by: yangjun <yangjun@cmss.chinamobile.com>
(cherry picked from commit 3457192c24a66ba499a7c9b1747bc29c79b34636)

Merge pull request #32245 from smithfarm/wip-41106-nautilus

nautilus: mds: add command that config individual client session

Reviewed-by: Ramana Raja <rraja@redhat.com>

rgw: support radosgw-admin zone/zonegroup placement get command

Signed-off-by: jiahuizeng <jhzeng93@foxmail.com>
(cherry picked from commit 5601517c3b4ea0ad9e5debf758fedd75a00c5c7c)

rgw: fix opslog operation field as per Amazon s3

According to Amazon s3[1], current Operation field are using
http method, which should be operation name.

[1] http://docs.aws.amazon.com/AmazonS3/latest/dev/LogFormat.html

Fixes: http://tracker.ceph.com/issues/20978
Signed-off-by: Jiaying Ren <renjiaying@cmss.chinamobile.com>
(cherry picked from commit 7949f81b71b718e1c17ade5ca0393979b22260bc)

rgw: the http response code of delete bucket should not be 204-no-content

Fixes: https://tracker.ceph.com/issues/41925
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
(cherry picked from commit 710d8d31d27af05de1f17f7a44f9f0b5862b00a2)

rgw: Incorrectly calling ceph::buffer::list::decode_base64 in bucket policy

Fixes: https://tracker.ceph.com/issues/42616
Signed-off-by: GaryHyg <huygbj@inspur.com>
(cherry picked from commit f56e9146f86f107bbf245700874ea849c57b5fef)

Merge pull request #32746 from jan--f/wip-43026-nautilus

nautilus: doc/ceph-volume: initial docs for zfs/inventory and zfs/api

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

rgw: Select the std::bitset to resolv ambiguity

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit 2853a48385ce07689eac476ce7b0ff0057b1c195)

rgw: fix bugs in listobjectsv1

Fixes: https://tracker.ceph.com/issues/40663
Signed-off-by: Albin Antony <aantony@redhat.com>
(cherry picked from commit c9a8e3acb130fd0e7f355440f4802f4f065d213f)

Conflicts:
src/rgw/rgw_rest_s3.cc
- deleted "if (is_truncated && !next_marker.empty()) {" conditional
  block is different in nautilus
- white space change not relevant to nautilus
- "s->formatter->close_section()" change not needed in nautilus

cls/rgw: when object is versioned and lc transition it, the object is becoming non-current

when object is versioned and lc transition it, the object is becoming non-current

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Fixes: https://tracker.ceph.com/issues/43444
(cherry picked from commit 5f37ca283141df76a6f698ae4d7a23772b8b9e1b)

rgw: return error if lock log shard fails

Fixes: https://github.com/ceph/ceph/pull/31344
Signed-off-by: zhangshaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit 0775c1b8754f488434ec72d262d1387627e6bf62)

rgw: move forward marker even in case of many non-existent indexes

Signed-off-by: Ilsoo Byun <ilsoobyun@linecorp.com>
(cherry picked from commit 2eaac132d491ef3c6f8cb50c34fc67f56e006c07)

Conflicts:
src/rgw/rgw_rados.cc
- comment is slightly different in nautilus
- nautilus does not have "if (*is_truncated && count < num_entries)"
  conditional block

rgw: fix a bug that bucket instance obj can't be removed after resharding completed.

Fixes: https://tracker.ceph.com/issues/42691
Signed-off-by: zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit 03039d9630e65088b525e9637dfc6f8f2c0a2702)

Conflicts:
src/rgw/rgw_reshard.cc
- in master, *store has type "rgw::sal::RGWRadosStore", while in nautilus
  it has type "RGWRados", but this line appears to be merely incidental
  to the patch

rgw:[bug] civetweb mg_write with offset.

data consistency error casued by rgw sent timeout.
Incomplete data transmission and incorrect data re-transmission

Signed-off-by: 李纲彬82225 <82225@sangfor.com>
(cherry picked from commit d9fccedda90ff889fbaaa35ebacf3f3b4574ea09)

rgw: build_linked_oids_for_bucket and build_buckets_instance_index
should return negative value if it fails

Fixes: https://tracker.ceph.com/issues/43452
Signed-off-by: zhangshaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit e9e6a3664efeda9abb4ad784b98975c272667844)

rgw: data sync markers include timestamp from datalog entry

this corrects the output of 'radosgw-admin data sync status' to show the
timestamp of the last datalog entry applied

Fixes: https://tracker.ceph.com/issues/43359
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b403e011185a98a7ae2d3fb6246f0b1131a5f9a9)

radosgw-admin: sync status displays id of shard furthest behind

'radosgw-admin sync status' gives you a timestamp for the "oldest
incremental change not applied", but doesn't tell you which shard that
came from. add [shard-id] to the output

Fixes: https://tracker.ceph.com/issues/43360
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d61beb74f66c2fde29fd217bba63892567fd6ac0)

Merge pull request #32469 from zhengchengyao/diff_nautilus

nautilus: librbd: diff iterate with fast-diff now correctly includes parent

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32299 from callithea/wip-42139-nautilus

nautilus: mgr/dashboard: Standby Dashboards don't handle all requests properly

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31980 from epuertat/wip-43103-nautilus

nautilus: mgr/dashboard,grafana: remove shortcut menu

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31942 from ricardoasmarques/wip-43083-nautilus

nautilus: mgr/dashboard: Trim IQN on iSCSI target form

Reviewed-by:  Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31941 from ricardoasmarques/wip-42955-nautilus

nautilus: mgr/dashboard: Unable to set boolean values to false when default is true

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

tests: add missing header cmath to test/mon/test_mon_memory_target.cc

While building ceph, the complier complains:
=================================================================
[ 86%] Building CXX object src/test/mon/CMakeFiles/ceph_test_mon_memory_target.dir/test_mon_memory_target.cc.o
/root/ceph/src/test/mon/test_mon_memory_target.cc: In int main(int, :
/root/ceph/src/test/mon/test_mon_memory_target.cc:66:16:  was not declared in this scope
   66 |   auto stdev = sqrt(sump / results.size());
      |                ^~~~
make[2]: *** [src/test/mon/CMakeFiles/ceph_test_mon_memory_target.dir/build.make:63: src/test/mon/CMakeFiles/ceph_test_mon_memory_target.dir/test_mon_memory_target.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:15575: src/test/mon/CMakeFiles/ceph_test_mon_memory_target.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
=================================================================

This's caused by lack of header cmath which is required by
std::sqrt().
Fix it by add the header then build continues.

Fixes: https://tracker.ceph.com/issues/41739
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
(cherry picked from commit 9acfb351a3965c28d390822fd8f2ec5675b29064)

Merge pull request #32123 from dvanders/dvanders_40891

nautilus: osd: set collection pool opts on collection create, pg load

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #32070 from smithfarm/wip-42993-nautilus

nautilus: monitoring: wait before firing osd full alert

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

Merge pull request #32069 from smithfarm/wip-42999-nautilus

DNM: nautilus: mgr/pg_autoscaler: default to pg_num[_min] = 16

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32068 from smithfarm/wip-42695-nautilus

nautilus: mgr/pg_autoscaler: implement shutdown method

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32733 from leseb/cv-raw-bp

nautilus: ceph-volume: add raw (--bluestore) mode

src/rgw/rgw_rest_conn.h: fix build with clang

Building with clang requires this little fix for

[ 60%] Building CXX object src/rgw/CMakeFiles/rgw_common.dir/services/svc_zone.cc.o
cd /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/src/rgw && /usr/bin/clang++  -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D__linux__ -I/<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/src/include -I/<<PKGBUILDDIR>>/src -I/usr/include/nss -I/usr/include/nspr -I/<<PKGBUILDDIR>>/src/dmclock/support/src -isystem /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/include -isystem /<<PKGBUILDDIR>>/src/xxHash -isystem /<<PKGBUILDDIR>>/src/rapidjson/include -isystem /<<PKGBUILDDIR>>/src/rgw/services  -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wall -Wtype-limits -Wignored-qualifiers -Winit-self -Wpointer-arith -Werror=format-security -fno-strict-aliasing -fsigned-char -Wno-unknown-pragmas -Wno-unused-function -Wno-unused-local-typedef -Wno-varargs -Wno-gnu-designator -Wno-missing-braces -Wno-parentheses -Wno-deprecated-register -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -ftemplate-depth-1024 -Wnon-virtual-dtor -Wno-unknown-pragmas -Wno-ignored-qualifiers -Wno-inconsistent-missing-override -Wno-mismatched-tags -Wno-unused-private-field -Wno-address-of-packed-member -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -fPIC   -DHAVE_CONFIG_H -D__CEPH__ -D_REENTRANT -D_THREAD_SAFE -D__STDC_FORMAT_MACROS -std=c++17 -o CMakeFiles/rgw_common.dir/services/svc_zone.cc.o -c /<<PKGBUILDDIR>>/src/rgw/services/svc_zone.cc
In file included from /<<PKGBUILDDIR>>/src/rgw/services/svc_zone.cc:7:
/<<PKGBUILDDIR>>/src/rgw/rgw_rest_conn.h:441:28: error: template parameter redefines default argument
template <class T, class E=int>
                           ^
/<<PKGBUILDDIR>>/src/rgw/rgw_rest_conn.h:437:32: note: previous default template argument defined here
  template <class T, class E = int>
                               ^
1 error generated.

This fixes the backport in 16aacb8a7ffeef7a22e4c654dc6a5475d606080d

Fixes: 16aacb8a7ffeef7a22e4c654dc6a5475d606080d
Fixes: https://tracker.ceph.com/issues/43437
Signed-off-by: Bernd Zeimetz <bernd@bzed.de>

doc/cephfs/client-auth: description and example are inconsistent

According to the path restriction example, the filesystem name
should be cephfs_a, not cephfs.  Converge on cephfs_a to avoid it
being confused with with the pool tag, which is always cephfs.

This was introduced in 160c4bfeb811 ("mon/AuthMonitor: Use new osd
auth caps for ceph fs authorize").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 267c00012a6562daba385186908598f348db059a)

Conflicts:
doc/cephfs/client-auth.rst [ commit e7a7cf429ef3 ("doc:
  filesystem to file system") not in nautilus ]

Merge pull request #31956 from dzafman/wip-balancer3-nautilus

nautilus: mgr: Balancer fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #31556 from jan--f/wip-42766-nautilus

nautilus: mgr/prometheus: assign a value to osd_dev_node when obj_store is not filestore or bluestore

Reviewed-by: Nathan Cutler <ncutler@suse.com>

rgw: apply_olh_log ignores RGW_ATTR_OLH_VER decode error

this resolves test failures that return 400 Bad Request after:

  apply_olh_log failed to decode olh ver ''

because olh_init_modification_impl() is writing an empty attribute:

  bufferlist verbl;
  op.setxattr(RGW_ATTR_OLH_VER, verbl);

on such decode errors, link_epoch will be set to 0, and will correctly
be overwritten by real epochs from the olh log

Fixes: https://tracker.ceph.com/issues/39142
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 36c54a601c62894d0dccec49322cf7ea6bbb7d02)

rgw: allow apply_olh_log to rewrite the same olh version

if two zones write an object version with the same olh epoch,
apply_olh_log needs to be able to write the more-current instance at the
same epoch

Fixes: https://tracker.ceph.com/issues/39142
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d78aa3f4d4e283726d9c3b35a477b238df6b5af1)

rgw: apply_olh_log filters out older instances of same epoch

when apply_olh_log() detects multiple instances with the same epoch, we
only promote the one that the bucket index lists as more current

Fixes: https://tracker.ceph.com/issues/39142
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 15aa3ad341ab0df128464d22cd97bd33f185e41a)

rgw: factor out decode_olh_info()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ef039b27df93bf89c95724d13ba82324813c3aae)

cls/rgw: only promote instance entries if they sort after current instance

to support bidirectional replication, where each zone could create its
own instances with the same olh epoch, the sort order of those instances
should be used to determine whether an entry should be promoted over the
current instance

Fixes: https://tracker.ceph.com/issues/39142
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6f52900c817c536431f76de8f72ea9c94d5f2a99)

test/rgw: add test_concurrent_versioned_object_incremental_sync

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a078120512260c6ccf74acefe6f9f50e78803996)

rgw: adding mfa code validation when bucket versioning status is changed.

When the user changes bucket versioning status from Enabled->Suspended
and vice versa, MFA code needs to be validated, if MFA has been enabled
for the bucket.

Fixes tracker issue #42911

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit db89c4af159301710e1cc32dbd6298c1ec24b006)

ceph-volume: fix device unittest, mock has_bluestore_label, lint

This also adds a test case for has_bluestore_label.

Fixes: https://tracker.ceph.com/issues/43611
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 91c7f5189d163ca517c643b8548a2e16176392cc)

qa/cephfs: test case for timeout config of individual session

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 4b28a4a7569ab516d75752c97226f598e6229ca2)

mds: add command that config individual client session

For now, the command only can change session's timeout config

Fixes: http://tracker.ceph.com/issues/40811
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit a9c21bdaef3473d99ac362122d441b9d869230dd)

Merge pull request #32743 from leseb/fast-shutdown-bkp

nautilus: osd: add osd_fast_shutdown option (default true)

doc/ceph-volume: docs for zfs/inventory

fixes: https://tracker.ceph.com/issues/42722
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit 799673a580da20ffffdc1f2922fa5f08123a86cb)

Merge pull request #31554 from jan--f/wip-42703-nautilus

nautilus: ceph-volume: reject disks smaller then 5GB in inventory

Merge pull request #31553 from jan--f/wip-42741-nautilus

nautilus: ceph-volume: use fsync for dd command

qa/standalone/ceph-helpers.sh: remove osd down check

A kill doesn't induce a mark-down of the OSD with osd_fast_shutdown=true.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3a62d166a7af5a62d5210417a5c10f101ce75229)

qa/standalone/ceph-helpers.sh: destroy_osd: mark osd down

Stopping the OSD doesn't guarantee that it will be marked down.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ede1d3677310bd152b3493e8a84da1e648fd3ea9)

osd: add osd_fast_shutdown option (default true)

If we get a SIGINT or SIGTERM or are deleted from the OSDMap, do a fast
shutdown by exiting immediately.  This has a few important benefits:

 - We immediately stop responding (binding) to any sockets, which means
   other OSDs will immediately decide we are down (and dead!).  This
   minimizes IO interruption.
 - We avoid the complex "clean" shutdown process, which is historically a
   source of bugs.

In reality, the only purpose of the "clean" shutdown is to try to tear down
everything in memory so we can do memory leak checking with valgrind.  Set
this option to false for valgrind QA runs so we can still do that.

Not that with the new read leases in octopus, we rely on the default
behavior that a ECONNREFUSED is taken to mean that the OSD is fully dead,
so that we don't have to wait for any leases to time out.  This works in
sane environments with normal IP networks, but that behavior could
conceivably be a bad idea if there are some weird network shenanigans
going on.  If osd_fast_fail_on_connection_refused were disabled, then this
fast shutdown procedure might be *worse* than the clean shutdown because
we would have to wait for the heartbeat timeout.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cf352c3ac0bd87d8b7e0c52ac724f94576ae5aa7)

Merge pull request #32078 from smithfarm/wip-43141-nautilus

nautilus: tools/cephfs: make 'cephfs-data-scan scan_links' fix dentry's first

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32077 from smithfarm/wip-43138-nautilus

nautilus: mds: remove unnecessary debug warning

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32075 from smithfarm/wip-43001-nautilus

nautilus: tests/cephfs: Ignore "ceph.dir.pin: No such attribute errors" in getfattr tests for old kernel client

Reviewed-by: Ramana Raja <rraja@redhat.com>