Merge pull request #45557 from vshankar/wip-53911

pacific: Revert "mds: kill session when mds do ms_handle_remote_reset"

Merge pull request #45489 from cfsnyder/wip-54041-pacific

pacific: rgw: remove bucket API returns NoSuchKey than NoSuchBucket

Merge pull request #45543 from ajarr/rraja-pacific-backport-pr-44795

pacific: mgr/nfs: allow dynamic update of cephfs nfs export

Merge pull request #45502 from cfsnyder/wip-54148-pacific

pacific: rgw: RGWPostObj::execute() may lost data.

Merge pull request #45500 from cfsnyder/wip-54147-pacific

pacific: rgw/admin: fix radosgw-admin datalog list max-entries issue

Merge pull request #45497 from cfsnyder/wip-54092-pacific

pacific: rgwlc:  warn on missing RGW_ATTR_LC

Merge pull request #45495 from cfsnyder/wip-54083-pacific

pacific: librgw: make rgw file handle versioned

Merge pull request #45487 from mchangir/wip-54532-pacific

pacific: mds,client: add new getvxattr op

Merge pull request #45555 from idryomov/wip-rbd-du-validate-pacific

pacific: test/librbd: add test to verify diff_iterate size

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #44551 from lxbsz/wip-53759

pacific: mds: reset heartbeat in each MDSContext complete()

Reviewed-by: Venky Shankar vshankar@redhat.com

Merge pull request #45509 from cfsnyder/wip-54152-pacific

pacific: rgw: in bucket reshard list, clarify new num shards is tentative

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45508 from phlogistonjohn/jjm-backport-pacific-pr-44845-44870-45211

pacific: mgr/nfs: support managing exports without orchestration enabled

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #45569 from mgfritch/backport-45420-pacific

pacific: cephadm: infer the default container image during pull

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #45582 from idryomov/wip-rbd-nbd-cookie-pacific

pacific: backport nbd cookie support

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #45580 from idryomov/wip-fix-pids-limit-pacific

pacific: cephadm: Remove containers pids-limit

Reviewed-by: Adam King adking@redhat.com

Merge pull request #45477 from cfsnyder/wip-53637-pacific

pacific: test/rgw: disable cls_rgw_gc test cases with defer_gc()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45476 from cfsnyder/wip-53518-pacific

pacific: rgw: add object null point judging when listing pubsub  topics

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45484 from cfsnyder/wip-53835-pacific

pacific: rgw: document S3 bucket replication support

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45483 from cfsnyder/wip-53818-pacific

pacific: rgw: RadosBucket::get_bucket_info() updates RGWBucketEnt

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45480 from cfsnyder/wip-53654-pacific

pacific: rgw: init bucket index only if putting bucket instance info succeeds

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45478 from cfsnyder/wip-53639-pacific

pacific: rgw/amqp: add default case to silence compiler warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45463 from cfsnyder/wip-53254-pacific

pacific: rgw: fix rgw.none statistics

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45596 from neha-ojha/wip-45512-pacific

pacific: ceph/admin: s/master/main

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45532 from idryomov/wip-fix-schedule-status-pacific

pacific: mgr/rbd_support: cast pool_id from int to str when collecting LevelSpec

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

rgw: fix rgw.none statistics

Only unaccount for objects being removed if they exist.

Also, clean up some formatting.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 95b662fa176cd841e26b76341f91bf2e6c266003)

ceph/admin: s/master/main

This PR changes the name "master" to "main" so
that builds (and, I assume, a great many other
things) will not fail.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 6a1dd3a8a2f3dc9fe8615d402c9041273516ff89)

qa/workunits/rbd: test map/attach with --show-cookie/--cookie options

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 2adfe55bcce08f8f46989e9ddbf46aa74e681992)

rbd-nbd: allow user to specify cookie at map

Allow user to specify cookie of choice at the time of map

$ rbd device attach rbd-pool/image --device /dev/nbd0 \
--cookie 6f85d970-10b2-456b-8baf-676aa4d782e4 --options try-netlink

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 16404dede832f3908d48fc36c7714891bfcfde20)

rbd: add --show-cookie/--cookie option for map/attach commands

Example:
$ rbd device map rbd-pool/image --show-cookie --try-netlink --device-type nbd

$ rbd device attach rbd-pool/image --device /dev/nbd0 \
      --cookie 6f85d970-10b2-456b-8baf-676aa4d782e4 --device-type nbd

older Kernel versions can use --force to skip the cookie validation

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 6df06c7a62bdd6e9ce61b26c738dcd1509e68273)

Conflicts:
src/tools/rbd/action/Device.cc [ commit e1b4811bc324 ("rbd: add
  missing switch arguments for recognition by get_command_spec()")
  already in pacific -- re-add show-cookie ]

rbd-nbd: allow attach without --cookie for old kernel versions

For backward compatibility allow attach without --cookie option:

[root@linux-vm1]# rbd-nbd attach rbd-pool/image0 --device /dev/nbd0
/dev/nbd0

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 833e71282ad82023237c5eb407b9fa4630280c15)

rbd-nbd: show per device cookie with list command

[root@linux-vm1]# rbd-nbd list-mapped
id    pool      namespace  image   snap  device     cookie
8133  rbd-pool             image0  -     /dev/nbd0  c704cb91-c6cf-466e-a335-0e935c0d5e47

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit efeb255d107075516c95f719f9a2ec9375d5bb2f)

rbd-nbd: mandate device cookie for attach command

[root@linux-vm1]# rbd-nbd attach rbd-pool/image0 --device /dev/nbd0 \
                          --cookie c704cb91-c6cf-466e-a335-0e935c0d5e47
/dev/nbd0

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 6ca70ee4f078aba86a6edd013bd8d95f690a46a1)

rbd-nbd: provide a flag to show device cookie with map command

[root@linux-vm1]# rbd-nbd map rbd-pool/image0 --try-netlink --show-cookie
/dev/nbd0 c704cb91-c6cf-466e-a335-0e935c0d5e47

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 59c08eabf12fdcf155b54dc422fb0f67d6ad747e)

rbd-nbd: generate and send device cookie with netlink connect request

[root@linux-vm1]# rbd-nbd map rbd-pool/image0 --try-netlink
/dev/nbd0

[root@linux-vm1]# cat /sys/block/nbd0/backend
c704cb91-c6cf-466e-a335-0e935c0d5e47

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit eee8d8a32e3f39440ac417b2e9ea46d93d4a1d73)

qa/workunits/rbd: use rbd cli for device attach/detach commands

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit e2aa4128f1e451a85200441f70a13336e843dad6)

rbd: improve conditional compilation specific checks

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 4d7db9e86a246bf77e8545d0d4e220d90f48ac29)

rbd: promote rbd-nbd attach and detach at rbd integrated cli

Example:
$ rbd device attach rbd-pool/image --device /dev/nbd0 --device-type nbd --force
$ rbd device detach rpool/image --device-type nbd

for now returning EOPNOTSUPP with krbd, ggate and wnbd

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 35e5352f719857bed99caa4c76ae19bb68918c8e)

Conflicts:
src/tools/rbd/action/Device.cc [ commit e1b4811bc324 ("rbd: add
  missing switch arguments for recognition by get_command_spec()")
  already in pacific ]
src/tools/rbd/action/Wnbd.cc [ ditto ]

qa/suites/orch/cephadm: restrict test_iscsi_pids_limit to CentOS

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit f0ade57458b93f8401de8670ae62bf2295a6c40c)

[ commit 1f714da81440 ("qa: fix or add missing .qa links") not in
  pacific -- added qa/suites/orch/cephadm/workunits/task/.qa ]

cephadm: remove containers pids-limit

The default pids-limit (docker 4096/podman 2048) prevent some
customization from working (http threads on RGW) or limits the number
of luns per iscsi target.

Fixes: https://tracker.ceph.com/issues/52898
Signed-off-by: Teoman ONAY <tonay@redhat.com>
(cherry picked from commit de8b3c2676e65eb61df54c65cfd3b3af1e68da56)

Merge pull request #45426 from cfsnyder/wip-51348-pacific

pacific: rgw/notification: send correct size in COPY events

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45236 from k0ste/wip-54451-pacific

pacific: mgr/prometheus: Added `avail_raw` field for Pools DF Prometheus mgr module

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45397 from kamoltat/wip-ksirivad-pacific-backport-45078

pacific: mon/MonCommands.h: fix target_size_ratio range

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45376 from ronen-fr/wip-rf-43303-pacific

pacific: osd: fix the truncation of an int by int division

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45373 from smithfarm/wip-54276-pacific

pacific: common: fix FTBFS due to dout & need_dynamic on GCC-12

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #45323 from ljflores/wip-54466-pacific

pacific: osd: require osd_pg_max_concurrent_snap_trims > 0

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>

Merge pull request #45319 from ljflores/wip-52078-pacific

pacific: test: fix wrong alarm (HitSetWrite)

Reviewed-by: Myoungwon Oh <omwmw@sk.com>

Merge pull request #45152 from chrisphoffman/wip-53644-pacific

pacific: mon: Omit MANY_OBJECTS_PER_PG warning when autoscaler is on

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>

Merge pull request #43698 from cfsnyder/wip-52636-pacific

pacific: mon: do not quickly mark mds laggy when MON_DOWN

Reviewed-by: Venky Shankar vshankar@redhat.com
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45459 from cfsnyder/wip-53079-pacific

pacific: src/rgw: Fix for malformed url

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45457 from cfsnyder/wip-53036-pacific

pacific: rgw: cls_bucket_list_unordered() might return one redundent entry every time is_truncated is true

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45442 from cfsnyder/wip-52107-pacific

pacific: radosgw-admin: 'sync status' is not behind if there are no mdlog entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45440 from cfsnyder/wip-52071-pacific

pacific: rgw: add the condition of lock mode conversion to PutObjRentention

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45427 from cfsnyder/wip-51351-pacific

pacific: rgw/amqp: remove the explicit "disconnect()" interface

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45445 from cfsnyder/wip-52115-pacific

pacific: qa/rgw: update apache-maven mirror for rgw/hadoop-s3a

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45439 from cfsnyder/wip-52050-pacific

pacific: rgw: default ms_mon_client_mode = secure

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45434 from cfsnyder/wip-51777-pacific

pacific: rgw: forward request in multisite for RGWDeleteBucketPolicy and RGWDeleteBucketPublicAccessBlock

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45430 from cfsnyder/wip-51701-pacific

pacific: rgw: url_decode before parsing copysource in copyobject

Merge pull request #43956 from cfsnyder/wip-52782-pacific

pacific: rgw/sts: adding role name and role session to ops log.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cephadm: infer the default container image during pull

Fixes: https://tracker.ceph.com/issues/54588
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 736af2b9c29e375a184b863220641b7d254e7d04)

Merge pull request #45455 from cfsnyder/wip-52990-pacific

pacific: rgw: have "bucket check --fix" fix pool ids correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45461 from cfsnyder/wip-53158-pacific

pacific: rgw:When KMS encryption is used and the key does not exist, we should…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45453 from cfsnyder/wip-52988-pacific

pacific: rgw: document rgw_lc_debug_interval configuration option

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45450 from cfsnyder/wip-52611-pacific

pacific: rgw: clean-up logging of function entering to make thorough and consistent

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45448 from cfsnyder/wip-52348-pacific

pacific: rgw: change order of xml elements in ListRoles response

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45444 from cfsnyder/wip-52113-pacific

pacific: qa/rgw: barbican and pykmip tasks upgrade pip before installing pytz

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45438 from cfsnyder/wip-51810-pacific

pacific: RGW - Zipper - Make default args match in get_obj_state

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45432 from cfsnyder/wip-51703-pacific

pacific: rgw: fix md5 not match for RGWBulkUploadOp upload when enable rgw com…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45428 from cfsnyder/wip-51352-pacific

pacific: rgw: write meta of a MP part to a correct pool

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #45421 from cfsnyder/wip-54426-pacific

pacific: rgw: add OPT_BUCKET_SYNC_RUN to gc_ops_list, so that

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45413 from cfsnyder/wip-54035-pacific

pacific: qa: move certificates for kmip task into /etc/ceph

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45411 from cfsnyder/wip-54494-pacific

pacific: rgw: fix segfault in UserAsyncRefreshHandler::init_fetch

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #44729 from cfsnyder/wip-53849-pacific

pacific: rgwlc:  remove lc entry on bucket delete

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #44885 from BenoitKnecht/wip-54118-pacific

pacific: mgr: Fix ceph_daemon label in ceph_rgw_* metrics

Reviewed-by: BenoitKnecht <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #45351 from nmshelke/wip-54478-pacific

pacific: ceph-fuse: perform cleanup if test_dentry_handling failed

Reviewed-by: Venky Shankar vshankar@redhat.com
Reviewed-by: Kotresh HR khiremat@redhat.com

Merge pull request #45227 from rkachach/backport_of_pr_44928

pacific: mgr/cephadm: validating service_id for MDS

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Adam King adking@redhat.com

Merge pull request #45205 from kotreshhr/pacific-mgr-volumes-backports-0

pacific: mgr/volumes: A few mgr volumes pacific backports

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar vshankar@redhat.com

Revert "mds: kill session when mds do ms_handle_remote_reset"

This reverts commit 749ee198ecfffac67a896448229edce2e19ba633.

Signed-off-by: Venky Shankar <vshankar@redhat.com>

test/librbd: add test to verify diff_iterate size

Add test case to verify diff size values of image and multiple
snapshots.

Fixes: https://tracker.ceph.com/issues/54440
Signed-off-by: Christopher Hoffman <choffman@redhat.com>
(cherry picked from commit d4e44df1be2bafa1c0ceabc73bb7243104fc7ad4)

cephfs/test_nfs: Add test for dynamic update export

Add test to verify that the NFS servers don't restart when the
access type of a CephFS NFS export is updated.

And check the NFS servers are restarted when the pseudo path of
a CephFS NFS export is updated.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit d7904e78d13c24b305b4e83f15ee2b86ab2085e5)

cephfs/test_nfs: fix _write_to_read_only_export()

... to fail the test if write to read only export succeeded.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 3c22f6eb716b4b4f2817566a5b14350f7dd3310e)

mgr/nfs: Don't notify ganesha rados object

... to update export if the ganesha server is going to be
restarted. The ganesha server restart will reload the exports.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 4df294b7fafc308611b75c4656a40c42d1d95361)

mgr/nfs: allow dynamic update of cephfs nfs export

mgr/nfs module's apply_export() method is used to update an existing
CephFS NFS export. The method always restarted the ganesha service (
ganesha server cluster) after updating the export object and notifying
the ganesha servers to reload their exports. The restart temporarily
affected the clients connections of all the exports served by the
ganesha servers.

It is not always necessary to restart the ganesha servers. Only
updating the export ID, path, or FSAL block of a CephFS NFS export
requires a restart. So modify apply_export() to only restart the
ganesha servers for such export updates.

The mgr/nfs module creates a FSAL ceph user with read-only or
read-write path restricted MDS caps for each export. To change the
access type of the CephFS NFS export, the MDS caps of the export's FSAL
ceph user must also be changed. Ganesha can dynamically enforce an
export's access type changes, but Ceph server daemons can't dynamically
enforce changes in caps of the Ceph clients. To allow dynamic updates
of CephFS NFS exports, always create a FSAL Ceph user with read-write
path restricted MDS caps per export. Rely on the ganesha servers to
enforce the export access type changes for the NFS clients.

Fixes: https://tracker.ceph.com/issues/54025
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 2415e03217b4afe9e430863da0b4503be254c425)

qa/workunits/rbd/cli_generic.sh: relax trash purge schedule status assert

Commit 08df6e0fd006 ("qa/workunits/rbd: expand LevelSpec parsing
coverage") didn't account for images with a separate data pool.  This
was missed because of small-cache-pool.yaml breakage.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 90a90ad47dd3140c796ef4da7263c9633d34e841)

rbd: fix a typo in "rbd trash purge schedule ls" formatted output

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit beffb7f9c6a4a19b6c4cfb65d04386575af50961)

qa/workunits/rbd: expand LevelSpec parsing coverage

Invoke "rbd mirror snapshot schedule ls -R" and "rbd mirror snapshot
schedule status" commands on all levels, consistently.  In particular,
make sure that an image level schedule is listed for a recursive query
at the pool level both before and after the schedule kicks in:

  $ rbd create --size 1G --mirror-image-mode snapshot -p foo bar
  $ rbd mirror snapshot schedule add -p foo --image bar 1m
  $ rbd mirror snapshot schedule ls -p foo -R
  POOL  NAMESPACE  IMAGE  SCHEDULE
  foo              bar    every 1m
  <wait for schedule to become visible in status>
  $ rbd mirror snapshot schedule ls -p foo -R
  POOL  NAMESPACE  IMAGE  SCHEDULE
  foo              bar    every 1m

Also, make sure that pool and image level status queries work:

  $ rbd mirror snapshot schedule status -p foo
  SCHEDULE TIME        IMAGE
  2022-03-04 07:14:00  foo/bar
  $ rbd mirror snapshot schedule status -p foo --image bar
  SCHEDULE TIME        IMAGE
  2022-03-04 07:14:00  foo/bar

Both of these issues are fixed by the previous commit.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 08df6e0fd00689649b802e975df320efe3b07c83)

rgw: in bucket reshard list, clarify new num shards is tentative

With dynamic bucket index resharding, when the average number of
objects per shard exceeds the configured value, that bucket is
scheduled for reshard. That bucket may receive more new objects before
the resharding takes place. As a result, the existing code
re-calculates the number of new shards just prior to resharding,
rather than waste a resharding opportunity with too low a value.

The same holds true for a user-scheduled resharding.

A user reported confusion that the number reported in `radosgw-admin
reshard list` wasn't the number that the reshard operation ultimately
used. This commit makes it clear that the new number of shards is
"tentative". And test_rgw_reshard.py is updated to reflect this
altered output.

Additionally this commit adds some modernization and efficiency to the
"reshard list" subcommand.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit aa0071ce8b8594b92c0bed2be7a9bf35bfff8cac)

Conflicts:
src/rgw/rgw_admin.cc

Cherry-pick notes:
- Confict caused by reshard.list not having dout prefix provider arg in Pacific

mgr/nfs: remove redundant check

Remove the extra check of the cluster id from _apply method. As _apply
is a "private" method that should be only called from other private
methods that have already validated the cluster_id. It also removes
a dependency on the orch-requiring func available_clusters.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit dd5a47f83349c8f2b539ba3881f58f5270024cbb)

mgr/nfs: fix unintentional recursion

The `exports` property of the ExportMgr exists to cache the exports
configuration found in the .nfs namespace. Using that property
within the property method is probably not intentional and is probably
only working due to the lucky construction of the _exports dict
immediately after the None check so that the _exports dict is returned
(and is a mutable type).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit daa455cd168d62cd8fbcaba4d7aa79b56e68ef0d)

mgr/nfs: add known_cluster_ids to generalize nfs cluster id fetching

The changes to the nfs module in 8c711afc are working but when I began
writing more test automation I found a few more places in the
export-configuration code path relying on the orchestration module
only. This change generalizes the logic to source nfs clusters from
orchestration when it's enabled but from the .nfs pool when
orchestration is disabled. It then uses that call when loading
the exports cache on the ExportMgr object.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 4d09660dea5696e5085a75694968aafe9253f47a)

doc/mgr/nfs: document that nfs exports related mgr call requirements

A recent change in the mgr/nfs module should enable the functioning
of export management commands/API calls as long as the rados namespaces
and objects have been already established. Document this fact, noting
that now only the `ceph nfs cluster ...` calls *require* an
orchestration module.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit b5b3e0bcb5e2a27375f50f7717786a3928cba711)

mgr/nfs: support managing exports without orchestration enabled

This change allows the `ceph nfs export ...` commands to function
without the entire mgr/nfs subsystem requiring orchestration to be
enabled.  When there's no orchestration available, the code falls back
to examining the namespaces in the ".nfs" rados pool to determine what
cluster_id values are valid.

This change does not add support for creating the rados objects and
namespace needed to manage a nfs cluster. As discussed with the
orchestration group on 2022-01-22, rook does not need the mgr module to
establish the namespace. So, for now, we'll defer the work needed to
create the namespace/objects when orchestration is disabled.

Fixes: https://tracker.ceph.com/issues/54043
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 8c711afc4ab898942a2569b619eb8379ee02ffba)

mgr/nfs: fix typo in error message

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 56323a2625133d5a53bf1ee1662346daa1b4f09b)

mgr/nfs: add unit test for normalize_path

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit ffa95fbc796aa5c00eaa32138291c0ef2a48949a)

mgr/nfs: change method format_path to function normalize_path

This function was not using self and thus has no need to be a method.
While we're at it, rename it to normalize_path because that's what
it is doing.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit f91dd1bf7bfab251d671a30d622bb544a4ce37d0)

mgr/nfs: clean up rados object naming code

The naming of rados objects used to store the nfs config was spread
all over the code, including inline f-strings, not-static methods,
etc.
This change unifies the naming by putting constant string prefixes
and name generating functions into the utils.py file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 88266144423e6876dc392bc6ea59e32393024323)

mgr/nfs: make _check_rados_notify a function

This was previously a staticmethod. This static method was only used by
NFSRados object. Staticmethods are nearly always better implemented as
functions, which is done so here.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit c51a6755b52954910512f804cde9e5255c1db9e7)

mgr/nfs: limit dependency of NFSRados object

Previously, the NFSRados object accepted the "Module" as the
first argument but only used the rados attribute (type rados.Rados).
It's better to limit the scope of types when reasonably possible
so we can see what the true dependencies are. So we restrict
NFSRados to accepting a rados.Rados as the argument.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d94b63830d94f21ba276452844a46d21e084fb3f)

rgw: RGWPostObj::execute() may lost data.

Signed-off-by: Lei Zhang <1091517373@qq.com>
(cherry picked from commit f241a330dcb5968f9ec1de1a382572258cb6daac)

rgw/admin: fix radosgw-admin datalog list max-entries issue

Fixes: https://tracker.ceph.com/issues/54116
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit bd429ed9bec8aa4dc17c61a07e30987f50f7e5f6)

rgwlc:  warn on missing RGW_ATTR_LC

This should not happen.  If it does (e.g., due to damaged bucket_info),
log the event to assist with debugging.

Fixes: https://tracker.ceph.com/issues/53728
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit ae1a75c09d11d8f0b626c781112c35de353c0c89)