Merge pull request #53634 from vshankar/wip-62584-pacific

pacific: mds: blocklist clients with "bloated" session metadata

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53552 from batrick/wip-62906-pacific

pacific: mds,qa: some balancer debug messages (<=5) not printed when debug_mds is >=5

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #54604 from lxbsz/wip-63586-p

pacific: osd: log the number of extents for sparse read

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

osd: add more debug logs for sparse read

This will be very important to get to know what exactly has happened
when client get a very large number of extents.

URL: https://tracker.ceph.com/issues/63586
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 1cf5ecb93fb73a5ed4a82935f635f98efccf7de6)

Merge pull request #54542 from aaSharma14/wip-63570-pacific

pacific: mgr/dashboard: Consider null values as zero in grafana panels

Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: Consider null values as zero in grafana panels

After upgrading from RHCS4 to RHCS5..some of the grafana charts broke.
This is because in RHCS5 we do not generate the metrics if its value is
zero as a result the null value from that metric breaks the grafana
charts or graphs. This PR is to fix the above mentioned issue.

Fixes: https://tracker.ceph.com/issues/63088
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 6f3f58cb8e4ce100cc7186858465b4d11d5c2c49)

Merge pull request #54010 from k0ste/wip-53152-pacific

pacific: radosgw-admin: allow 'bi purge' to delete index if entrypoint doesn't exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54160 from cbodley/wip-61351-pacific

pacific: rgw/s3: ListObjectsV2 returns correct object owners

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53764 from cbodley/wip-63043-pacific

pacific: rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53443 from trociny/wip-61872-pacific

pacific: rgw: pick http_date in case of http_x_amz_date absence

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #52996 from k0ste/wip-59026-pacific

pacific: rgw: use unique_ptr for flat_map emplace in BucketTrimWatche

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #52605 from cbodley/wip-62138-pacific

pacific: rgw: s3 object lock avoids overflow in retention date

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54016 from k0ste/wip-57199-pacific

pacific: rgw: 'bucket check' deletes index of multipart meta when its pending_map is nonempty

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54014 from k0ste/wip-59361-pacific

pacific: rgw: fix rgw cache invalidation after unregister_watch() error

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53775 from cbodley/wip-63055-pacific

pacific: rgw: improve buffer list utilization in the chunkupload scenario

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53772 from cbodley/wip-63052-pacific

pacific: rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53769 from cbodley/wip-63049-pacific

pacific: rgw: s3website doesn't prefetch for web_dir() check

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53759 from cbodley/wip-63058-pacific

pacific: rgw: fix unwatch crash at radosgw startup

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53593 from trociny/wip-58478-pacific

pacific: rgw: fix FP error when calculating enteries per bi shard

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53474 from k0ste/wip-55701-pacific

pacific: radosgw-admin: don't crash on --placement-id without --storage-class

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53472 from k0ste/wip-57635-pacific

pacific: rgw: Drain async_processor request queue during shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53439 from k0ste/wip-62823-pacific

pacific: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53410 from trociny/wip-62308-pacific

pacific: rgw/sync-policy: Correct "sync status" & "sync group" commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53400 from trociny/wip-62751-pacific

pacific: rgw: fix 2 null versionID after convert_plain_entry_to_versioned

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53376 from jzhu116-bloomberg/wip-59692-pacific

pacific: rgw/notification: remove non x-amz-meta-* attributes from bucket notifications

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #53356 from k0ste/wip-53658-pacific

pacific: rgw: fix UploadPartCopy error code when src object not exist and src bucket not exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #52936 from k0ste/wip-58902-pacific

pacific: rgw: Fix Browser POST content-length-range min value

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #52797 from cbodley/wip-62300-pacific

pacific: rgw: retry metadata cache notifications with INVALIDATE_OBJ

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #52729 from theanalyst/wip-58817

pacific: rgw: swift : check for valid key in POST forms

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #52113 from cbodley/wip-61728-pacific

pacific: rgw/beast: add max_header_size option with 16k default, up from 4k

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54167 from cbodley/wip-58238-pacific

pacific: rgw: beast frontend checks for local_endpoint() errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54120 from dparmar18/wip-63269-pacific

pacific: mds: report clients laggy due laggy OSDs only after checking any OSD is laggy

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53916 from kotreshhr/wip-63164-pacific

pacific: pybind/mgr/volumes: log mutex locks to help debug deadlocks

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53576 from mchangir/wip-57157-pacific

pacific: doc/cephfs: note regarding start time time zone

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #53556 from batrick/wip-62731-pacific

pacific: mds: add event for batching getattr/lookup

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53555 from batrick/wip-62897-pacific

pacific: qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53550 from batrick/wip-62902-pacific

pacific: mds: log message when exiting due to asok command

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #53495 from lxbsz/wip-62859

pacific: mds: fix deadlock between unlinking and linkmerge

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53486 from batrick/wip-62854-pacific

pacific: qa: ignore expected cluster warning from damage tests

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53453 from joscollin/wip-62834-pacific

pacific: cephfs-top: include the missing fields in --dump output

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #52852 into pacific

* refs/pull/52852/head:
mds: remove calculating caps after adding revokes back
test/libcephfs: add test case for revoking caps
client: issue a cap release immediately if no cap exists
mds: add the revoking caps back to _revokes list
mds: move confirm_receipt() to Capability.cc

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: typo fix when checking for perf counter - s/md_thresh_evicted/mdthresh_evicted

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5856a1e6b72dc77d3b4d9109385481b924e4fe36)

qa: lower mds_session_metadata_threshold for tests

... and increase the number of files that are created so as to
hit the threshold with a high probability.

Fixes: http://tracker.ceph.com/issues/62873
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 92200d9d10d30ed35f640fd1120be65577a3ce64)

PendingReleaseNotes: add a note for `mds_session_metadata_threshold` mds config

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit ac3ab1a203ad65c7db07bc2c02acb5620800fb25)

test: add test to verify that a buggy client is blocklisted

... when its session metadata is bloated due to buildup of
`completed_requests`.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 84df4b3d0c9e767a74cf5af80e8138239992df2c)

mds: add perf counter to track number of sessions evicted due to metadata threshold being exceeded

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 59dd587ddb3bc95a95d0fb8715511d194181ec6f)

mds: blocklist clients with "bloated" session metadata

Buggy clients (or maybe a MDS bug) causes a huge buildup of
`completed_requests` metadata in its session information.
This could cause the MDS to go read-only when its flushing
session metadata to the journal since the bloated metadata
causes the ODSOp payload to exceed the maximum write size.

Blocklist such clients so as to allow the MDS to continue
servicing requests.

Fixes: http://tracker.ceph.com/issues/61947
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bc6814d72a9fbec9c41ed75aee2314666cfca34b)

 Conflicts:
src/common/options/mds.yaml.in
src/mds/MDSRank.cc

Pacific uses old-style config file (src/common/options.cc), so
adjust to that.

Merge pull request #54268 from ronen-fr/wip-63372-pacific

pacific: osd: fix use-after-move in build_incremental_map_msg()

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #53662 from lxbsz/wip-62523

pacific: ceph: allow xlock state to be LOCK_PREXLOCK when putting it

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53645 from vshankar/wip-61803-pacific

pacific: cephfs-journal-tool: disambiguate usage of all keyword (in tool help).

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53640 from vshankar/wip-62949-pacific

pacific: cephfs-mirror: do not run concurrent C_RestartMirroring context

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53362 from k0ste/wip-57110-pacific

pacific: mds: replacing bootstrap session only if handle client session message

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53270 from mchangir/wip-59001-pacific

pacific: cephfs_mirror: correctly set top level dir permissions

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53169 from leonid-s-usov/bp/cap-throttle-event/pacific

pacific: mds/Server: mark a cap acquisition throttle event in the request

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #54294 from ajarr/wip-63385-pacific

pacific: qa/suites/rbd: add test to check rbd_support module recovery

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #54293 from ajarr/wip-63382-pacific

pacific: mgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #54256 from pkalever/wip-63351-pacific

pacific: rbd-nbd: fix stuck with disable request

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mgr/rbd_support: remove CreateSnapshotRequests __del__()

There is no need for CreateSnapshotRequests.__del__() that calls
CreateSnapshotRequests.wait_for_pending().
MirrorSnapshotScheduleHandler.shutdown() already calls
CreateSnapshotRequests.wait_for_pending().

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit fed1e87685a698876cf167b3681327e5b0066ee6)

Conflicts:
       src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py
 - Above conflict was due to commit e4a16e2
   ("mgr/rbd_support: add type annotation") not in pacific

mgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock

The MirrorSnapshotScheduleHandler's run thread issues asynchronous
create snapshot requests using a CreateSnapshotRequests instance. When
the thread invokes a CreateSnapshotRequests instance's get_ioctx(),
the instance's class variable lock is acquired. With the class
variable lock held, the garbage collection of a CreateSnapshotRequests
instance may race in the thread. The thread would then call
CreateSnapshotRequests __del__() that tries to acquire the class
variable lock that the thread already holds. Fix this
recursive deadlock by converting the CreateSnapshotRequests lock from
a class variable to an instance variable. There is no need to share
the lock across CreateSnapshotRequests instances.

Also convert MirrorSnapshotScheduleHandler, PerfHandler and
TrashPurgeScheduleHandler class variables to instance variables
that don't need to be shared across the instances.

Fixes: https://tracker.ceph.com/issues/62994
Signed-off-by: Ramana Raja <rraja@redhat.com>
Co-Authored-By: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 4452bc22d1c6c8499cf55d6e39090adf7ae1dcbf)

 Conflicts:
src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py
src/pybind/mgr/rbd_support/perf.py
src/pybind/mgr/rbd_support/trash_purge_schedule.py
 - Above conflicts were due to commit e4a16e2
   ("mgr/rbd_support: add type annotation") not in pacific

Merge pull request #53970 from Matan-B/wip-63179-pacific

pacific: osd/OSD: introduce reset_purged_snaps_last

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge PR #53103 into pacific

* refs/pull/53103/head:
libcephsqlite: fill 0s in unread portion of buffer

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #53984 from sseshasa/wip-63185-pacific

pacific: mon/ConfigMonitor: Show localized name in "config dump --format json" output

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #53430 from pdvian/wip-fix-scrubmsg

pacific: osd/scrub: Fix scrub starts messages spamming the cluster log

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

qa/suites/rbd: add test to check rbd_support module recovery

... on repeated blocklisting of its client.

There were issues with rbd_support module not being able to recover
from its RADOS client being repeatedly blocklisted. This occured for
example in clusters with OSDs slow to process RBD requests while the
module's mirror_snapshot_scheduler was taking mirror snapshots by
requesting exclusive locks on the RBD images and workloads were running
on the snapshotted images via kernel clients.

Fixes: https://tracker.ceph.com/issues/62891
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 2f2cd3bcff82afc3a4d251143eb462e700e7fc60)

osd: fix use-after-move in build_incremental_map_msg()

Fixes: https://tracker.ceph.com/issues/63310
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit 9e2b8b0e8235b36e55310aab49b8f760e8d57cad)

Merge pull request #53135 from ifed01/wip-ifed-verbose-open-col-pac

pacific: osd,bluestore: gracefully handle a failure during meta collection load

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #53587 from ifed01/wip-ifed-vselector-53906-pac

pacific: bluestore: Fix problem with volume selector

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #52948 from ifed01/wip-ifed-fix-55260-pac

pacific: os/bluestore: don't require bluestore_db_block_size when attaching new

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

test/librbd/fsx: wait for resize to propagate in krbd_resize()

With this changes resize request will not be blocked until the resize is
completed. Because of this the fsx test fails as it assumes that the
request to resize immediately implies changes on the device size.

Hence we have to add a wait in resize handler of fsx for the device to
actually get resized.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit 6f3d0f570f1a262b06d4c661582091d8ddb11bfa)

rbd-nbd: fix stuck with disable request

Problem:
-------
Trying to disable any feature on an rbd image mapped with nbd leads to stuck
in rbd-nbd.

The rbd-nbd registers a watcher callback to detect image resize in
NBDWatchCtx::handle_notify(). The handle_notify calls image info method, which
calls refresh_if_required and it got stuck there.

It is getting stuck in ImageState::refresh_if_required() because
DisableFeaturesRequest issues update notifications while still holding onto
the exclusive lock with everything that has to do with it blocked.

Solution:
--------
Set only notify flag as part of NBDWatchCtx::handle_notify() and handle
the resize detection part as part of a different thread.

Fixes: https://tracker.ceph.com/issues/58740
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
(cherry picked from commit dbb4daff404c5d2da32c33f4e852e84a257c0b8d)

Merge pull request #54053 from idryomov/wip-63028-pacific

pacific: pybind/rbd: don't produce info on errors in aio_mirror_image_get_info()

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #54144 from rishabh-d-dave/wip-62879-pacific

pacific: cephfs: upgrade cephfs-shell's path wherever necessary

rgw: beast frontend checks for local_endpoint() errors

socket.local_endpoint() throws on error. use the error_code overload
instead and return on failure

Fixes: https://tracker.ceph.com/issues/57784
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 60af907c91210f60d0009318b8ca2ccd87941bb9)

rgw/s3: ListObjectsV2 returns correct object owners

Fixes: https://tracker.ceph.com/issues/59471
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0761607ef11bce3bc879358903d34802bb90f7cf)

qa/tasks/ceph_manager: thrash - add reset_purged_snaps_last

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 680e088b8d48e51dfb3aaa4207de67025a0fdabd)

osd/OSD: introduce reset_purged_snaps_last

When the OSD preboots it sends a MMonGetPurgedSnaps message to
the monitor (`_get_purged_snaps`).
The monitor will reply with all the purged snapshots that their purged_epoch_ is in the
range of superblock.purged_snaps_last + 1 up to the last superblock.current_epoch + 1.
When the OSD will handle the reply from the mon (`handle_get_purged_snaps_reply`)
it will call `record_purged_snaps` to write those purged snapshots in the
OSD store as well (PSN_ keys).

Once purged_snaps_last is reset, in the following OSD reboot, the snapshots that were marked as
purged (purged_snaps_ keys) in the mon's store will be also marked,
correspondingly, in the OSD store.
That way `scrub_purged_snaps` will be able to re-trim the snapshots that weren't
marked as purged in the OSD side (for some reason)

Fixes: https://tracker.ceph.com/issues/62981
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 120ed0f0e8f65c18bfcd1d649617770c2c5af663)
Manual conflict fixes: 'scrubdebug' command was removed since it's
                       not part of the original commit.

                       write_superblock() parameters were changed

Merge pull request #53758 from cbodley/wip-63040-pacific

pacific: [CVE-2023-43040] rgw: Fix bucket validation against POST policies

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49477 from aaSharma14/wip-58299-pacific

pacific: mgr/dashboard: Fix CephPoolGrowthWarning alert

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

cephfs: upgrade cephfs-shell's path wherever necessary

Commit dc69033763cc116c6ccdf1f97149a74248691042 moves cephfs-shell from
"<CEPH-REPO-ROOT>/src/tools/cephfs/" to
"<CEPH-REPO-ROOT>/src/tools/cephfs/shell" but cephfs-shell's location in
src/vstart.sh and qa/tasks/cephfs/test_cephfs_shell.py is left
un-updated. This produces a broken vstart_environment.sh and broken
export command in test_cephfs_shell.py.

Introduced-by: dc69033763cc116c6ccdf1f97149a74248691042
Fixes: https://tracker.ceph.com/issues/58795
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 48ef0444774934dd6d0d3e026142d95e4098bebd)

 Conflicts:
qa/tasks/cephfs/test_cephfs_shell.py
- Comment present at the top of file was different in Pacific
  compared to main branch.

Merge pull request #53808 from cfsnyder/wip-62945-pacific

pacific: rgw: add radosgw-admin bucket check olh/unlinked commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53784 from idryomov/wip-63010-pacific

pacific: qa/suites/krbd: stress test for recovering from watch errors

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #53562 from cfsnyder/wip-58787-pacific

pacific: rgwlc: prevent lc for one bucket from exceeding time budget

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #53295 from ajarr/wip-62686-pacific

pacific: librbd: kick ExclusiveLock state machine on client being blocklisted when waiting for lock

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #53274 from idryomov/wip-61707-pacific

pacific: librbd: make CreatePrimaryRequest remove any unlinked mirror snapshots

Reviewed-by: Ramana Raja <rraja@redhat.com>

qa: enhance test cases

Fixes: https://tracker.ceph.com/issues/63105
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 9005451882371948359a1466fca10256476c5c37)

mds: erase clients getting evicted from laggy_clients

Fixes: https://tracker.ceph.com/issues/63105
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 754b6022fb9fda075d38dcb1d058482f75dcff4d)

mds: report clients laggy due laggy OSDs only after checking any OSD is laggy

Fixes: https://tracker.ceph.com/issues/63105
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 8a5677f956d1b18ebae22c27b690b83e82db13cc)

Merge pull request #53978 from adk3798/wip-63115-pacific

pacific: mgr/cephadm: ceph orch add fails when ipv6 address is surrounded by square brackets.

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #53977 from adk3798/wip-62802-pacific

pacific: cephadm: run tcmu-runner through script to do restart on failure

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #53975 from adk3798/wip-62469-pacific

pacific: cephadm: add tcmu-runner to logrotate config

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #53974 from adk3798/wip-62448-pacific

pacific: mgr/cephadm: Add "networks" parameter to orch apply rgw

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #53469 from adk3798/pacific-tcmu-custom-configs

pacific: cephadm: make custom_configs work for tcmu-runner container

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #52413 from adk3798/wip-61686-pacific

pacific: python-common/drive_group: handle fields outside of 'spec' even when 'spec' is provided

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #52412 from adk3798/wip-61683-pacific

pacific: python-common/drive_selection: lower log level of limit policy message

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #52411 from adk3798/wip-61544-pacific

pacific: cephadm: Adding support to configure public_network cfg section

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #52083 from adk3798/wip-61677-pacific

pacific: cephadm: allow ports to be opened in firewall during adoption, reconfig, redeploy

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

cephadm: make custom_configs work for tcmu-runner container

This is intended to be a temporary workaround to make
custom config files be able to be mounted into
the tcmu-runner container. The hope is to refactor
cephadm's iscsi handling for squid, but a patch
like this could be useful for iscsi in older
releases where currently custom config files
are unusable for the tcmu-runner container

What this patch actually does is have us write the
custom config files to a dir for the tcmu-runner
container so that the rest of the logic works without
change. I thought this would be easier to remove later
than a patch that integrates more with the container
mounts or general deployment

The use case in mind is something like

service_type: iscsi
service_id: foo
service_name: iscsi.foo
placement:
  hosts:
  - host1
custom_configs:
  -  mount_path: /etc/tcmu/tcmu.conf
     content: |
       log_level = 4
spec:
  api_password: admin
  api_port: 5000
  api_user: admin
  pool: foo

which would allow users to modify the logging of the
tcmu-runner container for debugging purposes

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit de92392708bf456bba975cc18b3138035d79ae05)

rgw: 'bucket check' deletes index of multipart meta when its pending_map is noempty

Signed-off-by: Huber-ming <zhangsm01@inspur.com>
(cherry picked from commit 67fc2bd386fe6224b2abd92a0b043ccfcca50ae7)

Conflicts:
  - path: src/rgw/rgw_rados.cc
    comment: manually resolve conflicts

mgr/rbd_support: make type hits on aio_mirror_image_*() callbacks better

Make it clear that mirror mode, mirror info and snap ID can be None if
the respective operation fails.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 01fff6a72a328459c1d153e5dc1de6a34e48a82f)

Conflicts:
src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py [ commit
  e4a16e261370 ("mgr/rbd_support: add type annotation") not in
  pacific ]

pybind/rbd: don't produce info on errors in aio_mirror_image_get_info()

Check completion return value before attemting to decode c_info.
Otherwise we are guaranteed to access invalid memory in decode_cstr()
while trying to compute global_id string length when the client is
blocklisted for example.

Fixes: https://tracker.ceph.com/issues/63028
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit a81bd2db3af4d7b53736be8e42a3eaa53028d60c)