doc/cephadm: clean "Adv. OSD Service Specs"

Clean the English grammar and the .RST in the section "Advanced OSD
Service Specifications" in the file doc/cephadm/services/osd.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit f6b56f9ee5d9f0b54dcaf1832d7a977895eb9c94)

Merge pull request #59035 from sseshasa/wip-67355-squid

squid: osd/scheduler: add mclock queue length perfcounter

Merge pull request #58785 from sseshasa/wip-67108-squid

squid: qa/suites/rados/verify/validater: increase heartbeat grace timeout

Merge pull request #60644 from zdover23/wip-doc-2024-11-07-backport-60643-to-squid

squid: doc/cephadm: link to "host pattern" matching sect

doc/cephadm: link to "host pattern" matching sect

Link to the "Placement by Pattern Matching" section in
doc/cephadm/services/index.rst from the "Advanced OSD Service
Specifications" section in doc/cephadm/services/osd.rst.

Fixes: https://tracker.ceph.com/issues/68812
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit cdfa299a2511ee35dfe0a857702af6d3ba257efd)

Merge pull request #60637 from zdover23/wip-doc-2024-11-06-backport-56599-to-squid

squid: doc/cephadm: Update operations.rst

doc/cephadm: Update operations.rst

Fixed typo 'no' to 'not' in sentence "If this test fails, cephadm will
no be able to manage services on that host."

Signed-off-by: rhkelson <119636079+rhkelson@users.noreply.github.com>
(cherry picked from commit 34bc35a73443fa56908395a83ea2dd404a8fc4a6)

Merge pull request #59217 from ifed01/wip-ifed-bs-fix-unused-squid

squid: os/bluestore: Mute warnings

Reviewed-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>

Merge pull request #58740 from ifed01/wip-ifed-async-compact-squ

squid: os/bluestore: enable async manual compactions

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #58676 from ifed01/wip-ifed-many-many-extents-read-squi

squid: blk/aio: fix long batch (64+K entries) submission

Reviewed-by: Md Mahamudur Rahaman Sajib <mahamudur.sajib@croit.io>

Merge pull request #57819 from NitzanMordhai/wip-66319-squid

squid: objclass: deprecate cls_cxx_gather

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #59974 from ronen-fr/wip-rf-59944-squid

squid: qa/standalone/scrub: increase status updates frequency

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #60609 from zdover23/wip-doc-2024-11-05-backport-60608-to-squid

doc/rgw/notification: add missing admin commands

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rgw/notification: add missing admin commands

also remove obsolete admin commands

Fixes: https://tracker.ceph.com/issues/68818
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit ec272797c4caa5cf62319cdec104cf11575727b1)

Merge pull request #59398 from lxbsz/wip-67432

squid: qa/cephfs: fix TestRenameCommand and unmount the clinet before failin…

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59406 from joscollin/wip-67668-squid

squid: cephfs_mirror: update peer status for invalid metadata in remote snapshot

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59487 from vshankar/wip-67775-squid

squid: client: calls to _ll_fh_exists() should hold client_lock

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59517 from batrick/wip-67745-squid

squid: mds: encode quiesce payload on demand

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59521 from batrick/wip-67806-squid

squid: qa: load all dirfrags before testing altname recovery

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59402 from Matan-B/wip-67664-squid

squid: mon/OSDMonitor: Add force-remove-snap mon command

Merge pull request #59284 from k0ste/wip-65042-squid

squid: librados: use CEPH_OSD_FLAG_FULL_FORCE for IoCtxImpl::remove

Merge pull request #59156 from badone/wip-tracker-59380-admin-socket-injectfull-squid

squid: qa/suites/rados: Cancel injectfull to allow cleanup

Merge pull request #59065 from YiteGu/wip-blk-discard-squid

squid: a series of optimizations for kerneldevice discard

Merge pull request #58587 from YiteGu/wip-64606-squid

squid: client: check mds down status before getting mds_gid_t from mdsmap

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59097 from batrick/wip-67438-squid

squid: mds: only authpin on wrlock when not a locallock

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59306 from batrick/wip-67613-squid

squid: mon/MonClient: handle ms_handle_fast_authentication return

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #60367 from rhcs-dashboard/squid-i18n

squid: mgr/dashboard: update translations for squid

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60593 from zdover23/wip-doc-2024-11-02-backport-60584-to-squid

squid: doc: s/Whereas,/Although/

doc: s/Whereas,/Although/

Change the subordinating conjunction "Whereas" followed by a comma to
the less grammatically-incorrect "Although". I've been meaning to do
this since 22 Mar 2023.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit ea1a470fb4809d312f854e72fc5f8081cbcb2c49)

Merge pull request #60545 from yuvalif/wip-yuval-fix-notif-test

squid: test/rgw/notification: fixing backport issues in the tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #60557 from zdover23/wip-doc-2024-10-30-backport-60520-to-squid

squid: doc/start: add supported Squid distros

test/rgw/notification: replace calls to get_ip_http with get_ip

this fix is complements the fix done
by: 4d74fe5e508ebdadf2d033b8b4c43d6b18986569
that was causing python errors when running the bucket notifications
http tests.
this is also adding a missing function from:
970113d4092567bae9c6fab060bc32d241f6c581
the above commit cannot be cherry-picked as-is, since it has other
dependencies to the product code (notification list json format).

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>

doc/start: add supported Squid distros

Add the distros that support Squid to the chart on
start/os-recommendations.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 2368dcb402e7b4425c1617ca12c923c2a2ad14bf)

Merge pull request #60537 from zdover23/wip-doc-2024-10-29-backport-60508-to-squid

squid: doc/rados: add blaum_roth coding guidance

Merge pull request #60525 from zdover23/wip-doc-2024-10-29-backport-60470-to-squid

squid: doc: explain the consequence of enabling mirroring through monitor co…

doc/rados: add blaum_roth coding guidance

Direct Ceph administrators using blaum_roth coding for erasure-coded
pools to change the default value of w=7 to a different value in order
to ensure that w+1 is prime.

This information was provided to the Ceph upstream by Benjamin Mare in
September of 2024.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 6648d94aed5ff5643f50fcd4f3f07d97b846e885)

Merge pull request #60486 from guits/wip-68599-squid

squid: orch: disk replacement enhancement

Reviewed-by: Adam King <adking@redhat.com>

doc: explain the consequence of enabling mirroring through monitor commands

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit ed3ce72fbc0b097131945cbb39caa0831401b94a)

Merge pull request #59671 from yuvalif/wip-67952-squid

squid: rgw/notifications: free completion pointer using unique_ptr

Reviewed-By: Yuri Weinstein <yweinste@ibm.com>, Daniel Gryniewicz <dang1@ibm.com>, Shilpa Jagannath <shilpa.jagannath@ibm.com>

Merge pull request #59885 from ronen-fr/wip-rf-59793-to-squid

squid: osd/scrub: reduce osd_requested_scrub_priority default value

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

Merge pull request #60500 from zdover23/wip-doc-2024-10-26-backport-60444-to-squid

squid: doc/rados: standardize markup of "clean"

Merge pull request #60503 from zdover23/wip-doc-2024-10-26-backport-60434-to-squid

squid: doc/glossary: add "PLP" to glossary

doc/glossary: add "PLP" to glossary

Add "PLP" to the glossary.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 599bdbf0a240ee718ed6da4104141b0f23330fbe)

doc/rados: standardize markup of "clean"

Standardize the markup around the status "clean" in the documentation so
that readers don't mistakenly get the idea that inconsistent
presentation of the word "clean" implies a never-stated difference
between one instance and the other.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c8b4155d2d27228917d4be1f11514504100c55c7)

Merge pull request #60496 from zdover23/wip-doc-2024-10-26-backport-58647-to-squid

squid: doc/cephfs: add doc for disabling mgr/volumes plugin

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/cephfs: add doc for disabling mgr/volumes plugin

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 373910de34fc9e1d98218aed0361cd9bf3badffa)

orch: disk replacement enhancement

This introduces a new `ceph orch device replace` command in order to
improve the user experience when it comes to replacing the underlying
device of an OSD.

Fixes: https://tracker.ceph.com/issues/68456
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 212c8740831a7650b5be86c27d14f8c0b6eacbef)

ceph-volume: address mypy errors in disk.py

typical error:

```
ceph_volume/util/disk.py:1374: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")  [assignment]
```

This commits addresses it.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit b5e7008d28a5acd63ea9cd0c6b27f400dad409af)

ceph-volume: address test_activate_dmcrypt_tpm

This mocks the call to `luks_close()`, otherwise this test
fails when run on a system where `cryptsetup` isn't available.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit dd2a150f40fb11abe6bd1ee51bca03419aaa7d7f)

Merge pull request #60482 from zdover23/wip-doc-2024-10-25-backport-60480-to-squid

squid: doc/cephfs: use 'p' flag to set layouts or quotas

doc/cephfs: use 'p' flag to set layouts or quotas

Remind users that clients must use the 'p' flag when writing file
layouts.

This PR is made in response to
https://www.reddit.com/r/ceph/comments/1gbad51/comment/ltkgd5v/

Co-authored-by: Zac Dover <zac.dover@proton.me>
Signed-off-by: TruongSinh Tran-Nguyen <i@truongsinh.pro>
(cherry picked from commit 13c7c9d99881c7b4491ba1a693283c047161ea8a)

Merge pull request #59603 from cbodley/wip-67882-squid

squid: qa/rgw/multisite: specify realm/zonegroup/zone args for 'account create'

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #60244 from cbodley/wip-67957-squid

squid: rgw/beast: optimize for accept when meeting error in listenning

Merge pull request #60065 from cbodley/wip-68330-squid

squid: rgw/rados: don't rely on IoCtx::get_last_version() for async ops

Merge pull request #59917 from yuvalif/wip-67648-squid

squid: test/rgw/notifications: don't check for full queue if topics expired

Merge pull request #59670 from mkogan1/wip-67922-squid

squid: rgw: decrement qlen/qactive perf counters on error

Merge pull request #59439 from cbodley/wip-67712-squid

squid: rgw/http: finish_request() after logging errors

Merge pull request #59413 from cbodley/wip-67686-squid

squid: rgw: load copy source bucket attrs in putobj

Merge pull request #59303 from yuvalif/wip-67609-squid

squid: test/rgw/notification: use real ip address instead of localhost

Merge pull request #59226 from cbodley/wip-67558-squid

squid: rgw/notify: visit() returns copy of owner string

Merge pull request #59223 from cbodley/wip-67557-squid

squid: rgwlc: fix typo in getlc (ObjectSizeGreaterThan)

Merge pull request #60461 from zdover23/wip-doc-2024-10-24-backport-60460-to-squid

squid: doc/start: add vstart install guide

doc/start: add vstart install guide

Add "vstart Cluster Installation and Configuration Procedure" to the
Beginner's Guide.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a42be085bd7b9159f4924853ac2b2d92c37884b0)

Merge pull request #60354 from guits/wip-68556-squid

squid: cephadm: pull container images from quay.io

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #60353 from guits/wip-68566-squid

squid: ceph-volume: drop unnecessary call to `get_single_lv()`

Merge pull request #60091 from guits/wip-68343-squid

squid: ceph-volume: add new class UdevData

Merge pull request #60435 from zdover23/wip-doc-2024-10-23-backport-60433-to-squid

squid: doc/cephfs: rearrange subvolume group information

doc/cephfs: rearrange subvolume group information

Rearranging the documentation to align it with CephFS encapsulation design.

Signed-off-by: Indira Sawant <indira.sawant@ibm.com>
(cherry picked from commit f6cf7ae5281f5740ac78bfa5e2a22913353b8167)

Merge pull request #60432 from zdover23/wip-doc-2024-10-23-backport-60423-to-squid

squid: doc/cephadm: correct "ceph orch apply" command

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

cephadm: pull container images from quay.io

Now that all required images are hosted and/or mirrored on quay.io
we can move away from docker.io

Fixes: https://tracker.ceph.com/issues/68323
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit fd895dde9d69ae7be7a78d8db37a2f94ded27080)

doc/cephadm: correct "ceph orch apply" command

s/orch apply/ceph orch apply/ in a command where "ceph" was missing.

This was reported by Eugen Block to Zac Dover on 22 Oct 2024 through
Slack.

Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c42066766428c72884ed6f5dd6fa88244f436a1f)

Merge pull request #60127 from adamemerson/wip-40692-squid

squid: global: Call getnam_r with a 64KiB buffer on the heap

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #59661 from idryomov/wip-67845-squid

squid: librbd/migration: prune snapshot extents in RawFormat::list_snaps()

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #58542 from k0ste/wip-66530-squid

squid: cephadm: fix apparmor profiles with spaces in the names

Reviewed-by: Adam King adking@redhat.com

Merge pull request #60171 from idryomov/wip-63184-squid

squid: librbd/crypto/LoadRequest: clone format for migration source image

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #59771 from nbalacha/wip-67764-squid

squid: rbd-mirror: use correct ioctx for namespace

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #59502 from idryomov/wip-67698-squid

squid: rbd: "rbd bench" always writes the same byte

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #59145 from idryomov/wip-53674-squid

squid: librbd/crypto: fix issue when live-migrating from encrypted export

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #60335 from aclamk/wip-68560-squid

squid: os/bluestore: Fix ceph-bluestore-tool allocmap command

Merge pull request #59969 from aclamk/wip-68222-squid

squid: os/bluestore: Fix BlueFS allocating bdev label reserved location

Merge pull request #60336 from aclamk/wip-68558-squid

squid: os/bluestore: Fix repair of multilabel when collides with BlueFS

Merge pull request #60338 from zdover23/wip-doc-2024-10-16-backport-60302-to-squid

squid: doc,mailmap: update my email / association to ibm

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #60003 from kamoltat/wip-68280-squid

squid: src/mon/ConnectionTracker.cc: Fix dump function
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #60378 from rhcs-dashboard/wip-68597-squid

squid: mgr/dashboard: fix lifecycle issues

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60279 from mchangir/wip-log-save-thread-name-in-log-entries-squid

squid: log: save/fetch thread name infra

Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>

mgr/dashboard: fix lifecycle issues

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-details/rgw-bucket-details.component.html

Fixes: https://tracker.ceph.com/issues/68434
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 87612f499f86c9864c3bf6371cdd46954176e5ab)

Merge pull request #60356 from rhcs-dashboard/wip-68565-squid

Squid: mgr/dashboard: The subvolumes are missing from the dropdown menu on the "Create NFS export" page

mgr/dashboard: update translations for squid

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #60346 from rhcs-dashboard/wip-68554-squid

Squid: mgr/dashboard: sync policy's in Object >> Multi-site >> Sync-policy, does not show the zonegroup to which policy belongs to

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60348 from afreen23/wip-68559-squid

squid: mgr/dashboard: fix group name bugs in the nvmeof API

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60332 from cbodley/wip-doc-rgw-multisite-secondary-commit-squid

squid: doc/radosgw/multisite: fix Configuring Secondary Zones -> Updating the Period

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58991 from cbodley/wip-67307-squid

squid: msg: insert PriorityDispatchers in sorted position

Merge pull request #60343 from zdover23/wip-doc-2024-10-16-backport-60256-to-squid

squid: doc: update Key Idea in cephfs-mirroring.rst

Squid: mgr/dashboard: The subvolumes are missing from the dropdown menu on the "Create NFS export" page

Fixes: https://tracker.ceph.com/issues/68519
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
(cherry picked from commit e036fde7e4b03241e617cbb3ef8f19a703aae716)

ceph-volume: drop unnecessary call to `get_single_lv()`

`Zap.zap_lv()` currently makes a call to `get_single_lv()`:

```
        lv = api.get_single_lv(filters={'lv_name': device.lv_name,
                                        'vg_name': device.vg_name})
```

this isn't needed and redundant as zap_lv() takes an instance of `Device()`
as argument which has already a `lv_api` attribute:

class Device in device.py:

```
            else:
                vgname, lvname = self.path.split('/')
                filters = {'lv_name': lvname, 'vg_name': vgname}
            lv = lvm.get_single_lv(filters=filters)         # <---- same call

        if lv:
            self.lv_api = lv
```

This implies a duplicate call to `subprocess.Popen()` unnecessarily.

Fixes: https://tracker.ceph.com/issues/68312
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit c0e05bf36067294420631f33c5e43c32077eeb82)

os/bluestore: Fix repair of multilabel when collides with BlueFS

The problem was that BDEV_FIRST_LABEL_POSITION was removed from
bdev_label_valid_locations set.
Now, if label at BDEV_FIRST_LABEL_POSITION is valid, it is in the set.

Fixes: https://tracker.ceph.com/issues/68558
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
(cherry picked from commit 7343be720870d4a5f82b55beee4685457a003067)

mgr/dashboard: fix group name bugs in the nvmeof API

there are 2 issues

1. in cephadm, i was always using the first daemon to populate the group
   in all the services for the dashboard config.

2. in the API, if there are more than 1 gateways listed in the config,
   rather than chosing a random gateway from the group, raise an
   exception and warn user to specify the gw_group parameter in the api
   request

Fixes: https://tracker.ceph.com/issues/68463
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit f9b50b2e88ae5d9ac4f2cab986527a0a12317da9)

 Conflicts:
src/pybind/mgr/cephadm/services/nvmeof.py

Merge pull request #60311 from rhcs-dashboard/wip-68410-squid

squid: mgr/dashboard: ignore exceptions raised when no cert/key found

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60321 from aaSharma14/wip-68546-squid

squid: mgr/cephadm: Update multi-site configs before deploying  daemons on rgw service create

Reviewed-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: sync policy's in Object >> Multi-site >> Sync-policy, does not show the zonegroup to which policy belongs to

Fixes: https://tracker.ceph.com/issues/68355
Fixes Includes: Added default zonegroup name with the sync policy details

Signed-off-by: Naman Munet <namanmunet@li-ff83bccc-26af-11b2-a85c-a4b04bfb1003.ibm.com>
(cherry picked from commit 517ab013e2a9bd23b482134121bcb85c5a32f028)