doc/cephfs: update cephfs-shell link

Repair the link to cephfs-shell.rst in doc/cephfs/cephfs-shell.rst that
was broken in https://github.com/ceph/ceph/pull/41165/ when
doc/cephfs/cephfs-shell.rst was moved to doc/man/8/cephfs-shell.rst.

This commit is made in response to a request by Lander Duncan that was
made on the [ceph-users] mailing list, and can be seen here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/F7V4CWLIYCAJ4JXI2JLNY6QPCFPR4SLA/

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4f8903cf80a94596a1b17b4a07affb874f21ae76)

Merge pull request #58359 from zdover23/wip-doc-2024-06-30-backport-58358-to-squid

squid: doc/cephfs: take Anthony's suggestion

doc/cephfs: take Anthony's suggestion

Incorporate Anthony's suggestion in
https://github.com/ceph/ceph/pull/57567#discussion_r1607016050 into
doc/cephfs/fs-volumes.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7a78dda0a92339db27359f91f0cb081703e2e7f9)

Merge pull request #58264 from amathuria/wip-66690-squid

squid: mon, qa: suites override ec profiles with --yes_i_really_mean_it; monitors accept that

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #58262 from amathuria/wip-66688-squid

squid: qa/suites/rados/thrash/workloads: remove cache tiering workload

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #57589 from NitzanMordhai/wip-66142-squid

squid: common/pick_address: check if address in subnet all public address

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #58307 from zdover23/wip-doc-2024-06-27-backport-58287-to-squid

squid: doc/rados: update how to install c++ header files

Merge pull request #58304 from zdover23/wip-doc-2024-06-27-backport-58299-to-squid

squid: doc/dev: Fix typos in encoding.rst

Merge pull request #58090 from vshankar/wip-66519-squid

squid: qa: use centos9 for fs:upgrade

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/rados: update how to install c++ header files

In this example librados2-devel only install C header files on fedora 40,
therefore I added libradospp-devel to the command to include C++ header files.

Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
(cherry picked from commit 7e87441601db2379488fab62e900cd834086f26d)

doc/dev: Fix typos in encoding.rst

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit 9f43c67a639958a13fef198ffe62cf5241f1f53c)

Merge pull request #58271 from zdover23/wip-doc-2024-06-26-backport-58226-to-squid

squid: doc/rados: edit troubleshooting-osd.rst

doc/rados: edit troubleshooting-osd.rst

Make minor changes to the "Debugging Slow Requests" section of
doc/rados/troubleshooting/troubleshooting-osd.rst in preparation
for an expansion of this section in response to a reqeust from Joel
Davidow.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8b211b9c7f9b173bb12028eb7108d13867883d6e)

Merge pull request #58257 from zdover23/wip-doc-2024-06-25-backport-58243-to-squid

squid: doc/rados: credit Prashant for a procedure

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mon: settle erasure-code-profile --yes-i-really-mean-it in asok's schema

Fixes: https://tracker.ceph.com/issues/65183
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 0e70f679d0f32fafd7db39229325cf7db7f6eb05)

qa: tests override ec profiles with --yes-i-really-mean-it

This fixes a fallout from 629ba7bd349d48cdaa6d094751e7cfce651ba2bc.
The problem has been nailed down by Laura Flores.

Fixes: https://tracker.ceph.com/issues/65183
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit 6f813236f8c617b3ff9517cf73ed256b5fede90e)

qa/suites/rados/thrash/workloads: remove cache tiering workload

Fixes: https://tracker.ceph.com/issues/65852
Signed-off-by: Laura Flores <lflores@ibm.com>
(cherry picked from commit 35505a7f1fd8bcad2047426a4d641b4c016180af)

doc/rados: credit Prashant for a procedure

Credit Prashant D for creating the stretch-mode workaround procedure for
retrieving the correct size of datacenters.

Follows: https://github.com/ceph/ceph/pull/58109

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 2e777cb4f8bd682b807b97074f930826aa902db9)

Merge pull request #57970 from adk3798/squid-replace-pkg_resources-with-importlib

squid: cephadm: use importlib.metadata for querying ceph_iscsi's version

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57400 from adk3798/squid-cephadm-timeout-ignore-refresh-failed

squid: qa/cephadm: ignore CEPHADM_REFRESH_FAILED on timeout test

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57768 from adk3798/wip-65722-squid

squid: cephadm: have agent check for errors before json loading mgr response

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57398 from adk3798/squid-cephadm-ignore-orch-paused

squid: qa/cephadm: ignore CEPHADM_PAUSED on test_orch_cli test

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57187 from adk3798/squid-stray-laundry

squid: qa/cephadm: ignore stray daemon warning during rados_api_tests

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57770 from adk3798/wip-65948-squid

squid: mgr/pybind/object_format: fix json-pretty being marked invalid

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57516 from adk3798/wip-65958-squid

squid: mgr/cephadm:  update default NVMEoF container image version

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #58202 from zdover23/wip-doc-2024-06-23-backport-58200-to-squid

squid: doc/start: add links to Beginner's Guide

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/start: add links to Beginner's Guide

Add two links to Sage Weil's overview talks about Ceph.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 785020f212504cdbf374e0d3ca843fd92309de98)

Merge pull request #57936 from zdover23/wip-doc-2024-06-08-backport-57923-to-squid

squid: doc/src: add inline literals (``) to variables

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58181 from zdover23/wip-doc-2024-06-21-backport-58109-to-squid

squid: doc/rados: add stretch_rule workaround

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58161 from zdover23/wip-doc-2024-06-20-backport-57156-to-squid

squid: doc/rados: followup to PR#58057

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58189 from zdover23/wip-doc-2024-06-21-backport-55675-to-squid

squid: doc/src: invadvisable is not a word

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58187 from zdover23/wip-doc-2024-06-21-backport-58112-to-squid

squid: doc/rbd: fix typos in NVMe-oF docs.

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58171 from idryomov/wip-qemu-task-version-squid

squid: qa/tasks/qemu: Fix OS version comparison

Reviewed-by: Ramana Raja <rraja@redhat.com>

doc/src: invadvisable is not a word

changed to inadvisable on line 4997

Signed-off-by: Doug Whitfield <douglasawh@gmail.com>
(cherry picked from commit fb4a5d09e11b75a2f38c42c489af7e6f148916a7)

doc/rbd: fix typos in NVMe-oF docs.

Signed-off-by: N Balachandran <nibalach@redhat.com>
(cherry picked from commit 0cd415c53b49b2e9168df488340dab91d4c6badc)

doc/rados: add stretch_rule workaround

Add a method for defining a CRUSH rule that returns the actual value of
the total available size.

Fixes: https://tracker.ceph.com/issues/56650
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 007385a3ef05bd92e006fc7d6aba3fbb51792ef7)

Merge pull request #57990 from idryomov/wip-66359-squid

squid: pybind/rbd: parse access and modify timestamps in UTC

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #58098 from adk3798/wip-66452-squid

squid: cephadm: _extract_host_info_from_*() refactor

Reviewed-by: John Mulligan <jmulligan@redhat.com

Merge pull request #58096 from adk3798/wip-66429-squid

squid: mgr/cephadm: make SMB and NVMEoF upgrade last in staggered upgrade

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #58095 from adk3798/wip-66427-squid

squid: cephadm: CephExporter doesn't bind to IPv6 in dual stack

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57972 from adk3798/squid-keepalive-security-context

squid: cephadm/services/ingress: configure security user in keepalived template

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57773 from adk3798/wip-65968-squid

squid: mgr/cephadm: make setting --cgroups=split configurable for adopted daemons

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57772 from adk3798/wip-65955-squid

squid: mgr/cephadm: Fix unfound progress events

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57771 from adk3798/wip-65953-squid

squid: qa/tasks/cephadm: fix pulling containers from private registries

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57769 from adk3798/wip-65724-squid

squid: python-common: handle "anonymous_access: false" in to_json of Grafana spec

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57517 from adk3798/wip-65951-squid

squid: mgr/cephadm: cleanup iscsi and nvmeof keyrings upon daemon removal

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57411 from adk3798/squid-cephadm-ignore-stray-on-upgrade

squid: qa/cephadm: ignore CEPHADM_STRAY_DAEMON in upgrade tests

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57397 from adk3798/squid-no-stray-recent-removal

squid: mgr/cephadm: don't mark daemons created/removed in the last minute as stray

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57396 from adk3798/squid-autotune-testing

squid: mgr/cephadm: additional debug logging for autotuner

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57233 from adk3798/wip-65762-squid

squid: mgr/cephadm: set OSD cap for NVMEoF daemon to "profile rbd"

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57186 from adk3798/squid-service-size-cleanup

squid: mgr/cephadm: cleanup service size logic block

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #57185 from adk3798/squid-cephadm-apply-spec-file

squid: cephadm: check if file exists when passing `--apply_spec`

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #57184 from adk3798/squid-bootstrap-custom-alerts

squid: cephadm: allow passing custom prometheus alerts to bootstrap

Reviewed-by: John Mulligan <jmulligan@redhat.com>

qa/tasks/qemu: Fix OS version comparison

See: https://sentry.ceph.com/share/issue/21ed88d705854238bdafbf6711e795ee/
They're strings, not floats.
This surfaced as a result of https://github.com/ceph/teuthology/pull/1953

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit 260a45a03148ba4d2bbfcac9485bcf9a8743cade)

Merge pull request #58130 from zdover23/wip-doc-2024-06-19-backport-58123-to-squid

squid: doc: clarify availability vs integrity

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

doc/rados: followup to PR#58057

Incorporate Anthony D'Atri's suggestions in
https://github.com/ceph/ceph/pull/58057

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 2e999a26ef2d3c12d81bd54c187a6225a5fcd7a6)

pybind/rbd: parse access and modify timestamps in UTC

It appears that commits 08cee16d0a4b ("pybind/rbd: always parse
timestamps in UTC") and 809c5430c292 ("librbd: add image access/last
modified timestamps") raced with each other and we ended up with two
more timezone-dependent timestamps.

Fixes: https://tracker.ceph.com/issues/66359
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fafa911b51199f61d0c97bc6dfc71d4f65134db4)

test/pybind/rbd: make timestamp tests meaningful

The existing asserts don't really test anything, with some of them
being for inequality against a literal of a mismatching type.  As
a result, a bug in access_timestamp() and modify_timestamp() went
unnoticed for years.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit a1b6ca863a384fe32e750b9ee8f10688f88dd1e3)

test/pybind/rbd: fix tests that compare strings with b''

assert_not_equal(b'', self.image.id()) is bogus because Image::id()
returns a string (str), not bytes.  If the types don't match, values
are guaranteed to not match.

The same goes for Image::block_name_prefix().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 05f7e4f2025c435dca8ae0b34f65425afbc212b5)

Merge pull request #57571 from idryomov/wip-65813-squid

squid: librbd: don't crash on a zero-length read if buffer is NULL

Merge pull request #57602 from ajarr/wip-66117-squid

squid: librbd: create rbd_trash object during pool initialization and namespace creation

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #57766 from idryomov/wip-66232-squid

squid: qa/suites/rbd: override extra_system_packages directly on install task

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #58044 from idryomov/wip-64662-squid

squid: librbd: allow cloning from non-user snapshots

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #56883 from guits/wip-65480-squid

squid: ceph-volume: osd objectstore refactor

Merge pull request #56931 from adk3798/wip-65515-squid

squid: mgr/dashboard: fix duplicate grafana panels when on mgr failover

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #58122 from ljflores/wip-66554-squid

squid: qa/rados: don't pass --max_attr_len to ceph_test_rados unconditionally

python-common/service_spec: fix some mypy complaints

The python/mypy combination on the jenkins nodes the CI
is running on don't seem to care, but locally I get

mypy: commands[0]> mypy --config-file=../mypy.ini -p ceph
ceph/deployment/service_spec.py: note: In member "validate" of class "NvmeofServiceSpec":
ceph/deployment/service_spec.py:1497: error: Unsupported operand types for > ("float" and "None")  [operator]
ceph/deployment/service_spec.py:1497: note: Left operand is of type "Optional[float]"
ceph/deployment/service_spec.py:1500: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1500: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1503: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1503: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1506: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1506: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1509: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1509: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1512: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1512: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1515: error: Unsupported operand types for > ("float" and "None")  [operator]
ceph/deployment/service_spec.py:1515: note: Left operand is of type "Optional[float]"
ceph/deployment/service_spec.py:1518: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1518: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1521: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1521: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1524: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1524: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1527: error: Unsupported operand types for > ("int" and "None")  [operator]
ceph/deployment/service_spec.py:1527: note: Left operand is of type "Optional[int]"
ceph/deployment/service_spec.py:1530: error: Unsupported operand types for > ("float" and "None")  [operator]
ceph/deployment/service_spec.py:1530: note: Left operand is of type "Optional[float]"
Found 12 errors in 1 file (checked 27 source files)

The errors make sense to me, so I think we should fix them

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 7520f65f261c61d01dd75b9722c84147229d9e87)

mgr/cephadm: update default NVMEoF container image version

Otherwise the nvmeof daemon fails to start up

i148-njpgei[69585]: Traceback (most recent call last):
i148-njpgei[69585]:   File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
i148-njpgei[69585]:     return _run_code(code, main_globals, None,
i148-njpgei[69585]:   File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
i148-njpgei[69585]:     exec(code, run_globals)
i148-njpgei[69585]:   File "/src/control/__main__.py", line 43, in <module>
i148-njpgei[69585]:     gateway.serve()
i148-njpgei[69585]:   File "/src/control/server.py", line 177, in serve
i148-njpgei[69585]:     omap_lock = OmapLock(omap_state, gateway_state)
i148-njpgei[69585]:   File "/src/control/state.py", line 201, in __init__
i148-njpgei[69585]:     self.omap_file_lock_retry_sleep_interval = self.omap_state.config.getint_with_default("gateway",
i148-njpgei[69585]:   File "/src/control/config.py", line 47, in getint_with_default
i148-njpgei[69585]:     return self.config.getint(section, param, fallback=value)
i148-njpgei[69585]:   File "/usr/lib64/python3.9/configparser.py", line 818, in getint
i148-njpgei[69585]:     return self._get_conv(section, option, int, raw=raw, vars=vars,
i148-njpgei[69585]:   File "/usr/lib64/python3.9/configparser.py", line 808, in _get_conv
i148-njpgei[69585]:     return self._get(section, conv, option, raw=raw, vars=vars,
i148-njpgei[69585]:   File "/usr/lib64/python3.9/configparser.py", line 803, in _get
i148-njpgei[69585]:     return conv(self.get(section, option, **kwargs))
i148-njpgei[69585]: ValueError: invalid literal for int() with base 10: '1.0'

I've been told 1.2.5 has a patch that allows this value to be a float

Fixes: https://tracker.ceph.com/issues/65718
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 0cebec2888d309e3c727ec2547692f680f3896af)

Merge pull request #58127 from petrutlucian94/wip-66547-squid

squid: ceph_test_rados_api_misc: adjust LibRadosMiscConnectFailure.ConnectTimeout timeout

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #58116 from zdover23/wip-doc-2024-06-18-backport-57352-to-squid

squid: doc/dev/crimson: CPU cores allocation

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #58105 from zdover23/wip-doc-2024-06-18-backport-58901-to-squid

squid: doc/dev: link to ceph.io leads list

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58067 from Matan-B/wip-57788-squid

squid: crimson/os/seastore: implement disk and writer level stats reporting

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #58066 from Matan-B/wip-57654-squid

squid: crimson/os/seastore: fix compile error in release build

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #58032 from Matan-B/wip-57279-squid

squid: crimson/osd/pg: trigger wait_for_active_blocker on replica osds when the activate event is committed

Reviewed-by: Samuel Just <sjust@redhat.com>

doc: clarify availability vs integrity

(cherry picked from commit 3888cf77075eceb079b1d5a8ce19092d12fd82bd)

ceph_test_rados_api_misc: adjust LibRadosMiscConnectFailure.ConnectTimeout timeout

The test sets a 2 second connection timeout and asserts that the
attempt takes less than 4 seconds.

Under load, the asserted timeout can be exceeded. In particular,
this affects the Windows CI job, which uses a VM with a reduced
amount of resources.

    [ RUN      ] LibRadosMiscConnectFailure.ConnectTimeout
    /home/ubuntu/ceph/src/test/librados/misc.cc:87: Failure
    Expected: (dur) < (utime_t(4, 0)), actual: 5.987712 vs 4.000000

We'll adjust the timeouts, using a 5s configured timeout, asserting
that the connection attempt doesn't take more than 15s.

Fixes: https://tracker.ceph.com/issues/66534
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 3a2b4a58898477ae9f62b506bd5be56ff9abacf0)

qa/rados: don't pass --max_attr_len to ceph_test_rados unconditionally

Fixes: https://tracker.ceph.com/issues/66321
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit 8a6d56e62fc03062e60828a3bc9f37ed7232b3e7)

doc/dev/crimson: CPU cores allocation

CPU cores allocation and
add crimson-smp vstart flag

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 35d257bfaa5ed7cf91c8a8b523ed4494fdd27667)

Merge pull request #58093 from Matan-B/wip-55831-squid

squid: crimson/os/seastore: lba pointer consolidation

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #58065 from Matan-B/wip-57383-squid

squid: crimson: fix user_version handling

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #58030 from Matan-B/wip-57561-squid

squid: crimson/osd/ops_executer: LIST_SNAPS only on CEPH_SNAPDIR

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #57862 from rhcs-dashboard/wip-66345-squid

squid: mgr/dashboard: add smb service management support

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #57861 from rhcs-dashboard/wip-66037-squid

squid: mgr/dashboard: add upgrade notification

qa: upgrade sub-suite upgraded_client from from n-1|n-2 releases

... and move to centos9.

Fixes: https://tracker.ceph.com/issues/65580
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 76589659480c6e9cd2ad09b3be21deeb6d1686d1)

qa: upgrade sub-suite nofs from n-1 and n-2 releases

also adjust the respective upgrade YAMLs and move to
centos9.

For squid branch, start upgrade from quincy/reef.

Fixes: https://tracker.ceph.com/issues/65580
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f00a69f714dd07d580731a326aff41cacca6d0b6)

qa: use supported releases for featureful_client

... and switch for centos9.

For squid branch, start upgrade from quincy/reef.

Fixes: https://tracker.ceph.com/issues/65580
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 3715ce671de97530dda30d59fe643cebc3ebc508)

crimson/osd: remove osd_op_params_t::user_at_version

osd_op_params_t::user_at_version was populated from
osd_op_params_t::at_version before the call to prepare_transaction,
which incremented osd_op_params_t::at_version.version.  As a result,
the value stored in object_info_t::user_version ended up one version
behind object_info_t::version.  The log entry, on the other hand,
ended up with the correct version as OpsExecutor::prepare_transaction
populates it directly from at_version.  As a result, the primary could
return different versions to the client depending on whether the IO was
already in the log.

This commit eliminates osd_op_params_t::user_at_version and updates
PGBackend::mutate_object to behave like prepare_transaction.  Because
the prior commit removes the prepare_transaction increment, this isn't
strictly necessary, but it is simpler.

Fixes: https://tracker.ceph.com/issues/65857
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 85fdadab2adcfbec83a82c6f1bae1f4f31a96f54)

crimson/osd/ops_executor: only increment osd_op_params_t::at_version for clone

Previously, we incremented prior to usage in both prepare_transaction
and execute_clone.  Because at_version is initialized from
PG::get_next_version(), this results in log entries skipping
values.  Instead, only increment after populating clone
object_info.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 7e0dde721d5203623b06351bb4b261b3c9fe2c33)

crimson/osd/ops_executor: use clone oi values in log entry

The fields from the clone_obc oi are what we're actually
trying to fill in here -- we might as well use them
directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 207e5a6e9ba54deb2fb18d5a5ad432d42072453d)

crimson/osd/ops_executor: update osd_op_params_t::at_version in execute_clone

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1ecf3d64853c2fa92bf467692b98a58b4670b212)

crimson/osd/pg: remove slightly confusing assert from PG::submit_transaction

This assert should always hold, so it's not wrong.  However,
osd_op_p.at_version is chosen during transaction construction time
using the current epoch via PG::get_next_version.  Asserting it here
is more confusing than helpful.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 7a04f6230d9376d155ac92a6c3bf1ab5eeca644f)

crimson/osd/pg_backend: remove uninformative #if 0 block

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 2bd7c38381bc22bc73130e6d138e001096e66faf)

Merge pull request #58064 from Matan-B/wip-57056-squid

squid:  crimson/osd: make osd_op_params::at_version coherent with last log entry

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #58063 from Matan-B/wip-57245-squid

squid: qa/config/crimson_qa_overrides: adjust mgr_stats_period

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #58024 from Matan-B/wip-57251-squid

squid: crimson/osd/pg_backend: only create new onode when it doesn't exist

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #57471 from guits/wip-66021-squid

squid: ceph-volume: use os.makedirs for mkdir_p

Merge pull request #57473 from guits/wip-66023-squid

squid: cephadm: change loki/promtail default image tags

Merge pull request #58004 from zdover23/wip-doc-2024-06-13-backport-57650-to-squid

squid: ceph-volume: use importlib from stdlib on Python 3.8 and up

doc/dev: link to ceph.io leads list

Remove the docs.ceph.com list of team leads, and link to the list of
leads on ceph.io.

Fixes: https://tracker.ceph.com/issues/66392
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 5134aeb78bb1d9d956442333ad372e1f397ff7b3)

cephadm: _extract_host_info_from_*() refactor

The current implementation doesn't take into account the format is Yaml

This can lead to issue when oob details are passed to host service spec.

For instance, with the following host spec:

```
---
service_type: host
addr: 1.2.3.4
hostname: node1
oob:
  username: root
  password: passw0rd
  addr: 127.0.0.1
```

it is converted to a list like the following:

```
['service_type: host', 'addr: 1.2.3.4', 'hostname: node1',
 'oob:', 'username: root', 'password: passw0rd', 'addr: 127.0.0.1']
```

It was (probably) assumed that the pattern `addr:` would be present only
once. With the introduction of node-proxy, this isn't true anymore.

Now that the cephadm binary can embed some external libraries we can leverage pyyaml.
The idea is to use proper yaml format instead so it is easier to process the data.

Fixes: https://tracker.ceph.com/issues/66165
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit f4ddb2750eb04d7563356a41c7433a3c5d6ded57)