Merge pull request #48078 from adk3798/wip-57524-quincy

quincy: mgr/cephadm: validating tuned profile specification

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #48079 from adk3798/wip-57521-quincy

quincy: mgr/cephadm: don't use "sudo" in commands if user is root

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47989 from guits/wip-57380-quincy

quincy: ceph-volume: add a retry in util.disk.remove_partition

Merge pull request #48034 from s0nea/wip-57488-quincy

quincy: monitoring/ceph-mixin: add RGW host to label info

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #48104 from ljflores/wip-quincy-revert-pr-47901

quincy: Revert PR 47901

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Revert "osd/PeeringState: fix missed `recheck_readable` from laggy"

This reverts commit c71ee14cdbcc499d0ccaa84f4714ad5f29821f40.

This commit caused a regression in the rados suite, as evidenced by:

- with the commit:
http://pulpito.front.sepia.ceph.com/lflores-2022-09-14_15:11:39-rados-quincy-release-distro-default-smithi/

- with the commit reverted:
http://pulpito.front.sepia.ceph.com/lflores-2022-09-14_17:02:02-rados-wip-lflores-testing-quincy-release-distro-default-smithi/

Fixes: https://tracker.ceph.com/issues/57546
Signed-off-by: Laura Flores <lflores@redhat.com>

Revert "osd/PeeringState: proc_lease_ack break once found from OSD"

This reverts commit 0d07b406dc2f854363f7ae9b970e980400f4f03e.

This commit caused a regression in the rados suite, as evidenced by:

- with the commit:
http://pulpito.front.sepia.ceph.com/lflores-2022-09-14_15:11:39-rados-quincy-release-distro-default-smithi/

- with the commit reverted:
http://pulpito.front.sepia.ceph.com/lflores-2022-09-14_17:02:02-rados-wip-lflores-testing-quincy-release-distro-default-smithi/

Fixes: https://tracker.ceph.com/issues/57546
Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #48061 from zdover23/wip-doc-2022-09-13-backport-47575-to-quincy

quincy: doc/rados: add prompts to pools.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #48063 from zdover23/wip-doc-2022-09-13-backport-47305-to-quincy

quincy: doc/monitoring: add min vers of apps in mon stack

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/cephadm: don't use "sudo" in commands if user is root

We had a patch earlier to make us not use sudo unless the
user is not root for our other commands, but this specific
one that just runs "true" with a timeout to check if the host
is online was missed.

Fixes: https://tracker.ceph.com/issues/57369
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit c638d295dd30e202777314571b1075de3334becc)

mgr/cephadm: validating tuned profile specification
fixes: https://tracker.ceph.com/issues/57192
fixes: https://tracker.ceph.com/issues/57191

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit d3e55f1521da8097f3a164d663bfbd086ac74028)

Conflicts:
src/cephadm/cephadm
src/pybind/mgr/cephadm/services/cephadmservice.py

Merge pull request #48047 from aaSharma14/wip-57499-quincy

quincy: mgr/dashboard: fix snapshot creation with duplicate name

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #47660 from adk3798/wip-57098-quincy

quincy: cephadm: support quotes around public/cluster network in config passed to bootstrap

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47949 from adk3798/wip-57397-quincy

quincy: qa/cephadm: remove fsid dir before bootstrap in test_cephadm.sh

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

doc/monitoring: add min vers of apps in mon stack

https://tracker.ceph.com/issues/45447

This PR adds recommended versions of grafana and
prometheus and alert manager.

This PR is a second attempt at getting the information
in the following PR into the docs:
https://github.com/ceph/ceph/pull/46000/files

Himadri Maheshwari deserves the credit for the work
in this commit.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
Signed-off-by: Himadri Maheshwari <himadri.maheshwari7915@gmail.com>
(cherry picked from commit 367695f5b09f75ee723d53116e2f4a6e45dd795d)

doc/rados: add prompts to pools.rst

This commit adds ".. prompt:: bash $"-style prompts to pools.rst.
This brings this file up to the standard established in 2020 when
Kefu added support for the ".. prompt::" directive.

This commit is a part of an initiative to modernize the presentation
of all BASH commands in the RADOS documentation.

The progress of this project can be tracked here:
https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 1bd64192568242b141d8e30fef6758bf162ec350)

Merge pull request #47954 from adk3798/quincy-tp-no-overwrite-doc2

quincy: doc/cephadm: add info about --no-overwrite to note about tuned-profiles

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47324 from zdover23/wip-doc-2022-07-28-backport-46024-to-quincy

quincy: doc/radosgw/STS: sts_key and user capabilities

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/dashboard: fix snapshot creation with duplicate name

Snapshot creation with same name on UI throwing 500 Internal Error, This PR intends to fix this issue.

Fixes: https://tracker.ceph.com/issues/57456
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 2ae1bca1d9c7a771be0fa9481bef2622ffd7cc45)

Merge pull request #47945 from adk3798/wip-57425-quincy

quincy: cephadm: Fix disk size calculation

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge pull request #48019 from rzarzynski/wip-doc-dups-trimming-relnotes-quincy

quincy: PendingReleaseNotes: document online and offline trimming of PG Log's…

Reviewed-By: Neha Ojha <nojha@redhat.com>

Merge pull request #47930 from rzarzynski/wip-56723-quincy

quincy: msg: fix deadlock when handling existing but closed v2 connection

Merge pull request #47929 from rzarzynski/wip-56655-quincy

quincy: osd: return ENOENT if pool information is invalid during tier-flush

Merge pull request #48013 from mchangir/wip-56541-quincy

quincy: mgr/snap_schedule: Use rados.Ioctx.remove_object() instead of remove().

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #47971 from vshankar/tr-55930

quincy: client: allow overwrites to file with size greater than the max_file_size

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48016 from idryomov/wip-ntstatus-quincy

quincy: common/win32,dokan: include bcrypt.h for NTSTATUS

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #47931 from rzarzynski/wip-57257-quincy

quincy:  msg: reset ProtocolV2's frame assembler in appropriate thread

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #47977 from neesingh-rh/wip-57440-quincy

quincy: cephfs-top: display average read/write/metadata latency

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #47658 from adk3798/wip-57149-quincy

quincy: mgr/cephadm: set dashboard grafana-api-password when user provides one

Reviewed-by: John Mulligan <jmulligan@redhat.com>

monitoring/ceph-mixin: add RGW host to label info

Add the missing information about the RGW instance to the labels of the
"Average GET/PUT Latencies" panel on the "RGW Overview" dashboard.

Fixes: https://tracker.ceph.com/issues/57166
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 15fa97d49d28aebdd57a2eb9276865c4513e0cb2)

monitoring/ceph-mixin: add entries to envlist

Add the missing entries `jsonnet-bundler-install` and
`jsonnet-bundler-update` to envlist.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 42ff9370a02b8685e737e958df39f8c3c26a519b)

Merge pull request #48025 from rhcs-dashboard/wip-57484-quincy

quincy: mgr/dashboard: fix unmanaged service creation

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #48023 from idryomov/wip-57116-cont-quincy

quincy: test/{librbd, rgw}: increase delay between and number of bind attempts

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

mgr/dashboard: fix unmanaged snmp service creation

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a35651ee8586133fa666de189b9d324c0846c3ca)

mgr/dashboard: fix unmanged iscsi service creation

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 7d157fe48e21945ef51e21f206639f9abfa44fa5)

mgr/dashboard: fix unable to create ingress unmanaged

the following snipped is the error from backend

```
 File "/lib/python3.6/site-packages/ceph/deployment/service_spec.py", line 698, in _from_json_impl
                                                  _cls.validate()
                                                File "/lib/python3.6/site-packages/ceph/deployment/service_spec.py", line 1058, in validate
                                                  'Cannot add ingress: No frontend_port specified')
                                              ceph.deployment.hostspec.SpecValidationError: Cannot add ingress: No frontend_port specified
```

It looks like even if we set unmanaged flag, we need to input the
backend_service, frontend_port, monitor_port and virtual_ip, because there is a
validation going for that in the backend.

https://github.com/ceph/ceph/blob/main/src/python-common/ceph/deployment/service_spec.py#L1050

Fixes: https://tracker.ceph.com/issues/57284
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit ea68560e148cf46dd69982120605519aba3a8955)

Merge pull request #47961 from rhcs-dashboard/wip-57436-quincy

quincy: mgr/dashboard: fix nfs exports form issues with squash field

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

test/{librbd, rgw}: increase delay between and number of bind attempts

Commit aa7885f7cc41 ("test/{librbd, rgw}: retry when bind fail with
port 0") reduced the frequency of sporadic unit test failures caused
by EADDRINUSE a lot, but not entirely.

Currently, it yields a cumulative sleep of ~9 seconds.  Let's increase
that to 1 minute.

Fixes: https://tracker.ceph.com/issues/57116
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 48016eaa1511ac8a39ed33084e0e230f3b1b5821)

include: fix IS_ERR on Windows

The "long" type uses 32b on x64 Windows platforms, which means
it's not large enough to store a pointer. intptr_t or uintptr_t
should be used instead.

This change fixes include/err.h, using the right types. There was
a previous patch on this topic but unfortunately it didn't address
all the type casts.

This issue was brought up by the unittest_crush test, which recently
started to fail as the CrushWrapper methods use IS_ERR.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit c95b6b6c774da05e989cd09e23eee1eeaa9e6ec2)

Merge pull request #47942 from adk3798/wip-57375-quincy

quincy: mgr/dashboard: enable addition of custom Prometheus alerts

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47932 from rzarzynski/wip-57209-quincy

quincy: test/lazy-omap-stats: Various enhancements

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #47536 from guits/wip-57089-quincy

quincy: ceph-volume: system.get_mounts() refactor

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #47608 from adk3798/quincy-incorrect-warning

quincy: cephadm: fix incorrect warning

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #47610 from adk3798/quincy-multiple-vips

quincy: Cephadm: Allow multiple virtual IP addresses for keepalived and haproxy

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #47626 from guits/wip-57132-quincy

quincy: cephadm/ceph-volume: fix rm-cluster --zap

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #47654 from adk3798/quincy-dup-args-loki-promtail

quincy: cephadm: Remove duplicated process args in promtail and loki

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47656 from adk3798/wip-57168-quincy

quincy: cephadm: support for Oracle Linux 8

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #47657 from adk3798/wip-57151-quincy

quincy: mgr/cephadm: fix handling of draining hosts with explicit placement specs

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47659 from adk3798/wip-57102-quincy

quincy: mgr/cephadm: recreate osd config when redeploy/reconfiguring

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47944 from adk3798/wip-57426-quincy

quincy: cephadm/mgr: adding logic to handle --no-overwrite for tuned profiles

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47946 from adk3798/wip-57423-quincy

quincy: mgr/cephadm: Fix how we check if a host belongs to public network

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47950 from adk3798/wip-57383-quincy

quincy: mgr/cephadm: Adding logic to store grafana cert/key per node

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47951 from adk3798/wip-57382-quincy

quincy: mgr/cephadm: allow binding to loopback for rgw daemons

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #47952 from adk3798/wip-57378-quincy

quincy: cephadm: return nonzero exit code when applying spec fails in bootstrap

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #47953 from adk3798/wip-57377-quincy

quincy: mgr/cephadm: don't try to write client/os tuning profiles to known offline hosts

Reviewed-by: John Mulligan <jmulligan@redhat.com>

dokan: cast variable to the expected type before comparison

to fix the FTBFS due to following warning:

```
/home/jenkins-build/build/workspace/ceph-windows-pull-requests/ceph/build.deps/src/dokany/dokan/dokan.h:723:22: error: narrowing conversion of '-1' from 'int' to 'long unsigned int' [-Wnarrowing]
  723 | #define DOKAN_ERROR -1
      |                      ^
```

also, clean up the following warning:

```
/home/jenkins-build/build/workspace/ceph-windows-pull-requests/ceph/src/dokan/dbg.cc:142:62: warning: NULL used in arithmetic [-Wpointer-arith]
  142 |   o << "\n\tIsDirectory: " << (DokanFileInfo->IsDirectory != NULL);
      |
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit f27e16ff9494e1d1f36324b64fb24a16fcf929b3)

PendingReleaseNotes: document online and offline trimming of PG Log's dups

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit dbfb50453c2f7fe5b0a06d846dd38693cceb63fe)

Conflicts:
    PendingReleaseNotes
- Moved the note under ">=17.2.4" section

Merge pull request #48004 from sseshasa/wip-57461-quincy

quincy: PendingReleaseNotes: Note the fix for high CPU utilization during recovery

Reviewed-by: Neha Ojha <nojha@redhat.com>

win32_deps_build.sh: bump curl to curl-7_84_0

to silence warnings like:

```
configure.ac:3671: warning: The macro `AC_HELP_STRING' is obsolete.
configure.ac:3671: You should run autoupdate.
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 6b43df3c58fd15501988b7cfbe411fb42f223689)

Merge pull request #47894 from kotreshhr/wip-57242-quincy

quincy: mgr/volumes: Few mgr volumes backports

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #47996 from idryomov/wip-52810-quincy

quincy: librbd: retry ENOENT in V2_REFRESH_PARENT as well

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

test/encoding: refactor EncodingException::Macros

so it is more compacted

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit b3151a9c373b472c7e88080205fc256a1304fe9d)

test/encoding: verify that e.what() starts with expected str

boost changes the way how it prints boost::system::system_error in
boost 1.79 -- it appends the stringified error_category at end of
exception::what(), and our buffer::malformed_input is a subclass
of boost::system::system_error.

so we cannot just compare the return value of what() with the
expected string, to be more future proof, let's check if i
starts with the expected string instead.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 0d9eec3c4feff30ab55601533bccf9ba6e568b9f)

common/win32,dokan: include bcrypt.h for NTSTATUS

to avoid the conflicting declaration of NTSTATUS from bcrypt.h and our
own typedef. as after switching to boost 1.79, we would have following compiling
failure:

In file included from ../src/dokan/options.cc:14:
../src/dokan/ceph_dokan.h:16:15: error: conflicting declaration 'typedef DWORD NTSTATUS'
   16 | typedef DWORD NTSTATUS;
      |               ^~~~~~~~
In file included from ../build.deps/mingw/boost/include/boost/asio/impl/connect_pipe.ipp:29,
                 from ../build.deps/mingw/boost/include/boost/asio/connect_pipe.hpp:79,
                 from ../build.deps/mingw/boost/include/boost/asio.hpp:64,
                 from ../src/include/win32/winsock_wrapper.h:20,
                 from <command-line>:
/usr/share/mingw-w64/include/bcrypt.h:27:16: note: previous declaration as 'typedef LONG NTSTATUS'
   27 |   typedef LONG NTSTATUS,*PNTSTATUS;
      |                ^~~~~~~~

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 1a09e520bf26f61cb048d9d3d3876f284af64b04)

mgr/snap_schedule: Use rados.Ioctx.remove_object() instead of remove().

This fixes the "AttributeError: 'rados.Ioctx' object has no attribute
'remove'" error.

Signed-off-by: Andreas Teuchert <a.teuchert@syseleven.de>
(cherry picked from commit bb958f18af615377852d1579da936ea6c62d6056)

PendingReleaseNotes: Note the fix for high CPU utilization during recovery

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit d6982022325a17dbe91e760530ab21832459a817)

Conflicts:
    PendingReleaseNotes
- Moved the note under ">=17.2.4" section

Merge pull request #47993 from soumyakoduri/wip-skoduri-quincy

rgw/backport/quincy: Fix crashes with Sync policy APIs

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #46005 from rzarzynski/wip-common-no-cpp17-second_round-quincy

quincy: common/bl: fix FTBFS on C++11 due to C++17's if-with-initializer

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #47901 from amathuria/wip-56736-quincy

quincy: osd/PeeringState: fix missed recheck_readable from laggy

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #45892 from nkshirsagar/wip-55297-quincy

quincy: Catch exception if thrown by __generate_command_map()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #46559 from pdvian/wip-55305-quincy

quincy: mgr, mon: Keep upto date metadata with mgr for MONs

Reviewed-by: Laura Flores <lflores@redhat.com>

librbd: make RefreshRequest tests compatible with clone v1

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 36f1d515ca92784631d29fa1c5d1465e957af2a7)

librbd: retry ENOENT in V2_REFRESH_PARENT as well

With auto-deletion of trashed snapshots, it is relatively easy to lose
a race to "rbd flatten" as follows:

- when V2_GET_PARENT runs, the image is technically still a clone
- when V2_REFRESH_PARENT runs, the image is fully flattened and the
  snapshot in the parent image is deleted

This results in a spurious ENOENT error, mainly when trying to open the
image (e.g. for "rbd info").  This race condition has always been there
but auto-deletion of trashed snapshots makes it much worse.

Retry ENOENT in V2_REFRESH_PARENT the same way as in V2_GET_SNAPSHOTS.

Fixes: https://tracker.ceph.com/issues/52810
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit bd885d75b2e4d728086f744e0d10e7cd12d3f15b)

librbd: limit the number of ENOENT retries in RefreshRequest

If the image header is corrupt, ENOENT error may be persistent.  Avoid
an infinite loop in that case.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 8570194b133462db6b7d4ab108383db0967b1cb9)

librbd: fix a bunch of issues with restarting RefreshRequest

Make RefreshRequest properly restartable, at least up until and including
V2_REFRESH_PARENT step:

- clear m_migration_spec when skipping GET_MIGRATION_HEADER
- don't rely on potentially stale m_incomplete_update on retry
- reset m_legacy_parent when retrying more than just V2_GET_PARENT
- don't rely on potentially stale m_parent_md.overlap and
  m_head_parent_overlap on retry
- clear m_metadata before fetching image metadata (but not before
  fetching pool metadata)
- clear m_op_features when skipping V2_GET_OP_FEATURES
- clear m_group_spec on EOPNOTSUPP error in V2_GET_GROUP
- reset m_legacy_snapshot when retrying more than just V2_GET_SNAPSHOTS
- don't rely on potentially stale m_snap_parents on retry

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 6bd89ea119520cf5a45ac93b0e16edf35ddd4e57)

librbd: check *result consistently in RefreshRequest

Stick to *result >= 0 checks everywhere and add missing checks for
op_features_get_finish() and image_group_get_finish() errors.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ce6dff89c0f005c1ae1dc71cadfbef9f82df37a4)

librbd: reflect V2_GET_SNAPSHOTS ENOENT retry in state diagram

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ca36ffa347f0c68115a7d6b54ebb47ac5e82698d)

radosgw-admin: fix crash with 'sync flow create/remove' cmd

Avoid dereferencing an empty optional "flow-type" (if not specified).

Fixes: https://tracker.ceph.com/issues/57275
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 539c5b87a2965ce43002430790abd586b98f620d)

rgw: Avoid dereferencing nullptr while configuring bucket sync policy

While configuring bucket sync policy, in "rgw_sync_bucket_entities::set_bucket()",
there could be a case where in bucket doesnt contain any value but is still being
dereferenced. This commit fixes the same.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 86cf8275224536a7ca77eaf8a6e59951b3f25261)

Merge pull request #47940 from idryomov/wip-56703-quincy

quincy: librbd/cache/pwl: narrow the scope of m_lock in write_image_cache_state()

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #47235 from cfsnyder/wip-55714-quincy

quincy: rgw_rest_user_policy: Fix GetUserPolicy & ListUserPolicies responses

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #46107 from BenoitKnecht/wip-55499-quincy

quincy: rgw: Avoid segfault when OPA authz is enabled

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45714 from cbodley/wip-55136

quincy: rgw: data sync uses yield_spawn_window()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47980 from tchaikov/quincy-pr-47962

quincy: test/{librbd, rgw}: retry when bind fail with port 0

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

ceph-volume: add a retry in util.disk.remove_partition

This fixes a possible race condition when zapping a device.
Due to some udev events, that race condition makes the key
`ID_PART_ENTRY_NUMBER` show up too late.

The idea here is to retry multiple times before actually failing.

Fixes: https://tracker.ceph.com/issues/57144
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3fedce6c34fb3acfa57b5a028cb750041ee8fbd6)

test/{librbd, rgw}: retry when bind fail with port 0

there is chance that the bind() call may fail if we have another test
happen to pick the free port picked by operating system. in this case,
we just retry up to 42 times.

in theory, this change does not fully address the racing, but it should
help to alleviate this issue.

See-also: https://tracker.ceph.com/issues/57116
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit aa7885f7cc41390fcc8eeb82bc7142c3ff6a53f9)

Merge pull request #47765 from rzarzynski/wip-get_or_fail-debug-louder-quincy

quincy: msg: Log at higher level when Throttle::get_or_fail() fails

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>

Merge pull request #47619 from tchaikov/quincy-pr-47449

quincy: cmake: disable LTO when building pmdk

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47302 from petrutlucian94/wip-56728-quincy

quincy: msg: Fix Windows IPv6 support

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #47909 from Matan-B/wip-57372-quincy

quincy: SimpleRADOSStriper: Avoid moving bufferlists by using deque in read()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc: include read, write, metadata average latencies in doc/man.

Also, the sample cephfs-top image in the doc is outdated. Update that!

Fixes: http://tracker.ceph.com/issues/48619
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit aa04f3faedb6edcb0897e802a8390904deb6f936)

cephfs-top: display latency in milliseconds

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bf261f2a07111361ec8db36a7a4b13b54ff5d891)

cephfs-top: switch to displaying average latencies and stdev

Do away with cumulative latencies -- those are not much useful.
However, these types need to be maintained since `perf stats`
command (via mgr/stats plugin) includes them. So, maintain a
legacy metrics list which is ignored when choosing metrics to
display.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 60f33a8ca3055ec5ae5c8d67fd03f571bcec8892)

mgr/stats: include average latencies and stdev in `perf stat` dump

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b2bc45223b02ded7a5cc921980b3961c5e1d5893)

mgr/stats: auto generate metrics names from configured metrics

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit aaeec93efd2ae03d740299a5b22bb9203fbd7b8d)

client: forward read, write, metadata average latency and stdev

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 9b4f210b58571c0d88e5e01b90e6106cd894c3be)

Conflicts:
src/client/Client.cc:Added the 'if' condition to
read, write and metadata latency

mds, mgr: plumb in new client metrics

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f1083c0b3d41e6691945e8b40df8aa707a261217)

client: track average read, write and metadata IO latencies

And also standard deviation for each to measure the variance
(volatility) of latencies.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 967e24fe5c0efd9d7eb870494610fd1b4412f1d6)

client: allow overwrites to files with size greater than the max_file_size cfg

Before this change, overwriting from file-offset >= max_file_size config
returns "File too large" (even though the data is being written)
This change allow overwrites as the file size is not further increasing.

Fixes: https://tracker.ceph.com/issues/24894
Signed-off-by: Tamar Shacked <tshacked@redhat.com>
(cherry picked from commit a451a3670b7bb783ca6dcb8b2a31a8e6ec396899)

Merge pull request #47955 from zdover23/wip-doc-2022-09-04-backport-47841-to-quincy

quincy: doc/start: update documenting-ceph branch names

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>