qa/testing: added pacific-x and quincy-x for reef release

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #48289 from adamemerson/wip-warn-warn-warn

src/: silence Clang and GCC Warnings

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #44564 from zdover23/wip-doc-2022-01-13-44150-cleanup-grafana-data-source-name

doc/mgr: name data source in "Man Install & Config"

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #48362 from zdover23/wip-doc-2022-10-05-rados-ops-crush-map-edits-grammar

doc/rados: refine English in crush-map-edits.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/rados: refine English in crush-map-edits.rst

This commit makes several refinements to the English in
rados/operations/crush-map-edits.rst, which refinements were suggested
by Cole Mitchell and Anthony D'Atri in the discussion of PR#48085.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48085 from zdover23/wip-doc-2022-09-14-rados-ops-crush-map-prompt-update

doc/rados: update prompts in crush-map-edits.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

Merge pull request #47458 from rzarzynski/wip-all-kickoff-r

kickoff v18 reef

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #46037 from mohmacht/ebdplugin

blk/kernel: add plugin system for devices with compression and move VDO support into plugin

Merge pull request #48346 from zdover23/wip-doc-2022-10-03-glossary-remove-redundancies

doc/glossary.rst: remove duplicates

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev/release-checklists: mention LIBRBD_VER_MINOR and VERSION

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

librbd: bump minor version to match R release

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #48218 from mchangir/qa-test_scrub_pause_and_resume_with_abort-wait-for-scrub-to-finish

qa: wait for scrub to finish

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

doc, qa/standalone/mon/misc: verify that len(monmap.features.persistent) == 10

Also updates the release checklist.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/dev/release-checklists.rst: document vX.0.0 tagging and mark it complete.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #48041 from cbodley/wip-rgw-notify-shutdown

rgw: rados service shuts down rados client on shutdown()

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #48152 from Vfrost98/GHC22OSD_Ceph

mgr/telemetry: fixed log exceptions as "exception" instead of "error"

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>

Merge pull request #48150 from li3317/ghc-osd-li3317

osd: break multi condition asserts into multiple asserts

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #47340 from kamoltat/wip-ksirivad-recreate-zilla-2104207

mon/OSDMonitor: Added extra check before mon.go_recovery_stretch_mode()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #46284 from votdev/issue_55673_cephadm_vagrantfile

mgr/cephadm: Deploying a cluster with the Vagrantfile fails

RGWServices_Def::shutdown() shuts down the rest of the services too

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: shut down RGWDataChangesLog before other services

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: rados service shuts down rados client on shutdown()

if we leave the rados client running, it will keep delivering
AioCompletions while we're shutting other things down

this resolves a valgrind use-after-free where rgw::notify::Manager
gets completions after its destruction

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #48155 from liangmingyuanneo/wip-rgw-aync-refcount

rgw: async refcount operate in copy_obj

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47186 from cfsnyder/wip-cfsnyder-56645

rgw: log deletion status of individual objects in multi object delete request

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

doc/glossary.rst: remove duplicates

This commit removes similar but distinct entries for the following:
   * CephFS
   * Ceph Client

Removal of a glossary term that is referred to in the body of the
documentation suite requires the alteration of the text string
that refers to the glossary term. Alterations of this kind have
been made to doc/architecture.rst and doc/rados/api/index.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48292 from kotreshhr/doc-symlink-fix

doc: Fix disaster recovery doc

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #46821 from batrick/i55470

qa: postgresql test suite workunit

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48179 from batrick/i57598

qa: use correct recovery procedure for alternate meta pools

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #47795 from batrick/i57091

mds: check for some dentry damage in scrub

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #48333 from zdover23/wip-doc-2022-10-02-glossary-alphabetize

doc/glossary.rst: alphabetize glossary terms

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/glossary.rst: alphabetize glossary terms

This commit (finally) alphabetizes the terms in the glossary.
This is not a grammar-correcting or usage-correcting commit.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

rgw: async refcount operate in copy_obj

Signed-off-by: Mingyuan Liang <liangmingyuan@baidu.com>

Merge pull request #48332 from zdover23/wip-doc-2022-10-01-rados-ops-ec-prompts

doc/rados: fix prompts in erasure-code.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

Merge pull request #48303 from PepperJo/cmp_write_test_bl_greater_fix

librbd: remove unnecessary write in compare-and-write test

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rados: fix prompts in erasure-code.rst

This commit adds unselectable prompts to doc/rados/
erasure-code.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48322 from zdover23/wip-doc-2022-10-01-rados-ops-ec-front-matter-rewrite

doc/rados: rewrite EC intro

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/rados: rewrite EC intro

This PR rewrites the front matter in the "Erasure Code"
section of the RADOS documentation. Previously, the information
in this section was syntactically confused. I have also fleshed
out the distinction between erasure coding and replication.

Co-Author: Cole Mitchell <cole.mitchell@gmail.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #43567 from jdurgin/wip-gov

doc/governance: add proposed structure

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

doc: merge new and old governance description

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

doc/governance: updates based on review comments

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

test/erasure-code: Fix signed compare warnings

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #48072 from ceph/wip-yuriw-release-17.2.4-main

doc: 17.2.4 Release Notes

doc: 17.2.4 Release Notes

Fixes: https://tracker.ceph.com/issues/57472
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #48259 from ktdreyer/build-win-fixes

win32: support building windows client on RHEL

Reviewed-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Reviewed-by: Justin Caratzas <jcaratza@redhat.com>

doc/governance: update based on review and CLT discussions

Clarified some parts of council that were discussed previously,
specifying the number of members and a staggered term.

Added a bit more about the steering committee - thinking the meetings
could be split between tactical (3/4 weeks) and strategic (monthly),
and still open to anyone to join the discussion, but restricted to
only members voting.

Removed the meeting section since that belongs more in a separate
place, like the ceph.io website.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

doc/governance: add proposed structure

This is Sage's summary of the Ceph leadership team discussions around
this topic. Still many details to be worked out, this is just one
concrete proposal as a basis for further discussion.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mon: Fix snprintf truncation warning by switching to fmtlib

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

mon: Use `ssize` to get rid of signed compare warning

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test/perf_local: Mark unused but useful variables as `maybe_unused`

Since they're used for a print that's commented out, keep them around
but silence the warning about them.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Prefer ssize or safe comparison to casts

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

test/librados: Use a call that can sleep for fractions of seconds

sleep(2) accepts an integral argument and `.01` rounds down to
zero. Use a sleep function that sleep for fractions of a second.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

doc: Fix disaster recovery doc

Fixes: https://tracker.ceph.com/issues/57734
Signed-off-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #47249 from nSedrickm/aria-labels-accessibility-update

mgr/dashboard: Improve level A accessibility for missing aria labels in tabs and listitems

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #48299 from neha-ojha/wip-crimson-nightlies

qa/crontab: run crimson-rados on the main branch

Reviewed-by: Sam Just <sjust@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #48215 from zmc/osd-containers-tasks

qa/tasks: Updates to enable containerized loop-backed OSDs

Merge pull request #48300 from zmc/db-e2e-url-fix

mgr/dashboard: Fix broken Fedora image URL

librbd: remove unnecessary write in cmp&write test

Remove write in compare and write test that tests
successful completion when bufferlist is greater then len.
The write was redundant and overwritten directly after.

Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>

mgr/dashboard: Fix broken Fedora image URL

This mirror apparently stopped working, breaking jobs that use this
script.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #47679 from cbodley/wip-rgw-sal-bootstrap

rgw: start on a rgw::sal::ConfigStore for boostrapping other stores

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

qa/crontab: run crimson-rados on the main branch

It isn't useful to run the crimson suite on Quincy because we don't
backport crimson fixes.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #48287 from andriytk/fix-rpmbuild

compressor: fix rpmbuild on RHEL-8

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #48194 from zmc/python36-prettytable

Merge pull request #48260 from phlogistonjohn/jjm-cephadm-import

cephadm: Clean up how test files import/use cephadm as a module

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #47814 from vshankar/wip-57072

pybind/mgr: check for empty metadata mgr_module:get_metadata()

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #48174 from batrick/i57597

qa: output debugging for cephfs-journal-tool/cephfs-data-scan

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #47991 from kotreshhr/qa-fix-57446

qa: Fix test_subvolume_snapshot_info_if_orphan_clone

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #47985 from kotreshhr/qa-fix-57205

qa: Fix test_subvolume_group_ls_filter_internal_directories

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #47879 from lxbsz/wip-57344

client: fix incorrectly showing the .snap size for stat

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48257 from batrick/i57657

qa: ignore expected scrub error

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #48264 from batrick/i57677

qa: ignore MDS_TRIM warnings when osd thrashing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

mds: check for some dentry damage in scrub

Fixes: https://tracker.ceph.com/issues/57091
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: remove unused method

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: note damaged dentry with first gt last

Only when loaded in cache.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: cluster log scrub failure for dirfrag

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: mark dirfrag good if repaired

Otherwise, we get wrong scrub errors after repair.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: only dump past_parent_snap if non-empty

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #48282 from ljflores/wip-telemetry-perf-fix

mgr/telemetry: handle daemons with complex ids

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

radosgw-admin: use ConfigStore and related zone APIs

replaces all of rgw_admin.cc's use of the RGWSystemMetaObj subclasses
with calls to the ConfigStore

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add higher-level functions to rgw_zone.*

duplicates some of the logic from member functions of
RGWRealm/Period/ZoneGroup/ZoneParams, but as free functions in terms
of the sal::ConfigStore APIs

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: fix --tier-config for 'zonegroup add'

ZONEGROUP_ADD applies changes to RGWZoneParams based on the
--tier-config option, but doesn't write those changes to rados

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: create a ConfigStore

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add create_config_store() factory

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw::sal::ImmutableConfigStore for json-based config

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw::sal::SQLiteConfigStore

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw::sal::RadosConfigStore

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw::sal::ConfigStore

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWRealm/Period/ZoneGroup/ZoneParams are fully public

Signed-off-by: Casey Bodley <cbodley@redhat.com>

compressor: fix rpmbuild on RHEL-8

When building rpms on latest RHEL-8.6:

    [  0%] Building CXX object src/compressor/CMakeFiles/compressor_objs.dir/Compressor.cc.o
    In file included from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/config_values.h:59,
                     from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/config.h:27,
                     from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/config_proxy.h:6,
                     from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/ceph_context.h:41,
                     from /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/compressor/Compressor.cc:23:
    /d/ceph-build/ceph-17.2.3-0-dff484dfc9e/src/common/options/legacy_config_opts.h:1:10: fatal error: global_legacy_options.h: No such file or directory
        1 | #include "global_legacy_options.h"
          |          ^~~~~~~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    make[2]: *** [src/compressor/CMakeFiles/compressor_objs.dir/build.make:76: src/compressor/CMakeFiles/compressor_objs.dir/Compressor.cc.o] Error 1
    make[1]: *** [CMakeFiles/Makefile2:7964: src/compressor/CMakeFiles/compressor_objs.dir/all] Error 2
    make: *** [Makefile:146: all] Error 2
    error: Bad exit status from /var/tmp/rpm-tmp.1CBUsC (%build)

    RPM build errors:
        Bad exit status from /var/tmp/rpm-tmp.1CBUsC (%build)

Solution: add legacy-option-headers dependency to compressor_objs.

Signed-off-by: Andriy Tkachuk <andriy.tkachuk@seagate.com>

qa: wait for scrub to finish

Wait for scrub to finish during test_scrub_pause_and_resume_with_abort
which otherwise races and fails with an incorrect assertion.

Fixes: https://tracker.ceph.com/issues/48812
Signed-off-by: Milind Changire <mchangir@redhat.com>

blk/kernel: add plugin system for devices with compression and move VDO support into plugin

The current VDO support implementation is buried inside the common/blkdev.cc
with a simple interface used by KernelDevice. It is not easily extendable
and can not be easily used for other devices providing similar capabilities.
This patch adds a plugin system that is based in its structure on the
erasure code plugin system and moves the VDO support code into a VDO plugin.

Signed-off-by: Martin Ohmacht <mohmacht@us.ibm.com>

mgr/telemetry: handle daemons with complex ids

Treating daemons as `<daemon_type>.x` caused a crash
in the Telemetry module since the current method does not cover a case
where a daemon id is more complex, i.e. `<daemon_type>.x.y`.

When we parse the daemon type and daemon id, we should
split it into a maximum of two pieces rather than splitting
it by every `.` character. Specifying `1` in the Python
.split() function will limit the split to a maximum of two items.

Fixes: https://tracker.ceph.com/issues/57700
Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #45379 from tobias-urdin/bump-openstack-versions

qa/suites/rgw: fix and update tempest and barbican tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: fix json format of ops log op data fields for multi object delete

The num_ok and num_err fields were being mistakenly dumped as bool values
vs. integers.

Signed-off-by: Cory Snyder <csnyder@iland.com>

rgw: log deletion status of individual objects in multi object delete request

Provides ops log visibility for individual objects that are deleted as part
of a multi-object delete request.

Fixes: https://tracker.ceph.com/issues/56645
Signed-off-by: Cory Snyder <csnyder@iland.com>

rgw: add hook for RGWOp subclasses to write custom metadata to ops logs

Adds a write_ops_log_entry method to the RGWOp class to allow it's
subclasses an opportunity to modify the request ops log entry.

Signed-off-by: Cory Snyder <csnyder@iland.com>

Merge pull request #48267 from rhcs-dashboard/nfs-exports-fs-permission-error

mgr/dashboard: handle the cephfs permission issue in nfs exports

Reviewed-by: Pedro González Gómez <pegonzal@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #48114 from athanatos/sjust/wip-crimson-multicore-for-review

crimson: multicore support for pgs in crimson-osd

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cephadm: rename import from cephadm to _cephadm

To be consistent with the other test_*.py files.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: rename import from cd to _cephadm

The `cd` name is an abbreviation of an obsolete name for cephadm.
As this is not widely known it could be confusing or conflict with
other common terms like `cd` for "change dir" or "compact disc".
Therefore we rename it to something much clearer.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: rename import from cd to _cephadm

The `cd` name is an abbreviation of an obsolete name for cephadm.
As this is not widely known it could be confusing or conflict with
other common terms like `cd` for "change dir" or "compact disc".
Therefore we rename it to something much clearer.

Signed-off-by: John Mulligan <jmulligan@redhat.com>