crimson/net: print connection memory address as the identity

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/test_messenger: add pause before marking down a connection

To make sure the ACK messages are consumed, otherwise there can be
unexpected connect attempts to break the test.

Fixes: https://tracker.ceph.com/issues/57693
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #48350 from athanatos/sjust/wip-57739

crimson/.../replicated_request,logmissing_request: retain reference to req

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzynski@redhat.com>

Merge pull request #48308 from cyx1231st/wip-seastore-offset

crimson/os/seastore: introduce (64-8)-bit signed device_off_t and 32-bit signed segment_off_t

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>

Merge pull request #48349 from rzarzynski/wip-crimson-cltreq-errreply-debug

crimson/osd: introduce debug to ClientRequest::reply_op_error()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>

Merge pull request #47754 from batrick/i57248

qa: cleanup volumes on unwind

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #48064 from lxbsz/asok_fs

qa: the default CEPH_ASOK_DIR is asok/ now

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #48161 from trociny/wip-57589

cephfs-data-scan: make scan_links more verbose

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48182 from lxbsz/wip-57580

qa: make mount to be compatible with nautilus for blocklist

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #48421 from tchaikov/wip-cephadm-pyfakefs

cephadm: run tests as root

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #48423 from zdover23/wip-doc-2022-10-11-releases-pacific-grammar

doc/releases: improve grammar in pacific.rst

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #48396 from rhcs-dashboard/fix-57792-main

mgr/dashboard: fix broken login links

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #48337 from batrick/first-python

tools/cephfs: convert first-damage.sh to python

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #46772 from cbodley/wip-rgw-doc-bucket-index

doc/dev/radosgw: start describing the bucket index

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

cephadm: pin pyfakefs to 5.0

so the test is more repeatable.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cephadm: fix the indent

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

cephadm: run tests as root

as we will be create/access directories which are only accessible by
root with pyfakefs, and pyfake respects the uid of current effective
user, so if we run the test using non-root user, these tests would fail
after the fix for https://github.com/pytest-dev/pyfakefs/issues/489
is addressed in the pyfakefs we are using.

in this change, we will run the test on behalf of root, so we can
create and access the directories as we did before.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

doc/releases: improve grammar in pacific.rst

This commit accepts the grammar suggestions that were made by Cole
Mitchell in https://github.com/ceph/ceph/pull/48404.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48331 from tchaikov/crimson-stop

crimson/osd: shutdown on osdmaps' demand

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48238 from sisco0/fix-rgw-amqp-switch-compiler-warning-2

rgw: fix rabbitmq version checking for AMQP_STATUS_SSL_SET_ENGINE_FAILED

Merge pull request #48401 from zdover23/wip-doc-2022-10-09-crush-pdf-link-update

doc/various: update link to CRUSH pdf

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/various: update link to CRUSH pdf

This commit updates link to the research paper that announces and
explains the CRUSH algorithm. This link was broken in the migration from
the old Ceph website to ceph.io.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48399 from underacloud/wip-doc-spellin-errors

doc/radosgw: Fixes spelling errors in vault.rst

doc: Fixes spelling errors in vault.rst

Signed-off-by: Mary Frances underacloud.tech@gmail.com

Merge pull request #48393 from rzarzynski/wip-crimson-drop-omapiter

crimson: drop OmapIterator, use omap_get_values() instead

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48073 from rzarzynski/wip-crimson-asok-dump_recovery_reservations

crimson/osd: bring the dump_recovery_reservations asok cmd

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>

crimson/os: drop the OmapIterator concept

`get_omap_values()` is powerful enough to be used instead.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

test/crimson: unittest-seastar uses omap_get_values() instead of OmapIterator

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: convert seastarized PGLog from OmapIterator to omap_get_values()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: read_omap_for_push_op() takes max_len as ref, not ptr

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: ReplicatedRecoveryBackend uses omap_get_values() instead of OmapIterator

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #48374 from rkachach/fix_issue_57769

doc/cephadm: fixing grafana cert/key generation documentation

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #48128 from alimaredia/wip-intrusive-lru-destructor-add

common:  intrusive_lru destructor add

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/cephadm: fixing grafana cet/key generation documentation
Fixes: https://tracker.ceph.com/issues/57769
Signed-off-by: Redouane Kachach <rkachach@redhat.com>

mgr/dashboard: fix broken login links

Fixes: https://tracker.ceph.com/issues/57792
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #48378 from ceph/wip-yuriw-crontab-master

qa/testing: added pacific-x and quincy-x for reef release

Reviewed-by: Laura Flores <lflores@redhat.com>

crimson/osd: bring the dump_recovery_reservations asok cmd

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #48275 from liu-chunmei/rados-api-test

crimson/qa: add rados_api_tests.yaml for seastore

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48162 from Matan-B/wip-matanb-c-rados-test

qa/suites/crimson-rados/thrash: Adding workloads / Simplifying thrashers

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48277 from cyx1231st/wip-crimson-tools

crimson/net: fix msgr tools and test helpers

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #48351 from athanatos/sjust/wip-57738

crimson/osd/replicated_backend: block _submit_transaction on sending the messages

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #47934 from phlogistonjohn/jjm-nfs-sectype

mgr/nfs: add sectype option

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #48301 from PepperJo/cmp_write_C_vec

librbd: add compare-and-write vector C API

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #48285 from adk3798/upgrade-start-version

qa/cephadm: do not start upgrades from Octopus

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #47903 from rkachach/fix_issue_57366

mgr/cephadm: reconfig prometheus when service-discovery port changes

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #48372 from idryomov/wip-57766

qa/tasks/rbd_fio: bump default to fio 3.32

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

PendingReleaseNotes: add rbd_aio_compare_and_writev note

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #47731 from mjenglish/multiple-actuator-osd-specification

python-common/ceph/deployment: filter drives by actuators when creating osds

Merge pull request #48319 from gregsfortytwo/wip-57737-cephfs-doc

doc: discuss the standard multi-tenant CephFS security model

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #48379 from zdover23/wip-doc-2022-10-06-install-clone-source-master-to-main

doc/install: clone-source.rst s/master/main

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/install: clone-source.rst s/master/main

s/master/main in doc/install/clone-source.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

qa/workunits/fs/damage: update first-damage test for python

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

tools/cephfs: convert first-damage.sh to python

To have only one RADOS instance and improve performance/reliability.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #46925 from rkachach/fix_issue_44461

mgr/cephadm: add a simple mechanism to check grafana cert/key

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #48353 from athanatos/sjust/wip-dump-live-ops

crimson: improve dump_ops_in_flight

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #48352 from athanatos/sjust/wip-57740

crimson: allow pg creation to be canceled if pool is removed

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #48195 from athanatos/sjust/wip-57617

crimson/osd: populate version/user_version in reply for duplicate ops

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

qa/testing: added pacific-x and quincy-x for reef release

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #48289 from adamemerson/wip-warn-warn-warn

src/: silence Clang and GCC Warnings

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

doc: discuss the standard multi-tenant CephFS security model

Fixes: https://tracker.ceph.com/issues/57737
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

doc/mgr/nfs: document --sectype option for export create commands

Add documentation for the option to specify the sectype (for enabling kerberos)
when creating a new export.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/nfs: add test cases to verify sectype can be set

Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/nfs: add --sectype flag to nfs export create functions

Fixes: https://tracker.ceph.com/issues/57404
Add the sectype argument, also for use as a CLI option, to the functions
that create nfs exports. NFS Ganesha can use the sectype configuration
option to decide what kind of security will be applied/required for
connection to an export.

NOTE: This option is only useful when Ganesha is configured along with
LDAP/Kerberos integration. Configuration of that integration is outside
the scope of these patches.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/nfs: reformat long function calls to one line per arg

Using a hand rolled version of the long function call style similar to
that produced by the "Black" tool, put every argument to the
create_export function on a line of its own. This makes the code easier
to read and, importantly for an upcoming change, easier to add new
arguments.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/nfs: add sectype argument to nfs export management funcs

If the caller has supplied a sectype argument, pass it along to the
ganesha configuration functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

mgr/nfs: add awareness of the SecType option to ganesha_conf.py

This NFS-Ganesha option may be used to select kerberos integration in the
nfs client connection. If the option is not present in the configuration
dict, the option will not be present in the ganesha config blocks, just
as it was before.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/cephadm: do not start upgrades from Octopus

now that we officially have v18 in main the upgrade
will fail if upgrading from Octopus (v15) to main
because they are now considered 3 major releases apart

Additionally adding more debugging to the end
of the upgrade tests as something like `ceph health detail`
can be very useful to see at the end of failed
upgrade tests

Fixes: https://tracker.ceph.com/issues/57695
Signed-off-by: Adam King <adking@redhat.com>

qa/tasks/rbd_fio: bump default to fio 3.32

fio 3.16 doesn't build against recent kernel headers due to removal of
linux/raw.h.

Fixes: https://tracker.ceph.com/issues/57766
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #44564 from zdover23/wip-doc-2022-01-13-44150-cleanup-grafana-data-source-name

doc/mgr: name data source in "Man Install & Config"

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #48362 from zdover23/wip-doc-2022-10-05-rados-ops-crush-map-edits-grammar

doc/rados: refine English in crush-map-edits.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/rados: refine English in crush-map-edits.rst

This commit makes several refinements to the English in
rados/operations/crush-map-edits.rst, which refinements were suggested
by Cole Mitchell and Anthony D'Atri in the discussion of PR#48085.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

crimson/osd/osd_operation: drop unused OSDOperationRegistry::dump_client_requests

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/osd: notify waiters if pool for creating pg does not exist

Fixes: https://tracker.ceph.com/issues/57740
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #48085 from zdover23/wip-doc-2022-09-14-rados-ops-crush-map-prompt-update

doc/rados: update prompts in crush-map-edits.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

Merge pull request #47458 from rzarzynski/wip-all-kickoff-r

kickoff v18 reef

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

librbd: add compare-and-write vector C API

Add compare and write vector C API.
Require cmp_iov and write_iov to be the same length.

Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>

Merge pull request #46037 from mohmacht/ebdplugin

blk/kernel: add plugin system for devices with compression and move VDO support into plugin

Merge pull request #48346 from zdover23/wip-doc-2022-10-03-glossary-remove-redundancies

doc/glossary.rst: remove duplicates

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev/release-checklists: mention LIBRBD_VER_MINOR and VERSION

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

librbd: bump minor version to match R release

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #48218 from mchangir/qa-test_scrub_pause_and_resume_with_abort-wait-for-scrub-to-finish

qa: wait for scrub to finish

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

doc, qa/standalone/mon/misc: verify that len(monmap.features.persistent) == 10

Also updates the release checklist.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/dev/release-checklists.rst: document vX.0.0 tagging and mark it complete.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: introduce debug to ClientRequest::reply_op_error()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #48041 from cbodley/wip-rgw-notify-shutdown

rgw: rados service shuts down rados client on shutdown()

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #48152 from Vfrost98/GHC22OSD_Ceph

mgr/telemetry: fixed log exceptions as "exception" instead of "error"

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>

Merge pull request #48150 from li3317/ghc-osd-li3317

osd: break multi condition asserts into multiple asserts

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #47340 from kamoltat/wip-ksirivad-recreate-zilla-2104207

mon/OSDMonitor: Added extra check before mon.go_recovery_stretch_mode()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #46284 from votdev/issue_55673_cephadm_vagrantfile

mgr/cephadm: Deploying a cluster with the Vagrantfile fails

RGWServices_Def::shutdown() shuts down the rest of the services too

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: shut down RGWDataChangesLog before other services

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: rados service shuts down rados client on shutdown()

if we leave the rados client running, it will keep delivering
AioCompletions while we're shutting other things down

this resolves a valgrind use-after-free where rgw::notify::Manager
gets completions after its destruction

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #48155 from liangmingyuanneo/wip-rgw-aync-refcount

rgw: async refcount operate in copy_obj

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #47186 from cfsnyder/wip-cfsnyder-56645

rgw: log deletion status of individual objects in multi object delete request

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

doc/glossary.rst: remove duplicates

This commit removes similar but distinct entries for the following:
   * CephFS
   * Ceph Client

Removal of a glossary term that is referred to in the body of the
documentation suite requires the alteration of the text string
that refers to the glossary term. Alterations of this kind have
been made to doc/architecture.rst and doc/rados/api/index.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48292 from kotreshhr/doc-symlink-fix

doc: Fix disaster recovery doc

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #46821 from batrick/i55470

qa: postgresql test suite workunit

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48179 from batrick/i57598

qa: use correct recovery procedure for alternate meta pools

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #47795 from batrick/i57091

mds: check for some dentry damage in scrub

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #48333 from zdover23/wip-doc-2022-10-02-glossary-alphabetize

doc/glossary.rst: alphabetize glossary terms

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>