msg/async: fix banner_v1 check in ProtocolV2

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>

Merge PR #26614 into master

* refs/pull/26614/head:
osd: Implement lazy omap usage statistics

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26453 into master

* refs/pull/26453/head:
rpm: build with lttng on ppc64le (SUSE)
rpm: no lttng build on openSUSE s390x

Reviewed-by: Tim Serong <tserong@suse.com>

Merge PR #26667 into master

* refs/pull/26667/head:
cmake: link rgw tests with ${rgw_libs}
cmake: link ${rgw_libs} not rgw_a and rabbitmq
cmake: move rgw tests targets down to end

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge PR #26681 into master

* refs/pull/26681/head:
mon/ConfigMonitor: add 'config ls'

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge pull request #26693 from trociny/wip-rbd-bench-interrupt

rbd: close image when bench is interrupted

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26639 from cbodley/wip-38479

rgw: data sync drains lease stack on lease failure

Merge pull request #26685 from ceph/wip-testinfra-ceph-volume

ceph-volume: use our own testinfra suite for functional testing

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #26650 from LenzGr/typo-fix

mgr/dashboard: Fixed typos in environment.build.js

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26588 from votdev/relocate_empty_pipe

mgr/dashboard: Relocate empty pipe

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26447 from votdev/fix_notify_show

mgr/dashboard: Remove param when calling notificationService.show

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Fixed typo in environment.build.js

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #26630 from sebastian-philipp/ssh-simple-type-error

mgr/ssh: fix type and doc errors

Reviewed-by: Noah Watkins <noahwatkins@gmail.com>

rbd: close image when bench is interrupted

Previously when `rbd bench` was interrupted it left the lock and
watcher and one had to wait for them to expire before reusing or
removing the image.

As a bonus now it prints the benchmark summary on exit even if it
is interrupted.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #26596 from smithfarm/wip-release-notes-12_2_11

doc: add missing 12.2.11 release note

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge PR #26663 into master

* refs/pull/26663/head:
doc: change ruleset to id in crush map file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26595 into master

* refs/pull/26595/head:
osd/PG: log tracked pg refs under debug_refs
osd/PG: include tag in get/put debug lines
osd: clean up oncommit contexts in _process shutdown

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26637 into master

* refs/pull/26637/head:
common/ceph_context: experimental kludge for crimson
auth/AuthRegistry: mark 'secure' con mode experimental for now
msg/async: set ms_*_mode to crc only (no secure)

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26640 into master

* refs/pull/26640/head:
mon/OSDMonitor: remove old v1 addrs when converting blacklist to new TYPE_ANY form

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26645 into master

* refs/pull/26645/head:
rpm: use %_python_buildid to specify python-remoto dep

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #26678 into master

* refs/pull/26678/head:
mon: handle racing ms_handle_reset and handle_auth_request

Reviewed-by: Ricardo Dias <rdias@suse.com>

ceph-volume: use our own testinfra suite for functional testing

This allows us to run a smaller subset of tests from the ones provided
by ceph-ansible and test only the things that pertain to ceph-volume.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #26648 into master

* refs/pull/26648/head:
msg/async: msgr2: cleanup frame classes hacks

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #26523 from dillaman/wip-38387

librbd: add missing shutdown states to managed lock helper

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #26598 into master

* refs/pull/26598/head:
doc/releases/nautilus: dashboard tweaks
doc/releases/nautilus: more rados highlights
doc/release/nautilus: added RBD major features
doc/releases/nautilus: add msgr2 enablement steps
doc/releases/nautilus: add dashboard items

Reviewed-by: Kai Wagner <kwagner@suse.com>

Merge pull request #26570 from jtlayton/wip-rook

mgr/orchestrator/rook: allow the creation of OSDs in directories

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

doc/releases/nautilus: dashboard tweaks

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #26671 into master

* refs/pull/26671/head:
doc: Replaced "plugin" with "module" in the Mgr documentation

Reviewed-by: Wido den Hollander <wido@42on.com>
Reviewed-by: Sage Weil <sage@redhat.com>

mon/ConfigMonitor: add 'config ls'

Signed-off-by: Sage Weil <sage@redhat.com>

mon: handle racing ms_handle_reset and handle_auth_request

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: log tracked pg refs under debug_refs

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: include tag in get/put debug lines

Signed-off-by: Sage Weil <sage@redhat.com>

osd: clean up oncommit contexts in _process shutdown

If we have queued PG completions in the workqueue and have to shut down,
delete them, so that the PG refs get cleaned up.

Fixes: http://tracker.ceph.com/issues/38431
Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: msgr2: cleanup frame classes hacks

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge PR #26628 into master

* refs/pull/26628/head:
msg/async: msgr2: expected tags validation
msg/async: msgr2: template specialized write function for frames
msg/async: msgr2: extract frame classes into its own header

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26227 from Tinatinak/api_documentation

mgr/dashboard: Improved support for generating OpenAPI Spec documentation

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #26575 from tspmelo/wip-multi-backstore-ui

mgr/dashboard: Add support for iSCSI's multi backstores (UI)

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #26622 from Songweibin/wip-fix-rbd-group

rbd: fix segmentation fault when rbd_group_image_list() getting -ENOENT

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #26432 from dillaman/wip-librbd-discard

librbd: skip small, unaligned discard extents by default

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

doc: Replaced "plugin" with "module" in the Mgr documentation

The documentation currently refers to Ceph Manager Modules as
"plugins" in many places, while the command line interface uses
"module" to enable/disable modules. Replaced all occurences
of "plugin" with "module" in the docs, to avoid confusion and to
be in alignment with the CLI. Also fixed the capitalizations of some
module chapters.

Fixes: https://tracker.ceph.com/issues/38481
Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

mgr/dashboard: Add support for iSCSI's multi backstores (UI)

Fixes: https://tracker.ceph.com/issues/38286
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Relocate empty pipe

Move empty pipe into shared/pipes directory.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #26586 from votdev/change_notify_icon_color

mgr/dashboard: Change icon color in notifications

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #26649 from tspmelo/wip-i18n

mgr/dashboard: Update I18N translation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26589 from Devp00l/issue-38382

mgr/dashboard: PGs will update as expected

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #26116 from votdev/feature_37934

mgr/dashboard: Configure all mgr modules in UI

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #26592 from nathan-weinberg/fs-breadcrumb

mgr/dashboard: Added breadcrumb tests to Filesystems menu

Merge pull request #26647 from tspmelo/wip-npm-update-19-03

mgr/dashboard: Update npm packages

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #26219 from yaozongyou/mon-cleanups

mon: some tiny cleanups related class forward declaration

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonell@redhat.com>

cmake: link rgw tests with ${rgw_libs}

they should not link against rabbitmq if WITH_RADOSGW_AMQP_ENDPOINT is
not enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: link ${rgw_libs} not rgw_a and rabbitmq

so we don't need to check the condition everywhere

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: move rgw tests targets down to end

so we can reference `RabbitMQ::RabbitMQ` which is defined after
`find_package(RabbitMQ REQUIRED)`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: change ruleset to id in crush map file

ruleset is not used after merging below patch
   commit f9a095deb1e74642d88bc9015684b0be113cc5f9
       crush: s/ruleset/id/ in decompiled output
       Moving away from the 'ruleset' terminology.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>

Merge pull request #26527 from iotcg/doc

use preferred commands for ceph config-key

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26558 from k0ste/prometheus_metadata

mgr/prometheus: get osd_objectstore once instead twice.

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

common/ceph_context: experimental kludge for crimson

Signed-off-by: Sage Weil <sage@redhat.com>

auth/AuthRegistry: mark 'secure' con mode experimental for now

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #26646 into master

* refs/pull/26646/head:
doc: fix typo in objectops.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

doc/releases/nautilus: more rados highlights

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26627 from jan--f/c-v-38472

ceph-volume: don't create osd['block.db'] by default

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

msg/async: msgr2: expected tags validation

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #26225 from jan--f/c-v-simple-activate-all

ceph-volume: add --all flag to simple activate

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #26481 from votdev/gitignore_wheelhouse

mgr: Ignore __pycache__ and wheelhouse dirs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: Update I18N translation

Add support for Chinese (Simplified), Czech, French, Indonesian and Spanish.

Fixed Portuguese (PT) locale source.

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #26338 from rhcs-dashboard/ts-cleanup

mgr/dashboard: typescript cleanup

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Introduce upperFirst pipe

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Configure all mgr modules in UI

Fixes: https://tracker.ceph.com/issues/37934
Signed-off-by: Volker Theile <vtheile@suse.com>

qa/suites/rbd: enable partial discard test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: align discard requests to default bluestore allocation size

This will match the new krbd behavior of aligning discards to the
default bluestore minimum allocation size. Requests that are too
small (or the misaligned head / tail of the request) will be
ignored since it will not result in freed space.

Fixes: http://tracker.ceph.com/issues/38146
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #26636 into master

* refs/pull/26636/head:
messages/MMonMgrDigest: use set_data() to avoid screwing up byte_throttler

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26130 from minshenglin/wip-37855

rgw_file: use correct secret key to check auth

mgr/dashboard: Update npm packages

Signed-off-by: Tiago Melo <tmelo@suse.com>

doc: fix typo in objectops.rst

Signed-off-by: Lan Liu <liulan@umcloud.com>

rpm: use %_python_buildid to specify python-remoto dep

Without this patch, the Python 2 version of python-remoto is always installed,
even on Python 3 systems.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

librbd: set *image_size to 0 when getting ENOENT in rbd_group_image_list()

Fixes: http://tracker.ceph.com/issues/38468
Signed-off-by: songweibin <song.weibin@zte.com.cn>

osd: Implement lazy omap usage statistics

Opportunistic gathering of omap statistics during deep scrub.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

doc: use preferred commands for ceph config-key

'del' & 'list' & 'put' are deprecated. It's preferred to
using 'rm' & 'ls' & 'set' for ceph config-key

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>

Merge pull request #26611 from falcon78921/wip-doc-log-based-pg

doc: updated reference link for log based PG

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

mon/OSDMonitor: remove old v1 addrs when converting blacklist to new TYPE_ANY form

Fixes: http://tracker.ceph.com/issues/38477
Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: set ms_*_mode to crc only (no secure)

'secure' mode will be experimental in the initial nautilus release.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #26617 into master

* refs/pull/26617/head:
rpm: Fix pyOpenSSL dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge PR #26623 into master

* refs/pull/26623/head:
mgr/PyModule: put mgr_module_path first in sys.path

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #26543 into master

* refs/pull/26543/head:
Revert "ceph.spec.in: python-kubernetes broken on rhel"

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge PR #26625 into master

* refs/pull/26625/head:
rpm,cmake: additional fixes to make amqp support optional

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22755 from zhangsw/feature-rgw-appendobj

rgw: Add append object api.

rgw: data sync drains lease stack on lease failure

on lease failure, call drain_all() to collect the spawned lease stack.
otherwise, the parent coroutine will inherit this spawned stack and may
not drain or collect it manually. RGWDataSyncShardControlCR, for
example, does not drain/collect because it only uses call() to run a
single RGWDataSyncShardCR, so all lease stacks from RGWDataSyncShardCR
just pile up in the RGWDataSyncShardControlCR

Fixes: http://tracker.ceph.com/issues/38479
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #26610 into master

* refs/pull/26610/head:
doc: updated governance link for Ceph project leads

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

librbd: skip partial discards by default

Avoid the IO expense of writing zeros mid-object when a discard
is attempting to free space.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: image replay should retry asok registration upon failure

If the asok registration fails (perhaps due to a race condition with
a deleted and recreated image of the same name), periodically attempt
to register the missing asok hook.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: failure to initialize pool replayer should stop leader

The leader watcher should not start processing requests if it failed
to initialize.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: fixed potential bootstrap cancel race condition

If the image replay was canceled prior to the start of the bootstrap
stage, the image replayer would be stuck attempting to shut down if
the bootstrap is paused behind an image sync.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: complete pool watcher initialization if object missing

If the mirroring object is missing, complete the initialization and
continue to retry in the background. This is useful for cases where
the remote doesn't (yet) have mirroring enabled but the remote
pool watcher initialization is delaying the leader watcher promotion
to the point where the leader is blacklisted by its peers.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: abort trash watcher initialization if create blacklisted

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26603 from dillaman/wip-38453

rbd-mirror: use active/active policy by default

Reviewed-by: Mykola Golub <mgolub@suse.com>

messages/MMonMgrDigest: use set_data() to avoid screwing up byte_throttler

Use the existing set_data() helper to avoid breaking the byte_throttler
accounting.

Fixes e4ae368ff7a5396194f8bdd5692429af5457998b

Fixes: http://tracker.ceph.com/issues/38295
Signed-off-by: Sage Weil <sage@redhat.com>

rbd-mirror: avoid attempting to reacquire lock if blacklisted

The pool replayer state machine will automatically be restarted
when a blacklist is detected.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: don't delete global perf counters before shut down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add missing shutdown states to managed lock helper

The PRE_SHUTTING_DOWN and SHUTTING_DOWN states were missed
in the 'is_state_shutdown' helper method. This resulted in
rbd-mirror potentially entering an infinite loop during
shutdown.

http://tracker.ceph.com/issues/38387
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd: delete pools before stopping rbd-mirror

This better mimics the behavior of teuthology and tests rbd-mirror
daemon's ability to handle a pool deletion.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mgr/rook: have it create bluestore OSDs in data_directories

When the data_directories field is populated, use that to populate the
spec.storage.nodes.<host>.directories field in the CephCluster object.
That will cue the rook operator to create an OSD in using an area under
that directory for storage.

Signed-off-by: Jeff Layton <jlayton@redhat.com>