msg/async: add async msg perf counter

Signed-off-by: Xinze Chi <xinze@xsky.com>
Updated to account for the new ProtocolV1/ProtocolV2

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

tools/crushtool: add new Option --bucket-tree to show bucket map tree

Signed-off-by: linbing <linbing@t2cloud.net>
Update the crushtool cli test template for new options.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

msg/async: bug fix 23600 by setting _fd = 1.

After listen_socket was closed, it entered the start() processf magically. so
I think we should make '_fd = -1' when listen_socket was closed. And in the
start process, determine the value of listen_socket, if it equal to -1, return
the process.

Fixes: https://tracker.ceph.com/issues/23600
Signed-off-by: shangfufei <shangfufei@inspur.com>
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #27581 from LenzGr/master-documentation

doc: Improved the dashboard proxy config section

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27593 from smithfarm/wip-39291

cmake: revert librados_tp.so version from 3 to 2

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #27514 from tchaikov/wip-crimson-auth-client

crimson: enable cephx for v2 msgr

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #27624 into master

* refs/pull/27624/head:
doc: fixed telemetry module reference link

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/auth: let AuthClient::get_auth_request return a named struct

better readability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: use const reference for Connection::auth_registry

it is not optional now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

auth/AuthRegistry: more constness

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: remove "#warning" in v1 implementation

since we've implemented the v2 support, these reminders are not needed
anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: reset promise using `{}`

better readability, less repeating.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonClient: remove unused AuthMethodList

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: use auth_registry for querying auth_methods

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fixed telemetry module reference link

Used :ref: label for telemetry reference in Nautilus.

Fixes: https://tracker.ceph.com/issues/39337
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Merge PR #27519 into master

* refs/pull/27519/head:
doc/rados/operations/health-checks: document new bluestore warnings
os/bluestore: alert on fm/bdev size mismatch
os/bluestore: introduce legacy statfs alert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27556 into master

* refs/pull/27556/head:
mgr/rook: Remove support for Rook older than v0.9

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
Reviewed-by: Blaine Gardner <bgardner@suse.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #27563 into master

* refs/pull/27563/head:
mon/OSDMonitor: respect crush node flags for can_mark_*()
osd/OSDMap: add get_crush_node_flags(int osd)
mon/OSDMonitor: make 'osd {add,rm}-{noin,noout,...}' support crush nodes
osd/OSDMap: raise OSD_FLAGS health alert for crush node flags, too
osd/OSDMap: add flags for crush nodes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #27568 into master

* refs/pull/27568/head:
doc/release/nautilus: proscribe minimum hammer tunables and straw2 on upgrade
common/options: update mon_crush_min_required_version=hammer

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #26461 from adamemerson/wip-aio-aio-aio-aio-queue-like-an-egyptian

♪ Ai-o Ai-o Ai-o Ai-o… queue like an Egyptian ♪

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27616 from yuriw/wip-yuriw-crontab-master

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Merge pull request #27381 from markhpc/wip-prioritycache-manager

common/PriorityCache: Implement a Cache Manager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #26089 from taoCH/wip-librgw-create-bucket-bug

rgw: unexpected crash when creating bucket in librgw

crimson: use lowres_system_clock for keepalive timestamps

we should use chrono types for representing time_point and duration,
utime_t could be used for the format on wire though.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use v2 addresses

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: register monc as an AuthClient

so msgrs will be able to turn to monc when they try to authenticate
themselves.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: implement AuthClient

for v2 msgr authentication support, let ceph::mon::Client implement
ceph::auth::AuthClient.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: rename Connection::authenticate()

rename Connection::authenticate() to Connection::authenticate_v1() to
differentiate it from Connection::authenticate_v2(). the latter will be
added for v2 msgr support.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: add setters for Connections

so we don't directly access its member variables

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: refactor Connection::create_auth() so it can be reused

Connection::create_auth() will be shared by v1 and v2 msgrs, so we need
to decouple it from v1 implementation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/auth: refactor AuthClient interface

* we'd better return value by "return" not by passing output prameters by
  pointers.
* remove unused parameters from AuthClient::handle_auth_done(),
  `session_key` and `connection_secret` are returned by setting
  corresponding member variables of `auth_meta`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/auth: rename auth::auth_error to auth::error

* s/auth::auth_error/auth::error/: less repeating this way
* auth/Errors.{h,cc}: remove them. as we don't need to use error code to
  construct exceptions in this context
* move auth::error into auth::AuthClient

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27564 into master

* refs/pull/27564/head:
ceph-objectstore-tool: Rename dump-import to dump-export
osd: Fix fast path to use version bumped value

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27546 into master

* refs/pull/27546/head:
mgr/progress: fix typo
mgr/progress: behave if pgs disappear (due to a racing pg merge)

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #27036 from votdev/issue_24453

mgr/dashboard: Manager should complain about wrong dashboard certificate

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #27433 from rhcs-dashboard/wip-36243-master

mgr/dashboard: Add frontend code documentation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

cmake: revert librados_tp.so version from 3 to 2

Post-mortem analysis:

librados.so and librados_tp.so are packaged together in the librados2
RPM.

c680fb10f598267f15a37fdb7bc44529c37c9318 bumped
the librados.so and librados_tp.so versions from 2 to 3.

Later, 7bf6b5ee1208a359826c74ab033e6bbbfc65969f reverted the librados.so
version from 3 to 2, but left the librados_tp.so version at 3.

Fixes: http://tracker.ceph.com/issues/39291
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: Improved the dashboard proxy config section

Added paragraph that explains how standby dashboard instances perform
HTTP redirections to the currently active ceph-mgr node.
Added note about current limitations (see Bug#24662 for details).

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

doc: dashboard: refined object gateway configuration

As discussed in BUG#39252, the dashboard currently does not
support all possible RGW configuration possibilities when trying
to obtain hostname and port number. Update the docs to better
reflect this.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #27530 from yuvalif/fix_pubsub_duplicates

rgw/pubsub: handle subscription conf errors better

Merge pull request #27552 from tchaikov/wip-corpus.rst

doc/corpus: update to adapt the change from autotools to cmake

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Changcheng Liu <changcheng.liu@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/corpus: update to adapt the change from autotools to cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/operations/health-checks: document new bluestore warnings

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27493 from yuvalif/pubsub_zone_conf_doc

rgw/pubsub: clarify pubsub zone configuration

Merge PR #27585 into master

* refs/pull/27585/head:
messages: remove MNop

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27578 from cbodley/wip-rgw-pubsub-warning

rgw: silence a -Wunused-function warning in pubsub

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #27178 from pritha-srivastava/wip-rgw-iam

rgw: Adding 'iam' namespace for Role and User Policy related REST APIs.

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

os/bluestore: alert on fm/bdev size mismatch

Relates to: https://tracker.ceph.com/issues/39151

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: introduce legacy statfs alert

Fixes: https://tracker.ceph.com/issues/39146
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #27553 from theanalyst/doc/releases/12.2.12

doc: releases/luminous: release notes for 12.2.12

Reviewed-By: Nathan Cutler <ncutler@suse.com>

doc: releases/12.2.12: fix changelog to add issues without trackers

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #27491 from sebastian-philipp/rook-service-ls-rgw

mgr/rook: Added missing `rgw` daemons in `service ls`

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #27462 from sebastian-philipp/rook-fix-rgw

mgr/rook: Fix RGW creation

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #27339 from david-z/wip-fix-rbd-map-hang

krbd: fix rbd map hang due to udev return subsystem unordered

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

messages: remove MNop

MNop was introduced along with xio messenger. since we've dropped
xiomessenger in #27021, there is no need to keep MNop around anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27517 into master

* refs/pull/27517/head:
mon: make mon_osd_down_out_subtree_limit update at runtime

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #27458 into master

* refs/pull/27458/head:
bluestore: fix for FreeBSD iocb structure

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27485 into master

* refs/pull/27485/head:
mgr/ActivePyModules: handle_command - fix broken lock

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27506 into master

* refs/pull/27506/head:
src/tests/crushtool: Add test for an empty default
crush/CrushCompiler: Fix __replacement_assert

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27271 from idryomov/wip-krbd-wac-on-dm

qa: add device mapper and lvm test cases for stable pages

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27533 from dillaman/wip-39257

rbd-mirror: periodically flush IO and commit positions

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #27229 from dillaman/wip-librbd-writearound

librbd: introduce new default write-around cache policy

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #27446 from trociny/wip-rbd-io-simple-scheduler-improvements

librbd: dispatch delayed requests only if read intersects

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27513 from ddiss/librados_silence_unused_vars

librbd: silence -Wunused-variable warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: silence a -Wunused-function warning in pubsub

Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/release/nautilus: proscribe minimum hammer tunables and straw2 on upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: update mon_crush_min_required_version=hammer

Hammer is a good point to set the minimum at:

 - it includes straw2 and vary_r, which are the worst offenders among the
   old tunables
 - it does not include chooseleaf_stable, which improves placement
   considerably but *also* shuffles basically all data in the cluster.

Getting everyone up to straw2 support means we can also proscribe the
'ceph osd crush set-all-straw-buckets-to-straw2' command.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: Rename dump-import to dump-export

If user specifies dump-import it will still work, but isn't
in the usage that way.

Fixes: http://tracker.ceph.com/issues/39284
Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix fast path to use version bumped value

Fix the code so that the version is in one place
for both the slow and fast path.

Caused by: d2ca3d2feb442f97ca89023c7d01178d96f517a6

Fixes: http://tracker.ceph.com/issues/39281
Signed-off-by: David Zafman <dzafman@redhat.com>

mgr/dashboard: Add frontend code documentation

Adds compodoc as auto-documentation tool for Angular frontend code.

Fixes: http://tracker.ceph.com/issues/36243
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mon/OSDMonitor: respect crush node flags for can_mark_*()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: add get_crush_node_flags(int osd)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27470 from cbodley/wip-39160

rgw: don't recalculate etags for slo/dlo

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

mon/OSDMonitor: make 'osd {add,rm}-{noin,noout,...}' support crush nodes

Set noin, noout, etc. flags on crush nodes.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27528 from cbodley/wip-qa-rgw-no-debuginfo

qa/rgw: disable debuginfo packages

osd/OSDMap: raise OSD_FLAGS health alert for crush node flags, too

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: Remove support for Rook older than v0.9

http://tracker.ceph.com/issues/39278

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

osd/OSDMap: add flags for crush nodes

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: write-around cache should honor FUA op flag

If force unit access is specified, the cache should be bypassed just
like other non-optimized IO through the cache.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: support new bench 'full-seq' io pattern option

This ensures IOs are issued in sequential order regardless of the current
IO thread setting.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: releases/luminous: release notes for 12.2.12

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mgr/progress: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/progress: behave if pgs disappear (due to a racing pg merge)

If one of the PGs we care about disappears, consider it complete.

Fixes: http://tracker.ceph.com/issues/38157
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #27473 into master

* refs/pull/27473/head:
mgr/DaemonServer: prevent pgp_num reductions from outpacing pg_num merges

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #27477 into master

* refs/pull/27477/head:
os/bluestore: check return value of func _open_db_and_around.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #27499 into master

* refs/pull/27499/head:
qa/standalone/osd/osd-markdown: fix dup command disabling

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #27516 into master

* refs/pull/27516/head:
mgr/rook: Fix Python 2 regression

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Gabriel Brascher <gabriel@apache.org>

Merge pull request #26816 from iotcg/doc

doc: correct example to use vstart to run up cluster

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27535 from yuriw/wip-yuriw-wip-yuriw-crontab-master

qa/tests: added rados on master, reduced fs, rbd, multimds

Merge pull request #27428 from cyx1231st/wip-seastar-msgr-write

crimson/net: extract do_write_dispatch_sweep()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

qa/tests: added rados on master, reduced fs, rbd, multimds

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #27515 from tchaikov/wip-seastar-fix-monc

test/crimson: fix build of unittest_seastar_monc

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

src/tests/crushtool: Add test for an empty default

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #27524 from tchaikov/wip-39164

install-deps.sh: install '*rpm-macros'

Reviewed-by: Boris Ranto <branto@redhat.com>

rbd-mirror: periodically flush prior to updating status

This ensure the most up-to-date commit position is included in the
status report.

Fixes: http://tracker.ceph.com/issues/39257
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: simplify image replayer IO / commit position flush path

There is no current user of the callback context provided to the
flush method and there are no longer any reason to have the flush
support methods be virtual.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27443 from Polynomial-C/tinfo

cmake: Fix build against ncurses with separate libtinfo

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27527 into master

* refs/pull/27527/head:
qa/tests: run luminous-x and mimic-x 2 times a week but with high priority

Reviewed-by: Sage Weil <sage@redhat.com>

rgw/pubsub: handle subscription conf errors better

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>