Merge PR #27595 into master

* refs/pull/27595/head:
osd: add 'ceph osd stop <osd.nnn>' command

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27619 into master

* refs/pull/27619/head:
mgr/BaseMgrModule: run MonCommandCompletion on the finisher
mgr/BaseMgrModule: fix leak

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge PR #27651 into master

* refs/pull/27651/head:
test/osd/safe-to-destroy.sh: fix typo

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27663 into master

* refs/pull/27663/head:
osd/OSDMap: Replace get_out_osds with get_out_existing_osds

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27631 from dmick/wip-telemetry-crasharray

mgr/telemetry: change crash dict to a list

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27599 from gregsfortytwo/wip-stale-prs

Miscellaneous lost fixes

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #27615 from ifed01/wip-ifed-verbose-bluefs-replay

os/bluestore: be more verbose doing bluefs log replay.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27509 from Aran85/fix-pg-dout

osd/PG: the warning seems more serious than what it wanna transmit

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27592 from tchaikov/wip-ref

message,mgr: drop MessageFactory and friends and use ref_t<> in mgr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #27633 from xiexingguo/wip-39340

osd/osd_types: pool_stat_t::dump - fix 'num_store_stats' field

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27658 from gregsfortytwo/wip-39367-crc-peer

msg: output peer address when detecting bad CRCs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27657 from anthonyeleven/patch-1

doc: operations: improve reweight-by-utilization

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: operations: reweight-by-utilization typo

Add a missing backquote delimiter.

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/BaseMgrModule: run MonCommandCompletion on the finisher

The completion takes the ActivePyModules::lock, but the caller may be
holding an internal lock for Objecter and/or MonClient.  Avoid deadlock
by putting the completion on our finisher.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/BaseMgrModule: fix leak

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #27685 into master

* refs/pull/27685/head:
doc: fix boot transition in mds state diagram

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27686 from dzafman/wip-39383

osd: Increase log level of messages which unnecessarily fill up logs

Reviewed-by: Neha Ojha <nojha@redhat.com>

osd: Increase log level of messages which unnecessarily fill up logs

Caused by: 834d3c19a774f1cc93903447d91d182776e12d18

Fixes: https://tracker.ceph.com/issues/39383
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #27037 from alimaredia/wip-rgw-dpp-rest-s3-swift

rgw: log refactoring for rgw_rest_s3/swift ops

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: fix boot transition in mds state diagram

MDSMonitor assigns a booting MDS immediately to standby. From there, the
standby may be promoted to standby-replay or some state leading to active.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27375 into master

* refs/pull/27375/head:
rgw: pass mostly_omap flag when opening/creating pools
rgw/rgw_rados: pass mostly_omap flag when opening/creating pools
rgw: move rgw_init_ioctx() to rgw_tools.cc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27576 from cbodley/wip-39287

rgw: RGWPeriodPusher uses zone system key for inter-zonegroup messages

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #27518 from jtlayton/vstart

vstart: add new option to pass list of block devices to bluestore

Merge pull request #27567 from cbodley/wip-39283

rgw: multisite log trimming only checks peers that sync from us

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #27275 from Devp00l/issue-39037

mgr/dashboard: Localization for date picker module

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

osd: add 'ceph osd stop <osd.nnn>' command

stop command can be used to force stopping a specified osd daemon, e.g.,
you don't have to pre-figure out where it located.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/OSDMap: Replace get_out_osds with get_out_existing_osds

Fixes: http://tracker.ceph.com/issues/39154
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge PR #27235 into master

* refs/pull/27235/head:
cephfs-shell: support non-octal mode
cephfs-shell: fix rm command
cephfs-shell: fix lls command
cephfs-shell: fix lcd command
cephfs-shell: fix umask command
cephfs-shell: fix cat command
cephfs-shell: fix chmod command
cephfs-shell: fix chdir command
cephfs-shell: fix mv command
cephfs-shell: fix rmdir command
cephfs-shell: fix ls command
cephfs-shell: fix get command
cephfs-shell: fix put command
cephfs-shell: fix mkdir command
cephfs-shell: helper functions and infra changes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

msg/Message: add SafeMessage which forbids put/get

This is an opt-in wrapper around Message inheritance to prevent
undesired get/put calls on the object. Instead, you must/should use a
smart ptr to adjust the reference count.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/MgrClient: use ref_t<M>

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/{Mgr,MgrStandby}: use ref_t<M>

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/DaemonState: pass const by reference

DaemonPerfCounters::update() does not change the `MMgrReport` message,
neither does it expect a nullptr. so pass by const reference.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/DaemonServer: use ref_t<M>

to avoid explicit use of RefCountedObj interface. and let
intrusive_ptr<> to take care of the life cycle of messages.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/Message: remove Message::const_ref

Signed-off-by: Kefu Chai <kchai@redhat.com>

common: use ref_t<M>

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/Message: drop MessageFactory and friends

we'd better decouple the helpers from the definition of Message types,
to keep the definition Messages simple and concise.

Signed-off-by: Kefu Chai <kchai@redhat.com>

messages/MOSDPeeringOp: do not include PGPeeringEvent.h

forward delcaration would suffice

Signed-off-by: Kefu Chai <kchai@redhat.com>

messages: do not use MessageSubType anymore

so we can drop it in future commit

Signed-off-by: Kefu Chai <kchai@redhat.com>

messages: #include necessary header

for instance, MLogRec is defined in PGPeeringEvent.h, we should not rely
other header(s) to do this for us. so just #include PGPeeringEvent.h for
its definition.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mds/MDSCache: init empty string_view with {}

nullptr is not a valid null-terminated character sequence

Signed-off-by: Kefu Chai <kchai@redhat.com>

src/: use make_message<Foo> instead of Foo::create()

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/Message: remove msgref_cast()

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/Message: add aliases for static_pointer_cast

strictly speaking, they are not "alias" of boost::static_pointer_cast,
but to avoid an extra conversion, instead of using
boost::static_pointer_cast<>, i am using std::static_cast<> directly for
creating boost::intrusive_ptr<> instance.

Signed-off-by: Kefu Chai <kchai@redhat.com>

client,mds: use ref_cast<Foo> instead of Foo::msgref_cast

Signed-off-by: Kefu Chai <kchai@redhat.com>

src/: use ref_t<Message> instead of Message::ref

Signed-off-by: Kefu Chai <kchai@redhat.com>

src/: use cref_t<Message> instead of Message::const_ref

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/RefCountedObj: move private members

To follow code-style guidelines.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

common/RefCountedObj: use type of nref

"int" may narrow the ref count (although very unlikely!).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

msg: add aliases of ref_t and cref_t

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27649 into master

* refs/pull/27649/head:
PendingReleaseNotes: 14.2.1 note on crush required version

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

msg: output peer address when detecting bad CRCs

Fixes: http://tracker.ceph.com/issues/39367
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #27600 from neha-ojha/wip-39006

osd/OSD.cc: make osd bench description consistent with parameters

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27481 into master

* refs/pull/27481/head:
qa: add a method to test if a file can be created
qa: add wrappers for vstart's run() with check_status as False
qa: add a method to create an empty file with any user
qa: add stdin parameter to run_shell
qa: command to be ran as a different user should be a single string
qa: looks for quotes in Python shell and get rid of them
qa: move checks on command arguments to a different method
qa: accept args as strings
qa: add a method to run given command as root user
qa: add a method to run a command as a different user
qa: allow keeping sudo in command arguments
qa: make execution abortion optional for command ran through run_shell()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

doc: operations: improve reweight-by-utilization

Add the missing `max_change`, `max_osds`, and `--no-increasing` parameters to `reweight-by-utilization` and `test-reweight-by-utilization`.  Minor adjustments to wording.

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #27282 from mogeb/wip-lvmcache

ceph-volume: add utility functions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

qa: add a method to test if a file can be created

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add wrappers for vstart's run() with check_status as False

Too often in tests it is required to run a command that we want to (or
will probably) fail so that we can check if it failed for expected
reason(s) (eg. a testcase where command must fail with "Permission
denied"). Therefore, add few wrapper methods for vstart_runner.py's
run() with check_status's default value as False so that we don't need
to pass check_status's value at every call to run() or it's other
wrappers.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add a method to create an empty file with any user

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add stdin parameter to run_shell

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: command to be ran as a different user should be a single string

Signed-off-by: Rishabh Dave <ridave@redhat.com>

test/osd/safe-to-destroy.sh: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async: add async msg perf counter

Signed-off-by: Xinze Chi <xinze@xsky.com>
Updated to account for the new ProtocolV1/ProtocolV2

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

tools/crushtool: add new Option --bucket-tree to show bucket map tree

Signed-off-by: linbing <linbing@t2cloud.net>
Update the crushtool cli test template for new options.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

msg/async: bug fix 23600 by setting _fd = 1.

After listen_socket was closed, it entered the start() processf magically. so
I think we should make '_fd = -1' when listen_socket was closed. And in the
start process, determine the value of listen_socket, if it equal to -1, return
the process.

Fixes: https://tracker.ceph.com/issues/23600
Signed-off-by: shangfufei <shangfufei@inspur.com>
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

os/bluestore: be more verbose doing bluefs log replay.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

PendingReleaseNotes: 14.2.1 note on crush required version

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27581 from LenzGr/master-documentation

doc: Improved the dashboard proxy config section

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

osd/osd_types: pool_stat_t::dump - fix 'num_store_stats' field

Fixes: http://tracker.ceph.com/issues/39340
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27593 from smithfarm/wip-39291

cmake: revert librados_tp.so version from 3 to 2

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

mgr/telemetry: change crash dict to a list

For ease of indexing.  There's no value in timestamp keys.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #27514 from tchaikov/wip-crimson-auth-client

crimson: enable cephx for v2 msgr

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #27624 into master

* refs/pull/27624/head:
doc: fixed telemetry module reference link

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/auth: let AuthClient::get_auth_request return a named struct

better readability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: use const reference for Connection::auth_registry

it is not optional now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

auth/AuthRegistry: more constness

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: remove "#warning" in v1 implementation

since we've implemented the v2 support, these reminders are not needed
anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: reset promise using `{}`

better readability, less repeating.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonClient: remove unused AuthMethodList

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: use auth_registry for querying auth_methods

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fixed telemetry module reference link

Used :ref: label for telemetry reference in Nautilus.

Fixes: https://tracker.ceph.com/issues/39337
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

rgw: refactor remaining rgw_rest_swift logging

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor remaining rgw_rest_s3 logging

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor logging in Swift ObjStore classes in rgw_rest_swift.cc

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor logging in S3 ObjStore classes in rgw_rest_s3.cc

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge PR #27519 into master

* refs/pull/27519/head:
doc/rados/operations/health-checks: document new bluestore warnings
os/bluestore: alert on fm/bdev size mismatch
os/bluestore: introduce legacy statfs alert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27556 into master

* refs/pull/27556/head:
mgr/rook: Remove support for Rook older than v0.9

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
Reviewed-by: Blaine Gardner <bgardner@suse.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #27563 into master

* refs/pull/27563/head:
mon/OSDMonitor: respect crush node flags for can_mark_*()
osd/OSDMap: add get_crush_node_flags(int osd)
mon/OSDMonitor: make 'osd {add,rm}-{noin,noout,...}' support crush nodes
osd/OSDMap: raise OSD_FLAGS health alert for crush node flags, too
osd/OSDMap: add flags for crush nodes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #27568 into master

* refs/pull/27568/head:
doc/release/nautilus: proscribe minimum hammer tunables and straw2 on upgrade
common/options: update mon_crush_min_required_version=hammer

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #26461 from adamemerson/wip-aio-aio-aio-aio-queue-like-an-egyptian

♪ Ai-o Ai-o Ai-o Ai-o… queue like an Egyptian ♪

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27616 from yuriw/wip-yuriw-crontab-master

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Merge pull request #27381 from markhpc/wip-prioritycache-manager

common/PriorityCache: Implement a Cache Manager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #26089 from taoCH/wip-librgw-create-bucket-bug

rgw: unexpected crash when creating bucket in librgw

crimson: use lowres_system_clock for keepalive timestamps

we should use chrono types for representing time_point and duration,
utime_t could be used for the format on wire though.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use v2 addresses

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: register monc as an AuthClient

so msgrs will be able to turn to monc when they try to authenticate
themselves.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: implement AuthClient

for v2 msgr authentication support, let ceph::mon::Client implement
ceph::auth::AuthClient.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: rename Connection::authenticate()

rename Connection::authenticate() to Connection::authenticate_v1() to
differentiate it from Connection::authenticate_v2(). the latter will be
added for v2 msgr support.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: add setters for Connections

so we don't directly access its member variables

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: refactor Connection::create_auth() so it can be reused

Connection::create_auth() will be shared by v1 and v2 msgrs, so we need
to decouple it from v1 implementation.

Signed-off-by: Kefu Chai <kchai@redhat.com>