Merge pull request #27509 from Aran85/fix-pg-dout

osd/PG: the warning seems more serious than what it wanna transmit

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27592 from tchaikov/wip-ref

message,mgr: drop MessageFactory and friends and use ref_t<> in mgr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #27633 from xiexingguo/wip-39340

osd/osd_types: pool_stat_t::dump - fix 'num_store_stats' field

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27658 from gregsfortytwo/wip-39367-crc-peer

msg: output peer address when detecting bad CRCs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27657 from anthonyeleven/patch-1

doc: operations: improve reweight-by-utilization

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: operations: reweight-by-utilization typo

Add a missing backquote delimiter.

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge PR #27685 into master

* refs/pull/27685/head:
doc: fix boot transition in mds state diagram

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27686 from dzafman/wip-39383

osd: Increase log level of messages which unnecessarily fill up logs

Reviewed-by: Neha Ojha <nojha@redhat.com>

osd: Increase log level of messages which unnecessarily fill up logs

Caused by: 834d3c19a774f1cc93903447d91d182776e12d18

Fixes: https://tracker.ceph.com/issues/39383
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #27037 from alimaredia/wip-rgw-dpp-rest-s3-swift

rgw: log refactoring for rgw_rest_s3/swift ops

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: fix boot transition in mds state diagram

MDSMonitor assigns a booting MDS immediately to standby. From there, the
standby may be promoted to standby-replay or some state leading to active.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27375 into master

* refs/pull/27375/head:
rgw: pass mostly_omap flag when opening/creating pools
rgw/rgw_rados: pass mostly_omap flag when opening/creating pools
rgw: move rgw_init_ioctx() to rgw_tools.cc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27576 from cbodley/wip-39287

rgw: RGWPeriodPusher uses zone system key for inter-zonegroup messages

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #27518 from jtlayton/vstart

vstart: add new option to pass list of block devices to bluestore

Merge pull request #27567 from cbodley/wip-39283

rgw: multisite log trimming only checks peers that sync from us

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #27275 from Devp00l/issue-39037

mgr/dashboard: Localization for date picker module

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #27235 into master

* refs/pull/27235/head:
cephfs-shell: support non-octal mode
cephfs-shell: fix rm command
cephfs-shell: fix lls command
cephfs-shell: fix lcd command
cephfs-shell: fix umask command
cephfs-shell: fix cat command
cephfs-shell: fix chmod command
cephfs-shell: fix chdir command
cephfs-shell: fix mv command
cephfs-shell: fix rmdir command
cephfs-shell: fix ls command
cephfs-shell: fix get command
cephfs-shell: fix put command
cephfs-shell: fix mkdir command
cephfs-shell: helper functions and infra changes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

msg/Message: add SafeMessage which forbids put/get

This is an opt-in wrapper around Message inheritance to prevent
undesired get/put calls on the object. Instead, you must/should use a
smart ptr to adjust the reference count.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/MgrClient: use ref_t<M>

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/{Mgr,MgrStandby}: use ref_t<M>

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/DaemonState: pass const by reference

DaemonPerfCounters::update() does not change the `MMgrReport` message,
neither does it expect a nullptr. so pass by const reference.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/DaemonServer: use ref_t<M>

to avoid explicit use of RefCountedObj interface. and let
intrusive_ptr<> to take care of the life cycle of messages.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/Message: remove Message::const_ref

Signed-off-by: Kefu Chai <kchai@redhat.com>

common: use ref_t<M>

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/Message: drop MessageFactory and friends

we'd better decouple the helpers from the definition of Message types,
to keep the definition Messages simple and concise.

Signed-off-by: Kefu Chai <kchai@redhat.com>

messages/MOSDPeeringOp: do not include PGPeeringEvent.h

forward delcaration would suffice

Signed-off-by: Kefu Chai <kchai@redhat.com>

messages: do not use MessageSubType anymore

so we can drop it in future commit

Signed-off-by: Kefu Chai <kchai@redhat.com>

messages: #include necessary header

for instance, MLogRec is defined in PGPeeringEvent.h, we should not rely
other header(s) to do this for us. so just #include PGPeeringEvent.h for
its definition.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mds/MDSCache: init empty string_view with {}

nullptr is not a valid null-terminated character sequence

Signed-off-by: Kefu Chai <kchai@redhat.com>

src/: use make_message<Foo> instead of Foo::create()

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/Message: remove msgref_cast()

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/Message: add aliases for static_pointer_cast

strictly speaking, they are not "alias" of boost::static_pointer_cast,
but to avoid an extra conversion, instead of using
boost::static_pointer_cast<>, i am using std::static_cast<> directly for
creating boost::intrusive_ptr<> instance.

Signed-off-by: Kefu Chai <kchai@redhat.com>

client,mds: use ref_cast<Foo> instead of Foo::msgref_cast

Signed-off-by: Kefu Chai <kchai@redhat.com>

src/: use ref_t<Message> instead of Message::ref

Signed-off-by: Kefu Chai <kchai@redhat.com>

src/: use cref_t<Message> instead of Message::const_ref

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/RefCountedObj: move private members

To follow code-style guidelines.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

common/RefCountedObj: use type of nref

"int" may narrow the ref count (although very unlikely!).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

msg: add aliases of ref_t and cref_t

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27649 into master

* refs/pull/27649/head:
PendingReleaseNotes: 14.2.1 note on crush required version

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

msg: output peer address when detecting bad CRCs

Fixes: http://tracker.ceph.com/issues/39367
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #27600 from neha-ojha/wip-39006

osd/OSD.cc: make osd bench description consistent with parameters

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27481 into master

* refs/pull/27481/head:
qa: add a method to test if a file can be created
qa: add wrappers for vstart's run() with check_status as False
qa: add a method to create an empty file with any user
qa: add stdin parameter to run_shell
qa: command to be ran as a different user should be a single string
qa: looks for quotes in Python shell and get rid of them
qa: move checks on command arguments to a different method
qa: accept args as strings
qa: add a method to run given command as root user
qa: add a method to run a command as a different user
qa: allow keeping sudo in command arguments
qa: make execution abortion optional for command ran through run_shell()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

doc: operations: improve reweight-by-utilization

Add the missing `max_change`, `max_osds`, and `--no-increasing` parameters to `reweight-by-utilization` and `test-reweight-by-utilization`.  Minor adjustments to wording.

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #27282 from mogeb/wip-lvmcache

ceph-volume: add utility functions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

qa: add a method to test if a file can be created

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add wrappers for vstart's run() with check_status as False

Too often in tests it is required to run a command that we want to (or
will probably) fail so that we can check if it failed for expected
reason(s) (eg. a testcase where command must fail with "Permission
denied"). Therefore, add few wrapper methods for vstart_runner.py's
run() with check_status's default value as False so that we don't need
to pass check_status's value at every call to run() or it's other
wrappers.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add a method to create an empty file with any user

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add stdin parameter to run_shell

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: command to be ran as a different user should be a single string

Signed-off-by: Rishabh Dave <ridave@redhat.com>

PendingReleaseNotes: 14.2.1 note on crush required version

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27581 from LenzGr/master-documentation

doc: Improved the dashboard proxy config section

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

osd/osd_types: pool_stat_t::dump - fix 'num_store_stats' field

Fixes: http://tracker.ceph.com/issues/39340
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27593 from smithfarm/wip-39291

cmake: revert librados_tp.so version from 3 to 2

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #27514 from tchaikov/wip-crimson-auth-client

crimson: enable cephx for v2 msgr

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #27624 into master

* refs/pull/27624/head:
doc: fixed telemetry module reference link

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/auth: let AuthClient::get_auth_request return a named struct

better readability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: use const reference for Connection::auth_registry

it is not optional now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

auth/AuthRegistry: more constness

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: remove "#warning" in v1 implementation

since we've implemented the v2 support, these reminders are not needed
anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: reset promise using `{}`

better readability, less repeating.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonClient: remove unused AuthMethodList

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: use auth_registry for querying auth_methods

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fixed telemetry module reference link

Used :ref: label for telemetry reference in Nautilus.

Fixes: https://tracker.ceph.com/issues/39337
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

rgw: refactor remaining rgw_rest_swift logging

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor remaining rgw_rest_s3 logging

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor logging in Swift ObjStore classes in rgw_rest_swift.cc

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor logging in S3 ObjStore classes in rgw_rest_s3.cc

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge PR #27519 into master

* refs/pull/27519/head:
doc/rados/operations/health-checks: document new bluestore warnings
os/bluestore: alert on fm/bdev size mismatch
os/bluestore: introduce legacy statfs alert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27556 into master

* refs/pull/27556/head:
mgr/rook: Remove support for Rook older than v0.9

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
Reviewed-by: Blaine Gardner <bgardner@suse.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #27563 into master

* refs/pull/27563/head:
mon/OSDMonitor: respect crush node flags for can_mark_*()
osd/OSDMap: add get_crush_node_flags(int osd)
mon/OSDMonitor: make 'osd {add,rm}-{noin,noout,...}' support crush nodes
osd/OSDMap: raise OSD_FLAGS health alert for crush node flags, too
osd/OSDMap: add flags for crush nodes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #27568 into master

* refs/pull/27568/head:
doc/release/nautilus: proscribe minimum hammer tunables and straw2 on upgrade
common/options: update mon_crush_min_required_version=hammer

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #26461 from adamemerson/wip-aio-aio-aio-aio-queue-like-an-egyptian

♪ Ai-o Ai-o Ai-o Ai-o… queue like an Egyptian ♪

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27616 from yuriw/wip-yuriw-crontab-master

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Merge pull request #27381 from markhpc/wip-prioritycache-manager

common/PriorityCache: Implement a Cache Manager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #26089 from taoCH/wip-librgw-create-bucket-bug

rgw: unexpected crash when creating bucket in librgw

crimson: use lowres_system_clock for keepalive timestamps

we should use chrono types for representing time_point and duration,
utime_t could be used for the format on wire though.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use v2 addresses

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: register monc as an AuthClient

so msgrs will be able to turn to monc when they try to authenticate
themselves.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: implement AuthClient

for v2 msgr authentication support, let ceph::mon::Client implement
ceph::auth::AuthClient.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: rename Connection::authenticate()

rename Connection::authenticate() to Connection::authenticate_v1() to
differentiate it from Connection::authenticate_v2(). the latter will be
added for v2 msgr support.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: add setters for Connections

so we don't directly access its member variables

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: refactor Connection::create_auth() so it can be reused

Connection::create_auth() will be shared by v1 and v2 msgrs, so we need
to decouple it from v1 implementation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/auth: refactor AuthClient interface

* we'd better return value by "return" not by passing output prameters by
  pointers.
* remove unused parameters from AuthClient::handle_auth_done(),
  `session_key` and `connection_secret` are returned by setting
  corresponding member variables of `auth_meta`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/auth: rename auth::auth_error to auth::error

* s/auth::auth_error/auth::error/: less repeating this way
* auth/Errors.{h,cc}: remove them. as we don't need to use error code to
  construct exceptions in this context
* move auth::error into auth::AuthClient

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27564 into master

* refs/pull/27564/head:
ceph-objectstore-tool: Rename dump-import to dump-export
osd: Fix fast path to use version bumped value

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27546 into master

* refs/pull/27546/head:
mgr/progress: fix typo
mgr/progress: behave if pgs disappear (due to a racing pg merge)

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #27036 from votdev/issue_24453

mgr/dashboard: Manager should complain about wrong dashboard certificate

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #27433 from rhcs-dashboard/wip-36243-master

mgr/dashboard: Add frontend code documentation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

cmake: revert librados_tp.so version from 3 to 2

Post-mortem analysis:

librados.so and librados_tp.so are packaged together in the librados2
RPM.

c680fb10f598267f15a37fdb7bc44529c37c9318 bumped
the librados.so and librados_tp.so versions from 2 to 3.

Later, 7bf6b5ee1208a359826c74ab033e6bbbfc65969f reverted the librados.so
version from 3 to 2, but left the librados_tp.so version at 3.

Fixes: http://tracker.ceph.com/issues/39291
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: Improved the dashboard proxy config section

Added paragraph that explains how standby dashboard instances perform
HTTP redirections to the currently active ceph-mgr node.
Added note about current limitations (see Bug#24662 for details).

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

doc: dashboard: refined object gateway configuration

As discussed in BUG#39252, the dashboard currently does not
support all possible RGW configuration possibilities when trying
to obtain hostname and port number. Update the docs to better
reflect this.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #27530 from yuvalif/fix_pubsub_duplicates

rgw/pubsub: handle subscription conf errors better

osd/PG: the warning seems more serious than what it wanna transmit

The missing items is picked from primary and peers, so it is normal if
some guys missing the same version, assertion on the next line will check
wrong circumstances.

Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>

osd/OSD.cc: make osd bench description consistent with parameters

Fixes: http://tracker.ceph.com/issues/39006
Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27552 from tchaikov/wip-corpus.rst

doc/corpus: update to adapt the change from autotools to cmake

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Changcheng Liu <changcheng.liu@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/corpus: update to adapt the change from autotools to cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/operations/health-checks: document new bluestore warnings

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27493 from yuvalif/pubsub_zone_conf_doc

rgw/pubsub: clarify pubsub zone configuration

Merge PR #27585 into master

* refs/pull/27585/head:
messages: remove MNop

Reviewed-by: Sage Weil <sage@redhat.com>