rbd: fixed memory leak in 'rbd mirror pool' multi-image actions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: async open/close should free ImageCtx before issuing callback

The destructor for ImageCtx attempts to access librados::IoCtx objects
that might have been destroyed immediately after the async open (failure)
and close actions completed.

Fixes: http://tracker.ceph.com/issues/39031
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27567 from cbodley/wip-39283

rgw: multisite log trimming only checks peers that sync from us

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #27275 from Devp00l/issue-39037

mgr/dashboard: Localization for date picker module

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #27235 into master

* refs/pull/27235/head:
cephfs-shell: support non-octal mode
cephfs-shell: fix rm command
cephfs-shell: fix lls command
cephfs-shell: fix lcd command
cephfs-shell: fix umask command
cephfs-shell: fix cat command
cephfs-shell: fix chmod command
cephfs-shell: fix chdir command
cephfs-shell: fix mv command
cephfs-shell: fix rmdir command
cephfs-shell: fix ls command
cephfs-shell: fix get command
cephfs-shell: fix put command
cephfs-shell: fix mkdir command
cephfs-shell: helper functions and infra changes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #27649 into master

* refs/pull/27649/head:
PendingReleaseNotes: 14.2.1 note on crush required version

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #27600 from neha-ojha/wip-39006

osd/OSD.cc: make osd bench description consistent with parameters

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27481 into master

* refs/pull/27481/head:
qa: add a method to test if a file can be created
qa: add wrappers for vstart's run() with check_status as False
qa: add a method to create an empty file with any user
qa: add stdin parameter to run_shell
qa: command to be ran as a different user should be a single string
qa: looks for quotes in Python shell and get rid of them
qa: move checks on command arguments to a different method
qa: accept args as strings
qa: add a method to run given command as root user
qa: add a method to run a command as a different user
qa: allow keeping sudo in command arguments
qa: make execution abortion optional for command ran through run_shell()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #27282 from mogeb/wip-lvmcache

ceph-volume: add utility functions

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

qa: add a method to test if a file can be created

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add wrappers for vstart's run() with check_status as False

Too often in tests it is required to run a command that we want to (or
will probably) fail so that we can check if it failed for expected
reason(s) (eg. a testcase where command must fail with "Permission
denied"). Therefore, add few wrapper methods for vstart_runner.py's
run() with check_status's default value as False so that we don't need
to pass check_status's value at every call to run() or it's other
wrappers.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add a method to create an empty file with any user

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: add stdin parameter to run_shell

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa: command to be ran as a different user should be a single string

Signed-off-by: Rishabh Dave <ridave@redhat.com>

PendingReleaseNotes: 14.2.1 note on crush required version

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27581 from LenzGr/master-documentation

doc: Improved the dashboard proxy config section

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27593 from smithfarm/wip-39291

cmake: revert librados_tp.so version from 3 to 2

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #27514 from tchaikov/wip-crimson-auth-client

crimson: enable cephx for v2 msgr

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #27624 into master

* refs/pull/27624/head:
doc: fixed telemetry module reference link

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

crimson/auth: let AuthClient::get_auth_request return a named struct

better readability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: use const reference for Connection::auth_registry

it is not optional now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

auth/AuthRegistry: more constness

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: remove "#warning" in v1 implementation

since we've implemented the v2 support, these reminders are not needed
anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: reset promise using `{}`

better readability, less repeating.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonClient: remove unused AuthMethodList

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: use auth_registry for querying auth_methods

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fixed telemetry module reference link

Used :ref: label for telemetry reference in Nautilus.

Fixes: https://tracker.ceph.com/issues/39337
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

Merge PR #27519 into master

* refs/pull/27519/head:
doc/rados/operations/health-checks: document new bluestore warnings
os/bluestore: alert on fm/bdev size mismatch
os/bluestore: introduce legacy statfs alert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27556 into master

* refs/pull/27556/head:
mgr/rook: Remove support for Rook older than v0.9

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
Reviewed-by: Blaine Gardner <bgardner@suse.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #27563 into master

* refs/pull/27563/head:
mon/OSDMonitor: respect crush node flags for can_mark_*()
osd/OSDMap: add get_crush_node_flags(int osd)
mon/OSDMonitor: make 'osd {add,rm}-{noin,noout,...}' support crush nodes
osd/OSDMap: raise OSD_FLAGS health alert for crush node flags, too
osd/OSDMap: add flags for crush nodes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #27568 into master

* refs/pull/27568/head:
doc/release/nautilus: proscribe minimum hammer tunables and straw2 on upgrade
common/options: update mon_crush_min_required_version=hammer

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #26461 from adamemerson/wip-aio-aio-aio-aio-queue-like-an-egyptian

♪ Ai-o Ai-o Ai-o Ai-o… queue like an Egyptian ♪

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27616 from yuriw/wip-yuriw-crontab-master

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Merge pull request #27381 from markhpc/wip-prioritycache-manager

common/PriorityCache: Implement a Cache Manager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #26089 from taoCH/wip-librgw-create-bucket-bug

rgw: unexpected crash when creating bucket in librgw

crimson: use lowres_system_clock for keepalive timestamps

we should use chrono types for representing time_point and duration,
utime_t could be used for the format on wire though.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use v2 addresses

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: register monc as an AuthClient

so msgrs will be able to turn to monc when they try to authenticate
themselves.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: implement AuthClient

for v2 msgr authentication support, let ceph::mon::Client implement
ceph::auth::AuthClient.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: rename Connection::authenticate()

rename Connection::authenticate() to Connection::authenticate_v1() to
differentiate it from Connection::authenticate_v2(). the latter will be
added for v2 msgr support.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/net: add setters for Connections

so we don't directly access its member variables

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: refactor Connection::create_auth() so it can be reused

Connection::create_auth() will be shared by v1 and v2 msgrs, so we need
to decouple it from v1 implementation.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/auth: refactor AuthClient interface

* we'd better return value by "return" not by passing output prameters by
  pointers.
* remove unused parameters from AuthClient::handle_auth_done(),
  `session_key` and `connection_secret` are returned by setting
  corresponding member variables of `auth_meta`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/auth: rename auth::auth_error to auth::error

* s/auth::auth_error/auth::error/: less repeating this way
* auth/Errors.{h,cc}: remove them. as we don't need to use error code to
  construct exceptions in this context
* move auth::error into auth::AuthClient

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27564 into master

* refs/pull/27564/head:
ceph-objectstore-tool: Rename dump-import to dump-export
osd: Fix fast path to use version bumped value

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27546 into master

* refs/pull/27546/head:
mgr/progress: fix typo
mgr/progress: behave if pgs disappear (due to a racing pg merge)

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #27036 from votdev/issue_24453

mgr/dashboard: Manager should complain about wrong dashboard certificate

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #27433 from rhcs-dashboard/wip-36243-master

mgr/dashboard: Add frontend code documentation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

cmake: revert librados_tp.so version from 3 to 2

Post-mortem analysis:

librados.so and librados_tp.so are packaged together in the librados2
RPM.

c680fb10f598267f15a37fdb7bc44529c37c9318 bumped
the librados.so and librados_tp.so versions from 2 to 3.

Later, 7bf6b5ee1208a359826c74ab033e6bbbfc65969f reverted the librados.so
version from 3 to 2, but left the librados_tp.so version at 3.

Fixes: http://tracker.ceph.com/issues/39291
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: Improved the dashboard proxy config section

Added paragraph that explains how standby dashboard instances perform
HTTP redirections to the currently active ceph-mgr node.
Added note about current limitations (see Bug#24662 for details).

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

doc: dashboard: refined object gateway configuration

As discussed in BUG#39252, the dashboard currently does not
support all possible RGW configuration possibilities when trying
to obtain hostname and port number. Update the docs to better
reflect this.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #27530 from yuvalif/fix_pubsub_duplicates

rgw/pubsub: handle subscription conf errors better

osd/OSD.cc: make osd bench description consistent with parameters

Fixes: http://tracker.ceph.com/issues/39006
Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27552 from tchaikov/wip-corpus.rst

doc/corpus: update to adapt the change from autotools to cmake

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Changcheng Liu <changcheng.liu@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/corpus: update to adapt the change from autotools to cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/operations/health-checks: document new bluestore warnings

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27493 from yuvalif/pubsub_zone_conf_doc

rgw/pubsub: clarify pubsub zone configuration

Merge PR #27585 into master

* refs/pull/27585/head:
messages: remove MNop

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27578 from cbodley/wip-rgw-pubsub-warning

rgw: silence a -Wunused-function warning in pubsub

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #27178 from pritha-srivastava/wip-rgw-iam

rgw: Adding 'iam' namespace for Role and User Policy related REST APIs.

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

os/bluestore: alert on fm/bdev size mismatch

Relates to: https://tracker.ceph.com/issues/39151

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: introduce legacy statfs alert

Fixes: https://tracker.ceph.com/issues/39146
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #27553 from theanalyst/doc/releases/12.2.12

doc: releases/luminous: release notes for 12.2.12

Reviewed-By: Nathan Cutler <ncutler@suse.com>

doc: releases/12.2.12: fix changelog to add issues without trackers

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #27491 from sebastian-philipp/rook-service-ls-rgw

mgr/rook: Added missing `rgw` daemons in `service ls`

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #27462 from sebastian-philipp/rook-fix-rgw

mgr/rook: Fix RGW creation

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #27339 from david-z/wip-fix-rbd-map-hang

krbd: fix rbd map hang due to udev return subsystem unordered

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

messages: remove MNop

MNop was introduced along with xio messenger. since we've dropped
xiomessenger in #27021, there is no need to keep MNop around anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27517 into master

* refs/pull/27517/head:
mon: make mon_osd_down_out_subtree_limit update at runtime

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #27458 into master

* refs/pull/27458/head:
bluestore: fix for FreeBSD iocb structure

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27485 into master

* refs/pull/27485/head:
mgr/ActivePyModules: handle_command - fix broken lock

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27506 into master

* refs/pull/27506/head:
src/tests/crushtool: Add test for an empty default
crush/CrushCompiler: Fix __replacement_assert

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27271 from idryomov/wip-krbd-wac-on-dm

qa: add device mapper and lvm test cases for stable pages

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27533 from dillaman/wip-39257

rbd-mirror: periodically flush IO and commit positions

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #27229 from dillaman/wip-librbd-writearound

librbd: introduce new default write-around cache policy

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #27446 from trociny/wip-rbd-io-simple-scheduler-improvements

librbd: dispatch delayed requests only if read intersects

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27513 from ddiss/librados_silence_unused_vars

librbd: silence -Wunused-variable warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: silence a -Wunused-function warning in pubsub

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cephfs-shell: support non-octal mode

Fixes: http://tracker.ceph.com/issues/38740
Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix rm command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix lls command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix lcd command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix umask command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix cat command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix chmod command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix chdir command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix mv command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix rmdir command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix ls command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix get command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix put command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: fix mkdir command

Signed-off-by: Milind Changire <mchangir@redhat.com>

cephfs-shell: helper functions and infra changes

Signed-off-by: Milind Changire <mchangir@redhat.com>

doc/release/nautilus: proscribe minimum hammer tunables and straw2 on upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: update mon_crush_min_required_version=hammer

Hammer is a good point to set the minimum at:

 - it includes straw2 and vary_r, which are the worst offenders among the
   old tunables
 - it does not include chooseleaf_stable, which improves placement
   considerably but *also* shuffles basically all data in the cluster.

Getting everyone up to straw2 support means we can also proscribe the
'ceph osd crush set-all-straw-buckets-to-straw2' command.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: multisite log trimming only checks peers that sync from us

if there are peer zones in our zonegroup that do not sync from us, they
won't have any sync status that corresponds to our logs. fetching their
status will give us empty markers, so we'll never make progress with log
trimming

the get_zone_data_notify_to_map() contains only the zones that are
configured to sync from us, so use that instead of get_zone_conn_map()

Fixes: http://tracker.ceph.com/issues/39283
Signed-off-by: Casey Bodley <cbodley@redhat.com>

ceph-objectstore-tool: Rename dump-import to dump-export

If user specifies dump-import it will still work, but isn't
in the usage that way.

Fixes: http://tracker.ceph.com/issues/39284
Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix fast path to use version bumped value

Fix the code so that the version is in one place
for both the slow and fast path.

Caused by: d2ca3d2feb442f97ca89023c7d01178d96f517a6

Fixes: http://tracker.ceph.com/issues/39281
Signed-off-by: David Zafman <dzafman@redhat.com>

rgw: use std::optional in bilog trim

clean up an ugly workaround for a bug in boost::optional

Signed-off-by: Casey Bodley <cbodley@redhat.com>