rgw: refactor remaining rgw_rest_swift logging

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor remaining rgw_rest_s3 logging

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor logging in Swift ObjStore classes in rgw_rest_swift.cc

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: refactor logging in S3 ObjStore classes in rgw_rest_s3.cc

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge PR #27519 into master

* refs/pull/27519/head:
doc/rados/operations/health-checks: document new bluestore warnings
os/bluestore: alert on fm/bdev size mismatch
os/bluestore: introduce legacy statfs alert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27556 into master

* refs/pull/27556/head:
mgr/rook: Remove support for Rook older than v0.9

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
Reviewed-by: Blaine Gardner <bgardner@suse.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #27563 into master

* refs/pull/27563/head:
mon/OSDMonitor: respect crush node flags for can_mark_*()
osd/OSDMap: add get_crush_node_flags(int osd)
mon/OSDMonitor: make 'osd {add,rm}-{noin,noout,...}' support crush nodes
osd/OSDMap: raise OSD_FLAGS health alert for crush node flags, too
osd/OSDMap: add flags for crush nodes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #27568 into master

* refs/pull/27568/head:
doc/release/nautilus: proscribe minimum hammer tunables and straw2 on upgrade
common/options: update mon_crush_min_required_version=hammer

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #26461 from adamemerson/wip-aio-aio-aio-aio-queue-like-an-egyptian

♪ Ai-o Ai-o Ai-o Ai-o… queue like an Egyptian ♪

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27616 from yuriw/wip-yuriw-crontab-master

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Merge pull request #27381 from markhpc/wip-prioritycache-manager

common/PriorityCache: Implement a Cache Manager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/tests: moved some runs from ovh, removed ceph-disk/nautilus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #26089 from taoCH/wip-librgw-create-bucket-bug

rgw: unexpected crash when creating bucket in librgw

Merge PR #27564 into master

* refs/pull/27564/head:
ceph-objectstore-tool: Rename dump-import to dump-export
osd: Fix fast path to use version bumped value

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27546 into master

* refs/pull/27546/head:
mgr/progress: fix typo
mgr/progress: behave if pgs disappear (due to a racing pg merge)

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #27036 from votdev/issue_24453

mgr/dashboard: Manager should complain about wrong dashboard certificate

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #27433 from rhcs-dashboard/wip-36243-master

mgr/dashboard: Add frontend code documentation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #27530 from yuvalif/fix_pubsub_duplicates

rgw/pubsub: handle subscription conf errors better

Merge pull request #27552 from tchaikov/wip-corpus.rst

doc/corpus: update to adapt the change from autotools to cmake

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Changcheng Liu <changcheng.liu@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/corpus: update to adapt the change from autotools to cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/operations/health-checks: document new bluestore warnings

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27493 from yuvalif/pubsub_zone_conf_doc

rgw/pubsub: clarify pubsub zone configuration

Merge PR #27585 into master

* refs/pull/27585/head:
messages: remove MNop

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27578 from cbodley/wip-rgw-pubsub-warning

rgw: silence a -Wunused-function warning in pubsub

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #27178 from pritha-srivastava/wip-rgw-iam

rgw: Adding 'iam' namespace for Role and User Policy related REST APIs.

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

os/bluestore: alert on fm/bdev size mismatch

Relates to: https://tracker.ceph.com/issues/39151

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: introduce legacy statfs alert

Fixes: https://tracker.ceph.com/issues/39146
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #27553 from theanalyst/doc/releases/12.2.12

doc: releases/luminous: release notes for 12.2.12

Reviewed-By: Nathan Cutler <ncutler@suse.com>

doc: releases/12.2.12: fix changelog to add issues without trackers

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #27491 from sebastian-philipp/rook-service-ls-rgw

mgr/rook: Added missing `rgw` daemons in `service ls`

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #27462 from sebastian-philipp/rook-fix-rgw

mgr/rook: Fix RGW creation

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #27339 from david-z/wip-fix-rbd-map-hang

krbd: fix rbd map hang due to udev return subsystem unordered

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

messages: remove MNop

MNop was introduced along with xio messenger. since we've dropped
xiomessenger in #27021, there is no need to keep MNop around anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #27517 into master

* refs/pull/27517/head:
mon: make mon_osd_down_out_subtree_limit update at runtime

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #27458 into master

* refs/pull/27458/head:
bluestore: fix for FreeBSD iocb structure

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27485 into master

* refs/pull/27485/head:
mgr/ActivePyModules: handle_command - fix broken lock

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #27506 into master

* refs/pull/27506/head:
src/tests/crushtool: Add test for an empty default
crush/CrushCompiler: Fix __replacement_assert

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27271 from idryomov/wip-krbd-wac-on-dm

qa: add device mapper and lvm test cases for stable pages

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27533 from dillaman/wip-39257

rbd-mirror: periodically flush IO and commit positions

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #27229 from dillaman/wip-librbd-writearound

librbd: introduce new default write-around cache policy

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #27446 from trociny/wip-rbd-io-simple-scheduler-improvements

librbd: dispatch delayed requests only if read intersects

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27513 from ddiss/librados_silence_unused_vars

librbd: silence -Wunused-variable warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: silence a -Wunused-function warning in pubsub

Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/release/nautilus: proscribe minimum hammer tunables and straw2 on upgrade

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: update mon_crush_min_required_version=hammer

Hammer is a good point to set the minimum at:

 - it includes straw2 and vary_r, which are the worst offenders among the
   old tunables
 - it does not include chooseleaf_stable, which improves placement
   considerably but *also* shuffles basically all data in the cluster.

Getting everyone up to straw2 support means we can also proscribe the
'ceph osd crush set-all-straw-buckets-to-straw2' command.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: Rename dump-import to dump-export

If user specifies dump-import it will still work, but isn't
in the usage that way.

Fixes: http://tracker.ceph.com/issues/39284
Signed-off-by: David Zafman <dzafman@redhat.com>

osd: Fix fast path to use version bumped value

Fix the code so that the version is in one place
for both the slow and fast path.

Caused by: d2ca3d2feb442f97ca89023c7d01178d96f517a6

Fixes: http://tracker.ceph.com/issues/39281
Signed-off-by: David Zafman <dzafman@redhat.com>

mgr/dashboard: Add frontend code documentation

Adds compodoc as auto-documentation tool for Angular frontend code.

Fixes: http://tracker.ceph.com/issues/36243
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mon/OSDMonitor: respect crush node flags for can_mark_*()

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: add get_crush_node_flags(int osd)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27470 from cbodley/wip-39160

rgw: don't recalculate etags for slo/dlo

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

mon/OSDMonitor: make 'osd {add,rm}-{noin,noout,...}' support crush nodes

Set noin, noout, etc. flags on crush nodes.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27528 from cbodley/wip-qa-rgw-no-debuginfo

qa/rgw: disable debuginfo packages

osd/OSDMap: raise OSD_FLAGS health alert for crush node flags, too

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: Remove support for Rook older than v0.9

http://tracker.ceph.com/issues/39278

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

osd/OSDMap: add flags for crush nodes

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: write-around cache should honor FUA op flag

If force unit access is specified, the cache should be bypassed just
like other non-optimized IO through the cache.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: support new bench 'full-seq' io pattern option

This ensures IOs are issued in sequential order regardless of the current
IO thread setting.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: releases/luminous: release notes for 12.2.12

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

mgr/progress: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/progress: behave if pgs disappear (due to a racing pg merge)

If one of the PGs we care about disappears, consider it complete.

Fixes: http://tracker.ceph.com/issues/38157
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #27473 into master

* refs/pull/27473/head:
mgr/DaemonServer: prevent pgp_num reductions from outpacing pg_num merges

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #27477 into master

* refs/pull/27477/head:
os/bluestore: check return value of func _open_db_and_around.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #27499 into master

* refs/pull/27499/head:
qa/standalone/osd/osd-markdown: fix dup command disabling

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #27516 into master

* refs/pull/27516/head:
mgr/rook: Fix Python 2 regression

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Gabriel Brascher <gabriel@apache.org>

Merge pull request #26816 from iotcg/doc

doc: correct example to use vstart to run up cluster

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27535 from yuriw/wip-yuriw-wip-yuriw-crontab-master

qa/tests: added rados on master, reduced fs, rbd, multimds

Merge pull request #27428 from cyx1231st/wip-seastar-msgr-write

crimson/net: extract do_write_dispatch_sweep()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

qa/tests: added rados on master, reduced fs, rbd, multimds

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #27515 from tchaikov/wip-seastar-fix-monc

test/crimson: fix build of unittest_seastar_monc

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

src/tests/crushtool: Add test for an empty default

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #27524 from tchaikov/wip-39164

install-deps.sh: install '*rpm-macros'

Reviewed-by: Boris Ranto <branto@redhat.com>

rbd-mirror: periodically flush prior to updating status

This ensure the most up-to-date commit position is included in the
status report.

Fixes: http://tracker.ceph.com/issues/39257
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: simplify image replayer IO / commit position flush path

There is no current user of the callback context provided to the
flush method and there are no longer any reason to have the flush
support methods be virtual.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27443 from Polynomial-C/tinfo

cmake: Fix build against ncurses with separate libtinfo

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #27527 into master

* refs/pull/27527/head:
qa/tests: run luminous-x and mimic-x 2 times a week but with high priority

Reviewed-by: Sage Weil <sage@redhat.com>

rgw/pubsub: handle subscription conf errors better

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>

librbd: enable the simple IO scheduler by default

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: switch to write-around cache policy by default

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: new alternative write-around cache layer

This is a replacement for the ObjectCacher-based writethrough/
writeback cache. It permits a configurable maximum byte size
of in-flight writes (bytes) to the OSDs where the writes are
immediately completed back to the caller. Any IO errors are
propagated on the next flush request.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: pass journal tid through flush object dispatch spec

This permits moving the journal flush and commit logic down to the
journal object dispatch layer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: remove special case for journal replay via writeback

With the future addition of a writearound cache (and to a much
lesser extent the IO scheduler), the code will become easier
to maintain via a single IO path that periodically injects IO
flushes to ensure the data is consistently committed to disk.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: moved object cacher writeback handler to cache sub-directory

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/buffer: hexdump should skip lines that are the same as previous

Previously, it would only skip similar lines if the data was all zeroed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/tests: run luminous-x and mimic-x 2 times a week but with high priority
hope is it will help with locking 5 nodes for tests

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge PR #27345 into master

* refs/pull/27345/head:
cephfs-shell: Change type of d_name to bytes array
cephfs-shell: Revert "cephfs.pyx: add py3 compatibility"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

install-deps.sh: install '*rpm-macros'

so `yum-builddep` can have access to the latest macros for preparing the
build dependencies

Fixes: http://tracker.ceph.com/issues/39164
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27504 from tchaikov/wip-qa-use-python36-for-py3-tests

qa/tasks/ceph_deploy: install python3.6 instead of python3.4 for py3 tests

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

mon: make mon_osd_down_out_subtree_limit update at runtime

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: Fix Python 2 regression

Fixes: http://tracker.ceph.com/issues/39250
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge PR #27323 into master

* refs/pull/27323/head:
msg/async: avoid creating unnecessary AsyncConnectionRef

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27444 from ifed01/wip-ifed-fix-21312

os/bluestore: call fault_range prior to looking for blob to reuse

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27445 from ifed01/wip-ifed-fix-39143

os/ceph-bluestore-tool: bluefs-bdev-expand asserts if no WAL

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

crimson: add move ctor for CephContext and mon::Client

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/crimson: add dummy_auth_handler to test_monc

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>

Merge pull request #27438 from Devp00l/wip-issue-37701

mgr/dashboard: Removes distracting search behavior

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

librbd: silence -Wunused-variable warnings

Signed-off-by: David Disseldorp <ddiss@suse.de>

rgw/pubsub: more info on notification deletion compatibility

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #27348 from Devp00l/wip-issue-39086

mgr/dashboard: Enable read only users to read again

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

cmake: Fix build against ncurses with separate libtinfo

Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>