Merge pull request #39361 from sebastian-philipp/orch-fix-flake8

mgr/orch: Fix flake8 error

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39332 from cyx1231st/wip-seastore-range-query

crimson/onode-staged-tree: implement Cursor::get_next() and comparators for range query

Reviewed-by: Samuel Just <sjust@redhat.com>

mgr/orch: Fix flake8 error

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #37729 from mgfritch/cephadm-podman-logdriver

cephadm: configure journald as the logdriver

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #37764 from mgfritch/cephadm-no-container-init

cephadm: run containers using `--init` by default

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #39153 from mgfritch/cephadm-flake8

mgr/cephadm: introduce flake8

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39331 from ceph/revert-37954-wip-F46226-log-dq-throttle-limit-new

Revert "msg,mon,common: log when DispatchQueue throttle limit is reached"

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39359 from tchaikov/wip-crimson-clang-build

crimson/os: fix build with clang

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

crimson/os: capture labels using a new variable

structured-binding does not define variables, so we cannot use "[=]" to
implicitly capture them. define variables in the capture list instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: mark unused variable [[maybe_unused]]

to silence warning like

key_layout.h:516:10: warning: unused variable 'nspace_masked'
[-Wunused-variable]
    auto nspace_masked = string_view_masked_t::decode(nspace, delta);
         ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: do not define assignment operator for Value

we cannot assign a reference.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: do not capture unused variable

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: mark ExitBarrier final

to silence warnings like:

crimson/common/operation.h:394:20: warning: class with destructor marked
'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
    ~ExitBarrier() final {
                   ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39350 from tchaikov/wip-cephadm-unicode

cephadm: do not cast subnet to unicode

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39266 from sebastian-philipp/cephadm-rm-keyring-on-failed-deploy

mgr/cephadm: cleanup daemon keyring on failed daemon deploy

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #38695 from pcuzner/orch-status-changes

mgr/orchestrator: minor improvements to orch status

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39347 from xxhdx1985126/wip-crimson-pg-bug-fix

crimson/osd: make sure ox be removed after all continuations are finished in PG::do_osd_ops

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Revert "msg,mon,common: log when DispatchQueue throttle limit is reached"

Signed-off-by: Kefu Chai <kchai@redhat.com>

cephadm: do not cast subnet to unicode

this change addresses a regression introduced by
fe4f4402fbcd87667613640f2808d5d0e07e749d, which was tested before
abd9287db0e4f4f7873864119f5ce62519af1d48 got merged. and the former was
merged after abd9287db0e4f4f7873864119f5ce62519af1d48. hence the
regression.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: make sure ox be removed after all continuations are finished in PG::do_osd_ops

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #39193 from tchaikov/wip-cephadm-py3

cephadm: drop support to python<3.3

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38897 from Daniel-Pivonka/mgroktostop

mgr/cephadm: ok-to-stop for mgr

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39082 from ifed01/wip-ifed-meta-in-objectstore-tool

tools/ceph-objectstore-tool: allow object operations on meta pgid

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39273 from tchaikov/wip-timer-mono

common/Timer: use mono_clock for clock_t

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39295 from tchaikov/wip-mgr-balancer-ann

pybind/mgr/balancer: add type annotations

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #39164 from tchaikov/wip-crimson-push-mov

crimson/osd: consume the push payload

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #39204 from TRYTOBE8TME/wip-rgw-create-bucket-dpp

src/rgw: Dpp addition to create bucket log

rgw: Dpp addition to create bucket log

This commit adds dpp to the create bucket log.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>

pybind/mgr/balancer: flake8 cleanups

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/balancer: define Plan.{dump,show}()

as they are called by the commands

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/alerts: flake8 cleanups

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/balancer: define options using Option

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/balancer: add type annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/balancer: define commands using CLICommand

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39336 from AlexWangD/patch-4

doc: fix the spelling mistake of “Topicd”

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix the spelling mistake of “Topicd”

“Topicd” is a spelling mistake of "Topics".

Merge pull request #37566 from AmnonHanuhov/wip-ObjectStore_EIO_Handling

crimson/osd: Recover missing object on read error

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/onode-staged-tree: cleanup -- switch to get_*_slot() interfaces

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #39303 from dillaman/wip-49173

librbd/io: conditionally disable move optimization

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #39312 from ivancich/wip-orphan-list-binary-data

rgw: allow rgw-orphan-list to handle intermediate files w/ binary data

Reviewed-by: Michael Kidd <linuxkidd@gmail.com>

mgr/cephadm: Redefinition of unused name from line n (F811)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Module imported but unused (F401)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Local variable name is assigned to but never used (F841)

add error handling when `CompletedProcess` fails during rgw relam,
zongroup create et al.

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Local variable name is assigned to but never used (F841)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Line break occurred after a binary operator (W504)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Trailing whitespace (W291)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: undefined name 'channel' (F821)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: use ==/!= to compare constant literals (F632)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: f-string is missing placeholders (F541)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Do not use variables named 'I', 'O', or 'l' (E741)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Do not assign a lambda expression, use a def (E731)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Do not use bare except, specify exception instead (E722)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Comparison to true should be 'if cond is true:' or 'if cond:' (E712)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Multiple statements on one line (def) (E704)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: statement ends with a semicolon (E703)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Expected 1 blank line before a nested definition (E306)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Expected 2 blank lines, found 0 (E302)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: at least two spaces before inline comment (E261)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: multiple spaces after ':' (E241)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Missing whitespace after ',', ';', or ':' (E231)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Missing whitespace around arithmetic operator (E226)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Missing whitespace around operator (E225)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Continuation line under-indented for visual indent (E128)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Continuation line over-indented for hanging indent (E126)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: closing bracket does not match visual indentation (E124)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Continuation line missing indentation or outdented (E122)

also resolves:
continuation line unaligned for hanging indent (E131)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Continuation line under-indented for hanging indent (E121)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Indentation is not a multiple of four (comment) (E114)

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: introduce flake8

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: configure journald as the logdriver

Introduced by e6792f306ab4d07251588fdca6ed3876ae3a092a

podman containers running in a detached `-d` state need to have a
configured logdriver for `cephadm logs` to be able to gather the daemon
output from journald

Fixes: https://tracker.ceph.com/issues/47916
Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: extend `--container-init` to the shell command

allow for the `--init` flag to be passed during `cephadm shell`

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: run containers using `--init` by default

generally all ceph containers need an init process to both reap any
zombie pids and/or perform signal handling (e.g. coredumps, etc.)

Signed-off-by: Michael Fritch <mfritch@suse.com>

doc/man/cephadm: add missing `--container-init` flag

adopt command allows for the `--container-init` flag

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39261 from adk3798/applytest

mgr/orchestrator: useful message when no service type in "ceph orch apply"

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39238 from liewegas/cephadm-conditional-osd-apply

mgr/cephadm: only apply osd specs if the devices have changed

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39259 from sebastian-philipp/cephadm-execnet-chmod-600

mgr/cephadm: on ssh connection error, advice chmod 0600

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39318 from AlexWangD/patch-3

doc: fix the spelling mistake of “Rersult”

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix the spelling mistake of “Rersult”

“Rersult” is a spelling mistake of "Result".

Signed-off-by: Alex Wang wangdashuai@inspur.com

crimson/onode-staged-tree: cleanup -- implement and use get_next_slot()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #39258 from Rethan/doc-remove-ms-options

doc/rados: remove some invalid ms option

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: allow rgw-orphan-list to handle intermediate files w/ binary data

Updates the grep commands with the --text command-line option since
there are cases where rados object names contain NUL characters.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #38538 from chenerqi/qos-skip-discard

librbd: optionally exclude ops from qos

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/cephadm:update test_orchestrator func signature

The available signature has changed to 3 parms, so this
patch is needed to ensure it conforms.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

Merge pull request #39176 from ivancich/wip-rgw-gap-list

rgw: tooling to locate rgw objects with missing rados components

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Michael Kidd <linuxkidd@gmail.com>

Merge PR #39213 into master

* refs/pull/39213/head:
qa/suites/upgrade/octopus-x/parallel: include cephfs in upgrade cluster

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39135 from orozery/librbd-crypto-performance

librbd: crypto performance optimization

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #39298 from dillaman/wip-rbd-suite-readahead

qa/suites/rbd: drop require-osd-release command

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #39147 into master

* refs/pull/39147/head:
qa/tasks/ceph_fuse: do not createfs
qa/tasks/cephfs/fuse_mount: pass admin_socket path
qa/suites/fs/cephadm/multivolume: add basic multivolume test
mgr/mds_autoscaler: some fixes and cleanup
mgr/volumes: deploy MDSs when creating fs

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #39262 from zdover23/wip-doc-removing-workbench-from-docs-2021-Feb-first

doc/dev: Remove workbench mentions

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/suites/rbd: drop require-osd-release command

Teuthology already defaults to quincy now and results in a failure
when trying to set to pacific. Additionally, drop the LUKS readbalance
test since it's unnecessary to duplicate that test.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38715 from lxbsz/rest_api

qa: add REST API method support for ceph-iscsi

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #39265 from zdover23/wip-doc-rados-realy-typo-2020-feb

doc/rados: s/realy/really/

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39263 from cbodley/wip-48963

cmake: partial revert of BOOST_USE_VALGRIND when ALLOCATOR=libc

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

librbd/io: conditionally disable move optimization

If the requested write length does not match the provided bufferlist
length, disable the move optimization and instead fallback to creating
a new sub-bufferlist for the object request.

Fixes: https://tracker.ceph.com/issues/49173
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #39182 from cbodley/wip-49065

rgw multisite: fix RGWCoroutine error handling

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #39241 from dillaman/wip-49117

test/rbd-mirror: fix broken ceph_test_rbd_mirror_random_write

Reviewed-by: Mykola Golub <mgolub@suse.com>

qa/suites/upgrade/octopus-x/parallel: include cephfs in upgrade cluster

Signed-off-by: Sage Weil <sage@newdream.net>

Merge PR #39210 into master

* refs/pull/39210/head:
qa/suites/rados: move thrash-old-clients out of cephadm

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39229 from rhcs-dashboard/label-badge

mgr/dashboard: Add badge to the Label column in Host List

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/MgrClient: use mono_clock for clock_t used by Timer

so we don't suffer from "missing event" when adjust time the system
clock

Signed-off-by: Kefu Chai <kchai@redhat.com>