doc/install/index_manual: remove cephadm from toctree

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge PR #39823 into master

* refs/pull/39823/head:
qa/tasks/cephadm: remove mirror code

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #39822 from mgfritch/cephadm-custom-container-sample

cephadm: split custom container args into argv

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39699 from mgfritch/cephadm-bin-flake8-quotes

cephadm: introduce flake8 and flake8-quotes

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

qa/tasks/cephadm: remove mirror code

This is being done from ansible now.  Also, it breaks when
the conf file has unqualified-search-registries but not 'registry'
entries.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #39831 from xxhdx1985126/wip-crimson-io-sequencer-bug

crimson/osd: do not pass lvalue of the lambda to seastar::futurize_invoke

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39656 from pritha-srivastava/wip-rgw-sts-user-fix

rgw/sts: fix for encoding/decoding user namespace

Merge pull request #39452 from rhcs-dashboard/wip-49283-master

mgr/dashboard: report mgr fsid

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

crimson/osd: do not pass lvalue of the lambda to seastar::futurize_invoke

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #39815 from sebastian-philipp/github-cephadm-minor

.github: Minor cephadm changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39827 from xxhdx1985126/wip-crimson-minor-bugs

crimson/osd: capture error_code by value in PG::handle_failed_op

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #39812 from rzarzynski/wip-crimson-admin-config-help

crimson/admin: add support for 'config help'

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: capture error_code by value in PG::handle_failed_op

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #39801 from rhcs-dashboard/49574-fix-pyjwt

mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #39755 from smithfarm/wip-49550

rpm: use PMDK system libraries on SUSE

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #39799 into master

* refs/pull/39799/head:
doc: update affiliation for myoungwon

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: update affiliation for myoungwon

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

cephadm: split custom container args into argv

introduced by usage of `shlex` in 3ea514c5

Fixes: https://tracker.ceph.com/issues/49590
Signed-off-by: Michael Fritch <mfritch@suse.com>

rpm: use PMDK system libraries on SUSE

As of a49d1dbb32e2436ff2836a85b2fa84418f0a5fff, when the rbd_rwl_cache and
rbd_ssd_cache bconds are enabled and WITH_SYSTEM_PMDK is disabled (as it is by
default), the RPM build attempts to

    git clone https://github.com/ceph/pmdk.git

but of course that won't work in the OBS, where the build workers have no
Internet connectivity.

Fortunately, the openSUSE/SLE versions targeted by Ceph master and pacific ship
the necessary PMDK libraries as RPM packages.

Fixes: a49d1dbb32e2436ff2836a85b2fa84418f0a5fff
Fixes: https://tracker.ceph.com/issues/49550
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #39765 from smithfarm/wip-ncpus-for-obs

rpm: set build parallelism from memory on SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>

Merge PR #39682 into master

* refs/pull/39682/head:
vstart_runner: remove duplicate methods from LocalCephManager

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #39780 into master

* refs/pull/39780/head:
qa/vstart_runner: dont log "not Ceph bin" msg too often

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge PR #39681 into master

* refs/pull/39681/head:
vstart_runner: define path to ceph binary and use it

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #39382 from gauravsitlani/master

rgw: added missing documentation on "pubsub" in rgw_enable_apis

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39762 from smithfarm/wip-support-leap-15.3

rpm/luarocks: simplify conditional and support Leap 15.3

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>

cephadm: fixup `alrady` -> `already`

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: Change outer quotes to avoid escaping inner quotes (Q003)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: Remove bad quotes from multiline string (Q001)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: Remove bad quotes (Q000)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: introduce flake8-quotes

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: line break after binary operator (W504)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: blank line contains whitespace (W293)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: trailing whitespace (W291)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: local variable 'e' is assigned to but never used (F841)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: 'select' imported but unused (F401)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: ambiguous variable name 'l' (E741)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: do not use bare 'except' (E722)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: statement ends with a semicolon (E703)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: module level import not at top of file (E402)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: expected 1 blank line before a nested definition (E306)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: expected 2 blank lines after end of function or class (E305)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: too many blank lines (E303)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: expected 2 blank lines, found 1 (E302)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: expected 1 blank line, found 0 (E301)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: too many leading '#' for block comment (E266)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: block comment should start with '# ' (E265)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: at least two spaces before inline comment (E261)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: unexpected spaces around keyword / parameter equals (E251)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: multiple spaces after ',' (E241)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: missing whitespace after ':' (E231)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: missing whitespace around arithmetic operator (E226)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: missing whitespace around operator (E225)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: whitespace before ':' (E203)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: whitespace after '{' (E201)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: continuation line unaligned for hanging indent (E131)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: continuation line under-indented for visual indent (E128)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: continuation line over-indented for visual indent (E127)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: continuation line over-indented for hanging indent (E126)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: continuation line with same indent as next logical line (E125)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: closing bracket does not match visual indentation (E124)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: ... does not match indentation of opening bracket's line (E123)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: continuation line missing indentation or outdented (E122)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: continuation line under-indented for hanging indent (E121)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: over-indented (E117)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: introduce flake8

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39789 from zdover23/wip-doc-cephadm-convert--bash-shells-2021-Mar-03

doc/cephadm: add prompts to adoption.rst

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39794 from cbodley/wip-qa-rgw-valgrind-supp-librados-delete

valgrind: suppress operator delete[] for tcmalloc

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

.github: Minor cephadm changes

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39558 from sebastian-philipp/doc-rados-rm-toctree-cephadm

doc/rados: avoid including cephadm into the toctree

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

rpm: drop old SUSE-specific OOM fix

This fix was needed some years ago, but no longer.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: limit build jobs by system memory on SUSE

43b441f9a3bc907c17d52385251001ffcd5d3ff9 removed a bunch of code which the SUSE
builds were relying on to avoid OOM. This commit brings back that code in
a much-streamlined form: the SUSE-specific %limit_build macro.

This also has the advantage of not breaking the build on older RPMs which only
know about %_smp_mflags, and not the newer %_smp_build_ncpus etc. macros.

Fixes: 43b441f9a3bc907c17d52385251001ffcd5d3ff9
Fixes: https://tracker.ceph.com/issues/49556
Signed-off-by: Nathan Cutler <ncutler@suse.com>

crimson/admin: add support for 'config help'

This commit solves a problem unveiled in teuthology testing
where `ceph --admin-daemon ... config help` were constantly
returning `EINVAL`:

```
2021-03-03T02:43:01.566 DEBUG:tasks.admin_socket:Testing config help with config {'branch': 'master'}
2021-03-03T02:43:01.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config help
2021-03-03T02:43:01.711 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 3 closest matches:
2021-03-03T02:43:01.712 INFO:teuthology.orchestra.run.smithi083.stderr:config get <var>
2021-03-03T02:43:01.712 INFO:teuthology.orchestra.run.smithi083.stderr:config set <var> <val>...
2021-03-03T02:43:01.712 INFO:teuthology.orchestra.run.smithi083.stderr:config show
2021-03-03T02:43:01.713 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command
2021-03-03T02:43:01.714 DEBUG:teuthology.orchestra.run:got remote process result: 22
2021-03-03T02:43:01.714 INFO:tasks.admin_socket:ceph cli returned an error, command not registered yet?
2021-03-03T02:43:01.715 INFO:tasks.admin_socket:sleeping and retrying ...
```

The root cause is no support for `config help` in crimson.
As shown below this command hasn't been registered at all:

```
DEBUG 2021-03-03 02:42:48,242 [shard 0] osd - start: asok socket path=/var/run/ceph/ceph-osd.0.asok
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): mark_unfound_lost)
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): query)
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): perf dump_seastar)
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): dump_pgstate_history)
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): flush_pg_stats)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): send_beacon)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): status)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): injectargs)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): config show)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): config set)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): config get)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): get_command_descriptions)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): help)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): git_version)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): version)
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Fixes: https://tracker.ceph.com/issues/49574
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #39399 from dang/wip-dang-zipper-10

RGW - Zipper 10: The Great Zippening

Merge PR #38909 into master

* refs/pull/38909/head:
PendingReleaseNotes: document option osd_fast_shutdown_notify_mon
osd: add osd_fast_shutdown_notify_mon option (default false)

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #39654 into master

* refs/pull/39654/head:
common/options: drop ms_async_max_op_threads
msg/async: drop Stack::num_workers
msg/async: s/num_workers/workers.size()/
msg/async: use range-based loop in NetworkStack
msg/async: do not pass worker id to Stack::spawn_worker()
async/Stack: pass Worker* to NetworkStack::add_thread()
async/rdma: do not reference worker id in RDMAStack::spawn_worker()
async/dpdk: do not use worker id when creating worker
async/PosixStack: do not reference worker id in ctor
async/rdma: initialize worker in RDMAStack::create_worker()
async/rdma: move RDMAStack::create_worker() to .cc

Reviewed-by: luo runbing <luo.runbing@zte.com.cn>
Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge PR #39665 into master

* refs/pull/39665/head:
qa/rados/dashbord use "random" objectore

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #39689 into master

* refs/pull/39689/head:
src/global/signal_handler.h: fix preprocessor logic for alpine

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #39750 into master

* refs/pull/39750/head:
mgr: create client messenger using ms_public_type

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39770 from liewegas/bug-46745

cephadm: add docker.service dependency in systemd units

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39795 from dillaman/wip-migration-import-docs

doc/rbd: clarify QCOW2 migration support

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #39555 from tchaikov/wip-drop-pre-nautilus

*: drop backward compatibility with pre-nautilus versions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39798 from tchaikov/wip-crimson-monc-copy-sent

crimson/mon: keep a copy of sent MMonCommand messages

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/mon: keep a copy of sent MMonCommand messages

as per Yingxin Cheng,

> The send process can be asynchronous (there is a conn.out_q, or if the
> underlying socket lives in a different core in the m:n model to be
> ordered there). If user really wants to reuse a message, they must be
> careful not to modify it because it may result in modifing the pending
> messages.
>
> I think the best way is to copy the message if user want to resend it,
> and keep the ceph_assert(!msg->get_seq()). It may looks good to reuse a
> message under lossy policy, but the correctness is now up to user not to
> modify it inplace.

see also https://github.com/ceph/ceph/pull/39786

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rbd: clarify QCOW2 migration support

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

valgrind: suppress operator delete[] for tcmalloc

suppresses a mismatched operator delete[] from the librados client's
destructor:

<error>
  <unique>0xeef7eb</unique>
  <tid>1</tid>
  <kind>MismatchedFree</kind>
  <what>Mismatched free() / delete / delete []</what>
  <stack>
    <frame>
      <ip>0x4C32EA0</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator delete[](void*, unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.16.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>660</line>
    </frame>
    <frame>
      <ip>0x5FBEB86</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5FBEF2C</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5F7F7C5</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
      <fn>librados::v14_2_0::Rados::shutdown()</fn>
    </frame>

Fixes: https://tracker.ceph.com/issues/49501
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #39730 into master

* refs/pull/39730/head:
cephadm: fix escaping/quoting of stderr-prefix arg for ceph daemons

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

doc/cephadm: add prompts to adoption.rst

This PR formats the bash prompts. It also formats the
bash output so that it appears in the correct (easily
copy-and-pasteable) format. This PR will be followed by
a grammar-improving PR, but this PR is just a
formatting PR.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39781 from idryomov/wip-krbd-xfstests-fixes

qa/suites/krbd: address recent issues caused by newer kernels

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #39723 into master

* refs/pull/39723/head:
mon/ConfigMonitor: make config changes via KVMonitor's pending set

Reviewed-by: Neha Ojha <nojha@redhat.com>

cephadm: fix escaping/quoting of stderr-prefix arg for ceph daemons

On some versions of podman doing

   --default-log-stderr-prefix="debug "

seems to work okay, but on others it does not, producing lines like

   /usr/bin/ceph-mon: "debug "2021-02-26T15:30:36.266+0000 7f3d8a955700  4 rocksdb:                    Options.db_write_buffer_size: 0

Quote things properly for the unit.run bash script.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge PR #39739 into master

* refs/pull/39739/head:
cephadm: set CEPH_USE_RANDOM_NONCE if using --init
msg/Messenger: use random nonce if CEPH_USE_RANDOM_NONCE or pid == 1
Revert "Merge PR #39482 into master"

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

RGW - Fix mock User class in Lua test

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

cephadm: add docker.service dependency in systemd units

This ensures that (1) we start after the docker daemon is running, and
(2) a restart of docker will also restart the ceph services.

Fixes: https://tracker.ceph.com/issues/46745
Signed-off-by: Sage Weil <sage@newdream.net>

Merge PR #39679 into master

* refs/pull/39679/head:
Revert "mon/MDSMonitor: add missing frozen checks"

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #38995 from mgfritch/cephadm-check-host-errors

cephadm: remove redundant `ERROR` during check-host

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39722 from p-se/wip-pse-cephadm-SUSE-alertmanager

cephadm: `cephadm ls` broken for SUSE downstream alertmanager container

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38915 from Daniel-Pivonka/clientsoktostop

mgr/cephadm: add ok-to-stop functions for ceph client services

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39352 from sebastian-philipp/orch-remove-promise

mgr/orch: Make orchestrator interface synchronous

Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #39612 from jmolmo/fix_rook_orch_ls

mgr/orchestrator: Fix ceph orch ls in Rook

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>