doc: fix a spelling error at /doc/radosgw/dynamicresharding.rst

Signed-off-by: bangmingcheng <bangmingcheng@gmail.com>

Merge PR #33135 into master

* refs/pull/33135/head:
mon: fix tell command registration
ceph.in: fix -h/help behavior for tell commands
ceph.in: remove '{osd,mds,mon} tell' deprecation warning

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33190 into master

* refs/pull/33190/head:
cephadm: report image name for stopped daemons
cephadm: record the image name to unit.image

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #33186 from yuriw/wip-yuriw-crontab-master

qa/tests: removed all runs for luminous - EOL

Merge PR #33179 into master

* refs/pull/33179/head:
mgr/orchestrator: Use `pickle` to pass exceptions across sub-interpreters

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33184 into master

* refs/pull/33184/head:
qa/suites/rados/multimon/tasks/mon_clock_with_skews: whitelist MOST_DOWN
qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33043 into master

* refs/pull/33043/head:
doc/releases: octopus draft notes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32965 from Dmole/patch-3

doc: quick-rbd.rst de-duplicate

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #33181 into master

* refs/pull/33181/head:
qa/workunits/cephadm/test_cephadm: fix typo

Reviewed-by: Sebastian Wagner <swagner@suse.com>

doc/releases: octopus draft notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33131 into master

* refs/pull/33131/head:
mgr/orch: 'ceph orchestrator ...' -> 'ceph orch ...'

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33162 into master

* refs/pull/33162/head:
cephadm: KillMode=none in unit file
cephadm: clean up unit string syntax a bit

Reviewed-by: Sebastian Wagner <swagner@suse.com>

cephadm: report image name for stopped daemons

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: record the image name to unit.image

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33185 from tchaikov/wip-44063

pybind/mgr: bump six to 1.14

Reviewed-by: Sebastian Wagner <swagner@suse.com>

qa/tests: removed all runs for luminous - EOL

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

doc: quick-rbd.rst de-duplicate

Remove code duplication from the dependency.

Signed-off-by: Tim <elatllat@gmail.com>

pybind/mgr: bump six to 1.14

otherwise some packages which uses six.ensure_text will fail, as it
was introduced by 1.12. and 1.14 is the latest stable version of
six. so let's use it instead.

Fixes: https://tracker.ceph.com/issues/44063
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #33137 into master

* refs/pull/33137/head:
mon/PGMap: fix digest summary of PGs by state
mon/PGMap: fix compat encoding of num_pg_by_state

Reviewed-by: yanjun <yan.jun8@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mon: fix tell command registration

Pass full cmddesc, not just prefix.

Broken by 3394b70ba18cffa49317141ff444ba9990d6170b

Fixes: https://tracker.ceph.com/issues/44050
Signed-off-by: Sage Weil <sage@redhat.com>

ceph.in: fix -h/help behavior for tell commands

- add "Tell $type commands" heading
- 'ceph tell mon.a -h' now works
- 'ceph tell mon.a prefix -h' also works

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orchestrator: Use `pickle` to pass exceptions across sub-interpreters

`copy.deepcopy` doesn't work to cross sub-interpreter boundaries.

Fixes: https://tracker.ceph.com/issues/43913
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #33035 from jdurgin/wip-target-ratio

mgr/pg_autoscaler: treat target ratios as weights

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/pg_autoscaler: add unit tests for effective_target_ratio

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/pg_autoscaler: remove unused imports

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/pg_autoscaler: cleanup pool_id type

Keep it as an int so we don't have to cast back and forth.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/pg_autoscaler: correct and simplify progress tracking

Reset the progress each time we make an adjustment, and track progress
from that initial state to that new target. Previously we were also
using the wrong target: the current pg_num_target, not the new value
(pg_num_final) that we set.

Look up the pool by name, not id, in _maybe_adjust(), since that is how it is
retrieved by osdmap.get_pools_by_name().

Dedupe some logic into PgAdjustmentProgress to simplify things.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

PendingReleaseNotes: mention target_size_ratio change

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

doc: update autoscaler docs for target ratio

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/pg_autoscaler: include effective target ratio in status

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/pg_autoscaler: add warning when target bytes and ratio are both set

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/pg_autoscaler: remove target ratio warning

Since the ratios are normalized, they cannot exceed 1.0 or overcommit
combined with target_bytes.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

qa: use ratios >1 in pg_autoscaler test

Also check for pg_num_target being set correctly, rather than pg_num,
so the test doesn't depend on merging/splitting speed.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/pg_autoscaler: treat target ratios as weights

Normalize across pools so that it's simpler to use - this way you
don't have to adjust every other pool when you add one.

Handle pools with target_bytes by taking their capacity off the top,
and dividing the rest into the pools with a target_ratio.

If both target bytes and ratio are specified, ignore bytes. This
matches the docs and makes accounting simpler.

Fixes: https://tracker.ceph.com/issues/43947
Signed-off-by: Josh Durgin <jdurgin@redhat.com>

mgr/orch: 'ceph orchestrator ...' -> 'ceph orch ...'

Less typing for all!

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephadm/test_cephadm: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33126 into master

* refs/pull/33126/head:
doc/mgr/orchestrator_cli: update support table
mgr/deepsea: remove
mgr/ansible: remove

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33136 into master

* refs/pull/33136/head:
cephadm: fix ceph version probe

Reviewed-by: Michael Fritch <mfritch@suse.com>

cephadm: fix ceph version probe

docker returns '<no value>' if the label isn't present, in which case we
still need to run ceph -v.

Also, don't probe non-ceph (e.g., monitoring) containers.

Also, only probe each image id once.

Add a simple test.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33133 into master

* refs/pull/33133/head:
qa/workunits/cephadm/test_cephadm.sh: make monitoring tests faster
qa/workunits/cephadm/test_cephadm: 2 OSDs is enough
cephadm: disable node-exporter cpu/memory limits for the time being

Reviewed-by: Michael Fritch <mfritch@suse.com>

qa/suites/rados/multimon/tasks/mon_clock_with_skews: whitelist MOST_DOWN

The skewed clock makes some mons miss elections.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33134 into master

* refs/pull/33134/head:
qa/workunits/cephadm/test_repos: don't try to use the refspec

Reviewed-by: Michael Fritch <mfritch@suse.com>

qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Fixes: https://tracker.ceph.com/issues/43889
Signed-off-by: Sage Weil <sage@redhat.com>

doc/mgr/orchestrator_cli: update support table

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/deepsea: remove

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33117 into master

* refs/pull/33117/head:
qa/suites/upgrade/nautilus-x-singleton: ensure hit sets behave across upgrade
osd/PrimaryLogPG: use legacy timestamp rendering for hit_set objects
include/utime: allow legacy rendering of timestamp

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/PGMap: fix digest summary of PGs by state

This fixes the PG state summary seen in 'ceph -s' to reflect PG state bits
beyond 32 bits.

Fixes: https://tracker.ceph.com/issues/44052
Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: fix compat encoding of num_pg_by_state

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #32928 from ljishen/wip-rados-bench-latency

rados bench: fix the delayed checking of completed ops

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #32934 from rzarzynski/wip-bl-32bytes

include, common: make ceph::bufferlist 32 bytes long on x86

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33099 from howard0su/wip_ceph_fix

ceph.in: print decoded output in interactive mode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33101 from tchaikov/wip-thread-join

pybind/ceph_argparse: avoid int overflow

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33068 from yuvalif/verify-pubsub-inc-sync

qa/rgw/pubsub: verify incremental sync is used in pubsub

Merge PR #32816 into master

* refs/pull/32816/head:
mds: check inode type when deciding if filelock should be in EXCL state
mds: don't delegate inos when handling replayed requests
mds: process re-sent async dir operations at clientreplay stage
mds: consider async dirops when checking directory empty
mds: always suppress issuing caps in Locker::issue_new_caps()
mds: try reconnect cap only when replayed request creates new inode
mds: set cap id to 1 for newly created inode

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

cephadm: KillMode=none in unit file

The unit already has a ExecStop command that runs 'podman|docker stop',
so systemd should not *also* try to kill the process or else it may
screw up the container state

From https://www.redhat.com/sysadmin/podman-shareable-systemd-services

"Note: It’s important to set the kill mode to none. Otherwise, systemd
will start competing with Podman to stop and kill the container processes.
which can lead to various undesired side effects and invalid states."

Hopefully-fixes: https://tracker.ceph.com/issues/43883
Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: clean up unit string syntax a bit

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephadm/test_cephadm.sh: make monitoring tests faster

The sleep 90 was way overkill.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephadm/test_cephadm: 2 OSDs is enough

Signed-off-by: Sage Weil <sage@redhat.com>

ceph.in: remove '{osd,mds,mon} tell' deprecation warning

The last one of the was removed in luminous.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: disable node-exporter cpu/memory limits for the time being

Ubuntu 18.04 kernel does not support these.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33092 into master

* refs/pull/33092/head:
doc/rados/operations: adjust docs a bit
mon/OSDMonitor: accept 'autoscale_mode' argument to 'osd pool create'

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #33094 into master

* refs/pull/33094/head:
mgr/telemetry: split entity_name only once (handle ids with dots)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33037 into master

* refs/pull/33037/head:
osd/OSD: choose more heartbeat peers from different subtrees

Reviewed-by: yanjun <yan.jun8@zte.com.cn>

qa/workunits/cephadm/test_repos: don't try to use the refspec

This is usually a sha1, and we can't reliably find packages based on
that.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33132 into master

* refs/pull/33132/head:
qa/workunits/cephadm/test_repos: apt update
qa/workunits/cephadm/test_repos: fix subst
qa/suites/rados/cephadm/.../test_repos: run without ulimit and coverage tools
qa/tasks/workunit: accept coverage_and_limits: false
qa/suites/rados/cephadm: move ubuntu_18.04_podman to shared location
qa/suites/rados/cephadm: fix conflicts, missing .qa link

Reviewed-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_repos: apt update

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephadm/test_repos: fix subst

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33061 from tchaikov/wip-pybind-compiler-flags-patch

pybind: refactor monkey_with_compiler()

Reviewed-By: Adam Emersen <aemerson@redhat.com>

Merge PR #33098 into master

* refs/pull/33098/head:
mgr/orch,cephadm: add 'host set-addr'
mgr/orch: include addr (and labels) in 'host ls'
mgr/cephadm: fix 'cephadm check-host'
mgr/cephadm: use addr to contact host
mgr/orch: pass HostSpec to add_host
mgr/orch: HostSpec -> HostPlacementSpec

Reviewed-by: Sebastian Wagner <swagner@suse.com>

qa/suites/rados/cephadm/.../test_repos: run without ulimit and coverage tools

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/workunit: accept coverage_and_limits: false

Allow workunits without teuthology tools (normally installed by ceph.py,
IIRC).

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/cephadm: move ubuntu_18.04_podman to shared location

Also set the registries.conf file so we can pull from docker.io.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/upgrade/nautilus-x-singleton: ensure hit sets behave across upgrade

Create a pool that generates hit sets before the upgrade, and ensure that
they (continue to) trim after the upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: use legacy timestamp rendering for hit_set objects

These objects exist prior to octopus and we need to be consistent with
the naming.  Stick with the legacy form.

Fixes: https://tracker.ceph.com/issues/44024
Signed-off-by: Sage Weil <sage@redhat.com>

include/utime: allow legacy rendering of timestamp

In 79d8d761cf8fb6a5679d1925f92889c950dc2be4 and
ec3ddcb9886e3c74b78aa8521bc05e695f6aeeab
we switched to a strict ISO8660 rendering for timestamps.  In some cases,
we need to render the timestamp in the legacy form: ' ' instead of 'T',
and no time zone suffix.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/cephadm: fix conflicts, missing .qa link

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33080 into master

* refs/pull/33080/head:
qa: specify random distros in multimds

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #33070 into master

* refs/pull/33070/head:
mgr/telemetry: use raise_for_status()
mgr/telemetry: factor post into helper
mgr/telemetry: catch exception during requests.put

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>

mgr/orch,cephadm: add 'host set-addr'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: include addr (and labels) in 'host ls'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: fix 'cephadm check-host'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: use addr to contact host

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: pass HostSpec to add_host

Distinguish between the hostname and the addr (dns name or IP) to reach
the host.  Include labels here too since it's convenient to do so.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: HostSpec -> HostPlacementSpec

This object is about describing where to place a service on a host: it
includes a host name and either an IP or network and possibly even a name
for the service.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33118 into master

* refs/pull/33118/head:
orchestrator cli: change 'rgw update' params order
mgr/cephadm: fix minor typo

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33062 into master

* refs/pull/33062/head:
qa/suites/rados/cephadm: collect all cephadm tests together here
qa/workunits/cephadm/test_repos: add test for the repo commands
cephadm: add '{add,rm}-repo', with initial centos/rhel/fedora/ubuntu support

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33119 into master

* refs/pull/33119/head:
mgr/upgrade: fix mgr self check
mgr/cephadm: fail upgrade if target image doesn't exist
mgr/cephadm: factor upgrade failure into helper
mgr/cephadm: refresh if we don't know a daemon's image_id
mgr/cephadm: refresh services in upgrade loop
mgr/cephadm: clean up upgrade messages a bit

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #33086 from guits/guits-fix_cv_rerun

ceph-volume: skip osd creation when already done

qa/suites/rados/cephadm: collect all cephadm tests together here

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/cephadm/test_repos: add test for the repo commands

This isn't a great test, but it is something.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: add '{add,rm}-repo', with initial centos/rhel/fedora/ubuntu support

Other distros to follow.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33111 into master

* refs/pull/33111/head:
qa/suites/rados/cephadm[-smoke]: test podman on ubuntu 18.04

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #33128 from pponnuvel/fix_incorrect_link_documentation

coding-style: update a link and fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

coding-style: update a link and fix typos.

Signed-off-by: Ponnuvel Palaniyappan <pponnuvel@gmail.com>

mgr/ansible: remove

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-volume: add unit test test_safe_prepare_osd_already_created

This commit adds a new unit test
`test_safe_prepare_osd_already_created()` in order to test when
`is_ceph_device()` returns `True` `RuntimeError` is well raised.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-volume: skip osd creation when already done

When rerunning ceph-volume lvm create on a device already prepared and
activated, ceph-volume should skip the creation.

This is a regression introduced by bb4de1a3fc238eaf9f717dc59c6bdf338ef6d657

Fixes: https://tracker.ceph.com/issues/43981
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge PR #33042 into master

* refs/pull/33042/head:
cephadm: bootstrap: warn on fqdn hostname

Reviewed-by: Sebastian Wagner <swagner@suse.com>

pybind/ceph_argparse: avoid int overflow

in python 2.6.8, `thread.join(timeout)` tries to convert the given
timeout to PyTime, but turns out `2 << 32` overflows when python
runtime converts the timeout from sec to ns. that's why
the `lock.acquire()` call always fail in
`Thread._wait_for_tstate_lock()`.
and we end up with an alive thread after calling `thread.join()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/cephadm[-smoke]: test podman on ubuntu 18.04

Signed-off-by: Sage Weil <sage@redhat.com>