]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agotest: Add test for scrub parallelism 40077/head
David Zafman [Fri, 15 Jan 2021 04:40:04 +0000 (20:40 -0800)]
test: Add test for scrub parallelism

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit dd63577ab372d1d4b2da284c6e506603b293d85d)

4 years agoosd: Try other PGs when reservation failures occur
David Zafman [Fri, 15 Jan 2021 04:39:32 +0000 (20:39 -0800)]
osd: Try other PGs when reservation failures occur

Fixes: https://tracker.ceph.com/issues/48843
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 08c3ede0844507074de2b43638423c094976b493)

4 years agoMerge pull request #39977 from singuliere/wip-49054-pacific
Yuri Weinstein [Thu, 11 Mar 2021 18:40:12 +0000 (10:40 -0800)]
Merge pull request #39977 from singuliere/wip-49054-pacific

pacific: common/mempool: Improve mempool shard selection

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39952 from rzarzynski/wip-tool-nul-in-rados-pacific
Yuri Weinstein [Thu, 11 Mar 2021 18:38:58 +0000 (10:38 -0800)]
Merge pull request #39952 from rzarzynski/wip-tool-nul-in-rados-pacific

pacific: tools/rados: add support for binary object names in the rados CLI

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39951 from rzarzynski/wip-librados-nul-objname-pacific
Yuri Weinstein [Thu, 11 Mar 2021 18:38:25 +0000 (10:38 -0800)]
Merge pull request #39951 from rzarzynski/wip-librados-nul-objname-pacific

 pacific: librados, tests: allow to list objects with the NUL character in names

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #39898 from singuliere/wip-49038-pacific
Yuri Weinstein [Thu, 11 Mar 2021 18:37:40 +0000 (10:37 -0800)]
Merge pull request #39898 from singuliere/wip-49038-pacific

pacific: blk/kernel: fix io_uring got (4) Interrupted system call

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39881 from singuliere/wip-49384-pacific
Yuri Weinstein [Thu, 11 Mar 2021 18:36:48 +0000 (10:36 -0800)]
Merge pull request #39881 from singuliere/wip-49384-pacific

pacific: os/bluestore/BlueFS: use iterator_impl::copy instead of bufferlist::c_str() to avoid bufferlist rebuild

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge PR #39807 into pacific
Sage Weil [Wed, 10 Mar 2021 20:27:27 +0000 (14:27 -0600)]
Merge PR #39807 into pacific

* refs/pull/39807/head:
cephadm: split custom container args into argv
cephadm: fix escaping/quoting of stderr-prefix arg for ceph daemons
cephadm: set CEPH_USE_RANDOM_NONCE if using --init
msg/Messenger: use random nonce if CEPH_USE_RANDOM_NONCE or pid == 1
Revert "Merge PR #39482 into master"
cephadm: remove redundant `ERROR` during check-host
cephadm: remove unused imports
cephadm: `cephadm ls` broken for SUSE downstream alertmanager container
cephadm: `cephadm ls` broken for SUSE downstream alertmanager container
mgr/cephadm: add ok-to-stop functions for ceph client services
mgr/test_orchestrator: Refactor create_osds
mgr/volumes: adapt to now orch interface
doc/mgr/orchestrator_modules: adapt to now orch interface
mgr/selftest: adapt to now orch interface
mgr/dashboard: adapt to now orch interface
mgr/mds_autoscaler: Add to tox.ini
mgr/mds_autoscaler: adapt to now orch interface
mgr/test_orchestrator: adapt to now orch interface
mgr/rook: Adapt to new orch interface
mgr/cephadm: Adapt cephadm to new orch interface
mgr/orch: Remove old tests
mgr/orch: adapt orchestrator CLI to new interface
mgr/orch: replace Completion with OrchResult(Generic[T])
mgr/orchestrator: Fix ceph orch ls in Rook
doc/cephadm: rewrite "install cephadm"
doc/cephadm: rewrite "b.strap a new cluster"
cephadm: add docker.service dependency in systemd units
cephadm: add multi-digest test
mgr/orchestrator: validate config options at apply time
mgr/cephadm: disallow managed options in ServiceSpec config section
mgr/cephadm: add config section to ServiceSpec
doc/cephadm: s/togeter/together
cephadm: provide meta during bootstrap
mgr/cephadm: put service_name in unit.meta and use it when available
cephadm: accept arbitrary dict via --meta-json
mgr/cephadm: incorporte memory_{usage,request,limit} from 'ls'
cephadm: accept --memory-{request,limit}
cephadm: include memory_usage in 'ls' output
doc/cephadm: remove Orchestrator CLI from cephadm toc
doc/cephadm: move host labels to host mgmt
doc/cephadm: group MDS sections into one chapter
doc/cephadm: Add iscsi
doc/cephadm: group NFS sections into one chapter
doc/cephadm: rename monitoring chapter title
doc/cephadm: group MON sections into one chapter
doc/cephadm: make custom containers its own chapter
doc/cephadm: group RGW mgmt sections into one chapter
doc/cephadm: move scheduler topic to service mgmt
doc/cephadm: move unmanaged=true to service-mgmt.rst
doc/cephadm: group general service mgmt sections into one chapter
doc/cephadm: group OSD mgmt sections into one chapter
doc/cephadm: Move FQDN chapter to host mgmt.rst
doc/cephadm: Move SSH config from operations to host-mgmt.rst
doc/cephadm: group host mgmt sections into one chapter
cephadm: fix bug in orphan-initial-daemons logic
mgr/orch: drop __all__ from __init__.py
mgr/cephadm: add DaemonDescriptionStatus
cephadm: version command hide traceback when login is needed
doc/cephadm: troubleshooting: manually deploy MGR
cephadm: fix port_in_use when IPv6 is disabled
cephadm: Allow to use paths in all <_devices> drivegroup sections
mgr/cephadm: error if service action called with daemonless service
mgr/cephadm: fix up the strings reporting osd ids
mgr/cephadm: remove daemon before osd destroy/purge
mgr/cephadm: simplify OSD __str__ for drain
mgr/cephadm: make drain adjust crush weight if not replacing
mgr/cephadm: less log noise from osd drain code
mgr/cephadm: fix 'orch daemon add osd ...'
mgr/cephadm/upgrade: fix typo
mgr/cephadm: remove spec from CephadmDaemonDeploySpec
mgr/cephadm/upgrade: restart mgr after mons upgrade to pacific
mgr/cephadm: use get_foreign_ceph_option() instead of 'config get' mon command

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #39943 from rhcs-dashboard/wip-49665-pacific
Ernesto Puerta [Wed, 10 Mar 2021 17:17:57 +0000 (18:17 +0100)]
Merge pull request #39943 from rhcs-dashboard/wip-49665-pacific

pacific: mgr/dashboard: Host Maintenance Feature

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
4 years agomgr/dashboard: Host Maintenance Feature 39943/head
Nizamudeen A [Tue, 2 Feb 2021 12:26:13 +0000 (17:56 +0530)]
mgr/dashboard: Host Maintenance Feature

In Cluster -> Hosts, I've added additional button to put the selected host on maintenance or exit out of the maintenance mode. Also for some hosts the ok-to-stop tests may trigger some warnings which requires a --force command to pass along with the maintenance enter command to enter a host into maintenance. In UI this is achieved using a confirmation Modal. In addition to this if the check error is It is NOT safe to stop the host then the host wont be able to put into maintenance mode.

Fixes: https://tracker.ceph.com/issues/49101
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 0780fc131d1484601318eb0ec39f14955c731e16)

4 years agoMerge pull request #39913 from rhcs-dashboard/wip-49652-pacific
Ernesto Puerta [Wed, 10 Mar 2021 11:00:02 +0000 (12:00 +0100)]
Merge pull request #39913 from rhcs-dashboard/wip-49652-pacific

pacific: mgr/dashboard: fix PUT - /api/host/{hostname} while adding labels

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #39963 from neha-ojha/wip-49670-pacific
Neha Ojha [Wed, 10 Mar 2021 01:24:14 +0000 (17:24 -0800)]
Merge pull request #39963 from neha-ojha/wip-49670-pacific

pacific: src/test/smoke.sh: increase timeout

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #39921 from neha-ojha/wip-revert-autoscaler-pacific
Yuri Weinstein [Tue, 9 Mar 2021 23:58:59 +0000 (15:58 -0800)]
Merge pull request #39921 from neha-ojha/wip-revert-autoscaler-pacific

pacific: Revert "mgr/pg_autoscaler: avoid scale-down until there is pressure"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
4 years agoMerge pull request #39844 from neha-ojha/wip-49565-pacific
Yuri Weinstein [Tue, 9 Mar 2021 23:57:38 +0000 (15:57 -0800)]
Merge pull request #39844 from neha-ojha/wip-49565-pacific

pacific: tests: ceph_test_rados_api_watch_notify: Allow for reconnect

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
4 years agocommon/mempool: Modify shard selection function 39977/head
Adam Kupczyk [Mon, 25 Jan 2021 10:45:06 +0000 (11:45 +0100)]
common/mempool: Modify shard selection function

Modify shard selection function, so no longer all threads use the same shard 0.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit fccbdc0905e3868666fbb10803bac6b73f687cb1)

4 years agocommon/mempool: Add test for mempool shards
Adam Kupczyk [Mon, 25 Jan 2021 10:33:28 +0000 (11:33 +0100)]
common/mempool: Add test for mempool shards

Add test that checks quality of mempool sharing.
It refuses to accept cases when variance is too large.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit e4e07de8f36d0d44b341d5784a8bf56192201b95)

4 years agosrc/test/smoke.sh: increase timeout 39963/head
Sage Weil [Thu, 4 Mar 2021 23:34:00 +0000 (17:34 -0600)]
src/test/smoke.sh: increase timeout

Fixes: https://tracker.ceph.com/issues/47419
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 69d7aa9bf1b0de490e70cedd0463d3ca765a5673)

4 years agoMerge pull request #39901 from tchaikov/pr-39461-pacific
Yuri Weinstein [Tue, 9 Mar 2021 17:35:48 +0000 (09:35 -0800)]
Merge pull request #39901 from tchaikov/pr-39461-pacific

pacific: rgw/test: allow for duplicate events in pubsub

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
4 years agotools/rados: assign to optional<> without deref'ing it 39952/head
Kefu Chai [Wed, 24 Feb 2021 14:24:50 +0000 (22:24 +0800)]
tools/rados: assign to optional<> without deref'ing it

this change addresses a regression introduced by
d333b35aa10bf03a8bc047994d5cf3fed019b49a

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 41d68594ae6df343782a1c382ac0bddc8cb7cb69)

4 years agotools/rados: replace omap_key_valid with std::optional.
Radoslaw Zarzynski [Fri, 5 Feb 2021 18:47:29 +0000 (18:47 +0000)]
tools/rados: replace omap_key_valid with std::optional.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit eb87a0646a3635a7a35b6326e6d26e4b0b5b9d37)

4 years agotools/rados: replace omap_key_pretty with prettify().
Radoslaw Zarzynski [Fri, 5 Feb 2021 18:43:23 +0000 (18:43 +0000)]
tools/rados: replace omap_key_pretty with prettify().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit cca92228f279d1334207b5e1f4bcfa225dca7f11)

4 years agotools/rados: add support for binary object names.
Radoslaw Zarzynski [Fri, 5 Feb 2021 15:28:04 +0000 (15:28 +0000)]
tools/rados: add support for binary object names.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit d333b35aa10bf03a8bc047994d5cf3fed019b49a)

4 years agotools: switch scratch tool to use rados_nobjects_list_next2(). 39951/head
Radoslaw Zarzynski [Thu, 4 Feb 2021 20:22:17 +0000 (20:22 +0000)]
tools: switch scratch tool to use rados_nobjects_list_next2().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 05dca8aa338fecd701c5a681242322b50bd11e67)

4 years agolibrados: NObjectIteratorImpl takes care about NUL in obj name.
Radoslaw Zarzynski [Thu, 4 Feb 2021 17:35:24 +0000 (17:35 +0000)]
librados: NObjectIteratorImpl takes care about NUL in obj name.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 22bb1498c93b1e1419920fe928e91885504f1f16)

4 years agotest: add unit test for rados_nobjects_list_next2().
Radoslaw Zarzynski [Fri, 5 Feb 2021 12:38:47 +0000 (12:38 +0000)]
test: add unit test for rados_nobjects_list_next2().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 8965506b8bd0b74ca5ec743e5eddd40dd4d37bac)

4 years agolibrados: introduce rados_nobjects_list_next2().
Radoslaw Zarzynski [Thu, 4 Feb 2021 17:33:13 +0000 (17:33 +0000)]
librados: introduce rados_nobjects_list_next2().

It's needed to properly list an object with the NUL
character in the name. Such names can appear mostly
as a consequence of bugs in clients.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 9d268b0dd72324347ce74408b9a7a2a95c37ad0f)

4 years agoMerge pull request #39829 from smithfarm/wip-49593-pacific
Yuri Weinstein [Tue, 9 Mar 2021 16:11:44 +0000 (08:11 -0800)]
Merge pull request #39829 from smithfarm/wip-49593-pacific

pacific: rpm: use PMDK system libraries on SUSE

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agocephadm: split custom container args into argv 39807/head
Michael Fritch [Wed, 3 Mar 2021 20:00:32 +0000 (13:00 -0700)]
cephadm: split custom container args into argv

introduced by usage of `shlex` in 3ea514c5

Fixes: https://tracker.ceph.com/issues/49590
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 1474477375761b3d18e221375201d23dd8a2ab79)

4 years agocephadm: fix escaping/quoting of stderr-prefix arg for ceph daemons
Sage Weil [Tue, 2 Mar 2021 16:35:41 +0000 (10:35 -0600)]
cephadm: fix escaping/quoting of stderr-prefix arg for ceph daemons

On some versions of podman doing

   --default-log-stderr-prefix="debug "

seems to work okay, but on others it does not, producing lines like

   /usr/bin/ceph-mon: "debug "2021-02-26T15:30:36.266+0000 7f3d8a955700  4 rocksdb:                    Options.db_write_buffer_size: 0

Quote things properly for the unit.run bash script.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3ea514c5525cd21722ef3fe9b90363c21e483596)

4 years agocephadm: set CEPH_USE_RANDOM_NONCE if using --init
Sage Weil [Sat, 27 Feb 2021 20:44:22 +0000 (15:44 -0500)]
cephadm: set CEPH_USE_RANDOM_NONCE if using --init

This ensures that daemon messenger nonces don't collide by using PIDs that are
no longer unique for the IP address.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 576823bcb67eec0d0899ad961be07da7c0856732)

4 years agomsg/Messenger: use random nonce if CEPH_USE_RANDOM_NONCE or pid == 1
Sage Weil [Sat, 27 Feb 2021 20:45:47 +0000 (15:45 -0500)]
msg/Messenger: use random nonce if CEPH_USE_RANDOM_NONCE or pid == 1

If we are in a container, then we do not have a unique pid, and need to
use a random nonce.  We normally detect this if our pid is 1, but that
doesn't work when we have a init process--we'll (probably?) have a small
pid (in my tests, the OSDs were getting pid 7).

To be safe, also check for an environment variable set by cephadm.

This avoids problems that arise when we don't have a unique address.

Fixes: https://tracker.ceph.com/issues/49534
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d18f60854e68d2060c1a3841299466f165f6552d)

4 years agoRevert "Merge PR #39482 into master"
Sage Weil [Sat, 27 Feb 2021 20:54:21 +0000 (15:54 -0500)]
Revert "Merge PR #39482 into master"

This reverts commit 9200b1ea8e5, reversing
changes made to e42bbba9ca8.

For running tests to narrow down the root cause of:
https://tracker.ceph.com/issues/49237

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit a16e46ef0589129bf990dc7dd24e5fa5c6defdbe)

4 years agocephadm: remove redundant `ERROR` during check-host
Michael Fritch [Wed, 20 Jan 2021 21:28:38 +0000 (14:28 -0700)]
cephadm: remove redundant `ERROR` during check-host

```
ERROR: ERROR: No time synchronization is active
```

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ace5d41dc580a1ea0d0aba19f07da355b06823ef)

4 years agocephadm: remove unused imports
Patrick Seidensal [Mon, 1 Mar 2021 14:36:18 +0000 (15:36 +0100)]
cephadm: remove unused imports

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 0d556267f72c68d5ae291b29d5f664adb30ffb67)

4 years agocephadm: `cephadm ls` broken for SUSE downstream alertmanager container
Patrick Seidensal [Fri, 26 Feb 2021 16:27:17 +0000 (17:27 +0100)]
cephadm: `cephadm ls` broken for SUSE downstream alertmanager container

Fixes: https://tracker.ceph.com/issues/49506
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 3c222a8705bd67e4bff4f03df822770cc2e4893f)

4 years agocephadm: `cephadm ls` broken for SUSE downstream alertmanager container
Patrick Seidensal [Fri, 26 Feb 2021 16:27:17 +0000 (17:27 +0100)]
cephadm: `cephadm ls` broken for SUSE downstream alertmanager container

Fixes: https://tracker.ceph.com/issues/49506
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 8f0dae05fb0c50fbfc0db952ecbfda3900a8a10c)

4 years agomgr/cephadm: add ok-to-stop functions for ceph client services
Daniel-Pivonka [Thu, 14 Jan 2021 22:18:43 +0000 (17:18 -0500)]
mgr/cephadm: add ok-to-stop functions for ceph client services

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 674912bfed92537a97e625bb79397bf97f10b24b)

4 years agomgr/test_orchestrator: Refactor create_osds
Sebastian Wagner [Mon, 1 Mar 2021 12:56:51 +0000 (13:56 +0100)]
mgr/test_orchestrator: Refactor create_osds

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0f22f24a35651a37430296e6c2eea454b3489e6a)

4 years agomgr/volumes: adapt to now orch interface
Sebastian Wagner [Wed, 10 Feb 2021 12:39:44 +0000 (13:39 +0100)]
mgr/volumes: adapt to now orch interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a455dff412525de847c6157242349309f941d64b)

4 years agodoc/mgr/orchestrator_modules: adapt to now orch interface
Sebastian Wagner [Tue, 9 Feb 2021 11:02:02 +0000 (12:02 +0100)]
doc/mgr/orchestrator_modules: adapt to now orch interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d529f1db8242d683df7ec0f405dc695f8203357d)

4 years agomgr/selftest: adapt to now orch interface
Sebastian Wagner [Tue, 9 Feb 2021 10:58:31 +0000 (11:58 +0100)]
mgr/selftest: adapt to now orch interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 828992d7fece771c207e7de01780db541165ec61)

4 years agomgr/dashboard: adapt to now orch interface
Sebastian Wagner [Mon, 8 Feb 2021 09:00:21 +0000 (10:00 +0100)]
mgr/dashboard: adapt to now orch interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b92dc7120588b53293f8fcff5180b428d7f4a1ff)

4 years agomgr/mds_autoscaler: Add to tox.ini
Sebastian Wagner [Mon, 1 Mar 2021 13:10:19 +0000 (14:10 +0100)]
mgr/mds_autoscaler: Add to tox.ini

Fixes: f69abe6291958a9ccfb85ce77c9b86e8af8ecf21
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit bca36ab3cf7499a99824b225433e72b99aeb5b43)

4 years agomgr/mds_autoscaler: adapt to now orch interface
Sebastian Wagner [Mon, 8 Feb 2021 08:54:07 +0000 (09:54 +0100)]
mgr/mds_autoscaler: adapt to now orch interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1d406328268b8d761a2a669889bceb5c5c9374b8)

4 years agomgr/test_orchestrator: adapt to now orch interface
Sebastian Wagner [Mon, 8 Feb 2021 00:56:16 +0000 (01:56 +0100)]
mgr/test_orchestrator: adapt to now orch interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1ecb93a47066aff0f68d9dd7abd145c2ca22dcd1)

4 years agomgr/rook: Adapt to new orch interface
Sebastian Wagner [Mon, 8 Feb 2021 00:47:42 +0000 (01:47 +0100)]
mgr/rook: Adapt to new orch interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 42d5c8991a9b806b960623386c95128515468c6d)

Conflicts:
src/pybind/mgr/rook/module.py

4 years agomgr/cephadm: Adapt cephadm to new orch interface
Sebastian Wagner [Mon, 8 Feb 2021 00:37:07 +0000 (01:37 +0100)]
mgr/cephadm: Adapt cephadm to new orch interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 68030bec52c98b19c398aff26677f6ff4a71fbe3)

4 years agomgr/orch: Remove old tests
Sebastian Wagner [Mon, 8 Feb 2021 00:24:38 +0000 (01:24 +0100)]
mgr/orch: Remove old tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b6c770a4cb4ab1fdcf738dd1df2fe6ca8801f1ad)

4 years agomgr/orch: adapt orchestrator CLI to new interface
Sebastian Wagner [Mon, 8 Feb 2021 00:21:37 +0000 (01:21 +0100)]
mgr/orch: adapt orchestrator CLI to new interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit e4d7d7676fdc5317631c2470208319f18cfc421a)

4 years agomgr/orch: replace Completion with OrchResult(Generic[T])
Sebastian Wagner [Mon, 8 Feb 2021 00:01:08 +0000 (01:01 +0100)]
mgr/orch: replace Completion with OrchResult(Generic[T])

Greatly simplify the orchestrator interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit fcf75136634aa7750a242c60b81b86619713fbbe)

4 years agomgr/orchestrator: Fix ceph orch ls in Rook
Juan Miguel Olmo Martínez [Mon, 22 Feb 2021 13:00:43 +0000 (14:00 +0100)]
mgr/orchestrator: Fix ceph orch ls in Rook

Fixes: https://tracker.ceph.com/issues/49411
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit d070caedb5971351de9521e3125d162838270c2b)

4 years agodoc/cephadm: rewrite "install cephadm"
Zac Dover [Sun, 28 Feb 2021 12:13:39 +0000 (22:13 +1000)]
doc/cephadm: rewrite "install cephadm"

This PR breaks the "Deploying a New Ceph Cluster"
section into several sub-sections, so that each sub-section
pertains to only one subject. I've also added some explanatory
text that puts the instructions into context more than they were
before.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 1b626b989538e4e9bf7e9a8e51f6fb1169efaa04)

4 years agodoc/cephadm: rewrite "b.strap a new cluster"
Zac Dover [Mon, 1 Mar 2021 14:01:05 +0000 (00:01 +1000)]
doc/cephadm: rewrite "b.strap a new cluster"

This PR rewrites the section "Bootstrap A New
Cluster" in the Cephadm Guide, in the Install
Chapter. I've broken this section up into what
seem to me to be the topics that the content
naturally divides into.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 2ec313caba9edb71ec3973e2a8eaa0fb23f3d122)

4 years agocephadm: add docker.service dependency in systemd units
Sage Weil [Mon, 1 Mar 2021 22:42:15 +0000 (17:42 -0500)]
cephadm: add docker.service dependency in systemd units

This ensures that (1) we start after the docker daemon is running, and
(2) a restart of docker will also restart the ceph services.

Fixes: https://tracker.ceph.com/issues/46745
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0b5bc8fc2e11f1ec84be96e7a2bdbb884e1d0de3)

4 years agocephadm: add multi-digest test
Michael Fritch [Sat, 27 Feb 2021 05:16:41 +0000 (22:16 -0700)]
cephadm: add multi-digest test

quick unit test for inspect of a container with multiple digests

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 22f9712aaa0aac7c06771bf6ceba2c3ad5845b8c)

4 years agomgr/orchestrator: validate config options at apply time
Sage Weil [Tue, 23 Feb 2021 21:51:09 +0000 (16:51 -0500)]
mgr/orchestrator: validate config options at apply time

Make sure config options are valid/exist.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit deb9eae5b19e9d1ac95a183c6211a9f8ec25833c)

4 years agomgr/cephadm: disallow managed options in ServiceSpec config section
Sage Weil [Tue, 23 Feb 2021 21:50:44 +0000 (16:50 -0500)]
mgr/cephadm: disallow managed options in ServiceSpec config section

If a service is managing a config option, prevent the user from specifying
it in the spec.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 48934492df463d04c1273161db4f26567c4a70ec)

4 years agomgr/cephadm: add config section to ServiceSpec
Sage Weil [Tue, 23 Feb 2021 18:47:59 +0000 (13:47 -0500)]
mgr/cephadm: add config section to ServiceSpec

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6b852c2e013566b252ada9c33a7865a4cb62d23e)

4 years agodoc/cephadm: s/togeter/together
Zac Dover [Fri, 26 Feb 2021 12:10:51 +0000 (22:10 +1000)]
doc/cephadm: s/togeter/together

Exactly as it says in the title; this is a typo
I noticed while reading over the cephadm guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 8c191fb5f7339bd3fcefd172b29a433113efca8e)

4 years agocephadm: provide meta during bootstrap
Sage Weil [Wed, 24 Feb 2021 16:59:44 +0000 (11:59 -0500)]
cephadm: provide meta during bootstrap

This is slightly gross, but we need ctx.meta_json for the bootstrap case,
which deploys a mon and mgr.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 42d11d46b5f0b6707befd0e3fbf51926290df004)

4 years agomgr/cephadm: put service_name in unit.meta and use it when available
Sage Weil [Tue, 23 Feb 2021 17:28:27 +0000 (12:28 -0500)]
mgr/cephadm: put service_name in unit.meta and use it when available

Inferring service_name from the daemon name is error-prone.

Fixes: https://tracker.ceph.com/issues/46219
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 852ca83933bd49a5b0df0be37a2cd1574bbc2ea7)

4 years agocephadm: accept arbitrary dict via --meta-json
Sage Weil [Tue, 23 Feb 2021 17:25:37 +0000 (12:25 -0500)]
cephadm: accept arbitrary dict via --meta-json

e.g., --meta-json '{"foo": "bar"}'

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f58485efb66548f2f7acd94e448e160480775f81)

4 years agomgr/cephadm: incorporte memory_{usage,request,limit} from 'ls'
Sage Weil [Fri, 5 Feb 2021 21:48:42 +0000 (15:48 -0600)]
mgr/cephadm: incorporte memory_{usage,request,limit} from 'ls'

Keep this in our cached inventory.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ab63356f4682588017174a1590707c56f5c429d0)

4 years agocephadm: accept --memory-{request,limit}
Sage Weil [Fri, 5 Feb 2021 21:48:06 +0000 (15:48 -0600)]
cephadm: accept --memory-{request,limit}

Set a limit on the pod.  Pass both request and limit as
POD_MEMORY_REQUEST and POD_MEMORY_LIMIT, for consistency with Rook.

Store the request and limit in a new unit.meta file, stored next to
unit.run.

Report everything in unit.meta with 'ls' result.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a49bebe0d9772cf60261d95d837be52634787feb)

4 years agocephadm: include memory_usage in 'ls' output
Sage Weil [Fri, 5 Feb 2021 21:45:26 +0000 (15:45 -0600)]
cephadm: include memory_usage in 'ls' output

Report of container memory usage

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7d78d21f7c0afc4d21cdbd28125960f67b2b30c8)

4 years agodoc/cephadm: remove Orchestrator CLI from cephadm toc
Sebastian Wagner [Thu, 25 Feb 2021 15:57:07 +0000 (16:57 +0100)]
doc/cephadm: remove Orchestrator CLI from cephadm toc

not much useful information there anymore.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f4f3fee6fd1bc8e46c9f5fbc6b1bdb810a472225)

4 years agodoc/cephadm: move host labels to host mgmt
Sebastian Wagner [Thu, 25 Feb 2021 14:17:36 +0000 (15:17 +0100)]
doc/cephadm: move host labels to host mgmt

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a4aef4a49db70c310372ebc59b3ef6941333b878)

4 years agodoc/cephadm: group MDS sections into one chapter
Sebastian Wagner [Thu, 25 Feb 2021 14:14:36 +0000 (15:14 +0100)]
doc/cephadm: group MDS sections into one chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1033c46ad79795479fb75bfc8cc5b8116210d501)

4 years agodoc/cephadm: Add iscsi
Sebastian Wagner [Thu, 18 Feb 2021 17:07:44 +0000 (18:07 +0100)]
doc/cephadm: Add iscsi

Fixes https://tracker.ceph.com/issues/49488

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 46a203a5c6f4552fd4618e659e8ac7abbba0f186)

4 years agodoc/cephadm: group NFS sections into one chapter
Sebastian Wagner [Thu, 18 Feb 2021 16:36:11 +0000 (17:36 +0100)]
doc/cephadm: group NFS sections into one chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 8d07e35b2f10f8b6b6bb039e6c066ff3d1056d86)

4 years agodoc/cephadm: rename monitoring chapter title
Sebastian Wagner [Thu, 18 Feb 2021 14:48:27 +0000 (15:48 +0100)]
doc/cephadm: rename monitoring chapter title

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 769642c77d83e1c70b61e064151e92498f92c8a5)

4 years agodoc/cephadm: group MON sections into one chapter
Sebastian Wagner [Thu, 18 Feb 2021 14:39:59 +0000 (15:39 +0100)]
doc/cephadm: group MON sections into one chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b3846f6f6a4ff3daf7c26eb087992bd230ee9a47)

4 years agodoc/cephadm: make custom containers its own chapter
Sebastian Wagner [Thu, 18 Feb 2021 14:34:04 +0000 (15:34 +0100)]
doc/cephadm: make custom containers its own chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0fb2ccb49b765d589f56b355589dc811fbf66d63)

4 years agodoc/cephadm: group RGW mgmt sections into one chapter
Sebastian Wagner [Thu, 18 Feb 2021 14:17:07 +0000 (15:17 +0100)]
doc/cephadm: group RGW mgmt sections into one chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 818ed4f2025d46a0822d025c5cae5eeb161099c3)

4 years agodoc/cephadm: move scheduler topic to service mgmt
Sebastian Wagner [Thu, 18 Feb 2021 14:24:21 +0000 (15:24 +0100)]
doc/cephadm: move scheduler topic to service mgmt

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 4f3305f3bb44d9169ef74b10cb9b95e14f177137)

Conflicts:
doc/cephadm/index.rst

4 years agodoc/cephadm: move unmanaged=true to service-mgmt.rst
Sebastian Wagner [Thu, 18 Feb 2021 14:08:44 +0000 (15:08 +0100)]
doc/cephadm: move unmanaged=true to service-mgmt.rst

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 31470cfd5c04011dc69c106a921fc4dbf561ce46)

4 years agodoc/cephadm: group general service mgmt sections into one chapter
Sebastian Wagner [Thu, 18 Feb 2021 14:06:31 +0000 (15:06 +0100)]
doc/cephadm: group general service mgmt sections into one chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 11fe5ef7cf7fb93440d58fee5594fad8bd7ef32b)

4 years agodoc/cephadm: group OSD mgmt sections into one chapter
Sebastian Wagner [Thu, 18 Feb 2021 13:48:33 +0000 (14:48 +0100)]
doc/cephadm: group OSD mgmt sections into one chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 74bcd7c5f6fe1c6ddb1b01bae8e964d71b57d5a1)

4 years agodoc/cephadm: Move FQDN chapter to host mgmt.rst
Sebastian Wagner [Thu, 18 Feb 2021 14:20:59 +0000 (15:20 +0100)]
doc/cephadm: Move FQDN chapter to host mgmt.rst

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a3036d974c8c25eda6c50e6f997c826d4b8cea02)

4 years agodoc/cephadm: Move SSH config from operations to host-mgmt.rst
Sebastian Wagner [Thu, 18 Feb 2021 13:52:22 +0000 (14:52 +0100)]
doc/cephadm: Move SSH config from operations to host-mgmt.rst

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1dbac10c710ee92ed41415d2b30af58a930eecf9)

4 years agodoc/cephadm: group host mgmt sections into one chapter
Sebastian Wagner [Thu, 18 Feb 2021 12:43:09 +0000 (13:43 +0100)]
doc/cephadm: group host mgmt sections into one chapter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 88e8e91f7289b4c47056ecae1d2d198c7e7259a8)

4 years agoMerge pull request #39925 from tchaikov/pacific-github-workflow-milestone
Kefu Chai [Tue, 9 Mar 2021 09:21:03 +0000 (17:21 +0800)]
Merge pull request #39925 from tchaikov/pacific-github-workflow-milestone

pacific: .github: add workflow for adding milestone

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agocephadm: fix bug in orphan-initial-daemons logic
Daniel Pivonka [Tue, 23 Feb 2021 20:13:12 +0000 (15:13 -0500)]
cephadm: fix bug in orphan-initial-daemons logic

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 0197c7b13290e1d9b2a719ed6f2f70987c9592e3)

4 years agomgr/orch: drop __all__ from __init__.py
Sebastian Wagner [Mon, 22 Feb 2021 12:00:34 +0000 (13:00 +0100)]
mgr/orch: drop __all__ from __init__.py

`__all__` is only required if you want to modify the `form .. import *`
behavior. We don't need it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b8dbdb6530ce3ce98df42abc3825e0876cf8388a)

4 years agomgr/cephadm: add DaemonDescriptionStatus
Daniel Pivonka [Thu, 28 Jan 2021 21:36:11 +0000 (16:36 -0500)]
mgr/cephadm: add DaemonDescriptionStatus

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ae83ffc7897a0912308001f55876fd51dd56ad40)

4 years agocephadm: version command hide traceback when login is needed
Daniel Pivonka [Thu, 18 Feb 2021 21:23:23 +0000 (16:23 -0500)]
cephadm: version command hide traceback when login is needed

if image comes from a authenticated registry and login has not been done yet the traceback should be hidden and only show the error message

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 4c1c656d009230894674d33efed9c31b2bc2581d)

4 years agodoc/cephadm: troubleshooting: manually deploy MGR
Sebastian Wagner [Mon, 15 Feb 2021 13:43:31 +0000 (14:43 +0100)]
doc/cephadm: troubleshooting: manually deploy MGR

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a4179acbce3b9e71abb526a5b46436d6a3b96c82)

4 years agocephadm: fix port_in_use when IPv6 is disabled
Patrick Seidensal [Thu, 11 Feb 2021 15:22:06 +0000 (16:22 +0100)]
cephadm: fix port_in_use when IPv6 is disabled

Do not return "port is in use" when the protocol family tested is not
supported (due to being deactivated).

Fixes: https://tracker.ceph.com/issues/49273
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit a0ffcec4af7a6f464439023d3dd0ccb62c8a856c)

4 years agocephadm: Allow to use paths in all <_devices> drivegroup sections
Juan Miguel Olmo Martínez [Thu, 11 Feb 2021 12:19:07 +0000 (13:19 +0100)]
cephadm: Allow to use paths in all <_devices> drivegroup sections

This will allow to use specific device paths in any osd component.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit d02683efbfcd5aa1437ebb9e73f89db38f7e7aa6)

4 years agomgr/cephadm: error if service action called with daemonless service
Adam King [Wed, 17 Feb 2021 21:51:30 +0000 (16:51 -0500)]
mgr/cephadm: error if service action called with daemonless service

Fixes: https://tracker.ceph.com/issues/49348
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 0618c69dc5255bd407595250c6980cf4929a8ce6)

4 years agomgr/cephadm: fix up the strings reporting osd ids
Sage Weil [Wed, 17 Feb 2021 21:21:02 +0000 (15:21 -0600)]
mgr/cephadm: fix up the strings reporting osd ids

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a1ff3a9952778c1f20836b806de9fa5606432137)

4 years agomgr/cephadm: remove daemon before osd destroy/purge
Sage Weil [Wed, 17 Feb 2021 21:20:22 +0000 (15:20 -0600)]
mgr/cephadm: remove daemon before osd destroy/purge

Otherwise it doesn't work!

Drop the fullname property: it is always "osd.{self.osd_id}".

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b5eab0ddfa0bb8ae7b1a6aec4ea2e4257a01a045)

4 years agomgr/cephadm: simplify OSD __str__ for drain
Sage Weil [Wed, 17 Feb 2021 20:57:10 +0000 (14:57 -0600)]
mgr/cephadm: simplify OSD __str__ for drain

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ca4050b057296d3c62deefca0ffcb4f640b30102)

4 years agomgr/cephadm: make drain adjust crush weight if not replacing
Sage Weil [Wed, 17 Feb 2021 16:28:05 +0000 (10:28 -0600)]
mgr/cephadm: make drain adjust crush weight if not replacing

If we are replacing an OSD, we should mark it out and then back in
again when a new device shows up.  However, if we are going to
destroy an OSD, we should just weight it to 0 in crush, so that data
doesn't move again once the OSD is purged.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4fc1309f281356db0a074da22aa6f2daa034df8d)

4 years agomgr/cephadm: less log noise from osd drain code
Sage Weil [Wed, 17 Feb 2021 20:26:14 +0000 (14:26 -0600)]
mgr/cephadm: less log noise from osd drain code

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e2f0e56ddf3197f220c5a43c79d6bc43e4b135ce)

4 years agomgr/cephadm: fix 'orch daemon add osd ...'
Sage Weil [Wed, 17 Feb 2021 20:31:57 +0000 (14:31 -0600)]
mgr/cephadm: fix 'orch daemon add osd ...'

When adding an osd daemon explicitly, there is no created timestamp
for the spec, and we should never not apply it.

Fixes: b129c1312113f56a227caeb535f656f5a090a85f
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e8643275e5d92af9539e60a7a80ef13d0f27af64)

4 years agomgr/cephadm/upgrade: fix typo
Sage Weil [Mon, 22 Feb 2021 23:07:26 +0000 (18:07 -0500)]
mgr/cephadm/upgrade: fix typo

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8e05d9759b607582deaed872d4e4d61c1cc30689)

4 years agomgr/cephadm: remove spec from CephadmDaemonDeploySpec
Sebastian Wagner [Tue, 16 Feb 2021 15:54:11 +0000 (16:54 +0100)]
mgr/cephadm: remove spec from CephadmDaemonDeploySpec

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d8bcaac10a1a7cdc8689375f37f5d59d86148314)

4 years agomgr/cephadm/upgrade: restart mgr after mons upgrade to pacific
Sage Weil [Wed, 17 Feb 2021 17:39:25 +0000 (12:39 -0500)]
mgr/cephadm/upgrade: restart mgr after mons upgrade to pacific

If we are upgrading to pacific, we need to restart the mgr *after* the
mons have a pacific quorum so that they can get the kv subscriptions.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4d7dd0012962202532bde4fbc47554186c95b9fb)

4 years agomgr/cephadm: use get_foreign_ceph_option() instead of 'config get' mon command
Sage Weil [Tue, 16 Feb 2021 23:05:37 +0000 (18:05 -0500)]
mgr/cephadm: use get_foreign_ceph_option() instead of 'config get' mon command

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9b5cbf40f8328060d7c6d04a474ee87d5bf03ca3)