]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agocephadm: allow pulling from insecure registries
Joao Eduardo Luis [Mon, 11 Oct 2021 17:20:47 +0000 (17:20 +0000)]
cephadm: allow pulling from insecure registries

Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit 0a86107dccfdffa7476c61312f0c3b5aae7da7b7)

3 years agomgr/cephadm: test: Move test_monitoring_ports to TestMonitoring
Sebastian Wagner [Mon, 18 Oct 2021 13:20:48 +0000 (15:20 +0200)]
mgr/cephadm: test: Move test_monitoring_ports to TestMonitoring

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit c1c1e961f677ea10f782cab412e8b845e100266b)

Conflicts:
    src/pybind/mgr/cephadm/tests/test_cephadm.py

3 years agomgr/cephadm: Monitoring: Verify config file generation
Sebastian Wagner [Mon, 18 Oct 2021 13:06:20 +0000 (15:06 +0200)]
mgr/cephadm: Monitoring: Verify config file generation

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit f4c154463365f7df5dcd9053e59de65f53bf0f5f)

Conflicts:
src/pybind/mgr/cephadm/tests/test_services.py

3 years agomgr/cephadm: Monitoring: Prometheus: Fix IPv6
Sebastian Wagner [Mon, 18 Oct 2021 13:05:43 +0000 (15:05 +0200)]
mgr/cephadm: Monitoring: Prometheus: Fix IPv6

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 406734a14226800f2c2ed01290a508e89e0dec6e)

3 years agomgr/cephadm: Monitoring: Alertmanager: Fix IPv6
Sebastian Wagner [Mon, 18 Oct 2021 13:05:27 +0000 (15:05 +0200)]
mgr/cephadm: Monitoring: Alertmanager: Fix IPv6

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d65ccf3a0f83fc4a79edfe46c9468cf77f655f83)

3 years agomgr/cephadm: Monitoring: Grafana: Fix IPv6
Sebastian Wagner [Mon, 18 Oct 2021 13:04:46 +0000 (15:04 +0200)]
mgr/cephadm: Monitoring: Grafana: Fix IPv6

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit bb2cb3ea3a9704cf124b5342fb2a974e8618cc01)

Conflicts:
src/pybind/mgr/cephadm/tests/test_ssh.py

3 years agodoc/cephadm: update typo in install.rst
Zac Dover [Wed, 20 Oct 2021 14:42:02 +0000 (00:42 +1000)]
doc/cephadm: update typo in install.rst

This PR changes the string

   use "--log-to-file" option during bootstrap

to

   use the "--log-to-file" option during bootstrap

The way that it was written makes it sound like it was
written by a native Russian speaker. But it was written
by me, a native English speaker, so this is on me.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit d5f6d8ef33413906c8a53a1196ad37a1e1e80289)

3 years agoFix option name osd_crush_chooseleaf_type
krafZLorG [Thu, 21 Oct 2021 07:27:14 +0000 (10:27 +0300)]
Fix option name osd_crush_chooseleaf_type

Signed-off-by: Dmitry Kvashnin <dm.kvashnin@gmail.com>
(cherry picked from commit 0e7c16ce2c94facb7b3979722d1ce50d28d4452f)

3 years agomgr/cephadm: update host when adding host that already exists
Daniel Pivonka [Thu, 12 Aug 2021 18:25:23 +0000 (14:25 -0400)]
mgr/cephadm: update host when adding host that already exists

update addr and labels of existing host
this prevents labels being removed if a host is readded
or addr being set to 127.0.... if readding host active mgr is on

Fixes: https://tracker.ceph.com/issues/51667
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 0facfac91fd8f71e5a8b869d818e7c2b07b93516)

3 years agomgr/cephadm: put list_servers output in debug logs
Adam King [Thu, 30 Sep 2021 12:34:58 +0000 (08:34 -0400)]
mgr/cephadm: put list_servers output in debug logs

Over time, seen a number of cases where daemons are being
marked stray for unknown reasons and it is always difficult
to debug due to the difficulty of finding what list_servers
is outputting. I think it would help to include this in the
debug logs. That way we have that side of the info available
and can easily compare it to daemon/host info cephadm is storing.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit bdc802354554ea9c5e7c0377e3fe695f719921a7)

3 years agocephadm: do not log commands sent to "call" function
Adam King [Thu, 14 Oct 2021 18:56:51 +0000 (14:56 -0400)]
cephadm: do not log commands sent to "call" function

They could potentially log sensitive info such as registry
login passwords

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 3205a835f6bfa1cea4415dc8a95888ac7dade181)

3 years agomgr/orchestrator: clean up 'orch device ls' output
Sage Weil [Tue, 3 Aug 2021 17:25:26 +0000 (13:25 -0400)]
mgr/orchestrator: clean up 'orch device ls' output

- headings in CAPS
- DEVICE ID, not just serial or model/vendor
- drop the health, LED fields from non-wide view
- sort devices
- 'Yes' or '', for easier visual parsing

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c2f82d7228a6ebbca9500f66fc118e0f67a0528a)

3 years agoqa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
Sage Weil [Fri, 15 Oct 2021 15:58:41 +0000 (11:58 -0400)]
qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8900775841c7950ec18c9963bdcb38c2e43cc2f5)

3 years agomgr/cephadm: make upgrade ls output structured
Sage Weil [Fri, 15 Oct 2021 15:09:07 +0000 (11:09 -0400)]
mgr/cephadm: make upgrade ls output structured

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit be3fd05fc59bd523c1171d18cb0a0784f6a6d5c1)

3 years agomgr/cephadm: add 'orch upgrade ls' to list available versions
Sage Weil [Tue, 12 Oct 2021 19:22:14 +0000 (15:22 -0400)]
mgr/cephadm: add 'orch upgrade ls' to list available versions

Optionally pass image name to query an alternative image/registry, or
--tags to list tags instead of ceph versions.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7f750fe31aa62f0055e43e86d4461d48baaaa7d3)

3 years agodoc/cephadm: Hosts: Add "initial CRUSH location"
Sebastian Wagner [Fri, 15 Oct 2021 11:13:02 +0000 (13:13 +0200)]
doc/cephadm: Hosts: Add "initial CRUSH location"

Followup on d2a9a35993

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8e4f9ea46ceec267748d60c6ed009b209d2352ab)

3 years agodoc/cephadm: update example of custom container spec file
Sebastian Wagner [Wed, 29 Sep 2021 10:02:28 +0000 (12:02 +0200)]
doc/cephadm: update example of custom container spec file

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit ce73047d25599b247ab889614fa62684b0c7aca0)

3 years agodoc/cephadm: use sphinx autoclass to document IscsiServiceSpec
Sebastian Wagner [Wed, 29 Sep 2021 10:00:42 +0000 (12:00 +0200)]
doc/cephadm: use sphinx autoclass to document IscsiServiceSpec

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 2e7b369373236c32e429463512521da1fc7e159d)

3 years agodoc/cephadm: use sphinx autoclass to document RGWSpec
Sebastian Wagner [Tue, 28 Sep 2021 14:57:41 +0000 (16:57 +0200)]
doc/cephadm: use sphinx autoclass to document RGWSpec

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8ef77a0bbcef710185c7ee6452d6ca8a133843c1)

3 years agomgr/cephadm: the display of 'orch osd rm status' is incorrect
jianglong01 [Tue, 7 Sep 2021 07:49:04 +0000 (15:49 +0800)]
mgr/cephadm: the display of 'orch osd rm status' is incorrect

After exec 'ceph orch osd rm --replace', 'orch osd rm status' has an wrong display.

wrong display:
OSD_ID  HOST                              STATE     PG_COUNT  REPLACE  FORCE  DRAIN_STARTED_AT
15      cephqa08.cpp.zzbm.qianxin-inc.cn  draining  0         True     True   2021-09-07 07:54:05.776906

correct display:
OSD_ID  HOST                              STATE     PG_COUNT  REPLACE  FORCE  DRAIN_STARTED_AT
15      cephqa08.cpp.zzbm.qianxin-inc.cn  draining  0         True     False  2021-09-07 07:35:34.731417

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
(cherry picked from commit 465f7e8fa34b37cf09e4d280da5239086924b942)

3 years agocephadm: add a test on 'ceph orch ls'
Guillaume Abrioux [Wed, 29 Sep 2021 06:42:33 +0000 (08:42 +0200)]
cephadm: add a test on 'ceph orch ls'

This adds a unittest in order to cover the output of 'orch ls' command
and check the reporting about the running osd number is displayed as
expected.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Co-Authored-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 1d848dc447fd4df18893903a053d58129fb8c13a)

3 years agocephadm: fix osd size reported in 'orch ls'
Guillaume Abrioux [Wed, 22 Sep 2021 07:35:54 +0000 (09:35 +0200)]
cephadm: fix osd size reported in 'orch ls'

The current code enforces ('running osds' + 1), see [1].
Let's simply print the number of running osds instead of a
meaningless calculation.

[1] https://github.com/ceph/ceph/commit/1c0a0a6f6153dc555085f263f161666cd171964a#diff-4f2fb7d330e74b64ac41457b7c7a723cd78db86433e0b0c398874531e5a7e39eR1852-R1859

Fixes: https://tracker.ceph.com/issues/50928
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 97219591dab8f54f239ccfb1891c45b01e7bec79)

3 years agomgr/cephadm: use json-pretty for ceph-volume calls
Adam King [Wed, 29 Sep 2021 02:10:07 +0000 (22:10 -0400)]
mgr/cephadm: use json-pretty for ceph-volume calls

To avoid a single line json string that is too long
and causes a chunk longer than limit error

Fixes: https://tracker.ceph.com/issues/52745
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit b20637714e366b44ec604a76bab9095d5f4a1ef2)

3 years agodoc/cephadm: document setting ssh-user during adopt
Daniel Pivonka [Mon, 11 Oct 2021 21:15:35 +0000 (17:15 -0400)]
doc/cephadm: document setting ssh-user during adopt

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit bafbeb6fca21374aebace7ae65b964c875d87d01)

3 years agodoc/cephadm: note that replacment osd needs to be on same host as removed osd
Daniel Pivonka [Thu, 21 Oct 2021 17:05:05 +0000 (13:05 -0400)]
doc/cephadm: note that replacment osd needs to be on same host as removed osd

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 441869220c97d739cbe2186a9e4ee0d8a2911f1d)

3 years agomgr/cephadm: set health check warning in `_apply_service_config` for invalid config...
Melissa Li [Thu, 5 Aug 2021 18:28:35 +0000 (14:28 -0400)]
mgr/cephadm: set health check warning in `_apply_service_config` for invalid config options and failures to set options

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit 0f8f085058dc3319c073fb50491d1862e45ba9a2)

3 years agomgr/cephadm: refactor `_check_for_strays` to use `remove_health_warning` helper
Melissa Li [Tue, 3 Aug 2021 17:50:26 +0000 (13:50 -0400)]
mgr/cephadm: refactor `_check_for_strays` to use `remove_health_warning` helper

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit f42992a1ef8f7827155e377013cef6c9a8cba15b)

3 years agomgr/cephadm: refactor `_refresh_hosts_and_daemons` to use `set_health_warning` and...
Melissa Li [Tue, 3 Aug 2021 17:46:21 +0000 (13:46 -0400)]
mgr/cephadm: refactor `_refresh_hosts_and_daemons` to use `set_health_warning` and `remove_health_warning` helpers

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit 2eacd6e7e936636b2eee82bd03ee59f67431cd2e)

3 years agomgr/cephadm: refactor `_update_paused_health` to use `set_health_warning` and `remove...
Melissa Li [Tue, 3 Aug 2021 17:40:39 +0000 (13:40 -0400)]
mgr/cephadm: refactor `_update_paused_health` to use `set_health_warning` and `remove_health_warning` helpers

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit cfd6ad0af2feac3a7611067dd7e88fed1b315023)

3 years agomgr/cephadm: refactor `_set_maintenance_health_check` to use `set_health_warning...
Melissa Li [Tue, 3 Aug 2021 17:34:06 +0000 (13:34 -0400)]
mgr/cephadm: refactor `_set_maintenance_health_check` to use `set_health_warning` helper

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit c3f8fba04a629f2f3e70d0f5443a42de54426bf2)

3 years agomgr/cephadm: add `set_health_warning` and `remove_health_warning` helpers in module.py
Melissa Li [Tue, 3 Aug 2021 17:28:40 +0000 (13:28 -0400)]
mgr/cephadm: add `set_health_warning` and `remove_health_warning` helpers in module.py

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit 6044afb2cead4d54c19cb6052c90afd56c28e0d0)

3 years agocephadm: bootstrap --apply-spec shouldn't enforce :z
Guillaume Abrioux [Thu, 7 Oct 2021 14:23:07 +0000 (16:23 +0200)]
cephadm: bootstrap --apply-spec shouldn't enforce :z

There's no need to mount this file with the `:z` option.
read-only access is enough for this file.

Fixes: https://tracker.ceph.com/issues/52855
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit edc07019c6e70950d697ec0d7f3756065ac64b92)

3 years agocephadm: shell --mount shouldnt enforce ':z' option
Guillaume Abrioux [Thu, 7 Oct 2021 08:49:25 +0000 (10:49 +0200)]
cephadm: shell --mount shouldnt enforce ':z' option

cephadm shouldn't enforce this option.
For instance, it can be an issue when you try to bindmount a file in /usr

Fixes: https://tracker.ceph.com/issues/52853
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit aeae360e2595f348f10ddc36cdb270b018f7eb02)

3 years agosrc/cephadm: Add mariner as a supported distro.
Roaa Sakr [Fri, 1 Oct 2021 22:40:11 +0000 (15:40 -0700)]
src/cephadm: Add mariner as a supported distro.

Using cephadm install commands on Mariner will result in an error
"Distro not supported" though the packages are supported and can be
installed manually. This patch adds mariner as a supported distro.
Fix validated on mariner.

Signed-off-by: Roaa Sakr <romoh@microsoft.com>
(cherry picked from commit 1612427ea549d3fc259c8eb0e37953a9323dcb31)

3 years agocephadm: add daemon_name in daemon description
Guillaume Abrioux [Thu, 20 May 2021 14:06:05 +0000 (16:06 +0200)]
cephadm: add daemon_name in daemon description

This adds the daemon_name in json output when asking for daemon
description.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 19ee16a46b90d92f1a49ace5ba1a3c47eb8c1403)

3 years agodoc/cephadm/services: unindent note section
Kefu Chai [Wed, 29 Sep 2021 23:22:09 +0000 (07:22 +0800)]
doc/cephadm/services: unindent note section

before this change the note on "apply" command is embedded in the note
on "_no_schedule". and they are not related. so let's move the former
out. also, highlight the yaml file sample in YAML.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 48f5cd5b9bb1ccf89e1eaad163cf11874ad3e176)

3 years agomgr/cephadm: populate trusted_ip_list in iscsi-gateway.cfg with mgr ips
Daniel Pivonka [Tue, 21 Sep 2021 20:04:51 +0000 (16:04 -0400)]
mgr/cephadm: populate trusted_ip_list in iscsi-gateway.cfg with mgr ips

Iscsi gateways do not show "UP" in dashboard without this

fixes: https://tracker.ceph.com/issues/52692
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit cda82c98a32f51cb392fc51ba854bcae409567f8)

Conflicts:
src/pybind/mgr/cephadm/module.py

3 years agocephadm/iscsi: fix a typo
Guillaume Abrioux [Mon, 27 Sep 2021 06:57:35 +0000 (08:57 +0200)]
cephadm/iscsi: fix a typo

This commit fixes a typo in the comment of unit.run of iscsi service.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5a4f29de7a6080b0476ad134426faf94c01fb316)

3 years agocephadm: fix iscsi test
Guillaume Abrioux [Mon, 27 Sep 2021 15:22:48 +0000 (17:22 +0200)]
cephadm: fix iscsi test

the current test is wrong because it generates the tcmu-runner part two
times.
given the function `deploy_daemon_units()` in cephadm already writes a
first time the tcmu-runner command, calling a second time
`get_tcmu_runner_container()` from the test makes `deploy_daemon_units()`
write the same command again.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7e9762e0ce36d773c34fc9353d2eacc694554f39)

3 years agocephadm: modify iscsi containers bindmount
Guillaume Abrioux [Mon, 27 Sep 2021 11:46:01 +0000 (13:46 +0200)]
cephadm: modify iscsi containers bindmount

tcmu-runner logs in `/var/log/tcmu-runner.log`, there's no option to
make it log to stdout/stderr so the log is only available from the
container.
Modifying the bindmount from `-v /var/log/ceph/<fsid>/:/var/log/rbd-target-api:z`
to `-v /var/log/ceph/<fsid>/:/var/log:z` makes it at least available
from the host.

Fixes: https://tracker.ceph.com/issues/47480
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b130214195ff84af57f447cf3c3410f0918becaf)

3 years agodoc/cephadm: move services into services/
Sebastian Wagner [Mon, 13 Sep 2021 15:15:33 +0000 (17:15 +0200)]
doc/cephadm: move services into services/

This is going to clean up the toctree a bit.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8c70398949f773d8b992ad8ae6c71383460d2932)

3 years agodoc/cephadm: move service-management to services/index.rst
Sebastian Wagner [Mon, 13 Sep 2021 15:12:01 +0000 (17:12 +0200)]
doc/cephadm: move service-management to services/index.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 5e25b19cc7bd5845eccdd0bbb3d6ee8a5852563d)

3 years agodoc/cephadm: Add `networks` to spec documentation
Sebastian Wagner [Tue, 14 Sep 2021 14:07:14 +0000 (16:07 +0200)]
doc/cephadm: Add `networks` to spec documentation

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit fd32c1651b95ac14662bb5048e42f92fc1f45e56)

3 years agodoc/cephadm: add mgr.rst
Sebastian Wagner [Tue, 14 Sep 2021 14:15:29 +0000 (16:15 +0200)]
doc/cephadm: add mgr.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 6773cb01b8f42a4a37b4360d7a032524df39a6f8)

3 years agodoc/cephadm: Add `Specifying Networks` to rgw.rst
Sebastian Wagner [Tue, 14 Sep 2021 14:08:11 +0000 (16:08 +0200)]
doc/cephadm: Add `Specifying Networks` to rgw.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 4243a7936a9972a1d9c677207ad6a4e56908e7af)

3 years agomgr/dashboard: More robust handling of timedelta() representation
Brad Hubbard [Fri, 24 Sep 2021 02:22:42 +0000 (12:22 +1000)]
mgr/dashboard: More robust handling of timedelta() representation

The representation of timedelta() appears to have changed recently so
we should try and be more accommodating in that regard.

Fixes: https://tracker.ceph.com/issues/52696
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit ad1a4f2410393cab6ac728c4ddb79d111e252559)

3 years agocephadm: Fix TestMaintenance trying to list /var/lib/ceph
Sebastian Wagner [Fri, 24 Sep 2021 10:55:03 +0000 (12:55 +0200)]
cephadm: Fix TestMaintenance trying to list /var/lib/ceph

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 767564864f15ea8afe9b925345cdf59db1815cb0)

3 years agocephadm: TestCheckHost: also mock `check_time_sync`
Sebastian Wagner [Fri, 24 Sep 2021 10:46:54 +0000 (12:46 +0200)]
cephadm: TestCheckHost: also mock `check_time_sync`

Fixes: https://tracker.ceph.com/issues/52722
```
TestCheckHost.test_container_engine fails at cephadm:5834: Error cephadm.Error: No time synchronization is active
```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit fc9d9b39144fd16f2b2a9621f80dcd1162459000)

3 years agomgr/cephadm: base maintenance enter/exit success off of return message
Adam King [Wed, 22 Sep 2021 21:43:43 +0000 (17:43 -0400)]
mgr/cephadm: base maintenance enter/exit success off of return message

rather than on whether there was any stdout from the command

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 393330bae6b259102bba83b0d3c6213970838c02)

3 years agomgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure...
Adam King [Wed, 22 Sep 2021 21:42:41 +0000 (17:42 -0400)]
mgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure messages

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 5350d58b9ad45359b7885d19592c2b4826e9aed7)

Conflicts:
src/pybind/mgr/cephadm/tests/test_cephadm.py

3 years agocephadm: unit tests for maitenance mode return values
Adam King [Wed, 22 Sep 2021 18:34:12 +0000 (14:34 -0400)]
cephadm: unit tests for maitenance mode return values

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 73e6aa8d9ca69c96361fae6998dd71d252808aa4)

3 years agocephadm: fix mypy complaints for ThreadedChildWatcher class
Adam King [Tue, 21 Sep 2021 18:00:11 +0000 (14:00 -0400)]
cephadm: fix mypy complaints for ThreadedChildWatcher class

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 8bdddfa02eea7331971c16a9f8806b8451f09c5b)

3 years agocephadm: fix exiting maintenance when systemd target doesn't exist
Adam King [Mon, 20 Sep 2021 12:02:44 +0000 (08:02 -0400)]
cephadm: fix exiting maintenance when systemd target doesn't exist

If the systemd target doesn't exist we need to just bypass enabling
it and return success or the host will just be stuck in maitnenance
mode.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 3a15f1dc81dfd5f963e1bf36316fa5b289a898b0)

3 years agocephadm: raise error during `pull` failure
Michael Fritch [Wed, 8 Sep 2021 23:16:31 +0000 (17:16 -0600)]
cephadm: raise error during `pull` failure

instead of a traceback to the console

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit d07e72c4b0bdc4d74cf1a0d8826061942f757d08)

3 years agomgr/cephadm: osd should not be zap when it is running
jianglong01 [Mon, 6 Sep 2021 10:37:06 +0000 (18:37 +0800)]
mgr/cephadm: osd should not be zap when it is running

When osd is running, we exec the command of "ceph orch device zap".
Then ceph-volume will exec dd command which cause the osd of lvs to be destroyed.
If the osd of systemctl unit restarts, it will be unable to boot.

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
(cherry picked from commit aed472f7b6663c487a5ae89fcdd788b32910b621)

3 years agodoc: from v16.2.6 you have to point to the quay.io image if using docker
Javier Cacheiro [Tue, 21 Sep 2021 09:37:42 +0000 (11:37 +0200)]
doc: from v16.2.6 you have to point to the quay.io image if using docker

New images are no longer uploaded to the Docker Hub registry, so Docker users should point the upgrade
command to the quay.io image.

Fixes: https://tracker.ceph.com/issues/52683
Signed-off-by: Javier Cacheiro <javier.cacheiro.lopez@cesga.es>
(cherry picked from commit 961a4fcffa9bae348810a8542c08ca38947ed111)

3 years agodoc/ceph-volume/activate: Users should call cephadm instead
Sebastian Wagner [Sat, 11 Sep 2021 17:27:23 +0000 (19:27 +0200)]
doc/ceph-volume/activate: Users should call cephadm instead

Fixes: https://tracker.ceph.com/issues/52575
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 925820e73cfeae290900d2077cc792e9085b5767)

3 years agodoc/cephadm/service-management: Move Deployment of Daemons
Sebastian Wagner [Tue, 14 Sep 2021 14:43:17 +0000 (16:43 +0200)]
doc/cephadm/service-management: Move Deployment of Daemons

.. into "Daemon Placement"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit b68c99cd739d8f92784a3398e238e3ed6903046f)

3 years agodoc/cephadm/service-management: Move "Updating Service Specifications"
Sebastian Wagner [Tue, 14 Sep 2021 14:45:54 +0000 (16:45 +0200)]
doc/cephadm/service-management: Move "Updating Service Specifications"

into "Service Specification"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 80f33a4e7df949759f9ac02c50b3bf99d03678e1)

3 years agocephadm: skip podman check during `rm-repo`
Michael Fritch [Fri, 10 Sep 2021 13:38:48 +0000 (07:38 -0600)]
cephadm: skip podman check during `rm-repo`

allow the `rm-repo` command to succeed when podman is not installed

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fd977773a57e12003fb02bdc762bf6bc89d785a1)

3 years agodoc/cephadm: Removing a service
Sebastian Wagner [Sat, 11 Sep 2021 17:15:38 +0000 (19:15 +0200)]
doc/cephadm: Removing a service

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 7af138e089bf0972a2067f84fe9dd6cd4588e7f8)

3 years agodoc/cephadm: Add lots of links to other chapters
Sebastian Wagner [Sat, 11 Sep 2021 18:02:44 +0000 (20:02 +0200)]
doc/cephadm: Add lots of links to other chapters

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d9ec8eb7a8da3c7dff40d7ed89feaebf7cadd37d)

3 years agocephadm: show podman version during `check-host`
Michael Fritch [Mon, 23 Aug 2021 13:47:56 +0000 (07:47 -0600)]
cephadm: show podman version during `check-host`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 44aee33945f285ed4366b960e9526ed9d1984382)

3 years agocephadm: avoid unhandled `AttributeError`
Michael Fritch [Thu, 19 Aug 2021 20:06:32 +0000 (14:06 -0600)]
cephadm: avoid unhandled `AttributeError`

when docker/podman are not present

Fixes: https://tracker.ceph.com/issues/51818
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4d5694a9f0977a22c2a6dac680d594ab3feb070b)

3 years agomgr/cephadm: show unhandled exceptions during `host add`
Michael Fritch [Thu, 19 Aug 2021 21:21:06 +0000 (15:21 -0600)]
mgr/cephadm: show unhandled exceptions during `host add`

138700e59bcd assumes stderr will always have a line containing the
prefix 'ERROR', which leads to an empty error reason when `check-host`
fails with an unhandled exception

Fixes: https://tracker.ceph.com/issues/51818
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit dac9225085a1f6d2eeaf209fc3d77c54208db2e8)

3 years agomgr/cephadm: Add OSDService.post_remove()
Sebastian Wagner [Tue, 31 Aug 2021 09:38:14 +0000 (11:38 +0200)]
mgr/cephadm: Add OSDService.post_remove()

Do not remove the osd.N keyring, if we failed to deploy the OSD, because
we cannot recover from it. The OSD keys are created by ceph-volume and not by
us.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d7a4c5794034e60e94dd12951f7dbf4685647686)

3 years agomgr/cephadm: Add MonService.post_remove()
Sebastian Wagner [Tue, 31 Aug 2021 09:01:11 +0000 (11:01 +0200)]
mgr/cephadm: Add MonService.post_remove()

We should never remove the mon keyring. Let's move
this piece of code into the MonService class

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 71eaf35aa755735574f8bc53b38fa1bac550792c)

3 years agocephadm: (re)add command argv logging
Michael Fritch [Mon, 30 Aug 2021 15:40:55 +0000 (09:40 -0600)]
cephadm: (re)add command argv logging

introduced by 81a7df0498d and inadvertently removed by 3afec2ab30c

Fixes: https://tracker.ceph.com/issues/52484
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6d18759bcb75c68c3a2d421e5d39c6cee8c18526)

3 years agocephadm: add thread ident to log messages
Michael Fritch [Mon, 30 Aug 2021 15:18:15 +0000 (09:18 -0600)]
cephadm: add thread ident to log messages

can be used to filter msgs from a specific cephadm command

Fixes: https://tracker.ceph.com/issues/52484
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 2f1482890bad99cf29623585c6e4f8abf15cecc5)

3 years agoqa/distros: Remove stale kubic distros
Sebastian Wagner [Fri, 3 Sep 2021 08:13:54 +0000 (10:13 +0200)]
qa/distros: Remove stale kubic distros

Cause they're broken

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 65e2cc084073ea7e05ecbc06f5773617676708ff)

3 years agoqa/distros/podman: Add rhel_8.rhel_8.4_container_tools_3.0.yaml
Sebastian Wagner [Thu, 2 Sep 2021 09:48:13 +0000 (11:48 +0200)]
qa/distros/podman: Add rhel_8.rhel_8.4_container_tools_3.0.yaml

mainly for the cephfs suite

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 0293580b502da3dc874430861c6cfac976403a67)

3 years agodoc/cephadm: monitoring: Further Reading
Sebastian Wagner [Mon, 30 Aug 2021 11:14:30 +0000 (13:14 +0200)]
doc/cephadm: monitoring: Further Reading

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit dc1180c485f91f86df0194b0769234d51a450816)

3 years ago.github/labeler: Add monitoring
Sebastian Wagner [Mon, 30 Aug 2021 10:45:56 +0000 (12:45 +0200)]
.github/labeler: Add monitoring

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 5624a62024ef64209d16d619e4a610870a244f37)

3 years agodoc/cephadm: monitoring: Add "Adding Alertmanager webhooks"
Sebastian Wagner [Mon, 30 Aug 2021 10:37:26 +0000 (12:37 +0200)]
doc/cephadm: monitoring: Add "Adding Alertmanager webhooks"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 90f4cc017a49859224f0817fefc31ae459f5deec)

3 years agodoc/cephadm: monitoring: Add "Setting up Grafana"
Sebastian Wagner [Mon, 30 Aug 2021 10:26:23 +0000 (12:26 +0200)]
doc/cephadm: monitoring: Add "Setting up Grafana"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d17613086697252f31b57cddef1813b3dc2625d8)

3 years agodoc/cephadm: monitoring: move "deploying w/o" up
Sebastian Wagner [Mon, 30 Aug 2021 10:23:13 +0000 (12:23 +0200)]
doc/cephadm: monitoring: move "deploying w/o" up

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 5aa5fea8ee3e57e7636627031f651113a1d31cb4)

3 years agodoc/cephadm: monitoring: default placements
Sebastian Wagner [Mon, 30 Aug 2021 10:20:53 +0000 (12:20 +0200)]
doc/cephadm: monitoring: default placements

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit efd79a4adcfefc887bb42f1dc6882eec18576c1d)

3 years agoMerge pull request #43748 from tchaikov/pacific-doc-build
Sebastian Wagner [Mon, 1 Nov 2021 13:38:31 +0000 (14:38 +0100)]
Merge pull request #43748 from tchaikov/pacific-doc-build

pacific: admin/doc-requirements.txt: pin Sphinx at 3.5.4

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoadmin/doc-requirements.txt: pin Sphinx at 3.5.4 43748/head
Kefu Chai [Sat, 30 Oct 2021 03:18:17 +0000 (11:18 +0800)]
admin/doc-requirements.txt: pin Sphinx at 3.5.4

* pin Sphinx at 3.5.4
* pin docutils at 0.18

at least the combination of these two versions
is known to compile.

to address the bug reported at
https://sourceforge.net/p/docutils/bugs/431/

the backtrace looks like:

/home/jenkins-build/build/workspace/ceph-pr-docs/build-doc/virtualenv/lib/python3.8/site-packages/sphinx/util/docutils.py:285:
RemovedInSphinx30Warning: function based directive support is now
deprecated. Use class based directive instead.
  warnings.warn('function based directive support is now deprecated. '

Exception occurred:
  File
"/home/jenkins-build/build/workspace/ceph-pr-docs/build-doc/virtualenv/lib/python3.8/site-packages/docutils/writers/html5_polyglot/__init__.py",
line 445, in section_title_tags
    if (ids and self.settings.section_self_link
AttributeError: 'Values' object has no attribute 'section_self_link'

please note this change is not cherry-picked from
master, because master already bumped Sphinx to 3.5.4
in 4968baa2523bd2a5ca6be147b26bc28906a864c9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
3 years agoMerge pull request #43543 from rhcs-dashboard/wip-52870-pacific
Yuri Weinstein [Thu, 28 Oct 2021 20:02:43 +0000 (13:02 -0700)]
Merge pull request #43543 from rhcs-dashboard/wip-52870-pacific

pacific: mgr/dashboard: clean-up controllers and API backward versioning compatibility

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
3 years agoMerge pull request #43417 from trociny/wip-51646-pacific
Yuri Weinstein [Wed, 27 Oct 2021 13:16:54 +0000 (06:16 -0700)]
Merge pull request #43417 from trociny/wip-51646-pacific

pacific: osd/OSD: mkfs need wait for transcation completely finish

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #43562 from lxbsz/vino_fix
Yuri Weinstein [Wed, 27 Oct 2021 13:15:50 +0000 (06:15 -0700)]
Merge pull request #43562 from lxbsz/vino_fix

Pacific: test/libcephfs: put inodes after lookup

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43559 from batrick/i52654-pacific
Yuri Weinstein [Wed, 27 Oct 2021 13:14:59 +0000 (06:14 -0700)]
Merge pull request #43559 from batrick/i52654-pacific

pacific: pybind/mgr/cephadm: set allow_standby_replay during CephFS upgrade

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
3 years agoMerge pull request #43475 from lxbsz/tracker_52876
Yuri Weinstein [Wed, 27 Oct 2021 13:13:26 +0000 (06:13 -0700)]
Merge pull request #43475 from lxbsz/tracker_52876

pacific: test: shutdown the mounter after test finishes

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43644 from aaSharma14/wip-52965-pacific
Ernesto Puerta [Wed, 27 Oct 2021 10:23:54 +0000 (12:23 +0200)]
Merge pull request #43644 from aaSharma14/wip-52965-pacific

pacific: mgr/dashboard: monitoring: grafonnet refactoring for radosgw dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #43619 from smithfarm/wip-53005-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:38:12 +0000 (13:38 -0700)]
Merge pull request #43619 from smithfarm/wip-53005-pacific

pacific: rgw/tracing: unify SO version numbers within librgw2 package

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43512 from neha-ojha/wip-52770-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:29:56 +0000 (13:29 -0700)]
Merge pull request #43512 from neha-ojha/wip-52770-pacific

pacific: os/bluestore: list obj which equals to pend

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
3 years agoMerge pull request #43513 from neha-ojha/wip-52620-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:29:11 +0000 (13:29 -0700)]
Merge pull request #43513 from neha-ojha/wip-52620-pacific

pacific: osd: fix partial recovery become whole object recovery after restart osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #43511 from neha-ojha/wip-52843-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:28:38 +0000 (13:28 -0700)]
Merge pull request #43511 from neha-ojha/wip-52843-pacific

pacific: msg/async/ProtocolV2: Set the recv_stamp at the beginning of receiving a message

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
3 years agoMerge pull request #43445 from k0ste/wip-52848-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:27:30 +0000 (13:27 -0700)]
Merge pull request #43445 from k0ste/wip-52848-pacific

pacific: mgr: Add check to prevent mgr from crashing

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agoMerge pull request #43437 from trociny/wip-52831-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:26:46 +0000 (13:26 -0700)]
Merge pull request #43437 from trociny/wip-52831-pacific

pacific: osd: re-cache peer_bytes on every peering state activate

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43421 from callithea/wip-52289-pacific
Yuri Weinstein [Tue, 26 Oct 2021 20:26:16 +0000 (13:26 -0700)]
Merge pull request #43421 from callithea/wip-52289-pacific

pacific: qa/tasks/mgr: skip test_diskprediction_local on python>=3.8

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43353 from kamoltat/wip-ksirivad-backport-pacific-37544
Yuri Weinstein [Tue, 26 Oct 2021 20:24:37 +0000 (13:24 -0700)]
Merge pull request #43353 from kamoltat/wip-ksirivad-backport-pacific-37544

pacific: mgr/progress: optimize global recovery && introduce 5 seconds interval

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agomgr/dashboard: monitoring: grafonnet refactoring for hosts dashboards 43644/head
Aashish Sharma [Fri, 8 Oct 2021 10:03:13 +0000 (15:33 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for hosts dashboards

This PR intends to refactor hosts dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit f7714de294dd7376a9a8ae5131aa429322b459c3)

Conflicts:
monitoring/grafana/dashboards/jsonnet/grafana_dashboards.jsonnet(merging all the jsonnet dashboards in one PR)

3 years agoMerge pull request #43646 from rhcs-dashboard/wip-53026-pacific
Ernesto Puerta [Mon, 25 Oct 2021 14:00:47 +0000 (16:00 +0200)]
Merge pull request #43646 from rhcs-dashboard/wip-53026-pacific

pacific: mgr/dashboard: pin a version for autopep8 and pyfakefs

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: pin a version for autopep8 and pyfakefs 43646/head
Nizamudeen A [Mon, 25 Oct 2021 08:42:57 +0000 (14:12 +0530)]
mgr/dashboard: pin a version for autopep8 and pyfakefs

Fixes: https://tracker.ceph.com/issues/53024
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 946dab4f608ec47e0a3cfefdf8e7d1afda69117f)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for cephfs dashboards
Aashish Sharma [Fri, 8 Oct 2021 10:07:17 +0000 (15:37 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for cephfs dashboards

This PR intends to refactor cephfs dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit ed954b0e6ce24fbae66f78f7e4f90416b9ed7749)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for osds dashboards
Aashish Sharma [Fri, 8 Oct 2021 09:58:13 +0000 (15:28 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for osds dashboards

This PR intends to refactor osds dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit e490e2f3abe707a2e891171f3c230d44e282c601)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for pools dashboards
Aashish Sharma [Fri, 8 Oct 2021 09:52:46 +0000 (15:22 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for pools dashboards

This PR intends to refactor pools dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 8c48821c21f7a6b248de10ff6750a63bab1e4948)

3 years agomgr/dashboard: monitoring: grafonnet refactoring for rbd dashboards
Aashish Sharma [Fri, 8 Oct 2021 09:42:41 +0000 (15:12 +0530)]
mgr/dashboard: monitoring: grafonnet refactoring for rbd dashboards

This PR intends to refactor rbd dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit e737aaa000a31e2f37ca90eb813f031a42edef3b)