]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agomds: skip journaling blocklisted clients when in `replay` state 43841/head
Venky Shankar [Fri, 1 Oct 2021 08:55:40 +0000 (04:55 -0400)]
mds: skip journaling blocklisted clients when in `replay` state

When a standby MDS is transitioning to active, it passes through
`replay` state. When the MDS is in this state, there are no journal
segments available for recording journal updates. If the MDS receives
an OSDMap update in this state, journaling blocklisted clients causes
a crash since no journal segments are available. This is a bit hard
to reproduce as it requires correct timing of an OSDMap update along
with various other factors.

Note that, when the MDS reaches `reconnect` state, it will journal
the blocklisted clients anyway.

This partially fixes tracker: https://tracker.ceph.com/issues/51589
which mentions a similar crash but in `reconnect` state. However,
that crash was seen in nautilus.

A couple of minor changes include removing hardcoded function names
and carving out reusable parts into a separate function.

Partially-fixes: https://tracker.ceph.com/issues/51589
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 6d6236dc8d15636af8060057e6e69c26c473f987)

3 years agoMerge pull request #43708 from cfsnyder/wip-52598-pacific
Guillaume Abrioux [Mon, 8 Nov 2021 13:19:10 +0000 (14:19 +0100)]
Merge pull request #43708 from cfsnyder/wip-52598-pacific

pacific: ceph-volume: util/prepare fix osd_id_available()

3 years agoMerge pull request #43812 from rhcs-dashboard/wip-53153-pacific
Ernesto Puerta [Mon, 8 Nov 2021 13:18:36 +0000 (14:18 +0100)]
Merge pull request #43812 from rhcs-dashboard/wip-53153-pacific

pacific: mgr/dashboard: fix missing alert rule details

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agomgr/dashboard: fix missing alert rule details 43812/head
Ernesto Puerta [Wed, 3 Nov 2021 17:57:53 +0000 (18:57 +0100)]
mgr/dashboard: fix missing alert rule details

Fixes: https://tracker.ceph.com/issues/53144
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit b47f9c83d87057bbb1a0c6052450088532a31f81)

3 years agoMerge pull request #43793 from ifed01/wip-ifed-fix-omap-upgrade-pac
Yuri Weinstein [Thu, 4 Nov 2021 18:37:45 +0000 (11:37 -0700)]
Merge pull request #43793 from ifed01/wip-ifed-fix-omap-upgrade-pac

pacific: os/bluestore: fix invalid omap name conversion when upgrading to per-pg

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43631 from rhcs-dashboard/wip-52803-pacific
Ernesto Puerta [Thu, 4 Nov 2021 09:22:07 +0000 (10:22 +0100)]
Merge pull request #43631 from rhcs-dashboard/wip-52803-pacific

pacific: mgr/dashboard,prometheus: fix handling of server_addr

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #43544 from trociny/wip-52936-pacific
Yuri Weinstein [Wed, 3 Nov 2021 22:35:55 +0000 (15:35 -0700)]
Merge pull request #43544 from trociny/wip-52936-pacific

pacific: osd: handle inconsistent hash info during backfill and deep scrub gracefully

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43457 from gregsfortytwo/wip-52868-pacific
Yuri Weinstein [Wed, 3 Nov 2021 22:35:16 +0000 (15:35 -0700)]
Merge pull request #43457 from gregsfortytwo/wip-52868-pacific

pacific: mon: Allow specifying new tiebreaker monitors

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43415 from trociny/wip-51909-pacific
Yuri Weinstein [Wed, 3 Nov 2021 22:34:29 +0000 (15:34 -0700)]
Merge pull request #43415 from trociny/wip-51909-pacific

pacific: crush: cancel upmaps with up set size != pool size

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #43740 from cfsnyder/wip-53091-pacific
Yuri Weinstein [Wed, 3 Nov 2021 19:08:46 +0000 (12:08 -0700)]
Merge pull request #43740 from cfsnyder/wip-53091-pacific

pacific: rgw: add abstraction for ops log destination and add file logger

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43695 from cfsnyder/wip-52960-pacific
Yuri Weinstein [Wed, 3 Nov 2021 19:08:11 +0000 (12:08 -0700)]
Merge pull request #43695 from cfsnyder/wip-52960-pacific

pacific: rgw/rgw_rados: make RGW request IDs non-deterministic

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #43662 from MrFreezeex/wip-53032-pacific
Yuri Weinstein [Wed, 3 Nov 2021 19:07:35 +0000 (12:07 -0700)]
Merge pull request #43662 from MrFreezeex/wip-53032-pacific

pacific: rbd-mirror: fix mirror image removal

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
3 years agoMerge pull request #43694 from rhcs-dashboard/wip-53065-pacific
Ernesto Puerta [Wed, 3 Nov 2021 15:35:09 +0000 (16:35 +0100)]
Merge pull request #43694 from rhcs-dashboard/wip-53065-pacific

pacific: monitoring: ethernet bonding filter in Network Load.

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoPendingReleaseNotes: document OMAP upgrade bug. 43793/head
Igor Fedotov [Tue, 2 Nov 2021 11:54:55 +0000 (14:54 +0300)]
PendingReleaseNotes: document OMAP upgrade bug.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit dfacf162afe55c6f15a8afb8aca18b52814e80ee)

3 years agoos/bluestore: fix invalid omap name conversion when upgrading to per-pg.
Igor Fedotov [Wed, 27 Oct 2021 10:59:34 +0000 (13:59 +0300)]
os/bluestore: fix invalid omap name conversion when upgrading to per-pg.

Fixes: https://tracker.ceph.com/issues/53062
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit cbc97018d883333f81ab9a3cfa99d2f68a9874cd)

3 years agotest/store_test: add a UT for omap format upgrade.
Igor Fedotov [Wed, 27 Oct 2021 15:58:52 +0000 (18:58 +0300)]
test/store_test: add a UT for omap format upgrade.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit ccb6fdf3b6ba700da1efea597deabe79329a4504)

3 years agoos/bluestore: permit legacy omap naming scheme in mkfs.
Igor Fedotov [Tue, 9 Feb 2021 15:37:41 +0000 (18:37 +0300)]
os/bluestore: permit legacy omap naming scheme in mkfs.

Primarily for debug purposes...

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 99e40a1e87c44705ffdb557d57bd54eaff31cfe3)

 Conflicts:
src/common/options/global.yaml.in
 old way of specifying config settings

3 years agorgw/rgw_rados: make RGW request IDs non-deterministic 43695/head
Cory Snyder [Tue, 5 Oct 2021 19:07:11 +0000 (19:07 +0000)]
rgw/rgw_rados: make RGW request IDs non-deterministic

Use a random number vs. incremental counter for first component of request ID.

Fixes: https://tracker.ceph.com/issues/52818
Signed-off-by: Cory Snyder <csnyder@iland.com>
(cherry picked from commit bce34dd68634d241b451111dcf2e931837eb4bfd)

3 years agoceph-volume: util/prepare fix osd_id_available() 43708/head
Guillaume Abrioux [Thu, 9 Sep 2021 08:23:43 +0000 (10:23 +0200)]
ceph-volume: util/prepare fix osd_id_available()

The current check only allows to request an OSD id that exists but
marked as 'destroyed'.
With this small fix, we can now use `--osd-id` with an id that doesn't
exist at all.

Fixes: https://tracker.ceph.com/issues/50880
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 73bfa5d2b0157f92721d8bf36619fd35ee265cdd)

3 years agoMerge pull request #43728 from sebastian-philipp/pacific-backport-42970-43021-43039...
Sebastian Wagner [Wed, 3 Nov 2021 11:37:45 +0000 (12:37 +0100)]
Merge pull request #43728 from sebastian-philipp/pacific-backport-42970-43021-43039-43010-42989-42859-43143-43141-43115-43162-

pacific: cephadm: October batch

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #43615 from batrick/i52999
Yuri Weinstein [Tue, 2 Nov 2021 13:34:49 +0000 (06:34 -0700)]
Merge pull request #43615 from batrick/i52999

pacific: mon/MDSMonitor: avoid crash when decoding old FSMap epochs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #43614 from batrick/i52998
Yuri Weinstein [Tue, 2 Nov 2021 13:33:41 +0000 (06:33 -0700)]
Merge pull request #43614 from batrick/i52998

pacific: mds/FSMap: do not assert allow_standby_replay on old FSMaps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agopybind/mgr/cephadm: upgrade MDS if no MDS is "up" 43728/head
Patrick Donnelly [Thu, 28 Oct 2021 00:26:55 +0000 (20:26 -0400)]
pybind/mgr/cephadm: upgrade MDS if no MDS is "up"

The upgrade process can get stuck if an MDS crashes. This should be rare
when straddling v16.2.5 where the compatset of the file system inherits
the FSMap "default". The MDS from pre-v16.2.5 do not yet share a
compatset with the mons so the mons will do no promotions, causing
upgrade task to get stuck.

Fixes: https://tracker.ceph.com/issues/53074
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d7717256644e45ba670165c6af0941461fb884f4)

3 years agomgr/orchestrator: log exception from wrapper
Sage Weil [Fri, 15 Oct 2021 15:44:59 +0000 (10:44 -0500)]
mgr/orchestrator: log exception from wrapper

This lets us see a proper traceback when we are calling across
modules.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9502953d4dfcd794fcc9400bb52ebf229abb5517)

3 years agomgr/orchestrator: add --zap flag to 'orch osd rm'
Cory Snyder [Fri, 10 Sep 2021 10:59:35 +0000 (06:59 -0400)]
mgr/orchestrator: add --zap flag to 'orch osd rm'

Adds the ability to zap OSD devices after removal, implemented as a flag
on the 'orch osd rm' command.

Fixes: https://tracker.ceph.com/issues/43692
Signed-off-by: Cory Snyder <csnyder@iland.com>
(cherry picked from commit ea507d6c86f6a6ed5850edf24116e12088b24036)

Conflicts:
src/pybind/mgr/cephadm/module.py
src/pybind/mgr/rook/module.py

3 years agodoc/cephadm/operations: extend CEPHADM_STRAY_DAEMON
Sebastian Wagner [Tue, 28 Sep 2021 12:15:13 +0000 (14:15 +0200)]
doc/cephadm/operations: extend CEPHADM_STRAY_DAEMON

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit eff5c76d885c8ebb3199db362730a7b02796c367)

3 years agodoc/cephadm: osd.rst: s/Note, /.. note:: /
Sebastian Wagner [Tue, 26 Oct 2021 09:37:40 +0000 (11:37 +0200)]
doc/cephadm: osd.rst: s/Note, /.. note:: /

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d749453ce4f9cd8f88bb70cb1421fb1963881248)

3 years agodoc/cephadm: osd.rst: s/DriveGroup/OSD spec/
Sebastian Wagner [Tue, 26 Oct 2021 09:31:14 +0000 (11:31 +0200)]
doc/cephadm: osd.rst: s/DriveGroup/OSD spec/

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 61fe2a21b7d3869a156b1994696b38fbb6587be9)

3 years agodoc/cephadm: osd.rst: Fix yaml comments
Sebastian Wagner [Tue, 26 Oct 2021 09:27:51 +0000 (11:27 +0200)]
doc/cephadm: osd.rst: Fix yaml comments

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 3b6ce664c3ed71aae9d5d4e182d42efe402a3161)

3 years agodoc/cephadm: Add "Deploying OSDs on multiple similar hosts"
Sebastian Wagner [Wed, 20 Oct 2021 12:38:40 +0000 (14:38 +0200)]
doc/cephadm: Add "Deploying OSDs on multiple similar hosts"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 358a5ae588d0d98ba3a7f69d753aae2a639a6700)

3 years agodoc/cephadm: indent yaml specs
Sebastian Wagner [Wed, 20 Oct 2021 12:45:03 +0000 (14:45 +0200)]
doc/cephadm: indent yaml specs

properly conform with other service types

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 84dce06aa85c2a20ec81b7be38aeed8e83615415)

3 years agomgr/cephadm:improve validation of orch device zap
Paul Cuzner [Thu, 14 Oct 2021 00:06:30 +0000 (13:06 +1300)]
mgr/cephadm:improve validation of orch device zap

Improve the logic in the function to :
1. check that the host exists
2. check the host isn't in maintenance
3. check the disk exists on the host
4. check the disk isn't supporting a known OSD
5. check that the disk doesn't belong to another cluster

In addition the command now returns a completion message so
you know it worked.

Fixes: https://tracker.ceph.com/issues/52919
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit b7782084ac9657be9b2da6ebd56b5029cf859225)

3 years agocephadm: allow pulling from insecure registries
Joao Eduardo Luis [Mon, 11 Oct 2021 17:20:47 +0000 (17:20 +0000)]
cephadm: allow pulling from insecure registries

Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit 0a86107dccfdffa7476c61312f0c3b5aae7da7b7)

3 years agomgr/cephadm: test: Move test_monitoring_ports to TestMonitoring
Sebastian Wagner [Mon, 18 Oct 2021 13:20:48 +0000 (15:20 +0200)]
mgr/cephadm: test: Move test_monitoring_ports to TestMonitoring

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit c1c1e961f677ea10f782cab412e8b845e100266b)

Conflicts:
    src/pybind/mgr/cephadm/tests/test_cephadm.py

3 years agomgr/cephadm: Monitoring: Verify config file generation
Sebastian Wagner [Mon, 18 Oct 2021 13:06:20 +0000 (15:06 +0200)]
mgr/cephadm: Monitoring: Verify config file generation

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit f4c154463365f7df5dcd9053e59de65f53bf0f5f)

Conflicts:
src/pybind/mgr/cephadm/tests/test_services.py

3 years agomgr/cephadm: Monitoring: Prometheus: Fix IPv6
Sebastian Wagner [Mon, 18 Oct 2021 13:05:43 +0000 (15:05 +0200)]
mgr/cephadm: Monitoring: Prometheus: Fix IPv6

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 406734a14226800f2c2ed01290a508e89e0dec6e)

3 years agomgr/cephadm: Monitoring: Alertmanager: Fix IPv6
Sebastian Wagner [Mon, 18 Oct 2021 13:05:27 +0000 (15:05 +0200)]
mgr/cephadm: Monitoring: Alertmanager: Fix IPv6

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d65ccf3a0f83fc4a79edfe46c9468cf77f655f83)

3 years agomgr/cephadm: Monitoring: Grafana: Fix IPv6
Sebastian Wagner [Mon, 18 Oct 2021 13:04:46 +0000 (15:04 +0200)]
mgr/cephadm: Monitoring: Grafana: Fix IPv6

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit bb2cb3ea3a9704cf124b5342fb2a974e8618cc01)

Conflicts:
src/pybind/mgr/cephadm/tests/test_ssh.py

3 years agodoc/cephadm: update typo in install.rst
Zac Dover [Wed, 20 Oct 2021 14:42:02 +0000 (00:42 +1000)]
doc/cephadm: update typo in install.rst

This PR changes the string

   use "--log-to-file" option during bootstrap

to

   use the "--log-to-file" option during bootstrap

The way that it was written makes it sound like it was
written by a native Russian speaker. But it was written
by me, a native English speaker, so this is on me.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit d5f6d8ef33413906c8a53a1196ad37a1e1e80289)

3 years agoFix option name osd_crush_chooseleaf_type
krafZLorG [Thu, 21 Oct 2021 07:27:14 +0000 (10:27 +0300)]
Fix option name osd_crush_chooseleaf_type

Signed-off-by: Dmitry Kvashnin <dm.kvashnin@gmail.com>
(cherry picked from commit 0e7c16ce2c94facb7b3979722d1ce50d28d4452f)

3 years agomgr/cephadm: update host when adding host that already exists
Daniel Pivonka [Thu, 12 Aug 2021 18:25:23 +0000 (14:25 -0400)]
mgr/cephadm: update host when adding host that already exists

update addr and labels of existing host
this prevents labels being removed if a host is readded
or addr being set to 127.0.... if readding host active mgr is on

Fixes: https://tracker.ceph.com/issues/51667
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 0facfac91fd8f71e5a8b869d818e7c2b07b93516)

3 years agomgr/cephadm: put list_servers output in debug logs
Adam King [Thu, 30 Sep 2021 12:34:58 +0000 (08:34 -0400)]
mgr/cephadm: put list_servers output in debug logs

Over time, seen a number of cases where daemons are being
marked stray for unknown reasons and it is always difficult
to debug due to the difficulty of finding what list_servers
is outputting. I think it would help to include this in the
debug logs. That way we have that side of the info available
and can easily compare it to daemon/host info cephadm is storing.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit bdc802354554ea9c5e7c0377e3fe695f719921a7)

3 years agocephadm: do not log commands sent to "call" function
Adam King [Thu, 14 Oct 2021 18:56:51 +0000 (14:56 -0400)]
cephadm: do not log commands sent to "call" function

They could potentially log sensitive info such as registry
login passwords

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 3205a835f6bfa1cea4415dc8a95888ac7dade181)

3 years agomgr/orchestrator: clean up 'orch device ls' output
Sage Weil [Tue, 3 Aug 2021 17:25:26 +0000 (13:25 -0400)]
mgr/orchestrator: clean up 'orch device ls' output

- headings in CAPS
- DEVICE ID, not just serial or model/vendor
- drop the health, LED fields from non-wide view
- sort devices
- 'Yes' or '', for easier visual parsing

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c2f82d7228a6ebbca9500f66fc118e0f67a0528a)

3 years agoqa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
Sage Weil [Fri, 15 Oct 2021 15:58:41 +0000 (11:58 -0400)]
qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8900775841c7950ec18c9963bdcb38c2e43cc2f5)

3 years agomgr/cephadm: make upgrade ls output structured
Sage Weil [Fri, 15 Oct 2021 15:09:07 +0000 (11:09 -0400)]
mgr/cephadm: make upgrade ls output structured

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit be3fd05fc59bd523c1171d18cb0a0784f6a6d5c1)

3 years agomgr/cephadm: add 'orch upgrade ls' to list available versions
Sage Weil [Tue, 12 Oct 2021 19:22:14 +0000 (15:22 -0400)]
mgr/cephadm: add 'orch upgrade ls' to list available versions

Optionally pass image name to query an alternative image/registry, or
--tags to list tags instead of ceph versions.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7f750fe31aa62f0055e43e86d4461d48baaaa7d3)

3 years agodoc/cephadm: Hosts: Add "initial CRUSH location"
Sebastian Wagner [Fri, 15 Oct 2021 11:13:02 +0000 (13:13 +0200)]
doc/cephadm: Hosts: Add "initial CRUSH location"

Followup on d2a9a35993

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8e4f9ea46ceec267748d60c6ed009b209d2352ab)

3 years agodoc/cephadm: update example of custom container spec file
Sebastian Wagner [Wed, 29 Sep 2021 10:02:28 +0000 (12:02 +0200)]
doc/cephadm: update example of custom container spec file

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit ce73047d25599b247ab889614fa62684b0c7aca0)

3 years agodoc/cephadm: use sphinx autoclass to document IscsiServiceSpec
Sebastian Wagner [Wed, 29 Sep 2021 10:00:42 +0000 (12:00 +0200)]
doc/cephadm: use sphinx autoclass to document IscsiServiceSpec

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 2e7b369373236c32e429463512521da1fc7e159d)

3 years agodoc/cephadm: use sphinx autoclass to document RGWSpec
Sebastian Wagner [Tue, 28 Sep 2021 14:57:41 +0000 (16:57 +0200)]
doc/cephadm: use sphinx autoclass to document RGWSpec

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8ef77a0bbcef710185c7ee6452d6ca8a133843c1)

3 years agomgr/cephadm: the display of 'orch osd rm status' is incorrect
jianglong01 [Tue, 7 Sep 2021 07:49:04 +0000 (15:49 +0800)]
mgr/cephadm: the display of 'orch osd rm status' is incorrect

After exec 'ceph orch osd rm --replace', 'orch osd rm status' has an wrong display.

wrong display:
OSD_ID  HOST                              STATE     PG_COUNT  REPLACE  FORCE  DRAIN_STARTED_AT
15      cephqa08.cpp.zzbm.qianxin-inc.cn  draining  0         True     True   2021-09-07 07:54:05.776906

correct display:
OSD_ID  HOST                              STATE     PG_COUNT  REPLACE  FORCE  DRAIN_STARTED_AT
15      cephqa08.cpp.zzbm.qianxin-inc.cn  draining  0         True     False  2021-09-07 07:35:34.731417

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
(cherry picked from commit 465f7e8fa34b37cf09e4d280da5239086924b942)

3 years agocephadm: add a test on 'ceph orch ls'
Guillaume Abrioux [Wed, 29 Sep 2021 06:42:33 +0000 (08:42 +0200)]
cephadm: add a test on 'ceph orch ls'

This adds a unittest in order to cover the output of 'orch ls' command
and check the reporting about the running osd number is displayed as
expected.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Co-Authored-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 1d848dc447fd4df18893903a053d58129fb8c13a)

3 years agocephadm: fix osd size reported in 'orch ls'
Guillaume Abrioux [Wed, 22 Sep 2021 07:35:54 +0000 (09:35 +0200)]
cephadm: fix osd size reported in 'orch ls'

The current code enforces ('running osds' + 1), see [1].
Let's simply print the number of running osds instead of a
meaningless calculation.

[1] https://github.com/ceph/ceph/commit/1c0a0a6f6153dc555085f263f161666cd171964a#diff-4f2fb7d330e74b64ac41457b7c7a723cd78db86433e0b0c398874531e5a7e39eR1852-R1859

Fixes: https://tracker.ceph.com/issues/50928
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 97219591dab8f54f239ccfb1891c45b01e7bec79)

3 years agomgr/cephadm: use json-pretty for ceph-volume calls
Adam King [Wed, 29 Sep 2021 02:10:07 +0000 (22:10 -0400)]
mgr/cephadm: use json-pretty for ceph-volume calls

To avoid a single line json string that is too long
and causes a chunk longer than limit error

Fixes: https://tracker.ceph.com/issues/52745
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit b20637714e366b44ec604a76bab9095d5f4a1ef2)

3 years agodoc/cephadm: document setting ssh-user during adopt
Daniel Pivonka [Mon, 11 Oct 2021 21:15:35 +0000 (17:15 -0400)]
doc/cephadm: document setting ssh-user during adopt

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit bafbeb6fca21374aebace7ae65b964c875d87d01)

3 years agodoc/cephadm: note that replacment osd needs to be on same host as removed osd
Daniel Pivonka [Thu, 21 Oct 2021 17:05:05 +0000 (13:05 -0400)]
doc/cephadm: note that replacment osd needs to be on same host as removed osd

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 441869220c97d739cbe2186a9e4ee0d8a2911f1d)

3 years agomgr/cephadm: set health check warning in `_apply_service_config` for invalid config...
Melissa Li [Thu, 5 Aug 2021 18:28:35 +0000 (14:28 -0400)]
mgr/cephadm: set health check warning in `_apply_service_config` for invalid config options and failures to set options

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit 0f8f085058dc3319c073fb50491d1862e45ba9a2)

3 years agomgr/cephadm: refactor `_check_for_strays` to use `remove_health_warning` helper
Melissa Li [Tue, 3 Aug 2021 17:50:26 +0000 (13:50 -0400)]
mgr/cephadm: refactor `_check_for_strays` to use `remove_health_warning` helper

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit f42992a1ef8f7827155e377013cef6c9a8cba15b)

3 years agomgr/cephadm: refactor `_refresh_hosts_and_daemons` to use `set_health_warning` and...
Melissa Li [Tue, 3 Aug 2021 17:46:21 +0000 (13:46 -0400)]
mgr/cephadm: refactor `_refresh_hosts_and_daemons` to use `set_health_warning` and `remove_health_warning` helpers

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit 2eacd6e7e936636b2eee82bd03ee59f67431cd2e)

3 years agomgr/cephadm: refactor `_update_paused_health` to use `set_health_warning` and `remove...
Melissa Li [Tue, 3 Aug 2021 17:40:39 +0000 (13:40 -0400)]
mgr/cephadm: refactor `_update_paused_health` to use `set_health_warning` and `remove_health_warning` helpers

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit cfd6ad0af2feac3a7611067dd7e88fed1b315023)

3 years agomgr/cephadm: refactor `_set_maintenance_health_check` to use `set_health_warning...
Melissa Li [Tue, 3 Aug 2021 17:34:06 +0000 (13:34 -0400)]
mgr/cephadm: refactor `_set_maintenance_health_check` to use `set_health_warning` helper

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit c3f8fba04a629f2f3e70d0f5443a42de54426bf2)

3 years agomgr/cephadm: add `set_health_warning` and `remove_health_warning` helpers in module.py
Melissa Li [Tue, 3 Aug 2021 17:28:40 +0000 (13:28 -0400)]
mgr/cephadm: add `set_health_warning` and `remove_health_warning` helpers in module.py

Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
(cherry picked from commit 6044afb2cead4d54c19cb6052c90afd56c28e0d0)

3 years agocephadm: bootstrap --apply-spec shouldn't enforce :z
Guillaume Abrioux [Thu, 7 Oct 2021 14:23:07 +0000 (16:23 +0200)]
cephadm: bootstrap --apply-spec shouldn't enforce :z

There's no need to mount this file with the `:z` option.
read-only access is enough for this file.

Fixes: https://tracker.ceph.com/issues/52855
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit edc07019c6e70950d697ec0d7f3756065ac64b92)

3 years agocephadm: shell --mount shouldnt enforce ':z' option
Guillaume Abrioux [Thu, 7 Oct 2021 08:49:25 +0000 (10:49 +0200)]
cephadm: shell --mount shouldnt enforce ':z' option

cephadm shouldn't enforce this option.
For instance, it can be an issue when you try to bindmount a file in /usr

Fixes: https://tracker.ceph.com/issues/52853
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit aeae360e2595f348f10ddc36cdb270b018f7eb02)

3 years agosrc/cephadm: Add mariner as a supported distro.
Roaa Sakr [Fri, 1 Oct 2021 22:40:11 +0000 (15:40 -0700)]
src/cephadm: Add mariner as a supported distro.

Using cephadm install commands on Mariner will result in an error
"Distro not supported" though the packages are supported and can be
installed manually. This patch adds mariner as a supported distro.
Fix validated on mariner.

Signed-off-by: Roaa Sakr <romoh@microsoft.com>
(cherry picked from commit 1612427ea549d3fc259c8eb0e37953a9323dcb31)

3 years agocephadm: add daemon_name in daemon description
Guillaume Abrioux [Thu, 20 May 2021 14:06:05 +0000 (16:06 +0200)]
cephadm: add daemon_name in daemon description

This adds the daemon_name in json output when asking for daemon
description.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 19ee16a46b90d92f1a49ace5ba1a3c47eb8c1403)

3 years agodoc/cephadm/services: unindent note section
Kefu Chai [Wed, 29 Sep 2021 23:22:09 +0000 (07:22 +0800)]
doc/cephadm/services: unindent note section

before this change the note on "apply" command is embedded in the note
on "_no_schedule". and they are not related. so let's move the former
out. also, highlight the yaml file sample in YAML.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 48f5cd5b9bb1ccf89e1eaad163cf11874ad3e176)

3 years agomgr/cephadm: populate trusted_ip_list in iscsi-gateway.cfg with mgr ips
Daniel Pivonka [Tue, 21 Sep 2021 20:04:51 +0000 (16:04 -0400)]
mgr/cephadm: populate trusted_ip_list in iscsi-gateway.cfg with mgr ips

Iscsi gateways do not show "UP" in dashboard without this

fixes: https://tracker.ceph.com/issues/52692
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit cda82c98a32f51cb392fc51ba854bcae409567f8)

Conflicts:
src/pybind/mgr/cephadm/module.py

3 years agocephadm/iscsi: fix a typo
Guillaume Abrioux [Mon, 27 Sep 2021 06:57:35 +0000 (08:57 +0200)]
cephadm/iscsi: fix a typo

This commit fixes a typo in the comment of unit.run of iscsi service.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5a4f29de7a6080b0476ad134426faf94c01fb316)

3 years agocephadm: fix iscsi test
Guillaume Abrioux [Mon, 27 Sep 2021 15:22:48 +0000 (17:22 +0200)]
cephadm: fix iscsi test

the current test is wrong because it generates the tcmu-runner part two
times.
given the function `deploy_daemon_units()` in cephadm already writes a
first time the tcmu-runner command, calling a second time
`get_tcmu_runner_container()` from the test makes `deploy_daemon_units()`
write the same command again.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7e9762e0ce36d773c34fc9353d2eacc694554f39)

3 years agocephadm: modify iscsi containers bindmount
Guillaume Abrioux [Mon, 27 Sep 2021 11:46:01 +0000 (13:46 +0200)]
cephadm: modify iscsi containers bindmount

tcmu-runner logs in `/var/log/tcmu-runner.log`, there's no option to
make it log to stdout/stderr so the log is only available from the
container.
Modifying the bindmount from `-v /var/log/ceph/<fsid>/:/var/log/rbd-target-api:z`
to `-v /var/log/ceph/<fsid>/:/var/log:z` makes it at least available
from the host.

Fixes: https://tracker.ceph.com/issues/47480
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b130214195ff84af57f447cf3c3410f0918becaf)

3 years agodoc/cephadm: move services into services/
Sebastian Wagner [Mon, 13 Sep 2021 15:15:33 +0000 (17:15 +0200)]
doc/cephadm: move services into services/

This is going to clean up the toctree a bit.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8c70398949f773d8b992ad8ae6c71383460d2932)

3 years agodoc/cephadm: move service-management to services/index.rst
Sebastian Wagner [Mon, 13 Sep 2021 15:12:01 +0000 (17:12 +0200)]
doc/cephadm: move service-management to services/index.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 5e25b19cc7bd5845eccdd0bbb3d6ee8a5852563d)

3 years agodoc/cephadm: Add `networks` to spec documentation
Sebastian Wagner [Tue, 14 Sep 2021 14:07:14 +0000 (16:07 +0200)]
doc/cephadm: Add `networks` to spec documentation

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit fd32c1651b95ac14662bb5048e42f92fc1f45e56)

3 years agodoc/cephadm: add mgr.rst
Sebastian Wagner [Tue, 14 Sep 2021 14:15:29 +0000 (16:15 +0200)]
doc/cephadm: add mgr.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 6773cb01b8f42a4a37b4360d7a032524df39a6f8)

3 years agodoc/cephadm: Add `Specifying Networks` to rgw.rst
Sebastian Wagner [Tue, 14 Sep 2021 14:08:11 +0000 (16:08 +0200)]
doc/cephadm: Add `Specifying Networks` to rgw.rst

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 4243a7936a9972a1d9c677207ad6a4e56908e7af)

3 years agomgr/dashboard: More robust handling of timedelta() representation
Brad Hubbard [Fri, 24 Sep 2021 02:22:42 +0000 (12:22 +1000)]
mgr/dashboard: More robust handling of timedelta() representation

The representation of timedelta() appears to have changed recently so
we should try and be more accommodating in that regard.

Fixes: https://tracker.ceph.com/issues/52696
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit ad1a4f2410393cab6ac728c4ddb79d111e252559)

3 years agocephadm: Fix TestMaintenance trying to list /var/lib/ceph
Sebastian Wagner [Fri, 24 Sep 2021 10:55:03 +0000 (12:55 +0200)]
cephadm: Fix TestMaintenance trying to list /var/lib/ceph

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 767564864f15ea8afe9b925345cdf59db1815cb0)

3 years agocephadm: TestCheckHost: also mock `check_time_sync`
Sebastian Wagner [Fri, 24 Sep 2021 10:46:54 +0000 (12:46 +0200)]
cephadm: TestCheckHost: also mock `check_time_sync`

Fixes: https://tracker.ceph.com/issues/52722
```
TestCheckHost.test_container_engine fails at cephadm:5834: Error cephadm.Error: No time synchronization is active
```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit fc9d9b39144fd16f2b2a9621f80dcd1162459000)

3 years agomgr/cephadm: base maintenance enter/exit success off of return message
Adam King [Wed, 22 Sep 2021 21:43:43 +0000 (17:43 -0400)]
mgr/cephadm: base maintenance enter/exit success off of return message

rather than on whether there was any stdout from the command

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 393330bae6b259102bba83b0d3c6213970838c02)

3 years agomgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure...
Adam King [Wed, 22 Sep 2021 21:42:41 +0000 (17:42 -0400)]
mgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure messages

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 5350d58b9ad45359b7885d19592c2b4826e9aed7)

Conflicts:
src/pybind/mgr/cephadm/tests/test_cephadm.py

3 years agocephadm: unit tests for maitenance mode return values
Adam King [Wed, 22 Sep 2021 18:34:12 +0000 (14:34 -0400)]
cephadm: unit tests for maitenance mode return values

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 73e6aa8d9ca69c96361fae6998dd71d252808aa4)

3 years agocephadm: fix mypy complaints for ThreadedChildWatcher class
Adam King [Tue, 21 Sep 2021 18:00:11 +0000 (14:00 -0400)]
cephadm: fix mypy complaints for ThreadedChildWatcher class

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 8bdddfa02eea7331971c16a9f8806b8451f09c5b)

3 years agocephadm: fix exiting maintenance when systemd target doesn't exist
Adam King [Mon, 20 Sep 2021 12:02:44 +0000 (08:02 -0400)]
cephadm: fix exiting maintenance when systemd target doesn't exist

If the systemd target doesn't exist we need to just bypass enabling
it and return success or the host will just be stuck in maitnenance
mode.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 3a15f1dc81dfd5f963e1bf36316fa5b289a898b0)

3 years agocephadm: raise error during `pull` failure
Michael Fritch [Wed, 8 Sep 2021 23:16:31 +0000 (17:16 -0600)]
cephadm: raise error during `pull` failure

instead of a traceback to the console

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit d07e72c4b0bdc4d74cf1a0d8826061942f757d08)

3 years agomgr/cephadm: osd should not be zap when it is running
jianglong01 [Mon, 6 Sep 2021 10:37:06 +0000 (18:37 +0800)]
mgr/cephadm: osd should not be zap when it is running

When osd is running, we exec the command of "ceph orch device zap".
Then ceph-volume will exec dd command which cause the osd of lvs to be destroyed.
If the osd of systemctl unit restarts, it will be unable to boot.

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
(cherry picked from commit aed472f7b6663c487a5ae89fcdd788b32910b621)

3 years agodoc: from v16.2.6 you have to point to the quay.io image if using docker
Javier Cacheiro [Tue, 21 Sep 2021 09:37:42 +0000 (11:37 +0200)]
doc: from v16.2.6 you have to point to the quay.io image if using docker

New images are no longer uploaded to the Docker Hub registry, so Docker users should point the upgrade
command to the quay.io image.

Fixes: https://tracker.ceph.com/issues/52683
Signed-off-by: Javier Cacheiro <javier.cacheiro.lopez@cesga.es>
(cherry picked from commit 961a4fcffa9bae348810a8542c08ca38947ed111)

3 years agodoc/ceph-volume/activate: Users should call cephadm instead
Sebastian Wagner [Sat, 11 Sep 2021 17:27:23 +0000 (19:27 +0200)]
doc/ceph-volume/activate: Users should call cephadm instead

Fixes: https://tracker.ceph.com/issues/52575
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 925820e73cfeae290900d2077cc792e9085b5767)

3 years agodoc/cephadm/service-management: Move Deployment of Daemons
Sebastian Wagner [Tue, 14 Sep 2021 14:43:17 +0000 (16:43 +0200)]
doc/cephadm/service-management: Move Deployment of Daemons

.. into "Daemon Placement"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit b68c99cd739d8f92784a3398e238e3ed6903046f)

3 years agodoc/cephadm/service-management: Move "Updating Service Specifications"
Sebastian Wagner [Tue, 14 Sep 2021 14:45:54 +0000 (16:45 +0200)]
doc/cephadm/service-management: Move "Updating Service Specifications"

into "Service Specification"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 80f33a4e7df949759f9ac02c50b3bf99d03678e1)

3 years agocephadm: skip podman check during `rm-repo`
Michael Fritch [Fri, 10 Sep 2021 13:38:48 +0000 (07:38 -0600)]
cephadm: skip podman check during `rm-repo`

allow the `rm-repo` command to succeed when podman is not installed

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fd977773a57e12003fb02bdc762bf6bc89d785a1)

3 years agodoc/cephadm: Removing a service
Sebastian Wagner [Sat, 11 Sep 2021 17:15:38 +0000 (19:15 +0200)]
doc/cephadm: Removing a service

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 7af138e089bf0972a2067f84fe9dd6cd4588e7f8)

3 years agodoc/cephadm: Add lots of links to other chapters
Sebastian Wagner [Sat, 11 Sep 2021 18:02:44 +0000 (20:02 +0200)]
doc/cephadm: Add lots of links to other chapters

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d9ec8eb7a8da3c7dff40d7ed89feaebf7cadd37d)

3 years agocephadm: show podman version during `check-host`
Michael Fritch [Mon, 23 Aug 2021 13:47:56 +0000 (07:47 -0600)]
cephadm: show podman version during `check-host`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 44aee33945f285ed4366b960e9526ed9d1984382)

3 years agocephadm: avoid unhandled `AttributeError`
Michael Fritch [Thu, 19 Aug 2021 20:06:32 +0000 (14:06 -0600)]
cephadm: avoid unhandled `AttributeError`

when docker/podman are not present

Fixes: https://tracker.ceph.com/issues/51818
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4d5694a9f0977a22c2a6dac680d594ab3feb070b)

3 years agomgr/cephadm: show unhandled exceptions during `host add`
Michael Fritch [Thu, 19 Aug 2021 21:21:06 +0000 (15:21 -0600)]
mgr/cephadm: show unhandled exceptions during `host add`

138700e59bcd assumes stderr will always have a line containing the
prefix 'ERROR', which leads to an empty error reason when `check-host`
fails with an unhandled exception

Fixes: https://tracker.ceph.com/issues/51818
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit dac9225085a1f6d2eeaf209fc3d77c54208db2e8)

3 years agomgr/cephadm: Add OSDService.post_remove()
Sebastian Wagner [Tue, 31 Aug 2021 09:38:14 +0000 (11:38 +0200)]
mgr/cephadm: Add OSDService.post_remove()

Do not remove the osd.N keyring, if we failed to deploy the OSD, because
we cannot recover from it. The OSD keys are created by ceph-volume and not by
us.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d7a4c5794034e60e94dd12951f7dbf4685647686)

3 years agomgr/cephadm: Add MonService.post_remove()
Sebastian Wagner [Tue, 31 Aug 2021 09:01:11 +0000 (11:01 +0200)]
mgr/cephadm: Add MonService.post_remove()

We should never remove the mon keyring. Let's move
this piece of code into the MonService class

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 71eaf35aa755735574f8bc53b38fa1bac550792c)

3 years agocephadm: (re)add command argv logging
Michael Fritch [Mon, 30 Aug 2021 15:40:55 +0000 (09:40 -0600)]
cephadm: (re)add command argv logging

introduced by 81a7df0498d and inadvertently removed by 3afec2ab30c

Fixes: https://tracker.ceph.com/issues/52484
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6d18759bcb75c68c3a2d421e5d39c6cee8c18526)