]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agomgr/cephadm: Introduce tox and autopep8
Volker Theile [Wed, 29 Jul 2020 10:13:01 +0000 (12:13 +0200)]
mgr/cephadm: Introduce tox and autopep8

This PR introduces tox and autopep8 to format the Python code according to PEP8. It does NOT include any lint related things.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 7d8a3b60321da10af6f4557a9d69eda09ca2c620)

Conflicts:
src/pybind/mgr/tox.ini

5 years agocephadm: Deploying alertmanager image is broken
Volker Theile [Mon, 3 Aug 2020 14:53:31 +0000 (16:53 +0200)]
cephadm: Deploying alertmanager image is broken

Fixes: https://tracker.ceph.com/issues/46814
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit e401946531d6b151cd696c3ea74fa75dcd209f43)

5 years agomgr/cephadm: correct to_json output from str->dict
Joshua Schmid [Fri, 31 Jul 2020 11:33:24 +0000 (13:33 +0200)]
mgr/cephadm: correct to_json output from str->dict

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 4b8384f0638b6138abed5264e3c9688350fa9987)

5 years agocephadm: add unwrap_ipv6 helper method
Matthew Oliver [Thu, 30 Jul 2020 03:18:48 +0000 (13:18 +1000)]
cephadm: add unwrap_ipv6 helper method

When we pass in a mon-ip that is ipv6 we want it wrapped, this is so it
can be properly inserted into the mon_addrv address.
But there are times we need to unwrap it to test it's a valid ipv6
address.

This patch adds a helper method `unwrap_ipv6` which takes a string and
returns it either unwrapped or as it is, so it's harmless to other types
of ips.

This allows us to check a wrapped ipv6 base_ip with the networks on the
host.

Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 13d3bad01954a25dd49b7a7d71bd74dd931f09c5)

5 years agobuild/debian: remove osd_support
Joshua Schmid [Tue, 28 Jul 2020 11:43:11 +0000 (13:43 +0200)]
build/debian: remove osd_support

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 8ff2824beb78dfd03710e94302f038ea70fb4561)

5 years agodoc/cephadm: adjust osd removal docs
Joshua Schmid [Tue, 28 Jul 2020 07:35:56 +0000 (09:35 +0200)]
doc/cephadm: adjust osd removal docs

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 9606f4d6023d2663469b0857ce879ea1ae41c166)

5 years agomgr/cephadm: add tests for new remove/replace logic
Joshua Schmid [Fri, 24 Jul 2020 13:25:31 +0000 (15:25 +0200)]
mgr/cephadm: add tests for new remove/replace logic

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit e78f5c8faeedf014aa8590ff47ebbd517a77204a)

5 years agomgr/cephadm: implement osd remove/replace logic in cephadm
Joshua Schmid [Fri, 24 Jul 2020 13:24:38 +0000 (15:24 +0200)]
mgr/cephadm: implement osd remove/replace logic in cephadm

Fixes: https://tracker.ceph.com/issues/44548
Fixes: https://tracker.ceph.com/issues/45594
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit a0f68c488f74c13534b1b6452a8018f4bb2802d1)

5 years agomgr/osd_support: remove module and all traces
Joshua Schmid [Fri, 24 Jul 2020 13:23:52 +0000 (15:23 +0200)]
mgr/osd_support: remove module and all traces

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit a55c1dd0fac826978edf22e448c3456e7634cbc3)

Conflicts:
src/mon/MgrMonitor.cc

5 years agomgr/cephadm: add qa tests for daemon restart
Paul Cuzner [Wed, 29 Jul 2020 21:02:04 +0000 (09:02 +1200)]
mgr/cephadm: add qa tests for daemon restart

Added tests to cove daemon stop/start and restart

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 5ff0a1d2daa7d36c9c3284523269c4a1a7daa437)

5 years agomgr/cephadm: fix call to cephadm for daemon restarts etc
Paul Cuzner [Wed, 29 Jul 2020 04:00:37 +0000 (16:00 +1200)]
mgr/cephadm: fix call to cephadm for daemon restarts etc

The call currently passes the daemon_type as the first
parameter - but the function expects the hostname. This
results in failures when attempting daemon restarts through the ceph orch daemon <action> <daemon_id>
command.

Fixes: https://tracker.ceph.com/issues/46740
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 94e418581c9126dc199def6078d6a8449bdec8df)

5 years agomgr/cephadm: Add migration to keep the service names consistent
Sebastian Wagner [Fri, 24 Jul 2020 14:20:22 +0000 (16:20 +0200)]
mgr/cephadm: Add migration to keep the service names consistent

After 15.2.4, we unified some service IDs: MONs, MGRs etc no longer have a service id.
Which means, the service names changed:

mon.foo -> mon
mgr.foo -> mgr

This fixes the data structure consistency

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f2c0b2efae730c972f088a9d25247375c229499f)

5 years agomgr/cephadm: minor cleanup in test_migration
Sebastian Wagner [Fri, 24 Jul 2020 14:17:45 +0000 (16:17 +0200)]
mgr/cephadm: minor cleanup in test_migration

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 775d5bedad2703014878d7744d23a25d13bd9bde)

5 years agodoc/mgr/orchestrator: Fix formatting and examples in create OSDs
Varsha Rao [Thu, 30 Jul 2020 13:28:47 +0000 (15:28 +0200)]
doc/mgr/orchestrator: Fix formatting and examples in create OSDs

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 195f28a6148c41c7502027b96e8f154a99342acf)

5 years agomgr/orchestrator: For OSDs, the service_id is the osdspec_affinity
Sebastian Wagner [Fri, 31 Jul 2020 09:09:04 +0000 (11:09 +0200)]
mgr/orchestrator: For OSDs, the service_id is the osdspec_affinity

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d04bfb846b78a0a53777a32526eae935e318a606)

5 years agomgr/cephadm: get_daemons_by_service should use dd.service_name
Sebastian Wagner [Fri, 31 Jul 2020 09:06:35 +0000 (11:06 +0200)]
mgr/cephadm: get_daemons_by_service should use dd.service_name

daemon_name does not necessarily starts with the service_name.

Especially not for OSDs.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 051361e22246120331795b70513beaaff29ceb51)

5 years agopython-common: don't assert when building rados url
Michael Fritch [Thu, 30 Jul 2020 17:34:33 +0000 (11:34 -0600)]
python-common: don't assert when building rados url

breaks the `orch ls --format yaml` command

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 26aa14820196143c25d57b4d5517adf6500b19f9)

5 years agomgr/cephadm: revamp ceph.conf distribution scheduling
Sebastian Wagner [Mon, 27 Jul 2020 12:27:12 +0000 (14:27 +0200)]
mgr/cephadm: revamp ceph.conf distribution scheduling

Having an in-memeory list doesn't work properly: Especially
when loading the mgr module, we didn't knwo if we should
deploy confs or not.

Now we only distribute ceph.confs, if there is a new mon_map.
We also store that info now in the config store

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit aa4c3b2a4e9a37f0f92f50d3bafbbd63dfb6fefa)

5 years agomgr/cephadm: Add test verifying the initializaiton order
Sebastian Wagner [Mon, 27 Jul 2020 10:09:30 +0000 (12:09 +0200)]
mgr/cephadm: Add test verifying the initializaiton order

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 537412cc96be40bc4ab920eb5625f7d1338f4569)

5 years agovstart: infer the mon public_network
Michael Fritch [Tue, 28 Jul 2020 17:24:21 +0000 (11:24 -0600)]
vstart: infer the mon public_network

set the mon public_network when deploying with the cephadm flag

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6442662bbe45af2591427e9522eef45604cebbfa)

5 years agomgr/orch: allow for multiline OrchestratorEvent message
Michael Fritch [Tue, 28 Jul 2020 19:57:17 +0000 (13:57 -0600)]
mgr/orch: allow for multiline OrchestratorEvent message

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit a10861255087b1122f69b6a863d5374ec113bb58)

5 years agomgr/cephadm: streamline rgw deployment
Daniel-Pivonka [Thu, 16 Jul 2020 12:24:47 +0000 (08:24 -0400)]
mgr/cephadm: streamline rgw deployment

cephadm will create realm, zonegroup, and zone if needed before creating rgw service

fixes: https://tracker.ceph.com/issues/43681
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit 131001c453dc3c5d14c56691cdf25353e0127882)

5 years agomgr/cephadm: add `orch host ok-to-stop` command
Michael Fritch [Tue, 21 Jul 2020 21:06:19 +0000 (15:06 -0600)]
mgr/cephadm: add `orch host ok-to-stop` command

$ ceph orch host ok-to-stop host1
It is presumed safe to stop host host1

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit d6fa2e23015c405775c89c2f1bd55d10c0d8a710)

5 years agomgr/cephadm: return HandleCommandResult from ok_to_stop
Michael Fritch [Tue, 21 Jul 2020 21:26:43 +0000 (15:26 -0600)]
mgr/cephadm: return HandleCommandResult from ok_to_stop

- return output from the result of the ok_to_stop command
- log ok-to-stop result during all invocations

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 2521a7cef1a5ac0d42f26aabe597a89926814152)

5 years agomgr/orch: add errno to OrchestratorError
Michael Fritch [Wed, 22 Jul 2020 23:43:05 +0000 (17:43 -0600)]
mgr/orch: add errno to OrchestratorError

add errno to OrchestratorError and ServiceSpecValidationError exceptions

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 60b99dcdc767d5520f4dddafe124de4404ee7459)

5 years agocephadm: use src/mypy.ini instead
Sebastian Wagner [Mon, 20 Jul 2020 11:55:09 +0000 (13:55 +0200)]
cephadm: use src/mypy.ini instead

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 148ca82f7483f6564d23472f25733fc1eb743c5e)

Conflicts:
src/cephadm/tox.ini

5 years agocephadm: Add tcmu-runner container when deploying ceph-iscsi
Matthew Oliver [Wed, 22 Jul 2020 07:09:12 +0000 (17:09 +1000)]
cephadm: Add tcmu-runner container when deploying ceph-iscsi

Currently when we deploy ceph-iscsi via cephadm it doesn't include a
running tcmu-runner. Which means initiators will be able to login but
you wont see the LUNS on the initiator.

This patch deploys an additional tcmu-runner container along side the
ceph-iscsi container that just runs the tcmu-runner service.

Fixes: https://tracker.ceph.com/issues/46540
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit eb604d3338b32a7a11e36e05df6556dbd95ac4a0)

5 years agomgr/orch: Add some more type annotations
Sebastian Wagner [Fri, 24 Jul 2020 15:29:28 +0000 (17:29 +0200)]
mgr/orch: Add some more type annotations

Made `orch.Completion` a generic type

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 2310d7fbd092dc28e053f18e354ed01514061610)

5 years agomgr/cephadm: allow use of authenticated registry
Adam King [Fri, 10 Jul 2020 12:09:39 +0000 (08:09 -0400)]
mgr/cephadm: allow use of authenticated registry

Add option to use custom authenticated registry during
bootstrap as well as a registry-login command in order
to let user change authenticated registry login info

Fixes: https://tracker.ceph.com/issues/44886
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit a63b030ded2f3ea885883530eb06549356655955)

5 years agomgr/cephadm: re-add `apply_drivegroups()`
Sebastian Wagner [Thu, 23 Jul 2020 08:43:17 +0000 (10:43 +0200)]
mgr/cephadm: re-add `apply_drivegroups()`

Fixes: d348d7bf8d3663140c089937b62a0b316b69176b
Fixes: https://tracker.ceph.com/issues/46681
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 6e8f40e4a413e5de291a32185196e31fbef49c90)

5 years agodoc/cephadm: `status` doesn't show a progress
Sebastian Wagner [Mon, 27 Jul 2020 14:50:01 +0000 (16:50 +0200)]
doc/cephadm: `status` doesn't show a progress

Fixes: https://tracker.ceph.com/issues/45858
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1125a805bd7c4c4663c7805d3f713d6dd143dcce)

5 years agomgr/cephadm: create OSDs in parallel
Sebastian Wagner [Wed, 22 Jul 2020 12:45:09 +0000 (14:45 +0200)]
mgr/cephadm: create OSDs in parallel

Should speedup day 1 OSD creation by something like factor 10

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit bd356bae1474884726ff295b23a3a02951a9aedc)

5 years agomgr/cephadm: move `forall_hosts()` to utils module
Sebastian Wagner [Wed, 22 Jul 2020 12:39:26 +0000 (14:39 +0200)]
mgr/cephadm: move `forall_hosts()` to utils module

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 031365d357f02cdc5a0743c31672cbc392df8816)

5 years agodoc/mgr/orchestrator: add missing OSD service_id
Michael Fritch [Wed, 22 Jul 2020 22:35:32 +0000 (16:35 -0600)]
doc/mgr/orchestrator: add missing OSD service_id

example for deploying multiple specs via yaml was missing the service_id

Fixes: https://tracker.ceph.com/issues/46377
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 7906460483c473b639f19d7452e899635552fd82)

5 years agopython-common: clean-up ServiceSpec.service_id handling
Michael Fritch [Tue, 30 Jun 2020 22:06:20 +0000 (16:06 -0600)]
python-common: clean-up ServiceSpec.service_id handling

service_id is required for iscsi, mds, nfs, osd, rgw.

any other service_type (mon, mgr, etc.) should not contain a service_id

Fixes: https://tracker.ceph.com/issues/46175
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit eecc8fcbcc2ee88e4c873579c69f2012dac8c652)

5 years agodoc/cephadm: Fix typos in 'Cephadm Concepts' document.
Volker Theile [Wed, 22 Jul 2020 08:00:18 +0000 (10:00 +0200)]
doc/cephadm: Fix typos in 'Cephadm Concepts' document.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 7f5244f6ec44339f0bea8c201ff86dec1b9bb5b5)

5 years agodoc: add notes about --dry-run
Joshua Schmid [Mon, 6 Jul 2020 09:59:49 +0000 (11:59 +0200)]
doc: add notes about --dry-run

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit a8e21a114bb25bec9b666db61412b2c1768680dc)

5 years agomgr/cephadm: rework --dry-run/previews
Joshua Schmid [Wed, 10 Jun 2020 08:33:36 +0000 (10:33 +0200)]
mgr/cephadm: rework --dry-run/previews

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit d348d7bf8d3663140c089937b62a0b316b69176b)

Conflicts:
src/pybind/mgr/orchestrator/module.py

5 years agomgr/cephadm: adapt tests to have OSDSPEC_AFFINITY
Joshua Schmid [Thu, 4 Jun 2020 07:53:32 +0000 (09:53 +0200)]
mgr/cephadm: adapt tests to have OSDSPEC_AFFINITY

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 917e043c7c7b94a40e6867bb6fd25ff005be76b3)

5 years agomgr/cephadm: re-enable osdspec affinity passing
Joshua Schmid [Thu, 4 Jun 2020 06:52:38 +0000 (08:52 +0200)]
mgr/cephadm: re-enable osdspec affinity passing

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit edbb4f2db4834800eb3d005e92ad9497d3451668)

5 years agoMerge pull request #36332 from yuriw/wip-yuriw-octopus-p2p-octopus
Yuri Weinstein [Mon, 3 Aug 2020 18:21:47 +0000 (11:21 -0700)]
Merge pull request #36332 from yuriw/wip-yuriw-octopus-p2p-octopus

qa/tests: changed rbd/test_librbd_python.sh to run on v15.2.4

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #36143 from rhcs-dashboard/wip-46576-octopus
Lenz Grimmer [Mon, 3 Aug 2020 15:14:24 +0000 (17:14 +0200)]
Merge pull request #36143 from rhcs-dashboard/wip-46576-octopus

octopus: mgr/dashboard: add API team to CODEOWNERS

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #36226 from bk201/wip-46351-octopus
Lenz Grimmer [Mon, 3 Aug 2020 15:11:29 +0000 (17:11 +0200)]
Merge pull request #36226 from bk201/wip-46351-octopus

octopus: mgr/dashboard: Re-enable OSD's table autoReload

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
5 years agoMerge pull request #36277 from s0nea/wip-46590-octopus
Lenz Grimmer [Mon, 3 Aug 2020 15:08:01 +0000 (17:08 +0200)]
Merge pull request #36277 from s0nea/wip-46590-octopus

octopus: mgr/dashboard: Use same required field message accross the UI

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #36322 from rhcs-dashboard/wip-46736-octopus
Lenz Grimmer [Mon, 3 Aug 2020 15:05:52 +0000 (17:05 +0200)]
Merge pull request #36322 from rhcs-dashboard/wip-46736-octopus

octopus: mgr/dashboard: cpu stats incorrectly displayed

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #36369 from bk201/wip-44458
Lenz Grimmer [Mon, 3 Aug 2020 15:02:46 +0000 (17:02 +0200)]
Merge pull request #36369 from bk201/wip-44458

octopus: mgr/dashboard: right-align dropdown menu of column filters

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
5 years agoMerge pull request #36247 from dillaman/wip-46674
Yuri Weinstein [Thu, 30 Jul 2020 16:06:07 +0000 (09:06 -0700)]
Merge pull request #36247 from dillaman/wip-46674

octopus: librbd: new 'write_zeroes' API methods to suppliment the `discard` APIs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
5 years agomgr/dashboard: right-align dropdown menu of column filters 36369/head
Kiefer Chang [Thu, 30 Jul 2020 09:34:42 +0000 (17:34 +0800)]
mgr/dashboard: right-align dropdown menu of column filters

When the value of a filter is too long, the text of value will run out
of the viewport. Right-aligning the menu makes the text visible.

NOTE: This change can't be backported from master because in master we had
switched the bootstrap library to ng-bootstrap and the menu is already
right-aligned in the master.

Fixes: https://tracker.ceph.com/issues/44458
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
5 years agoqa/tests: changed rbd/test_librbd_python.sh to run on v15.2.4 36332/head
Yuri Weinstein [Tue, 28 Jul 2020 19:40:32 +0000 (12:40 -0700)]
qa/tests: changed rbd/test_librbd_python.sh to run on v15.2.4

Fixes https://tracker.ceph.com/issues/46077
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #35995 from tchaikov/octopus-46408
Yuri Weinstein [Tue, 28 Jul 2020 14:59:52 +0000 (07:59 -0700)]
Merge pull request #35995 from tchaikov/octopus-46408

octopus: mgr: avoid false alarm of MGR_MODULE_ERROR

Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #35957 from smithfarm/wip-46372-octopus
Yuri Weinstein [Tue, 28 Jul 2020 14:59:17 +0000 (07:59 -0700)]
Merge pull request #35957 from smithfarm/wip-46372-octopus

octopus: osd: expose osdspec_affinity to osd_metadata

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #35893 from callithea/wip-46214-octopus
Yuri Weinstein [Tue, 28 Jul 2020 14:56:21 +0000 (07:56 -0700)]
Merge pull request #35893 from callithea/wip-46214-octopus

octopus: mgr/dashboard: Add host labels in UI

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agomgr/dashboard: cpu stats incorrectly displayed 36322/head
Avan Thakkar [Thu, 23 Jul 2020 06:27:32 +0000 (11:57 +0530)]
mgr/dashboard: cpu stats incorrectly displayed

Fixes: https://tracker.ceph.com/issues/46683
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit f039e5585d552c553e37a5f68713cfe2b109b97f)

5 years agoMerge PR #36224 into octopus
Patrick Donnelly [Fri, 24 Jul 2020 18:23:41 +0000 (11:23 -0700)]
Merge PR #36224 into octopus

* refs/pull/36224/head:
mgr/nfs: Remove NParts and Cache_Size from MDCACHE block
doc/cephfs/nfs: Update about deploying nfs-ganesha with cephadm in vstart

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #36265 from sebastian-philipp/octopus-backport-35955-35709-35744...
Sebastian Wagner [Fri, 24 Jul 2020 13:17:56 +0000 (15:17 +0200)]
Merge pull request #36265 from sebastian-philipp/octopus-backport-35955-35709-35744-35991-35838-35471-35862-35901-35456-35846-36152-3599

octopus: cephadm batch backport July (3)

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #36160 from rhcs-dashboard/wip-46602-octopus
Laura Paduano [Fri, 24 Jul 2020 08:35:31 +0000 (10:35 +0200)]
Merge pull request #36160 from rhcs-dashboard/wip-46602-octopus

octopus: mgr/dashboard: fix ui api endpoints

Reviewed-by: Fabrizio D'Angelo <fdangelo@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agomgr/dashboard: Use same required field message accross the UI 36277/head
Volker Theile [Tue, 7 Jul 2020 11:48:50 +0000 (13:48 +0200)]
mgr/dashboard: Use same required field message accross the UI

Some pages use the message 'Required field', but 99% use 'This field is required'. Use the latter everywhere.

Fixes: https://tracker.ceph.com/issues/46395
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit c70e5706652fb14cef078727749d4699aacc20fd)

5 years agocephadm: fix UnboundLocalError: `container_path` 36265/head
Sebastian Wagner [Mon, 20 Jul 2020 11:51:14 +0000 (13:51 +0200)]
cephadm: fix UnboundLocalError: `container_path`

Fixes: https://tracker.ceph.com/issues/46098
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 8532f5331111e2fd5a75df3e19ffb7d6b74c8498)

5 years agocephadm: remove ktdreyer's copr repository
Dimitri Savineau [Fri, 17 Jul 2020 17:38:14 +0000 (13:38 -0400)]
cephadm: remove ktdreyer's copr repository

This was needed as a temporary solution until all missing EL8 packages
are built on EPEL 8.
Because we now have all dependencies on EPEL 8 then we can remove that
repository.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 4cf99eb68e5c5e7e2423905dd83684b03e80b99b)

5 years agocephadm: Fix ipv6 network regex to support routes with expiry
Matthew Oliver [Thu, 16 Jul 2020 06:38:29 +0000 (06:38 +0000)]
cephadm: Fix ipv6 network regex to support routes with expiry

The regex I previously added didn't take into account ipv6 networks that
had an expiry, this patch updates the regex and includes an ipv6 route
version in the test.

Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit f4a986cbffccbf00b499d2d0546ac4623c12ddad)

5 years agomgr/osd_support: add unittests
Joshua Schmid [Thu, 9 Jul 2020 09:40:54 +0000 (11:40 +0200)]
mgr/osd_support: add unittests

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 0fcfc5455dde176bc1bc82fb3176c8be4788c15d)

5 years agoceph: Fix `tell` for cephadm
Sebastian Wagner [Fri, 17 Jul 2020 10:50:07 +0000 (12:50 +0200)]
ceph: Fix `tell` for cephadm

The tell target might contain dots now. Like
`mds.myfs.myhost.rnusmq` is now a valid
target.

Fixes: https://tracker.ceph.com/issues/46560
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d5b8e3d61cd42d23857d7066660e03aecc89d9ce)

5 years agocephadm: also retry on digest mismatch
Sebastian Wagner [Tue, 14 Jul 2020 13:11:46 +0000 (15:11 +0200)]
cephadm: also retry on digest mismatch

Fixes: https://tracker.ceph.com/issues/46534
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ea080e669a83bc2f7478587ef5f4c6a658738a62)

5 years agocephadm: Retry pull on transient error
Sebastian Wagner [Tue, 30 Jun 2020 09:26:58 +0000 (11:26 +0200)]
cephadm: Retry pull on transient error

Fixes: https://tracker.ceph.com/issues/46271
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f3e770f3541c716b4f568cf73255f25243c655e5)

5 years agomgr/cephadm: clean-up service module type checking
Michael Fritch [Fri, 17 Jul 2020 15:26:52 +0000 (09:26 -0600)]
mgr/cephadm: clean-up service module type checking

mypy cast to avoid:

cephadm/module.py: note: In member "_config_fn" of class "CephadmOrchestrator":
cephadm/module.py:1797: error: Incompatible return value type (got "Optional[function]", expected "Optional[Callable[[ServiceSpec], None]]")

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 502c574c36f49947cb7a501cafd5530455451773)

5 years agomgr/cephadm: assert the service_type during config/create
Michael Fritch [Wed, 15 Jul 2020 22:23:59 +0000 (16:23 -0600)]
mgr/cephadm: assert the service_type during config/create

ensure the passed ServiceSpec type matches the CephadmService type

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit f8186bd72f807df3c8a31b8c515d2a9e4bb69d09)

5 years agomgr/cephadm: clean-up service module type checking
Michael Fritch [Mon, 13 Jul 2020 22:22:59 +0000 (16:22 -0600)]
mgr/cephadm: clean-up service module type checking

- convert to py3 style type checks
- other misc type checking fixups

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 2bcdcef8ecc256cd86bc7744d392cda875341cb3)

5 years agomgr/cephadm: clean-up service module type checking
Michael Fritch [Tue, 14 Jul 2020 13:18:48 +0000 (07:18 -0600)]
mgr/cephadm: clean-up service module type checking

- add return type to `config` func
- add missing `NFSServiceSpec` annotation to `config` func

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 414e5507034f6380edac78de86419cd98dc59295)

5 years agomgr/cephadm: clean-up service module type checking
Michael Fritch [Mon, 13 Jul 2020 20:41:57 +0000 (14:41 -0600)]
mgr/cephadm: clean-up service module type checking

- add missing return type to `create` func

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6c020b697e400461760ef41f246e94913923f9df)

5 years agocephadm: fix indentation
Patrick Seidensal [Tue, 7 Jul 2020 12:29:49 +0000 (14:29 +0200)]
cephadm: fix indentation

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 8844ff41b6547d82c6f900493e830de8686660bc)

5 years agocephadm: remove unused/commented out code
Patrick Seidensal [Tue, 7 Jul 2020 12:29:12 +0000 (14:29 +0200)]
cephadm: remove unused/commented out code

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 6aa87f7b0dfd516439a6c75681e20d7dffacaf4d)

5 years agocephadm: fix creating custom Prometheus image
Patrick Seidensal [Tue, 7 Jul 2020 11:58:27 +0000 (13:58 +0200)]
cephadm: fix creating custom Prometheus image

Fix the code by improving a condition where a zero as uid or gid leads
to an additional attempt to extract the uid/gid instead of uid/gid
simply being used correctly.

Fixes: https://tracker.ceph.com/issues/46398
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 3c98f1553fe3b91777ee81d887d02462132475fb)

5 years agomgr/cephadm: Add CEPHADM_PAUSED to test_cli
Sebastian Wagner [Tue, 30 Jun 2020 13:13:51 +0000 (15:13 +0200)]
mgr/cephadm: Add CEPHADM_PAUSED to test_cli

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit aee1aed64131e615bbf5b9453f52fec5eef36bf9)

5 years agomgr/cephadm: Set exception context to populate orch events.
Sebastian Wagner [Sat, 6 Jun 2020 23:25:21 +0000 (01:25 +0200)]
mgr/cephadm: Set exception context to populate orch events.

Like when if daemon deployment fails

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1dd2c5c4fbce49495736c12c3aea2c96f6ad4696)

5 years agomgr/cephadm: Add event when deploying a daemon
Sebastian Wagner [Sat, 6 Jun 2020 22:54:35 +0000 (00:54 +0200)]
mgr/cephadm: Add event when deploying a daemon

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 960f6bfac0d18eb92a017ea21ca97efa1a12b7cf)

5 years agomgr/cephadm: Add an event when creating a service
Sebastian Wagner [Sat, 6 Jun 2020 22:53:34 +0000 (00:53 +0200)]
mgr/cephadm: Add an event when creating a service

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 05ade49a0900ae0714a8bfe53a8b993d46d5be50)

5 years agomgr/cephadm: Add inventory.EventStore
Sebastian Wagner [Sat, 6 Jun 2020 22:51:38 +0000 (00:51 +0200)]
mgr/cephadm: Add inventory.EventStore

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f5b0155e04ab23beecea5d832a59d4e14cd4e898)

5 years agomgr/orch: Add events to tests/test_orchestrator
Sebastian Wagner [Fri, 26 Jun 2020 10:44:44 +0000 (12:44 +0200)]
mgr/orch: Add events to tests/test_orchestrator

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 83c163683d3da644de9985c4da0f94ca31b48004)

5 years agomgr/orch: Add OrchestratorEvent class
Sebastian Wagner [Sat, 6 Jun 2020 22:49:43 +0000 (00:49 +0200)]
mgr/orch: Add OrchestratorEvent class

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b9ff9118470a1a0cfa94bf3ea4a3d341b25a6bd8)

5 years agovstart: Remove old ceph images if new cluster is deployed
Varsha Rao [Tue, 14 Jul 2020 10:26:07 +0000 (15:56 +0530)]
vstart: Remove old ceph images if new cluster is deployed

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit d8c6e0d97b20be8a0feb53ecaaee0caf16c812ff)

5 years agocephadm: refactor update_firewalld
Sebastian Wagner [Thu, 2 Jul 2020 14:28:14 +0000 (16:28 +0200)]
cephadm: refactor update_firewalld

* split into three different methods
* untangle service and port lists

This helps in making our firewall handling more dynamic.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Co-authored-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 4f341819ec746d0a04b76e88b84b64158905e5b6)

5 years agomgr/cephadm: check-host should not fail as hard using fqdn
Adam King [Tue, 30 Jun 2020 21:49:09 +0000 (17:49 -0400)]
mgr/cephadm: check-host should not fail as hard using fqdn

Print error message instead of traceback when check-host
fails in due to host address not being found

Fixes: https://tracker.ceph.com/issues/45724
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit fb1ebb9dc1071a96b3b2e4e85be5744e4a18416c)

5 years agomgr/cephadm: Add test for _check_daemons()
Sebastian Wagner [Thu, 18 Jun 2020 12:07:54 +0000 (14:07 +0200)]
mgr/cephadm: Add test for _check_daemons()

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1aeb0be49b3dc26f456da5d0d850b1d68aec88cf)

5 years agomgr/cephadm: refactor `_create_daemon()`
Sebastian Wagner [Mon, 8 Jun 2020 10:45:14 +0000 (12:45 +0200)]
mgr/cephadm: refactor `_create_daemon()`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 69ed3d9cffda243275ab7c9e89d5f9a615f2ab98)

5 years agomgr/cephadm: Add CephadmDaemonSpec class
Sebastian Wagner [Mon, 8 Jun 2020 07:36:20 +0000 (09:36 +0200)]
mgr/cephadm: Add CephadmDaemonSpec class

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7bc22f61cd69dcb5e7ee2aaa275f9f8cef3b3b26)

5 years agopython-common: RGW service_id might not contain a zone
Michael Fritch [Wed, 1 Jul 2020 14:27:02 +0000 (08:27 -0600)]
python-common: RGW service_id might not contain a zone

also adds missing RGWSpec.validate()

Fixes: https://tracker.ceph.com/issues/46268
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b54a800b748dc94cb847cc44e23b481c8c3a85b4)

5 years agodoc/cephadm: remove a confusing sentence fragment
Zac Dover [Thu, 11 Jun 2020 19:11:09 +0000 (05:11 +1000)]
doc/cephadm: remove a confusing sentence fragment

Removed a confusing sentence fragment: "For each file system:"

Fixes: https://tracker.ceph.com/issues/46546
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Co-authored-by: Varsha <rvarsha016@gmail.com>
(cherry picked from commit dcce296eea99fa025135d68f079008421de14fa3)

5 years agomgr/cephadm: Adapt Vagrantbox to el8
Volker Theile [Thu, 9 Jul 2020 10:41:49 +0000 (12:41 +0200)]
mgr/cephadm: Adapt Vagrantbox to el8

After el7 has been dropped with https://github.com/ceph/ceph/pull/35722 the Vagrantbox need to be adapted to el8.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit d8623ff67bb57ecb3e7b0ae814ea21f1fda54c9b)

Conflicts:
src/pybind/mgr/cephadm/Vagrantfile

5 years agodoc/mgr/orchestrator: Add information about unmanaged parameter
Juan Miguel Olmo Martínez [Wed, 24 Jun 2020 09:57:51 +0000 (11:57 +0200)]
doc/mgr/orchestrator: Add information about unmanaged parameter

Explain parameter 'unmanaged' in OSDs creation and deletion

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit da62f752c5c233b5bac687ac6be5e00fe35084b9)

5 years agodoc/cephadm: add yaml infor for orch apply mon
Zac Dover [Tue, 14 Jul 2020 13:51:08 +0000 (23:51 +1000)]
doc/cephadm: add yaml infor for orch apply mon

This commit breaks one of the long sentences into
three shorter sentences, and adds a parenthetical
comment walking the reader through a series of commands,
explaining what each command does and why only the last
of the commands ends up having a lasting effect on the Ceph
environment.

Fixes: https://tracker.ceph.com/issues/46133
Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit b02c98bfa5f0ebe442abd97fab16d65b0ebfbf83)

5 years agomgr/cephadm: do not accept filestore osds in osdspecs
Joshua Schmid [Tue, 7 Jul 2020 08:33:51 +0000 (10:33 +0200)]
mgr/cephadm: do not accept filestore osds in osdspecs

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 4974a2c1b8e67426db12b7bb2ea3d1af892ca16e)

5 years agodoc: add note about filter_logic
Joshua Schmid [Mon, 6 Jul 2020 09:40:36 +0000 (11:40 +0200)]
doc: add note about filter_logic

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 73475f3b396bf7892b921985b5b4fcf3e9732f42)

5 years agopython-common/drivegroups: implement option to change filter_logic
Joshua Schmid [Wed, 1 Jul 2020 09:10:14 +0000 (11:10 +0200)]
python-common/drivegroups: implement option to change filter_logic

Filters are applied to disks using an AND gate by default. In some
scenarios you can only achieve the desired behavior when using an OR
gate without expanding the available filters.

This patch allows to switch between `AND` and `OR` gates.

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit f00fba87925f6f72448128649694d98b4df3e748)

5 years agomgr/cephadm: also list vendor/model/rotational in device ls
Joshua Schmid [Mon, 6 Jul 2020 15:41:25 +0000 (17:41 +0200)]
mgr/cephadm: also list vendor/model/rotational in device ls

Fixes: https://tracker.ceph.com/issues/45872
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 7c18893f94b01f10aa35ebcec13ee3a3a785c94a)

5 years agoMerge pull request #35880 from ceph/wip-octopus-34744
Jan Fajerski [Thu, 23 Jul 2020 07:42:17 +0000 (09:42 +0200)]
Merge pull request #35880 from ceph/wip-octopus-34744

octopus: ceph-volume: handle idempotency with batch and explicit scenarios

5 years agorbd: use new 'write_zeroes' API to ensure zeroed extents will be zeroed 36247/head
Jason Dillaman [Tue, 30 Jun 2020 12:47:48 +0000 (08:47 -0400)]
rbd: use new 'write_zeroes' API to ensure zeroed extents will be zeroed

Fixes: https://tracker.ceph.com/issues/45986
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f319ecfbec3486917824eab84ab76347319b8bde)

5 years agopybind/rbd: add new write_zeroes/aio_write_zeroes API methods
Jason Dillaman [Tue, 30 Jun 2020 15:20:31 +0000 (11:20 -0400)]
pybind/rbd: add new write_zeroes/aio_write_zeroes API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c27d744a339d3644b756ee496445909f63c8903e)

5 years agolibrbd: add 'write_zeroes' public C/C++ API methods
Jason Dillaman [Tue, 30 Jun 2020 12:44:38 +0000 (08:44 -0400)]
librbd: add 'write_zeroes' public C/C++ API methods

Unlike the existing 'discard' option which is more of a hint to
attempt to release space, the new 'write_zeroes' APIs will ensure
that the entire provided extent is fully zeroed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ae6dd86b22e928dc23a385faf41cf76b0d293576)

Conflicts:
src/librbd/api/Io.h/cc: logic exists in ImageRequestWQ files
src/librbd/librbd.cc: trivial resolution due missing api::Io
src/test/librbd/test_librbd.cc: trivial resolution

5 years agoMerge pull request #36137 from rhcs-dashboard/wip-46570-octopus
Yuri Weinstein [Wed, 22 Jul 2020 16:05:25 +0000 (09:05 -0700)]
Merge pull request #36137 from rhcs-dashboard/wip-46570-octopus

octopus: mgr/dashboard: fix pool usage calculation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #36049 from smithfarm/wip-46009-octopus
Yuri Weinstein [Wed, 22 Jul 2020 16:04:34 +0000 (09:04 -0700)]
Merge pull request #36049 from smithfarm/wip-46009-octopus

octopus: tests: objectstore/store_test: kill ExcessiveFragmentation test case.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>