]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agoMerge pull request #36195 from smithfarm/wip-46591-octopus
Yuri Weinstein [Tue, 25 Aug 2020 22:09:03 +0000 (15:09 -0700)]
Merge pull request #36195 from smithfarm/wip-46591-octopus

octopus: common:  ignore SIGHUP prior to fork

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36126 from ShyamsundarR/wip-46528-octopus
Yuri Weinstein [Tue, 25 Aug 2020 22:08:20 +0000 (15:08 -0700)]
Merge pull request #36126 from ShyamsundarR/wip-46528-octopus

octopus: mgr/volumes: Deprecate protect/unprotect CLI calls for subvolume snapshots

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36100 from smithfarm/wip-46536-octopus
Yuri Weinstein [Tue, 25 Aug 2020 22:07:47 +0000 (15:07 -0700)]
Merge pull request #36100 from smithfarm/wip-46536-octopus

octopus: pybind/ceph_volume_client: Fix PEP-8 SyntaxWarning

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #35962 from smithfarm/wip-46389-octopus
Yuri Weinstein [Tue, 25 Aug 2020 22:06:56 +0000 (15:06 -0700)]
Merge pull request #35962 from smithfarm/wip-46389-octopus

octopus: client: introduce timeout for client shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #36621 from smithfarm/wip-46193-octopus
Yuri Weinstein [Tue, 25 Aug 2020 18:55:01 +0000 (11:55 -0700)]
Merge pull request #36621 from smithfarm/wip-46193-octopus

octopus: os/bluestore: fix bluefs log growth

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #36748 from sebastian-philipp/octopus-backport-36371-36446-36349...
Sebastian Wagner [Mon, 24 Aug 2020 11:47:53 +0000 (13:47 +0200)]
Merge pull request #36748 from sebastian-philipp/octopus-backport-36371-36446-36349-36463-36444-36426-36447-36373-36492-36497-36538-3645

octopus: cephadm batch backport August (2)

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
4 years agoMerge pull request #36668 from smithfarm/wip-46629-octopus
Yuri Weinstein [Sat, 22 Aug 2020 18:10:24 +0000 (11:10 -0700)]
Merge pull request #36668 from smithfarm/wip-46629-octopus

octopus: bluestore: blk:BlockDevice.cc: use pending_aios instead of iovec size as ios num

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #36667 from smithfarm/wip-46964-octopus
Yuri Weinstein [Sat, 22 Aug 2020 18:09:16 +0000 (11:09 -0700)]
Merge pull request #36667 from smithfarm/wip-46964-octopus

octopus: mgr: decrease pool stats if pg was removed

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #36666 from smithfarm/wip-46934-octopus
Yuri Weinstein [Sat, 22 Aug 2020 18:08:23 +0000 (11:08 -0700)]
Merge pull request #36666 from smithfarm/wip-46934-octopus

octopus: tools/rados: Set locator key when exporting or importing a pool

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36665 from smithfarm/wip-46739-octopus
Yuri Weinstein [Sat, 22 Aug 2020 18:06:57 +0000 (11:06 -0700)]
Merge pull request #36665 from smithfarm/wip-46739-octopus

octopus: mon: fix the 'Error ERANGE' message when conf "osd_objectstore" is filestore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #36664 from smithfarm/wip-46722-octopus
Yuri Weinstein [Sat, 22 Aug 2020 18:06:06 +0000 (11:06 -0700)]
Merge pull request #36664 from smithfarm/wip-46722-octopus

octopus: osd/OSD.cc: remove osd_lock for bench

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
4 years agoMerge pull request #36663 from smithfarm/wip-46709-octopus
Yuri Weinstein [Sat, 22 Aug 2020 18:04:59 +0000 (11:04 -0700)]
Merge pull request #36663 from smithfarm/wip-46709-octopus

octopus: osd/PeeringState: prevent peer's num_objects going negative

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #36633 from smithfarm/wip-46951-octopus
Yuri Weinstein [Fri, 21 Aug 2020 19:47:31 +0000 (12:47 -0700)]
Merge pull request #36633 from smithfarm/wip-46951-octopus

octopus: monclient: schedule first tick using mon_client_hunt_interval

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #36643 from smithfarm/wip-46931-octopus
Yuri Weinstein [Fri, 21 Aug 2020 18:30:07 +0000 (11:30 -0700)]
Merge pull request #36643 from smithfarm/wip-46931-octopus

octopus: librados: add LIBRADOS_SUPPORTS_GETADDRS support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36630 from smithfarm/wip-46958-octopus
Yuri Weinstein [Fri, 21 Aug 2020 18:28:48 +0000 (11:28 -0700)]
Merge pull request #36630 from smithfarm/wip-46958-octopus

octopus: mgr/status: metadata is fetched async

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
4 years agoMerge pull request #36611 from tspmelo/wip-46924-octopus
Yuri Weinstein [Fri, 21 Aug 2020 18:26:15 +0000 (11:26 -0700)]
Merge pull request #36611 from tspmelo/wip-46924-octopus

octopus: mgr/dashboard: Unable to edit iSCSI logged-in client

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #36577 from callithea/wip-46715-octopus
Yuri Weinstein [Fri, 21 Aug 2020 18:25:25 +0000 (11:25 -0700)]
Merge pull request #36577 from callithea/wip-46715-octopus

octopus: mgr/diskprediction_local: Fix array size error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #36717 from smithfarm/wip-47022-octopus
Yuri Weinstein [Fri, 21 Aug 2020 14:08:01 +0000 (07:08 -0700)]
Merge pull request #36717 from smithfarm/wip-47022-octopus

octopus: rbd: librbd: Align rbd_write_zeroes declarations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #36678 from bk201/wip-46944-octopus
Yuri Weinstein [Fri, 21 Aug 2020 14:07:25 +0000 (07:07 -0700)]
Merge pull request #36678 from bk201/wip-46944-octopus

octopus: mgr/dashboard: Fix host attributes like labels are not returned

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #36581 from callithea/wip-46717-octopus
Yuri Weinstein [Fri, 21 Aug 2020 14:06:12 +0000 (07:06 -0700)]
Merge pull request #36581 from callithea/wip-46717-octopus

octopus: mgr/prometheus: log time it takes to collect metrics

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
5 years agoMerge pull request #36302 from dillaman/wip-46724-octopus
Yuri Weinstein [Fri, 21 Aug 2020 14:05:16 +0000 (07:05 -0700)]
Merge pull request #36302 from dillaman/wip-46724-octopus

octopus: ceph-iscsi: selinux fixes

Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoqa/tasks/nfs: Instead of 'orch ls' use 'orch ps' 36748/head
Varsha Rao [Thu, 20 Aug 2020 14:36:47 +0000 (14:36 +0000)]
qa/tasks/nfs: Instead of 'orch ls' use 'orch ps'

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 5c4e659067fc2022e1c226bb3c9f72fb2e018b44)

5 years agoqa/suites/rados/test_orch_cli: Change mds name
Varsha Rao [Thu, 20 Aug 2020 14:28:57 +0000 (14:28 +0000)]
qa/suites/rados/test_orch_cli: Change mds name

As MDS name should not start with numeric digit.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 572f0aeceb01981cab01423250cf5694404d421a)

5 years agoqa/tasks/nfs: Delete nfs cluster after exports are created
Varsha Rao [Thu, 20 Aug 2020 12:09:33 +0000 (12:09 +0000)]
qa/tasks/nfs: Delete nfs cluster after exports are created

Fixes: https://tracker.ceph.com/issues/47009
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 049afe5a7e6c3ccfaf29407111ff02d6ca5ee9e3)

5 years agomgr/cephadm: use fixed config file path as argument
Patrick Seidensal [Wed, 5 Aug 2020 14:47:13 +0000 (16:47 +0200)]
mgr/cephadm: use fixed config file path as argument

Fixes other alertmanager containers which do not default to look for a
configuration file in `/etc/alertmanager/alertmanager.yml`.

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit cdd0ba13a5109e4319ffdaff8e8a17212e78dd15)

5 years agocephadm: fix custom alertmanager
Patrick Seidensal [Thu, 30 Jul 2020 09:37:46 +0000 (11:37 +0200)]
cephadm: fix custom alertmanager

When alertmanager image has been customized and does not have a
/etc/alertmanager directory, the uid/gid extraction fails. This change
assumes that instead the /etc/prometheus directory exists and can be
used to extract the uid and gid.

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit d218de4cde8d37f7f69a2a2a9474c0ac9afca41d)

5 years agocephadm: Allow users to use a custom dashboard ssl port
Juan Miguel Olmo Martínez [Wed, 19 Aug 2020 10:11:28 +0000 (12:11 +0200)]
cephadm: Allow users to use a custom dashboard ssl port

This modification allows the user to create a new bootstrap cluster using a predefined SSl port for the dashboard.
If firewall is enabled, any new manager daemon deployed in new hosts will take care of open the required ports for all the services enabled in the manager.

Two new parameters for cephadm tool (aka binary or standalone):
Command bootstrap:
--ssl-dashboard-port SSL_DASHBOARD_PORT
                      Port number used to connect with dashboard using SSL
Command deploy:
--tcp-ports TCP_PORTS
                      List of tcp ports to open in the host firewall

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 08c4a53e79a7ed63912f765fbbf86e3364d8f9be)

5 years agomgr/cephadm: mgr or mds scale-down should prefer non-active daemons
Adam King [Wed, 5 Aug 2020 21:04:46 +0000 (17:04 -0400)]
mgr/cephadm: mgr or mds scale-down should prefer non-active daemons

When removing daemons during a mgr/mds scale-down, prefer to remove
standby daemons so the active daemon is not killed

Fixes: https://tracker.ceph.com/issues/44252
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 0fbf12c9d17a98bf4315b0bff81c0aeb38458e0f)

5 years agodoc/cephadm: remove ssh user limitation troubleshooting
Daniel-Pivonka [Tue, 18 Aug 2020 20:46:35 +0000 (16:46 -0400)]
doc/cephadm: remove ssh user limitation troubleshooting

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit b59b480c3d6f0bcceff155c02770acb2ea842561)

5 years agopython-common: ban paths from non-data-devices
Joshua Schmid [Mon, 10 Aug 2020 11:09:25 +0000 (13:09 +0200)]
python-common: ban paths from non-data-devices

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit ac5839b9f5d4ff2c4269b9da0314837cdafa5360)

5 years agodocs/mgr: remove traces of ceph orch osd spec
Joshua Schmid [Tue, 11 Aug 2020 14:47:36 +0000 (16:47 +0200)]
docs/mgr: remove traces of ceph orch osd spec

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit f8b8e6f22900848ed12d166592f504e2d6b70554)

5 years agomgr/orchestrator: remove traces of ceph orch osd spec
Joshua Schmid [Tue, 11 Aug 2020 14:47:15 +0000 (16:47 +0200)]
mgr/orchestrator: remove traces of ceph orch osd spec

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 4109e981d969c5c61f6dbf66e1e8c3c17724f5f8)

5 years agomgr/cephadm: add default osdspec_affinity to module fixture
Joshua Schmid [Tue, 11 Aug 2020 07:53:23 +0000 (09:53 +0200)]
mgr/cephadm: add default osdspec_affinity to module fixture

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit eb1b37cfc34db90db86ef8449f645d5a192656b9)

5 years agomgr/cephadm: save osdspec_affinity to dd store
Joshua Schmid [Mon, 10 Aug 2020 10:38:00 +0000 (12:38 +0200)]
mgr/cephadm: save osdspec_affinity to dd store

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 0feae8893364729f2e9b9ee9bc53b6eba66cbc10)

5 years agomgr/cephadm: add by_type daemon retrieval method
Joshua Schmid [Wed, 12 Aug 2020 10:23:59 +0000 (12:23 +0200)]
mgr/cephadm: add by_type daemon retrieval method

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 1f8965f8cc48c9e901f3531259d446e3d9211135)

5 years agomgr/cephadm: also mock `ceph vesions`
Sebastian Wagner [Mon, 17 Aug 2020 11:07:03 +0000 (13:07 +0200)]
mgr/cephadm: also mock `ceph vesions`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1c8cb204ffdb654a2b954a3d39a1c1712c151684)

5 years agomgr/cephadm: add test_upgrade.py
Sebastian Wagner [Mon, 17 Aug 2020 10:34:02 +0000 (12:34 +0200)]
mgr/cephadm: add test_upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d71f3b42738eef2d234ec752bc27d3e8e53e744c)

5 years agomgr/cephadm/tests: move with_service to fixtures.py
Sebastian Wagner [Mon, 17 Aug 2020 09:11:21 +0000 (11:11 +0200)]
mgr/cephadm/tests: move with_service to fixtures.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 50ecd89d790fb33a7dc297ae1eeb7751dd58cba2)

5 years agopybind/mgr: Also mock the config store in pytest
Sebastian Wagner [Fri, 7 Aug 2020 14:21:18 +0000 (16:21 +0200)]
pybind/mgr: Also mock the config store in pytest

This will be used to dry-run mgr/cephadm/upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f44c1e71033f8f62e3bca17d274049b858e7b015)

5 years agomgr/cephadm: check existance of the RADOS pool
Michael Fritch [Thu, 6 Aug 2020 21:39:05 +0000 (15:39 -0600)]
mgr/cephadm: check existance of the RADOS pool

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4419c18f9fb961b54715c355194b85e57804adec)

5 years agomgr/cephadm: convert py2 -> py3 type checking
Michael Fritch [Fri, 7 Aug 2020 00:24:56 +0000 (18:24 -0600)]
mgr/cephadm: convert py2 -> py3 type checking

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit e05bf091724e318b547967c6ec81ed5cf5807b16)

5 years agomgr/cephadm: clean-up nfs service module imports
Michael Fritch [Fri, 7 Aug 2020 00:30:24 +0000 (18:30 -0600)]
mgr/cephadm: clean-up nfs service module imports

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4ad0636e81d1005c116ef6aae211e6fcead23ed4)

5 years agoqa/tasks/cephfs/nfs: Add tests for cluster config set and reset
Varsha Rao [Fri, 31 Jul 2020 10:27:28 +0000 (15:57 +0530)]
qa/tasks/cephfs/nfs: Add tests for cluster config set and reset

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit bba2d66176fa30b934aaa6fb7670cd0fecc0af78)

5 years agodoc/cephfs/nfs: Update the doc about 'reset' and 'set' config interfaces
Varsha Rao [Wed, 29 Jul 2020 13:48:06 +0000 (19:18 +0530)]
doc/cephfs/nfs: Update the doc about 'reset' and 'set' config interfaces

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 287da119215671fe24b371f9df01059569db4b3e)

5 years agomgr/volumes/nfs: Add interface for adding user defined configuration
Varsha Rao [Mon, 27 Jul 2020 12:56:45 +0000 (18:26 +0530)]
mgr/volumes/nfs: Add interface for adding user defined configuration

Code related to creating and removing rados object has been rearranged abd
modified. This patch introduces interface to set and reset user config.

nfs cluster config set <clusterid> -i <config_file>
nfs cluster config reset <clusterid>

Fixes: https://tracker.ceph.com/issues/45747
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 9535154794c1d6d116b81c1317c48ba1aecd7581)

5 years agomgr/orch: _interface.py: PEP8tify
Sebastian Wagner [Tue, 4 Aug 2020 15:49:01 +0000 (17:49 +0200)]
mgr/orch: _interface.py: PEP8tify

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b7caeed9b694a3191f06d96339a81f5d99b0ab88)

5 years agomgr/orch: _interface.py: Convert type annotations to py3
Sebastian Wagner [Tue, 4 Aug 2020 15:47:05 +0000 (17:47 +0200)]
mgr/orch: _interface.py: Convert type annotations to py3

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0bb09324e7b4cffc957a89c3c5188bdb823176aa)

5 years agodoc/cephadm: Remove shell alias
Sebastian Wagner [Mon, 27 Jul 2020 14:44:33 +0000 (16:44 +0200)]
doc/cephadm: Remove shell alias

Turns out, the alias doesn't work for files.

Fixes: https://tracker.ceph.com/issues/46701
Fixes:

```
$ ceph orch apply -i myfile.yaml
ERROR: no such file or directory: myfile.yaml
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 75baae35288cca79a6e958c3370dba3caab813f8)

5 years agomgr/cephadm: _get_config_and_keyring: `host` is required now
Sebastian Wagner [Fri, 7 Aug 2020 09:24:52 +0000 (11:24 +0200)]
mgr/cephadm: _get_config_and_keyring: `host` is required now

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 6105d1ba4c94e1278bf38896ad467a5cf7e5b92b)

5 years agomgr/cephadm: Some more typing
Sebastian Wagner [Thu, 6 Aug 2020 15:44:00 +0000 (17:44 +0200)]
mgr/cephadm: Some more typing

Make sure we don't mix daemon names, config entities and
auth entities. Cause all of them don't mix well together.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 01e2dafb6e5b7cf3f26a63c2c66d346a1ab0bffa)

5 years agomgr/cephadm: Prevent unnecessary SSH reconfiguration
Ricardo Marques [Thu, 6 Aug 2020 09:47:29 +0000 (10:47 +0100)]
mgr/cephadm: Prevent unnecessary SSH reconfiguration

When 'set-user', 'set-pub-key' or 'set-priv-key' are called two
times with the same value, we can ignore the second call.

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 4cbf587b686d3b539f98b5c4cab1f0320efbe68c)

5 years agomgr/orchestrator: Update service_action() docstring
Varsha Rao [Thu, 30 Jul 2020 08:27:26 +0000 (13:57 +0530)]
mgr/orchestrator: Update service_action() docstring

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 4d6365b6a9a9bda4375b84ad3661e43e5de9abf8)

5 years agomgr/cephadm: module.py: PEP8tify `ceph orch * ls`
Sebastian Wagner [Wed, 5 Aug 2020 10:50:01 +0000 (12:50 +0200)]
mgr/cephadm: module.py: PEP8tify `ceph orch * ls`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit be3f513239d45feb8557f30e05fc562ce9594f51)

5 years agomgr/cephadm: Make `ceph orch * --refresh` asynchronous
Sebastian Wagner [Tue, 4 Aug 2020 12:01:49 +0000 (14:01 +0200)]
mgr/cephadm: Make `ceph orch * --refresh` asynchronous

There is just no way we can allow those calls to be synchornous:

* All CLI commands are blocked, till the refresh completes.
* ^C doesn't help at all, as it just kills the client

Therefore they have to be scheduled in the bg.

Fixes: https://tracker.ceph.com/issues/46813
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7189575404262c0fbb4717f824c022d48ab58fcd)

5 years agomgr/cephadm: Refactor upgrade to call _daemon_action with new image
Sebastian Wagner [Mon, 3 Aug 2020 09:35:41 +0000 (11:35 +0200)]
mgr/cephadm: Refactor upgrade to call _daemon_action with new image

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 289f37d29674b9a26415147a964b1fcb26a7eaf7)

5 years agomgr/cephadm, Add --image to `ceph orch daemon redeploy <what>`
Sebastian Wagner [Fri, 31 Jul 2020 16:02:34 +0000 (18:02 +0200)]
mgr/cephadm, Add --image to `ceph orch daemon redeploy <what>`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 9d0480d18b6f30f929e8599ee8b32fdc6f91aa27)

5 years agomgr/cephadm: rgw update period after realm, zone creation
Daniel-Pivonka [Wed, 5 Aug 2020 21:50:12 +0000 (17:50 -0400)]
mgr/cephadm: rgw update period after realm, zone creation

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit 70b11a1a83d2d03062d7a2db81de8675b2f0b0f4)

5 years agomgr/cephadm: PEP8tify upgrade.py
Sebastian Wagner [Tue, 4 Aug 2020 09:13:49 +0000 (11:13 +0200)]
mgr/cephadm: PEP8tify  upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ffc1fd65c6b82a5565953f0da184a0b218d7ca24)

5 years agomgr/cephadm: Add UpgradeState class
Sebastian Wagner [Mon, 3 Aug 2020 13:51:53 +0000 (15:51 +0200)]
mgr/cephadm: Add UpgradeState class

My resons are:

* This increase mypy coverage
* Makes the code more discoverable.
* Allowes us to have intelligent UpgradeState properties

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 3072b1d8cbc8b5ac83697007767e12ff805a693d)

5 years agocephadm: don't add `ceph-volume lvm activate` for adopted simple OSDs
Tim Serong [Wed, 5 Aug 2020 06:34:20 +0000 (16:34 +1000)]
cephadm: don't add `ceph-volume lvm activate` for adopted simple OSDs

This changes the logic in deploy_daemon_units() to add either `chown` calls
for simple (ceph-disk style) OSDs, or to add `ceph-volume lvm activate` calls
for LVM OSDs, rather than always adding both.  When I was working on
https://github.com/ceph/ceph/pull/34703, I'd originally added an "osd_simple"
flag to figure out what type of OSD was being adopted/deployed, but passing
that around was kinda ugly, so was removed from that PR.  This time around
I'm checking if /etc/ceph/osd/$OSD_ID-$OSD_FSID.json.adopted-by-cephadm
exists, which seems pretty safe IMO.  My only concern with this method is:
what happens if someone adopts a simple OSD, then later wants to migrate it
to LVM.  Presumably that's a destroy and recreate, keeping the same OSD ID?
If that's true, then the JSON file probably still exists, so the subsequent
create will do the wrong thing, i.e. will add `chown` calls, not `ceph-volume
lvm activate` calls.  Any/all feedback appreciated...

Fixes: https://tracker.ceph.com/issues/46833
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 8112949851eb4849f0d559a118739d92051dcf8e)

5 years agomgr/caphadm/tests: Add `test_daemon_check_post`
Sebastian Wagner [Thu, 30 Jul 2020 11:47:26 +0000 (13:47 +0200)]
mgr/caphadm/tests: Add `test_daemon_check_post`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7d3deb2e23266ee5e206d76bb4ce33b436ac5ec2)

5 years agomgr/cephadm: Don't run `daemons_post` if _create_daemon fail
Sebastian Wagner [Thu, 30 Jul 2020 11:38:45 +0000 (13:38 +0200)]
mgr/cephadm: Don't run `daemons_post` if _create_daemon fail

We can't run daemon_check_post for the type, if a single daemon failed.
Mainly cause `daemon_check_post` is run by service type.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit fdaa00977e3d9ee3bd1c6f5be69a0e93d6825136)

5 years agomgr/cephadm: Add test_daemon_action_fail
Sebastian Wagner [Wed, 29 Jul 2020 12:23:06 +0000 (14:23 +0200)]
mgr/cephadm: Add test_daemon_action_fail

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f500b8fc747f1767a31e54e6fb4a9a596889d1a1)

5 years agomgr/cephadm/tests: Split `test_daemon_action`
Sebastian Wagner [Wed, 29 Jul 2020 12:09:00 +0000 (14:09 +0200)]
mgr/cephadm/tests: Split `test_daemon_action`

Into:

* `test_list_daemons`
* `test_daemon_action` (now without listing daemons)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 6fd599bd4d66c290ba94accd76c869a50a9dd5ce)

5 years agomgr/cephadm: guard _check_daemons
Sebastian Wagner [Wed, 29 Jul 2020 10:55:04 +0000 (12:55 +0200)]
mgr/cephadm: guard _check_daemons

Continue with other daemons, if one fails

Fixes: https://tracker.ceph.com/issues/46748
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 313e091fee6b3399852683575f7431c1ba235133)

5 years agomgr/cephadm: PEP8tify
Sebastian Wagner [Tue, 4 Aug 2020 10:25:33 +0000 (12:25 +0200)]
mgr/cephadm: PEP8tify

Those files should generate fewer conflicts for now.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 17db7ba7dfb5ad617ec2b3f81a37c6fdc69f9222)

5 years agomgr/cephadm: Enhance AlertManagerSpec to allow adding additional webhook receiver...
Volker Theile [Thu, 30 Jul 2020 12:24:34 +0000 (14:24 +0200)]
mgr/cephadm: Enhance AlertManagerSpec to allow adding additional webhook receiver URLs

Fixes: https://tracker.ceph.com/issues/46775
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 4b87eeb97861929130c4aa7e2c99e29d2a1652ac)

5 years agoMerge pull request #36260 from rhcs-dashboard/wip-46672-octopus
Lenz Grimmer [Fri, 21 Aug 2020 09:13:32 +0000 (11:13 +0200)]
Merge pull request #36260 from rhcs-dashboard/wip-46672-octopus

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agolibrbd: Align rbd_write_zeroes declarations 36717/head
Corey Bryant [Mon, 17 Aug 2020 12:19:25 +0000 (08:19 -0400)]
librbd: Align rbd_write_zeroes declarations

Fixes: https://tracker.ceph.com/issues/46928
Signed-off-by: Corey Bryant <corey.bryant@canonical.com>
(cherry picked from commit 32fcd679132fab93ea50ec791f559ad2b09d8a2b)

5 years agoselinux: Fix ceph-iscsi etc access 36302/head
Mike Christie [Thu, 9 Jan 2020 00:37:15 +0000 (18:37 -0600)]
selinux: Fix ceph-iscsi etc access

This fixes the selinux errors like this for /etc/target

-----------------------------------
Additional Information:
Source Context                system_u:system_r:ceph_t:s0
Target Context                system_u:object_r:targetd_etc_rw_t:s0
Target Objects                target [ dir ]
Source                        rbd-target-api
Source Path                   rbd-target-api
Port                          <Unknown>
Host                          ans8
Source RPM Packages
Target RPM Packages
Policy RPM                    selinux-policy-3.14.3-20.el8.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     ans8
Platform                      Linux ans8 4.18.0-147.el8.x86_64 #1 SMP
Thu Sep 26
                              15:52:44 UTC 2019 x86_64 x86_64
Alert Count                   1
First Seen                    2020-01-08 18:39:48 EST
Last Seen                     2020-01-08 18:39:48 EST
Local ID                      9a13ee18-eaf2-4f2a-872f-2809ee4928f6

Raw Audit Messages
type=AVC msg=audit(1578526788.148:69): avc:  denied  { search } for
pid=995 comm="rbd-target-api" name="target" dev="sda1" ino=52198
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:targetd_etc_rw_t:s0 tclass=dir permissive=1

Hash: rbd-target-api,ceph_t,targetd_etc_rw_t,dir,search

which are a result of the rtslib library the ceph-iscsi daemons use
accessing /etc/target to read/write a file which stores meta data the
target uses.

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 53be18165323a80895a34185df4ad9e8d37db618)

5 years agoselinux: Fix ceph-iscsi configfs access
Mike Christie [Thu, 9 Jan 2020 00:03:40 +0000 (18:03 -0600)]
selinux: Fix ceph-iscsi configfs access

This fixes the the following selinux error when using ceph-iscsi's
rbd-target-api daemon (rbd-target-gw has the same issue). They are
a result of the a python library, rtslib, which the daemons use.

Additional Information:
Source Context                system_u:system_r:ceph_t:s0
Target Context                system_u:object_r:configfs_t:s0
Target Objects
/sys/kernel/config/target/iscsi/iqn.2003-01.com.re
                              dhat:ceph-iscsi/tpgt_1/attrib/authentication
[
                              file ]
Source                        rbd-target-api
Source Path                   /usr/libexec/platform-python3.6
Port                          <Unknown>
Host                          ans8
Source RPM Packages           platform-python-3.6.8-15.1.el8.x86_64
Target RPM Packages
Policy RPM                    selinux-policy-3.14.3-20.el8.noarch
Selinux Enabled               True
Policy Type                   targeted
Enforcing Mode                Enforcing
Host Name                     ans8
Platform                      Linux ans8 4.18.0-147.el8.x86_64 #1 SMP
Thu Sep 26
                              15:52:44 UTC 2019 x86_64 x86_64
Alert Count                   1
First Seen                    2020-01-08 18:39:47 EST
Last Seen                     2020-01-08 18:39:47 EST
Local ID                      6f8c3415-7a50-4dc8-b3d2-2621e1d00ca3

Raw Audit Messages
type=AVC msg=audit(1578526787.577:68): avc:  denied  { ioctl } for
pid=995 comm="rbd-target-api"
path="/sys/kernel/config/target/iscsi/iqn.2003-01.com.redhat:ceph-iscsi/tpgt_1/attrib/authentication"
dev="configfs" ino=25703 ioctlcmd=0x5401
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:configfs_t:s0 tclass=file permissive=1

type=SYSCALL msg=audit(1578526787.577:68): arch=x86_64 syscall=ioctl
success=no exit=ENOTTY a0=34 a1=5401 a2=7ffd4f8f1f60 a3=3052cd2d95839b96
items=0 ppid=1 pid=995 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0
egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=rbd-target-api
exe=/usr/libexec/platform-python3.6 subj=system_u:system_r:ceph_t:s0
key=(null)

Hash: rbd-target-api,ceph_t,configfs_t,file,ioctl

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 8187235c91e897cb8c167ee4cbb90d910103709b)

5 years agoMerge pull request #36655 from smithfarm/wip-46968-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:42:51 +0000 (14:42 -0700)]
Merge pull request #36655 from smithfarm/wip-46968-octopus

octopus: rgw: introduce safe user-reset-stats

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #36654 from smithfarm/wip-46966-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:42:17 +0000 (14:42 -0700)]
Merge pull request #36654 from smithfarm/wip-46966-octopus

octopus: rgw: fix double slash (//) killing the gateway

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #36653 from smithfarm/wip-46955-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:41:43 +0000 (14:41 -0700)]
Merge pull request #36653 from smithfarm/wip-46955-octopus

octopus: rgw: fix shutdown crash in RGWAsyncReadMDLogEntries

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #36652 from smithfarm/wip-46949-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:40:48 +0000 (14:40 -0700)]
Merge pull request #36652 from smithfarm/wip-46949-octopus

octopus: cls/rgw: preserve olh entry's name on last unlink

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #36651 from smithfarm/wip-46938-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:39:54 +0000 (14:39 -0700)]
Merge pull request #36651 from smithfarm/wip-46938-octopus

octopus: qa/tasks/ragweed: always set ragweed_repo

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoMerge pull request #36650 from smithfarm/wip-46929-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:38:45 +0000 (14:38 -0700)]
Merge pull request #36650 from smithfarm/wip-46929-octopus

octopus: rgw: Empty reqs_change_state queue before unregistered_reqs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #36649 from smithfarm/wip-46640-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:37:58 +0000 (14:37 -0700)]
Merge pull request #36649 from smithfarm/wip-46640-octopus

octopus: rgw: add abort multipart date and rule-id header to init multipart upload response

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #36648 from smithfarm/wip-46593-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:37:14 +0000 (14:37 -0700)]
Merge pull request #36648 from smithfarm/wip-46593-octopus

octopus: rgw: add negative cache to the system object

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #36647 from smithfarm/wip-46518-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:36:31 +0000 (14:36 -0700)]
Merge pull request #36647 from smithfarm/wip-46518-octopus

octopus: rgw: fix boost::asio::async_write() does not return error...

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #36646 from smithfarm/wip-46510-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:35:31 +0000 (14:35 -0700)]
Merge pull request #36646 from smithfarm/wip-46510-octopus

octopus: rgw: Adding data cache and CDN capabilities

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #36645 from smithfarm/wip-46475-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:34:41 +0000 (14:34 -0700)]
Merge pull request #36645 from smithfarm/wip-46475-octopus

octopus: rgw/iam: correcting the result of get role policy.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #36636 from smithfarm/wip-46953-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:32:10 +0000 (14:32 -0700)]
Merge pull request #36636 from smithfarm/wip-46953-octopus

octopus: rgw: policy: reuse eval_principal to evaluate the policy principal

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #36026 from smithfarm/wip-46457-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:31:34 +0000 (14:31 -0700)]
Merge pull request #36026 from smithfarm/wip-46457-octopus

octopus: selinux: allow ceph_t amqp_port_t:tcp_socket

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #36018 from smithfarm/wip-45922-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:29:57 +0000 (14:29 -0700)]
Merge pull request #36018 from smithfarm/wip-45922-octopus

octopus: rgw: lc: add lifecycle perf counters

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
5 years agoMerge pull request #35929 from smithfarm/wip-46343-octopus
Yuri Weinstein [Tue, 18 Aug 2020 21:26:51 +0000 (14:26 -0700)]
Merge pull request #35929 from smithfarm/wip-46343-octopus

octopus: rgw: orphan-list timestamp fix

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agomgr/prometheus: log time it takes to collect metrics 36581/head
Patrick Seidensal [Tue, 9 Jun 2020 13:58:54 +0000 (15:58 +0200)]
mgr/prometheus: log time it takes to collect metrics

When debug is enabled, the time it takes to gather the various types of
metrics is logged by type.

Fixes: https://tracker.ceph.com/issues/46444
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit a5f86cb85fcbf8a9003bf511b57a6d6b4a0e1f41)

 Conflicts:
src/pybind/mgr/mgr_util.py
  - Some more imports in master compared to octopus;
    had to add "import time" (line 6)  manually, otherwise make check
    would fail with a mypy error ('time is not defined')

5 years agoMerge pull request #36638 from smithfarm/wip-46945-octopus
Yuri Weinstein [Tue, 18 Aug 2020 14:05:40 +0000 (07:05 -0700)]
Merge pull request #36638 from smithfarm/wip-46945-octopus

octopus: librbd:  global and pool-level config overrides require image refresh to apply

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>/
5 years agoMerge pull request #36414 from shyukri/wip-46322-octopus
Yuri Weinstein [Tue, 18 Aug 2020 14:04:56 +0000 (07:04 -0700)]
Merge pull request #36414 from shyukri/wip-46322-octopus

octopus: osd/OSDCap: rbd profile permits use of "rbd_info"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #36411 from shyukri/wip-46087-octopus
Yuri Weinstein [Tue, 18 Aug 2020 14:04:25 +0000 (07:04 -0700)]
Merge pull request #36411 from shyukri/wip-46087-octopus

octopus: mgr/prometheus: automatically discover RBD pools for stats gathering

Reviewed-by: Mykola Golub <mgolub@mirantis.com>/
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #36331 from dillaman/wip-librbd-aio-completion-octopus
Yuri Weinstein [Tue, 18 Aug 2020 14:03:50 +0000 (07:03 -0700)]
Merge pull request #36331 from dillaman/wip-librbd-aio-completion-octopus

octopus: librbd: potential race conditions handling API IO completions

Reviewed-by: Mykola Golub <mgolub@mirantis.com>/
5 years agoMerge pull request #36660 from smithfarm/wip-46721-octopus
Yuri Weinstein [Tue, 18 Aug 2020 13:58:32 +0000 (06:58 -0700)]
Merge pull request #36660 from smithfarm/wip-46721-octopus

octopus: rbd: immutable-object-cache: fixed crashes on start up

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #36659 from smithfarm/wip-46719-octopus
Yuri Weinstein [Tue, 18 Aug 2020 13:58:08 +0000 (06:58 -0700)]
Merge pull request #36659 from smithfarm/wip-46719-octopus

octopus: rbd: librbd: don't resend async_complete if watcher is unregistered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #36658 from smithfarm/wip-46711-octopus
Yuri Weinstein [Tue, 18 Aug 2020 13:57:38 +0000 (06:57 -0700)]
Merge pull request #36658 from smithfarm/wip-46711-octopus

octopus: rbd: librbd: flush all queued object IO from simple scheduler

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>/
5 years agoMerge pull request #36657 from smithfarm/wip-46309-octopus
Yuri Weinstein [Tue, 18 Aug 2020 13:56:56 +0000 (06:56 -0700)]
Merge pull request #36657 from smithfarm/wip-46309-octopus

octopus: tests/rbd_mirror: fix race on test shut down

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #36656 from smithfarm/wip-46150-octopus
Yuri Weinstein [Tue, 18 Aug 2020 13:56:07 +0000 (06:56 -0700)]
Merge pull request #36656 from smithfarm/wip-46150-octopus

octopus: rbd: librbd: race when disabling object map with overlapping in-flight writes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #36662 from smithfarm/wip-46595-octopus
Yuri Weinstein [Tue, 18 Aug 2020 13:55:35 +0000 (06:55 -0700)]
Merge pull request #36662 from smithfarm/wip-46595-octopus

octopus: crush/CrushWrapper: rebuild reverse maps after rebuilding crush map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge pull request #35772 from s0nea/wip-46205-octopus
Lenz Grimmer [Mon, 17 Aug 2020 15:00:26 +0000 (17:00 +0200)]
Merge pull request #35772 from s0nea/wip-46205-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #36350 from s0nea/wip-46751-octopus
Lenz Grimmer [Mon, 17 Aug 2020 13:32:27 +0000 (15:32 +0200)]
Merge pull request #36350 from s0nea/wip-46751-octopus

octopus: mgr/dashboard: Add hosts page unit tests

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agomgr/dashboard: Fix host attributes like labels are not returned 36678/head
Kiefer Chang [Mon, 3 Aug 2020 11:20:08 +0000 (19:20 +0800)]
mgr/dashboard: Fix host attributes like labels are not returned

The problem only happens when a host is reported both from Ceph and the
Orchestrator. There is an error on merging Ceph hosts with Orchestrator
hosts. The unit test is refined to test this case.

Fixes: https://tracker.ceph.com/issues/46761
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 8f393f8e1bc2f5b20d2582aa48cefef23edc2153)