ceph-ci.git
2 years ago16.2.4 v16.2.4
Jenkins Build Slave User [Thu, 13 May 2021 17:20:27 +0000 (17:20 +0000)]
16.2.4

2 years agomgr/dashboard: fix cookie injection issue
Ernesto Puerta [Thu, 13 May 2021 15:43:56 +0000 (17:43 +0200)]
mgr/dashboard: fix cookie injection issue

Fixes: CVE-2021-3509
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit b39922818bc57cde1b016e9ad41908b18063b93b)

2 years agosystemd: remove `ProtectClock=true` for `ceph-osd@.service`
Wong Hoi Sing Edison [Wed, 14 Apr 2021 07:36:17 +0000 (15:36 +0800)]
systemd: remove `ProtectClock=true` for `ceph-osd@.service`

Ceph 16.2.0 Pacific by https://github.com/ceph/ceph/commit/9a84d5a introduce following new systemd restriction:

    ProtectClock=true
    ProtectHostname=true
    ProtectKernelLogs=true
    RestrictSUIDSGID=true

BTW, `ceph-osd@.service` failed with `ProtectClock=true` unexpectly, also see:

  - <https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/TNBGGNN6STGDKARAQTQCIPTU4KLIVJQV/>
  - <https://serverfault.com/questions/1059317/bluestore-var-lib-ceph-osd-ceph-2-block-read-bdev-label-failed-to-open-var-l>

This PR intruduce:

  - Remove `ProtectClock=true` for our systemd service templates

Fixes: https://tracker.ceph.com/issues/50347
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
(cherry picked from commit 85bc551b179d940a50cbdfd0c20848e3187c70a6)

2 years agomgr/dashboard: fix set-ssl-certificate{,-key} commands
Alfonso Martínez [Mon, 3 May 2021 16:46:08 +0000 (18:46 +0200)]
mgr/dashboard: fix set-ssl-certificate{,-key} commands

- Now create-self-signed-cert command relies on set-ssl-certificate{,-key} commands.
- Simplify the command testing and increase the test coverage.

Fixes: https://tracker.ceph.com/issues/50519
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit fb20990bca55ffe1712a905c3a9741e8df004e74)
(cherry picked from commit 3bdbbb95a05b26e84e09a550d0bde3facf3a76d4)

2 years agomgr/dashboard: fix base-href: revert it to previous approach
Avan Thakkar [Fri, 7 May 2021 09:38:11 +0000 (15:08 +0530)]
mgr/dashboard: fix base-href: revert it to previous approach

Fixes: https://tracker.ceph.com/issues/50684
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit b6f92922f5c80223fd288d98ce85405a650c0135)
(cherry picked from commit 6b9c8ca076ef63df64ee12f89132ed2f46c4b739)

2 years agorgw: sanitize \r in s3 CORSConfiguration's ExposeHeader
Casey Bodley [Tue, 4 May 2021 12:32:58 +0000 (08:32 -0400)]
rgw: sanitize \r in s3 CORSConfiguration's ExposeHeader

follows up on 1524d3c0c5cb11775313ea1e2bb36a93257947f2 to escape \r as
well

Fixes: CVE-2021-3524
Reported-by: Sergey Bobrov <Sergey.Bobrov@kaspersky.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 87806f48e7a1b8891eb90711f1cedd26f1119aac)

2 years agorgw: RGWSwiftWebsiteHandler::is_web_dir checks empty subdir_name
Felix Hüttner [Thu, 6 May 2021 16:18:00 +0000 (12:18 -0400)]
rgw: RGWSwiftWebsiteHandler::is_web_dir checks empty subdir_name

checking for empty name avoids later assertion in RGWObjectCtx::set_atomic

Fixes: CVE-2021-3531
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7196a469b4470f3c8628489df9a41ec8b00a5610)

2 years ago16.2.3 v16.2.3
Jenkins Build Slave User [Thu, 6 May 2021 15:47:00 +0000 (15:47 +0000)]
16.2.3

2 years agocephadm: normalize image digest in 'ls' output too
Sage Weil [Wed, 5 May 2021 18:42:36 +0000 (14:42 -0400)]
cephadm: normalize image digest in 'ls' output too

We were previously only normalizing the 'inspect' output.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 06d849acfb77a189abf6e9c26b27aaaefdd77f14)

2 years ago16.2.2 v16.2.2
Jenkins Build Slave User [Tue, 4 May 2021 18:38:23 +0000 (18:38 +0000)]
16.2.2

2 years agoMerge PR #40922 into pacific
Sage Weil [Wed, 28 Apr 2021 15:44:21 +0000 (10:44 -0500)]
Merge PR #40922 into pacific

* refs/pull/40922/head:
pybind/ceph_argparse: print --format flag name in help descs
mgr/cephadm: don't list non ceph daemons as needing upgrade in upgrade check
qa/tasks/cephadm: ignore --keep-logs failure
qa/tasks/cephadm: use yaml.dump_all()
qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
qa/tasks/cephadm: tear down clsuter before gathering logs
qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
mgr/cephadm: remove virtual_ip check during scheduling
mgr/orchestrator: orch ls: leave off virtual_ip prefixlen
qa/tasks/cephadm: add wait_for_service
qa/tasks/cephadm: allow skip_monitor_stack=true
qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
qa/tasks/vip: add vip task to allocate virtual IPs
qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case
qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'
qa/tasks/cephadm: let cephadm.shell take string or list
doc/cephadm: wrong command for single daemon events
mgr/cephadm: place maximum on placement count based on host count
mgr/cephadm: fix nfs-rgw stray daemon
mgr/cephadm: skip-ssh flag enables cephadm mgr module
mgr/cephadm: report exception during upgrade in upgrade status
qa/suites/rados/thrash: shorten radosbench
mgr/cephadm: remove old haproxy and keepalived templates
mgr/orchestrator: validate lists in spec jsons
python-common: Verify service spec is not None
python-common: Verify data_devices is not None
mgr/orchestrator: DG loads properly the unmanaged attribute
mgr/orchestractor: rgw realm and zone flags must both be provided
mgr/cephadm: make prometheus scrape ingress haproxy
doc/cephadm: remove big warning about stability
doc/cepham/compatibility: rgw-ha -> ingress; note possibility of breaking changes
doc/cephadm: rewrite "dry run" section in osd.rst
doc/cephadm: rewrite part of "deploy osds"
doc/cephadm: rewrite osd.rst "Remove an OSD"
doc/cephadm: rewrite osd.rst - list devices
doc/cephadm: break mon section into sections
doc/cephadm: rewrite "deploying add. mons"
doc: fixes for cephadm documentation
doc/cephadm: remove warning about cephadm in production
doc/cephadm: Add Compatibility with Podman Versions
doc/cephadm: rewrite "index.rst"
doc/cephadm: explicitly show host requirments in adding host section
mgr/cephadm: ingress: add optional virtual_interface_networks
doc/cephadm/rgw: clean up example spec
mgr/cephadm/services/ingress: less verbose about prepare_create
doc/cephadm/rgw: add note about which ethernet interface is used
cephadm: make keepalived unit fiddle sysctl settings
mgr/orchestrator: report external endpoints from 'orch ls'
mgr/orchestrator: drop - when no ports
doc/cephadm/rgw: update docs for ingress service
mgr/cephadm: use per_host_daemon feature in scheduler
cephadm: fix a typo
mgr/cephadm/schedule: add per_host_daemon_type support
mgr/cephadm: HA_RGW -> Ingress
mgr/cephadm: include daemon_type in DaemonPlacement
mgr/cephadm: update list-networks to report interface names too
mgr/orchestrator: streamline 'orch ps' PORTS formatting
mgr/cephadm/schedule: handle multiple ports per daemon
mgr/cephadm/utils: resolve_ip(): prefer IPv4
cephadm: cleanup extra slash in runtime dir
cephadm: use split cgroup strategy for podman
cephadm: use class to represent container engine
mgr/cephadm: don't cleanup the daemon keyring on failed redeploy
mgr/cephadm: fix orch host add with multiple labels and no addr
doc/cephadm: remove keepalived_user from haproxy docs
rpm: re-disable SUSE lttng build on z390x
ceph.spec.in: enable tcmalloc and lttng on s390x

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2 years agoMerge pull request #41054 from leseb/bkp-41032
Sébastien Han [Wed, 28 Apr 2021 12:33:48 +0000 (14:33 +0200)]
Merge pull request #41054 from leseb/bkp-41032

pacific: ceph-volume: fix "device" output

2 years agoceph-volume: fix "device" output
Sébastien Han [Mon, 26 Apr 2021 16:01:19 +0000 (18:01 +0200)]
ceph-volume: fix "device" output

During
https://github.com/ceph/ceph/commit/ec0f5f3b22d24754c16131a1996e42b787e4255f
I changed "device" by "dev", let's revert that!

Fixes: https://tracker.ceph.com/issues/50478
Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit da1f9d215fa9e86fa30d82de7032df05c6cbefb1)

2 years agoMerge pull request #40612 from rhcs-dashboard/wip-50145-pacific
Ernesto Puerta [Tue, 27 Apr 2021 17:22:25 +0000 (19:22 +0200)]
Merge pull request #40612 from rhcs-dashboard/wip-50145-pacific

pacific: qa/tasks/vstart_runner.py: start max required mgrs

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: rishabh-d-dave <NOT@FOUND>
2 years agoMerge pull request #40887 from rhcs-dashboard/wip-50350-pacific
Ernesto Puerta [Tue, 27 Apr 2021 17:20:35 +0000 (19:20 +0200)]
Merge pull request #40887 from rhcs-dashboard/wip-50350-pacific

pacific: mgr/dashboard: improve telemetry opt-in reminder notification message

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #40929 from rhcs-dashboard/wip-49658-pacific
Ernesto Puerta [Tue, 27 Apr 2021 17:16:43 +0000 (19:16 +0200)]
Merge pull request #40929 from rhcs-dashboard/wip-49658-pacific

pacific: mgr/dashboard: test prometheus rules through promtool

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #40815 from rhcs-dashboard/wip-50171-pacific
Ernesto Puerta [Tue, 27 Apr 2021 17:07:07 +0000 (19:07 +0200)]
Merge pull request #40815 from rhcs-dashboard/wip-50171-pacific

pacific: mgr/dashboard: debug nodeenv hangs

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #40980 from rhcs-dashboard/wip-50418-pacific
Ernesto Puerta [Tue, 27 Apr 2021 17:06:10 +0000 (19:06 +0200)]
Merge pull request #40980 from rhcs-dashboard/wip-50418-pacific

pacific: mgr/dashboard: filesystem pool size should use stored stat

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #40691 from singuliere/wip-50124-pacific
Yuri Weinstein [Mon, 26 Apr 2021 21:33:28 +0000 (14:33 -0700)]
Merge pull request #40691 from singuliere/wip-50124-pacific

pacific: mon: Modifying trim logic to change paxos_service_trim_max dynamically

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
2 years agoMerge pull request #40989 from trociny/wip-50480-pacific
Yuri Weinstein [Mon, 26 Apr 2021 21:29:57 +0000 (14:29 -0700)]
Merge pull request #40989 from trociny/wip-50480-pacific

pacific: os/FileStore: don't propagate split/merge error to "create"/"remove"

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #40759 from smithfarm/wip-50154-pacific
Yuri Weinstein [Mon, 26 Apr 2021 21:29:32 +0000 (14:29 -0700)]
Merge pull request #40759 from smithfarm/wip-50154-pacific

pacific: osd/PeeringState: fix acting_set_writeable min_size check

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #40690 from singuliere/wip-50131-pacific
Yuri Weinstein [Mon, 26 Apr 2021 21:27:43 +0000 (14:27 -0700)]
Merge pull request #40690 from singuliere/wip-50131-pacific

pacific: monmaptool: Don't call set_port on an invalid address

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2 years agoMerge pull request #40679 from singuliere/wip-50121-pacific
Yuri Weinstein [Mon, 26 Apr 2021 21:27:10 +0000 (14:27 -0700)]
Merge pull request #40679 from singuliere/wip-50121-pacific

pacific:  crush/CrushLocation: do not print logging message in constructor

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #40677 from singuliere/wip-50212-pacific
Yuri Weinstein [Mon, 26 Apr 2021 21:26:46 +0000 (14:26 -0700)]
Merge pull request #40677 from singuliere/wip-50212-pacific

pacific: os/bluestore/BlueFS: do not _flush_range deleted files

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #40853 from batrick/i50285
Yuri Weinstein [Mon, 26 Apr 2021 17:18:35 +0000 (10:18 -0700)]
Merge pull request #40853 from batrick/i50285

pacific: qa: test standby_replay in workloads

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #40852 from batrick/i50287
Yuri Weinstein [Mon, 26 Apr 2021 17:17:50 +0000 (10:17 -0700)]
Merge pull request #40852 from batrick/i50287

pacific: qa: "log [ERR] : error reading sessionmap 'mds2_sessionmap'"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #40825 from batrick/i50253
Yuri Weinstein [Mon, 26 Apr 2021 17:17:09 +0000 (10:17 -0700)]
Merge pull request #40825 from batrick/i50253

pacific: mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #40680 from singuliere/wip-50082-pacific
Yuri Weinstein [Mon, 26 Apr 2021 14:45:10 +0000 (07:45 -0700)]
Merge pull request #40680 from singuliere/wip-50082-pacific

pacific: rbd-mirror: fix UB while registering perf counters

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2 years agoMerge pull request #40981 from rhcs-dashboard/wip-50476-pacific
Ernesto Puerta [Mon, 26 Apr 2021 08:12:39 +0000 (10:12 +0200)]
Merge pull request #40981 from rhcs-dashboard/wip-50476-pacific

pacific: mgr/dashboard: Remove username and password from request body

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #40990 from rhcs-dashboard/wip-50485-pacific
Ernesto Puerta [Fri, 23 Apr 2021 14:50:41 +0000 (16:50 +0200)]
Merge pull request #40990 from rhcs-dashboard/wip-50485-pacific

pacific: mgr/dashboard: fix duplicated rows when creating NFS export.

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #40688 from singuliere/wip-50086-pacific
Yuri Weinstein [Fri, 23 Apr 2021 14:48:52 +0000 (07:48 -0700)]
Merge pull request #40688 from singuliere/wip-50086-pacific

pacific: qa/tasks/cephfs: create enough subvolumes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #40686 from singuliere/wip-50180-pacific
Yuri Weinstein [Fri, 23 Apr 2021 14:48:23 +0000 (07:48 -0700)]
Merge pull request #40686 from singuliere/wip-50180-pacific

pacific: client: only check pool permissions for regular files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #40684 from singuliere/wip-50185-pacific
Yuri Weinstein [Fri, 23 Apr 2021 14:47:56 +0000 (07:47 -0700)]
Merge pull request #40684 from singuliere/wip-50185-pacific

pacific: test: disable mgr/mirroring for `test_mirroring_init_failure_with_recovery` test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #40683 from singuliere/wip-50190-pacific
Yuri Weinstein [Fri, 23 Apr 2021 14:46:52 +0000 (07:46 -0700)]
Merge pull request #40683 from singuliere/wip-50190-pacific

pacific: qa: fix ino_release_cb racy behavior

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #40682 from singuliere/wip-50225-pacific
Yuri Weinstein [Fri, 23 Apr 2021 14:46:26 +0000 (07:46 -0700)]
Merge pull request #40682 from singuliere/wip-50225-pacific

pacific: mds: skip the buffer in UnknownPayload::decode()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #40678 from singuliere/wip-50199-pacific
Yuri Weinstein [Fri, 23 Apr 2021 14:45:56 +0000 (07:45 -0700)]
Merge pull request #40678 from singuliere/wip-50199-pacific

pacific: tools/cephfs_mirror/PeerReplayer.cc: add missing include

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2 years agoMerge pull request #40630 from batrick/i50127
Yuri Weinstein [Fri, 23 Apr 2021 14:43:40 +0000 (07:43 -0700)]
Merge pull request #40630 from batrick/i50127

pacific: pybind/mgr/volumes: deadlock on async job hangs finisher thread

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge pull request #40627 from petrutlucian94/wip-50187-pacific
Yuri Weinstein [Fri, 23 Apr 2021 14:42:26 +0000 (07:42 -0700)]
Merge pull request #40627 from petrutlucian94/wip-50187-pacific

pacific: cephfs: minor ceph-dokan improvements

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agopybind/ceph_argparse: print --format flag name in help descs wip-sage2-testing-2021-04-23-0724-pacific
Adam King [Mon, 15 Mar 2021 17:29:46 +0000 (13:29 -0400)]
pybind/ceph_argparse: print --format flag name in help descs

Fixes: https://tracker.ceph.com/issues/49757
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 97825298b99121d8ac877eaec16d3983549f30ee)

2 years agomgr/cephadm: don't list non ceph daemons as needing upgrade in upgrade check
Adam King [Fri, 16 Apr 2021 16:49:59 +0000 (12:49 -0400)]
mgr/cephadm: don't list non ceph daemons as needing upgrade in upgrade check

Fixes: https://tracker.ceph.com/issues/50401
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 5bdf50e5316b742bb41252832a8211928f451352)

2 years agoqa/tasks/cephadm: ignore --keep-logs failure
Sage Weil [Fri, 16 Apr 2021 19:12:49 +0000 (14:12 -0500)]
qa/tasks/cephadm: ignore --keep-logs failure

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 65f2848e978b3ef3908124dfbfbeed1f7bf52087)

2 years agoqa/tasks/cephadm: use yaml.dump_all()
Sage Weil [Fri, 16 Apr 2021 15:30:05 +0000 (10:30 -0500)]
qa/tasks/cephadm: use yaml.dump_all()

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 97c143c3442e96b36de075ea7eefd6e17ede67bd)

2 years agoqa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
Sage Weil [Tue, 13 Apr 2021 16:53:24 +0000 (12:53 -0400)]
qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 16b30f2858aa8edb42cff79c4d40b40ec5ae72cc)

2 years agoqa/tasks/cephadm: tear down clsuter before gathering logs
Sage Weil [Thu, 15 Apr 2021 22:55:00 +0000 (17:55 -0500)]
qa/tasks/cephadm: tear down clsuter before gathering logs

We dont' always stop all services, because teuthology doesn't know about
things it didn't start.  Use rm-cluster to tear things down, but do not
remove the logs themselves.  After we get logs, we'll clean up completely.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit deec9074bb2fc42d29d6fa14c22b6b14b97c352f)

2 years agoqa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
Sage Weil [Thu, 15 Apr 2021 22:45:21 +0000 (17:45 -0500)]
qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress

Test this properly by downing each rgw and haproxy in turn and ensuring
that things remain up.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3ff3f697b474c9669bc4f51c472a9cad35e72266)

2 years agomgr/cephadm: remove virtual_ip check during scheduling
Sage Weil [Thu, 15 Apr 2021 22:22:26 +0000 (17:22 -0500)]
mgr/cephadm: remove virtual_ip check during scheduling

In 2f33c6ebbc8e2a6c3844a6921c857fb0796a1552 we made the keepalived task
set the necessary sysctls  to add a virtual_ip, so we don't need this
check anymore.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2382603162ec6785681700134e3c5764bd5aa99f)

2 years agomgr/orchestrator: orch ls: leave off virtual_ip prefixlen
Sage Weil [Thu, 15 Apr 2021 21:59:09 +0000 (16:59 -0500)]
mgr/orchestrator: orch ls: leave off virtual_ip prefixlen

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4498bbe77f58c59583bcb8b9ca1aae33296b329f)

2 years agoqa/tasks/cephadm: add wait_for_service
Sage Weil [Tue, 13 Apr 2021 16:52:49 +0000 (12:52 -0400)]
qa/tasks/cephadm: add wait_for_service

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ced2f7fe4a04ebaa09896376c342b7b866ab5bc7)

2 years agoqa/tasks/cephadm: allow skip_monitor_stack=true
Sage Weil [Thu, 15 Apr 2021 19:01:48 +0000 (14:01 -0500)]
qa/tasks/cephadm: allow skip_monitor_stack=true

(Useful for roleless when we want to go faster)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bb825157dccd5dee65fc75e63fde856c8bcc12e6)

2 years agoqa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
Sage Weil [Thu, 15 Apr 2021 19:01:18 +0000 (14:01 -0500)]
qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0d561e2741718498efa319e0c2b5ec3a902c67ca)

2 years agoqa/tasks/vip: add vip task to allocate virtual IPs
Sage Weil [Thu, 15 Apr 2021 19:00:57 +0000 (14:00 -0500)]
qa/tasks/vip: add vip task to allocate virtual IPs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2a8ab2d2b87f76dad9b8ecd7f8ce8370f8004b3c)

2 years agoqa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case
Sage Weil [Tue, 13 Apr 2021 14:21:41 +0000 (10:21 -0400)]
qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7e16bf3468b53d218ac02a81c01fdbbc002b5f1b)

2 years agoqa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'
Sage Weil [Tue, 13 Apr 2021 14:58:09 +0000 (10:58 -0400)]
qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'

'all' is ambiguous

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 08039576950697e7b9dd55c6e44068440d2a1553)

2 years agoqa/tasks/cephadm: let cephadm.shell take string or list
Sage Weil [Mon, 12 Apr 2021 20:12:01 +0000 (16:12 -0400)]
qa/tasks/cephadm: let cephadm.shell take string or list

Make it a bit more forgiving.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 779af3da6fd6405e13a2e522c57fa1d1512595a9)

2 years agodoc/cephadm: wrong command for single daemon events
Adam King [Thu, 8 Apr 2021 19:43:08 +0000 (15:43 -0400)]
doc/cephadm: wrong command for single daemon events

Fixes: https://tracker.ceph.com/issues/50257
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 31959dd6d361273cb125420338a47a6fcbf3998e)

2 years agomgr/cephadm: place maximum on placement count based on host count
Adam King [Wed, 24 Mar 2021 16:29:20 +0000 (12:29 -0400)]
mgr/cephadm: place maximum on placement count based on host count

Fixes: https://tracker.ceph.com/issues/49960
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 73532088915b8e7daf4a45d8b7968cdab55de9d1)

2 years agomgr/cephadm: fix nfs-rgw stray daemon
Daniel Pivonka [Thu, 8 Apr 2021 19:20:18 +0000 (15:20 -0400)]
mgr/cephadm: fix nfs-rgw stray daemon

nfs-rgw registers under a gid cephadm needs covert that to its known name during the stray daemon check

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit f94e0baf9e1897f803160eff8ba36df57aa433ac)

2 years agomgr/cephadm: skip-ssh flag enables cephadm mgr module
cypherean [Sun, 21 Mar 2021 22:13:46 +0000 (03:43 +0530)]
mgr/cephadm: skip-ssh flag enables cephadm mgr module

This commit fixes the use of skip-ssh flag. It disables ssh config and enables the cephadm mgr module.

Fixes: http://tracker.ceph.com/issues/49737
Signed-off-by: Shreyaa Sharma <shreyasharma.ss305@gmail.com>
(cherry picked from commit 777f236ad885b03b551dd820f41a00b9c89761eb)

2 years agomgr/cephadm: report exception during upgrade in upgrade status
Adam King [Wed, 14 Apr 2021 15:39:10 +0000 (11:39 -0400)]
mgr/cephadm: report exception during upgrade in upgrade status

Fixes: https://tracker.ceph.com/issues/50361
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 6119294b2871977e0a70b138d48dc5afc8abd45d)

2 years agoqa/suites/rados/thrash: shorten radosbench
Sage Weil [Tue, 13 Apr 2021 22:42:21 +0000 (17:42 -0500)]
qa/suites/rados/thrash: shorten radosbench

This is the longest of the thrash workloads; reducing it will bring
this test in line with the others (<= 45 min).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit babbab14f4ed8b000741fb92b76b97459510c689)

2 years agomgr/cephadm: remove old haproxy and keepalived templates
Sage Weil [Wed, 14 Apr 2021 16:47:54 +0000 (11:47 -0500)]
mgr/cephadm: remove old haproxy and keepalived templates

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0536364645ff06f285c3dd698df02898a920f1b9)

2 years agomgr/orchestrator: validate lists in spec jsons
Daniel Pivonka [Thu, 1 Apr 2021 17:56:48 +0000 (13:56 -0400)]
mgr/orchestrator: validate lists in spec jsons

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 7844ce0785595c951f5822d2c38d1381dc13c8c1)

2 years agopython-common: Verify service spec is not None
Sebastian Wagner [Thu, 11 Feb 2021 11:23:56 +0000 (12:23 +0100)]
python-common: Verify service spec is not None

Fixes: https://tracker.ceph.com/issues/48325
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 518edfae7522ada8c74b413cef6e1ae1f08a244b)

2 years agopython-common: Verify data_devices is not None
Sebastian Wagner [Thu, 11 Feb 2021 10:05:12 +0000 (11:05 +0100)]
python-common: Verify data_devices is not None

Add validation to verify that `data_devices` is not None

Fixes: https://tracker.ceph.com/issues/49191
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 55e9ecbc88bf6a33fe185e8b54491b9048d66adb)

2 years agomgr/orchestrator: DG loads properly the unmanaged attribute
Juan Miguel Olmo Martínez [Mon, 15 Mar 2021 13:19:33 +0000 (14:19 +0100)]
mgr/orchestrator: DG loads properly the unmanaged attribute

Fixes: https://tracker.ceph.com/issues/49805
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 0af4ad8614e426adf60eec32bd4b36974c5cb30b)

2 years agomgr/orchestractor: rgw realm and zone flags must both be provided
Daniel Pivonka [Fri, 9 Apr 2021 19:25:21 +0000 (15:25 -0400)]
mgr/orchestractor: rgw realm and zone flags must both be provided

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit c0803f8f271ea6b2c653b3a443f7807185303912)

2 years agomgr/cephadm: make prometheus scrape ingress haproxy
Sage Weil [Tue, 13 Apr 2021 22:20:21 +0000 (18:20 -0400)]
mgr/cephadm: make prometheus scrape ingress haproxy

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7a30e656b9c719ec0141e19bfc629c0f7ae89c9f)

2 years agodoc/cephadm: remove big warning about stability
Sage Weil [Tue, 13 Apr 2021 14:20:48 +0000 (10:20 -0400)]
doc/cephadm: remove big warning about stability

It's the first item on the toctree that follows.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d72c61b850a6a1102a58eaa731759afc41d73181)

2 years agodoc/cepham/compatibility: rgw-ha -> ingress; note possibility of breaking changes
Sage Weil [Tue, 13 Apr 2021 14:20:27 +0000 (10:20 -0400)]
doc/cepham/compatibility: rgw-ha -> ingress; note possibility of breaking changes

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 53477261194b50b6e5af0573f2aa82486092df8c)

2 years agodoc/cephadm: rewrite "dry run" section in osd.rst
Zac Dover [Wed, 24 Mar 2021 15:47:17 +0000 (01:47 +1000)]
doc/cephadm: rewrite "dry run" section in osd.rst

This rewrites the "dry run" section of the "OSD Service"
chapter of the Cephdam documentation. This commit makes
minor changes that reduce the cognitive load of the
reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit e61237f3a1f83b90d4ab22396c6e8291620a60fa)

2 years agodoc/cephadm: rewrite part of "deploy osds"
Zac Dover [Wed, 24 Mar 2021 14:39:01 +0000 (00:39 +1000)]
doc/cephadm: rewrite part of "deploy osds"

This reorganizes the section "Deploy OSDs"
in the "OSD Service" chapter of the Cephadm
Guide. Two new sections, "Listing Storage
Devices" and "Creating New OSDs" gather
information under headings in a sensible way,
making the information more accessible to someone
skimming this Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 5f1ce2f6e8df185673613df9a31bac2395a46438)

2 years agodoc/cephadm: rewrite osd.rst "Remove an OSD"
Zac Dover [Sun, 28 Mar 2021 19:23:08 +0000 (05:23 +1000)]
doc/cephadm: rewrite osd.rst "Remove an OSD"

This commit rewrites the entire "Remove an OSD"
section of the "OSD Service" chapter of the
cephadm book.

I got carried away and didn't break this one into
four smaller PRs, and I'm sorry in advance to
whomever ends up reviewing this. I'll break "Advanced
OSD Service Specifications", the next section in the
queue, into multiple sections.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 577e45c78b7fbb93e9d4cacf213f89f5d6a0abe4)

2 years agodoc/cephadm: rewrite osd.rst - list devices
Zac Dover [Tue, 23 Mar 2021 16:23:46 +0000 (02:23 +1000)]
doc/cephadm: rewrite osd.rst - list devices

This PR rewrites the "List Devices" section of
the OSD chapter of the Cephadm guide. This PR
is a simple grammar-and-elegance improvement.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 49352a3150b3cf19c5a6a65c270d69e81536990e)

2 years agodoc/cephadm: break mon section into sections
Zac Dover [Mon, 15 Mar 2021 15:03:06 +0000 (01:03 +1000)]
doc/cephadm: break mon section into sections

This PR breaks the "Deploy Additional Monitors" section
of the cephadm documentation into several subsections
whose titles spotlight the matter under discussion in
those respective subsections.

inb4: Another PR is on deck that rewrites the sentences
in this chapter of the cephadm documentation. I'd like
to get this chapter broken up into these subsections before
I rewrite those sentences. So I'm hoping for no grammatical
mission creep on this one. The grammar and clarity updates
are coming.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 25d9429d66a6edf446fd8bc3b7903b30de2aa31b)

2 years agodoc/cephadm: rewrite "deploying add. mons"
Zac Dover [Mon, 15 Mar 2021 15:03:06 +0000 (01:03 +1000)]
doc/cephadm: rewrite "deploying add. mons"

This rewrites the section "Deploying Additional
Monitors (Beyond the Default Three)" for elegance
and clarity.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit c605750db22e0807b887beae36648131805ede3c)

2 years agodoc: fixes for cephadm documentation
Jeff Layton [Fri, 29 Jan 2021 19:15:26 +0000 (14:15 -0500)]
doc: fixes for cephadm documentation

Be sure to note that python 3 is a prerequisite. Minimal centos 8
installs don't have it, for instance.

Also, we probably don't want to hardcode an octopus URL into the
suggested curl command. Change it to fill that in with
"|stable-release|", which should always point to the latest released
version name.

Fixes: https://tracker.ceph.com/issues/49806
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit bf69cdc68970789a7410928bd8a1af34d0d9b6a2)

2 years agodoc/cephadm: remove warning about cephadm in production
Sebastian Wagner [Wed, 3 Mar 2021 13:26:23 +0000 (14:26 +0100)]
doc/cephadm: remove warning about cephadm in production

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 230b78f35395c2c8b21cba6d2d1631971ebc752a)

2 years agodoc/cephadm: Add Compatibility with Podman Versions
Sebastian Wagner [Wed, 3 Mar 2021 13:00:51 +0000 (14:00 +0100)]
doc/cephadm: Add Compatibility with Podman Versions

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f15f0deccb93d0ec79beb4f7ba32c843e6e07e63)

2 years agodoc/cephadm: rewrite "index.rst"
Zac Dover [Tue, 23 Mar 2021 15:19:11 +0000 (01:19 +1000)]
doc/cephadm: rewrite "index.rst"

This PR rewrites the three paragraphs at the
front of the cephadm guide, increasing their
elegance and removing ambiguities.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit dfd205dca7889e325a6ec22892d3a9e058ad89d2)

2 years agodoc/cephadm: explicitly show host requirments in adding host section
Daniel Pivonka [Tue, 23 Mar 2021 17:50:33 +0000 (13:50 -0400)]
doc/cephadm: explicitly show host requirments in adding host section

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit b28fd9838ec3ad5b47a7b5e14015d986348f31e5)

2 years agomgr/cephadm: ingress: add optional virtual_interface_networks
Sage Weil [Mon, 12 Apr 2021 21:21:33 +0000 (17:21 -0400)]
mgr/cephadm: ingress: add optional virtual_interface_networks

It may be that the virtual IP we want to use is not in the same network
as any existing IPs on the host.  In that case, allow the spec to specify
a list of networks to match against existing IPs so that a match will
identify an ethernet interface to use.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bbf6a12752092b406abbec1e600533366ac59548)

2 years agodoc/cephadm/rgw: clean up example spec
Sage Weil [Mon, 12 Apr 2021 21:21:22 +0000 (17:21 -0400)]
doc/cephadm/rgw: clean up example spec

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 99b9f032de1d3611058caa748b5256ea2134446f)

2 years agomgr/cephadm/services/ingress: less verbose about prepare_create
Sage Weil [Mon, 12 Apr 2021 19:55:19 +0000 (15:55 -0400)]
mgr/cephadm/services/ingress: less verbose about prepare_create

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f7977c551db86dc46a1f6ceb6c25497aebd15c16)

2 years agodoc/cephadm/rgw: add note about which ethernet interface is used
Sage Weil [Mon, 12 Apr 2021 19:53:50 +0000 (15:53 -0400)]
doc/cephadm/rgw: add note about which ethernet interface is used

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 702829f7df462e10e1999a47fa587001fc61de1b)

2 years agocephadm: make keepalived unit fiddle sysctl settings
Sage Weil [Mon, 12 Apr 2021 17:50:12 +0000 (13:50 -0400)]
cephadm: make keepalived unit fiddle sysctl settings

No need to make the user adjust these manually.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2f33c6ebbc8e2a6c3844a6921c857fb0796a1552)

2 years agomgr/orchestrator: report external endpoints from 'orch ls'
Sage Weil [Fri, 9 Apr 2021 22:47:52 +0000 (18:47 -0400)]
mgr/orchestrator: report external endpoints from 'orch ls'

Add a PORTS column and report the external/virtual IP (and port(s)) from
'orch ls' output.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3f55c708b367da7b8890a00e48eb7c0498ef5d97)

2 years agomgr/orchestrator: drop - when no ports
Sage Weil [Sat, 10 Apr 2021 16:53:24 +0000 (12:53 -0400)]
mgr/orchestrator: drop - when no ports

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 60562414e10dfd29f0d42f2047da9bddc64b7d34)

2 years agodoc/cephadm/rgw: update docs for ingress service
Sage Weil [Fri, 9 Apr 2021 19:10:49 +0000 (15:10 -0400)]
doc/cephadm/rgw: update docs for ingress service

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ef2d92aab2df45e34d85d3db3c5a7fcb9f96eb4f)

2 years agomgr/cephadm: use per_host_daemon feature in scheduler
Sage Weil [Fri, 9 Apr 2021 18:43:59 +0000 (14:43 -0400)]
mgr/cephadm: use per_host_daemon feature in scheduler

This only affects ingress, at least for now.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6dcd0597bfea115eb7ec59817255bf1217a0b97e)

2 years agocephadm: fix a typo
Guillaume Abrioux [Tue, 30 Mar 2021 12:23:25 +0000 (14:23 +0200)]
cephadm: fix a typo

this adds a space in order to avoid displaying this:

```
"2021-03-29T10:51:32.595782Z service:rgw.default [ERROR] \"Failed while placing rgw.default.ceph-vasi-node5-osd-rgw-iscsi-gw.hpuesfon ceph-vasi-node5-osd-rgw-iscsi-gw
```

instead of:

```
"2021-03-29T10:51:32.595782Z service:rgw.default [ERROR] \"Failed while placing rgw.default.ceph-vasi-node5-osd-rgw-iscsi-gw.hpuesf on ceph-vasi-node5-osd-rgw-iscsi-rgw
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit a3cb119a7a5134e7a3e4006381da14ea2a927136)

2 years agomgr/cephadm/schedule: add per_host_daemon_type support
Sage Weil [Fri, 9 Apr 2021 18:43:45 +0000 (14:43 -0400)]
mgr/cephadm/schedule: add per_host_daemon_type support

This will be used to schedule a per-host keepalived alongside other
services.

Implement this as a final stage for place() that puts one per host and
also takes existing/stray daemons into consideration.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit db9f1930fee942c345a171508cb7250d6260c94b)

2 years agomgr/cephadm: HA_RGW -> Ingress
Sage Weil [Fri, 9 Apr 2021 16:34:20 +0000 (12:34 -0400)]
mgr/cephadm: HA_RGW -> Ingress

This is mostly a rename, with some simplification and cleanup.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0894773e95bd0d10ba85768ba8b9116fcd375f94)

2 years agomgr/cephadm: include daemon_type in DaemonPlacement
Sage Weil [Fri, 9 Apr 2021 18:19:37 +0000 (14:19 -0400)]
mgr/cephadm: include daemon_type in DaemonPlacement

Initially, this will always match the service_type.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d7b4a51a520d5d720fe784d64ac7864e828dea4a)

2 years agomgr/cephadm: update list-networks to report interface names too
Sage Weil [Thu, 1 Apr 2021 18:14:13 +0000 (14:14 -0400)]
mgr/cephadm: update list-networks to report interface names too

Also, minor fix in the ipv6 addr reporting: ignore networks that aren't in CIDR
form (no /).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1897d1cd15af385bd888da0a9ee944cd3a68af07)

2 years agomgr/orchestrator: streamline 'orch ps' PORTS formatting
Sage Weil [Tue, 23 Mar 2021 21:18:56 +0000 (17:18 -0400)]
mgr/orchestrator: streamline 'orch ps' PORTS formatting

"*:8000 *:8100" -> "*:8000,8100"

FWIW this matches the internal rendering used by DaemonPlacement

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f93c555c24336003770c182a4e3ccaae392c2d47)

2 years agomgr/cephadm/schedule: handle multiple ports per daemon
Sage Weil [Tue, 23 Mar 2021 20:37:20 +0000 (16:37 -0400)]
mgr/cephadm/schedule: handle multiple ports per daemon

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9256f1c374ab1e1e6d45f3a912048ab486357606)

2 years agomgr/cephadm/utils: resolve_ip(): prefer IPv4
Sage Weil [Tue, 23 Mar 2021 20:09:15 +0000 (16:09 -0400)]
mgr/cephadm/utils: resolve_ip(): prefer IPv4

On my system the first item in hte list is
'fe80::408d:35e7:510:e9fe%eno1np0'.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2e7808ccf80c78d424e1fc3db330ce0a6db1cb5d)

2 years agocephadm: cleanup extra slash in runtime dir
胡玮文 [Wed, 7 Apr 2021 13:18:52 +0000 (21:18 +0800)]
cephadm: cleanup extra slash in runtime dir

%t already contains a slash, no need to add an extra one

Signed-off-by: 胡玮文 <huww98@outlook.com>
(cherry picked from commit 9a864f086bedb7098060637ffb27ccc5dc92a88b)

2 years agocephadm: use split cgroup strategy for podman
胡玮文 [Thu, 11 Mar 2021 04:43:34 +0000 (12:43 +0800)]
cephadm: use split cgroup strategy for podman

Since systemd will create a cgroup for each service, we can instruct podman to
just split the current cgroup into sub-cgroups. This enables system admins to
use resource control features from systemd.

Signed-off-by: 胡玮文 <huww98@outlook.com>
(cherry picked from commit 1a76f4793ec96045b0fed5cd85b1a6b3dbcd732c)

2 years agocephadm: use class to represent container engine
胡玮文 [Thu, 11 Mar 2021 16:51:33 +0000 (00:51 +0800)]
cephadm: use class to represent container engine

This allow us to store additional information about engine apart from it's
path.

Signed-off-by: 胡玮文 <huww98@outlook.com>
(cherry picked from commit ca6a8fc90b1ad567ad4d777eaab402219d5d7ffb)