Jenkins Build Slave User [Tue, 30 Mar 2021 21:13:28 +0000 (21:13 +0000)]
16.2.0
Josh Durgin [Tue, 30 Mar 2021 21:00:12 +0000 (14:00 -0700)]
Merge pull request #40452 from smithfarm/wip-50029-pacific
pacific: rpm: drop extraneous explicit sqlite-libs runtime dependency
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Josh Durgin [Tue, 30 Mar 2021 20:12:01 +0000 (13:12 -0700)]
Merge pull request #40455 from liewegas/pacific-final-status
pacific: ceph_release: rc -> stable
Reviewed-by: Neha Ojha <nojha@redhat.com>
Josh Durgin [Tue, 30 Mar 2021 20:11:36 +0000 (13:11 -0700)]
Merge pull request #40217 from liewegas/cephadm-default-image-pacific
pacific: cephadm: update default image (DNM until right before final release)
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Sage Weil [Tue, 30 Mar 2021 00:25:54 +0000 (19:25 -0500)]
Merge PR #40473 into pacific
* refs/pull/40473/head:
mgr/cephadm/upgrade: ignore deployed_by until mgr is upgraded
Reviewed-by: Adam King <adking@redhat.com>
Sage Weil [Mon, 29 Mar 2021 13:42:03 +0000 (08:42 -0500)]
mgr/cephadm/upgrade: ignore deployed_by until mgr is upgraded
Until we upgrade the mgr itself, we will never be able to make our
deployed daemons have a deployed_by == target_digests. Ignore those
daemons until the mgr is the right version.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
bb00de5c40bf70e2b9cd14c3890d0be05d7d984b)
Sage Weil [Sun, 28 Mar 2021 21:37:26 +0000 (16:37 -0500)]
ceph_release: rc -> stable
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Sun, 28 Mar 2021 18:07:27 +0000 (13:07 -0500)]
Merge PR #40437 into pacific
* refs/pull/40437/head:
mgr/cephadm: make upgrade progress bar mention target version, not repo digest
doc/cephadm: fix rgw realm and zone flags
mgr/volumes: do not overwrite existant mds specs
mgr/cephadm: no-overwite flag for apply command
mgr/orchestrator: remove image name field from 'orch ps' and 'orch ls'
cephadm: fix parsing of keepalived version (drop leading 'v')
cephadm: keepalived needs --cap-add=NET_RAW
cephadm: fix --cap-add=NET_ADMIN
cephadm: fix quoting for keepalived env var
mgr/cephadm: ha-rgw: use correct port
cephadm: validate fsid during cephadm shell command
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Sage Weil [Sun, 28 Mar 2021 14:28:19 +0000 (09:28 -0500)]
Merge PR #40436 into pacific
* refs/pull/40436/head:
auth: require CEPHX_V2 by default
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Sage Weil [Sun, 28 Mar 2021 14:27:45 +0000 (09:27 -0500)]
Merge PR #40425 into pacific
* refs/pull/40425/head:
test_ipaddr: check that we correctly skip loopback
common/ipaddr: also skip just `lo`
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Sage Weil [Thu, 25 Mar 2021 20:05:02 +0000 (15:05 -0500)]
mgr/cephadm: make upgrade progress bar mention target version, not repo digest
The repo digest is super long and meaningless for a human user. Instead,
use the target version (as soon as we know what it is--until then, use
the target image name).
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
521cb8ab225561e0d29e215ffc63f293555ca00d)
Daniel Pivonka [Fri, 26 Mar 2021 19:13:18 +0000 (15:13 -0400)]
doc/cephadm: fix rgw realm and zone flags
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit
41a1df1b96903bf283cf4ffc40e3d18eb54c447e)
Adam King [Thu, 25 Mar 2021 20:29:03 +0000 (16:29 -0400)]
mgr/volumes: do not overwrite existant mds specs
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit
f6f7ae4c4c88324969f554d99b42b27d50209cfb)
Adam King [Wed, 24 Mar 2021 18:29:28 +0000 (14:29 -0400)]
mgr/cephadm: no-overwite flag for apply command
no-overwrite flag makes it so cephadm will only apply the spec
if the spec does not already exist.
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit
248173efcdebf67d4dae3bf8b4d26122d6eca911)
Nathan Cutler [Fri, 26 Mar 2021 10:03:34 +0000 (11:03 +0100)]
rpm: drop extraneous explicit sqlite-libs runtime dependency
Commit
75980798f19b8c11efd75ba4aae3e491d4c99f98 introduced a new package,
libcephsqlite, with a hard RPM dependency on a package "sqlite-libs" which
does not exist in openSUSE.
Since the runtime library dependencies of libcephsqlite are handled by RPM
transparently, this line is not needed.
Fixes: https://tracker.ceph.com/issues/50007
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
4025858570f270f8997478f95ffeec4eda8beafc)
Yuri Weinstein [Fri, 26 Mar 2021 19:18:37 +0000 (12:18 -0700)]
Merge pull request #40432 from smithfarm/wip-opensuse-usrmerged-pacific
pacific: ceph.spec: prepare openSUSE usrmerge (boo#
1029961)
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 17:47:55 +0000 (10:47 -0700)]
Merge pull request #40372 from batrick/49935
pacific: libcephfs: test termination "what(): Too many open files"
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 26 Mar 2021 17:31:05 +0000 (10:31 -0700)]
Merge PR #40422 into pacific
* refs/pull/40422/head:
src/tools/cephfs-top: fix flake8 line too long error
cephfs-top: include additional metrics reported by `fs perf stats`.
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Venky Shankar [Fri, 26 Mar 2021 17:19:50 +0000 (22:49 +0530)]
Merge pull request #40417 from racpatel/wip-cepfstop-fixinterval
pacific: cephfs-top: allow configurable stats refresh interval
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Venky Shankar [Fri, 26 Mar 2021 17:19:31 +0000 (22:49 +0530)]
Merge pull request #40402 from joscollin/wip-49973-pacific
pacific: cephfs-top: be resilient to missing client metadata keys
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 16:09:50 +0000 (09:09 -0700)]
Merge pull request #40358 from tchaikov/pacific-pr-39937
pacific: mgr: add mon metada using type of "mon"
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 15:34:03 +0000 (08:34 -0700)]
Merge pull request #40371 from batrick/i49932
pacific: MDS should return -ENODATA when asked to remove xattr that doesn't exist
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 15:33:25 +0000 (08:33 -0700)]
Merge pull request #40308 from varshar16/wip-49685-pacific
pacific: tools/cephfs-shell: fix listing of symbolic links
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 15:32:50 +0000 (08:32 -0700)]
Merge pull request #40307 from varshar16/wip-49713-pacific
pacific: mgr/volumes/nfs: Add command to update cephfs exports
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 15:32:25 +0000 (08:32 -0700)]
Merge pull request #40252 from singuliere/wip-49852-pacific
pacific: mds: fix race of fetching large dirfrag
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 15:31:55 +0000 (08:31 -0700)]
Merge pull request #40251 from singuliere/wip-49854-pacific
pacific: client: avoid cct being released while instances are still using it
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 15:31:08 +0000 (08:31 -0700)]
Merge pull request #40109 from lxbsz/wip-49520-pacific
pacific: client: wake up the front pos waiter
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Yuri Weinstein [Fri, 26 Mar 2021 15:30:19 +0000 (08:30 -0700)]
Merge pull request #40108 from lxbsz/wip-49609-pacific
pacific: qa: remove passed_validation check for test_damage
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Josh Durgin [Fri, 26 Mar 2021 15:25:01 +0000 (08:25 -0700)]
Merge pull request #40419 from badone/wip-pacific-revert-pr-39983
Revert "mgr/dashboard:test prometheus rules through promtool"
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Josh Durgin [Fri, 26 Mar 2021 15:18:47 +0000 (08:18 -0700)]
Merge pull request #40416 from neha-ojha/wip-onode-resiliant-to-split-cache-pacific
pacific: os/bluestore: Make Onode::put/get resiliant to split_cache
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Neha Ojha [Fri, 26 Mar 2021 15:12:34 +0000 (08:12 -0700)]
Merge pull request #40391 from neha-ojha/wip-49965-pacific
pacific: common/options: bluefs_buffered_io=true by default
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Adam King [Thu, 18 Mar 2021 17:20:46 +0000 (13:20 -0400)]
mgr/orchestrator: remove image name field from 'orch ps' and 'orch ls'
Now that we're typically using the image digests the name isn't as helpful. We also
end up in scenarios where some images use tags for their name and others use the
digest so the image name comes out as "mix" in orch ls despite it being the same image.
Fixes: https://tracker.ceph.com/issues/47333
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit
30c9210d9763ff01f4780194342fcfad6c6b6344)
Sage Weil [Wed, 24 Mar 2021 16:25:08 +0000 (12:25 -0400)]
cephadm: fix parsing of keepalived version (drop leading 'v')
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
f33bf0647bab441993777827903bf046c1c3d80a)
Sage Weil [Wed, 24 Mar 2021 16:06:48 +0000 (12:06 -0400)]
cephadm: keepalived needs --cap-add=NET_RAW
This makes
Mar 24 12:00:32 dael conmon[
3969650]: Wed Mar 24 16:00:32 2021: cant open raw socket. errno=1
go away and allows it to enter the MASTER state.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
3368844d02f52dddc25db982e586d2c2b303abf4)
Sage Weil [Wed, 24 Mar 2021 15:58:34 +0000 (11:58 -0400)]
cephadm: fix --cap-add=NET_ADMIN
Podman wants the = sign. This aligns us with the other --cap-add user
(SYS_PTRACE), which uses =.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
6a176b02b13c7551705ecffcff4285d52b58e526)
Sage Weil [Wed, 24 Mar 2021 15:57:45 +0000 (11:57 -0400)]
cephadm: fix quoting for keepalived env var
This was broken by
3ea514c5525cd21722ef3fe9b90363c21e483596
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
40e29b97863950a474adc20fccadf145e75dd8f6)
Sage Weil [Mon, 22 Mar 2021 19:30:42 +0000 (14:30 -0500)]
mgr/cephadm: ha-rgw: use correct port
The DaemonDescription includes the port that RGW is bound to; use that
in the haproxy configuration.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
401e7255062ef45e2c15d7f30dcddc3822cb67ed)
Daniel Pivonka [Wed, 10 Mar 2021 23:01:35 +0000 (18:01 -0500)]
cephadm: validate fsid during cephadm shell command
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit
9118b08365b24c3fe26f3dcdc4bf88d8ccbcbce0)
Ilya Dryomov [Wed, 24 Mar 2021 15:23:44 +0000 (16:23 +0100)]
auth: require CEPHX_V2 by default
It's been almost three years and support is present in all relevant
clients.
From the security perspective, roughly the same could be achieved
with "ceph osd set-require-min-compat-client nautilus", but this is
more user friendly as the client gets ENOTSUP instead of spinning on
"feature set mismatch" faults.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit
e5744672dbde2a897f5f4959339472b7b10c5688)
Sage Weil [Fri, 26 Mar 2021 12:17:42 +0000 (07:17 -0500)]
Merge PR #40355 into pacific
* refs/pull/40355/head:
mgr/cephadm: Fix dashboard gateway configuration when using IPV6
qa/workunits/cephadm/test_cephadm: specify image separately
mgr/cephadm: retry after JSONDecodeError in wait_for_mgr_restart()
cephadm: prevent podman from breaking socket.getfqdn()
qa/tasks/cephadm: use 'orch apply mon' to deploy mons
qa/suites/rados/cephadm/upgrade: add centos upgrade on latest octopus
mgr/cephadm/upgrade: do not crash if error races with user cancellation
doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port
cephadm: only bootstrap using image that matches cephadm version
mgr/cephadm: redeploy daemons deployed using old image during upgrade
mgr/cephadm: add container digests of mgr that deployed daemon to unit.meta
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Sage Weil [Fri, 26 Mar 2021 12:17:14 +0000 (07:17 -0500)]
Merge PR #40410 into pacific
* refs/pull/40410/head:
os/bluestore: separate omap per-pool vs per-pg alerts
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Varsha Rao [Fri, 26 Mar 2021 09:38:33 +0000 (15:08 +0530)]
src/tools/cephfs-top: fix flake8 line too long error
Fixes: https://tracker.ceph.com/issues/50005
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
51cf6c3d14883797faa2c2ea4a94f65f26bdf2ef)
Ludwig Nussel [Thu, 25 Mar 2021 09:32:21 +0000 (10:32 +0100)]
ceph.spec: prepare openSUSE usrmerge (boo#
1029961)
The compat symlink in /sbin is no longer required and actually in
the way in the usrmerge case.
Signed-off-by: Ludwig Nussel <ludwig.nussel@suse.de>
(cherry picked from commit
e4c4a4ce97fff8a5b4efa747d9cffeabcceedd25)
Dan van der Ster [Tue, 23 Mar 2021 10:28:37 +0000 (11:28 +0100)]
test_ipaddr: check that we correctly skip loopback
We should skip devices named 'lo' or of the form 'lo:0' regardless
of their IP address.
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Related-to: https://tracker.ceph.com/issues/49938
(cherry picked from commit
780125d1ed93cd7b17172752b3e76186a524103b)
Dan van der Ster [Tue, 23 Mar 2021 08:00:11 +0000 (09:00 +0100)]
common/ipaddr: also skip just `lo`
Skip iface's with name like 'lo' or of the form 'lo:0', 'lo:1'. This
brings back the original behavior from
b6d0fc9e0e515e50894c08217d688a8c94db7570
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Fixes: https://tracker.ceph.com/issues/49938
(cherry picked from commit
6147c0917157efd2d35610e759685656a4989abb)
Venky Shankar [Thu, 25 Mar 2021 09:40:28 +0000 (05:40 -0400)]
cephfs-top: include additional metrics reported by `fs perf stats`.
Without this, `cephfs-top` hits an exception since the additional
metrics keys were not configured.
Also, include a validation suring selftest that checks if `ceph
fs perf stats` metrics match what cephfs-top is configured to
report.
Fixes: http://tracker.ceph.com/issues/49974
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit
5a119dcc13354549071dddae96e9fb6f7df747e5)
Kefu Chai [Fri, 26 Mar 2021 01:27:23 +0000 (09:27 +0800)]
Merge pull request #40405 from tchaikov/pacific-pr-40400
pacific: run-make-check.sh: let ctest generate XML output
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Brad Hubbard [Thu, 25 Mar 2021 23:57:14 +0000 (09:57 +1000)]
Revert "mgr/dashboard:test prometheus rules through promtool"
Reverts: https://github.com/ceph/ceph/pull/39983
This is currently blocking testing on ubuntu on the eve of a pacific
release. The problems associated with this PR have been resolved
upstream but have not been backported yet and are non-trivial.
This reverts commit
be7f9e704c8d9ab70713a78c9a83481b5e26ee79.
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
Neha Ojha [Thu, 25 Mar 2021 17:42:01 +0000 (10:42 -0700)]
Merge pull request #40361 from sseshasa/wip-pacific-release-note
pacific: PendingReleaseNotes: Document mclock scheduler refinements and profiles
Reviewed-by: Neha Ojha <nojha@redhat.com>
Rachana Patel [Tue, 23 Mar 2021 08:47:34 +0000 (08:47 +0000)]
cephfs-top: signal main thread to exit on SIGINT/SIGTERM
Fixes: http://tracker.ceph.com/issues/49953
Signed-off-by: Rachana Patel <racpatel@redhat.com>
(cherry picked from commit
6ea5cb5ff3380ccf6c2e0ca3f838f803173d0582)
Rachana Patel [Tue, 23 Mar 2021 04:40:56 +0000 (04:40 +0000)]
cephfs-top: allow configurable stats refresh interval
Signed-off-by: Rachana Patel <racpatel@redhat.com>
(cherry picked from commit
abd4ae9f9b1bdf1f4d7ee7b10baa9c8ec03303fc)
Igor Fedotov [Fri, 19 Mar 2021 09:53:20 +0000 (12:53 +0300)]
os/bluestore: acquire proper lock in split_cache()
Fixes: https://tracker.ceph.com/issues/49900
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
35a3f7be8f2f204ad3b5e720d0534ca3e2a8587c)
Adam Kupczyk [Mon, 22 Mar 2021 10:20:11 +0000 (11:20 +0100)]
os/bluestore: Make Onode::put/get resiliant to split_cache
In
OnodeCacheShard* ocs = c->get_onode_cache();
std::lock_guard l(ocs->lock);
while waiting for lock, split_cache might have changed OnodeCacheShard.
This will result in adding Onode to improper OnodeCacheShard.
Such action is obviously bad, as we will operate in future (at least once) on
different OnodeCacheShard then we got lock for. Particulary sensitive to this
are _trim and split_cache functions, as they iterate over elements.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit
343b049a1328d39a69a8c4c9e9cb93ac6ac77280)
Yuri Weinstein [Thu, 25 Mar 2021 17:35:30 +0000 (10:35 -0700)]
Merge pull request #40325 from batrick/49930
pacific: mon/MDSMonitor: standby-replay daemons should be removed when the flag is turned off
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Thu, 25 Mar 2021 17:34:09 +0000 (10:34 -0700)]
Merge pull request #40267 from kotreshhr/wip-49905-pacific
pacific: mgr/volumes: Retain suid guid bits in clone
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Thu, 25 Mar 2021 17:33:27 +0000 (10:33 -0700)]
Merge pull request #40091 from batrick/i49610
pacific: qa: mds removed because trimming for too long with valgrind
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Thu, 25 Mar 2021 17:32:42 +0000 (10:32 -0700)]
Merge pull request #40069 from petrutlucian94/wip-49634-pacific
pacific: cephfs: Add ceph-dokan, providing Windows support
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Thu, 25 Mar 2021 17:32:07 +0000 (10:32 -0700)]
Merge pull request #39974 from singuliere/wip-49346-pacific
pacific: vstart: fix ganesha cluster id and rados url errors
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
Yuri Weinstein [Thu, 25 Mar 2021 17:30:35 +0000 (10:30 -0700)]
Merge pull request #39972 from singuliere/wip-49687-pacific
pacific: mds: add opened files/inodes and pinned i_caps metric payload support
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Venky Shankar [Thu, 25 Mar 2021 17:05:09 +0000 (22:35 +0530)]
Merge pull request #40206 from vshankar/wip-cephfs-mirror-pacific-backport-2
pacific: cephfs-mirror: peer bootstrap
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Thu, 25 Mar 2021 17:00:58 +0000 (10:00 -0700)]
Merge pull request #40349 from neha-ojha/wip-39729-pacific
pacific: os/bluestore: Add use_direct_io function
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Sage Weil [Tue, 23 Mar 2021 16:56:59 +0000 (11:56 -0500)]
os/bluestore: separate omap per-pool vs per-pg alerts
Currently the health alert raised does not match the docs, and the docs
do not describe what the health alert indicates.
Octopus added per-pool omap storage. This improves space accounting
and reporting.
Pacific added per-pg omap storage (object hash in key). This speeds up
PG removal.
Separate everthing out into two distinct alerts raised from bluestore
and surfaced as health alerts, with corresponding config options to
disable, and update the docs accordingly.
Also update the fsck options for warn vs error, and raise separate
errors for the per-pg and per-pool cases.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
f755e353e88b65fb923289464935ec635daf13b2)
Juan Miguel Olmo Martínez [Wed, 24 Mar 2021 14:30:38 +0000 (15:30 +0100)]
mgr/cephadm: Fix dashboard gateway configuration when using IPV6
Fixes: https://tracker.ceph.com/issues/49957
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit
1b18f4f9cb28708b544c62b3d07f9e1b4c701e41)
Sage Weil [Thu, 25 Mar 2021 12:33:26 +0000 (07:33 -0500)]
qa/workunits/cephadm/test_cephadm: specify image separately
On master, this will be IMAGE_MASTER. On pacific branch, we'll make it
IMAGE_PACIFIC so that we don't hit the release mismatch error.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
4d5c6044b062d7e61b1bb0c5c1efb705878517bf)
- adjusted IMAGE_DFEAULT to be pacific
Kefu Chai [Thu, 25 Mar 2021 09:08:48 +0000 (17:08 +0800)]
run-make-check.sh: let ctest generate XML output
to enable XUnit plugin of jenkins to consume the ctest output and
publish it in the dashboard, we need to
* let ctest generate XML output instead of plain text output
* do not fail the test if any test case fails. this allows the publisher
to do its job by checking the XML output.
* prevent ctest from compressing the output. see
https://issues.jenkins.io/browse/JENKINS-21737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
48ba39987d3958531589d7969750ea749e6a6d30)
Jos Collin [Wed, 17 Mar 2021 12:18:09 +0000 (17:48 +0530)]
tools/cephfs: fix missing keys in cephfs-top client_metadata
Fixes: https://tracker.ceph.com/issues/49736
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit
3fd63e97b41b9f648390e796d60451ddee2143db)
Dan van der Ster [Thu, 12 Nov 2020 16:14:37 +0000 (17:14 +0100)]
common/options: bluefs_buffered_io=true by default
Enable bluefs_buffered_io again because it makes a huge user-visible
improvement in metadata intensive scenarios, such as but not limited to
PG deletion.
In our environment, deleting PGs from 4 hybrid OSDs (sharing one SATA SSD block.db) saturates
the block.db at 350MB/s reads and causes slow reqs and flapping on the OSDs.
Those OSDs have 3GB osd_target_memory.
Enabling bluefs_buffered_io drops the SSD IO down to <1MBps and the OSDs
are performant again. (The underlying PG deletion inefficiency is being
solved separately, but the page cache is so much more effective than
the bluestore cache in this scenario).
Lastly, remove the comment about swap. We should separately advise
operators to disable swap on OSD machines, as it is much better in
our experience to OOM and restart than to chug along swapping.
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Related-to: https://tracker.ceph.com/issues/45765
Related-to: https://tracker.ceph.com/issues/47044
(cherry picked from commit
5ec8e8e63d409860c35e24a192090ac2b70af8f6)
Patrick Donnelly [Wed, 24 Mar 2021 23:11:03 +0000 (16:11 -0700)]
Merge PR #40317 into pacific
* refs/pull/40317/head:
cephsqlite: add julian day offset in milliseconds
doc: add libcephsqlite
ceph.spec,debian: package libcephsqlite
test/libcephsqlite,qa: add tests for libcephsqlite
libcephsqlite: rework architecture and backend
SimpleRADOSStriper: wait for finished aios after write
SimpleRADOSStriper: add new minimal async striper
mon: define simple-rados-client-with-blocklist profile
librados: define must renew lock flag
common: add timeval conversion for durations
Revert "libradosstriper: add function to read into char*"
test_libcephsqlite: test random inserts
cephsqlite: fix compiler errors
cmake: improve build inst for cephsqlite
libcephsqlite: sqlite interface to RADOS
libradosstriper: add function to read into char*
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
John Fulton [Wed, 17 Mar 2021 22:03:46 +0000 (18:03 -0400)]
mgr/cephadm: retry after JSONDecodeError in wait_for_mgr_restart()
'ceph mgr dump' does not always return valid JSON so cephadm
will throw an exception sometimes when applying a spec as per
the issue this PR closes. Add a try/except to catch a possible
JSONDecodeError and retry after sleeping.
Fixes: https://tracker.ceph.com/issues/49870
Signed-off-by: John Fulton <fulton@redhat.com>
(cherry picked from commit
0aba5704d9eb1a2df6dd437785fc1f8c558c0990)
Sage Weil [Thu, 18 Mar 2021 18:26:48 +0000 (14:26 -0400)]
cephadm: prevent podman from breaking socket.getfqdn()
socket.getfqdn() will return the reverse lookup for 127.0.1.1, which is
the last item listed for that IP in /etc/hosts. Podman, by default, will
append the container name (ceph-$fsid-$name) to that line, which is not
a valid hostname, and not what we want the dashbaord to use for the URI
it advertises in the service map.
Pass --no-hosts to podman to disable this.
Docker does not appear to modify /etc/hosts by default--or, more
importantly, does not add the container name there.
Explicitly instruct podman (and docker) to add a
Fixes: https://tracker.ceph.com/issues/49890
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
cfc1f914ce74f1fd1f45e2efd3ba2ddcb2da129a)
Sridhar Seshasayee [Wed, 24 Mar 2021 07:52:03 +0000 (13:22 +0530)]
PendingReleaseNotes: Document mclock scheduler refinements and profiles
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Ernesto Puerta [Wed, 24 Mar 2021 16:11:18 +0000 (17:11 +0100)]
Merge pull request #39999 from rhcs-dashboard/wip-49708-pacific
pacific: mgr/dashboard: cluster > hosts: replace inventory->devices table title
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Jeff Layton [Wed, 17 Mar 2021 15:52:05 +0000 (11:52 -0400)]
test: reduce number of threads to 32 in LibCephFS.ShutdownRace
We're still occasionally hitting file descriptor limits when running
this test. Reduce the thread count to 32 for now, since it was possible
to reproduce the original problem with 10 or so threads.
Fixes: https://tracker.ceph.com/issues/49559
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit
5aec283a1c33b6c21f877a27f57a1bc03b4894a0)
Jeff Layton [Tue, 16 Mar 2021 15:14:28 +0000 (11:14 -0400)]
test: add test for removing non-existent xattr
We should get back -ENODATA.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit
b94b668b7e340fa660462b75c97ab77dda9d5273)
Jeff Layton [Tue, 16 Mar 2021 16:22:56 +0000 (12:22 -0400)]
mds: fix removexattr logic when there aren't any
The MDS currently returns success on a removexattr if the xattr map is
completely empty. Fix the subtle logic bug and have it return -ENODATA
in that case.
Fixes: https://tracker.ceph.com/issues/49833
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit
85e73c7c7509cefbc50902436aca07a9a333eb23)
Patrick Donnelly [Wed, 24 Mar 2021 15:52:55 +0000 (08:52 -0700)]
Merge PR #39988 into pacific
* refs/pull/39988/head:
doc: fix broken links multimds and kcephfs
Reviewed-by: Jos Collin <jcollin@redhat.com>
Patrick Donnelly [Wed, 24 Mar 2021 15:49:18 +0000 (08:49 -0700)]
Merge PR #40250 into pacific
* refs/pull/40250/head:
doc: mds cap acquisition readdir throttle documentation
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Wed, 24 Mar 2021 15:48:51 +0000 (08:48 -0700)]
Merge PR #40306 into pacific
* refs/pull/40306/head:
doc/cephfs/nfs: Add rook pod restart note, export and log block example
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Casey Bodley [Wed, 24 Mar 2021 15:47:46 +0000 (11:47 -0400)]
Merge pull request #40255 from singuliere/wip-49743-pacific
pacific: rgw: limit rgw_gc_max_objs to RGW_SHARDS_PRIME_1
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Mar 2021 15:47:38 +0000 (11:47 -0400)]
Merge pull request #40261 from singuliere/wip-49310-pacific
WIP: pacific: rgw: add support for SSL encrypted AMQP connections
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Mar 2021 15:47:22 +0000 (11:47 -0400)]
Merge pull request #40338 from cbodley/wip-rgw-lc-lock-pacific
pacific: rgw/lc: Fix use-after-free in RGWLC::process
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Patrick Donnelly [Wed, 24 Mar 2021 15:45:12 +0000 (08:45 -0700)]
Merge PR #40362 into pacific
* refs/pull/40362/head:
doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Sage Weil [Wed, 24 Mar 2021 15:34:03 +0000 (10:34 -0500)]
Merge PR #40094 into pacific
* refs/pull/40094/head:
rgw/kms/vault - PendingReleaseNotes pointer
rgw/kms/vault - s3tests for both old and new test logic.
rgw/kms/vault - rework unit test logic for new transit logic.
rgw/kms/vault - 0 terminate before rapidjson
rgw/kms/vault - document configuration for new transit logic
rgw/kms/vault - new transit logic - fix compat logic
rgw/kms/vault - define attribute for new transit logic
rgw/kms/vault - "compat" option
rgw/kms/vault - encryption context - first part
rgw/kms/vault - define attribute to store encryption context
rgw/kms/vault - share get/set attr between rgw_crypt.cc and rgw_kms.cc
rgw/kms/vault - relax configuration parsing for rgw_crypt_vault_secret_engine
rgw/kms/vault - need libicu to make canonical json for encryption contexts.
rgw/kms/kmip - document configuration for a new feature: kmip kms
rgw/kms/kmip - rgw / kmip test integration.
rgw/kms/kmip - rgw / kmip test integration.
rgw/kms/kmip - rgw / kmip test integration.
rgw/kms/kmip - correct documentation.
rgw/kms/kmip - pykmip.py needs to make keys too.
rgw/kms/kmip - pykmip.py should actually run pykmip.
rgw/kms/kmip - python3 changes for testing.
rgw/kms/kmip - string handling cleanup.
teuthology/rgw: pykmip task
kmip: first pass at implementation logic.
kmip: configuration options.
Including cmake build logic inside of libkmip.
cmake glue to build libkmip.
Added libkmip as a submodule.
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Patrick Donnelly [Wed, 24 Mar 2021 00:34:45 +0000 (17:34 -0700)]
cephsqlite: add julian day offset in milliseconds
This magic number was copied from another VFS but was not adjusted for
the xCurrentTimeInt64 interface.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
9107e1a34743167d8f0cf969a440f5aaed2848af)
Casey Bodley [Wed, 24 Mar 2021 14:11:51 +0000 (10:11 -0400)]
Merge pull request #40262 from singuliere/wip-49176-pacific
pacific: rgw: Fix spurious error on empty datalog shard
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Mar 2021 14:11:31 +0000 (10:11 -0400)]
Merge pull request #40256 from singuliere/wip-49722-pacific
pacific: qa/rgw: add librgw_file unit tests to rgw/verify task
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Mar 2021 14:11:23 +0000 (10:11 -0400)]
Merge pull request #40257 from singuliere/wip-49601-pacific
pacific: rgw/sts: fix for encoding/decoding user namespace
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Mar 2021 14:11:15 +0000 (10:11 -0400)]
Merge pull request #40258 from singuliere/wip-49587-pacific
pacific: rgw: added missing documentation on "pubsub" in rgw_enable_apis
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Mar 2021 14:11:07 +0000 (10:11 -0400)]
Merge pull request #40259 from singuliere/wip-49381-pacific
pacific: rgw: multisite: fix single-part-MPU object etag misidentify problem
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Mar 2021 14:10:57 +0000 (10:10 -0400)]
Merge pull request #40260 from singuliere/wip-49362-pacific
pacific: rgw: fix sts get_session_token duration check failed
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Jason Dillaman [Wed, 24 Mar 2021 14:07:20 +0000 (10:07 -0400)]
Merge pull request #40339 from dillaman/wip-librbd-backports-pacific-10
pacific: librbd: miscellaneous backports
Reviewed-by: Mykola Golub <mgolub@suse.com>
Ernesto Puerta [Wed, 24 Mar 2021 11:58:18 +0000 (12:58 +0100)]
Merge pull request #40340 from rhcs-dashboard/wip-49941-pacific
pacific: mgr/dashboard: fix error shown when no rgw daemons are running.
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Ernesto Puerta [Wed, 24 Mar 2021 11:32:04 +0000 (12:32 +0100)]
Merge pull request #40360 from rhcs-dashboard/wip-49944-pacific
pacific: mgr/dashboard: Display users current quota usage
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Varsha Rao [Mon, 22 Mar 2021 11:51:53 +0000 (17:21 +0530)]
doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port
Fixes: https://tracker.ceph.com/issues/49921
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
fec8bbf5cd3e3dd72d05c4eb23209c84f056edda)
Avan Thakkar [Sat, 1 Aug 2020 10:36:13 +0000 (16:06 +0530)]
mgr/dashboard: Display users current quota usage
Fixes: https://tracker.ceph.com/issues/45965
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit
9456884c2996d31ce9c39e39011d295c5e62bcf7)
Kefu Chai [Tue, 9 Mar 2021 04:51:55 +0000 (12:51 +0800)]
mgr: add mon metada using type of "mon"
this change addresses a regression introduced by
c037f4cb5d7436879d58c34748ef516b5269781f
also remove the "P" before the json command.
see also: https://tracker.ceph.com/issues/48905
Fixes: https://tracker.ceph.com/issues/49661
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
8fc290bfba4d71a60d30c2374ce4bcba37e649de)
Sage Weil [Mon, 22 Mar 2021 13:58:39 +0000 (08:58 -0500)]
qa/tasks/cephadm: use 'orch apply mon' to deploy mons
The 'orch daemon add ...' command is not idempotent and can cause
duplicate (and failing) attempts to add the same mon.
Maintain the ability to add mons the old way for the benefit of testing
upgrades from early octopus versions of cephadm.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
a17593a177798e5b5371c224ede40dc411626d76)
Sage Weil [Fri, 12 Mar 2021 16:49:10 +0000 (10:49 -0600)]
qa/suites/rados/cephadm/upgrade: add centos upgrade on latest octopus
centos currently has the semi-broken podman 2.2.1 (no json function
for inspect), but the latest octopus (15.2.10) works around it.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
c6a75d05565f327e7b7a3971e89f5a3e730299a6)
Sage Weil [Mon, 22 Mar 2021 22:40:25 +0000 (18:40 -0400)]
mgr/cephadm/upgrade: do not crash if error races with user cancellation
If the user cancels the upgrade just before the upgrade thread runs into
a problem (and these things may be correlated!), ignore the failure
instead of crashing the module.
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit
2580de1a5db476b01093a269056d2f5bd0cc3d75)
Varsha Rao [Mon, 22 Mar 2021 11:51:53 +0000 (17:21 +0530)]
doc/cephfs/nfs: Add note about cephadm NFS-Ganesha daemon port
Fixes: https://tracker.ceph.com/issues/49921
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit
fec8bbf5cd3e3dd72d05c4eb23209c84f056edda)