Patrick Donnelly [Tue, 17 Dec 2019 23:10:34 +0000 (15:10 -0800)]
qa: add upgrade test for volume upgrade from legacy
This tests that volumes created using the ceph_volume_client.py library
continue to be accessible/function via the Nautilus/Octopus ceph-mgr
volumes plugin.
Fixes: https://tracker.ceph.com/issues/42723 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0c8899c985763826fb77fed599eeea5624263a35)
Conflicts:
qa/tasks/cephfs/fuse_mount.py
- use the coding format with nautilius
- use `require-osd-release nautilus` for this version
This commit adds the dmcrypt support in `ceph-volume raw` mode.
Note about `ceph-volume raw list` change:
Given `lsblk -J` (json output) option isn't available on all OS, I came up with
adding '--inverse' option to the existing command which allows us to get the
mapper devices list in that command output. Not listing root devices containing
partitions shouldn't have side effect since we are in `ceph-volume raw`
context.
example:
running `lsblk --paths --nodeps --output=NAME --noheadings` doesn't allow to
get the mapper list because the output is like following :
adding `--inverse` is a trick to get around this issue, the counterpart is that
we can't list root devices if they contain at least one partition but this
shouldn't be an issue in `ceph-volume raw` context given we only deal with
raw devices.
Casey Bodley [Tue, 14 Jan 2020 14:42:52 +0000 (09:42 -0500)]
qa/rgw: remove test against hadoop v2.8.5
the hadoop branch rel/release-2.8.5 fails to build with:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:37 min
[INFO] Finished at: 2020-01-14T13:09:02Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (create-parallel-tests-dirs) on project hadoop-aws: An Ant BuildException has occured: Unable to create javax script engine for javascript
Casey Bodley [Tue, 26 May 2020 19:03:03 +0000 (15:03 -0400)]
rgw: sanitize newlines in s3 CORSConfiguration's ExposeHeader
the values in the <ExposeHeader> element are sent back to clients in a
Access-Control-Expose-Headers response header. if the values are allowed
to have newlines in them, they can be used to inject arbitrary response
headers
this issue only affects s3, which gets these values from an xml document
in swift, they're given in the request header
X-Container-Meta-Access-Control-Expose-Headers, so the value itself
cannot contain newlines
Signed-off-by: Casey Bodley <cbodley@redhat.com> Reported-by: Adam Mohammed <amohammed@linode.com>
Nathan Cutler [Wed, 24 Jun 2020 19:08:40 +0000 (21:08 +0200)]
doc: PendingReleaseNotes: clean slate for 14.2.11
All of these Pending Release Notes have been included in the official
14.2.10 Release Notes, so keeping them in this file any longer would be
counterproductive.
Alfonso Martínez [Thu, 23 Jan 2020 10:16:27 +0000 (11:16 +0100)]
ceph.spec.in: fix 'make check' deps for centos8
When running 'FOR_MAKE_CHECK=1 ./install-deps.sh' in CentOS 8
these dependencies were not being installed.
Missing dependencies are provided by
https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-el8/
Kefu Chai [Tue, 24 Dec 2019 05:17:55 +0000 (13:17 +0800)]
ceph.spec.in: re-enable "make check" deps for el8
this change partially reverts e92cb7a0. as these packages are now
available in AppStream, BaseOS or PowerTools in el8, in this change,
they are re-enabled.
qa/test_exports: fix TestExports failure under new python3 compability changes
self.mount_a.client_remote.sh() returns an 'str' object rather than a StringIO object. Hence the p.stdout.getvalue() produces an error. This commit fixes this and also fix str and byte mismatch as byte and string were the same object in Python2 but this is not the case in Python3.
mgr/volumes: Create subvolume with isolated rados namespace
1. Add --namespace-isolated option to 'subvolume create' command
to create subvolume in a separate RADOS namespace
2. Add "pool_namespace" field to 'subvolume info' command
which displays the rados namespace if set else empty string
Jan Fajerski [Tue, 31 Mar 2020 14:07:45 +0000 (16:07 +0200)]
ceph-volume: add and delete lvm tags in a single lvchange call.
Otherwise we can end up in race-y situations when a concurrent c-v calls
sees only one tag but expects all tags to be present. Say if the
ceph.type tag is present, c-v expects ceph.osd_id to be present. By
setting/deleting tags in bulk, we use lvchange (and lvms internal
locking) as a sync mechanism.
Fixes: https://tracker.ceph.com/issues/44852 Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 20ecc309371e53fda5d6a5b6cf6de6110dbe5497)
Jeff Layton [Fri, 17 Apr 2020 13:55:41 +0000 (09:55 -0400)]
client: add a new inode release request callback
trim_caps() walks the list of caps on the session, and releases
non-auth caps, and attempts to trim dentries until the cache
size is under the max_caps value requested by MDS.
This is fine for FUSE, but doesn't really match the use-case of
nfs-ganesha. Ganesha typically looks up inodes by inode number, not
by dentry. It's quite possible that after a restart, we may have a
ton of outstanding inodes with no dentries associated with them.
Ganesha holds a reference to each inode, so libcephfs can't release
them, and we don't have a way to request that ganesha do so.
Add a new ino_release_callback and finisher. The intent is to allow
libcephfs to "upcall" to the application and request that it release
references to a specific inode.
Jeff Layton [Tue, 28 Apr 2020 18:00:13 +0000 (14:00 -0400)]
test: add a new program for testing ino_release_cb
Create a bunch of files and get their inode numbers. Remount, look them
all up by inode number and hold references. Stop looking up inodes as
soon as we get a callback from libcephfs. If we got the callback, return
success. Fail otherwise.
Since this has the same cluster setup as the other client_trim_caps
testcase, we can piggyback onto that task.
Jeff Layton [Tue, 21 Apr 2020 12:50:54 +0000 (08:50 -0400)]
client: only override umask_cb with non-NULL values
Client::init sets this, but if we later call ll_register_callbacks again
with a new set of function pointers that has umask_cb set to nullptr,
it'll override the value in the cmount.
Only reset umask_cb if the one in args is not nullptr.
Conflicts:
doc/cephfs/administration.rst
- nautilus has "filesystems" where master has "file systems"
- a difference that is not relevant to this backport
Kefu Chai [Mon, 25 May 2020 07:52:04 +0000 (15:52 +0800)]
qa/suites/rgw/tempest: bump up keystone to 17.0.0
Fixes: https://tracker.ceph.com/issues/45692
please note, the change on master also includes changes to use identity
v3. in this change, we only pick the change to use newer keystone to
address the pip install issue
Kefu Chai [Sun, 24 May 2020 15:06:08 +0000 (23:06 +0800)]
qa/tasks/tempest.py: install python3 for tempest test
install python3 for tempest's tox based test otherwise the test would
fail if python3.6 or python3.8 is not found.
this change is not cherry-picked from master, as our python bindings
are now built with python3.6, so python3.6 is always used. it's
supported by tempest's test.
Kefu Chai [Sun, 24 May 2020 14:51:05 +0000 (22:51 +0800)]
qa/tasks/keystone.py: install python3
install python3 for keystone's tox based test otherwise python34 would
be installed, and keystone's tox based test does not support python34.
this change is not cherry-picked from master, as our python bindings are
now built with python3.6, so python3.6 is always used. it's supported by
keystone's test.
Casey Bodley [Mon, 4 May 2020 20:39:59 +0000 (16:39 -0400)]
test/rgw: Cluster.admin() returns output as utf-8 str
fixes the TypeError:
File "rgw_multi/tests.py", line 1107, in test_bucket_sync_disable
check_buckets_sync_status_obj_not_exist(zone, buckets)
File "rgw_multi/tests.py", line 438, in check_buckets_sync_status_obj_not_exist
if log_list.find(':'+bucket+":") >= 0:
TypeError: argument should be integer or bytes-like object, not 'str'
Kefu Chai [Fri, 1 May 2020 08:52:41 +0000 (16:52 +0800)]
qa/tasks/qemu: open file in text mode
in python3, if a file is opened in 'rb' mode, we will have bytes when
reading from it, as a result, we have following failure like:
```
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested
vars.append(enter())
File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa/tasks/qemu.py", line 124, in generate_iso
test_setup = ''.join(f.readlines())
TypeError: sequence item 0: expected str instance, bytes found
```
qa/tasks/rbd: update NamedTemporaryFile usage for python3
- open the file in text mode to avoid 'bytes' objects
- 'bufsize' is now spelled 'buffering' and switching buffering off
with 'buffering=0' is only allowed in binary mode. As we need the
file to be either in the page cache or on disk before we copy it
to the remote, call flush().
1ac34a5ea3d1aca299b02e574b295dd4bf6167f4 is not backported to mimic, but
we are still running upgrade test from mimic. so in mimic, monitor
still complains at seeing too many pgs per osd.
projects / ceph.git / log