"ceph fs subvolume snapshot info <vol_name> <sub_name> <snap_name> [<group_name>]"
The output is in json format with following fields
created_at: time of creation of snapshot in the format "YYYY-MM-DD HH:MM:SS:ffffff"
data_pool: data pool the snapshot belongs to
has_pending_clones: "yes" if snapshot clone is in progress otherwise "no"
protected: "yes" if snapshot is protected otherwise "no"
size: snapshot size in bytes
Alfonso Martínez [Thu, 23 Jan 2020 10:16:27 +0000 (11:16 +0100)]
ceph.spec.in: fix 'make check' deps for centos8
When running 'FOR_MAKE_CHECK=1 ./install-deps.sh' in CentOS 8
these dependencies were not being installed.
Missing dependencies are provided by
https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-el8/
Kefu Chai [Tue, 24 Dec 2019 05:17:55 +0000 (13:17 +0800)]
ceph.spec.in: re-enable "make check" deps for el8
this change partially reverts e92cb7a0. as these packages are now
available in AppStream, BaseOS or PowerTools in el8, in this change,
they are re-enabled.
qa/test_exports: fix TestExports failure under new python3 compability changes
self.mount_a.client_remote.sh() returns an 'str' object rather than a StringIO object. Hence the p.stdout.getvalue() produces an error. This commit fixes this and also fix str and byte mismatch as byte and string were the same object in Python2 but this is not the case in Python3.
mgr/volumes: Create subvolume with isolated rados namespace
1. Add --namespace-isolated option to 'subvolume create' command
to create subvolume in a separate RADOS namespace
2. Add "pool_namespace" field to 'subvolume info' command
which displays the rados namespace if set else empty string
Jan Fajerski [Tue, 31 Mar 2020 14:07:45 +0000 (16:07 +0200)]
ceph-volume: add and delete lvm tags in a single lvchange call.
Otherwise we can end up in race-y situations when a concurrent c-v calls
sees only one tag but expects all tags to be present. Say if the
ceph.type tag is present, c-v expects ceph.osd_id to be present. By
setting/deleting tags in bulk, we use lvchange (and lvms internal
locking) as a sync mechanism.
Fixes: https://tracker.ceph.com/issues/44852 Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 20ecc309371e53fda5d6a5b6cf6de6110dbe5497)
Jeff Layton [Fri, 17 Apr 2020 13:55:41 +0000 (09:55 -0400)]
client: add a new inode release request callback
trim_caps() walks the list of caps on the session, and releases
non-auth caps, and attempts to trim dentries until the cache
size is under the max_caps value requested by MDS.
This is fine for FUSE, but doesn't really match the use-case of
nfs-ganesha. Ganesha typically looks up inodes by inode number, not
by dentry. It's quite possible that after a restart, we may have a
ton of outstanding inodes with no dentries associated with them.
Ganesha holds a reference to each inode, so libcephfs can't release
them, and we don't have a way to request that ganesha do so.
Add a new ino_release_callback and finisher. The intent is to allow
libcephfs to "upcall" to the application and request that it release
references to a specific inode.
Jeff Layton [Tue, 28 Apr 2020 18:00:13 +0000 (14:00 -0400)]
test: add a new program for testing ino_release_cb
Create a bunch of files and get their inode numbers. Remount, look them
all up by inode number and hold references. Stop looking up inodes as
soon as we get a callback from libcephfs. If we got the callback, return
success. Fail otherwise.
Since this has the same cluster setup as the other client_trim_caps
testcase, we can piggyback onto that task.
Jeff Layton [Tue, 21 Apr 2020 12:50:54 +0000 (08:50 -0400)]
client: only override umask_cb with non-NULL values
Client::init sets this, but if we later call ll_register_callbacks again
with a new set of function pointers that has umask_cb set to nullptr,
it'll override the value in the cmount.
Only reset umask_cb if the one in args is not nullptr.
Conflicts:
doc/cephfs/administration.rst
- nautilus has "filesystems" where master has "file systems"
- a difference that is not relevant to this backport
Kefu Chai [Mon, 25 May 2020 07:52:04 +0000 (15:52 +0800)]
qa/suites/rgw/tempest: bump up keystone to 17.0.0
Fixes: https://tracker.ceph.com/issues/45692
please note, the change on master also includes changes to use identity
v3. in this change, we only pick the change to use newer keystone to
address the pip install issue
Kefu Chai [Sun, 24 May 2020 15:06:08 +0000 (23:06 +0800)]
qa/tasks/tempest.py: install python3 for tempest test
install python3 for tempest's tox based test otherwise the test would
fail if python3.6 or python3.8 is not found.
this change is not cherry-picked from master, as our python bindings
are now built with python3.6, so python3.6 is always used. it's
supported by tempest's test.
Kefu Chai [Sun, 24 May 2020 14:51:05 +0000 (22:51 +0800)]
qa/tasks/keystone.py: install python3
install python3 for keystone's tox based test otherwise python34 would
be installed, and keystone's tox based test does not support python34.
this change is not cherry-picked from master, as our python bindings are
now built with python3.6, so python3.6 is always used. it's supported by
keystone's test.
Casey Bodley [Mon, 4 May 2020 20:39:59 +0000 (16:39 -0400)]
test/rgw: Cluster.admin() returns output as utf-8 str
fixes the TypeError:
File "rgw_multi/tests.py", line 1107, in test_bucket_sync_disable
check_buckets_sync_status_obj_not_exist(zone, buckets)
File "rgw_multi/tests.py", line 438, in check_buckets_sync_status_obj_not_exist
if log_list.find(':'+bucket+":") >= 0:
TypeError: argument should be integer or bytes-like object, not 'str'
Kefu Chai [Fri, 1 May 2020 08:52:41 +0000 (16:52 +0800)]
qa/tasks/qemu: open file in text mode
in python3, if a file is opened in 'rb' mode, we will have bytes when
reading from it, as a result, we have following failure like:
```
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested
vars.append(enter())
File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa/tasks/qemu.py", line 124, in generate_iso
test_setup = ''.join(f.readlines())
TypeError: sequence item 0: expected str instance, bytes found
```
qa/tasks/rbd: update NamedTemporaryFile usage for python3
- open the file in text mode to avoid 'bytes' objects
- 'bufsize' is now spelled 'buffering' and switching buffering off
with 'buffering=0' is only allowed in binary mode. As we need the
file to be either in the page cache or on disk before we copy it
to the remote, call flush().
1ac34a5ea3d1aca299b02e574b295dd4bf6167f4 is not backported to mimic, but
we are still running upgrade test from mimic. so in mimic, monitor
still complains at seeing too many pgs per osd.
qa/tasks: decode bytes returned by base64.b64encode()
we use the return value from `base64.b64encode()` as a string, this works
in python2, but in python3, instead of a string, `base64.b64encode()`
returns bytes. so for instance, if we try to compose command line
arguments using `pipes.quote()` by passing a byte instance, we will have
```
TypeError: cannot use a string pattern on a bytes-like object
```
in this change, the retval is always decoded using ASCII, because base64
only returns ASCII strings when encoding.
this change is not cherry-picked from master, as the changed files do
not exist in master anymore.
qa/tasks/openssl_keys.py: sort cert configs before creating certs
we cannot rely on the order in which items are arranged in a dict, the
order varies from version to another. in Python2, it happens to work,
and we can always have the self-signed cert added first. but in Python3,
it does not. and an exception is thrown
```
teuthology.exceptions.ConfigError: ssl: ca root not found for
certificate rgw.client.0
```
in this change, before creating certs, the settings are reordered so
that the self-signed ones are created first.
in boto, it tries to figure out the MIME type of a file by its name, if
the file-like objects has an attribute of "name". in Python2, the "name"
is always "<fdopen>", fortunately. while in Python3, `TemporaryFile` also
have a "name" which is its fd, and it is an integer now. so we have following
error when sending a `TemporaryFile` using
`upload_part_from_file()`:
```
2020-04-08T02:25:34.660 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last):
2020-04-08T02:25:34.661 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
2020-04-08T02:25:34.661 INFO:tasks.rgw_multisite_tests: self.test(*self.arg)
2020-04-08T02:25:34.662 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_tchaikov_ceph_wip-qa-py3/qa/tasks/rgw_multi/tests_ps.py", line 2567, in test_ps_creation_triggers
2020-04-08T02:25:34.662 INFO:tasks.rgw_multisite_tests: uploader.upload_part_from_file(fp, 1)
2020-04-08T02:25:34.663 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/boto/s3/multipart.py", line 260, in upload_part_from_file
2020-04-08T02:25:34.663 INFO:tasks.rgw_multisite_tests: query_args=query_args, size=size)
2020-04-08T02:25:34.664 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/boto/s3/key.py", line 1293, in set_contents_from_file
2020-04-08T02:25:34.664 INFO:tasks.rgw_multisite_tests: chunked_transfer=chunked_transfer, size=size)
2020-04-08T02:25:34.664 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/boto/s3/key.py", line 750, in send_file
2020-04-08T02:25:34.665 INFO:tasks.rgw_multisite_tests: chunked_transfer=chunked_transfer, size=size)
2020-04-08T02:25:34.665 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/boto/s3/key.py", line 920, in
_send_file_internal
2020-04-08T02:25:34.666 INFO:tasks.rgw_multisite_tests: self.content_type = mimetypes.guess_type(self.path)[0]
2020-04-08T02:25:34.666 INFO:tasks.rgw_multisite_tests: File "/usr/lib/python3.5/mimetypes.py", line 289, in guess_type
2020-04-08T02:25:34.667 INFO:tasks.rgw_multisite_tests: return _db.guess_type(url, strict)
2020-04-08T02:25:34.667 INFO:tasks.rgw_multisite_tests: File "/usr/lib/python3.5/mimetypes.py", line 114, in guess_type
2020-04-08T02:25:34.667 INFO:tasks.rgw_multisite_tests: scheme, url = urllib.parse.splittype(url)
2020-04-08T02:25:34.668 INFO:tasks.rgw_multisite_tests: File "/usr/lib/python3.5/urllib/parse.py", line 881, in splittype
2020-04-08T02:25:34.668 INFO:tasks.rgw_multisite_tests: match = _typeprog.match(url)
2020-04-08T02:25:34.669 INFO:tasks.rgw_multisite_tests:TypeError: expected string or bytes-like object
```
to address this issue, in this change, a `NamedTemporaryFile` is used
instead of `TemporaryFile`. the former does have a "name" which is a
`str`.
to address the test failures like
```
2020-04-07T15:44:58.693 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: ceph pg dump
pgs
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: pgid
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh: line 498: pgid: command not found
```
as the caller might want to `len(manager.get_osd_status()['raw'])`, and
`len()` does not accept a `filter` object.
also, the filtered osd statuses are printed out using `self.log()`, so
we should materialize the `filter` object before sending it to logging
facility. otherwise we will have something like:
```
2020-04-08T02:58:37.001 INFO:tasks.ceph.ceph_manager.ceph:<filter object at 0x7f5a080e1518>
```
in the logging message.
projects / ceph.git / log