Venky Shankar [Wed, 23 Dec 2020 05:14:31 +0000 (00:14 -0500)]
test: move snap delete operation to non-root directory
In teuthology, root directory in a Ceph Filesystem is owned
by root (0:0) and has sticky-bit enabled. Snapshots created
on the directory carry its UID/GID and mode.
This causes tests that remove snapshots on root directory to
fail. Sticky bit on a directory gives permission to delete a
file in that directory only if you own the file.
Venky Shankar [Wed, 23 Dec 2020 11:34:22 +0000 (06:34 -0500)]
client: optionally check client permission on rmsnap()
By default, Client::rmsnap() skips checking client permissions.
This was ok till now since it was not a user facing API (via
libcephfs).
With PR #37721 ("mds: store custom metadata on snapshot creation"),
rmsnap can be invoked via ceph_rmsnap() libcephfs call. Therefore,
the usual client side permission checks should be performed before
calling out to the MDS.
Patrick Donnelly [Tue, 22 Dec 2020 20:58:11 +0000 (12:58 -0800)]
Merge PR #37721 into master
* refs/pull/37721/head:
test: add tests for new snapshot APIs
pybind/cephfs: python bindings for new snapshot APIs
client: Snapshot cephfs APIs
mds: include snapshot metadata in trace reply to client
mds: include snap metadata on-disk on snapshot creation
mds: include metadata in SnapInfo structure
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Zac Dover [Thu, 19 Nov 2020 14:18:51 +0000 (00:18 +1000)]
doc/dev: update unit-tests tox section
This commit
- alters the structure of the
unit-test section of the Developer Guide
that discusses the tox tool, which is
used to run certain unit tests
- corrects some grammar
- updates the list of locations of tox.ini
files
- This also incorporates N. Cutler's suggestions.
Xuehan Xu [Fri, 18 Dec 2020 04:20:48 +0000 (12:20 +0800)]
crimson/osd: don't get recovery read lock in PGRecvery::on_local_recover()
Now we've changed drop_recovery_read to only care about the recovery_read flag, so
we shouldn't get read lock when acquiring recovery read, otherwise there would be
a chance in which the read lock can't get released
Jason Dillaman [Fri, 18 Dec 2020 15:14:13 +0000 (10:14 -0500)]
librbd: drop explicit masking of implicit feature bits
Now that the create image state machine is handling the masking
of implicit features, all callers to the state machine can skip
the need to perform the masking themselves.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Fri, 18 Dec 2020 14:55:30 +0000 (09:55 -0500)]
librbd/image: mask out all implicit features when creating an image
This will ensure that all paths to the create image state machine
properly handle this condition. Previously, it was up to the callers
of the state machine to clear the implicit feature bits.
Fixes: https://tracker.ceph.com/issues/48647 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Dan Mick [Thu, 17 Dec 2020 02:06:31 +0000 (02:06 +0000)]
qa/distros: configure podman to pull from docker-mirror
This also requires a change to cephadm.py to search for
registry entries for docker.io by 'prefix' or 'location',
and to only add the mirror if a mirror isn't already present
Venky Shankar [Tue, 24 Nov 2020 09:03:23 +0000 (04:03 -0500)]
client: Snapshot cephfs APIs
Introduce calls to create and delete a snapshot. The difference
b/w this and the traditional mkdir() call to create a snapshot
is the additional metadata field that can be persistent along
side a snapshot (on creation). However, rmsnap() does nothing
special and is included for completeness.
Additionally, get_snap_info() API allows to fetch snapshot info
which includes things like snaphot ID and metadata (if any, which
was set via mksnap() API).
Venky Shankar [Mon, 19 Oct 2020 09:38:47 +0000 (05:38 -0400)]
mds: include snapshot metadata in trace reply to client
MDS sends (custom user) metadata associated with a snapshot in
trace reply to the client. This is required to implement API for
fetching snapshot info (snap-id, metadata, etc...).
Venky Shankar [Mon, 19 Oct 2020 09:02:21 +0000 (05:02 -0400)]
mds: include metadata in SnapInfo structure
This is a free-flowing key/value metadata attached to a snapshot
and persisted in `SnapInfo` structure. The idea is to allow
custom metadata to be persisted when creating a snapshot.
Initial users for this feature would be cephfs-mirror daemon. The
idea is to persist the primary filesystems snapshot ID on snapshot
creation in secondary filesystem. This allows identification of
snapshot renames and recreates.
Jason Dillaman [Wed, 16 Dec 2020 15:15:28 +0000 (10:15 -0500)]
librbd/api: avoid retrieving more than max mirror image info records
This could otherwise result in an assertion failure in the API if
it failed to retrieve the status on an image and therefore required
a second iteration through the loop.
Fixes: https://tracker.ceph.com/issues/48522 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Ilya Dryomov [Thu, 3 Dec 2020 10:24:32 +0000 (11:24 +0100)]
qa: krbd_stable_pages_required.sh: move to stable_writes attribute
bdi/stable_pages_required attribute was deprecated in 5.10 and now
always returns 0. The replacement is queue/stable_writes. (It is
also writeable, so we can simplify these test cases somewhat in the
future.)
Kefu Chai [Thu, 17 Dec 2020 03:29:02 +0000 (11:29 +0800)]
crimson/osd: do not use the parallel_for_each() in prep_push()
the caller of prep_push() also loops in target pg shards, so there is no
need to do the loop in prep_push(). in this change, prep_push() just
takes care of a single pg_shard for a single oid instead of collecting
all push ops for all pg_shards to be recovered.
Kefu Chai [Thu, 17 Dec 2020 03:15:04 +0000 (11:15 +0800)]
crimson/osd: create the collected push ops use shared_ptr
do_with() uses a unique_ptr for keeping the captured variables alive. so
it is practically the same thing in concept. but `do_with()` is more
distracting and difficult to reason about, it increases the indent level
also.
so use a lw_shared_ptr() for collecting the push ops map.
Kefu Chai [Thu, 17 Dec 2020 02:05:04 +0000 (10:05 +0800)]
crimson/osd: trade do_with() for plain code
no need to capture the intervals or the data in
`ReplicatedRecoveryBackend::_handle_push()`, they are passed to
`submit_push_data()` by value right after being initialized. so
move the content of `do_with()` block out into the caller.
also, restructure the code preparing the parameters for
`submit_push_data()` to match with the structure of with the code in
`ReplicatedRecoveryBackend::_handle_pull_response()` for better
readability.
Kefu Chai [Thu, 17 Dec 2020 10:52:00 +0000 (18:52 +0800)]
install-deps.sh: do not install libboost-all-dev for old distros
ubuntu disco and ubuntu focal do not ship libboost 1.72 and up, and
we depend on libboost 1.72 or up, so it does not help to install
liboost 1.67 or libboost 1.71 anymore.
Alfonso Martínez [Tue, 15 Dec 2020 08:28:46 +0000 (09:28 +0100)]
mgr/dashboard: CLI commands: read passwords from file
Fixes: https://tracker.ceph.com/issues/48355 Signed-off-by: Alfonso Martínez <almartin@redhat.com> Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
projects / ceph.git / log