Venky Shankar [Fri, 21 Aug 2020 14:07:37 +0000 (10:07 -0400)]
mgr/volumes: maintain per subvolume trash directory
PR https://github.com/ceph/ceph/pull/36472 introduces changes
that disallow nested nested snapshots in a subtree (subvolume)
and renames across subvolumes. This effect asynchronous purge
in mgr/volumes as subvolume are moved to a trash directory for
asynchronous deletion by purge threads.
To workaround this, start maintaining a subvolume specific
trash directory. Use the trash directory as an index to the
subvolume specific trash directory entry.
This changes subvolume deletion logic which currently relies
on `--retain-snapshots` flag to decide if the subvolume user
directory should get purged or the subvolume base directory
itself. Deleting a subvolume moves the user facing directory
to its specific trash directory. Purge threads take care of
deleting user facing directories (in trash) and the subvolume
base directory if required (when certain conditions are met).
Kefu Chai [Fri, 21 Aug 2020 12:22:23 +0000 (20:22 +0800)]
qa/tasks/ceph: create a log file before redirecting to it
as it is shell who interprets ">>" and redirect the stderr to given
file, but the shell process is launched by ubuntu:ububunt without using
sudo, so the command fails with "Permission denied" failure. to address
this issue, in this change, a file with proper priviledges is created
beforehand using `install`, so shell is able to write to it.
also, instead of creating this file in `maybe_redirect_stderr()`, it
returns the command to create the log file.
Patrick Donnelly [Sat, 22 Aug 2020 01:44:06 +0000 (18:44 -0700)]
Merge PR #36681 into master
* refs/pull/36681/head:
mds: don't track change of config 'mds_replay_unsafe_with_closed_session'
mds: fix 'forward loop' when forward_all_requests_to_auth is set
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Sat, 22 Aug 2020 01:40:42 +0000 (18:40 -0700)]
Merge PR #36131 into master
* refs/pull/36131/head:
doc: document cephfs mirroring dev work
test: add tests for `ceph fs mirror` family of commands
mds: track filesystem mirror peers in fsmap
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 21 Aug 2020 23:12:32 +0000 (16:12 -0700)]
Merge PR #36472 into master
* refs/pull/36472/head:
qa/workunits/fs: add test for subvolume
mds: don't move inode with nlink > 1 to global snaprealm if it's in subvolume
mds: disallow hardlink across subvolume
mds: disallow across subvolume rename
mds: disallow creating snapshot on descendent directory of subvolume
mds: add vxattr that marks/clears subvolume flag
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Kefu Chai [Wed, 19 Aug 2020 17:03:41 +0000 (01:03 +0800)]
test/pybind: label test_rados tests with attributes
so we can pick the tests selectively. currently, some features are not
supported by crimson. but we will cover more and more of them. so
attaching different labels to tests cases helps us to run these tests
selectively.
Greg Farnum [Wed, 12 Aug 2020 23:44:11 +0000 (23:44 +0000)]
mon: mark pgtemp messages as no_reply more consistently in preprocess_pgtemp
If a message is forwarded, it's conceivable the leader's and peon's evaluation
will disagree about whether the message is useful or not, which could result
in the leader ignoring it and the peon having a dangling forwarded message.
Fix this by marking the op as no_reply whenever ignoring it.
Jason Dillaman [Thu, 30 Jul 2020 17:12:20 +0000 (13:12 -0400)]
rbd: let librados finish initializing the global context
librados will retrieve a bootstrap config update from the MONs
after connecting and then finish initializing the context. In
general, non-primitive config options cannot be updated after
initialization is complete. This prevents the rbd CLI from
supporting non-primitive config overrides from the MON.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 30 Jul 2020 17:00:22 +0000 (13:00 -0400)]
mon/MonClient: apply MON config before finishing bootstrap
In general, non-primitive config options cannot be updated after
the service threads have been marked as running. In librados, this
happens after the MonClient bootstrap process. However, the configs
received from the MON were applied in a separate thread which can
race with librados marking the service threads as running.
Now the received MON configs will be applied within the context of
the bootstrap function call instead of deferring to a separate
thread.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
cephadm: Allow users to use a custom dashboard ssl port
This modification allows the user to create a new bootstrap cluster using a predefined SSl port for the dashboard.
If firewall is enabled, any new manager daemon deployed in new hosts will take care of open the required ports for all the services enabled in the manager.
Two new parameters for cephadm tool (aka binary or standalone):
Command bootstrap:
--ssl-dashboard-port SSL_DASHBOARD_PORT
Port number used to connect with dashboard using SSL
Command deploy:
--tcp-ports TCP_PORTS
List of tcp ports to open in the host firewall
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Alfonso Martínez [Wed, 19 Aug 2020 14:39:53 +0000 (16:39 +0200)]
mgr/dashboard: use SASS colors in TypeScript
NOTE: Enabled TS compiler option esModuleInterop for the unit tests to pass, which is also highly recommended:
https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-7.html#support-for-import-d-from-cjs-from-commonjs-modules-with---esmoduleinterop
This implied updating lodash & momentjs imports (unneeded ones have been removed).
Fixes: https://tracker.ceph.com/issues/47036 Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Kefu Chai [Tue, 18 Aug 2020 16:57:58 +0000 (00:57 +0800)]
crimson/osd: errorator omap_get_values() ops
* os/alienstore: do not panic if retval is less than 0. instead,
translate ENOENT to enoent and other erros to input_output_error
* os/FuturizedStore: errorator omap_get_values() variants, so
they return errorator::future<> instead of plain seastar::future<>
* os/seastore: update accordingly.
* osd/pg_backend: use enodata for OI without omap in it. better
performance, and better readability this way.
Kefu Chai [Wed, 19 Aug 2020 04:54:41 +0000 (12:54 +0800)]
ceph: correct additional paths added to sys.path
pybind_path and pythonlib_path are already absolute paths, they are
prepared before being passed to `respawn_in_path()`. so let's drop
path components prepended to them.
Kefu Chai [Mon, 17 Aug 2020 07:44:44 +0000 (15:44 +0800)]
pybind/mgr/devicehealth: drop the cloud model of disk prediction
as the service offered by https://www.diskprophet.com/ is not accessible
anymore, the cloud model of disk prediction is not available. so let's
drop the code handling "cloud" model.
Kefu Chai [Mon, 17 Aug 2020 07:37:34 +0000 (15:37 +0800)]
doc/mgr/diskprediction.rst: remove references to diskprediction_cloud
since https://www.diskprophet.com is not accessible anymore, and this
feature is not available at the time of writing. let's drop the related
document.
the service offered by https://www.diskprophet.com is not accessible
anymore. before we have a replacement of it, let's drop
mgr-diskprediction-cloud. and remove its packaging.
Patrick Donnelly [Tue, 18 Aug 2020 16:27:49 +0000 (09:27 -0700)]
Merge PR #35824 into master
* refs/pull/35824/head:
vstart_runner.py: rotate logs between testcases
vstart.sh: write logrotate conf and rotate logs
vstart_runner: run one test method at a time...
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com> Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Ulrich Weigand [Tue, 18 Aug 2020 07:51:22 +0000 (09:51 +0200)]
messages,mds: Fix decoding of enum types on big-endian systems
When a struct member that has enum type needs to be encoded or
decoded, we need to use an explicit integer type, since there
are no encode routines for the enum type. (This is probably
to avoid introducing dependencies on implementation-defined
choices by the compiler to use a particular underlying type.)
This leads to code sequences along the lines of:
encode((int32_t)state, bl);
and
decode((int32_t&)(state), bl);
The encode line is actually fine, but the decode line is
incorrect on big-endian systems if the underlying type of
the enum differs from the explicitly chosen integer type.
This is because this performs in effect a pointer cast,
and will write the decoded int32_t value into the memory
backing the "state" member variable. If the sizes differ,
the value is written into the wrong bytes of "state" on
big-endian systems.
This patch fixes the problem by decoding into an intermediate
variable of the integer type first, and then casting the result
while assigning to the struct member of enum type.
This bug showed up initially as invalid health-status values
causing Ceph daemon aborts on s390x. I've tried to find and
fix all other instances of the same enum decode pattern as well.
projects / ceph.git / log