Since the v1.4.0 release there have been a few improvements to Zstandard
including improved compression ratios, faster compression, and faster
decompression.
Sebastian Wagner [Fri, 11 Sep 2020 19:10:37 +0000 (21:10 +0200)]
Merge pull request #35543 from sebastian-philipp/qa-cephadm-iscsi
qa/cephadm: Add iSCSI
Reviewed-by: Adam King <adking@redhat.com> Reviewed-by: Georgios Kyratsas <gkyratsas@suse.com> Reviewed-by: Matthew Oliver <moliver@suse.com> Reviewed-by: Michael Fritch <mfritch@suse.com>
Rishabh Dave [Mon, 24 Aug 2020 18:34:40 +0000 (00:04 +0530)]
qa/cephfs: backup mount object before running tests
Right now, only client IDs are stashed and restored but with the recent
changes (addition of more attributes to mount objects, specifically),
this is not enough. Saving and restoring these details before and after
tests respectively ensures that mount commands rus smoothly. Not doing
this typically leads to mount command failure for the second test in the
testsuite under execution since the client IDs are saved and restored in
CephFSTestCase.setUp and CephFSTestCase.tearDown respectively but the
rest of the details are not.
in future, string_view can be used almost every where string can be
used. in this change, `const char*` is instead passed to the constructor
of system_error, as we can ensure that the string_view instances are
always constructed from a `const char*` ended with `\0`.
we need this change for two reasons:
* better performance
* prefer for the world where string_view rules.
Rishabh Dave [Wed, 24 Jun 2020 13:30:56 +0000 (19:00 +0530)]
qa/cephfs: modify delete_all_filesystems() in filesystem.py
Modify filesystem.Filesystem.delete_all_filesystems() method to make it
more succinct, move it to class MDSCluster instead and update every call
to it accordingly.
rgw: RGWRESTOp no longer tracks separate error code
RGWOp_Bucket_Remove::execute() was storing failures from
bucket->remove_bucket() in op_ret, but left http_ret=0 so we responded
to the client with '200 OK'
to avoid bugs like this, remove the extra http_ret variable and only use
the op_ret from RGWOp
This commit exposes HACKING.rst through the
docs website. Formerly, HACKING.rst was just
a file that existed at /src/pybind/mgr/dashboard/
HACKING.rst. This commit also updates README.rst
to point to the new location of HACKING.rst.
crimson uses seastar logging facility for logging. and the latter uses
fmt::format(). currently, we collect the log message into a string and
pass it to log(fmt,...) as "fmt", but seastar/core/sstring.hh defines
the operator<<(ostream&, const vector<T>&) which is a better match than
our the operator<<(ostream&, const vector<T, Allocator>&). and seastar's
operator<<(ostream&, const vector<T>&) uses "{" and "}" to mark the
begin and end of a vector when printing it. and "{}" is also used by
libfmt to enclose its replacement fields. see
https://fmt.dev/latest/syntax.html. so when a vector is printed using
logging facility in crimson, libfmt chokes when trying to parse it as a
format string. so we have some options, like:
- disable seastar's operator<< implementation
- escape the "{}" when writing the vector to the output stream
- print the message as the args, and use "{}" as the fmt.
the 3rd one is the most straightforward, and probably more performant.
Rishabh Dave [Fri, 31 Jul 2020 08:57:16 +0000 (14:27 +0530)]
qa/cephfs: modify recreate() in filesystem.py
Modify cephfs.filesystem.Filesystem.recreate() method to delete only the
FS represented by the object instead of deleting the every FS on the
Ceph cluster.
Rishabh Dave [Wed, 24 Jun 2020 13:30:04 +0000 (19:00 +0530)]
qa/cephfs: add a method to destroy fs in filesystem.py
And reset_obj_attrs parameter to it so that the caller of the method can
choose to destroy the Ceph FS represented by the object without
disturbing the object attributes.
Rishabh Dave [Tue, 24 Mar 2020 11:34:14 +0000 (17:04 +0530)]
qa/cephfs: allow not aborting execution when mount command fails
This commit adds a new argument check_status to mount methods of
KernelMount, FuseMount, LocalKernelMount and LocalFuseMount. When value
of this argument is False, these methods would catch the
CommandFailedError exception and would return a tuple consisting of the
exception itself, and stdout and stderr of the mount command. This
allows reusing these mount methods while running negative tests for
commands.
The name "check_status" is selected so since teuthology's run() and
vstart_runner's run() use a variable with same name for the very same
purpose.
Rishabh Dave [Mon, 27 Jan 2020 06:07:44 +0000 (11:37 +0530)]
qa/cephfs: allow reusing mount objects and add remount method
This commit introduces following two set of changes -
First, make client keyring path, mountpoint on host FS and CephFS and
CephFS's name attributes of the object representing the mount
and update all the mount object creation calls accordingly. Also,
rewrite all the mount object creation to use keyword arguments instead
of positional arguments to avoid mistakes, especially since a new
argument was added in this commit.
Second, add remount method to mount.py so that it's possible to unmount
safely, modify the attributes of the object representing the mount and
mount again based on new state of the object *in a single call*. The
method is placed in mount.py to avoid duplication.
This change has two leads to two more changes: upgrading interface of
mount() and mount_wait() and upgrading testsuites to adapt to these
change.
This commit also makes sure that the old syntax 'allow rw path=<path>'
is supported for backwards compatibility. The old syntax would imply
'allow rw fsname=* path=<path>' and would grant read-write permission for
all FSs containing the path <path>.
Douglas Fuller [Thu, 7 Mar 2019 15:17:23 +0000 (10:17 -0500)]
cephfs: add auth caps based on fs names
Add new auth caps to restrict access to clients based on fsnames. To
specify this, for example:
mds 'allow rw fsname=cephfs1'
This will restrict client access to fs name "cephfs1" only. Messages to
active MDS assigned to any other FSMap will be dropped. Standby MDS not
associated with an FSMap will accept messages from clients. To allow
multiple file systems, create MDS cap as follows -
Douglas Fuller [Thu, 7 Mar 2019 15:15:33 +0000 (10:15 -0500)]
cephfs: store fs names associated with local MDSMap in MDSRank
Modify MMDSMap to include the name of the file system from the FSMap
when bringing up a new MDS. Store the name in the MDSRank structure for
use in security validation.
Fixes: http://tracker.ceph.com/issues/15070 Signed-off-by: Douglas Fuller <dfuller@redhat.com> Signed-off-by: Rishabh Dave <ridave@redhat.com>
Douglas Fuller [Fri, 1 Mar 2019 16:41:45 +0000 (11:41 -0500)]
mon/MDSMonitor: add mon auth caps for CephFS names
Add a 'fsname' clause to mon auth caps to restrict a client's view
of the FSMap. Example:
mon 'allow rw fsname=cephfs2'
This would restrict the client's view of the FSMap to the MDSMap for
cephfs2. Any MDS allocated to a different filesystem will be invisible.
Global standby daemons are always visible. To allow multiple CephFSs,
add multiple caps:
before this change, unittest_seastar_config set the specified option to
the shard id in parallel and expects that the values of the option are
consistent across all shards after setting the value on all shards. but
sharded::invoke_on_all() is executed in parallel, and so does
ConfigProxy::do_change(), so there are actually "n x n" continuations
racing each other. the order of the continuations storing the setting
cannot be determined. so we cannot expected that the last batch of
continuations hitting shareds store the same value.
in this change, only a single `conf.set_val()` call is performed with
a known value. and this value is checked with the values stored on all
shards.
crimson/osd: print notify_id instead of its address
this addresses the FTBFS like
/usr/include/fmt/core.h:902:19: error: static assertion failed: formatting of non-void pointers is disallowed
902 | static_assert(!sizeof(T), "formatting of non-void pointers is disallowed");
| ^~~~~~~~~~
also, i think it is more helpful in this context to have the
notify id instead of the address, as we can use the id to reference
the notification printed by other logging messages.
crimson/os: cast pointers to void* before printing them
to address the FTBFS like:
../src/crimson/os/seastore/lba_manager/btree/btree_range_pin.cc:110:14: required from here
/usr/include/fmt/core.h:902:19: error: static assertion failed: formatting of non-void pointers is disallowed
902 | static_assert(!sizeof(T), "formatting of non-void pointers is disallowed");
| ^~~~~~~~~~
crimson/os: instantiate future_type<> with future_stored_type
to accommodate the change on seastar's side, which started to use
monostate for representing `void` return type, and hide this convention
using future_stored_type_t.
crimson/net: print entity_addr_t not the in4_addr in it
for two reasons:
* we don't have operator<<(ostream&, in4_addr&) defined.
* we don't have any fmt::format() facility specialized for in4_addr
* to print entity_addr_t instead of in4_addr is more correct in
the sense that we should support IPv6
crimson/osd: use temporary RecoveryInfo for recovering object delete ops
Right now, recover_delete use WaitForObjectRecovery::pi, which may not be available
if there were no pulling, to pass recovery_info, so use temporary recovery_info instead
fmt/core.h:902:19: error: static assertion failed: formatting of non-void pointers is disallowed
902 | static_assert(!sizeof(T), "formatting of non-void pointers is disallowed");
| ^~~~~~~~~~
and i searched around, couldn't find any other
statement printing the address of message, so
i think it's not of much use to print their
addresses in a single place, unless we can cross
check them in different places.
projects / ceph.git / log