J. Eric Ivancich [Fri, 27 Sep 2019 16:05:17 +0000 (12:05 -0400)]
rgw: let newer RGWs work with older OSDs re: cls-side filtering
Previous commits moved bucket list filtering when a delimiter was
specified to the osd/cls layer. However, since rgw's are often
upgraded before osd's are, until we reach verison ceph version 16, an
rgw cannot assume that the osd/cls did the filtering. This is
addressed in the following ways....
First rgw_cls_list_ret now indicates whether filtering was done on the
osd/cls side.
And second, the old filtering code in the rgw is maintained in
RGWRados::Bucket::List::list_objects_ordered, so it can still be
triggered when all osd's are not doing the filtering.
Once we reach ceph version 16, and there is no chance that the rgw is
working with a osd running "young" version 14 code, we can remove the
backward compatibility code in
RGWRados::Bucket::List::list_objects_ordered.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
rgw: move some bucket index delimiter filtering logic to cls
Currently the cls code that returns bucket index entries does no
filtering when a delimiter is specified. This means that the cls code
will return many entries that the rgw code does not need. This means
unnecessary bucket index entries will be encoded, decoded, and consume
network bandwidth.
A change is made to send a delimiter, if it's specified, to the cls
code, so the filtering can be done there.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
rgw: rename search variable to better indicate semantics
cls_cxx_map_get_vals takes a parameter of where in the sequence of
keys to start returning values. In code it's often referred to as the
"start_key", but in fact it only returns values who have keys *after*
it. To clarify the semantics, rename the variable to "start_after_key"
in the bucket listing functions in both rgw and cls.
Also, declare a type RGWRados::check_filter_t, so the type does not
have to be fully literalized multiple times.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
rgw: move after-delimiter logic so available in both rgw & cls/osd
Since we need to be able to calculate a key a path subdirectory both
in rgw and cls, move the code that calculates that to cls_rgw_types.h,
which is shared by both rgw and cls/osd.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Sage Weil [Fri, 24 Jan 2020 15:30:11 +0000 (09:30 -0600)]
Merge PR #32686 into master
* refs/pull/32686/head:
mgr/cephadm: speed up when not refresh is needed
mgr/cephadm: track age of service metadata
mgr/orchestrator: include age of 'service ls' metadata
mgr/mgr_util: add to_pretty_timedelta
Patrick Donnelly [Thu, 23 Jan 2020 23:37:56 +0000 (15:37 -0800)]
qa: update xfstests version
Building xfstests on CentOS 8 has build errors:
2020-01-23T21:15:14.939 INFO:tasks.workunit.client.0.smithi155.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests/src/fssum.c:616: undefined reference to `major'
2020-01-23T21:15:14.939 INFO:tasks.workunit.client.0.smithi155.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests/src/fssum.c:617: undefined reference to `minor'
2020-01-23T21:15:14.940 INFO:tasks.workunit.client.0.smithi155.stderr:collect2: error: ld returned 1 exit status
Michael Fritch [Thu, 23 Jan 2020 15:12:10 +0000 (08:12 -0700)]
cephadm: require fsid if name contains a daemon id
$ cephadm shell --name mon.123
INFO:cephadm:Found fsids []
Traceback (most recent call last):
File "./cephadm", line 2805, in <module>
r = args.func()
File "./cephadm", line 646, in _infer_fsid
return func()
File "./cephadm", line 1992, in command_shell
no_config=True if args.config else False)
File "./cephadm", line 1009, in get_container_mounts
data_dir = get_data_dir(fsid, daemon_type, daemon_id)
File "./cephadm", line 670, in get_data_dir
return os.path.join(args.data_dir, fsid, '%s.%s' % (t, n))
File "/usr/lib64/python3.7/posixpath.py", line 94, in join
genericpath._check_arg_types('join', a, *p)
File "/usr/lib64/python3.7/genericpath.py", line 149, in _check_arg_types
(funcname, s.__class__.__name__)) from None
TypeError: join() argument must be str or bytes, not 'NoneType'
Sage Weil [Thu, 23 Jan 2020 14:12:49 +0000 (08:12 -0600)]
Merge PR #32794 into master
* refs/pull/32794/head:
cephadm: normalize image_id in 'pull' output too
cephadm: fix image_id normalization
cephadm: pull: remove newline from ceph_version
mgr/cephadm: base upgrade check on return code, not stderr
mgr/cephadm: fix ok-to-stop
mgr/cephadm: some upgrade debug output
mgr/cephadm: remove dup check
mgr/cephadm: only need_upgrade_self if image_id doesn't match
mgr/cephadm: safe to restart mgr and other non-mon/osd/mds daemons
Ulrich Weigand [Thu, 23 Jan 2020 12:07:08 +0000 (13:07 +0100)]
cmake: Improve test for 16-byte atomic support on IBM Z
Commit d1b9d14324586d02dc6c8dd0a9bdf4b98ae16416 added a check for
16-byte atomics without library support on IBM Z. Unfortunately
it turns out this test only works correctly when the test case
is built without optimization. (This is normally true, but it
may not be the case when passing explicit CXXFLAGS to cmake.)
The underlying reason is that GCC may choose to use either an
inline implementation of the 16-byte atomics or library calls,
depending on whether or not it is able to prove the atomic
variable is properly aligned. At -O0 it is never able to prove
that, but at higher optimization levels it depends on the complexity
of the expression (in particular, whether GCC can track down the
definition of the underlying object).
As the test case uses a very simple expression, it may happen that
this test can be built without requiring library support, but some
of the "real" uses of atomics in Ceph code cannot. This defeats
the whole purpose of the test at configure time.
Fixed by making the access pattern in the test more complex, so
that the test fails even at high optimization levels.
Alfonso Martínez [Thu, 23 Jan 2020 10:16:27 +0000 (11:16 +0100)]
ceph.spec.in: fix 'make check' deps for centos8
When running 'FOR_MAKE_CHECK=1 ./install-deps.sh' in CentOS 8
these dependencies were not being installed.
Missing dependencies are provided by
https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-el8/
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Stephan Müller [Fri, 14 Jun 2019 13:45:53 +0000 (15:45 +0200)]
mgr: Add get_rates_from_data to mgr_util.py
The function calculates the difference between the values
in a time series list.
The previous location of it was in the dashboard, that's why the
function has doc tests. In order to be able to test them as part of the
manager utilities a bit of tox magic was added.
Fixes: https://tracker.ceph.com/issues/40365 Signed-off-by: Stephan Müller <smueller@suse.com>
* always use the helpers from `futurize` instead of using
`::seastar::make_exception_future()` directly. as
`make_exception_future(future_state_base&& state)` is located in
`seastar::internal` namespace
* add `noexcept` specifier in both constructors. to match with the
helpers of `make_exception_future2()` which dispatch the call to
different constructor variants.
Samuel Just [Sat, 21 Dec 2019 02:04:47 +0000 (02:04 +0000)]
fastbmap_allocator_impl: avoid std::vector[] for oob index
This was relatively harmless as the pointer won't actually be
dereferenced, but it runs afoul of std::vector's bounds checks with
_GLIBCXX_ASSERTIONS enabled.
Neha [Wed, 22 Jan 2020 15:21:21 +0000 (15:21 +0000)]
mgr/pg_autoscaler: default to pg_num[_min] = 32
78bf92448002eece7501da01b67f900a84207e70 increased the default to 16.
Increasing it further to 32 will provide enough parallelism to improve
out of the box performance for new users.
projects / ceph-ci.git / log