]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agoscript/run-make.sh: retry if dpkg was interrupted 42743/head
Kefu Chai [Wed, 11 Aug 2021 08:29:10 +0000 (16:29 +0800)]
script/run-make.sh: retry if dpkg was interrupted

there is chance that apt-get is interrupted in the middle when a new PR
cancels the running jenkins job, the next job running apt-get or dpkg
would run into issues like:

E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
Build step 'Execute shell' marked build as failure

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42715 from tchaikov/wip-crimson-tls-int-cond
Kefu Chai [Mon, 9 Aug 2021 13:31:15 +0000 (21:31 +0800)]
Merge pull request #42715 from tchaikov/wip-crimson-tls-int-cond

crimson/common: instantiate interrupt_cond in .cc

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #42662 from trociny/wip-52063
Ilya Dryomov [Mon, 9 Aug 2021 12:58:25 +0000 (14:58 +0200)]
Merge pull request #42662 from trociny/wip-52063

rbd-mirror: fix potential async op tracker leak in start_image_replayers

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoMerge pull request #42708 from cbodley/wip-rgw-meta-sync-behind-after-trim
Casey Bodley [Mon, 9 Aug 2021 12:55:57 +0000 (08:55 -0400)]
Merge pull request #42708 from cbodley/wip-rgw-meta-sync-behind-after-trim

radosgw-admin: 'sync status' is not behind if there are no mdlog entries

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
3 years agocrimson: build without "using namespace std" 42715/head
Kefu Chai [Mon, 9 Aug 2021 11:12:05 +0000 (19:12 +0800)]
crimson: build without "using namespace std"

* add "std::" prefix in headers
* add "using" declarations in .cc files.

so we don't reply on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42611 from s0nea/wip-doc-transifex-resource
Ernesto Puerta [Mon, 9 Aug 2021 08:59:32 +0000 (10:59 +0200)]
Merge pull request #42611 from s0nea/wip-doc-transifex-resource

doc/dev: Dashboard translations release update

Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
3 years agocrimson/common: instantiate interrupt_cond in .cc
Kefu Chai [Sun, 8 Aug 2021 17:21:38 +0000 (01:21 +0800)]
crimson/common: instantiate interrupt_cond in .cc

so we can explicitly instantiate it.

this should address the segfault when accessing interrupt_cond when
it is defined as a plain thread local storage template variable in the
header file.

it seems Clang is not able to identify the access to TLS variable and
the value of %fs segment register of the main thread is always zero if
interrupt_cond is defined as a plain global variable stored in
thread local storage.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agotest/crimson: drop copy assignment operator
Kefu Chai [Sun, 8 Aug 2021 13:41:34 +0000 (21:41 +0800)]
test/crimson: drop copy assignment operator

because of the const reference member variable of `parent`:

const delta_overlay_t &parent;

we are not allowed to mutate it. so the explicitly defaulted copy assignment
operator is implicitly deleted

this change silences the warnings from Clang like:

 ../src/test/crimson/seastore/test_transaction_manager.cc:159:12: warning: explicitly defaulted copy assignment operator is implicitly deleted [-Wdefaulted-function-deleted]
         iterator &operator=(const iterator &) = default;
                   ^
 ../src/test/crimson/seastore/test_transaction_manager.cc:98:25: note: copy assignment operator of 'iterator' is implicitly deleted because field 'parent' is of reference type 'const transaction_manager_$
         const delta_overlay_t &parent;
                                ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42711 from tchaikov/wip-seastar
Kefu Chai [Sun, 8 Aug 2021 13:24:33 +0000 (21:24 +0800)]
Merge pull request #42711 from tchaikov/wip-seastar

seastar: pick up change adding ioctl and fcntl support

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoseastar: pick up change adding ioctl and fcntl support 42711/head
Kefu Chai [Sat, 7 Aug 2021 03:56:35 +0000 (11:56 +0800)]
seastar: pick up change adding ioctl and fcntl support

prepare for the better support of ZNS.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42668 from tchaikov/wip-doc-ci
Kefu Chai [Sat, 7 Aug 2021 02:59:14 +0000 (10:59 +0800)]
Merge pull request #42668 from tchaikov/wip-doc-ci

doc/dev/continuous-integration: document chacra

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agodoc/dev/continuous-integration: document chacra 42668/head
Kefu Chai [Thu, 5 Aug 2021 08:21:31 +0000 (16:21 +0800)]
doc/dev/continuous-integration: document chacra

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42149 from hualongfeng/fix_sync_point
Ilya Dryomov [Fri, 6 Aug 2021 21:09:44 +0000 (23:09 +0200)]
Merge pull request #42149 from hualongfeng/fix_sync_point

librbd/cache/pwl: avoid stack overflow caused by nested shared_ptr destruction

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
3 years agoradosgw-admin: 'sync status' is not behind if there are no mdlog entries 42708/head
Casey Bodley [Fri, 6 Aug 2021 19:14:26 +0000 (15:14 -0400)]
radosgw-admin: 'sync status' is not behind if there are no mdlog entries

if remote mdlogs are trimmed prematurely, sync status will report
that it's behind the remote's max-marker even if there are no mdlog
entries to sync

for each behind shard, we fetch the next mdlog entry from the remote. if
we get an empty listing, remove that shard from behind_shards. this
logic now has to run before we print "behind shards:" so that empty
shards aren't listed

Fixes: https://tracker.ceph.com/issues/52091
Signed-off-by: Casey Bodley <cbodley@redhat.com>
3 years agolibrbd/cache/pwl: avoid stack overflow caused by nested shared_ptr destruction 42149/head
Hualong Feng [Fri, 2 Jul 2021 06:22:06 +0000 (14:22 +0800)]
librbd/cache/pwl: avoid stack overflow caused by nested shared_ptr destruction

Destruction of nested shared_ptr will cause stack overflow.
With the explicit assignment of nullptr, the deleted node
is completely disconnected from the current linked list

-------              *******               -------
|sync | <--earlier-- |sync | <--earlier-x- |sync |
|point| --later----> |point| --later----x> |point|
-------              *******               -------
   |                    |                     |
   V                    V                     V
-------              -------               -------
|log_ | ---next----> |log_ | ---next----x> |log_ |
|entry|              |entry|               |entry|
-------              -------               -------

earlier: earlier_sync_point
later:   later_sync_point
next:    next_sync_point_entry

Fixes: https://tracker.ceph.com/issues/51418
Signed-off-by: Feng Hualong <hualong.feng@intel.com>
3 years agoMerge pull request #42705 from tchaikov/wip-clang-fixes
Kefu Chai [Fri, 6 Aug 2021 18:05:30 +0000 (02:05 +0800)]
Merge pull request #42705 from tchaikov/wip-clang-fixes

crimson, test/crimson, msg: clang fixes

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #42660 from zdover23/wip-doc-2021-08-04rados-config-storage-device...
Josh Durgin [Fri, 6 Aug 2021 16:59:48 +0000 (09:59 -0700)]
Merge pull request #42660 from zdover23/wip-doc-2021-08-04rados-config-storage-devices-filestore

doc/rados: rewrite "FileStore" section

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agocrimson/os/seastore: do not capture unused variable 42705/head
Kefu Chai [Fri, 6 Aug 2021 16:42:31 +0000 (00:42 +0800)]
crimson/os/seastore: do not capture unused variable

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agotest/crimson/seastore: drop duplicated i++
Kefu Chai [Fri, 6 Aug 2021 16:39:11 +0000 (00:39 +0800)]
test/crimson/seastore: drop duplicated i++

for two reasons:

* to silence Clang warning
* it's very unusual to do this. and this is a bad smell.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agotest/crimson/seastore: capture labels by defining new variables
Kefu Chai [Fri, 6 Aug 2021 16:37:28 +0000 (00:37 +0800)]
test/crimson/seastore: capture labels by defining new variables

structured binding does not define variables, so we have to define
variables in capture list for capturing them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agomsg/async/compression_onwire: include used header
Kefu Chai [Fri, 6 Aug 2021 16:36:39 +0000 (00:36 +0800)]
msg/async/compression_onwire: include used header

this fixes the FTBFS with Clang + libc++.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42700 from tchaikov/wip-crimson-cleanup
Kefu Chai [Fri, 6 Aug 2021 15:44:35 +0000 (23:44 +0800)]
Merge pull request #42700 from tchaikov/wip-crimson-cleanup

crimson/os/seastore: do not capture unused variable

Reviewed-by: Samuel Just <sjust@redhat.com>
3 years agoMerge pull request #36517 from giladmaya/on_wire_compression
Neha Ojha [Fri, 6 Aug 2021 15:07:58 +0000 (08:07 -0700)]
Merge pull request #36517 from giladmaya/on_wire_compression

msgr: added support in on-wire compression for osd-osd communication

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agoMerge pull request #42686 from dsavineau/cephadm_almalinux_distro
Sebastian Wagner [Fri, 6 Aug 2021 14:51:32 +0000 (16:51 +0200)]
Merge pull request #42686 from dsavineau/cephadm_almalinux_distro

cephadm: add AlmaLinux to supported distro

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42608 from joscollin/wip-B51975-keyerror-null-foo
Venky Shankar [Fri, 6 Aug 2021 12:16:04 +0000 (17:46 +0530)]
Merge pull request #42608 from joscollin/wip-B51975-keyerror-null-foo

pybind/mgr/stats: validate cmdtag

Reviewed-by: Venky Shankar <vshankar@redhat.com>
3 years agotest/crimson: mark final class "final" 42700/head
Kefu Chai [Fri, 6 Aug 2021 11:48:19 +0000 (19:48 +0800)]
test/crimson: mark final class "final"

silences warning from Clang like:

../src/test/crimson/seastore/test_object_data_handler.cc:33:16: warning: class with destructor marked 'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
  ~TestOnode() final = default;
               ^
../src/test/crimson/seastore/test_object_data_handler.cc:20:7: note: mark 'TestOnode' as 'final' to silence this warning
class TestOnode : public Onode {
      ^
1 warning generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agocrimson/os/seastore: do not capture unused variable
Kefu Chai [Fri, 6 Aug 2021 11:24:34 +0000 (19:24 +0800)]
crimson/os/seastore: do not capture unused variable

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42698 from tchaikov/wip-cmake-attr
Kefu Chai [Fri, 6 Aug 2021 11:21:25 +0000 (19:21 +0800)]
Merge pull request #42698 from tchaikov/wip-cmake-attr

cmake: fail on unknown attribute

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
3 years agoMerge pull request #42607 from tchaikov/wip-ceph-dencoder
Kefu Chai [Fri, 6 Aug 2021 10:39:36 +0000 (18:39 +0800)]
Merge pull request #42607 from tchaikov/wip-ceph-dencoder

tools/ceph-dencoder: refactor to address segfault when compiled w/ clang

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
3 years agocmake: fail on unknown attribute 42698/head
Kefu Chai [Fri, 6 Aug 2021 09:26:16 +0000 (17:26 +0800)]
cmake: fail on unknown attribute

on Clang, the option for detecting unknown attribute is
-Wunknown-attributes, so "-Wattributes -Werror" does not fail the test
when the C compiler is Clang.

in this change, we just turn all warnings into errors.
this should fail the test if the compiler does not understand
`__attribute__((__symver__ ...))`

Fixes: https://tracker.ceph.com/issues/40060
Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42602 from b-ranto/wip-lto
Kefu Chai [Fri, 6 Aug 2021 05:45:01 +0000 (13:45 +0800)]
Merge pull request #42602 from b-ranto/wip-lto

rados+rpm: Update symver defs and re-enable LTO

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42564 from myoungwon/wip-fix-45423
Kefu Chai [Fri, 6 Aug 2021 04:26:32 +0000 (12:26 +0800)]
Merge pull request #42564 from myoungwon/wip-fix-45423

test: fix wrong alarm (HitSetWrite)

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #42277 from tchaikov/wip-vstart-runner-cleanups
Kefu Chai [Fri, 6 Aug 2021 02:33:19 +0000 (10:33 +0800)]
Merge pull request #42277 from tchaikov/wip-vstart-runner-cleanups

qa/tasks/vstart_runner: do not send SIGTERM if no matched pid

Reviewed-by: Rishabh Dave <ridave@redhat.com>
3 years agoMerge pull request #42663 from cyx1231st/wip-fix-lba-empty-delta
Kefu Chai [Fri, 6 Aug 2021 02:28:46 +0000 (10:28 +0800)]
Merge pull request #42663 from cyx1231st/wip-fix-lba-empty-delta

crimson/os/seastore/lba_manager: remove assert non-empty delta

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
3 years agoMerge pull request #42669 from tchaikov/wip-doc-ext-confval
Kefu Chai [Fri, 6 Aug 2021 01:16:30 +0000 (09:16 +0800)]
Merge pull request #42669 from tchaikov/wip-doc-ext-confval

doc/_ext: print out the module which fails the build

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #42681 from tchaikov/wip-fmt-a
Kefu Chai [Fri, 6 Aug 2021 01:14:13 +0000 (09:14 +0800)]
Merge pull request #42681 from tchaikov/wip-fmt-a

cmake: link bundled fmt statically

Reviewed-by: Boris Ranto <branto@redhat.com>
3 years agoMerge pull request #42593 from ceph/wip-yuriw-release-15.2.14-master
David Galloway [Thu, 5 Aug 2021 23:12:48 +0000 (19:12 -0400)]
Merge pull request #42593 from ceph/wip-yuriw-release-15.2.14-master

doc: 15.2.14 Release Notes

3 years agodoc/releases/octopus.rst: add notable changes for mgr and core 42593/head
Neha Ojha [Thu, 5 Aug 2021 16:47:56 +0000 (16:47 +0000)]
doc/releases/octopus.rst: add notable changes for mgr and core

Signed-off-by: Neha Ojha <nojha@redhat.com>
3 years agodoc: add note about mon trimming of old MDSMaps
Patrick Donnelly [Tue, 3 Aug 2021 17:57:28 +0000 (10:57 -0700)]
doc: add note about mon trimming of old MDSMaps

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agodoc: 15.2.14 Release Notes
Yuri Weinstein [Tue, 3 Aug 2021 16:25:39 +0000 (09:25 -0700)]
doc: 15.2.14 Release Notes

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Signed-off-by: David Galloway <dgallowa@redhat.com>
3 years agoMerge pull request #42505 from wzbxqt327/put_obj_rentention
Casey Bodley [Thu, 5 Aug 2021 19:22:29 +0000 (15:22 -0400)]
Merge pull request #42505 from wzbxqt327/put_obj_rentention

rgw: add the condition of lock mode conversion to PutObjRentention

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #42531 from ivancich/wip-user-utilized-stats
Casey Bodley [Thu, 5 Aug 2021 19:22:05 +0000 (15:22 -0400)]
Merge pull request #42531 from ivancich/wip-user-utilized-stats

rgw: user stats showing 0 value for "size_utilized" and "size_kb_utilized" fields

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #42597 from ivancich/wip-rename-f
Casey Bodley [Thu, 5 Aug 2021 19:21:45 +0000 (15:21 -0400)]
Merge pull request #42597 from ivancich/wip-rename-f

rgw: clean up some code

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
3 years agoMerge pull request #42125 from ivancich/wip-cls-empty-listing
Casey Bodley [Thu, 5 Aug 2021 19:21:22 +0000 (15:21 -0400)]
Merge pull request #42125 from ivancich/wip-cls-empty-listing

rgw: resolve empty ordered bucket listing results w/ CLS filtering

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agoMerge pull request #42182 from tengfWang/rgw-swift-check-container-name
Casey Bodley [Thu, 5 Aug 2021 19:21:00 +0000 (15:21 -0400)]
Merge pull request #42182 from tengfWang/rgw-swift-check-container-name

rgw: remove the three-character limit when creating a container with swift interface

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
3 years agoMerge pull request #42397 from mkogan1/rgw-d3n-rem-ops-log
Casey Bodley [Thu, 5 Aug 2021 19:20:24 +0000 (15:20 -0400)]
Merge pull request #42397 from mkogan1/rgw-d3n-rem-ops-log

rgw: d3n: remove legacy requirement to enable rgw_enable_ops_log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
3 years agocephadm: add AlmaLinux to supported distro 42686/head
Dimitri Savineau [Thu, 5 Aug 2021 17:47:56 +0000 (13:47 -0400)]
cephadm: add AlmaLinux to supported distro

AlmaLinux [1] is an open-source 1:1 binary compatible fork of RHEL.

[1] https://almalinux.org/

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
3 years agoMerge pull request #42673 from guits/update_cephadm_doc
Kefu Chai [Thu, 5 Aug 2021 15:34:19 +0000 (23:34 +0800)]
Merge pull request #42673 from guits/update_cephadm_doc

doc/cephadm: update documentation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42664 from sebastian-philipp/cephadm-FakePipeWrapper
Sebastian Wagner [Thu, 5 Aug 2021 14:55:10 +0000 (16:55 +0200)]
Merge pull request #42664 from sebastian-philipp/cephadm-FakePipeWrapper

cephadm: fix unit tests

Reviewed-by: Sage Weil <sage@newdream.net>
3 years agoMerge pull request #42665 from sebastian-philipp/cephadm-doc-upgrade-offline
Sebastian Wagner [Thu, 5 Aug 2021 14:45:22 +0000 (16:45 +0200)]
Merge pull request #42665 from sebastian-philipp/cephadm-doc-upgrade-offline

doc/cephadm: Offline hosts will pause upgrades

Reviewed-by: Sage Weil <sage@newdream.net>
3 years agocmake: link bundled fmt statically 42681/head
Kefu Chai [Thu, 5 Aug 2021 13:52:50 +0000 (21:52 +0800)]
cmake: link bundled fmt statically

instead of building the bundled fmt library as shared library, built
static library. otherwise we'd have to link against libfmt.so.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agolibrados/librados_c: check for both symvers 42602/head
Boris Ranto [Thu, 5 Aug 2021 07:08:08 +0000 (09:08 +0200)]
librados/librados_c: check for both symvers

We should check if either asm symver or attribute symver is defined and
not assume that attribute symver implies asm symver.

Fixes: https://tracker.ceph.com/issues/40060
Signed-off-by: Boris Ranto <branto@redhat.com>
3 years agocmake: Check -flto flags
Boris Ranto [Thu, 5 Aug 2021 06:36:21 +0000 (08:36 +0200)]
cmake: Check -flto flags

We should check if -flto-partition=none is defined when the compiler
does not support symver attribute and fail the build if it is not.

Fixes: https://tracker.ceph.com/issues/40060
Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Signed-off-by: Boris Ranto <branto@redhat.com>
3 years agolibrados/librados_c: Use symver attribute if available
Boris Ranto [Tue, 3 Aug 2021 18:44:47 +0000 (20:44 +0200)]
librados/librados_c: Use symver attribute if available

Fixes: https://tracker.ceph.com/issues/40060
Signed-off-by: Boris Ranto <branto@redhat.com>
3 years agorpm: Re-enable LTO on supported systems
Boris Ranto [Tue, 3 Aug 2021 08:11:58 +0000 (10:11 +0200)]
rpm: Re-enable LTO on supported systems

We can now use LTO when building ceph. The symver issue was fixed by
using the gcc __symver__ attribute. The systems that support it can now
re-enable LTO.

Fixes: https://tracker.ceph.com/issues/40060
Signed-off-by: Boris Ranto <branto@redhat.com>
3 years agopybind/mgr/stats: check if cmdtag is a valid key in client_metadata 42608/head
Jos Collin [Tue, 3 Aug 2021 13:07:00 +0000 (18:37 +0530)]
pybind/mgr/stats: check if cmdtag is a valid key in client_metadata

Invalid cmdtag values ('', foo) are found in random testing and in the
qa runs. So check if cmdtag is a valid key in client_metadata.

Fixes: https://tracker.ceph.com/issues/51975
Signed-off-by: Jos Collin <jcollin@redhat.com>
3 years agodoc/cephadm: update documentation 42673/head
Guillaume Abrioux [Thu, 5 Aug 2021 10:02:19 +0000 (12:02 +0200)]
doc/cephadm: update documentation

This commit fixes a couple of typos in the cephadm documentation.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agodoc/_ext: print out the module which fails the build 42669/head
Kefu Chai [Thu, 5 Aug 2021 08:20:53 +0000 (16:20 +0800)]
doc/_ext: print out the module which fails the build

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agorados: Update symver defs
Boris Ranto [Mon, 2 Aug 2021 19:32:15 +0000 (21:32 +0200)]
rados: Update symver defs

The gcc compiler now supports symver attribute. We should update the
symvers to be able to support LTO.

Fixes: https://tracker.ceph.com/issues/40060
Signed-off-by: Boris Ranto <branto@redhat.com>
3 years agorados: Fix function ordering
Boris Ranto [Tue, 3 Aug 2021 07:26:16 +0000 (09:26 +0200)]
rados: Fix function ordering

The build fails because the function definitions are out of order. This
also reuses the LIBRADOS_C_API_DEFAULT_F macro for function definitions.

Fixes: https://tracker.ceph.com/issues/40060
Signed-off-by: Boris Ranto <branto@redhat.com>
3 years agodoc/cephadm: Offline hosts will pause upgrades 42665/head
Sebastian Wagner [Thu, 5 Aug 2021 08:27:01 +0000 (10:27 +0200)]
doc/cephadm: Offline hosts will pause upgrades

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: fix unit tests. 42664/head
Sebastian Wagner [Thu, 5 Aug 2021 07:53:09 +0000 (09:53 +0200)]
cephadm: fix unit tests.

Fixes AttributeError: 'FakePipeWrapper' object has no attribute 'readable'

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42242 from sebastian-philipp/cephadm-container-name-dashes
Sebastian Wagner [Thu, 5 Aug 2021 07:47:00 +0000 (09:47 +0200)]
Merge pull request #42242 from sebastian-philipp/cephadm-container-name-dashes

cephadm: use dashes for container names

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agocrimson/os/seastore/lba_manager: remove assert non-empty delta 42663/head
Yingxin Cheng [Tue, 3 Aug 2021 06:36:36 +0000 (14:36 +0800)]
crimson/os/seastore/lba_manager: remove assert non-empty delta

Delta can be empty when measure the effort of invalidated transactions.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
3 years agoMerge pull request #42462 from sebastian-philipp/mon-ip-change
Sebastian Wagner [Thu, 5 Aug 2021 07:44:58 +0000 (09:44 +0200)]
Merge pull request #42462 from sebastian-philipp/mon-ip-change

doc/cephadm: Move MONs to a different network

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agorbd-mirror: fix potential async op tracker leak in start_image_replayers 42662/head
Mykola Golub [Thu, 5 Aug 2021 07:03:20 +0000 (08:03 +0100)]
rbd-mirror: fix potential async op tracker leak in start_image_replayers

Fixes: https://tracker.ceph.com/issues/52063
Signed-off-by: Mykola Golub <mgolub@suse.com>
3 years agodoc/rados: rewrite "FileStore" section 42660/head
Zac Dover [Wed, 4 Aug 2021 21:57:24 +0000 (07:57 +1000)]
doc/rados: rewrite "FileStore" section

This PR makes minor improvements to the
syntax of the sentences in the "FileStore"
material in the Configuration chapter of
the RADOS manual.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
3 years agoMerge pull request #42619 from jdurgin/wip-release-notes-script
Josh Durgin [Wed, 4 Aug 2021 21:26:07 +0000 (14:26 -0700)]
Merge pull request #42619 from jdurgin/wip-release-notes-script

script/ceph-release-notes: improve output for markdown and limit verbosity

Reviewed-by: David Galloway <dgallowa@redhat.com>
3 years agoscript/ceph-release-notes: don't include PR body by default, just the title 42619/head
Josh Durgin [Wed, 4 Aug 2021 00:07:13 +0000 (20:07 -0400)]
script/ceph-release-notes: don't include PR body by default, just the title

This avoids overly long output for merges involving lots of PRs,
e.g. cephadm batch backports.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
3 years agoscript/ceph-release-notes: infer actual title from ptl-tool merges
Josh Durgin [Wed, 4 Aug 2021 00:05:50 +0000 (20:05 -0400)]
script/ceph-release-notes: infer actual title from ptl-tool merges

This helps the notes stay accurate when the commit title has changed,
but the PR title is out of date.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #42636 from zdover23/wip-doc-rados-config-storage-devices-bluestor...
Josh Durgin [Wed, 4 Aug 2021 15:34:21 +0000 (08:34 -0700)]
Merge pull request #42636 from zdover23/wip-doc-rados-config-storage-devices-bluestore-2021-08-03

doc/rados: rewrite BlueStore material

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #41897 from ivancich/wip-versioned-deletes
J. Eric Ivancich [Wed, 4 Aug 2021 14:15:28 +0000 (10:15 -0400)]
Merge pull request #41897 from ivancich/wip-versioned-deletes

rgw: when deleted obj removed in versioned bucket, extra del-marker added

Reviewed-by: Casey Bodley <cbodley@redhat.com>
3 years agocephadm: Introduce unit.stop 42242/head
Sebastian Wagner [Wed, 21 Jul 2021 12:13:52 +0000 (14:13 +0200)]
cephadm: Introduce unit.stop

Reason is we now have to stop two containers named differently. This is possible
with `bash -c ... echo %i | tr . -`. But we're not gainig anything readability
compared to putting it into a unit.stop script.

As not all daemons have this stop script, we still have to call podman for old daemons.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: don't log on exit.
Sebastian Wagner [Wed, 21 Jul 2021 09:46:16 +0000 (11:46 +0200)]
cephadm: don't log on exit.

At this point, the garbage collector already colleceted the `open` function

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: `stats` might return `--` for containers
Sebastian Wagner [Mon, 19 Jul 2021 10:13:11 +0000 (12:13 +0200)]
cephadm: `stats` might return `--` for containers

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: exec: also search for old cname
Sebastian Wagner [Wed, 14 Jul 2021 13:15:50 +0000 (15:15 +0200)]
cephadm: exec: also search for old cname

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: dashes: use both cnames for `inspect`
Sebastian Wagner [Tue, 13 Jul 2021 14:56:06 +0000 (16:56 +0200)]
cephadm: dashes: use both cnames for `inspect`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: Add CephContainer.for_daemon()
Sebastian Wagner [Mon, 19 Jul 2021 09:45:54 +0000 (11:45 +0200)]
cephadm: Add CephContainer.for_daemon()

To avoid calling get_container()

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agocephadm: use dashes for container names
Sebastian Wagner [Thu, 8 Jul 2021 09:52:52 +0000 (11:52 +0200)]
cephadm: use dashes for container names

podman adds the current container name to the /etc/hosts
file. Turns out, python's `socket.getfqdn()` differs from
`hostname -f`, when we have the container names containing
dots in it.:

[root@sebastians-laptop /]# cat /etc/hosts
127.0.0.1   localhost
::1         localhost
127.0.1.1   sebastians-laptop foo.bar.baz.com
[root@sebastians-laptop /]# hostname -f
sebastians-laptop
[root@sebastians-laptop /]# python3 -c 'import socket; print(socket.getfqdn())'
foo.bar.baz.com

Fascinatingly, this doesn't happen when using dashes.

Fixes: https://tracker.ceph.com/issues/51590
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42395 from sebastian-philipp/cephadm-improve-no-enigne-msg
Sebastian Wagner [Wed, 4 Aug 2021 12:49:41 +0000 (14:49 +0200)]
Merge pull request #42395 from sebastian-philipp/cephadm-improve-no-enigne-msg

cephadm: Clarify no container engine message

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agodoc/cephadm: MON IP change 42462/head
Sebastian Wagner [Fri, 23 Jul 2021 08:09:08 +0000 (10:09 +0200)]
doc/cephadm: MON IP change

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42587 from liewegas/rgw-mon-secure
Casey Bodley [Wed, 4 Aug 2021 12:31:40 +0000 (08:31 -0400)]
Merge pull request #42587 from liewegas/rgw-mon-secure

rgw: default ms_mon_client_mode = secure

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
3 years agoMerge pull request #42610 from rhcs-dashboard/52022-fix-rgw-perf-counters
Ernesto Puerta [Wed, 4 Aug 2021 11:55:48 +0000 (13:55 +0200)]
Merge pull request #42610 from rhcs-dashboard/52022-fix-rgw-perf-counters

mgr/dashboard: show perf. counters for rgw svc. on Cluster > Hosts

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #42554 from rhcs-dashboard/fix-rgw-ssl-cert-validation
Ernesto Puerta [Wed, 4 Aug 2021 11:37:33 +0000 (13:37 +0200)]
Merge pull request #42554 from rhcs-dashboard/fix-rgw-ssl-cert-validation

mgr/dashboard: fix ssl cert validation for rgw service creation

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #42575 from tchaikov/wip-venv
Kefu Chai [Wed, 4 Aug 2021 10:37:45 +0000 (18:37 +0800)]
Merge pull request #42575 from tchaikov/wip-venv

*: s/virtualenv/python -m venv/

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42594 from sebastian-philipp/mypy-ini-disallow-untyped-root
Kefu Chai [Wed, 4 Aug 2021 10:35:38 +0000 (18:35 +0800)]
Merge pull request #42594 from sebastian-philipp/mypy-ini-disallow-untyped-root

mypy.ini: Globally enable `disallow_untyped_defs = True`

Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42571 from tchaikov/wip-disable-kvs
Kefu Chai [Wed, 4 Aug 2021 10:31:40 +0000 (18:31 +0800)]
Merge pull request #42571 from tchaikov/wip-disable-kvs

cmake: disable kvs rados cls by default

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #42614 from soumyakoduri/wip-skoduri-sqlite3
Kefu Chai [Wed, 4 Aug 2021 09:56:18 +0000 (17:56 +0800)]
Merge pull request #42614 from soumyakoduri/wip-skoduri-sqlite3

cmake: Remove duplicate SQLITE module

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
3 years agomgr/dashboard: show perf. counters for rgw svc. on Cluster > Hosts 42610/head
Alfonso Martínez [Wed, 4 Aug 2021 09:11:50 +0000 (11:11 +0200)]
mgr/dashboard: show perf. counters for rgw svc. on Cluster > Hosts

- Use service_map_id to retrieve perf. counters for rgw services.
- Move logic from controller to CephService.

Fixes: https://tracker.ceph.com/issues/52022
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
3 years agorgw: allow ordered bucket listing to work when many filtered out entries 42125/head
J. Eric Ivancich [Mon, 19 Jul 2021 18:24:11 +0000 (14:24 -0400)]
rgw: allow ordered bucket listing to work when many filtered out entries

A previous PR moved the much of the filtering that's part of bucket
listing to the CLS layer. One unanticipated result was that it is now
possible for a call to return 0 entries. In such a case we want to
retry the call with the marker moved forward (i.e., advanced),
repeatedly if necessary, in order to either retrieve some entries or
to hit the end of the entries. This PR adds that functionality.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agorgw: allow CLSRGWConcurrentIO to handle "advancing" retries
J. Eric Ivancich [Mon, 19 Jul 2021 18:23:42 +0000 (14:23 -0400)]
rgw: allow CLSRGWConcurrentIO to handle "advancing" retries

When doing an asynchronous/concurrent bucket index operation against
multiple bucket index shards, a special error code is set aside to
indicate that an "advancing" retry of a/some shard(s) is necessary. In
that case another asynchronous call is made on the indicated shard(s)
from the client (i.e., CLSRGWConcurrentIO).  It is up to the subclass
of CLSRGWConcurrentIO to handle the retry such that it "advances" and
simply doesn't get stuck, looping forever.

The retry functionality only works when the "need_multiple_rounds"
functionality is not in use.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agorgw: de-conflate shard_id and request_id in CLSRGWConcurrentIO
J. Eric Ivancich [Fri, 16 Jul 2021 19:31:35 +0000 (15:31 -0400)]
rgw: de-conflate shard_id and request_id in CLSRGWConcurrentIO

When using asynchronous (concurrent) IO for bucket index requests,
there are two int ids that are used that need to be kept separate --
shard id and request id. In many cases they're the same -- shard 0
gets request 0, and so forth.

But in preparation for re-requests, those ids can diverge, where
request 13 maps to shard 2. The existing code maintained the OIDs that
went with each request. This PR also maintains the shard id as
well. Documentation has been beefed up to help future developers
navigate this.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agocmake: disable kvs rados cls by default 42571/head
Kefu Chai [Sat, 31 Jul 2021 03:36:37 +0000 (11:36 +0800)]
cmake: disable kvs rados cls by default

libcls_kvs was introduced back in
73d016fdb304ad19bba8aed3f2877b4bdb6ed32e, but we don't have an internal
user so far. to reduce the build time. let's disable the build of it by
default.

Signed-off-by: Kefu Chai <kchai@redhat.com>
3 years agoMerge pull request #42567 from dsavineau/issue_51973
Kefu Chai [Wed, 4 Aug 2021 00:56:37 +0000 (08:56 +0800)]
Merge pull request #42567 from dsavineau/issue_51973

cephadm: set global default container for ingress

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
3 years agoMerge pull request #42559 from tchaikov/wip-rpm-dwz
Kefu Chai [Wed, 4 Aug 2021 00:33:38 +0000 (08:33 +0800)]
Merge pull request #42559 from tchaikov/wip-rpm-dwz

rpm: enable dwz

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
3 years agoscript/ceph-release-notes: sort PRs case-insensitively
Josh Durgin [Tue, 3 Aug 2021 21:22:06 +0000 (17:22 -0400)]
script/ceph-release-notes: sort PRs case-insensitively

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
3 years agomgr/dashboard: fix ssl cert validation for rgw service creation 42554/head
Avan Thakkar [Thu, 29 Jul 2021 22:53:43 +0000 (04:23 +0530)]
mgr/dashboard: fix ssl cert validation for rgw service creation

Fixes: https://tracker.ceph.com/issues/51965
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
3 years agoscript/ceph-release-notes: adapt markdown output for new website
Josh Durgin [Tue, 3 Aug 2021 17:25:48 +0000 (13:25 -0400)]
script/ceph-release-notes: adapt markdown output for new website

* escape underscores
* avoid automatic links from PR titles containing .

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
3 years agorgw: clean up some code 42597/head
J. Eric Ivancich [Mon, 2 Aug 2021 19:46:47 +0000 (15:46 -0400)]
rgw: clean up some code

In rgw_admin.cc a variable `f` is used frequently. However due to the
sheer size of the file and variable scopes involved, finding the
declaration of `f`, which is a common character, can be
challenging. So it has been renamed `stream_flusher`, which better
indicates its purpose and makes it easier to find the corresponding
declaration.

Additionally some unneeded variables in RGWBucket::check_object_index
were removed. rgw::sal::Bucket::ListParams initializes parameters as
one would hope/expect.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agocmake: Remove duplicate SQLite module 42614/head
Soumya Koduri [Tue, 3 Aug 2021 17:46:06 +0000 (23:16 +0530)]
cmake: Remove duplicate SQLite module

Signed-off-by: Soumya Koduri <skoduri@redhat.com>