]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agodoc/cephadm: rewrite "cancelling the upgrade" 41188/head
Zac Dover [Thu, 6 May 2021 00:34:26 +0000 (10:34 +1000)]
doc/cephadm: rewrite "cancelling the upgrade"

This PR corrects a single prompt in the section
"Cancelling the Upgrade".

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #40937 from MrFreezeex/snap-remove-fix
Ilya Dryomov [Wed, 5 May 2021 20:42:42 +0000 (22:42 +0200)]
Merge pull request #40937 from MrFreezeex/snap-remove-fix

librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
4 years agoMerge PR #41148 into master
Sage Weil [Wed, 5 May 2021 18:53:22 +0000 (14:53 -0400)]
Merge PR #41148 into master

* refs/pull/41148/head:
Add Rocky Linux to supported DISTRO_NAMES

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge PR #41147 into master
Sage Weil [Wed, 5 May 2021 18:08:11 +0000 (14:08 -0400)]
Merge PR #41147 into master

* refs/pull/41147/head:
ceph_test_rados_api_service: stop threads before asserting
ceph_test_rados_api_service: more retries for servicemkap

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #40551 from alimaredia/wip-rgw-dpp-update-master
J. Eric Ivancich [Wed, 5 May 2021 16:28:53 +0000 (12:28 -0400)]
Merge pull request #40551 from alimaredia/wip-rgw-dpp-update-master

rgw,test: Add even more DoutPrefixProvider's to the RGW codebase

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #40063 from rhcs-dashboard/cyclomatic-complexity
Ernesto Puerta [Wed, 5 May 2021 16:11:03 +0000 (18:11 +0200)]
Merge pull request #40063 from rhcs-dashboard/cyclomatic-complexity

mgr/dashboard: reduce cyclomatic complexity

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #40715 from pcuzner/pool-overview-enhancement
Ernesto Puerta [Wed, 5 May 2021 16:08:58 +0000 (18:08 +0200)]
Merge pull request #40715 from pcuzner/pool-overview-enhancement

mgr/dashboard:include compression stats on pool dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge PR #41149 into master
Sage Weil [Wed, 5 May 2021 15:57:28 +0000 (11:57 -0400)]
Merge PR #41149 into master

* refs/pull/41149/head:
mgr/cephadm: remove haproxy+keepalived container image from service spec

Reviewed-by: Adam King <adking@redhat.com>
4 years agoMerge pull request #41081 from votdev/issue_49718_disable_nfs_v3
Ernesto Puerta [Wed, 5 May 2021 15:47:58 +0000 (17:47 +0200)]
Merge pull request #41081 from votdev/issue_49718_disable_nfs_v3

mgr/dashboard: disable NFSv3 support in dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #41133 from nh2/patch-1
Kefu Chai [Wed, 5 May 2021 14:54:52 +0000 (22:54 +0800)]
Merge pull request #41133 from nh2/patch-1

cephfs.pyx: Fix docstring of get_layout

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/cephfs: Fix docstring of get_layout 41133/head
Niklas Hambüchen [Tue, 4 May 2021 02:58:06 +0000 (04:58 +0200)]
pybind/cephfs: Fix docstring of get_layout

It was copy-pasted from another function.
Fixed by copy-pasting the docstring from the C docs.

Signed-off-by: Niklas Hambuechen <mail@nh2.me>
4 years agoMerge pull request #41165 from tchaikov/wip-cephfs-shell-manpage
Kefu Chai [Wed, 5 May 2021 09:43:26 +0000 (17:43 +0800)]
Merge pull request #41165 from tchaikov/wip-cephfs-shell-manpage

doc/man: build and install cephfs-shell manpage

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #40977 from xxhdx1985126/wip-seastore-xattr
Kefu Chai [Wed, 5 May 2021 07:23:00 +0000 (15:23 +0800)]
Merge pull request #40977 from xxhdx1985126/wip-seastore-xattr

crimson/os/seastore: formal extended attributes support

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41109 from markhpc/wip-bluefs_buffered_io-desc
Kefu Chai [Wed, 5 May 2021 06:45:40 +0000 (14:45 +0800)]
Merge pull request #41109 from markhpc/wip-bluefs_buffered_io-desc

src/common: Update bluefs_buffered_io descrption.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agodoc/man: build and install cephfs-shell manpage 41165/head
Kefu Chai [Tue, 4 May 2021 02:59:03 +0000 (10:59 +0800)]
doc/man: build and install cephfs-shell manpage

otherwise the manpage of cephfs-shell is not built and installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41142 from tchaikov/wip-cmake-global
Kefu Chai [Wed, 5 May 2021 04:52:53 +0000 (12:52 +0800)]
Merge pull request #41142 from tchaikov/wip-cmake-global

cmake: let libglobal_obj depend on legacy-option-headers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agodoc/man/8/cephfs-shell: reformat options
Kefu Chai [Tue, 4 May 2021 03:08:15 +0000 (11:08 +0800)]
doc/man/8/cephfs-shell: reformat options

* format global options using option directive
* fix the header, so man/conf.py is able to parse
  the description
* define "Synopsis" section to be consistent with other manpages.
* drop reference to glossary using "term" as manapge does not have
  reference to glossary entries.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41160 into master
Patrick Donnelly [Wed, 5 May 2021 01:00:27 +0000 (18:00 -0700)]
Merge PR #41160 into master

* refs/pull/41160/head:
Revert "Merge PR #41134 into master"

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoRevert "Merge PR #41134 into master" 41160/head
Patrick Donnelly [Tue, 4 May 2021 21:30:39 +0000 (14:30 -0700)]
Revert "Merge PR #41134 into master"

This reverts commit 93bca2d0e9d771e3019af5e7228af7986c20ab61, reversing
changes made to 7e70b15c437a2fbe207b515a98b8b21880980867.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agorgw: add DPP's to logging for most ops 40551/head
Ali Maredia [Sat, 30 Jan 2021 05:47:55 +0000 (00:47 -0500)]
rgw: add DPP's to logging for most ops

This commit contains the following major
changes:

- Most log output lines for major ops now have
DoutPrefixProviders.

- Create new macro ldpp_subdout. This macro is meant
as a replacement for lsubdout for headerfiles that
do not define the rgw subsys.

- Changes to RGWObjManifest begin and end iterators.

- Make it so that rgw_dencoder.cc should only contain the
logic related to encoding.

Also add dpp to ldouts and lderr's already using
req_state and replace sync_env/env->dpp's with dpp's

Signed-off-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #41152 from linuxbox2/wip-restore-rgw-lua
Casey Bodley [Tue, 4 May 2021 19:14:29 +0000 (15:14 -0400)]
Merge pull request #41152 from linuxbox2/wip-restore-rgw-lua

cmake: restore Lua scripting support for RGW

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #41024 from rhcs-dashboard/generate-manifest
Ernesto Puerta [Tue, 4 May 2021 18:52:30 +0000 (20:52 +0200)]
Merge pull request #41024 from rhcs-dashboard/generate-manifest

mgr/dashboard: Generate NPM dependencies manifest

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #41062 from rhcs-dashboard/50519-fix-ssl-cmd
Ernesto Puerta [Tue, 4 May 2021 18:46:05 +0000 (20:46 +0200)]
Merge pull request #41062 from rhcs-dashboard/50519-fix-ssl-cmd

mgr/dashboard: fix set-ssl-certificate{,-key} commands

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agomgr/cephadm: remove haproxy+keepalived container image from service spec 41149/head
Sage Weil [Tue, 4 May 2021 16:12:53 +0000 (12:12 -0400)]
mgr/cephadm: remove haproxy+keepalived container image from service spec

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agocmake: restore Lua scripting support for RGW 41152/head
Matt Benjamin [Tue, 4 May 2021 16:09:45 +0000 (12:09 -0400)]
cmake: restore Lua scripting support for RGW

RGW linkage with Lua was broken by the following commit:

commit 7471fbfa4bf015c972c0e577da2a1ee87f62adfd
Author: Kefu Chai <kchai@redhat.com>
Date:   Mon Apr 26 22:44:16 2021 +0800

    cmake: require lua if WITH_CEPHFS is enabled

    in src/mds/Mantle.cc, the lua C binding APIs are used, so we should only
    require lua if WITH_CEPHFS is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
4 years agoceph_test_rados_api_service: stop threads before asserting 41147/head
Sage Weil [Tue, 4 May 2021 16:06:17 +0000 (11:06 -0500)]
ceph_test_rados_api_service: stop threads before asserting

Otherwise, if we assert, we'll hang here:

Thread 1 (Thread 0x7f74eba79580 (LWP 1688617)):
#0  0x00007f74eb2aa529 in futex_wait (private=<optimized out>, expected=132, futex_word=0x7ffd642b4b54) at ../sysdeps/unix/sysv/linux/futex-internal.h:61
#1  futex_wait_simple (private=<optimized out>, expected=132, futex_word=0x7ffd642b4b54) at ../sysdeps/nptl/futex-internal.h:135
#2  __pthread_cond_destroy (cond=0x7ffd642b4b30) at pthread_cond_destroy.c:54

#3  0x0000563ff2e5a891 in LibRadosService_StatusFormat_Test::TestBody (this=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:78
#4  0x0000563ff2e9dc3a in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (location=0x563ff2ea72e4 "the test body", method=<optimized out>, object=0x563ff422a6d0)
    at ./src/googletest/googletest/src/gtest.cc:2605
#5  testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=object@entry=0x563ff422a6d0, method=<optimized out>, location=location@entry=0x563ff2ea72e4 "the test body")
    at ./src/googletest/googletest/src/gtest.cc:2641
#6  0x0000563ff2e908c3 in testing::Test::Run (this=0x563ff422a6d0) at ./src/googletest/googletest/src/gtest.cc:2680
#7  0x0000563ff2e90a25 in testing::TestInfo::Run (this=0x563ff41a3b70) at ./src/googletest/googletest/src/gtest.cc:2858
#8  0x0000563ff2e90ec1 in testing::TestSuite::Run (this=0x563ff41b6230) at ./src/googletest/googletest/src/gtest.cc:3012
#9  0x0000563ff2e92bdc in testing::internal::UnitTestImpl::RunAllTests (this=<optimized out>) at ./src/googletest/googletest/src/gtest.cc:5723
#10 0x0000563ff2e9e14a in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (location=0x563ff2ea8728 "auxiliary test code (environments or event listeners)",
    method=<optimized out>, object=0x563ff41a2d10) at ./src/googletest/googletest/src/gtest.cc:2605
#11 testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x563ff41a2d10, method=<optimized out>,
    location=location@entry=0x563ff2ea8728 "auxiliary test code (environments or event listeners)") at ./src/googletest/googletest/src/gtest.cc:2641
#12 0x0000563ff2e90ae8 in testing::UnitTest::Run (this=0x563ff30c0660 <testing::UnitTest::GetInstance()::instance>) at ./src/googletest/googletest/src/gtest.cc:5306

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoceph_test_rados_api_service: more retries for servicemkap
Sage Weil [Tue, 4 May 2021 16:01:38 +0000 (11:01 -0500)]
ceph_test_rados_api_service: more retries for servicemkap

Otherwise, this fails intermittently with

/build/ceph-16.2.1-257-g717ce59b/src/test/librados/service.cc:166: Failure
Expected: (0) != (retry), actual: 0 vs 0

Fixes: https://tracker.ceph.com/issues/50595
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoAdd Rocky Linux to supported DISTRO_NAMES 41148/head
Dennis Körner [Tue, 4 May 2021 15:59:59 +0000 (17:59 +0200)]
Add Rocky Linux to supported DISTRO_NAMES

Rocky Linux is a RHEL clone. I did a test-installation of ceph pacific on Rocky Linux RC1 with cephadm. As far as I can see, everything works as expected.

Signed-off-by: Dennis Körner <koerner@netzwerge.de>
4 years agoMerge PR #41134 into master
Patrick Donnelly [Tue, 4 May 2021 14:32:51 +0000 (07:32 -0700)]
Merge PR #41134 into master

* refs/pull/41134/head:
doc/man: build and install cephfs-shell manpage
doc/man/8/cephfs-shell: reformat options

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocmake: let libglobal_obj depend on legacy-option-headers 41142/head
Kefu Chai [Tue, 4 May 2021 13:07:01 +0000 (21:07 +0800)]
cmake: let libglobal_obj depend on legacy-option-headers

to address following build failure:

FAILED: src/global/CMakeFiles/libglobal_objs.dir/global_init.cc.o ...
src/global/CMakeFiles/libglobal_objs.dir/global_init.cc.o -MF src/global/CMakeFiles/libglobal_objs.dir/global_init.cc.o.d -o src/global/CMakeFiles/libglobal_objs.dir/global_init.cc.o -c
../src/global/global_init.cc
In file included from ../src/global/global_init.cc:26:
In file included from ../src/common/config.h:26:
In file included from ../src/common/config_values.h:59:
../src/common/options/legacy_config_opts.h:1:10: fatal error: 'global_legacy_options.h' file not found
         ^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40926 from ivancich/wip-orphan-list-pools
J. Eric Ivancich [Tue, 4 May 2021 13:08:48 +0000 (09:08 -0400)]
Merge pull request #40926 from ivancich/wip-orphan-list-pools

rgw: allow rgw-orphan-list to process multiple data pools

Reviewed-by: Michael Kidd <linuxkidd@gmail.com>
4 years agoMerge pull request #40631 from cbodley/wip-49780
Casey Bodley [Tue, 4 May 2021 12:08:51 +0000 (08:08 -0400)]
Merge pull request #40631 from cbodley/wip-49780

rgw/sts: read_obj_policy() consults iam_user_policies on ENOENT

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
4 years agolibrbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer 40937/head
Arthur Outhenin-Chalandre [Tue, 20 Apr 2021 11:51:45 +0000 (13:51 +0200)]
librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer

CreatePrimaryRequest could create some UnlinkPeerRequest with an already
unlinked peer in a scenario where you have multiple peers. This request
will not remove the peer (as it's already not linked to the requested
peer) and will skip deletion of the mirror snapshot if another peer
remains. Eventually the code will go through an infinite recursive loop
between CreatePrimaryRequest and UnlinkPeerRequest and segfault.

This commit adds an extra condition to make sure to not submit a
UnlinkPeerRequest if the peer is not linked to the current snapshot. If
there is already no peer in the list it will submit a UnlinkPeerRequest
to remove the snapshot.

Fixes: https://tracker.ceph.com/issues/50439
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
4 years agoMerge pull request #41033 from tchaikov/wip-osx-build
Kefu Chai [Tue, 4 May 2021 11:11:52 +0000 (19:11 +0800)]
Merge pull request #41033 from tchaikov/wip-osx-build

cmake,common: various cleanup for building on MacOS

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agotest/crimson: add tests for enhanced get/set attrs apis 40977/head
Xuehan Xu [Sun, 2 May 2021 07:22:16 +0000 (15:22 +0800)]
test/crimson: add tests for enhanced get/set attrs apis

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/os/seastore: allow upper layer to set attrs other than OI_ATTR and SS_ATTR
Xuehan Xu [Tue, 20 Apr 2021 07:47:51 +0000 (15:47 +0800)]
crimson/os/seastore: allow upper layer to set attrs other than OI_ATTR and SS_ATTR

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/os: change get/set attrs value to the bufferlist type
Xuehan Xu [Tue, 20 Apr 2021 07:45:43 +0000 (15:45 +0800)]
crimson/os: change get/set attrs value to the bufferlist type

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #41045 from jmolmo/fix_cv_lvm_lock
Sebastian Wagner [Tue, 4 May 2021 09:08:39 +0000 (11:08 +0200)]
Merge pull request #41045 from jmolmo/fix_cv_lvm_lock

mgr/cephadm: ceph-volume verbose only when fails

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #40313 from jmolmo/purge_iscsi_config
Sebastian Wagner [Tue, 4 May 2021 08:43:57 +0000 (10:43 +0200)]
Merge pull request #40313 from jmolmo/purge_iscsi_config

mgr/cephadm: Remove gateway.conf from iscsi pool when service is removed

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Sage Weil <sage@newdream.net>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #40799 from myoungwon/wip-fix-null-promote-obc
Samuel Just [Tue, 4 May 2021 07:55:33 +0000 (00:55 -0700)]
Merge pull request #40799 from myoungwon/wip-fix-null-promote-obc

osd: fix not set promote_obc when manifest object is rollbacked

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agodoc/man: build and install cephfs-shell manpage 41134/head
Kefu Chai [Tue, 4 May 2021 02:59:03 +0000 (10:59 +0800)]
doc/man: build and install cephfs-shell manpage

otherwise the manpage of cephfs-shell is not built and installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/man/8/cephfs-shell: reformat options
Kefu Chai [Tue, 4 May 2021 03:08:15 +0000 (11:08 +0800)]
doc/man/8/cephfs-shell: reformat options

* format global options using option directive
* fix the header, so man/conf.py is able to parse
  the description
* define "Synopsis" section to be consistent with other manpages.
* drop reference to glossary using "term" as manapge does not have
  reference to glossary entries.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41066 into master
Patrick Donnelly [Tue, 4 May 2021 00:51:37 +0000 (17:51 -0700)]
Merge PR #41066 into master

* refs/pull/41066/head:
mds: session dumps num_completed_flushes
mds: completed_requests -> num_completed_requests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #40963 into master
Patrick Donnelly [Tue, 4 May 2021 00:46:16 +0000 (17:46 -0700)]
Merge PR #40963 into master

* refs/pull/40963/head:
mds: do not trim the inodes from the lru list in standby_replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #40962 into master
Patrick Donnelly [Tue, 4 May 2021 00:43:47 +0000 (17:43 -0700)]
Merge PR #40962 into master

* refs/pull/40962/head:
test: add test to validate snap synchronization with parent directory snapshots
cephfs-mirror: ignore parent directory snapshots when building snap map

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #40927 into master
Patrick Donnelly [Tue, 4 May 2021 00:39:41 +0000 (17:39 -0700)]
Merge PR #40927 into master

* refs/pull/40927/head:
mds: reset the return value for heap command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #40903 into master
Patrick Donnelly [Tue, 4 May 2021 00:37:34 +0000 (17:37 -0700)]
Merge PR #40903 into master

* refs/pull/40903/head:
test: add test case for MDS privated inos accessing
mds: make the lost+found dir accessible from clients
mds: move the inos 1,2 and 3 macros to ceph_fs.h

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #40882 into master
Patrick Donnelly [Tue, 4 May 2021 00:35:24 +0000 (17:35 -0700)]
Merge PR #40882 into master

* refs/pull/40882/head:
client: Fix executeable access check for the root user

Reviewed-by: Jeff Layton <jlayton@redhat.com>
4 years agoMerge PR #41120 into master
Patrick Donnelly [Mon, 3 May 2021 21:51:55 +0000 (14:51 -0700)]
Merge PR #41120 into master

* refs/pull/41120/head:
doc/_ext: ignore desc if it is unavailable
doc/_ext: check "default" for None
doc/_ext: print 0B if option value is 0
doc/cephfs: render options using confval directive

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #41102 from soumyakoduri/set_acl_fix
Casey Bodley [Mon, 3 May 2021 18:27:58 +0000 (14:27 -0400)]
Merge pull request #41102 from soumyakoduri/set_acl_fix

rgw/acl: set_acl shouldn't overwrite bucket attrs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agomgr/dashboard: fix set-ssl-certificate{,-key} commands 41062/head
Alfonso Martínez [Mon, 3 May 2021 16:46:08 +0000 (18:46 +0200)]
mgr/dashboard: fix set-ssl-certificate{,-key} commands

- Now create-self-signed-cert command relies on set-ssl-certificate{,-key} commands.
- Simplify the command testing and increase the test coverage.

Fixes: https://tracker.ceph.com/issues/50519
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
4 years agomgr/dashboard: disable NFSv3 support in dashboard 41081/head
Volker Theile [Thu, 29 Apr 2021 11:21:11 +0000 (13:21 +0200)]
mgr/dashboard: disable NFSv3 support in dashboard

Fixes: https://tracker.ceph.com/issues/49718
Related to: https://github.com/ceph/ceph/pull/40154

Signed-off-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #41093 from idryomov/wip-rbd-pwl-ssd-boundary
Ilya Dryomov [Mon, 3 May 2021 13:27:18 +0000 (15:27 +0200)]
Merge pull request #41093 from idryomov/wip-rbd-pwl-ssd-boundary

librbd/cache/pwl/ssd/WriteLog: don't crash on split log entries

Reviewed-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
4 years agoMerge PR #41105 into master
Sage Weil [Mon, 3 May 2021 12:28:15 +0000 (08:28 -0400)]
Merge PR #41105 into master

* refs/pull/41105/head:
qa/workunits/cephadm/test_cephadm: test zap-osds
cephadm: add --zap-osds argument to rm_cluster
cephadm: implement zap-osds --fsid ... command

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agolibrbd/cache/pwl/ssd/WriteLog: don't crash on split log entries 41093/head
Ilya Dryomov [Thu, 29 Apr 2021 13:03:46 +0000 (15:03 +0200)]
librbd/cache/pwl/ssd/WriteLog: don't crash on split log entries

write_log_entries() will split a log entry at the end of the log, the
remainder is written to the beginning at DATA_RING_BUFFER_OFFSET.  On
the read side aio_read_data_block() doesn't handle this case and just
crashes.  Unless the workload in use is <= 4K, the image is rendered
unusable sooner or later.

Fixes: https://tracker.ceph.com/issues/50589
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agoMerge pull request #41068 from idryomov/wip-rbd-pwl-ssd-capacity
Ilya Dryomov [Mon, 3 May 2021 12:00:48 +0000 (14:00 +0200)]
Merge pull request #41068 from idryomov/wip-rbd-pwl-ssd-capacity

librbd/cache/pwl/ssd/WriteLog: decrement m_bytes_allocated when retiring

Reviewed-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
4 years agoosd: fix not set promote_obc when manifest object is rollbacked 40799/head
myoungwon oh [Sun, 11 Apr 2021 15:37:11 +0000 (00:37 +0900)]
osd: fix not set promote_obc when manifest object is rollbacked

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
4 years agomgr/dashboard:include compression stats on pool dashboard 40715/head
Paul Cuzner [Thu, 8 Apr 2021 04:43:22 +0000 (16:43 +1200)]
mgr/dashboard:include compression stats on pool dashboard

This is a replacement dashboard configuration for the
pool overview page. It provides a cluster wide view of
capacity consumed and compression effectiveness, and
breaks this down by each pool within the configuration.

Fixes: https://tracker.ceph.com/issues/50226
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
4 years agoMerge pull request #41042 from tchaikov/wip-rbd-cls-sparse-size
Ilya Dryomov [Sun, 2 May 2021 21:31:06 +0000 (23:31 +0200)]
Merge pull request #41042 from tchaikov/wip-rbd-cls-sparse-size

librbd: 32-bit and libc++ portability fixups

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
4 years agorgw/acl: set_acl shouldn't overwrite bucket attrs 41102/head
Soumya Koduri [Thu, 29 Apr 2021 20:40:06 +0000 (02:10 +0530)]
rgw/acl: set_acl shouldn't overwrite bucket attrs

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
4 years agodoc/_ext: ignore desc if it is unavailable 41120/head
Kefu Chai [Sun, 2 May 2021 11:57:27 +0000 (19:57 +0800)]
doc/_ext: ignore desc if it is unavailable

there is chance that we don't have desc, desc_long or fmt_desc, in that
case, we should just skip desc before checking its length. so, just use
'if desc' which is able to check for None or empty string.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/_ext: check "default" for None
Kefu Chai [Sun, 2 May 2021 10:17:00 +0000 (18:17 +0800)]
doc/_ext: check "default" for None

default could be None and we don't reference the default value using
`opt.default`, so this variable is always defined. to avoid printing
a None default value, check for a None value. see
https://jinja.palletsprojects.com/en/2.11.x/templates/#none

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/_ext: print 0B if option value is 0
Kefu Chai [Sun, 2 May 2021 09:50:36 +0000 (17:50 +0800)]
doc/_ext: print 0B if option value is 0

before this change, the rendered Default is "0Ei" if the default size is 0,
would be more readable if it is "0B" or "0".

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/cephfs: render options using confval directive
Kefu Chai [Sun, 2 May 2021 09:13:33 +0000 (17:13 +0800)]
doc/cephfs: render options using confval directive

less repeating, better maintainability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41085 from idryomov/wip-rbd-nuke-source-spec-file
Mykola Golub [Sun, 2 May 2021 09:11:10 +0000 (12:11 +0300)]
Merge pull request #41085 from idryomov/wip-rbd-nuke-source-spec-file

rbd: --source-spec-file should be --source-spec-path

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agolibrados/librados_c: check .symver support using cmake 41033/head
Kefu Chai [Tue, 27 Apr 2021 03:15:32 +0000 (11:15 +0800)]
librados/librados_c: check .symver support using cmake

the __asm__(".asmver ..") is a support provided by the compiler, so
would be better to detect it by either checking the compiler identifer
or just try it out.

in this change, instead of checking the building platform, we check this
feature using check_c_source_compiles().

in future, we could support versioned symbols using function attriubte
or symbol tables or version-script.

on platform where symbol versioning is not supported, we might need to
go with a different approach.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/util: enable cgroup only on linux
Kefu Chai [Tue, 27 Apr 2021 02:11:53 +0000 (10:11 +0800)]
common/util: enable cgroup only on linux

cgroups, i.e. control groups, are a linux kernel feature. this is not
available on FreeBSD, Win32 or MacOS. furthermore, we are reading the
information of it by accessing files exposed using a tmpfs filesystem
mounted at /sys/fs/cgroup, which is not available on other systems.

in this change, instead of guarding it using `#ifndef _WIN32`, it is now
guraded using `defined(__linux__)`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoinclude/compat.h: declare cpu_set_t for MacOS
Kefu Chai [Mon, 26 Apr 2021 16:29:37 +0000 (00:29 +0800)]
include/compat.h: declare cpu_set_t for MacOS

to appease the compiler as it needs to compile numa.cc

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/Throttle: remove unused member variable
Kefu Chai [Mon, 26 Apr 2021 16:02:14 +0000 (00:02 +0800)]
common/Throttle: remove unused member variable

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/compat: detect memset_s() using check_function_exists()
Kefu Chai [Mon, 26 Apr 2021 16:00:37 +0000 (00:00 +0800)]
common/compat: detect memset_s() using check_function_exists()

more robust this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/ceph_time: do not include non-existent header file
Kefu Chai [Mon, 26 Apr 2021 15:46:04 +0000 (23:46 +0800)]
common/ceph_time: do not include non-existent header file

i have following error when building on MacOS

ceph/src/common/ceph_time.cc:29:10: fatal error: 'ostringstream' file not found
         ^~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/StackStringStream: make override method with "final"
Kefu Chai [Mon, 26 Apr 2021 15:36:04 +0000 (23:36 +0800)]
common/StackStringStream: make override method with "final"

this change also silences the warnings like:

src/common/StackStringStream.h:75:7: warning: 'overflow' overrides a member function but is not marked 'override' [-Winconsistent-missing-override]
  int overflow(int c)
      ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: detect suseconds_t using CHECK_TYPE_SIZE
Kefu Chai [Mon, 26 Apr 2021 15:29:51 +0000 (23:29 +0800)]
cmake: detect suseconds_t using CHECK_TYPE_SIZE

instead assuming it's a macro use CHECK_TYPE_SIZE to check it.

on Darwin, it's defined as `__darwin_suseconds_t`

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: do not build libblk if libblk_srcs is empty
Kefu Chai [Mon, 26 Apr 2021 15:17:02 +0000 (23:17 +0800)]
cmake: do not build libblk if libblk_srcs is empty

if we don't or just not able to buld libblk, let's skip it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: require lua if WITH_CEPHFS is enabled
Kefu Chai [Mon, 26 Apr 2021 14:44:16 +0000 (22:44 +0800)]
cmake: require lua if WITH_CEPHFS is enabled

in src/mds/Mantle.cc, the lua C binding APIs are used, so we should only
require lua if WITH_CEPHFS is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: check for empty string by its value
Kefu Chai [Mon, 26 Apr 2021 14:38:35 +0000 (22:38 +0800)]
cmake: check for empty string by its value

the behavior of cmake 3.17 is a little bit different as it consider

NOT _library STREQUAL ""

as true, if ${_library} is "".

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41116 from tchaikov/wip-in_addr_t
Kefu Chai [Sun, 2 May 2021 00:09:03 +0000 (08:09 +0800)]
Merge pull request #41116 from tchaikov/wip-in_addr_t

common/pick_address: define in_addr_t if it is not defined

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #41117 from tchaikov/wip-doc-cephfs-conf
Kefu Chai [Sat, 1 May 2021 15:34:11 +0000 (23:34 +0800)]
Merge pull request #41117 from tchaikov/wip-doc-cephfs-conf

doc/cephfs/mds-config-ref: remove duplicated confval

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocommon/pick_address: define in_addr_t if it is not defined 41116/head
Kefu Chai [Sat, 1 May 2021 15:30:18 +0000 (23:30 +0800)]
common/pick_address: define in_addr_t if it is not defined

neither mingw not not have in_addr_t defined, see
https://docs.microsoft.com/en-us/windows/win32/api/winsock2/ns-winsock2-in_addr,
so define it if it is not defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/cephfs/mds-config-ref: remove duplicated confval 41117/head
Kefu Chai [Sat, 1 May 2021 15:05:36 +0000 (23:05 +0800)]
doc/cephfs/mds-config-ref: remove duplicated confval

this change addresses the warning of:

/home/jenkins-build/build/workspace/ceph-pr-docs/doc/cephfs/mds-config-ref.rst:2: WARNING: duplicate confval_option description of mds_cache_memory_limit, other instance in cephfs/cache-configuration
/home/jenkins-build/build/workspace/ceph-pr-docs/doc/cephfs/mds-config-ref.rst:2: WARNING: duplicate confval_option description of mds_cache_reservation, other instance in cephfs/cache-configuration

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41043 from ifed01/wip-ifed-fix-use-ratio
Kefu Chai [Sat, 1 May 2021 04:59:41 +0000 (12:59 +0800)]
Merge pull request #41043 from ifed01/wip-ifed-fix-use-ratio

osd: compute OSD's space usage ratio via raw space utilization

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41012 into master
Patrick Donnelly [Sat, 1 May 2021 04:03:17 +0000 (21:03 -0700)]
Merge PR #41012 into master

* refs/pull/41012/head:
doc/cephfs/cache-config: use config item macros

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40961 from tchaikov/wip-50456
Kefu Chai [Sat, 1 May 2021 01:11:30 +0000 (09:11 +0800)]
Merge pull request #40961 from tchaikov/wip-50456

common/pick_address: filter out loopback addresses

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #41071 from tchaikov/wip-cmake-disable-vta
Kefu Chai [Sat, 1 May 2021 01:09:05 +0000 (09:09 +0800)]
Merge pull request #41071 from tchaikov/wip-cmake-disable-vta

cmake: disable "variable tracking" when building rados python binding

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #41091 into master
Patrick Donnelly [Sat, 1 May 2021 00:15:55 +0000 (17:15 -0700)]
Merge PR #41091 into master

* refs/pull/41091/head:
doc/cephfs/nfs: add section about ganesha logs
doc/cephfs/nfs: Replace volume/nfs with nfs
doc/cephfs/nfs: add note about export management with volume/nfs interface only

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #41004 from sseshasa/wip-mclock-max-backfill-misc-fix
Sridhar Seshasayee [Fri, 30 Apr 2021 17:28:21 +0000 (22:58 +0530)]
Merge pull request #41004 from sseshasa/wip-mclock-max-backfill-misc-fix

osd: Override recovery, backfill and sleep related config options during OSD and mclock scheduler initialization.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
4 years agosrc/common: Update bluefs_buffered_io descrption. 41109/head
Mark Nelson [Fri, 30 Apr 2021 15:28:18 +0000 (15:28 +0000)]
src/common: Update bluefs_buffered_io descrption.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
4 years agoqa/workunits/cephadm/test_cephadm: test zap-osds 41105/head
Sage Weil [Fri, 30 Apr 2021 14:16:16 +0000 (10:16 -0400)]
qa/workunits/cephadm/test_cephadm: test zap-osds

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agocephadm: add --zap-osds argument to rm_cluster
Sage Weil [Fri, 30 Apr 2021 14:13:23 +0000 (10:13 -0400)]
cephadm: add --zap-osds argument to rm_cluster

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agocephadm: implement zap-osds --fsid ... command
Sage Weil [Fri, 30 Apr 2021 13:54:40 +0000 (09:54 -0400)]
cephadm: implement zap-osds --fsid ... command

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #40967 from dang/wip-dang-zipper-user
Daniel Gryniewicz [Fri, 30 Apr 2021 14:03:58 +0000 (10:03 -0400)]
Merge pull request #40967 from dang/wip-dang-zipper-user

RGW Zipper consistent load/store APIs for User

4 years agoMerge pull request #40946 from ifed01/wip-ifed-dump-pool-statfs
Kefu Chai [Fri, 30 Apr 2021 11:43:18 +0000 (19:43 +0800)]
Merge pull request #40946 from ifed01/wip-ifed-dump-pool-statfs

osd: admin cmd to dump pools statfs for a specific OSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41014 from smithfarm/wip-mempool-cacheline-49781
Kefu Chai [Fri, 30 Apr 2021 11:36:17 +0000 (19:36 +0800)]
Merge pull request #41014 from smithfarm/wip-mempool-cacheline-49781

qa: verify the benefits of mempool cacheline optimization

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #40998 from cyx1231st/wip-seastore-onode-tree-delete
Samuel Just [Fri, 30 Apr 2021 06:22:42 +0000 (23:22 -0700)]
Merge pull request #40998 from cyx1231st/wip-seastore-onode-tree-delete

crimson/onode-staged-tree: implement/validate features to erase values from tree

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agodoc/cephfs/nfs: add section about ganesha logs 41091/head
Varsha Rao [Thu, 29 Apr 2021 17:42:20 +0000 (23:12 +0530)]
doc/cephfs/nfs: add section about ganesha logs

Fixes: https://tracker.ceph.com/issues/50008
Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/cephfs/nfs: Replace volume/nfs with nfs
Varsha Rao [Thu, 29 Apr 2021 12:33:53 +0000 (18:03 +0530)]
doc/cephfs/nfs: Replace volume/nfs with nfs

Recently, nfs related code was moved out of volumes plugin[1]. So using the
name volume/nfs for the interface is not appropriate.

[1] https://github.com/ceph/ceph/pull/40526

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agodoc/cephfs/nfs: add note about export management with volume/nfs interface only
Varsha Rao [Thu, 29 Apr 2021 12:25:45 +0000 (17:55 +0530)]
doc/cephfs/nfs: add note about export management with volume/nfs interface only

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agoqa: verify the benefits of mempool cacheline optimization 41014/head
Loïc Dachary [Fri, 19 Mar 2021 07:29:08 +0000 (08:29 +0100)]
qa: verify the benefits of mempool cacheline optimization

There already is a test to verify the mempool sharding works, in the sense that
it uses at least half of the variables available to count the number of
allocated objects and their total size. This new test verifies that, with
sharding, object counting is at least twice faster than without sharding. It
also collects cacheline contention data with the perf c2c tool. The manual
analysis of this data shows the optimization gain is indeed related to cacheline
contention.

Fixes: https://tracker.ceph.com/issues/49896
Signed-off-by: Loïc Dachary <loic@dachary.org>
4 years agolibrbd/cache/pwl: include head and tail pointers in STATS 41068/head
Ilya Dryomov [Wed, 28 Apr 2021 12:27:12 +0000 (14:27 +0200)]
librbd/cache/pwl: include head and tail pointers in STATS

While at it, reduce the number of calls to operator<< and drop
the trailing comma.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agolibrbd/cache/pwl: bump "Waiting for allocation" and "Retiring" dout level
Ilya Dryomov [Wed, 28 Apr 2021 12:27:12 +0000 (14:27 +0200)]
librbd/cache/pwl: bump "Waiting for allocation" and "Retiring" dout level

Bump "Waiting for allocation" to 5.

"Retiring" is at 20 for rwl and 1 for ssd.  Bump the latter to 20 as
well.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>