]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agomds/Mutation: initialize MutationImpl object's initiated_at 41371/head
Ramana Raja [Mon, 17 May 2021 22:54:40 +0000 (18:54 -0400)]
mds/Mutation: initialize MutationImpl object's initiated_at

... to current time, ceph_clock_now(). This is a more useful
initialized value than utime_t().

Fixes: https://tracker.ceph.com/issues/42516
Signed-off-by: Ramana Raja <rraja@redhat.com>
Co-Authored-By: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #40969 from zdover23/wip-doc-cephadm-osd-filters-colons-2021-apr-21
zdover23 [Wed, 21 Apr 2021 19:23:07 +0000 (05:23 +1000)]
Merge pull request #40969 from zdover23/wip-doc-cephadm-osd-filters-colons-2021-apr-21

doc/cephadm: osd.rst -- removing colons

Reviewed-by: Sage Weil <sweil@redhat.com>
4 years agodoc/cephadm: osd.rst -- removing colons 40969/head
Zac Dover [Wed, 21 Apr 2021 18:14:24 +0000 (04:14 +1000)]
doc/cephadm: osd.rst -- removing colons

This PR removes colons that weren't necessary.
This is a companion PR to PR#40996.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #40705 from orozery/libcryptsetup-return-checks
Mykola Golub [Wed, 21 Apr 2021 18:10:32 +0000 (21:10 +0300)]
Merge pull request #40705 from orozery/libcryptsetup-return-checks

librbd/crypto: fix bad return checks from libcryptsetup

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge PR #40966 into master
Sage Weil [Wed, 21 Apr 2021 17:35:40 +0000 (13:35 -0400)]
Merge PR #40966 into master

* refs/pull/40966/head:
doc/cephadm: osd: rewrite "additional opts"

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agodoc/cephadm: osd: rewrite "additional opts" 40966/head
Zac Dover [Wed, 21 Apr 2021 15:01:01 +0000 (01:01 +1000)]
doc/cephadm: osd: rewrite "additional opts"

This PR rewrites the "Additional Options"
subsection of the OSD chapter of the cephadm
guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #40914 from zdover23/wip-doc-cephadm-osd-advanced-up-to-filters...
zdover23 [Wed, 21 Apr 2021 14:21:38 +0000 (00:21 +1000)]
Merge pull request #40914 from zdover23/wip-doc-cephadm-osd-advanced-up-to-filters-2021-apr-18

doc/cephadm: rewrite "advanced osd s. specs"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agolibrbd/crypto: fix bad return checks from libcryptsetup 40705/head
Or Ozeri [Thu, 8 Apr 2021 17:13:49 +0000 (20:13 +0300)]
librbd/crypto: fix bad return checks from libcryptsetup

This commit fixes the return checks for libcryptsetup functions
that may return non-zero success codes.

Fixes: https://tracker.ceph.com/issues/50461
Signed-off-by: Or Ozeri <oro@il.ibm.com>
4 years agoMerge pull request #40935 from TRYTOBE8TME/wip-rgw-bootstrap-fix
Yuval Lifshitz [Wed, 21 Apr 2021 08:39:03 +0000 (11:39 +0300)]
Merge pull request #40935 from TRYTOBE8TME/wip-rgw-bootstrap-fix

test/rgw: Removing unrecognized option from bootstrap file

4 years agoMerge pull request #40953 from tchaikov/wip-doc-confval
Kefu Chai [Wed, 21 Apr 2021 06:12:57 +0000 (14:12 +0800)]
Merge pull request #40953 from tchaikov/wip-doc-confval

common/options,doc/rados/configuration: extract crimson options, use confval directive

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agodoc/conf.py: use glob.glob() to find .yaml.in files 40953/head
Kefu Chai [Wed, 21 Apr 2021 05:32:35 +0000 (13:32 +0800)]
doc/conf.py: use glob.glob() to find .yaml.in files

improve the maintainability of .yaml.in files.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: use file(GLOB ..) to find .yaml.in files
Kefu Chai [Wed, 21 Apr 2021 05:28:52 +0000 (13:28 +0800)]
cmake: use file(GLOB ..) to find .yaml.in files

instead of hardcode the option file names, use file(GLOB ..) to find
and collect them instead, for better maintainability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/optinos: extract crimson options out
Kefu Chai [Wed, 21 Apr 2021 04:46:22 +0000 (12:46 +0800)]
common/optinos: extract crimson options out

prepare for the change to split the options by services

in future, osd will have its own osd.yaml.in, while crimson will consume
both osd.yaml.in and crimson.yaml.in.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/mon-config-ref: use confval directive
Kefu Chai [Wed, 21 Apr 2021 04:36:43 +0000 (12:36 +0800)]
doc/rados/configuration/mon-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/general-config-ref: use confval directive
Kefu Chai [Wed, 21 Apr 2021 04:22:53 +0000 (12:22 +0800)]
doc/rados/configuration/general-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/osd-config-ref: use confval directive
Kefu Chai [Wed, 21 Apr 2021 04:13:14 +0000 (12:13 +0800)]
doc/rados/configuration/osd-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40915 from rzarzynski/wip-crimson-backtrace-3
Kefu Chai [Wed, 21 Apr 2021 03:54:32 +0000 (11:54 +0800)]
Merge pull request #40915 from rzarzynski/wip-crimson-backtrace-3

crimson: make backtraces more human-readable with boost::stacktrace and addr2line

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40939 from tchaikov/wip-doc-confval
Kefu Chai [Wed, 21 Apr 2021 02:15:46 +0000 (10:15 +0800)]
Merge pull request #40939 from tchaikov/wip-doc-confval

doc/rados/configuration/osd-config-ref:  use confval directive

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agosrc/test: Removing unrecognized option from bootstrap file 40935/head
Kalpesh [Tue, 20 Apr 2021 11:07:29 +0000 (16:37 +0530)]
src/test: Removing unrecognized option from bootstrap file

Removing '--distribute' option from a command in bootstrap file which seemed to be the reason of failure in:
http://qa-proxy.ceph.com/teuthology/dang-2021-04-19_12:28:43-rgw-wip-dang-zipper-sysobj-distro-basic-smithi/6059513/teuthology.log

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
4 years agoMerge PR #39742 into master
Patrick Donnelly [Tue, 20 Apr 2021 21:01:32 +0000 (14:01 -0700)]
Merge PR #39742 into master

* refs/pull/39742/head:
client: make Inode to inherit from RefCountedObject
client: minor cleanup to Inode class to simplify the code
client: minor cleanup to Inode related code

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #40191 from dang/wip-dang-zipper-sysobj
Daniel Gryniewicz [Tue, 20 Apr 2021 18:19:47 +0000 (14:19 -0400)]
Merge pull request #40191 from dang/wip-dang-zipper-sysobj

RGW Zipper - Remove rgw_pool from API

4 years agodoc/rados/configuration/osd-config-ref: use confval directive 40939/head
Kefu Chai [Mon, 19 Apr 2021 15:28:49 +0000 (23:28 +0800)]
doc/rados/configuration/osd-config-ref:  use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40928 from idryomov/wip-gid-reclaim-notes
Ilya Dryomov [Tue, 20 Apr 2021 08:46:29 +0000 (10:46 +0200)]
Merge pull request #40928 from idryomov/wip-gid-reclaim-notes

doc: 14.2.20, 15.2.11 and 16.2.1 releases and notes (CVE-2021-20288)

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
4 years agodoc/releases/index: update for 14.2.20, 15.2.11 and 16.2.1 40928/head
Ilya Dryomov [Tue, 20 Apr 2021 08:05:28 +0000 (10:05 +0200)]
doc/releases/index: update for 14.2.20, 15.2.11 and 16.2.1

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agoMerge pull request #40877 from mgfritch/doc-rgw-python3-print
Kefu Chai [Tue, 20 Apr 2021 07:02:49 +0000 (15:02 +0800)]
Merge pull request #40877 from mgfritch/doc-rgw-python3-print

doc/rgw: update to python3 print syntax

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40846 from athanatos/sjust/wip-object-data
Samuel Just [Tue, 20 Apr 2021 06:50:22 +0000 (23:50 -0700)]
Merge pull request #40846 from athanatos/sjust/wip-object-data

seastore: initial object data support

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
4 years agoMerge pull request #40911 from tchaikov/wip-doc-confval
Kefu Chai [Tue, 20 Apr 2021 00:03:48 +0000 (08:03 +0800)]
Merge pull request #40911 from tchaikov/wip-doc-confval

doc/rados/configuration/auth-config-ref: drop section of keyrings

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agodoc/cephadm: rewrite "advanced osd s. specs" 40914/head
Zac Dover [Mon, 19 Apr 2021 14:38:44 +0000 (00:38 +1000)]
doc/cephadm: rewrite "advanced osd s. specs"

This PR improves the readbility and elegance of
the "Advanced OSD Service Specifications" section
of the OSD chapter of the cephadm guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agocrimson/common: do backtrace via boost::stackstrace and addr2line. 40915/head
Radoslaw Zarzynski [Mon, 19 Apr 2021 14:41:37 +0000 (14:41 +0000)]
crimson/common: do backtrace via boost::stackstrace and addr2line.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/common: don't use Seastar's handle_signal() for fatal signals.
Radoslaw Zarzynski [Mon, 19 Apr 2021 14:32:05 +0000 (14:32 +0000)]
crimson/common: don't use Seastar's handle_signal() for fatal signals.

The `handle_signal()` way requires going through the reactor, and
thus is unsuitable for situations when it's no longer operational.
SIGSEGV generated as a result of actual fault (not `kill()`) is
the intended audience of this change.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/common: let's dump the core on fatal signal.
Radoslaw Zarzynski [Mon, 19 Apr 2021 13:59:25 +0000 (13:59 +0000)]
crimson/common: let's dump the core on fatal signal.

Removing the call to engine's `exit()` should allow the singnal
handler to exit. In the case of `SIGSEGV` this would return back
to the problematic instruction, and thus retrigger the fault.
As our handler already restores `SIG_DFL`, it's expected to get
a core dump.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: handle fatal signals
Kefu Chai [Tue, 8 Sep 2020 05:44:44 +0000 (13:44 +0800)]
crimson/osd: handle fatal signals

seastar sets the signal handler for SIGSEGV and SIGABRT, and its signal
handler prints the stacktrace, but the the stack frames are adresses,
which are not human readable without the help of addr2line.

since crimson is linked with -rdynamic option, we have the symbols added
to the dynamic symbol table already. let print out the symbolized
addresses instead using our own stacktrace utility.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6e26243b6db49bb9813f8b8aeade68da07dc6065)

 Conflicts:
src/crimson/osd/main.cc

4 years agodoc/man/8/ceph-authtool: fix the indent of option 40911/head
Kefu Chai [Mon, 19 Apr 2021 14:09:52 +0000 (22:09 +0800)]
doc/man/8/ceph-authtool: fix the indent of option

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40899 from malcolmholmes/malcolmholmes/fix-json-dashboard-error
Ernesto Puerta [Mon, 19 Apr 2021 14:09:39 +0000 (16:09 +0200)]
Merge pull request #40899 from malcolmholmes/malcolmholmes/fix-json-dashboard-error

mgr/Dashboard: Remove erroneous elements in hosts-overview Grafana dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: p-se <NOT@FOUND>
4 years agoMerge pull request #40898 from rzarzynski/wip-crimson-only-mgrclt-noconn-report
Kefu Chai [Mon, 19 Apr 2021 14:06:27 +0000 (22:06 +0800)]
Merge pull request #40898 from rzarzynski/wip-crimson-only-mgrclt-noconn-report

crimson/mgr: don't report if there is no connection available.

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/mon-config-ref: use confval directive
Kefu Chai [Mon, 19 Apr 2021 13:56:14 +0000 (21:56 +0800)]
doc/rados/configuration/mon-config-ref:  use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/auth-config-ref: drop section of keyrings
Kefu Chai [Mon, 19 Apr 2021 13:45:05 +0000 (21:45 +0800)]
doc/rados/configuration/auth-config-ref: drop section of keyrings

as per Sage, the caps are out of date (should be based on profiles) and
the keyring locations aren't useful information

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40897 from tchaikov/wip-doc-more-conf-vals
Kefu Chai [Mon, 19 Apr 2021 13:43:43 +0000 (21:43 +0800)]
Merge pull request #40897 from tchaikov/wip-doc-more-conf-vals

doc/rados/configuration: use confval directive to define options.

Reviewed-by: Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoRGW Zipper - Remove rgw_pool/sysobj from API 40191/head
Daniel Gryniewicz [Wed, 17 Mar 2021 13:22:22 +0000 (09:22 -0400)]
RGW Zipper - Remove rgw_pool/sysobj from API

rgw_pool and sysobj are part of the RADOS backend, and do not belong in
the API.  Instead, pull the objects using them into the API.  These are:
RGWOIDCProvider, RGWRole, and a new object LuaScript.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #40905 from idryomov/wip-rbd-kms-config-map-doc
Ilya Dryomov [Mon, 19 Apr 2021 13:40:21 +0000 (15:40 +0200)]
Merge pull request #40905 from idryomov/wip-rbd-kms-config-map-doc

doc/rbd/rbd-kubernetes: update for recent ceph-csi versions

Reviewed-by: Madhu Rajanna <madhupr007@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #40378 from zdover23/wip-doc-cephadm-osd-declarative-state-rewrite...
zdover23 [Mon, 19 Apr 2021 13:03:21 +0000 (23:03 +1000)]
Merge pull request #40378 from zdover23/wip-doc-cephadm-osd-declarative-state-rewrite-2021-Mar-25

doc/cephadm: rewrite "delcarative state" in osd.rst

Reviewed-by: Sage Weil <sweil@redhat.com>
4 years agoMerge PR #40817 into master
Sage Weil [Mon, 19 Apr 2021 12:30:42 +0000 (08:30 -0400)]
Merge PR #40817 into master

* refs/pull/40817/head:
mgr/MgrStandby: fix config observer
mgr/MgrStandby: respawn if mgr_standby_modules changes
qa/tasks/mgr/test_dashboard: skip test_standby if mgr_standby_modules=false
qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
qa/suites/rados/cephadm/smoke-singlehost: test --single-host-defaults
cephadm: add --single-host-defaults option to bootstrap
mgr/cephadm: allow mgr colo if mgr_standby_modules=false
mgr/MgrStandby: add mgr_standby_modules option

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge PR #40893 into master
Sage Weil [Mon, 19 Apr 2021 12:30:11 +0000 (08:30 -0400)]
Merge PR #40893 into master

* refs/pull/40893/head:
mgr/cephadm: don't list non ceph daemons as needing upgrade in upgrade check

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #37730 from thotz/rgwkmsvaultsslsupport
Matt Benjamin [Mon, 19 Apr 2021 11:37:29 +0000 (07:37 -0400)]
Merge pull request #37730 from thotz/rgwkmsvaultsslsupport

rgw: extending existing ssl support for vault KMS

4 years agoMerge pull request #40881 from tchaikov/wip-50393
Kefu Chai [Mon, 19 Apr 2021 10:54:40 +0000 (18:54 +0800)]
Merge pull request #40881 from tchaikov/wip-50393

qa/workunits/mon/test_mon_config_key: use subprocess.run() instead of…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agodoc/rbd/rbd-kubernetes: expose volume expansion functionality 40905/head
Ilya Dryomov [Mon, 19 Apr 2021 09:08:22 +0000 (11:08 +0200)]
doc/rbd/rbd-kubernetes: expose volume expansion functionality

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agodoc/rbd/rbd-kubernetes: imageFeatures is required
Ilya Dryomov [Mon, 19 Apr 2021 07:37:01 +0000 (09:37 +0200)]
doc/rbd/rbd-kubernetes: imageFeatures is required

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agodoc/rbd/rbd-kubernetes: mention KMS config map
Ilya Dryomov [Mon, 19 Apr 2021 07:37:01 +0000 (09:37 +0200)]
doc/rbd/rbd-kubernetes: mention KMS config map

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agodoc/rados/configuration/mon-config-ref: use confval directive 40897/head
Kefu Chai [Mon, 19 Apr 2021 09:24:55 +0000 (17:24 +0800)]
doc/rados/configuration/mon-config-ref:  use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/msgr2: use confval directive
Kefu Chai [Mon, 19 Apr 2021 06:37:34 +0000 (14:37 +0800)]
doc/rados/configuration/msgr2: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/auth-config-ref: use "describe" to define default keyrings
Kefu Chai [Mon, 19 Apr 2021 06:28:02 +0000 (14:28 +0800)]
doc/rados/configuration/auth-config-ref: use "describe" to define default keyrings

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/_ext: use iec_size() instead of filesizeformat()
Kefu Chai [Sun, 18 Apr 2021 00:41:53 +0000 (08:41 +0800)]
doc/_ext: use iec_size() instead of filesizeformat()

to format size options in the same format supported by our C++
strict_iec_cast() parser. so they are more consistent from user's
perspective.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration: merge ms-ref into network-conf-ref
Kefu Chai [Sun, 18 Apr 2021 00:17:31 +0000 (08:17 +0800)]
doc/rados/configuration: merge ms-ref into network-conf-ref

the latter provides more context, and "messenger" is how the networking
infra is implemented.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/ceph-conf: use "describe" to define option types
Kefu Chai [Sat, 17 Apr 2021 23:55:39 +0000 (07:55 +0800)]
doc/rados/configuration/ceph-conf: use "describe" to define option types

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration: fix formatting of code block
Kefu Chai [Sat, 17 Apr 2021 17:52:52 +0000 (01:52 +0800)]
doc/rados/configuration: fix formatting of code block

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc: use "confsec" role for configure section
Kefu Chai [Sat, 17 Apr 2021 16:40:15 +0000 (00:40 +0800)]
doc: use "confsec" role for configure section

and rename confval_section to confsec for shorter name

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/ceph-conf: use confval directive
Kefu Chai [Sat, 17 Apr 2021 16:40:01 +0000 (00:40 +0800)]
doc/rados/configuration/ceph-conf: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc: use ":confval:" role to reference options defined using confval
Kefu Chai [Sat, 17 Apr 2021 16:11:14 +0000 (00:11 +0800)]
doc: use ":confval:" role to reference options defined using confval

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/osd-config-ref: use confval directive
Kefu Chai [Sat, 17 Apr 2021 15:27:15 +0000 (23:27 +0800)]
doc/rados/configuration/osd-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/_ext: render see_also field
Kefu Chai [Sat, 17 Apr 2021 16:39:41 +0000 (00:39 +0800)]
doc/_ext: render see_also field

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/_ext: render more fields in an option
Kefu Chai [Sat, 17 Apr 2021 14:28:57 +0000 (22:28 +0800)]
doc/_ext: render more fields in an option

* print "enum_value" as a list
* print "note" and "warning"

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/mclock-config-ref: use confval directive
Kefu Chai [Sat, 17 Apr 2021 04:46:13 +0000 (12:46 +0800)]
doc/rados/configuration/mclock-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/pool-pg-config-ref: use confval directive
Kefu Chai [Sat, 17 Apr 2021 04:35:32 +0000 (12:35 +0800)]
doc/rados/configuration/pool-pg-config-ref: use confval directive

for defining options.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados/configuration/bluestore-config-ref: use confval directive
Kefu Chai [Sat, 17 Apr 2021 04:15:45 +0000 (12:15 +0800)]
doc/rados/configuration/bluestore-config-ref: use confval directive

for defining options.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40906 from tchaikov/rzarzynski/wip-crimson-internal_client_reques...
Kefu Chai [Mon, 19 Apr 2021 08:45:46 +0000 (16:45 +0800)]
Merge pull request #40906 from tchaikov/rzarzynski/wip-crimson-internal_client_request-cleanup

crimson/osd: various cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #40874 from rhcs-dashboard/fix-cephfs-pool-size-stat
Kefu Chai [Mon, 19 Apr 2021 08:36:04 +0000 (16:36 +0800)]
Merge pull request #40874 from rhcs-dashboard/fix-cephfs-pool-size-stat

mgr/dashboard: filesystem pool size should use stored stat

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
4 years agoMerge pull request #40879 from myoungwon/wip-fix-50299
Kefu Chai [Mon, 19 Apr 2021 08:29:08 +0000 (16:29 +0800)]
Merge pull request #40879 from myoungwon/wip-fix-50299

osd: fix reference leak when ManifestOp is not used

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #40878 from tchaikov/wip-mop-cleanup
Kefu Chai [Mon, 19 Apr 2021 08:26:56 +0000 (16:26 +0800)]
Merge pull request #40878 from tchaikov/wip-mop-cleanup

osd: remove C_SetManifestRefCountDone::mop

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Sam Just <sjust@redhat.com>
4 years agoMerge pull request #40896 from jdurgin/wip-yaml-options-fix
Kefu Chai [Mon, 19 Apr 2021 08:26:04 +0000 (16:26 +0800)]
Merge pull request #40896 from jdurgin/wip-yaml-options-fix

common/options: fix several out of date defaults and options added during yaml conversion

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40884 from majianpeng/osd-watch-remove-unused-code
Kefu Chai [Mon, 19 Apr 2021 08:25:30 +0000 (16:25 +0800)]
Merge pull request #40884 from majianpeng/osd-watch-remove-unused-code

osd/Watch.h: remove unused code.

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoqa/workunits/mon/test_mon_config_key: use subprocess.run() instead of proc.communicate() 40881/head
Kefu Chai [Fri, 16 Apr 2021 04:09:14 +0000 (12:09 +0800)]
qa/workunits/mon/test_mon_config_key: use subprocess.run() instead of proc.communicate()

the loop of proc.communicate() on python3.6, where we always are able to
get something out of stdout and/or stderr PIPEs. and the `stdout` and
`stderr` keep growing until out of memory. and teuthology considers
the command crashed after a while.

Fixes: https://tracker.ceph.com/issues/50393
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: fix formatting around ClientRequest::do_process(). 40906/head
Radoslaw Zarzynski [Thu, 18 Mar 2021 09:39:58 +0000 (09:39 +0000)]
crimson/osd: fix formatting around ClientRequest::do_process().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: fix header inclusion order in ops_executer.h.
Radoslaw Zarzynski [Thu, 18 Mar 2021 12:27:57 +0000 (12:27 +0000)]
crimson/osd: fix header inclusion order in ops_executer.h.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: PG::with_locked_obc() doesn't depend on Op anymore.
Radoslaw Zarzynski [Mon, 15 Mar 2021 19:22:03 +0000 (19:22 +0000)]
crimson/osd: PG::with_locked_obc() doesn't depend on Op anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: drop namespace for arg in PG::with_locked_obc().
Radoslaw Zarzynski [Thu, 18 Mar 2021 09:54:40 +0000 (09:54 +0000)]
crimson/osd: drop namespace for arg in PG::with_locked_obc().

It's unnecessary.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: avoid unneeded obc catching in PG::do_osd_ops_execute().
Radoslaw Zarzynski [Mon, 22 Mar 2021 17:18:20 +0000 (17:18 +0000)]
crimson/osd: avoid unneeded obc catching in PG::do_osd_ops_execute().

Actually, `OpsExecuter` already holds `ObjectContextRef` and even
has a (private till now) getter for `hobject_t` extraction.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #40872 from wjwithagen/wjw-fix-pidfile-order
Kefu Chai [Mon, 19 Apr 2021 07:50:33 +0000 (15:50 +0800)]
Merge pull request #40872 from wjwithagen/wjw-fix-pidfile-order

core: fix compiler warning due to difference in order of struct members

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40749 from tchaikov/wip-python3-spec
Kefu Chai [Mon, 19 Apr 2021 07:49:37 +0000 (15:49 +0800)]
Merge pull request #40749 from tchaikov/wip-python3-spec

ceph.spec: consolidate python3-{setuptools,Cython} builds deps

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
4 years agoMerge pull request #40869 from tchaikov/wip-run-make-check.sh-py3
Kefu Chai [Mon, 19 Apr 2021 07:48:02 +0000 (15:48 +0800)]
Merge pull request #40869 from tchaikov/wip-run-make-check.sh-py3

run-make-check.sh: do not specify -DWITH_PYTHON=3

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocmake: rearrange configure_file() 40869/head
Kefu Chai [Thu, 15 Apr 2021 10:46:11 +0000 (18:46 +0800)]
cmake: rearrange configure_file()

AddCephTest and googletest's CMake scripts also call
find_package(Python3...), but they do not specify the required minor
version of Python3. by default, find_package(Python3...) picks the highest
available python3. so, if we have multiple python3 versions installed in the
system, and the highest python3 version is not the one specified by the
-DWITH_PYTHON3=3.x.y in the cmake command line, we might end up using a
different python3 for the ceph CLI. and even worse, the required python3
package might not available for the picked python3 interpreter found by
googletest. as, in general, only a single python3 has the full access to
prepackaged python3-* shipped by a GNU/Linux distro.

in this change, the configure_file() calls are rearranged to the top of
src/CMakeLists.txt, so they have less chance to use the "polluted" cmake
variable for their subvars.

this change address the test failure where we have, for instance, python3.8
installed on RHEL8/CentOS8, where python3.6 is the python3 which has
the access to the python3-* packages.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agorun-make-check.sh: do not specify -DWITH_PYTHON=3
Kefu Chai [Thu, 15 Apr 2021 09:22:31 +0000 (17:22 +0800)]
run-make-check.sh: do not specify -DWITH_PYTHON=3

should leave it to do_cmake.sh to decide which python3 version to use,
there is case that we have multiple python3 installed, but only one of them
is fully supported by the distro, in the sense that python3-* packages
are packaged for that python3.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40901 from tchaikov/wip-mgr-rook
Kefu Chai [Mon, 19 Apr 2021 03:25:23 +0000 (11:25 +0800)]
Merge pull request #40901 from tchaikov/wip-mgr-rook

cmake: let WITH_MGR_ROOK_CLIENT depend on WITH_MGR

Reviewed-by: Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocmake: let WITH_MGR_ROOK_CLIENT depend on WITH_MGR 40901/head
Kefu Chai [Sun, 18 Apr 2021 13:38:23 +0000 (21:38 +0800)]
cmake: let WITH_MGR_ROOK_CLIENT depend on WITH_MGR

it does not depend on WITH_MGR_DASHBOARD_FRONTEND, which is disabled by
default and is used for enable/disable the inclusion of dashboard
support. while root client is used by orchestrator. so it should depend
on WITH_MGR not WITH_MGR_DASHBOARD_FRONTEND.

this change addresses the regression introduced by
1003f1ffeee0d016e353f2c8c4ba692b5fdc2805

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os/seastore: wire in object_data_handler 40846/head
Samuel Just [Wed, 14 Apr 2021 04:51:50 +0000 (21:51 -0700)]
crimson/os/seastore: wire in object_data_handler

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/object_data_handler: add logic for handling object data
Samuel Just [Sat, 10 Apr 2021 01:04:59 +0000 (18:04 -0700)]
crimson/os/seastore/object_data_handler: add logic for handling object data

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/transaction_manager: expose block size
Samuel Just [Sat, 10 Apr 2021 01:04:15 +0000 (18:04 -0700)]
crimson/os/seastore/transaction_manager: expose block size

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/transaction_manager: add zero region reservation
Samuel Just [Sat, 10 Apr 2021 01:02:57 +0000 (18:02 -0700)]
crimson/os/seastore/transaction_manager: add zero region reservation

Add logical mappings for zero regions for use in reserving contiguous
sections of laddr space.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/transaction_manager: expose find_hole
Samuel Just [Thu, 4 Mar 2021 09:04:22 +0000 (01:04 -0800)]
crimson/os/seastore/transaction_manager: expose find_hole

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/transaction_manager: replace read_extents, expose pins
Samuel Just [Sat, 27 Mar 2021 06:10:08 +0000 (23:10 -0700)]
crimson/os/seastore/transaction_manager: replace read_extents, expose pins

read_extents in all except one case was used to read a known single extent
-- replace those users with read_extent.  store-nbd uses read_extents as
intended, but other users will need to be able to deal with zero mappings.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/.../btree_range_pin: update parent in duplicate
Samuel Just [Mon, 12 Apr 2021 23:11:56 +0000 (16:11 -0700)]
crimson/os/seastore/.../btree_range_pin: update parent in duplicate

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/cache: assert rather than discard in get_extent
Samuel Just [Mon, 12 Apr 2021 23:11:18 +0000 (16:11 -0700)]
crimson/os/seastore/cache: assert rather than discard in get_extent

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: clean up write helper param types
Samuel Just [Sat, 6 Mar 2021 08:03:34 +0000 (08:03 +0000)]
crimson/os/seastore: clean up write helper param types

By convention, we want to pass these by value or rval
ref as their lifetimes need to be managed.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agomonitoring/grafana: Remove erroneous elements in hosts-overview Grafana dashboard 40899/head
Malcolm Holmes [Sat, 17 Apr 2021 22:11:48 +0000 (23:11 +0100)]
monitoring/grafana: Remove erroneous elements in hosts-overview Grafana dashboard

The hosts-overview Grafana dashboard json file contains a repeated element, making
it invalid JSON. Some JSON parsers handle this. However, this prevents Jsonnet
from parsing the dashboard, which prevents the deployment of this dashboard via
Jsonnet.

Fixes: https://tracker.ceph.com/issues/50410
Signed-off-by: Malcolm Holmes <mdh@odoko.co.uk>
4 years agocrimson/mgr: don't report if there is no connection available. 40898/head
Radoslaw Zarzynski [Sat, 17 Apr 2021 17:14:06 +0000 (17:14 +0000)]
crimson/mgr: don't report if there is no connection available.

During a teuthology run [1] following crash happended:

```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-04-08_10:14:11-rados-master-distro-basic-smithi/6028696$ less remote/smithi052/log/ceph-osd.3.log.gz
...
DEBUG 2021-04-08 10:32:58,548 [shard 0] ms - [osd.3(client) v2:172.21.15.52:6813/30889@62168 >> mon.0 v2:172.21.15.52:3300/0] <== #3 === mgrmap(e 4) v1 (1796)
INFO  2021-04-08 10:32:58,549 [shard 0] ms - [osd.3(client) v2:172.21.15.52:6813/30889@62056 >> mgr.4100 v2:172.21.15.52:6800/30259] closing: reset no, replace no
DEBUG 2021-04-08 10:32:58,549 [shard 0] ms - [osd.3(client) v2:172.21.15.52:6813/30889@62056 >> mgr.4100 v2:172.21.15.52:6800/30259] TRIGGER CLOSING, was READY
INFO  2021-04-08 10:32:58,549 [shard 0] ms - [osd.3(client) v2:172.21.15.52:6813/30889@62056 >> mgr.4100 v2:172.21.15.52:6800/30259] execute_ready(): protocol aborted at CLOSING -- std::system_error (error crimson::net:4, read eof)
DEBUG 2021-04-08 10:32:58,549 [shard 0] ms - [osd.3(client) v2:172.21.15.52:6813/30889@62056 >> mgr.4100 v2:172.21.15.52:6800/30259] closed!
Segmentation fault on shard 0.
Backtrace:
  0x000000000151765c
  0x00000000014d9600
  0x00000000014d9902
  0x00000000014d9972
  /lib64/libpthread.so.0+0x0000000000012b1f
  0x0000000000e59cba
  0x00000000014dc8a6
  0x00000000014cdd1c
  0x0000000001503053
  0x000000000149fab7
  0x00000000006e0ef5
  /lib64/libc.so.6+0x00000000000237b2
  0x000000000072a23d
daemon-helper: command crashed with signal 11
```
[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-04-08_10:14:11-rados-master-distro-basic-smithi/6028696/

GDB testifies the `conn` during the execution of `ceph::mgr:report()` was null:

```
(gdb) frame 7
154 in /usr/src/debug/ceph-17.0.0-2935.g4153f8c2.el8.x86_64/src/crimson/mgr/client.cc
(gdb) print conn
$1 = {_b = 0x0, _p = 0x0}
```

Taken altogether with the `mgr.4100 v2:172.21.15.52:6800/30259] closed!`
debug this suggests that a call to `report()` occurred (likely from the
timer) but we were in the middle of the unatomic reconnect sequence:

```cpp
seastar::future<> Client::reconnect()
{
  if (conn) {
    conn->mark_down();
    conn = {};
  }
  // ...
  return seastar::sleep(a_while).then([this] {
    // ...
    conn = msgr.connect(peer, CEPH_ENTITY_TYPE_MGR);
  });
}
```

This commit alters the `mgr::report()` to skip reporting is the `conn`
is unavailable.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge PR #40892 into master
Patrick Donnelly [Sat, 17 Apr 2021 17:10:15 +0000 (10:10 -0700)]
Merge PR #40892 into master

* refs/pull/40892/head:
qa/crontab: reduce pacific priority

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #40890 from tchaikov/wip-yamlize-options-doc
Kefu Chai [Sat, 17 Apr 2021 02:43:58 +0000 (10:43 +0800)]
Merge pull request #40890 from tchaikov/wip-yamlize-options-doc

common/options,doc: extract formatted desc into .yaml.in

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agocommon/options/global.yaml.in: remove unused mon options 40896/head
Josh Durgin [Fri, 16 Apr 2021 22:16:31 +0000 (18:16 -0400)]
common/options/global.yaml.in: remove unused mon options

These were removed in a5f9ca1571cb2b9965d57c9102239aac28a5cc2a

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #40888 into master
Sage Weil [Fri, 16 Apr 2021 22:10:46 +0000 (18:10 -0400)]
Merge PR #40888 into master

* refs/pull/40888/head:
qa/tasks/cephadm: ignore --keep-logs failure
qa/tasks/cephadm: use yaml.dump_all()
qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
qa/tasks/cephadm: tear down clsuter before gathering logs
qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
mgr/cephadm: remove virtual_ip check during scheduling
mgr/orchestrator: orch ls: leave off virtual_ip prefixlen
qa/tasks/cephadm: add wait_for_service
qa/tasks/cephadm: allow skip_monitor_stack=true
qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
qa/tasks/vip: add vip task to allocate virtual IPs
qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case
qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'
qa/tasks/cephadm: let cephadm.shell take string or list

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agocommon/options/y2c.py: add newline after set_enum
Josh Durgin [Fri, 16 Apr 2021 21:29:25 +0000 (17:29 -0400)]
common/options/y2c.py: add newline after set_enum

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
4 years agocommon/options/immutable-object-cache.in.yaml: fix watermark after yaml conversion
Josh Durgin [Fri, 16 Apr 2021 21:15:52 +0000 (17:15 -0400)]
common/options/immutable-object-cache.in.yaml: fix watermark after yaml conversion

This was set higher by 2093c882298e0dc363fc56727b8e1c4a14d0c2f2

Signed-off-by: Josh Durgin <jdurgin@redhat.com>