crimson/os/seastore: limit the max size of objects' ns and oid

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

test/crimson/seastore: add set/get attrs test

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

test/crimson/seastore: add write/read_meta tests

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/os/seastore: add write/read_meta methods for seastore

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/os/seastore: add get/set attr methods

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #40942 from athanatos/sjust/wip-retired-placeholder

crimson/os/seastore: track retired extents until in-progress transactions complete

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40956 from rzarzynski/wip-crimson-depends-on-binutils

ceph.spec.in: crimson depends on binutils b/c of addr2line.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40970 from neha-ojha/wip-delete-some

osd: remove references to _delete_some()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40971 from tchaikov/wip-common-option-osd_erasure_code_plugins

common/options: set default variable of osd_erasure_code_plugins

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40964 from tchaikov/wip-disable-rook-mgr-module

vstart.sh: disable "rook" mgr module by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #40580 from anthonyeleven/anthonyeleven/clean-up-cephfs-standby

doc/cephfs: refine cephfs/standby

Reviewed-by: Zac Dover <zac.dover@gmail.com>

crimson/os/seastore/cached_extent: fix maybe_generate_relative

The only case we really care about here is that record relative addresses
for initial_pending extents need to be turned into block relative extents.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: retain placeholders for retired, uncached extents

We need to track extents retired without first being in cache.  Create
RetiredExtentPlaceholder extent type for conflict detection on those
cases.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: speed up mount with debugging enabled

Logging allocation information really slows down initial mount.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/cache: check cached extents for is_retired

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/cache: ensure retired extents remain until transactions complete

Adds a structure to ensure that retired extents remain in the index until
any transactions which might reference them complete.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore_types: fix NULL_SEG_OFF declaration

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #40974 from tchaikov/wip-doc-more-conf-vals

doc/rados/configuration/mon-config-ref: use confval directive

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40725 from sseshasa/wip-mclock-internal-doc

doc/dev: Add internal documentation for mclock scheduler

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/_ext/ceph_confval: print out option name if it does not render

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/mon-config-ref: link by ref instead of relative hyperlink

the latter is more fragile.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/mon-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration: add missing options

and link to them when appropriate

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: add description to osd_op_num_shards*

so they can be rendered

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40968 from rzarzynski/wip-crimson-only-fix-fwd-in-interruptor

crimson/common: fix forwarding in non_futurized_call_with_interruption().

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40904 from inspur-wyq/patch-3

doc/radosgw/multisite.rst: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40932 from lyarwood/pybind_rados_connect_timeout

pybind/rados: Document that timeout arg is ignored by connect

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #40938 into master

* refs/pull/40938/head:
mgr/cephadm: less noise about osd specs
mgr/cephadm: kick serve loop when adding/removing labels
mgr/cephadm: do not place osds on _no_schedule hosts
doc/cephadm: document _no_schedule label
mgr/cephadm: fix 'orch ls' count to reflect schedulable hosts
mgr/cephadm: do not schedule on _no_schedule hosts

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

common/options: set default variable of osd_erasure_code_plugins

5ddda38da4f introduced a regression which failed to substitude
osd_erasure_code_plugins's default value with the the corresponding
CMake variable

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: less noise about osd specs

We don't want this message every time around the serve() loop

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: kick serve loop when adding/removing labels

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: do not place osds on _no_schedule hosts

Signed-off-by: Sage Weil <sage@newdream.net>

doc/cephadm: document _no_schedule label

Signed-off-by: Sage Weil <sage@newdream.net>

osd: remove references to _delete_some()

It was replaced by do_delete_work() in b927419dd40987728684cc773e85130d29d06c39

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40969 from zdover23/wip-doc-cephadm-osd-filters-colons-2021-apr-21

doc/cephadm: osd.rst -- removing colons

Reviewed-by: Sage Weil <sweil@redhat.com>

doc/cephadm: osd.rst -- removing colons

This PR removes colons that weren't necessary.
This is a companion PR to PR#40996.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #40705 from orozery/libcryptsetup-return-checks

librbd/crypto: fix bad return checks from libcryptsetup

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #40966 into master

* refs/pull/40966/head:
doc/cephadm: osd: rewrite "additional opts"

Reviewed-by: Sage Weil <sage@redhat.com>

crimson/common: fix forwarding in non_futurized_call_with_interruption().

This commit fixes a memory corruption problem found on the intersection
of `seastar::thread` and `interruptor` by a teuthology job [1].
It turns out to be caused by a use-after-free which happens due to
an issue with type deduction in `non_futurized_call_with_interruption`.
However, it might be beneficial to audit other members of the family too.

If `Result` is deduced to be a non-reference type (i.e. the `Func`
returns e.g. `ceph::bufferlist`), `std::forward<Result>` forwards
it as a r-value reference (`Result&&`). `decltype(auto)` makes this
reference the return type of the entire function which isn't what
we want.

Debugging
=========

Instrumentation
---------------
```diff
--- a/src/crimson/osd/ops_executer.cc
+++ b/src/crimson/osd/ops_executer.cc
@@ -28,6 +28,21 @@ namespace {

 namespace crimson::osd {

+struct scope_dbgr {
+  scope_dbgr() {
+    logger().debug("{}:{} on {}", __func__, __LINE__, (void*)this);
+  }
+  scope_dbgr(const scope_dbgr& src) {
+    logger().debug("{}:{} copy on {}; src {}", __func__, __LINE__, (void*)this, (void*)&src);
+  }
+  scope_dbgr(scope_dbgr&& src) noexcept {
+    logger().debug("{}:{} move on {}; src {}", __func__, __LINE__, (void*)this, (void*)&src);
+  }
+  ~scope_dbgr() {
+    logger().debug("{}:{} on {}", __func__, __LINE__, (void*)this);
+  }
+};
+
 OpsExecuter::call_ierrorator::future<> OpsExecuter::do_op_call(OSDOp& osd_op)
 {
   std::string cname, mname;
@@ -84,14 +99,16 @@ OpsExecuter::call_ierrorator::future<> OpsExecuter::do_op_call(OSDOp& osd_op)
       ceph::bufferlist outdata;
       auto cls_context = reinterpret_cast<cls_method_context_t>(this);
       const auto ret = method->exec(cls_context, indata, outdata);
-      return std::make_pair(ret, std::move(outdata));
+      logger().debug("do_ops_call: outdata just after exec: {}", outdata);
+      return std::make_tuple(ret, std::move(outdata), scope_dbgr{});
     }
   ).then_interruptible(
     [this, prev_rd, prev_wr, &osd_op, flags]
     (auto outcome) -> call_errorator::future<> {
-      auto& [ret, outdata] = outcome;
+      auto& [ret, outdata, dbgr] = outcome;
       osd_op.rval = ret;
-
+      logger().debug("do_ops_call: outdata just after then_interruptible: {}",
+                     outdata);
       logger().debug("do_op_call: method returned ret={}, outdata.length()={}"
                      " while num_read={}, num_write={}",
                      ret, outdata.length(), num_read, num_write);
```

After the fix
-------------
```
DEBUG 2021-04-21 15:32:30,012 [shard 0] bluestore - get_attr
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - do_ops_call: outdata just after exec: buffer::list(len=18,
        buffer::ptr(0~18 0x6000000ef000 in raw 0x6000000ef000 len 4000 nref 1)
)
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:33 on 0x60000085feb3
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x60000085ff00; src 0x60000085feb3
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x60000085feb3
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x60000085fed0; src 0x60000085ff00
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x60000085ff00
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x60000085ff40; src 0x60000085fed0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x60000085fed0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x60000015a740; src 0x60000085ff40
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x60000085ff40
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x7ffc4d018df0; src 0x60000015a740
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x60000015a740
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x6000005196d8; src 0x7ffc4d018df0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x7ffc4d018df0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x7ffc4d018df0; src 0x6000005196d8
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x6000005196d8
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x7ffc4d018db0; src 0x7ffc4d018df0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x7ffc4d018df0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x600000807b58; src 0x7ffc4d018db0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x7ffc4d018db0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - scope_dbgr:39 move on 0x7ffc4d018d30; src 0x600000807b58
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - do_ops_call: outdata just after then_interruptible: buffer::list(len=18,
        buffer::ptr(0~18 0x6000000ef000 in raw 0x6000000ef000 len 4000 nref 1)
)
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - do_op_call: method returned ret=0, outdata.length()=18 while num_read=1, num_write=0
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x7ffc4d018d30
DEBUG 2021-04-21 15:32:30,012 [shard 0] osd - ~scope_dbgr:42 on 0x600000807b58
```

Before
------
```
DEBUG 2021-04-21 16:06:24,354 [shard 0] bluestore - get_attr
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - do_ops_call: outdata just after exec: buffer::list(len=18,
        buffer::ptr(0~18 0x6000000f7000 in raw 0x6000000f7000 len 4000 nref 1)
)
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:33 on 0x60000085fe84
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x60000085fed0; src 0x60000085fe84
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x60000085fe84
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x60000085fed0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x60000085fea0; src 0x60000085fed0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x60000085ff40; src 0x60000085fea0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x60000085fea0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x6000008360b0; src 0x60000085ff40
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x60000085ff40
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x7ffe48b3ee00; src 0x6000008360b0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x6000008360b0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x600000801a38; src 0x7ffe48b3ee00
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x7ffe48b3ee00
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x7ffe48b3ee00; src 0x600000801a38
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x600000801a38
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x7ffe48b3edc0; src 0x7ffe48b3ee00
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x7ffe48b3ee00
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x600000807bf8; src 0x7ffe48b3edc0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x7ffe48b3edc0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - scope_dbgr:39 move on 0x7ffe48b3ed40; src 0x600000807bf8
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - do_ops_call: outdata just after then_interruptible: buffer::list(len=18,
        buffer:ptr(0~18 no raw)
)
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - do_op_call: method returned ret=0, outdata.length()=18 while num_read=1, num_write=0
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x7ffe48b3ed40
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - ~scope_dbgr:42 on 0x600000807bf8
DEBUG 2021-04-21 16:06:24,354 [shard 0] osd - do_osd_ops: osd_op(client.4131.0:4 1.e 1:70fc7f5f:test-rados-api-o06-82166-2::foo:head {call lock.get_info in=31b out=18b}
 snapc 0={} ondisk+read+known_if_redirected e6) v8 - object 1:70fc7f5f:test-rados-api-o06-82166-2::foo:head all operations successful
Reactor stalled for 521 ms on shard 0.
Backtrace:
  0x00000000017101b8
  0x00000000016d7960
  0x00000000016d7edc
  0x00000000016d810f
  /lib64/libpthread.so.0+0x0000000000012b2f
  0x00000000016b563e
  0x00000000016b801c
  0x00000000016b8504
  0x0000000000d68120
```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-04-19_12:11:51-rados-master-distro-basic-smithi/6058778/

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/cephadm: osd: rewrite "additional opts"

This PR rewrites the "Additional Options"
subsection of the OSD chapter of the cephadm
guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #40914 from zdover23/wip-doc-cephadm-osd-advanced-up-to-filters-2021-apr-18

doc/cephadm: rewrite "advanced osd s. specs"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

vstart.sh: disable "rook" mgr module by default

as we don't have "jsonpatch" installed for running "make check", "rook"
module always fails to load, and the error message when mgr is
unable to load is misleading and distracting. let's just disabled it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/PyModuleRegistry: use vector<> when appropriate

no need to use a set<string> for storing module name, they are
inherently unique.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev: Add internal documentation for mclock scheduler

The document details the refinements to mclock scheduler and presents the
results of a comparison study performed between the mclock scheduler and
the WPQ scheduler.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

librbd/crypto: fix bad return checks from libcryptsetup

This commit fixes the return checks for libcryptsetup functions
that may return non-zero success codes.

Fixes: https://tracker.ceph.com/issues/50461
Signed-off-by: Or Ozeri <oro@il.ibm.com>

ceph.spec.in: crimson depends on binutils b/c of addr2line.

This small utility is needed due to the backtrace processing.
See: d3dcf510c35228a525311732fa5f7385c4a736ba.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #40935 from TRYTOBE8TME/wip-rgw-bootstrap-fix

test/rgw: Removing unrecognized option from bootstrap file

Merge pull request #40953 from tchaikov/wip-doc-confval

common/options,doc/rados/configuration: extract crimson options, use confval directive

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/conf.py: use glob.glob() to find .yaml.in files

improve the maintainability of .yaml.in files.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: use file(GLOB ..) to find .yaml.in files

instead of hardcode the option file names, use file(GLOB ..) to find
and collect them instead, for better maintainability.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/optinos: extract crimson options out

prepare for the change to split the options by services

in future, osd will have its own osd.yaml.in, while crimson will consume
both osd.yaml.in and crimson.yaml.in.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/mon-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/general-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/osd-config-ref: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40915 from rzarzynski/wip-crimson-backtrace-3

crimson: make backtraces more human-readable with boost::stacktrace and addr2line

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40939 from tchaikov/wip-doc-confval

doc/rados/configuration/osd-config-ref:  use confval directive

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

src/test: Removing unrecognized option from bootstrap file

Removing '--distribute' option from a command in bootstrap file which seemed to be the reason of failure in:
http://qa-proxy.ceph.com/teuthology/dang-2021-04-19_12:28:43-rgw-wip-dang-zipper-sysobj-distro-basic-smithi/6059513/teuthology.log

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>

Merge PR #39742 into master

* refs/pull/39742/head:
client: make Inode to inherit from RefCountedObject
client: minor cleanup to Inode class to simplify the code
client: minor cleanup to Inode related code

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40191 from dang/wip-dang-zipper-sysobj

RGW Zipper - Remove rgw_pool from API

crimson/os/seastore/cache: factor out remove_from_dirty

Signed-off-by: Samuel Just <sjust@redhat.com>

mgr/cephadm: fix 'orch ls' count to reflect schedulable hosts

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: do not schedule on _no_schedule hosts

Reuse and rename existing helper to return schedulable hosts.

Signed-off-by: Sage Weil <sage@newdream.net>

doc/rados/configuration/osd-config-ref:  use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/rados: Document that timeout arg is ignored by connect

While this argument is accepted it has always been ignored. To avoid
confusion it should at least be called out in the documentation.

Signed-off-by: Lee Yarwood <lyarwood@redhat.com>

Merge pull request #40928 from idryomov/wip-gid-reclaim-notes

doc: 14.2.20, 15.2.11 and 16.2.1 releases and notes (CVE-2021-20288)

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc/releases/index: update for 14.2.20, 15.2.11 and 16.2.1

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #40877 from mgfritch/doc-rgw-python3-print

doc/rgw: update to python3 print syntax

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/os/seastore/cache: swap dirty_from users to get_dirty_from()

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore: route all transaction creations through cache

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #40846 from athanatos/sjust/wip-object-data

seastore: initial object data support

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #40911 from tchaikov/wip-doc-confval

doc/rados/configuration/auth-config-ref: drop section of keyrings

Reviewed-by: Sage Weil <sage@redhat.com>

doc/cephadm: rewrite "advanced osd s. specs"

This PR improves the readbility and elegance of
the "Advanced OSD Service Specifications" section
of the OSD chapter of the cephadm guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

crimson/common: do backtrace via boost::stackstrace and addr2line.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/common: don't use Seastar's handle_signal() for fatal signals.

The `handle_signal()` way requires going through the reactor, and
thus is unsuitable for situations when it's no longer operational.
SIGSEGV generated as a result of actual fault (not `kill()`) is
the intended audience of this change.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/common: let's dump the core on fatal signal.

Removing the call to engine's `exit()` should allow the singnal
handler to exit. In the case of `SIGSEGV` this would return back
to the problematic instruction, and thus retrigger the fault.
As our handler already restores `SIG_DFL`, it's expected to get
a core dump.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: handle fatal signals

seastar sets the signal handler for SIGSEGV and SIGABRT, and its signal
handler prints the stacktrace, but the the stack frames are adresses,
which are not human readable without the help of addr2line.

since crimson is linked with -rdynamic option, we have the symbols added
to the dynamic symbol table already. let print out the symbolized
addresses instead using our own stacktrace utility.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6e26243b6db49bb9813f8b8aeade68da07dc6065)

 Conflicts:
src/crimson/osd/main.cc

doc/man/8/ceph-authtool: fix the indent of option

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40899 from malcolmholmes/malcolmholmes/fix-json-dashboard-error

mgr/Dashboard: Remove erroneous elements in hosts-overview Grafana dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: p-se <NOT@FOUND>

Merge pull request #40898 from rzarzynski/wip-crimson-only-mgrclt-noconn-report

crimson/mgr: don't report if there is no connection available.

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/mon-config-ref:  use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/auth-config-ref: drop section of keyrings

as per Sage, the caps are out of date (should be based on profiles) and
the keyring locations aren't useful information

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40897 from tchaikov/wip-doc-more-conf-vals

doc/rados/configuration: use confval directive to define options.

Reviewed-by: Reviewed-by: Sage Weil <sage@redhat.com>

RGW Zipper - Remove rgw_pool/sysobj from API

rgw_pool and sysobj are part of the RADOS backend, and do not belong in
the API.  Instead, pull the objects using them into the API.  These are:
RGWOIDCProvider, RGWRole, and a new object LuaScript.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #40905 from idryomov/wip-rbd-kms-config-map-doc

doc/rbd/rbd-kubernetes: update for recent ceph-csi versions

Reviewed-by: Madhu Rajanna <madhupr007@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #40378 from zdover23/wip-doc-cephadm-osd-declarative-state-rewrite-2021-Mar-25

doc/cephadm: rewrite "delcarative state" in osd.rst

Reviewed-by: Sage Weil <sweil@redhat.com>

Merge PR #40817 into master

* refs/pull/40817/head:
mgr/MgrStandby: fix config observer
mgr/MgrStandby: respawn if mgr_standby_modules changes
qa/tasks/mgr/test_dashboard: skip test_standby if mgr_standby_modules=false
qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
qa/suites/rados/cephadm/smoke-singlehost: test --single-host-defaults
cephadm: add --single-host-defaults option to bootstrap
mgr/cephadm: allow mgr colo if mgr_standby_modules=false
mgr/MgrStandby: add mgr_standby_modules option

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #40893 into master

* refs/pull/40893/head:
mgr/cephadm: don't list non ceph daemons as needing upgrade in upgrade check

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #37730 from thotz/rgwkmsvaultsslsupport

rgw: extending existing ssl support for vault KMS

Merge pull request #40881 from tchaikov/wip-50393

qa/workunits/mon/test_mon_config_key: use subprocess.run() instead of…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/rbd/rbd-kubernetes: expose volume expansion functionality

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

doc/rbd/rbd-kubernetes: imageFeatures is required

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

doc/rbd/rbd-kubernetes: mention KMS config map

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

doc/rados/configuration/mon-config-ref:  use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/msgr2: use confval directive

for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/auth-config-ref: use "describe" to define default keyrings

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/_ext: use iec_size() instead of filesizeformat()

to format size options in the same format supported by our C++
strict_iec_cast() parser. so they are more consistent from user's
perspective.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration: merge ms-ref into network-conf-ref

the latter provides more context, and "messenger" is how the networking
infra is implemented.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration/ceph-conf: use "describe" to define option types

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/configuration: fix formatting of code block

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: use "confsec" role for configure section

and rename confval_section to confsec for shorter name

Signed-off-by: Kefu Chai <kchai@redhat.com>