]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agocrimson/onode-staged-tree: convert ghobject_t to key_hobj_t to print 41772/head
Yingxin Cheng [Wed, 9 Jun 2021 03:06:28 +0000 (11:06 +0800)]
crimson/onode-staged-tree: convert ghobject_t to key_hobj_t to print

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: merge with single-value tail internal node
Yingxin Cheng [Wed, 9 Jun 2021 02:48:43 +0000 (10:48 +0800)]
crimson/onode-staged-tree: merge with single-value tail internal node

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: switch to and test larger node sizes
Yingxin Cheng [Tue, 8 Jun 2021 07:19:33 +0000 (15:19 +0800)]
crimson/onode-staged-tree: switch to and test larger node sizes

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: support unaligned read to leaf_sub_items_t::num_keys
Yingxin Cheng [Tue, 8 Jun 2021 06:11:52 +0000 (14:11 +0800)]
crimson/onode-staged-tree: support unaligned read to leaf_sub_items_t::num_keys

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: add the missing retired validation
Yingxin Cheng [Tue, 8 Jun 2021 06:03:07 +0000 (14:03 +0800)]
crimson/onode-staged-tree: add the missing retired validation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: encode/decode stage_size instead
Yingxin Cheng [Tue, 8 Jun 2021 01:59:32 +0000 (09:59 +0800)]
crimson/onode-staged-tree: encode/decode stage_size instead

end-offset may risk overflow the node_offset_t, but stage_size will not.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: use extent_len_t and node_offset_t correctly
Yingxin Cheng [Tue, 8 Jun 2021 01:55:15 +0000 (09:55 +0800)]
crimson/onode-staged-tree: use extent_len_t and node_offset_t correctly

extent_len_t represents a value that may include the node size, but
node_offset_t cannot and may overflow. Also add validations when
try to cast a larger type to node_offset_t.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: remove unused code
Yingxin Cheng [Tue, 8 Jun 2021 01:42:34 +0000 (09:42 +0800)]
crimson/onode-staged-tree: remove unused code

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: validate node size where possible
Yingxin Cheng [Tue, 8 Jun 2021 01:41:44 +0000 (09:41 +0800)]
crimson/onode-staged-tree: validate node size where possible

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: remove compile-time NODE_BLOCK_SIZE
Yingxin Cheng [Mon, 7 Jun 2021 05:57:13 +0000 (13:57 +0800)]
crimson/onode-staged-tree: remove compile-time NODE_BLOCK_SIZE

Switch to run-time node_size.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: pass node_size to lower node stages
Yingxin Cheng [Mon, 7 Jun 2021 05:15:58 +0000 (13:15 +0800)]
crimson/onode-staged-tree: pass node_size to lower node stages

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: remove compile-time node SIZE
Yingxin Cheng [Mon, 7 Jun 2021 02:45:18 +0000 (10:45 +0800)]
crimson/onode-staged-tree: remove compile-time node SIZE

Switch to run-time node_extent_t::node_size instead.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: cleanup internal_fields_3_t template
Yingxin Cheng [Mon, 7 Jun 2021 02:43:31 +0000 (10:43 +0800)]
crimson/onode-staged-tree: cleanup internal_fields_3_t template

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: pass node_size into node_extent_t
Yingxin Cheng [Thu, 3 Jun 2021 02:28:50 +0000 (10:28 +0800)]
crimson/onode-staged-tree: pass node_size into node_extent_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: extend num_keys_t to support larger nodes
Yingxin Cheng [Wed, 2 Jun 2021 15:44:07 +0000 (23:44 +0800)]
crimson/onode-staged-tree: extend num_keys_t to support larger nodes

Also adjust the accroding layout related unit tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: remove hard-coded node_stage_t::EXTENT_SIZE
Yingxin Cheng [Wed, 2 Jun 2021 08:25:05 +0000 (16:25 +0800)]
crimson/onode-staged-tree: remove hard-coded node_stage_t::EXTENT_SIZE

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: check the maximum tree height
Yingxin Cheng [Wed, 2 Jun 2021 06:07:33 +0000 (14:07 +0800)]
crimson/onode-staged-tree: check the maximum tree height

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: validate safe-split from tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 05:36:11 +0000 (13:36 +0800)]
crimson/onode-staged-tree: validate safe-split from tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: add and use node sizes in tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 05:33:50 +0000 (13:33 +0800)]
crimson/onode-staged-tree: add and use node sizes in tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: implement size upper-bounds to value
Yingxin Cheng [Wed, 2 Jun 2021 05:18:48 +0000 (13:18 +0800)]
crimson/onode-staged-tree: implement size upper-bounds to value

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: implement size upper-bounds to ns/oid
Yingxin Cheng [Wed, 2 Jun 2021 04:47:55 +0000 (12:47 +0800)]
crimson/onode-staged-tree: implement size upper-bounds to ns/oid

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: introduce compile-time tree_conf_t
Yingxin Cheng [Wed, 2 Jun 2021 03:52:33 +0000 (11:52 +0800)]
crimson/onode-staged-tree: introduce compile-time tree_conf_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: cleanup, decouple test_item_t from TestValue
Yingxin Cheng [Mon, 31 May 2021 07:58:30 +0000 (15:58 +0800)]
crimson/onode-staged-tree: cleanup, decouple test_item_t from TestValue

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/seastore: introduce TransactionManager::read_extent(t, offset)
Yingxin Cheng [Mon, 31 May 2021 02:11:01 +0000 (10:11 +0800)]
crimson/seastore: introduce TransactionManager::read_extent(t, offset)

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: correct the node size equation
Yingxin Cheng [Wed, 26 May 2021 06:26:59 +0000 (14:26 +0800)]
crimson/onode-staged-tree: correct the node size equation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: try merge peer if node has one value
Yingxin Cheng [Wed, 26 May 2021 04:45:27 +0000 (12:45 +0800)]
crimson/onode-staged-tree: try merge peer if node has one value

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/onode-staged-tree: cleanup and introduce has_single_value()
Yingxin Cheng [Wed, 26 May 2021 04:44:13 +0000 (12:44 +0800)]
crimson/onode-staged-tree: cleanup and introduce has_single_value()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #38135 from joke-lee/fix-list-parts-show-id-and-displayname
J. Eric Ivancich [Tue, 8 Jun 2021 17:20:05 +0000 (13:20 -0400)]
Merge pull request #38135 from joke-lee/fix-list-parts-show-id-and-displayname

rgw: RGWListMultipart show Owner ID and DisplayName

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #40975 from trociny/wip-50415
J. Eric Ivancich [Tue, 8 Jun 2021 17:19:33 +0000 (13:19 -0400)]
Merge pull request #40975 from trociny/wip-50415

cls/rgw: look for plain entries in non-ascii plain namespace too

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #41065 from pponnuvel/tracker_50554
J. Eric Ivancich [Tue, 8 Jun 2021 17:19:16 +0000 (13:19 -0400)]
Merge pull request #41065 from pponnuvel/tracker_50554

rgw: Improve error message on email id reuse

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
4 years agoMerge pull request #41678 from ivancich/wip-multipart-delete-marker
J. Eric Ivancich [Tue, 8 Jun 2021 17:18:25 +0000 (13:18 -0400)]
Merge pull request #41678 from ivancich/wip-multipart-delete-marker

rgw: completion of multipart upload leaves delete marker

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Matt Benjamin mbenjamin@redhat.com
4 years agoMerge pull request #41760 from zdover23/wip-doc-2021-06-09-dev-guide-reposotory-spelling
zdover23 [Tue, 8 Jun 2021 16:14:25 +0000 (02:14 +1000)]
Merge pull request #41760 from zdover23/wip-doc-2021-06-09-dev-guide-reposotory-spelling

doc/dev: s/reposotory/repository/ (really)

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/dev: s/reposotory/repository/ (really) 41760/head
Zac Dover [Tue, 8 Jun 2021 15:57:13 +0000 (01:57 +1000)]
doc/dev: s/reposotory/repository/ (really)

This corrects the heinous misspelling described in the
substitution expression in the title. This misspelling is
all the more egregious because it appears in a title, and
therefore would be used to create links if it had not been
caught.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #41523 from rhcs-dashboard/fix-50955-master
Ernesto Puerta [Tue, 8 Jun 2021 15:48:25 +0000 (17:48 +0200)]
Merge pull request #41523 from rhcs-dashboard/fix-50955-master

mgr/dashboard: introduce separate front-end component for API docs

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #41703 from tchaikov/wip-options-mgr-mon
Kefu Chai [Tue, 8 Jun 2021 15:33:38 +0000 (23:33 +0800)]
Merge pull request #41703 from tchaikov/wip-options-mgr-mon

common/options: extract mgr and mon options out

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #41444 from Daniel-Pivonka/cephadm-monitoring-ip-bind
Kefu Chai [Tue, 8 Jun 2021 15:32:24 +0000 (23:32 +0800)]
Merge pull request #41444 from Daniel-Pivonka/cephadm-monitoring-ip-bind

mgr/cephadm: allow monitoring stack ip binding and custom ports

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Adam King <adking@redhat.com>
4 years agoMerge pull request #41555 from mgfritch/cephadm-pytest-bootstrap
Kefu Chai [Tue, 8 Jun 2021 15:30:47 +0000 (23:30 +0800)]
Merge pull request #41555 from mgfritch/cephadm-pytest-bootstrap

cephadm: add bootstrap unit tests

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #41567 from ybwang0211/listxattr
Kefu Chai [Tue, 8 Jun 2021 15:30:13 +0000 (23:30 +0800)]
Merge pull request #41567 from ybwang0211/listxattr

doc: Added the explanation of the 'listxattr <obj-name>' and 'getxattr <obj-name> attr' commands

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41724 from tchaikov/wip-cmdparse-cleanup
Kefu Chai [Tue, 8 Jun 2021 15:29:27 +0000 (23:29 +0800)]
Merge pull request #41724 from tchaikov/wip-cmdparse-cleanup

common/cmdparse: do not create temp string unless necessary

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #41656 from rhcs-dashboard/49885-fix-bucket-object-lock-fields
Ernesto Puerta [Tue, 8 Jun 2021 14:53:10 +0000 (16:53 +0200)]
Merge pull request #41656 from rhcs-dashboard/49885-fix-bucket-object-lock-fields

mgr/dashboard: simplify object locking fields in 'Bucket Creation' form

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #41743 from tchaikov/wip-rgw-lua-exception
Kefu Chai [Tue, 8 Jun 2021 13:33:50 +0000 (21:33 +0800)]
Merge pull request #41743 from tchaikov/wip-rgw-lua-exception

rgw/rgw_lua_utils: return error using luaL_error()

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
4 years agoMerge pull request #41689 from tchaikov/wip-vstart-restful
Kefu Chai [Tue, 8 Jun 2021 12:23:40 +0000 (20:23 +0800)]
Merge pull request #41689 from tchaikov/wip-vstart-restful

vstart.sh: disable restful by default

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #41756 from rzarzynski/wip-crimson-monc-fix-races-in-on_session_opened
Kefu Chai [Tue, 8 Jun 2021 11:40:38 +0000 (19:40 +0800)]
Merge pull request #41756 from rzarzynski/wip-crimson-monc-fix-races-in-on_session_opened

crimson/monc: fix races between on_session_opened() and the reset sequence

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41747 from dengchl01/dengchl01-commit
Kefu Chai [Tue, 8 Jun 2021 11:11:35 +0000 (19:11 +0800)]
Merge pull request #41747 from dengchl01/dengchl01-commit

mgr/mgr_module: update docstring of MgrModule.get()

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/monc: fix races between on_session_opened() and the reset sequence. 41756/head
Radoslaw Zarzynski [Tue, 8 Jun 2021 10:33:31 +0000 (10:33 +0000)]
crimson/monc: fix races between on_session_opened() and the reset sequence.

The `active_con` can get invalidated every single time when there is
a preemption point. This includes even the middle connection open
sequence as it's spread across multiple continuations!

Unfortunately, we don't check for `active_con` in the lambdas inside
the `on_session_opened()` method. That was the reason of the following
crash at Sepia [1]:

```
INFO  2021-06-08 09:36:23,992 [shard 0] monc - do_auth_single: connection closed
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4967-g96cdf983/rpm/el8/BUILD/ceph-17.0.0-4967-g96cdf983/src/crimson/mon/MonClient.cc:399:10: runtime error: member access within null pointer of type 'struct Connection'
Segmentation fault on shard 0.
Backtrace:
 0# 0x000055C3C1CA860F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007FAAE1713B20 in /lib64/libpthread.so.0
 4# crimson::mon::Connection::get_conn() in ceph-osd
 5# 0x000055C3C2532DA8 in ceph-osd
 6# 0x000055C3C2535CB5 in ceph-osd
 7# 0x000055C3BBC9FC70 in ceph-osd
 8# 0x000055C3C76FAE5F in ceph-osd
 9# 0x000055C3C77003A0 in ceph-osd
10# 0x000055C3C78B240B in ceph-osd
11# 0x000055C3C740FE8A in ceph-osd
12# 0x000055C3C7419FAE in ceph-osd
13# main in ceph-osd
14# __libc_start_main in /lib64/libc.so.6
15# _start in ceph-osd
Fault at location: 0x98
```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-06-08_09:11:05-rados-master-distro-basic-smithi/6159602

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agomgr/mgr_module: add docstring of MgrModule.get()-mgr_map,modified_config_options... 41747/head
dengchl01 [Tue, 8 Jun 2021 07:42:27 +0000 (15:42 +0800)]
mgr/mgr_module: add docstring of MgrModule.get()-mgr_map,modified_config_options,service_map,mds_metadata,pg_status,osd_pool_stats,mgr_ips,have_local_config_map

Signed-off-by: dengchl01 <dengchl01@inspur.com>
4 years agoMerge pull request #41745 from tchaikov/wip-crimson-osd-cleanup
Kefu Chai [Tue, 8 Jun 2021 05:57:03 +0000 (13:57 +0800)]
Merge pull request #41745 from tchaikov/wip-crimson-osd-cleanup

crimson/osd: do not capture unused variable

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/osd: do not capture unused variable 41745/head
Kefu Chai [Tue, 8 Jun 2021 05:12:22 +0000 (13:12 +0800)]
crimson/osd: do not capture unused variable

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agorgw/rgw_lua*: return unknown field error using luaL_error() 41743/head
Kefu Chai [Tue, 8 Jun 2021 04:44:21 +0000 (12:44 +0800)]
rgw/rgw_lua*: return unknown field error using luaL_error()

it's found on aarch64, the exception is not caught even if we do
catch exactly the same type of thrown exception, and the uncaught
exception ends up with a std::terminate() call. it could be the ABI
mismatch in aarch64, so the C++ runtime failed to find the catch block.

in this change, luaL_error() is used to populate the error to the
caller instead to workaround this issue.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agorgw/rgw_lua_utils: return error using luaL_error()
Kefu Chai [Tue, 8 Jun 2021 04:30:37 +0000 (12:30 +0800)]
rgw/rgw_lua_utils: return error using luaL_error()

it's found on aarch64, the exception is not caught even if we do
catch exactly the same type of thrown exception, and the uncaught
exception ends up with a std::terminate() call. it could be the ABI
mismatch in aarch64, so the C++ runtime failed to find the catch block.

in this change, luaL_error() is used to populate the error to the
caller instead to workaround this issue.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41742 from lmgdlmgd/master
Kefu Chai [Tue, 8 Jun 2021 01:26:43 +0000 (09:26 +0800)]
Merge pull request #41742 from lmgdlmgd/master

mgr/mgr_module: correct docstring of MgrModule.get()

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41736 from rzarzynski/wip-crimson-alienstore-open_collection_dne
Kefu Chai [Tue, 8 Jun 2021 00:59:36 +0000 (08:59 +0800)]
Merge pull request #41736 from rzarzynski/wip-crimson-alienstore-open_collection_dne

crimson/os/alienstore: open_collection() returns nullptr if DNE.

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41730 from tchaikov/wip-fc35-python
Kefu Chai [Tue, 8 Jun 2021 00:58:48 +0000 (08:58 +0800)]
Merge pull request #41730 from tchaikov/wip-fc35-python

do_cmake.sh: use python3.10 for fedora 35

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
4 years agomgr/mgr_module: correct docstring of MgrModule.get() 41742/head
靳登科 [Tue, 8 Jun 2021 00:55:20 +0000 (08:55 +0800)]
mgr/mgr_module: correct docstring of MgrModule.get()

In annotation, one of param of Mgr_module.get() is "osd_crush_map_text", but in PyObject *ActivePyModules::get_python and selftest, the param is "osdmap_crush_map_text", so update annotation from "osd_crush_map_text" to "osdmap_crush_map_text"

Signed-off-by: lmgdlmgd <1105050771@qq.com>
4 years agoMerge pull request #40659 from PepperJo/data_allocate_fraction_v2
Andrew Schoen [Mon, 7 Jun 2021 19:06:52 +0000 (14:06 -0500)]
Merge pull request #40659 from PepperJo/data_allocate_fraction_v2

ceph-volume,python-common: Data allocate fraction

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
4 years agocrimson/os/alienstore: open_collection() returns nullptr if DNE. 41736/head
Radoslaw Zarzynski [Mon, 7 Jun 2021 16:40:06 +0000 (16:40 +0000)]
crimson/os/alienstore: open_collection() returns nullptr if DNE.

This is an AlienStore's counterpart of 9fbd60131b5396beb81ce745c9ccf01388f7e303
which becomes necessary as, since 6b66c30b2331f0d42c8a51446ab04714047084b6,
we call `open_collection()` to check the existence of OSD's superblock.

Without this fix the following crash happens:

```
[rzarzynski@o06 build]$ MDS=0 MGR=0 OSD=1 ../src/vstart.sh -l -n --without-dashboard --crimson
...
crimson-osd: boost/include/boost/smart_ptr/intrusive_ptr.hpp:199: T* boost::intrusive_ptr<T>::operator->() const [with T = ObjectStore::CollectionImpl]: Assertion `px != 0' failed.
Aborting on shard 0.
Backtrace:
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/boost/include/boost/stacktrace/stacktrace.hpp:129
 1# FatalSignal::signaled(int, siginfo_t const*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::operator()(int, siginfo_t*, void*) const at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:41
 3# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:36
 4# 0x00007F8BF534DB30 in /lib64/libpthread.so.0
 5# gsignal in /lib64/libc.so.6
 6# abort in /lib64/libc.so.6
 7# 0x00007F8BF3948B19 in /lib64/libc.so.6
 8# 0x00007F8BF3956DF6 in /lib64/libc.so.6
 9# boost::intrusive_ptr<ObjectStore::CollectionImpl>::operator->() const at /home/rzarzynski/ceph1/build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:200
10# crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}::operator()(boost::intrusive_ptr<ObjectStore::CollectionImpl>) const at /home/rzarzynski/ceph1/build/../src/crimson/os/alienstore/alien_store.cc:214
11# seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > seastar::futurize<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >::invoke<crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:2135
12# auto seastar::futurize_invoke<crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:2167
13# _ZZN7seastar6futureIN5boost13intrusive_ptrIN11ObjectStore14CollectionImplEEEE4thenIZN7crimson2os10AlienStore15open_collectionERK6coll_tEUlS5_E0_NS0_INS2_INS9_19FuturizedCollectionEEEEEEET0_OT_ENUlDpOT_E_clIJS5_EEEDaSN_ at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:1526
14# _ZN7seastar20noncopyable_functionIFNS_6futureIN5boost13intrusive_ptrIN7crimson2os19FuturizedCollectionEEEEEONS3_IN11ObjectStore14CollectionImplEEEEE17direct_vtable_forIZNS1_ISB_E4thenIZNS5_10AlienStore15open_collectionERK6coll_tEUlSB_E0_S8_EET0_OT_EUlDpOT_E_E4callEPKSE_SC_ at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/util/noncopyable_function.hh:125
15# seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>::operator()(boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) const at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/util/noncopyable_function.hh:210
16# seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > std::__invoke_impl<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> >, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(std::__invoke_other, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/bits/invoke.h:60
17# std::__invoke_result<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >::type std::__invoke<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/bits/invoke.h:97
18# std::invoke_result<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >::type std::invoke<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/functional:83
19# auto seastar::internal::future_invoke<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:1213
20# seastar::future<boost::intrusive_ptr<ObjectStore::CollectionImpl> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&)#1}::operator()(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&) const::{lambda()#1}::operator()() const at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:1575
21# void seastar::futurize<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >::satisfy_with_result_of<seastar::future<boost::intrusive_ptr<ObjectStore::CollectionImpl> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&)#1}::operator()(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&) const::{lambda()#1}>(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&) at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:2120
22# seastar::future<boost::intrusive_ptr<ObjectStore::CollectionImpl> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&)#1}::operator()(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&) const at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:1571
23# seastar::continuation<seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<ObjectStore::CollectionImpl> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&)#1}, boost::intrusive_ptr<ObjectStore::CollectionImpl> >::run_and_dispose() at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:771
24# seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /home/rzarzynski/ceph1/build/../src/seastar/src/core/reactor.cc:2237
25# seastar::reactor::run_some_tasks() at /home/rzarzynski/ceph1/build/../src/seastar/src/core/reactor.cc:2646 (discriminator 1)
26# seastar::reactor::run() at /home/rzarzynski/ceph1/build/../src/seastar/src/core/reactor.cc:2805
27# seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) at /home/rzarzynski/ceph1/build/../src/seastar/src/core/app-template.cc:207 (discriminator 7)
28# seastar::app_template::run(int, char**, std::function<seastar::future<int> ()>&&) at /home/rzarzynski/ceph1/build/../src/seastar/src/core/app-template.cc:115 (discriminator 2)
29# main at /home/rzarzynski/ceph1/build/../src/crimson/osd/main.cc:206 (discriminator 1)
30# __libc_start_main in /lib64/libc.so.6
31# _start in /home/rzarzynski/ceph1/build/bin/crimson-osd
Reactor stalled for 33157 ms on shard 0. Backtrace: 0xb14ab 0xa6cd418 0xa6a496d 0xa54fd22 0xa566e3d 0xa56383c 0xa5639fc 0xa566808 0x12b2f 0x3780e 0x21c44 0x21b18 0x2fdf5 0x7cfdf93 0x7b78622 0x7c3fcb7 0x7c1bcb2 0x7c1bdf1 0x7c4004a 0x7d913c7 0x7d875ed 0x7d777d7 0x7d5f626 0x7d47ca4 0x7d47b5a 0x7d5f770 0x7d47931 0x7d9bb74 0xa5998b5 0xa59e1ff 0xa5a3a95 0xa43546c 0xa433331 0x3f01992 0x237c2 0x3c7f9bd
../src/vstart.sh: line 28: 665993 Aborted                 (core dumped) PATH=$CEPH_BIN:$PATH "$@"
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #41733 from rzarzynski/wip-crimson-clientreq-cleanup
Kefu Chai [Mon, 7 Jun 2021 15:24:46 +0000 (23:24 +0800)]
Merge pull request #41733 from rzarzynski/wip-crimson-clientreq-cleanup

crimson/osd: clean up mutability and fix log in ClientRequest

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: fix debug in ClientRequest::do_process(). 41733/head
Radoslaw Zarzynski [Mon, 7 Jun 2021 14:30:14 +0000 (14:30 +0000)]
crimson/osd: fix debug in ClientRequest::do_process().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: drop unneeded lambda mutability in ClientRequest.
Radoslaw Zarzynski [Mon, 7 Jun 2021 14:29:29 +0000 (14:29 +0000)]
crimson/osd: drop unneeded lambda mutability in ClientRequest.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocommon/cmdparse: do not create temp string for writing ostream 41724/head
Kefu Chai [Mon, 7 Jun 2021 06:57:28 +0000 (14:57 +0800)]
common/cmdparse: do not create temp string for writing ostream

ostream is able to handle std::string_view, so no need to create
temporary std::string instance for writing to a std::ostream

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/cmdparse: do not create temp string for cmd_getval()
Kefu Chai [Mon, 7 Jun 2021 06:54:02 +0000 (14:54 +0800)]
common/cmdparse: do not create temp string for cmd_getval()

cmd_getval() consumes string_view, so no need to create a temporary
std::string instance for it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41709 into master
Sage Weil [Mon, 7 Jun 2021 14:03:09 +0000 (10:03 -0400)]
Merge PR #41709 into master

* refs/pull/41709/head:
cephadm: improve is_container_running()

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41509 into master
Sage Weil [Mon, 7 Jun 2021 14:02:52 +0000 (10:02 -0400)]
Merge PR #41509 into master

* refs/pull/41509/head:
common/cmdparse: fix CephBool validation for tell commands
mgr/nfs: fix 'nfs export create' argument order
common/cmdparse: emit proper json
mon/MonCommands: add -- seperator to example
qa/tasks/cephfs/test_nfs: fix export create test
mgr: make mgr commands compat with pre-quincy mon
doc/_ext/ceph_commands: handle non-positional args in docs
mgr: fix reweight-by-utilization cephbool flag
mon/MonCommands: convert some CephChoices to CephBool
mgr/k8sevents: fix help strings
pybind/mgr/mgr_module: fix help desc formatting
mgr/orchestrator: clean up 'orch {daemon add,apply} rgw' args
mgr/orchestrator: add end_positional to a few methods
mgr/orchestrator: reformat a few methods
pybind/ceph_argparse: stop parsing when we run out of positional args
pybind/ceph_argparse: remove dead code
pybind/mgr/mgr_module: infer non-positional args
pybind/mgr/mgr_module: add separator for non-positional args
command/cmdparse: use -- to separate positional from non-positional args
pybind/ceph_argparse: adjust help text for non-positional args
pybind/ceph_argparse: track a 'positional' property on cli args

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41725 from tchaikov/wip-cmake-prom-warning
Kefu Chai [Mon, 7 Jun 2021 13:56:38 +0000 (21:56 +0800)]
Merge pull request #41725 from tchaikov/wip-cmake-prom-warning

cmake: do not print warning in each cmake run

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #41728 from tchaikov/wip-cmake-win32-cleanup
Kefu Chai [Mon, 7 Jun 2021 13:55:39 +0000 (21:55 +0800)]
Merge pull request #41728 from tchaikov/wip-cmake-win32-cleanup

cmake: let vstart-base depend on cython_rados if NOT WIN32

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #41704 from tchaikov/wip-ceph-daemon-ann
Kefu Chai [Mon, 7 Jun 2021 13:46:49 +0000 (21:46 +0800)]
Merge pull request #41704 from tchaikov/wip-ceph-daemon-ann

pybind/ceph_daemon: add missing type annotations

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agoMerge pull request #41732 from rzarzynski/wip-crimson-opsequencer-prev_op-gt-last_unb...
Kefu Chai [Mon, 7 Jun 2021 13:43:52 +0000 (21:43 +0800)]
Merge pull request #41732 from rzarzynski/wip-crimson-opsequencer-prev_op-gt-last_unblocked

crimson/osd: fix assertion failure in OpSequencer on replay.

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41705 from tchaikov/wip-script-ceph-debug-docker
Kefu Chai [Mon, 7 Jun 2021 13:32:02 +0000 (21:32 +0800)]
Merge pull request #41705 from tchaikov/wip-script-ceph-debug-docker

script/ceph-debug-docker: s/x86_64/$(arch)/

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocrimson/osd: fix assertion failure in OpSequencer on replay. 41732/head
Radoslaw Zarzynski [Mon, 7 Jun 2021 13:03:51 +0000 (13:03 +0000)]
crimson/osd: fix assertion failure in OpSequencer on replay.

Commit b5efdc6f1c9563357d7dfd33a8f379053592a215 has unified
the interruption handling among `InternalClientRequest` and
`ClientRequest`. Unfortunately, a call to `maybe_reset()` of
`OpSequencer` has been overlooked and dropped leading to the
`assert(prev_op > last_unblocked)` assertion failure in
`start_op()`.

This was the root cause of the following problem at Sepia:

```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-05-26_12:20:26-rados-master-distro-basic-smithi/6136929$ less ./remote/smithi194/log/ceph-osd.6.log.gz
...
DEBUG 2021-05-26 20:24:53,988 [shard 0] ms - [osd.6(client) v2:172.21.15.194:6804/34047 >> client.4453 172.21.15.67:0/3814935464@37042] <== #1
 === osd_op(client.4453.0:5 12.6 12.7fc1f406 (undecoded) ondisk+write+known_if_redirected e52) v8 (42)
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12.7fc1f406 (undecoded) ondisk+write+known_if_redirected e52) v8): start
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12.7fc1f406 (undecoded) ondisk+write+known_if_redirected e52) v8): in repeat
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12.7fc1f406 (undecoded) ondisk+write+known_if_redirected e52) v8) same_interval_since: 19
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - do_recover_missing check for recovery, 12:602f83fe:::foo:head
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12:602f83fe:::foo:head {write 0~128 in=128b} snapc 0={} ondisk+write+known_if_redirected e52) v8): got obc lock
...
DEBUG 2021-05-26 20:25:21,810 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12:602f83fe:::foo:head {write 0~128 in=1
28b} snapc 0={} ondisk+write+known_if_redirected e52) v8): in repeat
...
DEBUG 2021-05-26 20:25:21,809 [shard 0] osd - should_abort_request operation restart, acting set changed
...
DEBUG 2021-05-26 20:25:21,813 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12:602f83fe:::foo:head {write 0~128 in=128b} snapc 0={} ondisk+write+known_if_redirected e52) v8) same_interval_since: 55
...
DEBUG 2021-05-26 20:25:21,813 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12:602f83fe:::foo:head {write 0~128 in=128b} snapc 0={} ondisk+write+known_if_redirected e52) v8) same_interval_since: 55
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4622-gaa1dc559/rpm/el8/BUILD/ceph-17.0.0-4622-gaa1dc559/src/crimson/osd/osd_operation_sequencer.h:52: seastar::futurize_t<Result> crimson::osd::OpSequencer::start_op(HandleT&, uint64_t, uint64_t, FuncT&&) [with HandleT = crimson::PipelineHandle; FuncT = crimson::interruptible::interruptor<InterruptCond>::wrap_function(Func&&) [with Func = crimson::osd::ClientRequest::start()::<lambda()> mutable::<lambda(Ref<crimson::osd::PG>)> mutable::<lambda()> mutable::<lambda()>; InterruptCond = crimson::osd::IOInterruptCondition]::<lambda()>; Result = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<> >; seastar::futurize_t<Result> = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<> >; uint64_t = long unsigned int]: Assertion `prev_op > last_unblocked' failed.
Aborting on shard 0.
Backtrace:
 0# 0x000055F440239C1F in ceph-osd
 1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
 2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
 3# 0x00007F4788336B20 in /lib64/libpthread.so.0
 4# gsignal in /lib64/libc.so.6
 5# abort in /lib64/libc.so.6
 6# 0x00007F4786931B09 in /lib64/libc.so.6
 7# 0x00007F478693FDE6 in /lib64/libc.so.6
 8# 0x000055F43BA3DD17 in ceph-osd
 9# 0x000055F43BA419A8 in ceph-osd
10# seastar::continuation<seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<boost::intrusive_ptr<crimson::osd::PG> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > >(seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >&&, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&, seastar::future_state<boost::intrusive_ptr<crimson::osd::PG> >&&)#1}, boost::intrusive_ptr<crimson::osd::PG> >::run_and_dispose() in ceph-osd
11# 0x000055F445C5C70F in ceph-osd
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agodo_cmake.sh: use python3.10 for fedora 35 41730/head
Kefu Chai [Mon, 7 Jun 2021 09:50:49 +0000 (17:50 +0800)]
do_cmake.sh: use python3.10 for fedora 35

fedora34 was released in April 2021 with Python3.9. while fedora 35
(still Rawhide) will include python3.10. see https://fedoraproject.org/wiki/Releases/35/ChangeSet#Python_3.10

so update the script accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodo_cmake.sh: refactor code to set PYBUILD
Kefu Chai [Mon, 7 Jun 2021 09:49:24 +0000 (17:49 +0800)]
do_cmake.sh: refactor code to set PYBUILD

so it is more consistent with rhel/centos part.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: let vstart-base depend on cython_rados if NOT WIN32 41728/head
Kefu Chai [Mon, 7 Jun 2021 08:00:23 +0000 (16:00 +0800)]
cmake: let vstart-base depend on cython_rados if NOT WIN32

to disable this custom target if NOT WIN32 without explanations is difficult
to maintain, so add a comment, and limit the scope to just
`cython_rados`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: do not print warning in each cmake run 41725/head
Kefu Chai [Mon, 7 Jun 2021 07:09:42 +0000 (15:09 +0800)]
cmake: do not print warning in each cmake run

cmake prints the warning of

> run-promtool-unittests is skipped due to missing promtool

everytime if the test is skipped per the test. but we only need to
get this message the first time cmake is executed. more importantly,
the value of `PROMTOOL_EXECUTABLE` is cached. so the new error message
does not provide updated information, so just stop doing so if the
cached variable is found.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41695 from tchaikov/wip-crimson-net-move
Kefu Chai [Mon, 7 Jun 2021 01:52:36 +0000 (09:52 +0800)]
Merge pull request #41695 from tchaikov/wip-crimson-net-move

crimson/net: move from out_q into sent queue

Reviewed-by: Amnon Hanuhov <ahanukov@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #41708 from tchaikov/wip-seastore-open-coll
Kefu Chai [Sun, 6 Jun 2021 01:45:16 +0000 (09:45 +0800)]
Merge pull request #41708 from tchaikov/wip-seastore-open-coll

crimson/os/seastore: open_collection() returns nullptr if DNE

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge PR #41665 into master
Sage Weil [Sat, 5 Jun 2021 20:43:36 +0000 (16:43 -0400)]
Merge PR #41665 into master

* refs/pull/41665/head:
mgr/cephadm:fix alerts sent to wrong URL

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
4 years agocephadm: improve is_container_running() 41709/head
Sage Weil [Sat, 5 Jun 2021 15:05:48 +0000 (10:05 -0500)]
cephadm: improve is_container_running()

The 'podman ps' command sporatically exits with 125, I suspect due
to some race/bug in podman itself.

However, our goal here is just to check if a specific container
is running.  Use inspect for that instead--it's even (a bit) faster.

Fixes: https://tracker.ceph.com/issues/51109
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #40652 from ronen-fr/wip-ronenf-cscrub-class
Kefu Chai [Sat, 5 Jun 2021 16:06:32 +0000 (00:06 +0800)]
Merge pull request #40652 from ronen-fr/wip-ronenf-cscrub-class

osd/scrub: modify "classic" OSD scrub state-machine to support Crimson

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agocommon/cmdparse: fix CephBool validation for tell commands 41509/head
Sage Weil [Sat, 5 Jun 2021 15:23:35 +0000 (11:23 -0400)]
common/cmdparse: fix CephBool validation for tell commands

The tell/asok validation uses a different validation path; make it work
for CephBool arguments.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #41154 from rzarzynski/wip-global-backtrace-bug-50647
Kefu Chai [Sat, 5 Jun 2021 13:41:00 +0000 (21:41 +0800)]
Merge pull request #41154 from rzarzynski/wip-global-backtrace-bug-50647

global: fault handlers cope with simultaneous faults now.

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41604 from t-msn/wip-51030
Kefu Chai [Sat, 5 Jun 2021 13:33:00 +0000 (21:33 +0800)]
Merge pull request #41604 from t-msn/wip-51030

osd/ECBackend: Fix null pointer dereference when enabling jaeger tracing

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
4 years agoMerge pull request #41501 from aclamk/wip-bluefs-safer-flush
Kefu Chai [Sat, 5 Jun 2021 13:26:04 +0000 (21:26 +0800)]
Merge pull request #41501 from aclamk/wip-bluefs-safer-flush

os/bluestore: Remove possibility of replay log and file inconsistency

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #41506 from ceph/wip-cv-batch-fixes
Kefu Chai [Sat, 5 Jun 2021 13:23:13 +0000 (21:23 +0800)]
Merge pull request #41506 from ceph/wip-cv-batch-fixes

ceph-volume: fix batch report and respect ceph.conf config values

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
4 years agoMerge pull request #41688 from tchaikov/wip-debian-rook
Kefu Chai [Sat, 5 Jun 2021 13:17:24 +0000 (21:17 +0800)]
Merge pull request #41688 from tchaikov/wip-debian-rook

debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-roo…

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agocrimson/os/seastore: open_collection() returns nullptr if DNE 41708/head
Kefu Chai [Sat, 5 Jun 2021 09:39:25 +0000 (17:39 +0800)]
crimson/os/seastore: open_collection() returns nullptr if DNE

we check for the existence of meta collection by trying to open it,
if it exists, we continue check for the superblock stored in it, if
the superblock does not exist or corrupted, we consider it as a failure.

before this change, open_collection() always return a valud Collection
even if the store does not have the collection with specified cid. this
behavior could be misleading in the use case above.

after this change, open_collection() looks up the collections stored in
root collection node for the specfied cid, and return nullptr if it does
not exist.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os/seastore: use structured binding
Kefu Chai [Sat, 5 Jun 2021 09:22:35 +0000 (17:22 +0800)]
crimson/os/seastore: use structured binding

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoscript/ceph-debug-docker: s/x86_64/$(arch)/ 41705/head
Kefu Chai [Sat, 5 Jun 2021 04:19:41 +0000 (12:19 +0800)]
script/ceph-debug-docker: s/x86_64/$(arch)/

try to avoid hardwiring to a certain architecture if possible.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind: add tox.ini 41704/head
Kefu Chai [Sat, 5 Jun 2021 03:53:50 +0000 (11:53 +0800)]
pybind: add tox.ini

also add ceph_daemon to mypy.ini

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/ceph_daemon: add missing type annotations
Kefu Chai [Sat, 5 Jun 2021 04:08:00 +0000 (12:08 +0800)]
pybind/ceph_daemon: add missing type annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/ceph_daemon: correct type annotation of admin_socket()
Kefu Chai [Sat, 5 Jun 2021 03:53:03 +0000 (11:53 +0800)]
pybind/ceph_daemon: correct type annotation of admin_socket()

we always pass a list of string to it, but since it does not assume the
container type of string, just specify Sequence[str].

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/ceph_daemon: fix the format string in Termsize.__repr__()
Kefu Chai [Sat, 5 Jun 2021 03:40:18 +0000 (11:40 +0800)]
pybind/ceph_daemon: fix the format string in Termsize.__repr__()

we intended to print the class name usign "%s".

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/ceph_daemon: do not handle get_command_descriptions in admin_socket()
Kefu Chai [Sat, 5 Jun 2021 03:38:17 +0000 (11:38 +0800)]
pybind/ceph_daemon: do not handle get_command_descriptions in admin_socket()

we never pass a single string to admin_socket() as its 2nd parameter.
instead, we always pass a list of string. so no need to handle it as a
special case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/options: extract mgr and mon options out 41703/head
Kefu Chai [Sat, 5 Jun 2021 02:43:02 +0000 (10:43 +0800)]
common/options: extract mgr and mon options out

this change is a part of a sequence of changes like
9b5e54d996a771f63b97620d4640c08f6406d843 and
dff996d9e24c6259fffce7e704e47e066c367b6a. because we need to audit
the consumers of an option before extracting it to the dedicated
.yaml.in file, it takes considerable time to get this done.

so let's do this piecemeal.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41581 from tchaikov/wip-options-mgr-mon
Kefu Chai [Sat, 5 Jun 2021 02:06:07 +0000 (10:06 +0800)]
Merge pull request #41581 from tchaikov/wip-options-mgr-mon

common/options: extract mgr and mon options out

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #40073 from jmolmo/delete_service_causes_osd_removal
Kefu Chai [Sat, 5 Jun 2021 00:44:42 +0000 (08:44 +0800)]
Merge pull request #40073 from jmolmo/delete_service_causes_osd_removal

mgr/cephadm: Warn about OSDs to remove manually when deleting an OSD service

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
4 years agomgr/nfs: fix 'nfs export create' argument order
Sage Weil [Fri, 4 Jun 2021 17:35:46 +0000 (13:35 -0400)]
mgr/nfs: fix 'nfs export create' argument order

Put path before --readonly so that it can still be positional.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agocommon/cmdparse: emit proper json
Sage Weil [Tue, 1 Jun 2021 20:55:44 +0000 (16:55 -0400)]
common/cmdparse: emit proper json

Instead of '"req": "false"', emit '"req": false'.  Same for conditional.

Luckily, the clients don't really care about this change, as
ceph_argparse.py argdesc interpets the JSON like so:

            self.req = req in (True, 'True', 'true')
            self.positional = positional in (True, 'True', 'true')

Clean up command definitions to use lowercase 'false', but tolerate
both for backward compat during upgrade and to tolerate future errors.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomon/MonCommands: add -- seperator to example
Sage Weil [Tue, 1 Jun 2021 18:43:47 +0000 (14:43 -0400)]
mon/MonCommands: add -- seperator to example

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/tasks/cephfs/test_nfs: fix export create test
Sage Weil [Tue, 1 Jun 2021 17:39:03 +0000 (13:39 -0400)]
qa/tasks/cephfs/test_nfs: fix export create test

Everything after --readonly is non-positional.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr: make mgr commands compat with pre-quincy mon
Sage Weil [Sat, 29 May 2021 02:19:27 +0000 (22:19 -0400)]
mgr: make mgr commands compat with pre-quincy mon

If the mon is not yet quincy, do not share the quincy-ism of
'positional=false'.

Signed-off-by: Sage Weil <sage@newdream.net>