Abutalib Aghayev [Thu, 10 Jun 2021 19:59:45 +0000 (15:59 -0400)]
os/bluestore: Fix the size of the block in the Allocator base class to avoid
the confusing log message about the block size.
Signed-off-by: Abutalib Aghayev <agayev@psu.edu>
Kefu Chai [Tue, 8 Jun 2021 13:33:50 +0000 (21:33 +0800)]
Merge pull request #41743 from tchaikov/wip-rgw-lua-exception
rgw/rgw_lua_utils: return error using luaL_error()
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 12:23:40 +0000 (20:23 +0800)]
Merge pull request #41689 from tchaikov/wip-vstart-restful
vstart.sh: disable restful by default
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 11:40:38 +0000 (19:40 +0800)]
Merge pull request #41756 from rzarzynski/wip-crimson-monc-fix-races-in-on_session_opened
crimson/monc: fix races between on_session_opened() and the reset sequence
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 11:11:35 +0000 (19:11 +0800)]
Merge pull request #41747 from dengchl01/dengchl01-commit
mgr/mgr_module: update docstring of MgrModule.get()
Reviewed-by: Kefu Chai <kchai@redhat.com>
Radoslaw Zarzynski [Tue, 8 Jun 2021 10:33:31 +0000 (10:33 +0000)]
crimson/monc: fix races between on_session_opened() and the reset sequence.
The `active_con` can get invalidated every single time when there is
a preemption point. This includes even the middle connection open
sequence as it's spread across multiple continuations!
Unfortunately, we don't check for `active_con` in the lambdas inside
the `on_session_opened()` method. That was the reason of the following
crash at Sepia [1]:
```
INFO 2021-06-08 09:36:23,992 [shard 0] monc - do_auth_single: connection closed
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/
17.0.0-4967-g96cdf983/rpm/el8/BUILD/
ceph-17.0.0-4967-g96cdf983/src/crimson/mon/MonClient.cc:399:10: runtime error: member access within null pointer of type 'struct Connection'
Segmentation fault on shard 0.
Backtrace:
0# 0x000055C3C1CA860F in ceph-osd
1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
3# 0x00007FAAE1713B20 in /lib64/libpthread.so.0
4# crimson::mon::Connection::get_conn() in ceph-osd
5# 0x000055C3C2532DA8 in ceph-osd
6# 0x000055C3C2535CB5 in ceph-osd
7# 0x000055C3BBC9FC70 in ceph-osd
8# 0x000055C3C76FAE5F in ceph-osd
9# 0x000055C3C77003A0 in ceph-osd
10# 0x000055C3C78B240B in ceph-osd
11# 0x000055C3C740FE8A in ceph-osd
12# 0x000055C3C7419FAE in ceph-osd
13# main in ceph-osd
14# __libc_start_main in /lib64/libc.so.6
15# _start in ceph-osd
Fault at location: 0x98
```
[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-06-08_09:11:05-rados-master-distro-basic-smithi/
6159602
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
dengchl01 [Tue, 8 Jun 2021 07:42:27 +0000 (15:42 +0800)]
mgr/mgr_module: add docstring of MgrModule.get()-mgr_map,modified_config_options,service_map,mds_metadata,pg_status,osd_pool_stats,mgr_ips,have_local_config_map
Signed-off-by: dengchl01 <dengchl01@inspur.com>
Kefu Chai [Tue, 8 Jun 2021 05:57:03 +0000 (13:57 +0800)]
Merge pull request #41745 from tchaikov/wip-crimson-osd-cleanup
crimson/osd: do not capture unused variable
Reviewed-by: Samuel Just <sjust@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 05:12:22 +0000 (13:12 +0800)]
crimson/osd: do not capture unused variable
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 04:44:21 +0000 (12:44 +0800)]
rgw/rgw_lua*: return unknown field error using luaL_error()
it's found on aarch64, the exception is not caught even if we do
catch exactly the same type of thrown exception, and the uncaught
exception ends up with a std::terminate() call. it could be the ABI
mismatch in aarch64, so the C++ runtime failed to find the catch block.
in this change, luaL_error() is used to populate the error to the
caller instead to workaround this issue.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 04:30:37 +0000 (12:30 +0800)]
rgw/rgw_lua_utils: return error using luaL_error()
it's found on aarch64, the exception is not caught even if we do
catch exactly the same type of thrown exception, and the uncaught
exception ends up with a std::terminate() call. it could be the ABI
mismatch in aarch64, so the C++ runtime failed to find the catch block.
in this change, luaL_error() is used to populate the error to the
caller instead to workaround this issue.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 01:26:43 +0000 (09:26 +0800)]
Merge pull request #41742 from lmgdlmgd/master
mgr/mgr_module: correct docstring of MgrModule.get()
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 00:59:36 +0000 (08:59 +0800)]
Merge pull request #41736 from rzarzynski/wip-crimson-alienstore-open_collection_dne
crimson/os/alienstore: open_collection() returns nullptr if DNE.
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 8 Jun 2021 00:58:48 +0000 (08:58 +0800)]
Merge pull request #41730 from tchaikov/wip-fc35-python
do_cmake.sh: use python3.10 for fedora 35
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
靳登科 [Tue, 8 Jun 2021 00:55:20 +0000 (08:55 +0800)]
mgr/mgr_module: correct docstring of MgrModule.get()
In annotation, one of param of Mgr_module.get() is "osd_crush_map_text", but in PyObject *ActivePyModules::get_python and selftest, the param is "osdmap_crush_map_text", so update annotation from "osd_crush_map_text" to "osdmap_crush_map_text"
Signed-off-by: lmgdlmgd <1105050771@qq.com>
Andrew Schoen [Mon, 7 Jun 2021 19:06:52 +0000 (14:06 -0500)]
Merge pull request #40659 from PepperJo/data_allocate_fraction_v2
ceph-volume,python-common: Data allocate fraction
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Radoslaw Zarzynski [Mon, 7 Jun 2021 16:40:06 +0000 (16:40 +0000)]
crimson/os/alienstore: open_collection() returns nullptr if DNE.
This is an AlienStore's counterpart of
9fbd60131b5396beb81ce745c9ccf01388f7e303
which becomes necessary as, since
6b66c30b2331f0d42c8a51446ab04714047084b6,
we call `open_collection()` to check the existence of OSD's superblock.
Without this fix the following crash happens:
```
[rzarzynski@o06 build]$ MDS=0 MGR=0 OSD=1 ../src/vstart.sh -l -n --without-dashboard --crimson
...
crimson-osd: boost/include/boost/smart_ptr/intrusive_ptr.hpp:199: T* boost::intrusive_ptr<T>::operator->() const [with T = ObjectStore::CollectionImpl]: Assertion `px != 0' failed.
Aborting on shard 0.
Backtrace:
0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/boost/include/boost/stacktrace/stacktrace.hpp:129
1# FatalSignal::signaled(int, siginfo_t const*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::operator()(int, siginfo_t*, void*) const at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:41
3# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:36
4# 0x00007F8BF534DB30 in /lib64/libpthread.so.0
5# gsignal in /lib64/libc.so.6
6# abort in /lib64/libc.so.6
7# 0x00007F8BF3948B19 in /lib64/libc.so.6
8# 0x00007F8BF3956DF6 in /lib64/libc.so.6
9# boost::intrusive_ptr<ObjectStore::CollectionImpl>::operator->() const at /home/rzarzynski/ceph1/build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:200
10# crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}::operator()(boost::intrusive_ptr<ObjectStore::CollectionImpl>) const at /home/rzarzynski/ceph1/build/../src/crimson/os/alienstore/alien_store.cc:214
11# seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > seastar::futurize<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >::invoke<crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:2135
12# auto seastar::futurize_invoke<crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(crimson::os::AlienStore::open_collection(coll_t const&)::{lambda(boost::intrusive_ptr<ObjectStore::CollectionImpl>)#2}&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:2167
13# _ZZN7seastar6futureIN5boost13intrusive_ptrIN11ObjectStore14CollectionImplEEEE4thenIZN7crimson2os10AlienStore15open_collectionERK6coll_tEUlS5_E0_NS0_INS2_INS9_19FuturizedCollectionEEEEEEET0_OT_ENUlDpOT_E_clIJS5_EEEDaSN_ at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:1526
14# _ZN7seastar20noncopyable_functionIFNS_6futureIN5boost13intrusive_ptrIN7crimson2os19FuturizedCollectionEEEEEONS3_IN11ObjectStore14CollectionImplEEEEE17direct_vtable_forIZNS1_ISB_E4thenIZNS5_10AlienStore15open_collectionERK6coll_tEUlSB_E0_S8_EET0_OT_EUlDpOT_E_E4callEPKSE_SC_ at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/util/noncopyable_function.hh:125
15# seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>::operator()(boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) const at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/util/noncopyable_function.hh:210
16# seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > std::__invoke_impl<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> >, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(std::__invoke_other, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/bits/invoke.h:60
17# std::__invoke_result<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >::type std::__invoke<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/bits/invoke.h:97
18# std::invoke_result<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >::type std::invoke<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/functional:83
19# auto seastar::internal::future_invoke<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl> >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>&&) at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:1213
20# seastar::future<boost::intrusive_ptr<ObjectStore::CollectionImpl> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&)#1}::operator()(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&) const::{lambda()#1}::operator()() const at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:1575
21# void seastar::futurize<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >::satisfy_with_result_of<seastar::future<boost::intrusive_ptr<ObjectStore::CollectionImpl> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&)#1}::operator()(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&) const::{lambda()#1}>(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&) at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:2120
22# seastar::future<boost::intrusive_ptr<ObjectStore::CollectionImpl> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&)#1}::operator()(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&) const at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:1571
23# seastar::continuation<seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<ObjectStore::CollectionImpl> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>, seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > >(seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<boost::intrusive_ptr<crimson::os::FuturizedCollection> >&&, seastar::noncopyable_function<seastar::future<boost::intrusive_ptr<crimson::os::FuturizedCollection> > (boost::intrusive_ptr<ObjectStore::CollectionImpl>&&)>&, seastar::future_state<boost::intrusive_ptr<ObjectStore::CollectionImpl> >&&)#1}, boost::intrusive_ptr<ObjectStore::CollectionImpl> >::run_and_dispose() at /home/rzarzynski/ceph1/build/../src/seastar/include/seastar/core/future.hh:771
24# seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /home/rzarzynski/ceph1/build/../src/seastar/src/core/reactor.cc:2237
25# seastar::reactor::run_some_tasks() at /home/rzarzynski/ceph1/build/../src/seastar/src/core/reactor.cc:2646 (discriminator 1)
26# seastar::reactor::run() at /home/rzarzynski/ceph1/build/../src/seastar/src/core/reactor.cc:2805
27# seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) at /home/rzarzynski/ceph1/build/../src/seastar/src/core/app-template.cc:207 (discriminator 7)
28# seastar::app_template::run(int, char**, std::function<seastar::future<int> ()>&&) at /home/rzarzynski/ceph1/build/../src/seastar/src/core/app-template.cc:115 (discriminator 2)
29# main at /home/rzarzynski/ceph1/build/../src/crimson/osd/main.cc:206 (discriminator 1)
30# __libc_start_main in /lib64/libc.so.6
31# _start in /home/rzarzynski/ceph1/build/bin/crimson-osd
Reactor stalled for 33157 ms on shard 0. Backtrace: 0xb14ab 0xa6cd418 0xa6a496d 0xa54fd22 0xa566e3d 0xa56383c 0xa5639fc 0xa566808 0x12b2f 0x3780e 0x21c44 0x21b18 0x2fdf5 0x7cfdf93 0x7b78622 0x7c3fcb7 0x7c1bcb2 0x7c1bdf1 0x7c4004a 0x7d913c7 0x7d875ed 0x7d777d7 0x7d5f626 0x7d47ca4 0x7d47b5a 0x7d5f770 0x7d47931 0x7d9bb74 0xa5998b5 0xa59e1ff 0xa5a3a95 0xa43546c 0xa433331 0x3f01992 0x237c2 0x3c7f9bd
../src/vstart.sh: line 28: 665993 Aborted (core dumped) PATH=$CEPH_BIN:$PATH "$@"
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 15:24:46 +0000 (23:24 +0800)]
Merge pull request #41733 from rzarzynski/wip-crimson-clientreq-cleanup
crimson/osd: clean up mutability and fix log in ClientRequest
Reviewed-by: Kefu Chai <kchai@redhat.com>
Radoslaw Zarzynski [Mon, 7 Jun 2021 14:30:14 +0000 (14:30 +0000)]
crimson/osd: fix debug in ClientRequest::do_process().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Mon, 7 Jun 2021 14:29:29 +0000 (14:29 +0000)]
crimson/osd: drop unneeded lambda mutability in ClientRequest.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Sage Weil [Mon, 7 Jun 2021 14:03:09 +0000 (10:03 -0400)]
Merge PR #41709 into master
* refs/pull/41709/head:
cephadm: improve is_container_running()
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Sage Weil [Mon, 7 Jun 2021 14:02:52 +0000 (10:02 -0400)]
Merge PR #41509 into master
* refs/pull/41509/head:
common/cmdparse: fix CephBool validation for tell commands
mgr/nfs: fix 'nfs export create' argument order
common/cmdparse: emit proper json
mon/MonCommands: add -- seperator to example
qa/tasks/cephfs/test_nfs: fix export create test
mgr: make mgr commands compat with pre-quincy mon
doc/_ext/ceph_commands: handle non-positional args in docs
mgr: fix reweight-by-utilization cephbool flag
mon/MonCommands: convert some CephChoices to CephBool
mgr/k8sevents: fix help strings
pybind/mgr/mgr_module: fix help desc formatting
mgr/orchestrator: clean up 'orch {daemon add,apply} rgw' args
mgr/orchestrator: add end_positional to a few methods
mgr/orchestrator: reformat a few methods
pybind/ceph_argparse: stop parsing when we run out of positional args
pybind/ceph_argparse: remove dead code
pybind/mgr/mgr_module: infer non-positional args
pybind/mgr/mgr_module: add separator for non-positional args
command/cmdparse: use -- to separate positional from non-positional args
pybind/ceph_argparse: adjust help text for non-positional args
pybind/ceph_argparse: track a 'positional' property on cli args
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 13:56:38 +0000 (21:56 +0800)]
Merge pull request #41725 from tchaikov/wip-cmake-prom-warning
cmake: do not print warning in each cmake run
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 13:55:39 +0000 (21:55 +0800)]
Merge pull request #41728 from tchaikov/wip-cmake-win32-cleanup
cmake: let vstart-base depend on cython_rados if NOT WIN32
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 13:46:49 +0000 (21:46 +0800)]
Merge pull request #41704 from tchaikov/wip-ceph-daemon-ann
pybind/ceph_daemon: add missing type annotations
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 13:43:52 +0000 (21:43 +0800)]
Merge pull request #41732 from rzarzynski/wip-crimson-opsequencer-prev_op-gt-last_unblocked
crimson/osd: fix assertion failure in OpSequencer on replay.
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 13:32:02 +0000 (21:32 +0800)]
Merge pull request #41705 from tchaikov/wip-script-ceph-debug-docker
script/ceph-debug-docker: s/x86_64/$(arch)/
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Radoslaw Zarzynski [Mon, 7 Jun 2021 13:03:51 +0000 (13:03 +0000)]
crimson/osd: fix assertion failure in OpSequencer on replay.
Commit
b5efdc6f1c9563357d7dfd33a8f379053592a215 has unified
the interruption handling among `InternalClientRequest` and
`ClientRequest`. Unfortunately, a call to `maybe_reset()` of
`OpSequencer` has been overlooked and dropped leading to the
`assert(prev_op > last_unblocked)` assertion failure in
`start_op()`.
This was the root cause of the following problem at Sepia:
```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-05-26_12:20:26-rados-master-distro-basic-smithi/
6136929$ less ./remote/smithi194/log/ceph-osd.6.log.gz
...
DEBUG 2021-05-26 20:24:53,988 [shard 0] ms - [osd.6(client) v2:172.21.15.194:6804/34047 >> client.4453 172.21.15.67:0/
3814935464@37042] <== #1
=== osd_op(client.4453.0:5 12.6 12.
7fc1f406 (undecoded) ondisk+write+known_if_redirected e52) v8 (42)
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12.
7fc1f406 (undecoded) ondisk+write+known_if_redirected e52) v8): start
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12.
7fc1f406 (undecoded) ondisk+write+known_if_redirected e52) v8): in repeat
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12.
7fc1f406 (undecoded) ondisk+write+known_if_redirected e52) v8) same_interval_since: 19
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - do_recover_missing check for recovery, 12:
602f83fe:::foo:head
DEBUG 2021-05-26 20:24:53,988 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12:
602f83fe:::foo:head {write 0~128 in=128b} snapc 0={} ondisk+write+known_if_redirected e52) v8): got obc lock
...
DEBUG 2021-05-26 20:25:21,810 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12:
602f83fe:::foo:head {write 0~128 in=1
28b} snapc 0={} ondisk+write+known_if_redirected e52) v8): in repeat
...
DEBUG 2021-05-26 20:25:21,809 [shard 0] osd - should_abort_request operation restart, acting set changed
...
DEBUG 2021-05-26 20:25:21,813 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12:
602f83fe:::foo:head {write 0~128 in=128b} snapc 0={} ondisk+write+known_if_redirected e52) v8) same_interval_since: 55
...
DEBUG 2021-05-26 20:25:21,813 [shard 0] osd - client_request(id=4, detail=osd_op(client.4453.0:5 12.6 12:
602f83fe:::foo:head {write 0~128 in=128b} snapc 0={} ondisk+write+known_if_redirected e52) v8) same_interval_since: 55
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/
17.0.0-4622-gaa1dc559/rpm/el8/BUILD/
ceph-17.0.0-4622-gaa1dc559/src/crimson/osd/osd_operation_sequencer.h:52: seastar::futurize_t<Result> crimson::osd::OpSequencer::start_op(HandleT&, uint64_t, uint64_t, FuncT&&) [with HandleT = crimson::PipelineHandle; FuncT = crimson::interruptible::interruptor<InterruptCond>::wrap_function(Func&&) [with Func = crimson::osd::ClientRequest::start()::<lambda()> mutable::<lambda(Ref<crimson::osd::PG>)> mutable::<lambda()> mutable::<lambda()>; InterruptCond = crimson::osd::IOInterruptCondition]::<lambda()>; Result = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<> >; seastar::futurize_t<Result> = crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<> >; uint64_t = long unsigned int]: Assertion `prev_op > last_unblocked' failed.
Aborting on shard 0.
Backtrace:
0# 0x000055F440239C1F in ceph-osd
1# FatalSignal::signaled(int, siginfo_t const*) in ceph-osd
2# FatalSignal::install_oneshot_signal_handler<6>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) in ceph-osd
3# 0x00007F4788336B20 in /lib64/libpthread.so.0
4# gsignal in /lib64/libc.so.6
5# abort in /lib64/libc.so.6
6# 0x00007F4786931B09 in /lib64/libc.so.6
7# 0x00007F478693FDE6 in /lib64/libc.so.6
8# 0x000055F43BA3DD17 in ceph-osd
9# 0x000055F43BA419A8 in ceph-osd
10# seastar::continuation<seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<boost::intrusive_ptr<crimson::osd::PG> >::then_impl_nrvo<seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>, seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > >(seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >&&, seastar::noncopyable_function<seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > (boost::intrusive_ptr<crimson::osd::PG>&&)>&, seastar::future_state<boost::intrusive_ptr<crimson::osd::PG> >&&)#1}, boost::intrusive_ptr<crimson::osd::PG> >::run_and_dispose() in ceph-osd
11# 0x000055F445C5C70F in ceph-osd
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 09:50:49 +0000 (17:50 +0800)]
do_cmake.sh: use python3.10 for fedora 35
fedora34 was released in April 2021 with Python3.9. while fedora 35
(still Rawhide) will include python3.10. see https://fedoraproject.org/wiki/Releases/35/ChangeSet#Python_3.10
so update the script accordingly.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 09:49:24 +0000 (17:49 +0800)]
do_cmake.sh: refactor code to set PYBUILD
so it is more consistent with rhel/centos part.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 08:00:23 +0000 (16:00 +0800)]
cmake: let vstart-base depend on cython_rados if NOT WIN32
to disable this custom target if NOT WIN32 without explanations is difficult
to maintain, so add a comment, and limit the scope to just
`cython_rados`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 07:09:42 +0000 (15:09 +0800)]
cmake: do not print warning in each cmake run
cmake prints the warning of
> run-promtool-unittests is skipped due to missing promtool
everytime if the test is skipped per the test. but we only need to
get this message the first time cmake is executed. more importantly,
the value of `PROMTOOL_EXECUTABLE` is cached. so the new error message
does not provide updated information, so just stop doing so if the
cached variable is found.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 7 Jun 2021 01:52:36 +0000 (09:52 +0800)]
Merge pull request #41695 from tchaikov/wip-crimson-net-move
crimson/net: move from out_q into sent queue
Reviewed-by: Amnon Hanuhov <ahanukov@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Kefu Chai [Sun, 6 Jun 2021 01:45:16 +0000 (09:45 +0800)]
Merge pull request #41708 from tchaikov/wip-seastore-open-coll
crimson/os/seastore: open_collection() returns nullptr if DNE
Reviewed-by: Samuel Just <sjust@redhat.com>
Sage Weil [Sat, 5 Jun 2021 20:43:36 +0000 (16:43 -0400)]
Merge PR #41665 into master
* refs/pull/41665/head:
mgr/cephadm:fix alerts sent to wrong URL
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Sage Weil [Sat, 5 Jun 2021 15:05:48 +0000 (10:05 -0500)]
cephadm: improve is_container_running()
The 'podman ps' command sporatically exits with 125, I suspect due
to some race/bug in podman itself.
However, our goal here is just to check if a specific container
is running. Use inspect for that instead--it's even (a bit) faster.
Fixes: https://tracker.ceph.com/issues/51109
Signed-off-by: Sage Weil <sage@newdream.net>
Kefu Chai [Sat, 5 Jun 2021 16:06:32 +0000 (00:06 +0800)]
Merge pull request #40652 from ronen-fr/wip-ronenf-cscrub-class
osd/scrub: modify "classic" OSD scrub state-machine to support Crimson
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Sage Weil [Sat, 5 Jun 2021 15:23:35 +0000 (11:23 -0400)]
common/cmdparse: fix CephBool validation for tell commands
The tell/asok validation uses a different validation path; make it work
for CephBool arguments.
Signed-off-by: Sage Weil <sage@newdream.net>
Kefu Chai [Sat, 5 Jun 2021 13:41:00 +0000 (21:41 +0800)]
Merge pull request #41154 from rzarzynski/wip-global-backtrace-bug-50647
global: fault handlers cope with simultaneous faults now.
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 13:33:00 +0000 (21:33 +0800)]
Merge pull request #41604 from t-msn/wip-51030
osd/ECBackend: Fix null pointer dereference when enabling jaeger tracing
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 13:26:04 +0000 (21:26 +0800)]
Merge pull request #41501 from aclamk/wip-bluefs-safer-flush
os/bluestore: Remove possibility of replay log and file inconsistency
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Kefu Chai [Sat, 5 Jun 2021 13:23:13 +0000 (21:23 +0800)]
Merge pull request #41506 from ceph/wip-cv-batch-fixes
ceph-volume: fix batch report and respect ceph.conf config values
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 13:17:24 +0000 (21:17 +0800)]
Merge pull request #41688 from tchaikov/wip-debian-rook
debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-roo…
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 09:39:25 +0000 (17:39 +0800)]
crimson/os/seastore: open_collection() returns nullptr if DNE
we check for the existence of meta collection by trying to open it,
if it exists, we continue check for the superblock stored in it, if
the superblock does not exist or corrupted, we consider it as a failure.
before this change, open_collection() always return a valud Collection
even if the store does not have the collection with specified cid. this
behavior could be misleading in the use case above.
after this change, open_collection() looks up the collections stored in
root collection node for the specfied cid, and return nullptr if it does
not exist.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 09:22:35 +0000 (17:22 +0800)]
crimson/os/seastore: use structured binding
for better readability
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 04:19:41 +0000 (12:19 +0800)]
script/ceph-debug-docker: s/x86_64/$(arch)/
try to avoid hardwiring to a certain architecture if possible.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 03:53:50 +0000 (11:53 +0800)]
pybind: add tox.ini
also add ceph_daemon to mypy.ini
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 04:08:00 +0000 (12:08 +0800)]
pybind/ceph_daemon: add missing type annotations
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 03:53:03 +0000 (11:53 +0800)]
pybind/ceph_daemon: correct type annotation of admin_socket()
we always pass a list of string to it, but since it does not assume the
container type of string, just specify Sequence[str].
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 03:40:18 +0000 (11:40 +0800)]
pybind/ceph_daemon: fix the format string in Termsize.__repr__()
we intended to print the class name usign "%s".
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 03:38:17 +0000 (11:38 +0800)]
pybind/ceph_daemon: do not handle get_command_descriptions in admin_socket()
we never pass a single string to admin_socket() as its 2nd parameter.
instead, we always pass a list of string. so no need to handle it as a
special case.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 02:06:07 +0000 (10:06 +0800)]
Merge pull request #41581 from tchaikov/wip-options-mgr-mon
common/options: extract mgr and mon options out
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kefu Chai [Sat, 5 Jun 2021 00:44:42 +0000 (08:44 +0800)]
Merge pull request #40073 from jmolmo/delete_service_causes_osd_removal
mgr/cephadm: Warn about OSDs to remove manually when deleting an OSD service
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Sage Weil [Fri, 4 Jun 2021 17:35:46 +0000 (13:35 -0400)]
mgr/nfs: fix 'nfs export create' argument order
Put path before --readonly so that it can still be positional.
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Tue, 1 Jun 2021 20:55:44 +0000 (16:55 -0400)]
common/cmdparse: emit proper json
Instead of '"req": "false"', emit '"req": false'. Same for conditional.
Luckily, the clients don't really care about this change, as
ceph_argparse.py argdesc interpets the JSON like so:
self.req = req in (True, 'True', 'true')
self.positional = positional in (True, 'True', 'true')
Clean up command definitions to use lowercase 'false', but tolerate
both for backward compat during upgrade and to tolerate future errors.
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Tue, 1 Jun 2021 18:43:47 +0000 (14:43 -0400)]
mon/MonCommands: add -- seperator to example
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Tue, 1 Jun 2021 17:39:03 +0000 (13:39 -0400)]
qa/tasks/cephfs/test_nfs: fix export create test
Everything after --readonly is non-positional.
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Sat, 29 May 2021 02:19:27 +0000 (22:19 -0400)]
mgr: make mgr commands compat with pre-quincy mon
If the mon is not yet quincy, do not share the quincy-ism of
'positional=false'.
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Tue, 25 May 2021 14:53:01 +0000 (10:53 -0400)]
doc/_ext/ceph_commands: handle non-positional args in docs
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Mon, 24 May 2021 19:17:58 +0000 (15:17 -0400)]
mgr: fix reweight-by-utilization cephbool flag
This was broken, since it was trying to get a CephChoices as a bool.
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Sun, 23 May 2021 14:47:35 +0000 (10:47 -0400)]
mon/MonCommands: convert some CephChoices to CephBool
These are old bool options that never got converted to CephBool.
Signed-off-by: Sage Weil <sage@newdream.net>
Patrick Donnelly [Fri, 4 Jun 2021 20:07:42 +0000 (13:07 -0700)]
Merge PR #41697 into master
* refs/pull/41697/head:
script: add a few more volume mounts for sepia
script: drop ceph-fuse from docker debugging
script: enable centos debuginfo repo for debugging
script: update repo url for multi-arch builds
script: fetch autobuild.asc key via HTTPS
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 17:57:03 +0000 (01:57 +0800)]
Merge pull request #41690 from tchaikov/wip-test-alloc_aging
test/objectstore/unittest_alloc_aging: init cct
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Kefu Chai [Fri, 4 Jun 2021 17:23:35 +0000 (01:23 +0800)]
Merge pull request #41698 from tchaikov/wip-qa-rook
qa/suites/orch/rook/smoke: stop testing on ubuntu 18.04
Reviewed-by: Sage Weil <sage@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 17:11:13 +0000 (01:11 +0800)]
qa/suites/orch/rook/smoke: stop testing on ubuntu 18.04
even rook does not really install ceph packages in the host directly, it
uses the ceph container image. but teuthology insists on checking the
existence of debian packages by querying shaman server when it sees a
teuthology facet file which includes:
os_type: ubuntu
os_version: "18.04"
but since we've stopped building ubuntu/bionic packages, teuthology
just complains when we are scheduling test suites which are composed
from facets in qa/suites/orch/rook/smoke.
in this change, the ubuntu_18.04.yaml is dropped because ubuntu/bionic
does not really increase the test coverage of ceph. it helps to test
the rook and container runtime though.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Patrick Donnelly [Fri, 4 Jun 2021 16:33:54 +0000 (09:33 -0700)]
script: add a few more volume mounts for sepia
We now have a few Ceph file systems with various possible mount points
depending which lab machine you're using.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 4 Jun 2021 16:33:30 +0000 (09:33 -0700)]
script: drop ceph-fuse from docker debugging
Install this on the fly as necessary...
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 4 Jun 2021 16:32:52 +0000 (09:32 -0700)]
script: enable centos debuginfo repo for debugging
So we can fetch e.g. the sqlite debuginfo packages.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 4 Jun 2021 16:31:19 +0000 (09:31 -0700)]
script: update repo url for multi-arch builds
Brad suggested this change based on his commit [1]. Thank you!
[1] https://github.com/ceph/ceph-ansible/commit/
267cce9e8360fc8cb9c192fde2406e5dca724610
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Fri, 4 Jun 2021 16:30:04 +0000 (09:30 -0700)]
script: fetch autobuild.asc key via HTTPS
Rather than relying the key being avaiable on the LRC /ceph file system.
(Someone appears to have deleted it recently.)
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Jonas Pfefferle [Fri, 4 Jun 2021 11:22:46 +0000 (13:22 +0200)]
python-common: data_allocate_fraction translate test
Add DriveGroupSpec translate test with data_allocate_fraction
Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
Jonas Pfefferle [Fri, 4 Jun 2021 11:14:54 +0000 (13:14 +0200)]
ceph-volume: data_allocate_fraction: batch tests
lvm batch tests with data allocate fraction
Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
Kefu Chai [Fri, 4 Jun 2021 12:19:30 +0000 (20:19 +0800)]
crimson/net: move from out_q into sent queue
to avoid the refcounting of underlying RefCountedObject.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 12:13:54 +0000 (20:13 +0800)]
Merge pull request #41679 from AmnonHanuhov/wip-get_rid_of_pending_q
crimson/net: Use out_q instead of pending_q
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Amnon Hanuhov [Thu, 3 Jun 2021 13:57:41 +0000 (16:57 +0300)]
crimson/net: Use out_q instead of pending_q
pending_q contains the same messages as in out_q and it is only used
for creating a bytestream out of these messages. We can just use out_q for that.
Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 09:15:06 +0000 (17:15 +0800)]
Merge pull request #41631 from tchaikov/wip-keyring-decode
auth/KeyRing: always decode keying as plaintext
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 09:00:48 +0000 (17:00 +0800)]
Merge pull request #41587 from cfsnyder/bugfix_47738
mgr/DaemonServer.cc: prevent mgr crashes caused by integer underflow that is triggered by large increases to pg_num/pgp_num
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 08:59:24 +0000 (16:59 +0800)]
Merge pull request #41592 from tchaikov/wip-ceph-default-confffile
ceph.in: use rados.Rados.DEFAULT_CONF_FILES
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 08:58:59 +0000 (16:58 +0800)]
Merge pull request #41594 from tchaikov/wip/test/librados/list
test/librados/list: print reason why test fails
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 08:57:41 +0000 (16:57 +0800)]
Merge pull request #36941 from hoamer/patch-1
doc/mgr/administrator: add a more precise description for creating key
Reviewed-by: Kefu Chai <kchai@redhat.com>
hoamer [Wed, 2 Sep 2020 07:13:12 +0000 (09:13 +0200)]
doc/mgr/administrator: add a more precise description for creating key
added a more precise description to handle filename when creating key for mgr
Signed-off-by: hoamer <kontakt@sebastian-neugebauer.de>
Kefu Chai [Fri, 4 Jun 2021 03:25:12 +0000 (11:25 +0800)]
debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-rook anymore
per https://www.debian.org/doc/debian-policy/ch-relationships.html
> Recommends
> This declares a strong, but not absolute, dependency.
>
> The Recommends field should list packages that would be found together
> with this one in all but unusual installations.
ceph-mgr-modules-core provides a set of ceph-mgr modules which are
always enabeld. but the rook module enables ceph-mgr to install and
configure a Ceph cluster using Rook. this module is very useful but
it does not have such a strong connection with ceph-mgr-modules-core.
we can always install it separately for using better intergration with
Rook.
See-also: https://tracker.ceph.com/issues/45574
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 2 Jun 2021 09:54:18 +0000 (17:54 +0800)]
test/objectstore/unittest_alloc_aging: init cct
* initialize the cct use by test, otherwise g_ceph_context is
not set at all.
* instead of using g_ceph_context, use static member variable cct.
less dependency to the global instance.
* setup and teardown the cct for test suite, because global_init()
initialize g_ceph_context, which cannot be set multiple times.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 2 Jun 2021 09:38:49 +0000 (17:38 +0800)]
test/objectstore: s/TearDownTestCase/TearDownTestSuite/
TearDownTestCase is deprecated by GTest. let's use the new API instead.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 05:56:33 +0000 (13:56 +0800)]
vstart.sh: disable restful by default
since we are going to deprecate restful in favor of dashboard's RESTful
API. see
https://lists.ceph.io/hyperkitty/list/dev@ceph.io/message/LBKLNXH7UQL7TLFU5G52Y2SYVME4RS6P/
and https://tracker.ceph.com/issues/47066, let's disable this module
by default. and change the option of --without-restful to
--with-restful.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 05:50:38 +0000 (13:50 +0800)]
Merge pull request #41652 from tchaikov/wip-qa-asock-or
qa/tasks/admin_socket: support "foo || bar" as command
Reviewed-by: Samuel Just <sjust@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 04:30:23 +0000 (12:30 +0800)]
Merge pull request #41686 from t-msn/update-trace-doc
doc/dev: update how to use lttng/blkin trace
Reviewed-by: Kefu Chai <kchai@redhat.com>
Misono Tomohiro [Fri, 4 Jun 2021 02:36:49 +0000 (11:36 +0900)]
doc/dev: update how to use lttng/blkin trace
Update doc to reflect current status.
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
Sage Weil [Fri, 4 Jun 2021 02:04:55 +0000 (22:04 -0400)]
Merge PR #41553 into master
* refs/pull/41553/head:
ceph-volume: replace __ with _ in device_id
Reviewed-by: Kefu Chai <kchai@redhat.com>
Sage Weil [Fri, 4 Jun 2021 02:04:32 +0000 (22:04 -0400)]
Merge PR #41636 into master
* refs/pull/41636/head:
mgr/cephadm/inventory: do not try to resolve current mgr host
pybind/mgr/mgr_module: make get_mgr_ip() return mgr's IP from mgrmap
mgr/restful: use get_mgr_ip() instead of hostname
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Fri, 4 Jun 2021 01:44:58 +0000 (09:44 +0800)]
Merge pull request #41674 from tchaikov/wip-vstart-without-restful
vstart.sh: add an option named --without-restful
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Paul Cuzner [Wed, 2 Jun 2021 23:34:19 +0000 (11:34 +1200)]
mgr/cephadm:fix alerts sent to wrong URL
The path_prefix in prometheus.yml was specifying an
endpoint prefix, which was invalid. This resulted in 404
errors when trying to send alerts to alertmanager and
blocked alerts being sent on to the ceph-dashboard API
receiver. This fix remves this prefix.
Fixes: https://tracker.ceph.com/issues/51073
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
Kefu Chai [Thu, 3 Jun 2021 23:50:44 +0000 (07:50 +0800)]
Merge pull request #41670 from tchaikov/wip-op-tracking-spin-off-0
crimson, common: improve const-correctness of Operation::dump()s.
Reviewed-by: Samuel Just <sjust@redhat.com>
Kefu Chai [Thu, 3 Jun 2021 23:50:21 +0000 (07:50 +0800)]
Merge pull request #41672 from tchaikov/wip-crimson-test-handle-fut
test/crimson/seastore: always handle returned future<>
Reviewed-by: Samuel Just <sjust@redhat.com>
Patrick Donnelly [Thu, 3 Jun 2021 20:34:54 +0000 (13:34 -0700)]
Merge PR #41654 into master
* refs/pull/41654/head:
mds: do not infinitely recursively print a metric
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Patrick Donnelly [Thu, 3 Jun 2021 20:33:58 +0000 (13:33 -0700)]
Merge PR #41639 into master
* refs/pull/41639/head:
mds/scrub: write root inode backtrace at creation
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 3 Jun 2021 20:33:27 +0000 (13:33 -0700)]
Merge PR #41499 into master
* refs/pull/41499/head:
qa/tasks/mds_thrash: fix thrash iteration never skip
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 3 Jun 2021 20:23:17 +0000 (13:23 -0700)]
Merge PR #41443 into master
* refs/pull/41443/head:
test: update log-ignorelist for fs:mirror test
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Thu, 3 Jun 2021 20:22:23 +0000 (13:22 -0700)]
Merge PR #39910 into master
* refs/pull/39910/head:
test: Add test for mgr hang when osd is full
mgr: Set client_check_pool_perm to false
mds: Add full caps to avoid osd full check
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Dan Mick [Thu, 3 Jun 2021 18:32:24 +0000 (11:32 -0700)]
Merge pull request #41559 from dmick/wip-grafana-container
monitoring/grafana/build/Makefile: revamp for arm64 builds, pushes to docker and quay, jenkins