Leonid Usov [Thu, 28 Mar 2024 05:32:26 +0000 (01:32 -0400)]
mds/quiesce: prevent an overflow of the wait duration
QuiesceTimeInterval::max() may overflow inside of a call to
std::condition_variable::wait_for and result in a busy-loop,
making the call to timeout immediately
The solution is to cap the wait duration to a value which can
certainly fit in whichever clock std library is using internally.
Fixes: https://tracker.ceph.com/issues/65276 Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
This commit broke the import of the "mgr_module" module
within the python modules in the mgr at least on python 3.6.8
that we currently use in our centos 8 stream based containers
Failures would look like (removing beginning of log lines)
Loading python module 'alerts'
Module not found: 'mgr_module'
Class not found in module 'alerts'
Error loading module 'alerts': (22) Invalid argument
Afreen [Wed, 27 Mar 2024 09:35:58 +0000 (15:05 +0530)]
mgr/dashboard:Use advanced fieldset for rbd image
Fixes https://tracker.ceph.com/issues/65025
- using cd-advanced-fieldset component for advanced section
- this will align advanced sections of forms in dashboard
- changed heading of sections of rbd-configuration to h5 to better represent it
as a sub section.
- added unit tests for rbd image config
we allocate a hitset without freeing it in this test, and LeakSanitizer
points this out
```
Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x557a0841a9dd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_hitset+0x1ae9dd) (BuildId: ad9be2b52b3d6fb1a567b262c3becaab6373e88d)
#1 0x557a0843b98e in ExplicitHashHitSetTest::ExplicitHashHitSetTest() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/hitset.cc:128:46
#2 0x557a0843b918 in ExplicitHashHitSetTest_Construct_Test::ExplicitHashHitSetTest_Construct_Test() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/hitset.cc:133:1
#3 0x557a0843b8cb in testing::internal::TestFactoryImpl<ExplicitHashHitSetTest_Construct_Test>::CreateTest() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
#4 0x557a08532406 in testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
#5 0x557a084eb892 in testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
#6 0x557a0849dd55 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2848:22
#7 0x557a0849f3bb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
#8 0x557a084bc848 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
#9 0x557a0853a6d6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
#10 0x557a084f1222 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
#11 0x557a084bbbd2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
#12 0x557a08441c10 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
#13 0x557a08441ba1 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
#14 0x7faa493d6d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```
in this change, we just free it in the dtor. this should address
the warning from the sanitizer.
Kefu Chai [Sun, 31 Mar 2024 01:14:21 +0000 (09:14 +0800)]
test/on_exit: use static variables for on_exit hooks
before this change, we allocate memory chunks using malloc(), but
we never free them. and LeakSanitizer points this out
```
Direct leak of 4 byte(s) in 1 object(s) allocated from:
#0 0x5588bfe532de in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_on_exit+0xa52de) (BuildId: 7c7a92bf5719592938c5307214bcd9b080bd847f)
#1 0x5588bfe911d7 in func_scope() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:33:22
#2 0x5588bfe90804 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:64:3
#3 0x7f23081c1d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
Direct leak of 4 byte(s) in 1 object(s) allocated from:
#0 0x5588bfe532de in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_on_exit+0xa52de) (BuildId: 7c7a92bf5719592938c5307214bcd9b080bd847f)
#1 0x5588bfe91160 in func_scope() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:29:22
#2 0x5588bfe90804 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:64:3
#3 0x7f23081c1d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```
in this change, instead of allocating the variables using `malloc()`,
we keep them in static variables, so that they can be accessed by
`OnExitManager` even if it is a static variable.
with this change, the memory leak reports for this source file go away.
Niklas Hambüchen [Sat, 30 Mar 2024 16:42:48 +0000 (17:42 +0100)]
doc/rados/operations: Improve crush_location docs
* Fix incorrect syntax
* Use underscores for config options, like other ceph docs did
* Fix incorrect statement that crush_location_hook adds fiels; it replaces
* Explain `root=default host=HOSTNAME` is not set if `crush_location` is given
* Remove duplication across sections
* Point out that `root=default` is important
in BlueFS.test_shared_alloc, we keep the return value of
`fs.get_perf_counters()`, and deference it after umounting the fs,
but the `PerfCounters*` pointer returned from `fs.get_perf_counters()`
is destroyed in `BlueFS::_shutdown_logger()` which is in turn called
by `BlueFS::umount()`. so ASan points this out:
```
==1662613==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110000b2d80 at pc 0x7f0eefc30644 bp 0x7ffcdbab6430 sp 0x7ffcdbab6428
READ of size 8 at 0x6110000b2d80 thread T0
#0 0x7f0eefc30643 in ceph::common::PerfCounters::get(int) const /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:246:8
#1 0x557595ddfc15 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1182:3
#2 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
#3 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
#4 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
#5 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
#6 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
#7 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
#8 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
#9 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
#10 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
#11 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
#12 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
#13 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#14 0x7f0eed083e3f in __libc_start_main csu/../csu/libc-start.c:392:3
#15 0x557595cd46a4 in _start (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x2856a4) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
0x6110000b2d80 is located 0 bytes inside of 208-byte region [0x6110000b2d80,0x6110000b2e50)
freed by thread T0 here:
#0 0x557595d92b1d in operator delete(void*) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x343b1d) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
#1 0x557595f31c43 in BlueFS::_shutdown_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:462:3
#2 0x557595f54ab5 in BlueFS::umount(bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:1076:3
#3 0x557595ddfbd7 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1180:6
#4 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
#5 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
#6 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
#7 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
#8 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
#9 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
#10 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
#11 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
#12 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
#13 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
#14 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
#15 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
previously allocated by thread T0 here:
#0 0x557595d922bd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x3432bd) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
#1 0x7f0eefc33180 in ceph::common::PerfCountersBuilder::PerfCountersBuilder(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:537:21
#2 0x557595f30ac9 in BlueFS::_init_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:221:23
#3 0x557595f42bc6 in BlueFS::mount() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:977:3
#4 0x557595ddd339 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1139:3
#5 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
#6 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
#7 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
#8 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
#9 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
#10 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
#11 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
#12 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
#13 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
#14 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
#15 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
#16 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```
in this change, instead of keeping `logger` across the `umount()` and
`mount()` calls, we get another instance of `logger`, query it for
the perf counter that we are interested, and compare the value
to see if it is unchanged.
John Mulligan [Fri, 29 Mar 2024 18:04:33 +0000 (14:04 -0400)]
ceph.spec.in: remove command-with-macro line
A comment clearly left as a breadcrumb for a node-proxy manpage is
causing (intermittent) build failures. Remove the line and hope
the manpage is added if/when appropriate.
Fixes: 0dd73643649ddc2366e60de4fe6c078b6e112091 Signed-off-by: John Mulligan <jmulligan@redhat.com>
Fixes: f68248944815acbf5fb08ef66f36da6d5baef824
It seems the shaman CentOS8 builders, which serve as the base for container images, aren't current with the ":=" syntax. Amend to support RHEL8, whose default python is python 3.6.8.
Patrick Donnelly [Thu, 28 Mar 2024 18:40:39 +0000 (14:40 -0400)]
Merge PR #56536 into main
* refs/pull/56536/head:
mds: do not dispatch aborted internal requests
qa: add test for killing quiesce_inode with outstanding remote authpin requests
mds: add lockup debugging command