Rishabh Dave [Wed, 3 Apr 2024 05:41:29 +0000 (11:11 +0530)]
qa/vstart_runner: don't let command run after timeout
Parameter "timeout" is accepted by LocalRemote.run() but the method
doesn't do anything about it besides accepting it. Thus, this parameter
has no effect.
In LocalRemote.run(), pass this parameter to LocalRemoteProcess.wait()
and from this method pass it to subprocess.Popen.communicate(). Thus,
command will be terminated by subprocess module at seconds specified by
"timeout" parameter. IOW, "timeout" parameter will have an effect.
Fixes: https://tracker.ceph.com/issues/65533 Signed-off-by: Rishabh Dave <ridave@redhat.com>
As the thrash tests were introduced, some options were disabled
until the tests are stabilized.
Re-enable chance_down option (default is 0.4) to detect bugs on restart.
Since it will probably take few iterations before thrash and recovery tests ('default.yaml')
will pass successfully, add anoter 'simple.yaml' which should remain stable.
crimson/common/operation: fix and move exit() after entering the next phase
If exit/unlock the barrier before entering the next phase, it is
possible for the next request to exit the barrier at the same time, and
enters the next phase first, causing reorder issues.
Instruct readers to use "mkdir /mnt/cephfs1" to create a mountpoint
before using "ceph-fuse" to mount a filesystem, if "/mnt/cephfs1"
doesn't already exist. cf.
https://github.com/ceph/ceph/pull/56831#discussion_r1561102227
The function is typically invoked on client errors like NoSuchBucket. Logging these errors with level 1 may initially suggest a significant issue, when in fact it's just a client error. Consider raising the logging level to 20 for better clarity.
- Moves "features" section in rbd image create form to "Advanced" section.
- makes rbd configuration section to be expanded by default rather than
being collapsed as it has only single section. This will improve user experience as it will not
require two clicks.
- updates e2e test
Rongqi Sun [Fri, 12 Apr 2024 06:51:34 +0000 (06:51 +0000)]
bluestore/bluestore_types: check 'it' valid before using
When sanitizer is enabled, unittest_bluestore_types fails as following
```
[ RUN ] sb_info_space_efficient_map_t.basic
=================================================================
==143714==ERROR: AddressSanitizer: heap-buffer-overflow on address 0xffff99f8b7f4 at pc 0xaaaab50bde18 bp 0xffffebefcdb0 sp 0xffffebefcda8
READ of size 8 at 0xffff99f8b7f4 thread T0
#0 0xaaaab50bde14 in sb_info_t::get_sbid() const /root/ceph/src/os/bluestore/bluestore_types.h:1337:30
#1 0xaaaab50a5908 in sb_info_space_efficient_map_t::find(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1385:10
#2 0xaaaab50bd638 in sb_info_space_efficient_map_t::_add(long) /root/ceph/src/os/bluestore/bluestore_types.h:1424:15
#3 0xaaaab50a52bc in sb_info_space_efficient_map_t::add_maybe_stray(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1358:12
#4 0xaaaab4fec03c in sb_info_space_efficient_map_t_basic_Test::TestBody() /root/ceph/src/test/objectstore/test_bluestore_types.cc:113:11
#5 0xaaaab51e9a40 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
#6 0xaaaab5197040 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
#7 0xaaaab51488a4 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
#8 0xaaaab514a7e8 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
#9 0xaaaab514bde8 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
#10 0xaaaab5167bac in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
#11 0xaaaab51f3940 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
#12 0xaaaab519e5d8 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
#13 0xaaaab5167024 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
#14 0xaaaab50b4d6c in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
#15 0xaaaab50a1080 in main /root/ceph/src/test/objectstore/test_bluestore_types.cc:2847:10
#16 0xffff9d6c73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#17 0xffff9d6c74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
#18 0xaaaab4f3812c in _start (/root/ceph/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)
0xffff99f8b7f4 is located 0 bytes to the right of 20-byte region [0xffff99f8b7e0,0xffff99f8b7f4)
allocated by thread T0 here:
#0 0xaaaab4fe636c in operator new[](unsigned long) (/root/ceph/build/bin/unittest_bluestore_types+0xef636c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)
#1 0xaaaab50c0d2c in mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t>::allocate(unsigned long, void*) /root/ceph/src/include/mempool.h:375:33
#2 0xaaaab50c0c0c in std::allocator_traits<mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::allocate(mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t>&, unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:318:20
#3 0xaaaab50c044c in std::_Vector_base<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:346:20
#4 0xaaaab50bf954 in void std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::_M_realloc_insert<long&>(__gnu_cxx::__normal_iterator<sb_info_t*, std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> > >, long&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:440:33
#5 0xaaaab50be0d8 in sb_info_t& std::vector<sb_info_t, mempool::pool_allocator<(mempool::pool_index_t)11, sb_info_t> >::emplace_back<long&>(long&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:121:4
#6 0xaaaab50bd760 in sb_info_space_efficient_map_t::_add(long) /root/ceph/src/os/bluestore/bluestore_types.h:1429:24
#7 0xaaaab50a5e78 in sb_info_space_efficient_map_t::add_or_adopt(unsigned long) /root/ceph/src/os/bluestore/bluestore_types.h:1361:15
#8 0xaaaab4feb07c in sb_info_space_efficient_map_t_basic_Test::TestBody() /root/ceph/src/test/objectstore/test_bluestore_types.cc:103:11
#9 0xaaaab51e9a40 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
#10 0xaaaab5197040 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
#11 0xaaaab51488a4 in testing::Test::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2680:5
#12 0xaaaab514a7e8 in testing::TestInfo::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:2858:11
#13 0xaaaab514bde8 in testing::TestSuite::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:3012:28
#14 0xaaaab5167bac in testing::internal::UnitTestImpl::RunAllTests() /root/ceph/src/googletest/googletest/src/gtest.cc:5723:44
#15 0xaaaab51f3940 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2605:10
#16 0xaaaab519e5d8 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /root/ceph/src/googletest/googletest/src/gtest.cc:2641:14
#17 0xaaaab5167024 in testing::UnitTest::Run() /root/ceph/src/googletest/googletest/src/gtest.cc:5306:10
#18 0xaaaab50b4d6c in RUN_ALL_TESTS() /root/ceph/src/googletest/googletest/include/gtest/gtest.h:2486:46
#19 0xaaaab50a1080 in main /root/ceph/src/test/objectstore/test_bluestore_types.cc:2847:10
#20 0xffff9d6c73f8 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#21 0xffff9d6c74c8 in __libc_start_main csu/../csu/libc-start.c:392:3
#22 0xaaaab4f3812c in _start (/root/ceph/build/bin/unittest_bluestore_types+0xe4812c) (BuildId: cb75399658026f83a4e89012de8fb02f08f6d239)
SUMMARY: AddressSanitizer: heap-buffer-overflow /root/ceph/src/os/bluestore/bluestore_types.h:1337:30 in sb_info_t::get_sbid() const
Shadow bytes around the buggy address:
0x200ff33f16a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff33f16b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff33f16c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff33f16d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff33f16e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x200ff33f16f0: fa fa fa fa fa fa fa fa fa fa fa fa 00 00[04]fa
0x200ff33f1700: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff33f1710: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff33f1720: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff33f1730: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x200ff33f1740: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==143714==ABORTING
```
'it' might be invalid, so before using 'it', need to figure validity out
Patrick Donnelly [Thu, 11 Apr 2024 23:00:53 +0000 (19:00 -0400)]
Merge PR #56839 into main
* refs/pull/56839/head:
script/ptl-tool: push branch to shaman for qa runs
script/ptl-tool: add release name to branch with switch
script/ptl-tool: alphabetize arguments
script/ptl-tool: avoid repo specific remotes entirely
script/ptl-tool: improve help text for envvars
Adds move constructor and move assignment operator to the librbd::Image.
Also marks copy ctor/assign op as deleted, and makes them public for better compiler diagnostics.
Patrick Donnelly [Thu, 11 Apr 2024 17:10:59 +0000 (13:10 -0400)]
Merge PR #56835 into main
* refs/pull/56835/head:
script/ptl-tool: create qa trackers for test branches
script/ptl-tool: add switch for debugging
script/ptl-tool: add --stop-at-built flag
When sanitizer is enabled, unittest_mds_quiesce_agent fails as following
```
[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from QuiesceAgentTest
[ RUN ] QuiesceAgentTest.ThreadManagement
[ OK ] QuiesceAgentTest.ThreadManagement (3 ms)
[ RUN ] QuiesceAgentTest.DbUpdates
[ OK ] QuiesceAgentTest.DbUpdates (1 ms)
[ RUN ] QuiesceAgentTest.QuiesceProtocol
[ OK ] QuiesceAgentTest.QuiesceProtocol (3 ms)
[ RUN ] QuiesceAgentTest.DuplicateQuiesceRequest
[ OK ] QuiesceAgentTest.DuplicateQuiesceRequest (2 ms)
[ RUN ] QuiesceAgentTest.TimeoutBeforeComplete
[ OK ] QuiesceAgentTest.TimeoutBeforeComplete (2 ms)
[----------] 5 tests from QuiesceAgentTest (11 ms total)
[----------] Global test environment tear-down
[==========] 5 tests from 1 test suite ran. (11 ms total)
[ PASSED ] 5 tests.
Rongqi Sun [Mon, 1 Apr 2024 07:49:18 +0000 (07:49 +0000)]
test/cephadm: extend timeout limit to fix timeout issue
When run make check(arm64) on omani series servers, eg.
https://jenkins.ceph.com/job/ceph-pull-requests-arm64/54751/.
Shows:
python: timeout after 30 seconds
INFO root:call_wrappers.py:284 Non-zero exit code 124 from /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/cephadm/.tox/py3/bin/python -c for i in range(1000000): print(i, flush=True)
Fixes: https://tracker.ceph.com/issues/65355 Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
qa/rgw/s3tests: remove 'client.0' from bucket prefix
new sns test cases are using this for topic names, but the '.' is not
allowed there:
> api_params = {'Name': 'test-client.0-n3bdgre5el2jk8v-606'}
> botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the CreateTopic operation: Name must be made up of only uppercase and lowercase ASCII letters, numbers, underscores, and hyphens
rgw/notify: support cross-tenant and cross-account notifications
a bucket's notification configuration may refer to topics from several
different tenants or accounts. when publishing to a given topic, look in
the correct namespace for each topic instead of defaulting to the
requesting user's tenant namespace