]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
15 months agocrimson/os/seastore/btree_lba_manager: cleanup methods that return std::pair
Zhang Song [Wed, 3 Apr 2024 09:07:52 +0000 (17:07 +0800)]
crimson/os/seastore/btree_lba_manager: cleanup methods that return std::pair

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
15 months agocrimson/os/seastore: introduce extent_ref_count_t
Zhang Song [Wed, 3 Apr 2024 08:27:50 +0000 (16:27 +0800)]
crimson/os/seastore: introduce extent_ref_count_t

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
15 months agocrimson/os/seastore: remove unused return value of RecordScanner::scan_valid_records
Zhang Song [Wed, 3 Apr 2024 08:02:22 +0000 (16:02 +0800)]
crimson/os/seastore: remove unused return value of RecordScanner::scan_valid_records

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
15 months agocrimson/common/interruptible_future: add discard_result
Zhang Song [Wed, 3 Apr 2024 08:01:59 +0000 (16:01 +0800)]
crimson/common/interruptible_future: add discard_result

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
15 months agoMerge pull request #56509 from rhcs-dashboard/replace-table-panel-grafana
Aashish Sharma [Wed, 3 Apr 2024 07:40:00 +0000 (13:10 +0530)]
Merge pull request #56509 from rhcs-dashboard/replace-table-panel-grafana

mgr/dashboard: replace deprecated table panel in grafana with a newer table panel

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge PR #56228 into main
Venky Shankar [Wed, 3 Apr 2024 05:27:56 +0000 (10:57 +0530)]
Merge PR #56228 into main

* refs/pull/56228/head:
doc: fixing doc/cephfs/fs-volumes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
15 months agoMerge pull request #56625 from tchaikov/wip-osd-hitset-free
Kefu Chai [Wed, 3 Apr 2024 00:43:58 +0000 (08:43 +0800)]
Merge pull request #56625 from tchaikov/wip-osd-hitset-free

test/osd/hitset: free allocated HitSet

Reviewed-by: Samuel Just <sjust@redhat.com>
15 months agoMerge PR #56565 into main
Patrick Donnelly [Tue, 2 Apr 2024 23:30:08 +0000 (19:30 -0400)]
Merge PR #56565 into main

* refs/pull/56565/head:
msg: trivial destructor should be default

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
15 months agoMerge PR #56566 into main
Patrick Donnelly [Tue, 2 Apr 2024 22:30:30 +0000 (18:30 -0400)]
Merge PR #56566 into main

* refs/pull/56566/head:
qa: add missing pg_health fragment links in fs suite
qa: ignore PG health warnings in CephFS QA

Reviewed-by: Samuel Just <sjust@redhat.com>
15 months agoMerge PR #56529 into main
Patrick Donnelly [Tue, 2 Apr 2024 22:28:53 +0000 (18:28 -0400)]
Merge PR #56529 into main

* refs/pull/56529/head:
mds: remove unnecssary quiesce finisher variable
mds: attach quiesce_path mdr to finisher at creation not dispatch

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
15 months agoMerge PR #55647 into main
Patrick Donnelly [Tue, 2 Apr 2024 22:26:30 +0000 (18:26 -0400)]
Merge PR #55647 into main

* refs/pull/55647/head:
mds: use cephfs errno for fragment size check

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
15 months agoMerge pull request #56617 from adk3798/wip-revert-55346
Adam King [Tue, 2 Apr 2024 21:59:22 +0000 (17:59 -0400)]
Merge pull request #56617 from adk3798/wip-revert-55346

Revert "mgr: use un-deprecated APIs to initialize Python interpretor"

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
15 months agodoc: fixing doc/cephfs/fs-volumes 56228/head
neeraj pratap singh [Fri, 15 Mar 2024 17:36:18 +0000 (23:06 +0530)]
doc: fixing doc/cephfs/fs-volumes

Fixes: https://tracker.ceph.com/issues/65267
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
15 months agoMerge pull request #55976 from cbodley/wip-qa-quincy-upgrade-distro
Casey Bodley [Tue, 2 Apr 2024 13:14:32 +0000 (14:14 +0100)]
Merge pull request #55976 from cbodley/wip-qa-quincy-upgrade-distro

qa/upgrade/quincy: bump ubuntu from 20->22

Reviewed-by: Adam King <adking@redhat.com>
15 months agoRevert "Merge pull request #55436 from tchaikov/mgr-python-3.12" 56617/head
Adam King [Tue, 2 Apr 2024 12:47:22 +0000 (08:47 -0400)]
Revert "Merge pull request #55436 from tchaikov/mgr-python-3.12"

This reverts commit 8dffa8707dc665eb7ca6e2f48fa34e0fa8ac5b51, reversing
changes made to 80374da12bcc708d1fa306484842b3f97ac42acb.

This commit broke the import of the "mgr_module" module
within the python modules in the mgr at least on python 3.6.8
that we currently use in our centos 8 stream based containers

Failures would look like (removing beginning of log lines)

Loading python module 'alerts'
Module not found: 'mgr_module'
Class not found in module 'alerts'
Error loading module 'alerts': (22) Invalid argument

Signed-off-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #56279 from ceph/long_processing_timeout
Casey Bodley [Tue, 2 Apr 2024 12:38:33 +0000 (13:38 +0100)]
Merge pull request #56279 from ceph/long_processing_timeout

rgw/s3select: s3select response handler refactor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agoMerge pull request #56614 from zdover23/wip-doc-2024-03-30-mgr-zabbix
Zac Dover [Tue, 2 Apr 2024 11:21:15 +0000 (21:21 +1000)]
Merge pull request #56614 from zdover23/wip-doc-2024-03-30-mgr-zabbix

doc/mgr: update zabbix information

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agotest/osd/hitset: free allocated HitSet 56625/head
Kefu Chai [Tue, 2 Apr 2024 06:51:35 +0000 (14:51 +0800)]
test/osd/hitset: free allocated HitSet

we allocate a hitset without freeing it in this test, and LeakSanitizer
points this out
```
Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x557a0841a9dd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_hitset+0x1ae9dd) (BuildId: ad9be2b52b3d6fb1a567b262c3becaab6373e88d)
    #1 0x557a0843b98e in ExplicitHashHitSetTest::ExplicitHashHitSetTest() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/hitset.cc:128:46
    #2 0x557a0843b918 in ExplicitHashHitSetTest_Construct_Test::ExplicitHashHitSetTest_Construct_Test() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/hitset.cc:133:1
    #3 0x557a0843b8cb in testing::internal::TestFactoryImpl<ExplicitHashHitSetTest_Construct_Test>::CreateTest() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
    #4 0x557a08532406 in testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #5 0x557a084eb892 in testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #6 0x557a0849dd55 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2848:22
    #7 0x557a0849f3bb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #8 0x557a084bc848 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #9 0x557a0853a6d6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #10 0x557a084f1222 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #11 0x557a084bbbd2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #12 0x557a08441c10 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #13 0x557a08441ba1 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #14 0x7faa493d6d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we just free it in the dtor. this should address
the warning from the sanitizer.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agomgr/dashboard: replace deprecated table panel in grafana with a newer 56509/head
Aashish Sharma [Wed, 27 Mar 2024 08:16:24 +0000 (13:46 +0530)]
mgr/dashboard: replace deprecated table panel in grafana with a newer
table panel

Fixes: https://tracker.ceph.com/issues/65174
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
15 months agoMerge pull request #53679 from rhcs-dashboard/alerts-v2-api
Aashish Sharma [Tue, 2 Apr 2024 04:22:01 +0000 (09:52 +0530)]
Merge pull request #53679 from rhcs-dashboard/alerts-v2-api

mgr/dashboard: start using alertmanager v2

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
15 months agoMerge pull request #56596 from tchaikov/wip-on-exit-fix-leaks
Kefu Chai [Tue, 2 Apr 2024 00:43:42 +0000 (08:43 +0800)]
Merge pull request #56596 from tchaikov/wip-on-exit-fix-leaks

test/on_exit: use static variables for on_exit hooks

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
15 months agoMerge pull request #55979 from athanatos/sjust/wip-64546-max-creating-pgs
Samuel Just [Mon, 1 Apr 2024 20:18:15 +0000 (13:18 -0700)]
Merge pull request #55979 from athanatos/sjust/wip-64546-max-creating-pgs

mon: always create pgs in the epoch in which the pool was created

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
15 months agoMerge pull request #56533 from phlogistonjohn/jjm-cpatch-before
Adam King [Mon, 1 Apr 2024 15:32:36 +0000 (11:32 -0400)]
Merge pull request #56533 from phlogistonjohn/jjm-cpatch-before

cpatch: add --run-before command line option

Reviewed-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #56136 from NitzanMordhai/wip-nitzan-init-hash-in-chunk-ref-by...
Yuri Weinstein [Mon, 1 Apr 2024 15:32:12 +0000 (08:32 -0700)]
Merge pull request #56136 from NitzanMordhai/wip-nitzan-init-hash-in-chunk-ref-by-hash

cls/cas/cls_cas_internal: Initialize 'hash' value before decoding

Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>
15 months agoMerge pull request #54991 from ifed01/wip-ifed-some-recover-stuff
Yuri Weinstein [Mon, 1 Apr 2024 15:29:57 +0000 (08:29 -0700)]
Merge pull request #54991 from ifed01/wip-ifed-some-recover-stuff

tools/ceph-objectstore-tool: tricks to tolerate disk errors for "pg export" command.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
15 months agodoc/mgr: update zabbix information 56614/head
Zac Dover [Fri, 29 Mar 2024 11:06:28 +0000 (21:06 +1000)]
doc/mgr: update zabbix information

Include information from John Jasen about how to install Zabbix.

Signed-off-by: Zac Dover <zac.dover@proton.me>
15 months agoMerge pull request #56512 from rhcs-dashboard/remove-token-authentication-multi-cluster
Aashish Sharma [Mon, 1 Apr 2024 14:00:56 +0000 (19:30 +0530)]
Merge pull request #56512 from rhcs-dashboard/remove-token-authentication-multi-cluster

mgr/dashboard: Remove token authentication multi cluster

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agomgr/dashboard: remove the option for authentication with token in 56512/head
Aashish Sharma [Wed, 27 Mar 2024 09:45:59 +0000 (15:15 +0530)]
mgr/dashboard: remove the option for authentication with token in
multi-cluster form

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
15 months agoMerge pull request #55860 from ceph/add-multi-cluster-token-ttl
Aashish Sharma [Mon, 1 Apr 2024 07:50:37 +0000 (13:20 +0530)]
Merge pull request #55860 from ceph/add-multi-cluster-token-ttl

mgr/dashboard: Add multi cluster token ttl

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56600 from zdover23/wip-doc-2024-04-01-dev-essentials-leads-list
Zac Dover [Mon, 1 Apr 2024 01:36:42 +0000 (11:36 +1000)]
Merge pull request #56600 from zdover23/wip-doc-2024-04-01-dev-essentials-leads-list

doc/dev: update leads list

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agodoc/dev: update leads list 56600/head
Zac Dover [Mon, 1 Apr 2024 01:17:39 +0000 (11:17 +1000)]
doc/dev: update leads list

Remove Joao Luis from the list of component leads.

Signed-off-by: Zac Dover <zac.dover@proton.me>
15 months agorefactor of s3select response handler, adding continuation-response to parquet flow... 56279/head
Gal Salomon [Tue, 19 Mar 2024 11:45:58 +0000 (13:45 +0200)]
refactor of s3select response handler, adding continuation-response to parquet flow, refacor of error response

bug fix (init_success_response).
s3select submodule

fix for json-error-flow

Signed-off-by: Gal Salomon <gal.salomon@gmail.com>
15 months agotest/on_exit: use static variables for on_exit hooks 56596/head
Kefu Chai [Sun, 31 Mar 2024 01:14:21 +0000 (09:14 +0800)]
test/on_exit: use static variables for on_exit hooks

before this change, we allocate memory chunks using malloc(), but
we never free them. and LeakSanitizer points this out

```
Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x5588bfe532de in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_on_exit+0xa52de) (BuildId: 7c7a92bf5719592938c5307214bcd9b080bd847f)
    #1 0x5588bfe911d7 in func_scope() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:33:22
    #2 0x5588bfe90804 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:64:3
    #3 0x7f23081c1d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x5588bfe532de in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_on_exit+0xa52de) (BuildId: 7c7a92bf5719592938c5307214bcd9b080bd847f)
    #1 0x5588bfe91160 in func_scope() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:29:22
    #2 0x5588bfe90804 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:64:3
    #3 0x7f23081c1d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, instead of allocating the variables using `malloc()`,
we keep them in static variables, so that they can be accessed by
`OnExitManager` even if it is a static variable.
with this change, the memory leak reports for this source file go away.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agoMerge pull request #56591 from tchaikov/wip-unittest_rocksdb_option-fix-leak
Kefu Chai [Sun, 31 Mar 2024 08:32:09 +0000 (16:32 +0800)]
Merge pull request #56591 from tchaikov/wip-unittest_rocksdb_option-fix-leak

tests: unittest_rocksdb_option: do not leak RocksDBStore

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
15 months agoMerge pull request #56592 from nh2/crush-location-doc-fixes
Zac Dover [Sun, 31 Mar 2024 00:08:50 +0000 (10:08 +1000)]
Merge pull request #56592 from nh2/crush-location-doc-fixes

doc/rados/operations: Improve crush_location docs

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>
15 months agodoc/rados/operations: Improve crush_location docs 56592/head
Niklas Hambüchen [Sat, 30 Mar 2024 16:42:48 +0000 (17:42 +0100)]
doc/rados/operations: Improve crush_location docs

* Fix incorrect syntax
* Use underscores for config options, like other ceph docs did
* Fix incorrect statement that crush_location_hook adds fiels; it replaces
* Explain `root=default host=HOSTNAME` is not set if `crush_location` is given
* Remove duplication across sections
* Point out that `root=default` is important

Signed-off-by: Niklas Hambüchen <mail@nh2.me>
15 months agoMerge pull request #56572 from ceph/wip-yuriw-release-process-main
Yuri Weinstein [Sat, 30 Mar 2024 14:57:37 +0000 (07:57 -0700)]
Merge pull request #56572 from ceph/wip-yuriw-release-process-main

doc: added jammy to the supported distro list for quincy

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agotests: unittest_rocksdb_option: do not leak RocksDBStore 56591/head
Kefu Chai [Sat, 30 Mar 2024 13:21:25 +0000 (21:21 +0800)]
tests: unittest_rocksdb_option: do not leak RocksDBStore

before this change, we allocate an instance of `RocksDBStore` with
`new`, but we never free it. and LeanSanitizer points this out:

```
Direct leak of 952 byte(s) in 1 object(s) allocated from:
    #0 0x55f31440bc2d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_rocksdb_option+0xaebc2d) (BuildId: 81b849dbc41cbc6b05d5e603d9ba8a002dab2d24)
    #1 0x55f3144132fd in RocksDBOption_simple_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/TestRocksdbOptionParse.cc:17:22
    #2 0x55f3144ecf26 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x55f3144a4312 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x55f314453ccc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x55f314455d02 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x55f31445733b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x55f3144747c8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x55f3144f5576 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x55f3144ab1a2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x55f314473b52 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x55f31440f690 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x55f31440e4c3 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/unit.cc:45:10
    #13 0x7f0d32551d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we manage the life cycle of `RocksDBStore` using
a smart pointer. this should address the leak.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agoMerge pull request #56587 from tchaikov/wip-bluefs-use-after-free
Kefu Chai [Sat, 30 Mar 2024 11:47:19 +0000 (19:47 +0800)]
Merge pull request #56587 from tchaikov/wip-bluefs-use-after-free

test/objectstore/test_bluefs: fix heap-use-after-free

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
15 months agoMerge pull request #56581 from zdover23/wip-doc-2024-03-29-dev-essentials-leads-list
Zac Dover [Sat, 30 Mar 2024 11:05:10 +0000 (21:05 +1000)]
Merge pull request #56581 from zdover23/wip-doc-2024-03-29-dev-essentials-leads-list

doc/dev: update leads list

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agoMerge pull request #55436 from tchaikov/mgr-python-3.12
Kefu Chai [Sat, 30 Mar 2024 05:40:49 +0000 (13:40 +0800)]
Merge pull request #55436 from tchaikov/mgr-python-3.12

mgr: use un-deprecated APIs to initialize Python interpretor

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
15 months agotest/objectstore/test_bluefs: fix heap-use-after-free 56587/head
Kefu Chai [Sat, 30 Mar 2024 04:43:56 +0000 (12:43 +0800)]
test/objectstore/test_bluefs: fix heap-use-after-free

in BlueFS.test_shared_alloc, we keep the return value of
`fs.get_perf_counters()`, and deference it after umounting the fs,
but the `PerfCounters*` pointer returned from `fs.get_perf_counters()`
is destroyed in `BlueFS::_shutdown_logger()` which is in turn called
by `BlueFS::umount()`. so ASan points this out:
```
==1662613==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110000b2d80 at pc 0x7f0eefc30644 bp 0x7ffcdbab6430 sp 0x7ffcdbab6428
READ of size 8 at 0x6110000b2d80 thread T0
    #0 0x7f0eefc30643 in ceph::common::PerfCounters::get(int) const /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:246:8
    #1 0x557595ddfc15 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1182:3
    #2 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #13 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #14 0x7f0eed083e3f in __libc_start_main csu/../csu/libc-start.c:392:3
    #15 0x557595cd46a4 in _start (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x2856a4) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)

0x6110000b2d80 is located 0 bytes inside of 208-byte region [0x6110000b2d80,0x6110000b2e50)
freed by thread T0 here:
    #0 0x557595d92b1d in operator delete(void*) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x343b1d) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
    #1 0x557595f31c43 in BlueFS::_shutdown_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:462:3
    #2 0x557595f54ab5 in BlueFS::umount(bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:1076:3
    #3 0x557595ddfbd7 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1180:6
    #4 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #5 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #6 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #7 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #8 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #9 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #10 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #13 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #14 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #15 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

previously allocated by thread T0 here:
    #0 0x557595d922bd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x3432bd) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
    #1 0x7f0eefc33180 in ceph::common::PerfCountersBuilder::PerfCountersBuilder(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:537:21
    #2 0x557595f30ac9 in BlueFS::_init_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:221:23
    #3 0x557595f42bc6 in BlueFS::mount() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:977:3
    #4 0x557595ddd339 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1139:3
    #5 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #6 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #7 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #8 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #9 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #10 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #11 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #12 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #13 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #14 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #15 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #16 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, instead of keeping `logger` across the `umount()` and
`mount()` calls, we get another instance of `logger`, query it for
the perf counter that we are interested, and compare the value
to see if it is unchanged.

this should address the ASan warning above.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agoMerge pull request #56544 from baum/fix_f68248944815acbf5fb08ef66f36da6d5baef824
Kefu Chai [Sat, 30 Mar 2024 01:30:02 +0000 (09:30 +0800)]
Merge pull request #56544 from baum/fix_f68248944815acbf5fb08ef66f36da6d5baef824

pybind: support RHEL8, whose default python is python 3.6.8

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
15 months agoqa: add missing pg_health fragment links in fs suite 56566/head
Patrick Donnelly [Thu, 28 Mar 2024 20:43:03 +0000 (16:43 -0400)]
qa: add missing pg_health fragment links in fs suite

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoqa: ignore PG health warnings in CephFS QA
Patrick Donnelly [Thu, 28 Mar 2024 20:31:33 +0000 (16:31 -0400)]
qa: ignore PG health warnings in CephFS QA

Fixes: https://tracker.ceph.com/issues/51282
Fixes: https://tracker.ceph.com/issues/65018
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge pull request #56584 from phlogistonjohn/jjm-rpmbuild-fix
Dan Mick [Fri, 29 Mar 2024 18:14:19 +0000 (11:14 -0700)]
Merge pull request #56584 from phlogistonjohn/jjm-rpmbuild-fix

Merged without waiting for tests.  Obvious fix.

15 months agoceph.spec.in: remove command-with-macro line 56584/head
John Mulligan [Fri, 29 Mar 2024 18:04:33 +0000 (14:04 -0400)]
ceph.spec.in: remove command-with-macro line

A comment clearly left as a breadcrumb for a node-proxy manpage is
causing (intermittent) build failures. Remove the line and hope
the manpage is added if/when appropriate.

Fixes: 0dd73643649ddc2366e60de4fe6c078b6e112091
Signed-off-by: John Mulligan <jmulligan@redhat.com>
15 months agoqa/upgrade/quincy: bump ubuntu from 20->22 55976/head
Casey Bodley [Tue, 5 Mar 2024 19:42:24 +0000 (14:42 -0500)]
qa/upgrade/quincy: bump ubuntu from 20->22

Signed-off-by: Casey Bodley <cbodley@redhat.com>
15 months agoMerge pull request #56540 from tchaikov/wip-fix-test-leaks-ceph-context
Kefu Chai [Fri, 29 Mar 2024 16:10:20 +0000 (00:10 +0800)]
Merge pull request #56540 from tchaikov/wip-fix-test-leaks-ceph-context

test: : do not increase ref when creating intrusive_ptr<CephContext>

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agopybind: use syntax supported by python3.6 56544/head
Alexander Indenbaum [Thu, 28 Mar 2024 07:20:11 +0000 (07:20 +0000)]
pybind: use syntax supported by python3.6

Fixes: f68248944815acbf5fb08ef66f36da6d5baef824
It seems the shaman CentOS8 builders, which serve as the base for container images, aren't current with the ":=" syntax. Amend to support RHEL8, whose default python is python 3.6.8.

Error example from https://shaman.ceph.com/builds/ceph/ceph-nvmeof-mon/27188b78bdbaa3c33de518b8514906de629e774a/default/388913/

  File "/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2505-g27188b78/rpm/el8/BUILD/ceph-19.0.0-2505-g27188b78/src/pybind/cephfs/setup.py", line 120
    if ldflags := os.environ.get('LDFLAGS'):
                ^
SyntaxError: invalid syntax

Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>
15 months agodoc: added jammy to the supported distro list for quincy 56572/head
Yuri Weinstein [Thu, 28 Mar 2024 22:57:34 +0000 (15:57 -0700)]
doc: added jammy to the supported distro list for quincy

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
15 months agodoc/dev: update leads list 56581/head
Zac Dover [Fri, 29 Mar 2024 11:14:21 +0000 (21:14 +1000)]
doc/dev: update leads list

Update the list of component leads in
doc/dev/developer_guide/essentials.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
15 months agomgr/dashboard: allow the user to add ttl for the connected cluster's 55860/head
Aashish Sharma [Thu, 29 Feb 2024 16:16:23 +0000 (21:46 +0530)]
mgr/dashboard: allow the user to add ttl for the connected cluster's
token expiration

Fixes: https://tracker.ceph.com/issues/65055
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
15 months agoMerge pull request #56190 from rhcs-dashboard/disable-multi-cluster-non-hub
Aashish Sharma [Fri, 29 Mar 2024 05:39:06 +0000 (11:09 +0530)]
Merge pull request #56190 from rhcs-dashboard/disable-multi-cluster-non-hub

mgr/dashboard: disable multi-cluster feature for non-hub clusters

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agoMerge pull request #54633 from joscollin/wip-F61334-use-snapdiff-api
Venky Shankar [Fri, 29 Mar 2024 05:36:35 +0000 (11:06 +0530)]
Merge pull request #54633 from joscollin/wip-F61334-use-snapdiff-api

cephfs_mirror: use snapdiff api for incremental syncing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
15 months agoMerge pull request #54301 from lxbsz/wip-sparse-read-test
Xiubo Li [Fri, 29 Mar 2024 04:26:26 +0000 (12:26 +0800)]
Merge pull request #54301 from lxbsz/wip-sparse-read-test

ceph_test_rados_api_io_pp: add multiple extents test for sparse-read

15 months agoMerge pull request #56570 from batrick/release-checklist-docs-2
Ilya Dryomov [Thu, 28 Mar 2024 23:06:07 +0000 (00:06 +0100)]
Merge pull request #56570 from batrick/release-checklist-docs-2

doc: reconcile release checklist

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
15 months agodoc: reconcile release checklist 56570/head
Patrick Donnelly [Thu, 28 Mar 2024 22:35:40 +0000 (18:35 -0400)]
doc: reconcile release checklist

Updating the releases documentation existed already but that section should be
moved to after dev freeze when the branch is forked.

Fixes: 7b5017913a9ecf27ca7281367a5472a9b9b043ec
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge pull request #56568 from batrick/release-checklist-docs
Ilya Dryomov [Thu, 28 Mar 2024 22:11:42 +0000 (23:11 +0100)]
Merge pull request #56568 from batrick/release-checklist-docs

doc: update release checklist

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
15 months agodoc: update release checklist 56568/head
Patrick Donnelly [Thu, 28 Mar 2024 21:06:41 +0000 (17:06 -0400)]
doc: update release checklist

To include note that release notes must be removed after forking the release
branch.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agomsg: trivial destructor should be default 56565/head
Patrick Donnelly [Mon, 11 Dec 2023 20:27:58 +0000 (15:27 -0500)]
msg: trivial destructor should be default

See: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rc-five

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge PR #55337 into main
Patrick Donnelly [Thu, 28 Mar 2024 19:10:15 +0000 (15:10 -0400)]
Merge PR #55337 into main

* refs/pull/55337/head:
mds: explain clients retaining Fb during xlock

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
15 months agoMerge PR #56536 into main
Patrick Donnelly [Thu, 28 Mar 2024 18:40:39 +0000 (14:40 -0400)]
Merge PR #56536 into main

* refs/pull/56536/head:
mds: do not dispatch aborted internal requests
qa: add test for killing quiesce_inode with outstanding remote authpin requests
mds: add lockup debugging command

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
15 months agoMerge pull request #56534 from athanatos/sjust/wip-64975-crimson-scrub-snapmapper-2
Samuel Just [Thu, 28 Mar 2024 16:57:45 +0000 (09:57 -0700)]
Merge pull request #56534 from athanatos/sjust/wip-64975-crimson-scrub-snapmapper-2

crimson: introduce INTERNAL_PG_LOCAL_NS, use for snapmapper

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
15 months agoMerge pull request #56516 from afreen23/wip-ceph-logo
Nizamudeen A [Thu, 28 Mar 2024 15:23:09 +0000 (20:53 +0530)]
Merge pull request #56516 from afreen23/wip-ceph-logo

mgr/dashboard: make ceph logo redirect to dashboard

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56543 from jrchyang/main
Igor Fedotov [Thu, 28 Mar 2024 09:13:54 +0000 (12:13 +0300)]
Merge pull request #56543 from jrchyang/main

os/bluestore: remove unnecessary std::hex in dout

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
15 months agoMerge pull request #56504 from rhcs-dashboard/replace-active_mds-active_nfs
Nizamudeen A [Thu, 28 Mar 2024 08:20:30 +0000 (13:50 +0530)]
Merge pull request #56504 from rhcs-dashboard/replace-active_mds-active_nfs

mgr/dashboard: s/active_mds/active_nfs in fs attach form

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agoMerge pull request #56468 from rhcs-dashboard/clone-name-exists-valid-fix
Nizamudeen A [Thu, 28 Mar 2024 08:19:59 +0000 (13:49 +0530)]
Merge pull request #56468 from rhcs-dashboard/clone-name-exists-valid-fix

mgr/dashboard: fix clone unique validator for name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agoMerge pull request #56470 from rhcs-dashboard/fix-full-name-validation-rgw
Nizamudeen A [Thu, 28 Mar 2024 08:19:50 +0000 (13:49 +0530)]
Merge pull request #56470 from rhcs-dashboard/fix-full-name-validation-rgw

mgr/dashboard: fix rgw display name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agoos/bluestore: remove unnecessary std::hex in dout 56543/head
jrchyang [Thu, 28 Mar 2024 07:16:58 +0000 (07:16 +0000)]
os/bluestore: remove unnecessary std::hex in dout

Signed-off-by: Jrchyang Yu <yuzhiqiang_yewu@cmss.chinamobile.com>
15 months agotest: : do not increase ref when creating intrusive_ptr<CephContext> 56540/head
Kefu Chai [Thu, 28 Mar 2024 02:01:46 +0000 (10:01 +0800)]
test: : do not increase ref when creating intrusive_ptr<CephContext>

before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance. and LeakSanitizer points this out:
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x558d341d837d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_ipaddr+0x19b37d) (BuildId: 1b7e7e5abfc2b58ce2334712e4c00b2441c25870)
    #1 0x7fd74c957559 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7fd74c956933 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7fd74c956933 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7fd74c956690 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7fd74c950562 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circ
ular_buffer/base.hpp:1039:9
    #6 0x7fd74c9435b4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7fd74bc1891d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7fd74bc178eb in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x558d341f97e9 in pick_address_filtering_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_ipaddr.cc:774:47
    #10 0x558d3430c4f6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x558d342c3fc2 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x558d342749dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #13 0x558d34276a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #14 0x558d3427804b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #15 0x558d342954d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #16 0x558d34314d26 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #17 0x558d342ca932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #18 0x558d34294862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #19 0x558d34218d80 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #20 0x558d34218d11 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #21 0x7fd749331d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so, in this change, we do not increase the refcount when creating cct.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agomds: do not dispatch aborted internal requests 56536/head
Patrick Donnelly [Wed, 27 Mar 2024 16:03:12 +0000 (12:03 -0400)]
mds: do not dispatch aborted internal requests

Matching the behavior of Server::dispatch_client_request.

This logic can be unified in a future refactor but keeping this as a targetted
fix for the purposes of the bug.

Fixes: https://tracker.ceph.com/issues/65182
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoqa: add test for killing quiesce_inode with outstanding remote authpin requests
Patrick Donnelly [Wed, 27 Mar 2024 17:27:23 +0000 (13:27 -0400)]
qa: add test for killing quiesce_inode with outstanding remote authpin requests

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge pull request #56241 from tchaikov/wip-cmake-enable-sanitizers
Kefu Chai [Wed, 27 Mar 2024 23:16:06 +0000 (07:16 +0800)]
Merge pull request #56241 from tchaikov/wip-cmake-enable-sanitizers

cmake: enable building with Sanitizers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agomds: add lockup debugging command
Patrick Donnelly [Wed, 27 Mar 2024 16:24:28 +0000 (12:24 -0400)]
mds: add lockup debugging command

To artificially make the rank unresponsive for a time.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge pull request #56378 from guojidan/snap-stats
Samuel Just [Wed, 27 Mar 2024 21:03:06 +0000 (14:03 -0700)]
Merge pull request #56378 from guojidan/snap-stats

crimson/osd/osd_operations/snaptrim_event: update PG's stats

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
15 months agoscript/cpatch.py: run container build with host networking 56533/head
John Mulligan [Wed, 27 Mar 2024 18:32:10 +0000 (14:32 -0400)]
script/cpatch.py: run container build with host networking

Applies host networking to building the image. This is handy when using
custom networking hacks that redirect repos, etc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
15 months agoscript/cpatch.py: add a --run-before argument to run commands
John Mulligan [Wed, 27 Mar 2024 18:30:01 +0000 (14:30 -0400)]
script/cpatch.py: add a --run-before argument to run commands

Add a `--run-before` command line argument to cpatch.py that allows
commands to be executed before doing any other layer generation.
This supports actions like customizing packages installed in the
image. Something that can be handy when developing new features.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
15 months agomds: remove unnecssary quiesce finisher variable 56529/head
Patrick Donnelly [Wed, 27 Mar 2024 16:05:44 +0000 (12:05 -0400)]
mds: remove unnecssary quiesce finisher variable

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agomds: attach quiesce_path mdr to finisher at creation not dispatch
Patrick Donnelly [Wed, 27 Mar 2024 16:04:57 +0000 (12:04 -0400)]
mds: attach quiesce_path mdr to finisher at creation not dispatch

No functional difference but this is cleaner.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge pull request #56274 from phlogistonjohn/jjm-mypy-bump
Adam King [Wed, 27 Mar 2024 15:18:16 +0000 (11:18 -0400)]
Merge pull request #56274 from phlogistonjohn/jjm-mypy-bump

python: mypy version bump

Reviewed-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #56304 from phlogistonjohn/jjm-tox-default-envs
Adam King [Wed, 27 Mar 2024 14:51:09 +0000 (10:51 -0400)]
Merge pull request #56304 from phlogistonjohn/jjm-tox-default-envs

python: allow unit tests to use tox default envs

Reviewed-by: Adam King <adking@redhat.com>
15 months agomgr/dashboard: disable multi-cluster feature for non-hub clusters 56190/head
Aashish Sharma [Thu, 14 Mar 2024 11:57:29 +0000 (17:27 +0530)]
mgr/dashboard: disable multi-cluster feature for non-hub clusters

Fixes: https://tracker.ceph.com/issues/65056
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
15 months agomgr/dashboard: make ceph logo redirect to dashboard 56516/head
Afreen [Wed, 27 Mar 2024 10:01:12 +0000 (15:31 +0530)]
mgr/dashboard: make ceph logo redirect to dashboard

Fixes https://tracker.ceph.com/issues/64734

Clicking on ceph logo in navigation takes back to overview dashboard.

Signed-off-by: Afreen <afreen23.git@gmail.com>
15 months agoMerge pull request #56449 from afreen23/wip-security
Nizamudeen A [Wed, 27 Mar 2024 07:28:23 +0000 (12:58 +0530)]
Merge pull request #56449 from afreen23/wip-security

mgr/dashboard:Update encryption and tags in bucket form

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agomgr/dashboard: s/active_mds/active_nfs in fs attach form 56504/head
Nizamudeen A [Wed, 27 Mar 2024 05:12:39 +0000 (10:42 +0530)]
mgr/dashboard: s/active_mds/active_nfs in fs attach form

Fixes: https://tracker.ceph.com/issues/65173
Signed-off-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56239 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard...
Nizamudeen A [Wed, 27 Mar 2024 04:39:20 +0000 (10:09 +0530)]
Merge pull request #56239 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-follow-redirects-1.15.6

mgr/dashboard: bump follow-redirects from 1.15.3 to 1.15.6 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56463 from rhcs-dashboard/cephfs-name-validation
Nizamudeen A [Wed, 27 Mar 2024 04:25:14 +0000 (09:55 +0530)]
Merge pull request #56463 from rhcs-dashboard/cephfs-name-validation

mgr/dashboard: fix cephfs name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agocrimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run 56534/head
Samuel Just [Tue, 26 Mar 2024 06:17:10 +0000 (23:17 -0700)]
crimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run

Fixes: https://tracker.ceph.com/issues/64975
Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/.../scrub_events: convert ScrubScan::run to coroutine
Samuel Just [Tue, 26 Mar 2024 04:20:48 +0000 (21:20 -0700)]
crimson/.../scrub_events: convert ScrubScan::run to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run
Samuel Just [Wed, 27 Mar 2024 01:43:06 +0000 (01:43 +0000)]
crimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/.../scrub_events: convert ScrubFindRange::run to a coroutine
Samuel Just [Wed, 27 Mar 2024 01:39:17 +0000 (01:39 +0000)]
crimson/.../scrub_events: convert ScrubFindRange::run to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocommon/hobject: make hobject_t formatter methods const
Samuel Just [Tue, 26 Mar 2024 06:16:31 +0000 (23:16 -0700)]
common/hobject: make hobject_t formatter methods const

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson: move snapmapper object into INTERNAL_PG_LOCAL_NS
Samuel Just [Tue, 26 Mar 2024 05:44:37 +0000 (22:44 -0700)]
crimson: move snapmapper object into INTERNAL_PG_LOCAL_NS

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects
Samuel Just [Tue, 26 Mar 2024 05:36:07 +0000 (22:36 -0700)]
crimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine
Samuel Just [Tue, 26 Mar 2024 05:31:08 +0000 (05:31 +0000)]
crimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads
Samuel Just [Tue, 26 Mar 2024 05:13:40 +0000 (22:13 -0700)]
crimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agoMerge pull request #56384 from Svelar/drop_fmt_version
Rongqi Sun [Wed, 27 Mar 2024 02:36:23 +0000 (10:36 +0800)]
Merge pull request #56384 from Svelar/drop_fmt_version

run-make: do not pass '-DWITH_FMT_VERSION' to cmake

15 months agoqa/lsan.supp: suppress MallocExtension::Initialize 56241/head
Kefu Chai [Sun, 24 Mar 2024 16:15:15 +0000 (00:15 +0800)]
qa/lsan.supp: suppress MallocExtension::Initialize

LeakSanitizer reports
```
==688591==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 45 byte(s) in 1 object(s) allocated from:
    #0 0x55f8dd9969dd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_fastbmap_allocator+0x1f89dd) (BuildId: cac39eac8ef1e8774f9dd48e6e3f677fdd864776)
    #1 0x55f8dd99c730 in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x55f8dd99c690 in std::allocator<char>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x55f8dd99c690 in std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    #4 0x55f8dd99c393 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:153:14
    #5 0x55f8dda96a6c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:307:21
    #6 0x55f8dda96852 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:395:8
    #7 0x7f4a751ab6f0 in
    MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId:
    eeef3d1257388a806e122398dbce3157ee568ef4)
```

this is a global object allocated by the allocator, so we can suppress this report.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agocmake: suppress LeakSanitizer reports of known leaks
Kefu Chai [Sun, 24 Mar 2024 15:20:38 +0000 (23:20 +0800)]
cmake: suppress LeakSanitizer reports of known leaks

there are known leaks, which are tracked by qa/lsan.suppr, in Ceph.
so let's reuse it so we don't see them when running unit test with
ASan enabled.

see also https://clang.llvm.org/docs/AddressSanitizer.html#issue-suppression

Signed-off-by: Kefu Chai <tchaikov@gmail.com>