]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
15 months agoRGW: move CacheDriver instance to D4N Filter
Samarah [Mon, 19 Jun 2023 18:00:16 +0000 (14:00 -0400)]
RGW: move CacheDriver instance to D4N Filter

Signed-off-by: Samarah <samarah.uriarte@ibm.com>
15 months agoRGW: Fix rgw crash issue
Samarah [Mon, 19 Jun 2023 15:58:36 +0000 (11:58 -0400)]
RGW: Fix rgw crash issue

Signed-off-by: Samarah <samarah.uriarte@ibm.com>
15 months agoRGW: Fix find_client method; rgw crash issue
Samarah [Mon, 19 Jun 2023 13:44:01 +0000 (09:44 -0400)]
RGW: Fix find_client method; rgw crash issue

Signed-off-by: Samarah <samarah.uriarte@ibm.com>
15 months agorgw/d4n: This commit squashes the following commits related to
Samarah [Thu, 25 May 2023 18:58:26 +0000 (14:58 -0400)]
rgw/d4n: This commit squashes the following commits related to
d4n policy and filter driver.

RGW: Add cache driver base class - add d4n filter driver.
RGW: Update cache and policy files; add RedisDriver into D4N filter
RGW: Add D4N classes and unit testing; update cpp_redis submodule
QA: Add D4N teuthology suite
RGW: Improve D4N readability and structure
RGW: Add base D4N policy and test
RGW: Add GWF policy to D4N
RGW: Add global weight property
RGW: Added D4N namespace
RGW: Update policy driver interface
RGW: Update unit tests
RGW: Add Address struct to all files
RGW: Update D4N names and structure
RGW: Update structure and fix errors
RGW: Add more features to policy driver
RGW: Access local weight in policy code
RGW: Work on D4N workflows
RGW: Add object directory class and remove copy_value method
RGW: Establish object directory in read workflow
RGW: Update cache and policy files; add RedisDriver into D4N filter
RGW: Switch out D4N cache methods with Redis driver methods
RGW: Update RedisDriver to match new CacheDriver structure; define set_attrs method
RGW: Fix D4N read workflow crashes
RGW: Update D4N files to match CacheDriver changes
RGW: fix key_exists method for RedisDriver and clean up rgw_sal_d4n.cc
RGW: Use correct get_block method
RGW: Make CachePolicy a virtual class
RGW: Initialize localWeight if not found and develop find_victim method
RGW: Debugging network failure
RGW: Rebase and debugging network failure
RGW: Update RedisDriver::list_entries and usage in D4N policy driver
RGW: Fix network failure issue; add entries and entry methods
RGW: Update D4N test files to match rebase
RGW: Update D4N policy and RedisDriver with entries
RGW: make localWeight an attribute
rgw/cache: commit to fix compilation issues.

Signed-off-by: Samarah <samarah.uriarte@ibm.com>
15 months agorgw/cache: this commit squashes the following commits for ssd backed cache driver.
Pritha Srivastava [Mon, 5 Jun 2023 08:04:32 +0000 (13:34 +0530)]
rgw/cache: this commit squashes the following commits for ssd backed cache driver.

initial implementation of ssd backed cache driver.This contains implementations
for methods related to Entry and Partition.

adding methods for put, get and delete data.
They are all synchronous.

added CacheAioRequest to handle async read and write
requests and added a corresponding read op in aio.h and aio.cc.

added get_async method implementation and
initial implementation to handle async reads using asio.

changing the signature of list_entries to match that with
base class and making it public.

adding implementation for asynchronous get.

modifying namespace from rgw::cal to rgw::cache.
Modifying the SSDDriver constructor for Partition related
modifications.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
15 months agoRGW: add entries and entry methods
Samarah [Tue, 13 Jun 2023 16:33:18 +0000 (12:33 -0400)]
RGW: add entries and entry methods

Signed-off-by: Samarah <samarah.uriarte@ibm.com>
15 months agorgw/cache: This commit squashes the following commits related to list_entries cache...
Samarah [Thu, 8 Jun 2023 17:10:58 +0000 (13:10 -0400)]
rgw/cache: This commit squashes the following commits related to list_entries cache api.

RGW: Add list_entries method to CacheDriver and to Redis driver.
RGW: Change CacheDriver::list_entries output type
RGW: Update RedisDriver::list_entries and usage in D4N policy driver

Signed-off-by: Samarah <samarah.uriarte@ibm.com>
15 months agorgw/cache: This commit squashes the following commits for redis driver.
Samarah [Fri, 22 Dec 2023 11:10:13 +0000 (16:40 +0530)]
rgw/cache: This commit squashes the following commits for redis driver.

RGW: fix key_exists method for RedisDriver and clean up rgw_sal_d4n.cc
RGW: Implement RedisDriver::get_free_space
rgw/cache: modifying namespace from rgw::cal to rgw::cache.
RGW: Update D4N files to match CacheDriver changes
RGW: Fix D4N read workflow crashes
RGW: Update RedisDriver to match new CacheDriver structure; define set_attrs method
RGW: Switch out D4N cache methods with Redis driver methods
RGW: Update Cache Driver structure
RGW: Update cache files.
RGW: create redis cache driver files

Signed-off-by: Samarah <samarah.uriarte@ibm.com>
15 months agorgw/cache: This commit squashes modifications to base class.
Pritha Srivastava [Fri, 2 Jun 2023 10:16:23 +0000 (15:46 +0530)]
rgw/cache: This commit squashes modifications to base class.

1. Rename namespace from rgw::cal to rgw::cache
2. Remove Partition from the CacheDriver constructor.
3. Removing rgw_cache_driver.cc since all methods
are pure virtual in rgw_cache_driver.h.
4. Moving structures Partition and Entry out of
class CacheDriver. And modifying the signatures of methods
get_current_partition_info() and get_free_space().
5. modifying the cache driver base class.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
15 months agoRGW: Add cache driver base class
Samarah [Thu, 25 May 2023 18:58:26 +0000 (14:58 -0400)]
RGW: Add cache driver base class

Signed-off-by: Samarah <samarah.uriarte@ibm.com>
15 months agoMerge pull request #55976 from cbodley/wip-qa-quincy-upgrade-distro
Casey Bodley [Tue, 2 Apr 2024 13:14:32 +0000 (14:14 +0100)]
Merge pull request #55976 from cbodley/wip-qa-quincy-upgrade-distro

qa/upgrade/quincy: bump ubuntu from 20->22

Reviewed-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #56279 from ceph/long_processing_timeout
Casey Bodley [Tue, 2 Apr 2024 12:38:33 +0000 (13:38 +0100)]
Merge pull request #56279 from ceph/long_processing_timeout

rgw/s3select: s3select response handler refactor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agoMerge pull request #56614 from zdover23/wip-doc-2024-03-30-mgr-zabbix
Zac Dover [Tue, 2 Apr 2024 11:21:15 +0000 (21:21 +1000)]
Merge pull request #56614 from zdover23/wip-doc-2024-03-30-mgr-zabbix

doc/mgr: update zabbix information

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agoMerge pull request #53679 from rhcs-dashboard/alerts-v2-api
Aashish Sharma [Tue, 2 Apr 2024 04:22:01 +0000 (09:52 +0530)]
Merge pull request #53679 from rhcs-dashboard/alerts-v2-api

mgr/dashboard: start using alertmanager v2

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
15 months agoMerge pull request #56596 from tchaikov/wip-on-exit-fix-leaks
Kefu Chai [Tue, 2 Apr 2024 00:43:42 +0000 (08:43 +0800)]
Merge pull request #56596 from tchaikov/wip-on-exit-fix-leaks

test/on_exit: use static variables for on_exit hooks

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
15 months agoMerge pull request #55979 from athanatos/sjust/wip-64546-max-creating-pgs
Samuel Just [Mon, 1 Apr 2024 20:18:15 +0000 (13:18 -0700)]
Merge pull request #55979 from athanatos/sjust/wip-64546-max-creating-pgs

mon: always create pgs in the epoch in which the pool was created

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
15 months agoMerge pull request #56533 from phlogistonjohn/jjm-cpatch-before
Adam King [Mon, 1 Apr 2024 15:32:36 +0000 (11:32 -0400)]
Merge pull request #56533 from phlogistonjohn/jjm-cpatch-before

cpatch: add --run-before command line option

Reviewed-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #56136 from NitzanMordhai/wip-nitzan-init-hash-in-chunk-ref-by...
Yuri Weinstein [Mon, 1 Apr 2024 15:32:12 +0000 (08:32 -0700)]
Merge pull request #56136 from NitzanMordhai/wip-nitzan-init-hash-in-chunk-ref-by-hash

cls/cas/cls_cas_internal: Initialize 'hash' value before decoding

Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>
15 months agoMerge pull request #54991 from ifed01/wip-ifed-some-recover-stuff
Yuri Weinstein [Mon, 1 Apr 2024 15:29:57 +0000 (08:29 -0700)]
Merge pull request #54991 from ifed01/wip-ifed-some-recover-stuff

tools/ceph-objectstore-tool: tricks to tolerate disk errors for "pg export" command.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
15 months agodoc/mgr: update zabbix information 56614/head
Zac Dover [Fri, 29 Mar 2024 11:06:28 +0000 (21:06 +1000)]
doc/mgr: update zabbix information

Include information from John Jasen about how to install Zabbix.

Signed-off-by: Zac Dover <zac.dover@proton.me>
15 months agoMerge pull request #56512 from rhcs-dashboard/remove-token-authentication-multi-cluster
Aashish Sharma [Mon, 1 Apr 2024 14:00:56 +0000 (19:30 +0530)]
Merge pull request #56512 from rhcs-dashboard/remove-token-authentication-multi-cluster

mgr/dashboard: Remove token authentication multi cluster

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agomgr/dashboard: remove the option for authentication with token in 56512/head
Aashish Sharma [Wed, 27 Mar 2024 09:45:59 +0000 (15:15 +0530)]
mgr/dashboard: remove the option for authentication with token in
multi-cluster form

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
15 months agoMerge pull request #55860 from ceph/add-multi-cluster-token-ttl
Aashish Sharma [Mon, 1 Apr 2024 07:50:37 +0000 (13:20 +0530)]
Merge pull request #55860 from ceph/add-multi-cluster-token-ttl

mgr/dashboard: Add multi cluster token ttl

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56600 from zdover23/wip-doc-2024-04-01-dev-essentials-leads-list
Zac Dover [Mon, 1 Apr 2024 01:36:42 +0000 (11:36 +1000)]
Merge pull request #56600 from zdover23/wip-doc-2024-04-01-dev-essentials-leads-list

doc/dev: update leads list

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agodoc/dev: update leads list 56600/head
Zac Dover [Mon, 1 Apr 2024 01:17:39 +0000 (11:17 +1000)]
doc/dev: update leads list

Remove Joao Luis from the list of component leads.

Signed-off-by: Zac Dover <zac.dover@proton.me>
15 months agorefactor of s3select response handler, adding continuation-response to parquet flow... 56279/head
Gal Salomon [Tue, 19 Mar 2024 11:45:58 +0000 (13:45 +0200)]
refactor of s3select response handler, adding continuation-response to parquet flow, refacor of error response

bug fix (init_success_response).
s3select submodule

fix for json-error-flow

Signed-off-by: Gal Salomon <gal.salomon@gmail.com>
15 months agotest/on_exit: use static variables for on_exit hooks 56596/head
Kefu Chai [Sun, 31 Mar 2024 01:14:21 +0000 (09:14 +0800)]
test/on_exit: use static variables for on_exit hooks

before this change, we allocate memory chunks using malloc(), but
we never free them. and LeakSanitizer points this out

```
Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x5588bfe532de in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_on_exit+0xa52de) (BuildId: 7c7a92bf5719592938c5307214bcd9b080bd847f)
    #1 0x5588bfe911d7 in func_scope() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:33:22
    #2 0x5588bfe90804 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:64:3
    #3 0x7f23081c1d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x5588bfe532de in malloc (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_on_exit+0xa52de) (BuildId: 7c7a92bf5719592938c5307214bcd9b080bd847f)
    #1 0x5588bfe91160 in func_scope() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:29:22
    #2 0x5588bfe90804 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/on_exit.cc:64:3
    #3 0x7f23081c1d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, instead of allocating the variables using `malloc()`,
we keep them in static variables, so that they can be accessed by
`OnExitManager` even if it is a static variable.
with this change, the memory leak reports for this source file go away.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agoMerge pull request #56591 from tchaikov/wip-unittest_rocksdb_option-fix-leak
Kefu Chai [Sun, 31 Mar 2024 08:32:09 +0000 (16:32 +0800)]
Merge pull request #56591 from tchaikov/wip-unittest_rocksdb_option-fix-leak

tests: unittest_rocksdb_option: do not leak RocksDBStore

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
15 months agoMerge pull request #56592 from nh2/crush-location-doc-fixes
Zac Dover [Sun, 31 Mar 2024 00:08:50 +0000 (10:08 +1000)]
Merge pull request #56592 from nh2/crush-location-doc-fixes

doc/rados/operations: Improve crush_location docs

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>
15 months agodoc/rados/operations: Improve crush_location docs 56592/head
Niklas Hambüchen [Sat, 30 Mar 2024 16:42:48 +0000 (17:42 +0100)]
doc/rados/operations: Improve crush_location docs

* Fix incorrect syntax
* Use underscores for config options, like other ceph docs did
* Fix incorrect statement that crush_location_hook adds fiels; it replaces
* Explain `root=default host=HOSTNAME` is not set if `crush_location` is given
* Remove duplication across sections
* Point out that `root=default` is important

Signed-off-by: Niklas Hambüchen <mail@nh2.me>
15 months agoMerge pull request #56572 from ceph/wip-yuriw-release-process-main
Yuri Weinstein [Sat, 30 Mar 2024 14:57:37 +0000 (07:57 -0700)]
Merge pull request #56572 from ceph/wip-yuriw-release-process-main

doc: added jammy to the supported distro list for quincy

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agotests: unittest_rocksdb_option: do not leak RocksDBStore 56591/head
Kefu Chai [Sat, 30 Mar 2024 13:21:25 +0000 (21:21 +0800)]
tests: unittest_rocksdb_option: do not leak RocksDBStore

before this change, we allocate an instance of `RocksDBStore` with
`new`, but we never free it. and LeanSanitizer points this out:

```
Direct leak of 952 byte(s) in 1 object(s) allocated from:
    #0 0x55f31440bc2d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_rocksdb_option+0xaebc2d) (BuildId: 81b849dbc41cbc6b05d5e603d9ba8a002dab2d24)
    #1 0x55f3144132fd in RocksDBOption_simple_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/TestRocksdbOptionParse.cc:17:22
    #2 0x55f3144ecf26 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x55f3144a4312 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x55f314453ccc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x55f314455d02 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x55f31445733b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x55f3144747c8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x55f3144f5576 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x55f3144ab1a2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x55f314473b52 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x55f31440f690 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x55f31440e4c3 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/unit.cc:45:10
    #13 0x7f0d32551d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we manage the life cycle of `RocksDBStore` using
a smart pointer. this should address the leak.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agoMerge pull request #56587 from tchaikov/wip-bluefs-use-after-free
Kefu Chai [Sat, 30 Mar 2024 11:47:19 +0000 (19:47 +0800)]
Merge pull request #56587 from tchaikov/wip-bluefs-use-after-free

test/objectstore/test_bluefs: fix heap-use-after-free

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
15 months agoMerge pull request #56581 from zdover23/wip-doc-2024-03-29-dev-essentials-leads-list
Zac Dover [Sat, 30 Mar 2024 11:05:10 +0000 (21:05 +1000)]
Merge pull request #56581 from zdover23/wip-doc-2024-03-29-dev-essentials-leads-list

doc/dev: update leads list

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agoMerge pull request #55436 from tchaikov/mgr-python-3.12
Kefu Chai [Sat, 30 Mar 2024 05:40:49 +0000 (13:40 +0800)]
Merge pull request #55436 from tchaikov/mgr-python-3.12

mgr: use un-deprecated APIs to initialize Python interpretor

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
15 months agotest/objectstore/test_bluefs: fix heap-use-after-free 56587/head
Kefu Chai [Sat, 30 Mar 2024 04:43:56 +0000 (12:43 +0800)]
test/objectstore/test_bluefs: fix heap-use-after-free

in BlueFS.test_shared_alloc, we keep the return value of
`fs.get_perf_counters()`, and deference it after umounting the fs,
but the `PerfCounters*` pointer returned from `fs.get_perf_counters()`
is destroyed in `BlueFS::_shutdown_logger()` which is in turn called
by `BlueFS::umount()`. so ASan points this out:
```
==1662613==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110000b2d80 at pc 0x7f0eefc30644 bp 0x7ffcdbab6430 sp 0x7ffcdbab6428
READ of size 8 at 0x6110000b2d80 thread T0
    #0 0x7f0eefc30643 in ceph::common::PerfCounters::get(int) const /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:246:8
    #1 0x557595ddfc15 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1182:3
    #2 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #13 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #14 0x7f0eed083e3f in __libc_start_main csu/../csu/libc-start.c:392:3
    #15 0x557595cd46a4 in _start (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x2856a4) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)

0x6110000b2d80 is located 0 bytes inside of 208-byte region [0x6110000b2d80,0x6110000b2e50)
freed by thread T0 here:
    #0 0x557595d92b1d in operator delete(void*) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x343b1d) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
    #1 0x557595f31c43 in BlueFS::_shutdown_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:462:3
    #2 0x557595f54ab5 in BlueFS::umount(bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:1076:3
    #3 0x557595ddfbd7 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1180:6
    #4 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #5 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #6 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #7 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #8 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #9 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #10 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #13 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #14 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #15 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

previously allocated by thread T0 here:
    #0 0x557595d922bd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_bluefs+0x3432bd) (BuildId: 5439261504ca3d7549fe9bcda1d17ef6d4d9b644)
    #1 0x7f0eefc33180 in ceph::common::PerfCountersBuilder::PerfCountersBuilder(ceph::common::CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/perf_counters.cc:537:21
    #2 0x557595f30ac9 in BlueFS::_init_logger() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:221:23
    #3 0x557595f42bc6 in BlueFS::mount() /home/jenkins-build/build/workspace/ceph-pull-requests/src/os/bluestore/BlueFS.cc:977:3
    #4 0x557595ddd339 in BlueFS_test_shared_alloc_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1139:3
    #5 0x557595eeef66 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #6 0x557595ea8b22 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #7 0x557595e5974c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #8 0x557595e5b782 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #9 0x557595e5cdbb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #10 0x557595e7a248 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #11 0x557595ef7816 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #12 0x557595eaf5c2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #13 0x557595e795d2 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #14 0x557595e05370 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #15 0x557595dfc1f5 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/objectstore/test_bluefs.cc:1603:10
    #16 0x7f0eed083d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, instead of keeping `logger` across the `umount()` and
`mount()` calls, we get another instance of `logger`, query it for
the perf counter that we are interested, and compare the value
to see if it is unchanged.

this should address the ASan warning above.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agoMerge pull request #56544 from baum/fix_f68248944815acbf5fb08ef66f36da6d5baef824
Kefu Chai [Sat, 30 Mar 2024 01:30:02 +0000 (09:30 +0800)]
Merge pull request #56544 from baum/fix_f68248944815acbf5fb08ef66f36da6d5baef824

pybind: support RHEL8, whose default python is python 3.6.8

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
16 months agoMerge pull request #56584 from phlogistonjohn/jjm-rpmbuild-fix
Dan Mick [Fri, 29 Mar 2024 18:14:19 +0000 (11:14 -0700)]
Merge pull request #56584 from phlogistonjohn/jjm-rpmbuild-fix

Merged without waiting for tests.  Obvious fix.

16 months agoceph.spec.in: remove command-with-macro line 56584/head
John Mulligan [Fri, 29 Mar 2024 18:04:33 +0000 (14:04 -0400)]
ceph.spec.in: remove command-with-macro line

A comment clearly left as a breadcrumb for a node-proxy manpage is
causing (intermittent) build failures. Remove the line and hope
the manpage is added if/when appropriate.

Fixes: 0dd73643649ddc2366e60de4fe6c078b6e112091
Signed-off-by: John Mulligan <jmulligan@redhat.com>
16 months agoqa/upgrade/quincy: bump ubuntu from 20->22 55976/head
Casey Bodley [Tue, 5 Mar 2024 19:42:24 +0000 (14:42 -0500)]
qa/upgrade/quincy: bump ubuntu from 20->22

Signed-off-by: Casey Bodley <cbodley@redhat.com>
16 months agoMerge pull request #56540 from tchaikov/wip-fix-test-leaks-ceph-context
Kefu Chai [Fri, 29 Mar 2024 16:10:20 +0000 (00:10 +0800)]
Merge pull request #56540 from tchaikov/wip-fix-test-leaks-ceph-context

test: : do not increase ref when creating intrusive_ptr<CephContext>

Reviewed-by: Casey Bodley <cbodley@redhat.com>
16 months agopybind: use syntax supported by python3.6 56544/head
Alexander Indenbaum [Thu, 28 Mar 2024 07:20:11 +0000 (07:20 +0000)]
pybind: use syntax supported by python3.6

Fixes: f68248944815acbf5fb08ef66f36da6d5baef824
It seems the shaman CentOS8 builders, which serve as the base for container images, aren't current with the ":=" syntax. Amend to support RHEL8, whose default python is python 3.6.8.

Error example from https://shaman.ceph.com/builds/ceph/ceph-nvmeof-mon/27188b78bdbaa3c33de518b8514906de629e774a/default/388913/

  File "/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-2505-g27188b78/rpm/el8/BUILD/ceph-19.0.0-2505-g27188b78/src/pybind/cephfs/setup.py", line 120
    if ldflags := os.environ.get('LDFLAGS'):
                ^
SyntaxError: invalid syntax

Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>
16 months agodoc: added jammy to the supported distro list for quincy 56572/head
Yuri Weinstein [Thu, 28 Mar 2024 22:57:34 +0000 (15:57 -0700)]
doc: added jammy to the supported distro list for quincy

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
16 months agodoc/dev: update leads list 56581/head
Zac Dover [Fri, 29 Mar 2024 11:14:21 +0000 (21:14 +1000)]
doc/dev: update leads list

Update the list of component leads in
doc/dev/developer_guide/essentials.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
16 months agomgr/dashboard: allow the user to add ttl for the connected cluster's 55860/head
Aashish Sharma [Thu, 29 Feb 2024 16:16:23 +0000 (21:46 +0530)]
mgr/dashboard: allow the user to add ttl for the connected cluster's
token expiration

Fixes: https://tracker.ceph.com/issues/65055
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
16 months agoMerge pull request #56190 from rhcs-dashboard/disable-multi-cluster-non-hub
Aashish Sharma [Fri, 29 Mar 2024 05:39:06 +0000 (11:09 +0530)]
Merge pull request #56190 from rhcs-dashboard/disable-multi-cluster-non-hub

mgr/dashboard: disable multi-cluster feature for non-hub clusters

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
16 months agoMerge pull request #54633 from joscollin/wip-F61334-use-snapdiff-api
Venky Shankar [Fri, 29 Mar 2024 05:36:35 +0000 (11:06 +0530)]
Merge pull request #54633 from joscollin/wip-F61334-use-snapdiff-api

cephfs_mirror: use snapdiff api for incremental syncing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
16 months agoMerge pull request #54301 from lxbsz/wip-sparse-read-test
Xiubo Li [Fri, 29 Mar 2024 04:26:26 +0000 (12:26 +0800)]
Merge pull request #54301 from lxbsz/wip-sparse-read-test

ceph_test_rados_api_io_pp: add multiple extents test for sparse-read

16 months agoMerge pull request #56570 from batrick/release-checklist-docs-2
Ilya Dryomov [Thu, 28 Mar 2024 23:06:07 +0000 (00:06 +0100)]
Merge pull request #56570 from batrick/release-checklist-docs-2

doc: reconcile release checklist

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
16 months agodoc: reconcile release checklist 56570/head
Patrick Donnelly [Thu, 28 Mar 2024 22:35:40 +0000 (18:35 -0400)]
doc: reconcile release checklist

Updating the releases documentation existed already but that section should be
moved to after dev freeze when the branch is forked.

Fixes: 7b5017913a9ecf27ca7281367a5472a9b9b043ec
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
16 months agoMerge pull request #56568 from batrick/release-checklist-docs
Ilya Dryomov [Thu, 28 Mar 2024 22:11:42 +0000 (23:11 +0100)]
Merge pull request #56568 from batrick/release-checklist-docs

doc: update release checklist

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
16 months agodoc: update release checklist 56568/head
Patrick Donnelly [Thu, 28 Mar 2024 21:06:41 +0000 (17:06 -0400)]
doc: update release checklist

To include note that release notes must be removed after forking the release
branch.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
16 months agoMerge PR #55337 into main
Patrick Donnelly [Thu, 28 Mar 2024 19:10:15 +0000 (15:10 -0400)]
Merge PR #55337 into main

* refs/pull/55337/head:
mds: explain clients retaining Fb during xlock

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
16 months agoMerge PR #56536 into main
Patrick Donnelly [Thu, 28 Mar 2024 18:40:39 +0000 (14:40 -0400)]
Merge PR #56536 into main

* refs/pull/56536/head:
mds: do not dispatch aborted internal requests
qa: add test for killing quiesce_inode with outstanding remote authpin requests
mds: add lockup debugging command

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
16 months agoMerge pull request #56534 from athanatos/sjust/wip-64975-crimson-scrub-snapmapper-2
Samuel Just [Thu, 28 Mar 2024 16:57:45 +0000 (09:57 -0700)]
Merge pull request #56534 from athanatos/sjust/wip-64975-crimson-scrub-snapmapper-2

crimson: introduce INTERNAL_PG_LOCAL_NS, use for snapmapper

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
16 months agoMerge pull request #56516 from afreen23/wip-ceph-logo
Nizamudeen A [Thu, 28 Mar 2024 15:23:09 +0000 (20:53 +0530)]
Merge pull request #56516 from afreen23/wip-ceph-logo

mgr/dashboard: make ceph logo redirect to dashboard

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
16 months agoMerge pull request #56543 from jrchyang/main
Igor Fedotov [Thu, 28 Mar 2024 09:13:54 +0000 (12:13 +0300)]
Merge pull request #56543 from jrchyang/main

os/bluestore: remove unnecessary std::hex in dout

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
16 months agoMerge pull request #56504 from rhcs-dashboard/replace-active_mds-active_nfs
Nizamudeen A [Thu, 28 Mar 2024 08:20:30 +0000 (13:50 +0530)]
Merge pull request #56504 from rhcs-dashboard/replace-active_mds-active_nfs

mgr/dashboard: s/active_mds/active_nfs in fs attach form

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
16 months agoMerge pull request #56468 from rhcs-dashboard/clone-name-exists-valid-fix
Nizamudeen A [Thu, 28 Mar 2024 08:19:59 +0000 (13:49 +0530)]
Merge pull request #56468 from rhcs-dashboard/clone-name-exists-valid-fix

mgr/dashboard: fix clone unique validator for name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
16 months agoMerge pull request #56470 from rhcs-dashboard/fix-full-name-validation-rgw
Nizamudeen A [Thu, 28 Mar 2024 08:19:50 +0000 (13:49 +0530)]
Merge pull request #56470 from rhcs-dashboard/fix-full-name-validation-rgw

mgr/dashboard: fix rgw display name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
16 months agoos/bluestore: remove unnecessary std::hex in dout 56543/head
jrchyang [Thu, 28 Mar 2024 07:16:58 +0000 (07:16 +0000)]
os/bluestore: remove unnecessary std::hex in dout

Signed-off-by: Jrchyang Yu <yuzhiqiang_yewu@cmss.chinamobile.com>
16 months agotest: : do not increase ref when creating intrusive_ptr<CephContext> 56540/head
Kefu Chai [Thu, 28 Mar 2024 02:01:46 +0000 (10:01 +0800)]
test: : do not increase ref when creating intrusive_ptr<CephContext>

before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance. and LeakSanitizer points this out:
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x558d341d837d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_ipaddr+0x19b37d) (BuildId: 1b7e7e5abfc2b58ce2334712e4c00b2441c25870)
    #1 0x7fd74c957559 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7fd74c956933 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7fd74c956933 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7fd74c956690 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7fd74c950562 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circ
ular_buffer/base.hpp:1039:9
    #6 0x7fd74c9435b4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7fd74bc1891d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7fd74bc178eb in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x558d341f97e9 in pick_address_filtering_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_ipaddr.cc:774:47
    #10 0x558d3430c4f6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x558d342c3fc2 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x558d342749dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #13 0x558d34276a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #14 0x558d3427804b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #15 0x558d342954d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #16 0x558d34314d26 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #17 0x558d342ca932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #18 0x558d34294862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #19 0x558d34218d80 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #20 0x558d34218d11 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #21 0x7fd749331d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so, in this change, we do not increase the refcount when creating cct.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agomds: do not dispatch aborted internal requests 56536/head
Patrick Donnelly [Wed, 27 Mar 2024 16:03:12 +0000 (12:03 -0400)]
mds: do not dispatch aborted internal requests

Matching the behavior of Server::dispatch_client_request.

This logic can be unified in a future refactor but keeping this as a targetted
fix for the purposes of the bug.

Fixes: https://tracker.ceph.com/issues/65182
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
16 months agoqa: add test for killing quiesce_inode with outstanding remote authpin requests
Patrick Donnelly [Wed, 27 Mar 2024 17:27:23 +0000 (13:27 -0400)]
qa: add test for killing quiesce_inode with outstanding remote authpin requests

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
16 months agoMerge pull request #56241 from tchaikov/wip-cmake-enable-sanitizers
Kefu Chai [Wed, 27 Mar 2024 23:16:06 +0000 (07:16 +0800)]
Merge pull request #56241 from tchaikov/wip-cmake-enable-sanitizers

cmake: enable building with Sanitizers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
16 months agomds: add lockup debugging command
Patrick Donnelly [Wed, 27 Mar 2024 16:24:28 +0000 (12:24 -0400)]
mds: add lockup debugging command

To artificially make the rank unresponsive for a time.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
16 months agoMerge pull request #56378 from guojidan/snap-stats
Samuel Just [Wed, 27 Mar 2024 21:03:06 +0000 (14:03 -0700)]
Merge pull request #56378 from guojidan/snap-stats

crimson/osd/osd_operations/snaptrim_event: update PG's stats

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
16 months agoscript/cpatch.py: run container build with host networking 56533/head
John Mulligan [Wed, 27 Mar 2024 18:32:10 +0000 (14:32 -0400)]
script/cpatch.py: run container build with host networking

Applies host networking to building the image. This is handy when using
custom networking hacks that redirect repos, etc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
16 months agoscript/cpatch.py: add a --run-before argument to run commands
John Mulligan [Wed, 27 Mar 2024 18:30:01 +0000 (14:30 -0400)]
script/cpatch.py: add a --run-before argument to run commands

Add a `--run-before` command line argument to cpatch.py that allows
commands to be executed before doing any other layer generation.
This supports actions like customizing packages installed in the
image. Something that can be handy when developing new features.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
16 months agoMerge pull request #56274 from phlogistonjohn/jjm-mypy-bump
Adam King [Wed, 27 Mar 2024 15:18:16 +0000 (11:18 -0400)]
Merge pull request #56274 from phlogistonjohn/jjm-mypy-bump

python: mypy version bump

Reviewed-by: Adam King <adking@redhat.com>
16 months agoMerge pull request #56304 from phlogistonjohn/jjm-tox-default-envs
Adam King [Wed, 27 Mar 2024 14:51:09 +0000 (10:51 -0400)]
Merge pull request #56304 from phlogistonjohn/jjm-tox-default-envs

python: allow unit tests to use tox default envs

Reviewed-by: Adam King <adking@redhat.com>
16 months agomgr/dashboard: disable multi-cluster feature for non-hub clusters 56190/head
Aashish Sharma [Thu, 14 Mar 2024 11:57:29 +0000 (17:27 +0530)]
mgr/dashboard: disable multi-cluster feature for non-hub clusters

Fixes: https://tracker.ceph.com/issues/65056
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
16 months agomgr/dashboard: make ceph logo redirect to dashboard 56516/head
Afreen [Wed, 27 Mar 2024 10:01:12 +0000 (15:31 +0530)]
mgr/dashboard: make ceph logo redirect to dashboard

Fixes https://tracker.ceph.com/issues/64734

Clicking on ceph logo in navigation takes back to overview dashboard.

Signed-off-by: Afreen <afreen23.git@gmail.com>
16 months agoMerge pull request #56449 from afreen23/wip-security
Nizamudeen A [Wed, 27 Mar 2024 07:28:23 +0000 (12:58 +0530)]
Merge pull request #56449 from afreen23/wip-security

mgr/dashboard:Update encryption and tags in bucket form

Reviewed-by: Nizamudeen A <nia@redhat.com>
16 months agomgr/dashboard: s/active_mds/active_nfs in fs attach form 56504/head
Nizamudeen A [Wed, 27 Mar 2024 05:12:39 +0000 (10:42 +0530)]
mgr/dashboard: s/active_mds/active_nfs in fs attach form

Fixes: https://tracker.ceph.com/issues/65173
Signed-off-by: Nizamudeen A <nia@redhat.com>
16 months agoMerge pull request #56239 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard...
Nizamudeen A [Wed, 27 Mar 2024 04:39:20 +0000 (10:09 +0530)]
Merge pull request #56239 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-follow-redirects-1.15.6

mgr/dashboard: bump follow-redirects from 1.15.3 to 1.15.6 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Nizamudeen A <nia@redhat.com>
16 months agoMerge pull request #56463 from rhcs-dashboard/cephfs-name-validation
Nizamudeen A [Wed, 27 Mar 2024 04:25:14 +0000 (09:55 +0530)]
Merge pull request #56463 from rhcs-dashboard/cephfs-name-validation

mgr/dashboard: fix cephfs name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
16 months agocrimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run 56534/head
Samuel Just [Tue, 26 Mar 2024 06:17:10 +0000 (23:17 -0700)]
crimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run

Fixes: https://tracker.ceph.com/issues/64975
Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agocrimson/.../scrub_events: convert ScrubScan::run to coroutine
Samuel Just [Tue, 26 Mar 2024 04:20:48 +0000 (21:20 -0700)]
crimson/.../scrub_events: convert ScrubScan::run to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agocrimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run
Samuel Just [Wed, 27 Mar 2024 01:43:06 +0000 (01:43 +0000)]
crimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run

Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agocrimson/.../scrub_events: convert ScrubFindRange::run to a coroutine
Samuel Just [Wed, 27 Mar 2024 01:39:17 +0000 (01:39 +0000)]
crimson/.../scrub_events: convert ScrubFindRange::run to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agocommon/hobject: make hobject_t formatter methods const
Samuel Just [Tue, 26 Mar 2024 06:16:31 +0000 (23:16 -0700)]
common/hobject: make hobject_t formatter methods const

Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agocrimson: move snapmapper object into INTERNAL_PG_LOCAL_NS
Samuel Just [Tue, 26 Mar 2024 05:44:37 +0000 (22:44 -0700)]
crimson: move snapmapper object into INTERNAL_PG_LOCAL_NS

Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agocrimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects
Samuel Just [Tue, 26 Mar 2024 05:36:07 +0000 (22:36 -0700)]
crimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects

Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agocrimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine
Samuel Just [Tue, 26 Mar 2024 05:31:08 +0000 (05:31 +0000)]
crimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agocrimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads
Samuel Just [Tue, 26 Mar 2024 05:13:40 +0000 (22:13 -0700)]
crimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads

Signed-off-by: Samuel Just <sjust@redhat.com>
16 months agoMerge pull request #56384 from Svelar/drop_fmt_version
Rongqi Sun [Wed, 27 Mar 2024 02:36:23 +0000 (10:36 +0800)]
Merge pull request #56384 from Svelar/drop_fmt_version

run-make: do not pass '-DWITH_FMT_VERSION' to cmake

16 months agoqa/lsan.supp: suppress MallocExtension::Initialize 56241/head
Kefu Chai [Sun, 24 Mar 2024 16:15:15 +0000 (00:15 +0800)]
qa/lsan.supp: suppress MallocExtension::Initialize

LeakSanitizer reports
```
==688591==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 45 byte(s) in 1 object(s) allocated from:
    #0 0x55f8dd9969dd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_fastbmap_allocator+0x1f89dd) (BuildId: cac39eac8ef1e8774f9dd48e6e3f677fdd864776)
    #1 0x55f8dd99c730 in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x55f8dd99c690 in std::allocator<char>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x55f8dd99c690 in std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    #4 0x55f8dd99c393 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:153:14
    #5 0x55f8dda96a6c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:307:21
    #6 0x55f8dda96852 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:395:8
    #7 0x7f4a751ab6f0 in
    MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId:
    eeef3d1257388a806e122398dbce3157ee568ef4)
```

this is a global object allocated by the allocator, so we can suppress this report.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agocmake: suppress LeakSanitizer reports of known leaks
Kefu Chai [Sun, 24 Mar 2024 15:20:38 +0000 (23:20 +0800)]
cmake: suppress LeakSanitizer reports of known leaks

there are known leaks, which are tracked by qa/lsan.suppr, in Ceph.
so let's reuse it so we don't see them when running unit test with
ASan enabled.

see also https://clang.llvm.org/docs/AddressSanitizer.html#issue-suppression

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agocmake: prevent ASAN_OPTIONS from detect_odr_violation
Kefu Chai [Sun, 24 Mar 2024 13:40:28 +0000 (21:40 +0800)]
cmake: prevent ASAN_OPTIONS from detect_odr_violation

turns out we have multiple copies of following symbol defined by rbd executable:
```
AddressSanitizer: odr-violation: global 'ceph::buffer::list::always_empty_bptr' at /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/buffer.cc:1267:34
```
before addressing it. let's disable this warning.

Refs https://tracker.ceph.com/issues/65098

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agopybind: use LDFLAGS in env variable when check for building env
Kefu Chai [Sun, 24 Mar 2024 12:23:00 +0000 (20:23 +0800)]
pybind: use LDFLAGS in env variable when check for building env

when building python bindings extensions using Cython, we first perform
sanity checks by building an executable and linking it against the
shared library of the C language binding. if the executable builds,
we consider the sanity test passes.

before this change, we don't add `LDFLAGS` specified in environmental
variable to the ldflags when building the executable. but we need to
link against libasan, so that the symbols used by, for instance,
librados.so, can be found by the executable.

in this change, we always check `LDFLAGS`, and add it to the ldflags
when building the executable for testing, if it's set.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agocmake: add ${CMAKE_SHARED_LINKER_FLAGS} to LDFLAGS when building python extensions
Kefu Chai [Sun, 17 Mar 2024 15:40:27 +0000 (23:40 +0800)]
cmake: add ${CMAKE_SHARED_LINKER_FLAGS} to LDFLAGS when building python extensions

if sanitizers are enabled, we have to populate the required link flags
to python extensions's building workflow. otherwise ld would fail to
link like:
```
/usr/bin/ld: /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libceph-common.so.2: undefined reference to `__asan_stack_free_10'
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agocmake: do not override CMAKE_EXE_LINKER_FLAGS
Kefu Chai [Sun, 17 Mar 2024 13:36:23 +0000 (21:36 +0800)]
cmake: do not override CMAKE_EXE_LINKER_FLAGS

instead of overriding CMAKE_EXE_LINKER_FLAGS, let's append to it.
so that the existing `CMAKE_EXE_LINKER_FLAGS` is not overriden.

this should enable us to build with Clang and with sanitizer(s) enabled.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agocmake: error out on UBSan error
Kefu Chai [Sun, 17 Mar 2024 11:35:04 +0000 (19:35 +0800)]
cmake: error out on UBSan error

so we can be alerted if UBSan identify something wrong.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agocmake: quote a list using quotes
Kefu Chai [Sun, 17 Mar 2024 10:53:03 +0000 (18:53 +0800)]
cmake: quote a list using quotes

otherwised we'd have

```
CMake Error at cmake/modules/FindSanitizers.cmake:17 (if):
  if given arguments:

    "address" "IN_LIST" "address" "thread" "undefined_behavior" "OR" "leak" "IN_LIST" "address" "thread" "undefined_behavior"

  Unknown arguments specified
```

when enabling TSan with WITH_TSAN=ON.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
16 months agoMerge pull request #56461 from tchaikov/wip-test-test_config_map-fix-leaks
Kefu Chai [Tue, 26 Mar 2024 23:31:34 +0000 (07:31 +0800)]
Merge pull request #56461 from tchaikov/wip-test-test_config_map-fix-leaks

test/mon/test_config_map: fix leaks identified by LeakSanitizer

Reviewed-by: Casey Bodley <cbodley@redhat.com>
16 months agoMerge pull request #56450 from tahajahangir/patch-2
Zac Dover [Tue, 26 Mar 2024 20:11:54 +0000 (06:11 +1000)]
Merge pull request #56450 from tahajahangir/patch-2

docs/rados: remove incorrect ceph command

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
16 months agoMerge pull request #56458 from adk3798/rgw-bootstrap-hostnames
Adam King [Tue, 26 Mar 2024 19:31:06 +0000 (15:31 -0400)]
Merge pull request #56458 from adk3798/rgw-bootstrap-hostnames

mgr/cephadm: update rgw zonegroup w/ zonegroup_hostnames from rgw spec

Reviewed-by: Ali Maredia <amaredia@redhat.com>
16 months agomgr/dashboard: start using alertmanager v2 53679/head
Nizamudeen A [Tue, 26 Sep 2023 16:08:51 +0000 (21:38 +0530)]
mgr/dashboard: start using alertmanager v2

I was looking into sorting the alerts and saw there is an api v2 for
alertmanager which also has an endpoint like `alerts/groups` which might
be something that is useful for us.

Refer: https://github.com/prometheus/alertmanager/blob/main/api/v2/openapi.yaml

Fixes: https://tracker.ceph.com/issues/65070
Signed-off-by: Nizamudeen A <nia@redhat.com>
16 months agoMerge PR #56454 into main
Patrick Donnelly [Tue, 26 Mar 2024 15:08:10 +0000 (11:08 -0400)]
Merge PR #56454 into main

* refs/pull/56454/head:
vstart: require debugging switch for all mds debug configs

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>