]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
15 months agomsg: trivial destructor should be default 56565/head
Patrick Donnelly [Mon, 11 Dec 2023 20:27:58 +0000 (15:27 -0500)]
msg: trivial destructor should be default

See: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rc-five

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge PR #55337 into main
Patrick Donnelly [Thu, 28 Mar 2024 19:10:15 +0000 (15:10 -0400)]
Merge PR #55337 into main

* refs/pull/55337/head:
mds: explain clients retaining Fb during xlock

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
15 months agoMerge PR #56536 into main
Patrick Donnelly [Thu, 28 Mar 2024 18:40:39 +0000 (14:40 -0400)]
Merge PR #56536 into main

* refs/pull/56536/head:
mds: do not dispatch aborted internal requests
qa: add test for killing quiesce_inode with outstanding remote authpin requests
mds: add lockup debugging command

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
15 months agoMerge pull request #56534 from athanatos/sjust/wip-64975-crimson-scrub-snapmapper-2
Samuel Just [Thu, 28 Mar 2024 16:57:45 +0000 (09:57 -0700)]
Merge pull request #56534 from athanatos/sjust/wip-64975-crimson-scrub-snapmapper-2

crimson: introduce INTERNAL_PG_LOCAL_NS, use for snapmapper

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
15 months agoMerge pull request #56516 from afreen23/wip-ceph-logo
Nizamudeen A [Thu, 28 Mar 2024 15:23:09 +0000 (20:53 +0530)]
Merge pull request #56516 from afreen23/wip-ceph-logo

mgr/dashboard: make ceph logo redirect to dashboard

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56543 from jrchyang/main
Igor Fedotov [Thu, 28 Mar 2024 09:13:54 +0000 (12:13 +0300)]
Merge pull request #56543 from jrchyang/main

os/bluestore: remove unnecessary std::hex in dout

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
15 months agoMerge pull request #56504 from rhcs-dashboard/replace-active_mds-active_nfs
Nizamudeen A [Thu, 28 Mar 2024 08:20:30 +0000 (13:50 +0530)]
Merge pull request #56504 from rhcs-dashboard/replace-active_mds-active_nfs

mgr/dashboard: s/active_mds/active_nfs in fs attach form

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agoMerge pull request #56468 from rhcs-dashboard/clone-name-exists-valid-fix
Nizamudeen A [Thu, 28 Mar 2024 08:19:59 +0000 (13:49 +0530)]
Merge pull request #56468 from rhcs-dashboard/clone-name-exists-valid-fix

mgr/dashboard: fix clone unique validator for name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agoMerge pull request #56470 from rhcs-dashboard/fix-full-name-validation-rgw
Nizamudeen A [Thu, 28 Mar 2024 08:19:50 +0000 (13:49 +0530)]
Merge pull request #56470 from rhcs-dashboard/fix-full-name-validation-rgw

mgr/dashboard: fix rgw display name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agoos/bluestore: remove unnecessary std::hex in dout 56543/head
jrchyang [Thu, 28 Mar 2024 07:16:58 +0000 (07:16 +0000)]
os/bluestore: remove unnecessary std::hex in dout

Signed-off-by: Jrchyang Yu <yuzhiqiang_yewu@cmss.chinamobile.com>
15 months agomds: do not dispatch aborted internal requests 56536/head
Patrick Donnelly [Wed, 27 Mar 2024 16:03:12 +0000 (12:03 -0400)]
mds: do not dispatch aborted internal requests

Matching the behavior of Server::dispatch_client_request.

This logic can be unified in a future refactor but keeping this as a targetted
fix for the purposes of the bug.

Fixes: https://tracker.ceph.com/issues/65182
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoqa: add test for killing quiesce_inode with outstanding remote authpin requests
Patrick Donnelly [Wed, 27 Mar 2024 17:27:23 +0000 (13:27 -0400)]
qa: add test for killing quiesce_inode with outstanding remote authpin requests

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge pull request #56241 from tchaikov/wip-cmake-enable-sanitizers
Kefu Chai [Wed, 27 Mar 2024 23:16:06 +0000 (07:16 +0800)]
Merge pull request #56241 from tchaikov/wip-cmake-enable-sanitizers

cmake: enable building with Sanitizers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agomds: add lockup debugging command
Patrick Donnelly [Wed, 27 Mar 2024 16:24:28 +0000 (12:24 -0400)]
mds: add lockup debugging command

To artificially make the rank unresponsive for a time.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge pull request #56378 from guojidan/snap-stats
Samuel Just [Wed, 27 Mar 2024 21:03:06 +0000 (14:03 -0700)]
Merge pull request #56378 from guojidan/snap-stats

crimson/osd/osd_operations/snaptrim_event: update PG's stats

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
15 months agoMerge pull request #56274 from phlogistonjohn/jjm-mypy-bump
Adam King [Wed, 27 Mar 2024 15:18:16 +0000 (11:18 -0400)]
Merge pull request #56274 from phlogistonjohn/jjm-mypy-bump

python: mypy version bump

Reviewed-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #56304 from phlogistonjohn/jjm-tox-default-envs
Adam King [Wed, 27 Mar 2024 14:51:09 +0000 (10:51 -0400)]
Merge pull request #56304 from phlogistonjohn/jjm-tox-default-envs

python: allow unit tests to use tox default envs

Reviewed-by: Adam King <adking@redhat.com>
15 months agomgr/dashboard: make ceph logo redirect to dashboard 56516/head
Afreen [Wed, 27 Mar 2024 10:01:12 +0000 (15:31 +0530)]
mgr/dashboard: make ceph logo redirect to dashboard

Fixes https://tracker.ceph.com/issues/64734

Clicking on ceph logo in navigation takes back to overview dashboard.

Signed-off-by: Afreen <afreen23.git@gmail.com>
15 months agoMerge pull request #56449 from afreen23/wip-security
Nizamudeen A [Wed, 27 Mar 2024 07:28:23 +0000 (12:58 +0530)]
Merge pull request #56449 from afreen23/wip-security

mgr/dashboard:Update encryption and tags in bucket form

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agomgr/dashboard: s/active_mds/active_nfs in fs attach form 56504/head
Nizamudeen A [Wed, 27 Mar 2024 05:12:39 +0000 (10:42 +0530)]
mgr/dashboard: s/active_mds/active_nfs in fs attach form

Fixes: https://tracker.ceph.com/issues/65173
Signed-off-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56239 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard...
Nizamudeen A [Wed, 27 Mar 2024 04:39:20 +0000 (10:09 +0530)]
Merge pull request #56239 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-follow-redirects-1.15.6

mgr/dashboard: bump follow-redirects from 1.15.3 to 1.15.6 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56463 from rhcs-dashboard/cephfs-name-validation
Nizamudeen A [Wed, 27 Mar 2024 04:25:14 +0000 (09:55 +0530)]
Merge pull request #56463 from rhcs-dashboard/cephfs-name-validation

mgr/dashboard: fix cephfs name validation

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agocrimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run 56534/head
Samuel Just [Tue, 26 Mar 2024 06:17:10 +0000 (23:17 -0700)]
crimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run

Fixes: https://tracker.ceph.com/issues/64975
Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/.../scrub_events: convert ScrubScan::run to coroutine
Samuel Just [Tue, 26 Mar 2024 04:20:48 +0000 (21:20 -0700)]
crimson/.../scrub_events: convert ScrubScan::run to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run
Samuel Just [Wed, 27 Mar 2024 01:43:06 +0000 (01:43 +0000)]
crimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/.../scrub_events: convert ScrubFindRange::run to a coroutine
Samuel Just [Wed, 27 Mar 2024 01:39:17 +0000 (01:39 +0000)]
crimson/.../scrub_events: convert ScrubFindRange::run to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocommon/hobject: make hobject_t formatter methods const
Samuel Just [Tue, 26 Mar 2024 06:16:31 +0000 (23:16 -0700)]
common/hobject: make hobject_t formatter methods const

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson: move snapmapper object into INTERNAL_PG_LOCAL_NS
Samuel Just [Tue, 26 Mar 2024 05:44:37 +0000 (22:44 -0700)]
crimson: move snapmapper object into INTERNAL_PG_LOCAL_NS

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects
Samuel Just [Tue, 26 Mar 2024 05:36:07 +0000 (22:36 -0700)]
crimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine
Samuel Just [Tue, 26 Mar 2024 05:31:08 +0000 (05:31 +0000)]
crimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agocrimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads
Samuel Just [Tue, 26 Mar 2024 05:13:40 +0000 (22:13 -0700)]
crimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads

Signed-off-by: Samuel Just <sjust@redhat.com>
15 months agoMerge pull request #56384 from Svelar/drop_fmt_version
Rongqi Sun [Wed, 27 Mar 2024 02:36:23 +0000 (10:36 +0800)]
Merge pull request #56384 from Svelar/drop_fmt_version

run-make: do not pass '-DWITH_FMT_VERSION' to cmake

15 months agoqa/lsan.supp: suppress MallocExtension::Initialize 56241/head
Kefu Chai [Sun, 24 Mar 2024 16:15:15 +0000 (00:15 +0800)]
qa/lsan.supp: suppress MallocExtension::Initialize

LeakSanitizer reports
```
==688591==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 45 byte(s) in 1 object(s) allocated from:
    #0 0x55f8dd9969dd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_fastbmap_allocator+0x1f89dd) (BuildId: cac39eac8ef1e8774f9dd48e6e3f677fdd864776)
    #1 0x55f8dd99c730 in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x55f8dd99c690 in std::allocator<char>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x55f8dd99c690 in std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    #4 0x55f8dd99c393 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:153:14
    #5 0x55f8dda96a6c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:307:21
    #6 0x55f8dda96852 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:395:8
    #7 0x7f4a751ab6f0 in
    MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId:
    eeef3d1257388a806e122398dbce3157ee568ef4)
```

this is a global object allocated by the allocator, so we can suppress this report.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agocmake: suppress LeakSanitizer reports of known leaks
Kefu Chai [Sun, 24 Mar 2024 15:20:38 +0000 (23:20 +0800)]
cmake: suppress LeakSanitizer reports of known leaks

there are known leaks, which are tracked by qa/lsan.suppr, in Ceph.
so let's reuse it so we don't see them when running unit test with
ASan enabled.

see also https://clang.llvm.org/docs/AddressSanitizer.html#issue-suppression

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agocmake: prevent ASAN_OPTIONS from detect_odr_violation
Kefu Chai [Sun, 24 Mar 2024 13:40:28 +0000 (21:40 +0800)]
cmake: prevent ASAN_OPTIONS from detect_odr_violation

turns out we have multiple copies of following symbol defined by rbd executable:
```
AddressSanitizer: odr-violation: global 'ceph::buffer::list::always_empty_bptr' at /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/buffer.cc:1267:34
```
before addressing it. let's disable this warning.

Refs https://tracker.ceph.com/issues/65098

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agopybind: use LDFLAGS in env variable when check for building env
Kefu Chai [Sun, 24 Mar 2024 12:23:00 +0000 (20:23 +0800)]
pybind: use LDFLAGS in env variable when check for building env

when building python bindings extensions using Cython, we first perform
sanity checks by building an executable and linking it against the
shared library of the C language binding. if the executable builds,
we consider the sanity test passes.

before this change, we don't add `LDFLAGS` specified in environmental
variable to the ldflags when building the executable. but we need to
link against libasan, so that the symbols used by, for instance,
librados.so, can be found by the executable.

in this change, we always check `LDFLAGS`, and add it to the ldflags
when building the executable for testing, if it's set.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agocmake: add ${CMAKE_SHARED_LINKER_FLAGS} to LDFLAGS when building python extensions
Kefu Chai [Sun, 17 Mar 2024 15:40:27 +0000 (23:40 +0800)]
cmake: add ${CMAKE_SHARED_LINKER_FLAGS} to LDFLAGS when building python extensions

if sanitizers are enabled, we have to populate the required link flags
to python extensions's building workflow. otherwise ld would fail to
link like:
```
/usr/bin/ld: /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libceph-common.so.2: undefined reference to `__asan_stack_free_10'
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agocmake: do not override CMAKE_EXE_LINKER_FLAGS
Kefu Chai [Sun, 17 Mar 2024 13:36:23 +0000 (21:36 +0800)]
cmake: do not override CMAKE_EXE_LINKER_FLAGS

instead of overriding CMAKE_EXE_LINKER_FLAGS, let's append to it.
so that the existing `CMAKE_EXE_LINKER_FLAGS` is not overriden.

this should enable us to build with Clang and with sanitizer(s) enabled.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agocmake: error out on UBSan error
Kefu Chai [Sun, 17 Mar 2024 11:35:04 +0000 (19:35 +0800)]
cmake: error out on UBSan error

so we can be alerted if UBSan identify something wrong.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agocmake: quote a list using quotes
Kefu Chai [Sun, 17 Mar 2024 10:53:03 +0000 (18:53 +0800)]
cmake: quote a list using quotes

otherwised we'd have

```
CMake Error at cmake/modules/FindSanitizers.cmake:17 (if):
  if given arguments:

    "address" "IN_LIST" "address" "thread" "undefined_behavior" "OR" "leak" "IN_LIST" "address" "thread" "undefined_behavior"

  Unknown arguments specified
```

when enabling TSan with WITH_TSAN=ON.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agoMerge pull request #56461 from tchaikov/wip-test-test_config_map-fix-leaks
Kefu Chai [Tue, 26 Mar 2024 23:31:34 +0000 (07:31 +0800)]
Merge pull request #56461 from tchaikov/wip-test-test_config_map-fix-leaks

test/mon/test_config_map: fix leaks identified by LeakSanitizer

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agoMerge pull request #56450 from tahajahangir/patch-2
Zac Dover [Tue, 26 Mar 2024 20:11:54 +0000 (06:11 +1000)]
Merge pull request #56450 from tahajahangir/patch-2

docs/rados: remove incorrect ceph command

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
15 months agoMerge pull request #56458 from adk3798/rgw-bootstrap-hostnames
Adam King [Tue, 26 Mar 2024 19:31:06 +0000 (15:31 -0400)]
Merge pull request #56458 from adk3798/rgw-bootstrap-hostnames

mgr/cephadm: update rgw zonegroup w/ zonegroup_hostnames from rgw spec

Reviewed-by: Ali Maredia <amaredia@redhat.com>
15 months agoMerge PR #56454 into main
Patrick Donnelly [Tue, 26 Mar 2024 15:08:10 +0000 (11:08 -0400)]
Merge PR #56454 into main

* refs/pull/56454/head:
vstart: require debugging switch for all mds debug configs

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
15 months agomgr/dashboard: fix rgw display name validation 56470/head
Nizamudeen A [Tue, 26 Mar 2024 09:04:23 +0000 (14:34 +0530)]
mgr/dashboard: fix rgw display name validation

the dot(.) character is allowed in full name

Fixes: https://tracker.ceph.com/issues/65146
Signed-off-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56351 from afreen23/wip-advanced-sec
Nizamudeen A [Tue, 26 Mar 2024 13:51:29 +0000 (19:21 +0530)]
Merge pull request #56351 from afreen23/wip-advanced-sec

mgr/dashboard: Add advanced fieldset component

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agomgr/dashboard: fix cephfs name validation 56463/head
Nizamudeen A [Tue, 26 Mar 2024 07:53:39 +0000 (13:23 +0530)]
mgr/dashboard: fix cephfs name validation

allow volume name to start with dot (.)

Fixes: https://tracker.ceph.com/issues/65143
Signed-off-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56462 from zhscn/fix-uaf
Yingxin [Tue, 26 Mar 2024 09:08:30 +0000 (17:08 +0800)]
Merge pull request #56462 from zhscn/fix-uaf

crimson/os/seastore: fix use after free error

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
15 months agomgr/dashboard: fix clone unique validator for name validation 56468/head
Nizamudeen A [Tue, 26 Mar 2024 08:50:25 +0000 (14:20 +0530)]
mgr/dashboard: fix clone unique validator for name validation

Fixes: https://tracker.ceph.com/issues/65145
Signed-off-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56436 from zdover23/wip-doc-2024-03-25-start-intro
Zac Dover [Tue, 26 Mar 2024 08:29:26 +0000 (18:29 +1000)]
Merge pull request #56436 from zdover23/wip-doc-2024-03-25-start-intro

doc/start: improve MDS explanation

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agoMerge pull request #56328 from rhcs-dashboard/cors-through-cli
Aashish Sharma [Tue, 26 Mar 2024 08:14:09 +0000 (13:44 +0530)]
Merge pull request #56328 from rhcs-dashboard/cors-through-cli

mgr/dashboard: introduce cli commands to set cors urls

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
15 months agoMerge pull request #55783 from rhcs-dashboard/multi-cluster-imrovements
Aashish Sharma [Tue, 26 Mar 2024 07:32:28 +0000 (13:02 +0530)]
Merge pull request #55783 from rhcs-dashboard/multi-cluster-imrovements

mgr/dashboard: Multi cluster improvements

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agomgr/dashboard:Update encryption and tags in bucket form 56449/head
Afreen [Mon, 25 Mar 2024 08:39:08 +0000 (14:09 +0530)]
mgr/dashboard:Update encryption and tags in bucket form

Fixes https://tracker.ceph.com/issues/65110

- renamed Security to Encryption
- aligned security fieldset with rest of the form using `cd-help-text`
  and aligning it with object locking in UX
- aligned security fieldset top use `cd-help-text`
- changed help text of Tags

Signed-off-by: Afreen <afreen23.git@gmail.com>
15 months agoMerge pull request #56390 from rhcs-dashboard/rgw-add-system-user
Nizamudeen A [Tue, 26 Mar 2024 07:03:02 +0000 (12:33 +0530)]
Merge pull request #56390 from rhcs-dashboard/rgw-add-system-user

mgr/dashboard: add system users to rgw user form

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agomgr/dashboard: Add advanced fieldset component 56351/head
Afreen [Fri, 1 Mar 2024 07:26:25 +0000 (12:56 +0530)]
mgr/dashboard: Add advanced fieldset component

Fixes https://tracker.ceph.com/issues/65024

- adds a new shared component for displaying advanced fields in /shared
- utilizes that component in bucket form
- placement targets are under advanced fieldset
- minor help text changes in object locking help text
- updated e2e tests

Signed-off-by: Afreen <afreen23.git@gmail.com>
15 months agocrimson/os/seastore: fix use after free error 56462/head
Zhang Song [Tue, 26 Mar 2024 06:15:35 +0000 (14:15 +0800)]
crimson/os/seastore: fix use after free error

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
15 months agoMerge pull request #56055 from rhcs-dashboard/add-nfsv3-support
Nizamudeen A [Tue, 26 Mar 2024 05:58:15 +0000 (11:28 +0530)]
Merge pull request #56055 from rhcs-dashboard/add-nfsv3-support

mgr/dashboard: add support for NFSv3 exports

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agomgr/dashboards: multi-cluster improvements and bug fixes 55783/head
Aashish Sharma [Mon, 26 Feb 2024 04:17:34 +0000 (09:47 +0530)]
mgr/dashboards: multi-cluster improvements and bug fixes

Fixes: https://tracker.ceph.com/issues/64880
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
15 months agoMerge pull request #56332 from cyx1231st/wip-fix-racing-get-or-create-pg-mapping
Yingxin [Tue, 26 Mar 2024 03:41:42 +0000 (11:41 +0800)]
Merge pull request #56332 from cyx1231st/wip-fix-racing-get-or-create-pg-mapping

crimson/os/pg_map: allow multiple shards to create new pg mappings at the same time

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
15 months agotest/mon/test_config_map: free CrushWrapper created with new 56461/head
Kefu Chai [Tue, 26 Mar 2024 03:03:38 +0000 (11:03 +0800)]
test/mon/test_config_map: free CrushWrapper created with new

before this change, we create a new CrushWrapper instance with `new`, but
we never free this instance after done with it. and LeakSanitizer
points this out:

```
Direct leak of 544 byte(s) in 1 object(s) allocated from:
    #0 0x561afe148fed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_config_map+0x1c2fed) (BuildId: 3ce9eeed38cee335628fa74fdd08cd215b15019e)
    #1 0x561afe151cbd in ConfigMap_result_sections_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/test_config_map.cc:93:16
    #2 0x561afe2689b6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x561afe221262 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x561afe1d1f7c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x561afe1d3fb2 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x561afe1d55eb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x561afe1f2a78 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x561afe2711e6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x561afe227bd2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x561afe1f1e02 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x561afe176ec0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x561afe176e51 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #13 0x7f37d9397d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so in this change, we manage the `CrushWrapper` pointer with a smart
pointer. because the size of `CrushWrapper` is relatively large, we
don't create it on stack.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agotest/mon/test_config_map: free CephContext created with new
Kefu Chai [Tue, 26 Mar 2024 02:50:16 +0000 (10:50 +0800)]
test/mon/test_config_map: free CephContext created with new

before this change, we create a new cct instance with `new`, but
we never free this instance after done with it. and LeakSanitizer
points this out:

```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x561afe148fed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_config_map+0x1c2fed) (BuildId: 3ce9eeed38cee335628fa74fdd08cd215b15019e)
    #1 0x7f37dc9ac189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7f37dc9ab563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7f37dc9ab563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7f37dc9ab2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7f37dc9a5192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    #6 0x7f37dc9981e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7f37dbc6e96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7f37dbc6d93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x561afe14e983 in ConfigMap_add_option_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/test_config_map.cc:58:18
    #10 0x561afe2689b6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x561afe221262 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x561afe1d1f7c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #13 0x561afe1d3fb2 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #14 0x561afe1d55eb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #15 0x561afe1f2a78 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #16 0x561afe2711e6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #17 0x561afe227bd2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #18 0x561afe1f1e02 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #19 0x561afe176ec0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #20 0x561afe176e51 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #21 0x7f37d9397d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so in this change, we manage the `CephContext` pointer with a smart
pointer. because the size of CephContext could be large, we don't create
it on stack.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agoMerge pull request #56455 from tchaikov/wip-test-test_rgw_iam_policy-fix-leak
Kefu Chai [Tue, 26 Mar 2024 02:09:56 +0000 (10:09 +0800)]
Merge pull request #56455 from tchaikov/wip-test-test_rgw_iam_policy-fix-leak

test/rgw/test_rgw_iam_policy: do not increase ref when creating intru…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agorun-make: do not pass '-DWITH_FMT_VERSION' to cmake 56384/head
Rongqi Sun [Fri, 22 Mar 2024 07:39:02 +0000 (07:39 +0000)]
run-make: do not pass '-DWITH_FMT_VERSION' to cmake

Clean up this opt because 'WITH_FMT_VERSION' is removed from
src/CMakeLists.txt.
See: https://github.com/ceph/ceph/blob/686dd3d838b6de6bf1101f0b61f729426b0bccad/src/CMakeLists.txt

Fixes: https://tracker.ceph.com/issues/65067
Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
15 months agoMerge pull request #55068 from phlogistonjohn/jjm-cephadm-smb-svc-spec
Adam King [Mon, 25 Mar 2024 20:24:54 +0000 (16:24 -0400)]
Merge pull request #55068 from phlogistonjohn/jjm-cephadm-smb-svc-spec

cephadm: add a new SMB service to the mgr module

Reviewed-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #56142 from Matan-B/wip-matanb-fix_hobject_to_str
Yuri Weinstein [Mon, 25 Mar 2024 18:30:45 +0000 (11:30 -0700)]
Merge pull request #56142 from Matan-B/wip-matanb-fix_hobject_to_str

common/hobject: fix hobject_t::to_str()

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
15 months agoMerge pull request #55676 from baergj/cod-for-all-upstream
Yuri Weinstein [Mon, 25 Mar 2024 18:29:46 +0000 (11:29 -0700)]
Merge pull request #55676 from baergj/cod-for-all-upstream

kv/RocksDBStore: Configure compact-on-deletion for all CFs

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
15 months agoMerge pull request #55599 from ifed01/wip-ifed-kill-build-warns
Yuri Weinstein [Mon, 25 Mar 2024 18:28:58 +0000 (11:28 -0700)]
Merge pull request #55599 from ifed01/wip-ifed-kill-build-warns

os/bluestore: eliminate build warnings

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
15 months agoMerge pull request #55562 from Matan-B/wip-matanb-fix-lookup-purged-snaps
Yuri Weinstein [Mon, 25 Mar 2024 18:27:16 +0000 (11:27 -0700)]
Merge pull request #55562 from Matan-B/wip-matanb-fix-lookup-purged-snaps

osd/SnapMapper: fix _lookup_purged_snap

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
15 months agomgr/cephadm: update zonegroup w/ zonegroup_hostnames from rgw spec 56458/head
Adam King [Mon, 25 Mar 2024 17:47:12 +0000 (13:47 -0400)]
mgr/cephadm: update zonegroup w/ zonegroup_hostnames from rgw spec

If users provide this field to the rgw spec, we need to modify
the zonegroup to includes these hostnames. This most just makes
use of another commit that added the `ceph rgw zonegroup modify`
command to the rgw module for this specific purpose.

Signed-off-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #55609 from ljflores/wip-telemetry-opt-in-to-new-collection
Laura Flores [Mon, 25 Mar 2024 17:13:43 +0000 (12:13 -0500)]
Merge pull request #55609 from ljflores/wip-telemetry-opt-in-to-new-collection

qa/workunits: opt in to new collection after upgrade from reef to main

15 months agomgr/rgw: add support to modify zonegroup parameters
Adam King [Mon, 25 Mar 2024 16:13:32 +0000 (12:13 -0400)]
mgr/rgw: add support to modify zonegroup parameters

This is being done with `radosgw-admin zonegroup set`
rather than `radosgw-admin zonegroup modify` as I don't
think the hostnames parameter (which is the primary focus
for this specific change) can be set using the modify
command. The nice bit about that is it should in theory
make it easy to extend this to allow setting other parameters
to be modified in the zonegroup in the future.

Signed-off-by: Adam King <adking@redhat.com>
15 months agotest/rgw/test_rgw_iam_policy: do not increase ref when creating intrusive_ptr<CephCon... 56455/head
Kefu Chai [Mon, 25 Mar 2024 14:56:28 +0000 (22:56 +0800)]
test/rgw/test_rgw_iam_policy: do not increase ref when creating intrusive_ptr<CephContext>

before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance. and LeakSanitizer points this out:

```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac359c7c8 in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/bin/unittest_rgw_iam_policy+0x211c7c8) (BuildId: 060fadb10da261b52fd5757c7b1e9812d34542f1)
    #1 0xffff96f764e4 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0xffff96f757cc in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0xffff96f757cc in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:2396:39
    #4 0xffff96f75500 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:2494:18
    #5 0xffff96f6ec4c in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:1039:9
    #6 0xffff96f63528 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/log/Log.cc:53:5
    #7 0xffff96045300 in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/common/ceph_context.cc:729:16
    #8 0xffff960446ec in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/common/ceph_context.cc:697:5
    #9 0xaaaac3629238 in IPPolicyTest::IPPolicyTest() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/rgw/test_rgw_iam_policy.cc:864:15
    #10 0xaaaac3628da0 in IPPolicyTest_MaskedIPOperations_Test::IPPolicyTest_MaskedIPOperations_Test() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/rgw/test_rgw_iam_policy.cc:869:1
    #11 0xaaaac3628d3c in testing::internal::TestFactoryImpl<IPPolicyTest_MaskedIPOperations_Test>::CreateTest() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
```

so, in this change, we do not increase the refcount when creating cct.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agoMerge pull request #56437 from tchaikov/wip-cephcontext-leak-fixes
Kefu Chai [Mon, 25 Mar 2024 14:18:26 +0000 (22:18 +0800)]
Merge pull request #56437 from tchaikov/wip-cephcontext-leak-fixes

test: fix the leakages of `CephContext` identified by LeakSanitizer

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agovstart: require debugging switch for all mds debug configs 56454/head
Patrick Donnelly [Mon, 25 Mar 2024 14:13:24 +0000 (10:13 -0400)]
vstart: require debugging switch for all mds debug configs

mds_debug_subtrees wrecks vstart cluster performance particularly egregiously.
Do not turn it on by default.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
15 months agoMerge pull request #56173 from Matan-B/wip-matanb-clone-overlap-doc
Matan Breizman [Mon, 25 Mar 2024 10:15:22 +0000 (12:15 +0200)]
Merge pull request #56173 from Matan-B/wip-matanb-clone-overlap-doc

doc/dev: osd_internals/snaps.rst: add clone_overlap doc

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>
15 months agodocs/rados: remove incorrect ceph command 56450/head
Taha Jahangir [Mon, 25 Mar 2024 10:02:12 +0000 (13:32 +0330)]
docs/rados: remove incorrect ceph command

The removed line was (incorrectly changed) output of the previous command.

Signed-off-by: Taha Jahangir <mtjahangir@gmail.com>
15 months agoMerge pull request #56357 from rhcs-dashboard/sanitize-users-form
Nizamudeen A [Mon, 25 Mar 2024 10:01:15 +0000 (15:31 +0530)]
Merge pull request #56357 from rhcs-dashboard/sanitize-users-form

mgr/dashboard: sanitize dashboard user creation

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agocrimson/osd/osd_operations/snaptrim_event: update PG's stats 56378/head
junxiang Mu [Fri, 22 Mar 2024 02:44:49 +0000 (22:44 -0400)]
crimson/osd/osd_operations/snaptrim_event: update PG's stats

Fixes: https://tracker.ceph.com/issues/63307
Signed-off-by: junxiang Mu <1948535941@qq.com>
15 months agoMerge pull request #56438 from tchaikov/wip-test-unittest-mem-fix-leak
Kefu Chai [Mon, 25 Mar 2024 07:38:17 +0000 (15:38 +0800)]
Merge pull request #56438 from tchaikov/wip-test-unittest-mem-fix-leak

test/common: do not leak in MemoryIsZeroSmallTest

Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>
15 months agocrimson/os/pg_map: allow multiple shards to create new pg mappings at the same time 56332/head
Yingxin Cheng [Wed, 20 Mar 2024 07:25:33 +0000 (15:25 +0800)]
crimson/os/pg_map: allow multiple shards to create new pg mappings at the same time

Also:
* Better detections in case of inconsistent racings, such as:
  * The new mapping is creating towards different cores.
  * Mapping creation is racing with its eracing.
  * Multiple shards are erasing the same mapping at the same time.
* Add more logs to debug in case of unexpected issues.

Fixes: https://tracker.ceph.com/issues/64934
Fixes: https://tracker.ceph.com/issues/64009
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
15 months agomgr/dashboard: add system users to rgw user form 56390/head
Pedro Gonzalez Gomez [Fri, 22 Mar 2024 14:20:48 +0000 (15:20 +0100)]
mgr/dashboard: add system users to rgw user form

Fixes: https://tracker.ceph.com/issues/65074
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
15 months agoMerge PR #56276 into main
Venky Shankar [Mon, 25 Mar 2024 04:57:42 +0000 (10:27 +0530)]
Merge PR #56276 into main

* refs/pull/56276/head:
MDS metrics typo fix

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
15 months agomgr/dashboard: introduce cli commands to set cors urls 56328/head
Nizamudeen A [Wed, 20 Mar 2024 05:51:03 +0000 (11:21 +0530)]
mgr/dashboard: introduce cli commands to set cors urls

The benefit is that this will take care of cherrypy reconfiguration

Fixes: https://tracker.ceph.com/issues/65104
Signed-off-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge PR #56293 into main
Venky Shankar [Mon, 25 Mar 2024 04:51:58 +0000 (10:21 +0530)]
Merge PR #56293 into main

* refs/pull/56293/head:
qa/suites/fs: link in pg_health ignorelist
qa/cephfs: add probabilistic ignorelist for pg_health

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
15 months agoMerge pull request #56164 from rhcs-dashboard/wip-64890-main
Nizamudeen A [Mon, 25 Mar 2024 04:49:40 +0000 (10:19 +0530)]
Merge pull request #56164 from rhcs-dashboard/wip-64890-main

mgr/dashboard: fix NVMeoF API

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56295 from rhcs-dashboard/cpu-usage-rm-danger-indicator
Nizamudeen A [Mon, 25 Mar 2024 04:46:09 +0000 (10:16 +0530)]
Merge pull request #56295 from rhcs-dashboard/cpu-usage-rm-danger-indicator

mgr/dashboard: rm warning/error threshold for cpu usage

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: ivoalmeida <NOT@FOUND>
15 months agoMerge pull request #56297 from rhcs-dashboard/grafana-issue-multi-cluster
Nizamudeen A [Mon, 25 Mar 2024 04:44:44 +0000 (10:14 +0530)]
Merge pull request #56297 from rhcs-dashboard/grafana-issue-multi-cluster

mgr/dashboard: update grafana api url when switching cluster

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
15 months agotest/common: do not leak in MemoryIsZeroSmallTest 56438/head
Kefu Chai [Mon, 25 Mar 2024 03:19:35 +0000 (11:19 +0800)]
test/common: do not leak in MemoryIsZeroSmallTest

before this change, we allocate memory chunks with specified
size using `new []`, but we never free them. when testing with
LeakSanitizer enabled, it rightly points identifies the leakage:

```
Direct leak of 8754 byte(s) in 184 object(s) allocated from:
    #0 0x55c0b2470f0d in operator new[](unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_memory+0x196f0d) (BuildId: d3267dd8819427b804c4729e0467dbe7601fb321)
    #1 0x55c0b247456c in MemoryIsZeroSmallTest_MemoryIsZeroTestSmall_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/common/test_memory.cc:33:18
    #2 0x55c0b2598ee6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x55c0b2553b92 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x55c0b25049dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x55c0b2506a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x55c0b250804b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x55c0b25254d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x55c0b25a16f6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x55c0b255a502 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x55c0b2524862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x55c0b24ab4c0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x55c0b24ab451 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #13 0x7f45e065ad8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we free the allocate memory.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agotest/common: avoid leakage of CephContext 56437/head
Kefu Chai [Sun, 24 Mar 2024 23:43:26 +0000 (07:43 +0800)]
test/common: avoid leakage of CephContext

before this change, in test_util.cc, we increment the refcount of
when constructing it. but at that moment, nobody really owns it.
also, `CephContext` 's refcount is set to 1 in its constructor.
so, we should not do this. otherwise, the created `CephContext`
is leaked as LeakSanitizer rightly points out:
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x5632320d27ed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_util+0x1917ed) (BuildId: ff1df1455bd07b651ad580584a17ea204afeb36e)
    #1 0x7ff9d535b189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7ff9d535a563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7ff9d535a563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7ff9d535a2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7ff9d5354192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    #6 0x7ff9d53471e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7ff9d461d96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7ff9d461c93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x5632320d52e0 in util_collect_sys_info_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/common/test_util.cc:34:27
    #10 0x563232205c16 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x5632321c2742 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x5632321736dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
```
in this change, instead of using a raw pointer, let's
use `boost::intrusive_ptr<CephContext>` to manage the lifecyle
of `CephContext`, this also address the leakage reported by
LeakSanitizer.

the same applies to common/test_context.cc

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agotest: do not increase ref when creating intrusive_ptr<CephContext>
Kefu Chai [Sun, 24 Mar 2024 23:37:58 +0000 (07:37 +0800)]
test: do not increase ref when creating intrusive_ptr<CephContext>

before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance, this not only annoys ASan, and defeats
the purpose of 14d878c8.
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x5564d173537d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_ipaddr+0x19b37d) (BuildId: 45c0c7f28b253c04fcb7bb1a43aed52a5526d734)
    #1 0x7fe7f2ccd189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7fe7f2ccc563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7fe7f2ccc563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7fe7f2ccc2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7fe7f2cc6192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    #6 0x7fe7f2cb91e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7fe7f1f8f96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7fe7f1f8e93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x5564d1752eb9 in pick_address_find_ip_in_subnet_list_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_ipaddr.cc:706:47
    #10 0x5564d18694d6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x5564d1820fc2 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x5564d17d19dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #13 0x5564d17d3a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #14 0x5564d17d504b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #15 0x5564d17f24d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #16 0x5564d1871d06 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #17 0x5564d1827932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #18 0x5564d17f1862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #19 0x5564d1775d80 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #20 0x5564d1775d11 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
```

so, in this change, we do not increase the refcount when
creating cct.

the same applies to `test/common/test_fault_injector.cc`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
15 months agodoc/start: improve MDS explanation 56436/head
Zac Dover [Sun, 24 Mar 2024 21:19:24 +0000 (07:19 +1000)]
doc/start: improve MDS explanation

Improve the explanation of MDS as requested by Anthony D'Atri here:

https://github.com/ceph/ceph/pull/56367#discussion_r1536667249

Signed-off-by: Zac Dover <zac.dover@proton.me>
15 months agoMerge pull request #56287 from rzarzynski/wip-ec-profile-set-paranoid-on-override
Zac Dover [Sun, 24 Mar 2024 20:57:30 +0000 (06:57 +1000)]
Merge pull request #56287 from rzarzynski/wip-ec-profile-set-paranoid-on-override

mon, doc: overriding ec profile requires --yes-i-really-mean-it

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
15 months agodoc/dev: osd_internals/snaps.rst: add clone_overlap doc 56173/head
Matan Breizman [Wed, 13 Mar 2024 14:19:59 +0000 (14:19 +0000)]
doc/dev: osd_internals/snaps.rst: add clone_overlap doc

snapdir concept was removed in: https://github.com/ceph/ceph/pull/17579

Co-authored-by: Zac Dover <zac.dover@proton.me>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
15 months agoMerge pull request #56367 from zdover23/wip-doc-2024-03-21-intro-mds
Zac Dover [Sun, 24 Mar 2024 07:27:20 +0000 (17:27 +1000)]
Merge pull request #56367 from zdover23/wip-doc-2024-03-21-intro-mds

doc/start: improve MDS explanation

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agoMerge pull request #56375 from athanatos/sjust/wip-64996-deepscrub-crash
Samuel Just [Sun, 24 Mar 2024 02:42:51 +0000 (19:42 -0700)]
Merge pull request #56375 from athanatos/sjust/wip-64996-deepscrub-crash

crimson/.../scrub_events: fix progress lifetime in deep_scan_object

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
15 months agoMerge PR #56354 into main
Patrick Donnelly [Sat, 23 Mar 2024 13:02:43 +0000 (09:02 -0400)]
Merge PR #56354 into main

* refs/pull/56354/head:
qa/suites/fs/workload: enable snap_schedule early

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
15 months agomgr/dashboard: sanitize dashboard user creation 56357/head
Pedro Gonzalez Gomez [Thu, 21 Mar 2024 07:56:58 +0000 (08:56 +0100)]
mgr/dashboard: sanitize dashboard user creation

Fixes: https://tracker.ceph.com/issues/65030
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
15 months agoMerge PR #56296 into main
Patrick Donnelly [Fri, 22 Mar 2024 17:54:04 +0000 (13:54 -0400)]
Merge PR #56296 into main

* refs/pull/56296/head:
qa/cephfs: add mgr debugging

Reviewed-by: Venky Shankar <vshankar@redhat.com>
15 months agomgr/dashboard: fix NVMeoF API 56164/head
Ernesto Puerta [Wed, 13 Mar 2024 13:06:10 +0000 (14:06 +0100)]
mgr/dashboard: fix NVMeoF API

* Update NVMe-oF gRPC Proto to 1.0.0
* Error handling,
* Missing PATCH for certain namespace ops (resize, set QoS, set balance
  groups),
* Stop bypassing gRPC payloads and validate those in the back-end,
* Fix incorrect HTTP 1.1 semantics for some POST/DELETE and URIs.
* Catch errors/exceptions.
* Clean-up EndpointDoc Params
* Run Black linter.
* Remove most of NVMeoFClient glue code between gRPC and controller.
* Fix namespace delete endpoint by exposing trsvcid
* nvmeof io_stats support

Fixes: https://tracker.ceph.com/issues/64890
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
15 months agoMerge pull request #56374 from athanatos/sjust/wip-64935-osd-osdmap-destruction-order
Samuel Just [Fri, 22 Mar 2024 17:49:47 +0000 (10:49 -0700)]
Merge pull request #56374 from athanatos/sjust/wip-64935-osd-osdmap-destruction-order

crimson/osd/osd.h: declare osdmap after OSDSingletonState

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>