]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
14 months agocmake: disable WITH_QATLIB/ZIP on non-x86 57520/head
Ken Dreyer [Tue, 14 May 2024 18:53:51 +0000 (14:53 -0400)]
cmake: disable WITH_QATLIB/ZIP on non-x86

This feature is only relevant to x86 hosts.

Signed-off-by: Ken Dreyer <kdreyer@ibm.com>
Fixes: https://tracker.ceph.com/issues/66016
Signed-off-by: Ken Dreyer <kdreyer@ibm.com>
(cherry picked from commit 487cd2fddbab784269af9f48206a130e63f1eca3)

14 months agoMerge pull request #57316 from Matan-B/wip-56027-squid
Matan Breizman [Thu, 16 May 2024 11:36:50 +0000 (14:36 +0300)]
Merge pull request #57316 from Matan-B/wip-56027-squid

squid: doc/dev/crimson: add Code Walkthroughs

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57484 from Matan-B/wip-56534-squid
Matan Breizman [Thu, 16 May 2024 11:36:13 +0000 (14:36 +0300)]
Merge pull request #57484 from Matan-B/wip-56534-squid

squid: crimson: introduce INTERNAL_PG_LOCAL_NS, use for snapmapper

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57483 from Matan-B/wip-56378-squid
Matan Breizman [Thu, 16 May 2024 07:38:03 +0000 (10:38 +0300)]
Merge pull request #57483 from Matan-B/wip-56378-squid

squid: crimson/osd/osd_operations/snaptrim_event: update PG's stats

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57482 from Matan-B/wip-56246-squid
Matan Breizman [Thu, 16 May 2024 07:37:37 +0000 (10:37 +0300)]
Merge pull request #57482 from Matan-B/wip-56246-squid

squid: crimson/os/seastore/btree: check for reserved ptrs when determining children stability

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57481 from Matan-B/wip-55947-squid
Matan Breizman [Thu, 16 May 2024 07:37:01 +0000 (10:37 +0300)]
Merge pull request #57481 from Matan-B/wip-55947-squid

squid: crimson/os/seastore: handle OP_COLL_MOVE_RENAME

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57480 from Matan-B/wip-55708-squid
Matan Breizman [Thu, 16 May 2024 07:36:31 +0000 (10:36 +0300)]
Merge pull request #57480 from Matan-B/wip-55708-squid

squid: crimson/osd/main: Enable multicore cluster messenger

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57270 from Matan-B/wip-55632-squid
Matan Breizman [Thu, 16 May 2024 07:36:00 +0000 (10:36 +0300)]
Merge pull request #57270 from Matan-B/wip-55632-squid

squid: debian/control: add new dependencies protobuf for crismon

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57287 from Matan-B/wip-55751-squid
Matan Breizman [Wed, 15 May 2024 16:19:55 +0000 (19:19 +0300)]
Merge pull request #57287 from Matan-B/wip-55751-squid

squid: test/crimson/test_socket: complete write-side errors in test_unexpected_down()

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agocommon/hobject: make hobject_t formatter methods const 57484/head
Matan Breizman [Wed, 8 May 2024 09:52:19 +0000 (12:52 +0300)]
common/hobject: make hobject_t formatter methods const

this is an alternative to 6021129f0c5c1c133436e471741c73f6e315c5dd
Since https://github.com/ceph/ceph/pull/55583 is not backported to S.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
14 months agocrimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run
Samuel Just [Tue, 26 Mar 2024 06:17:10 +0000 (23:17 -0700)]
crimson/.../scrub_events: use PGBackend::list_objects in ScrubScan::run

Fixes: https://tracker.ceph.com/issues/64975
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit e69883ad2a05afab9d9ab02b098ad2418a4eec8a)

14 months agocrimson/.../scrub_events: convert ScrubScan::run to coroutine
Samuel Just [Tue, 26 Mar 2024 04:20:48 +0000 (21:20 -0700)]
crimson/.../scrub_events: convert ScrubScan::run to coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 0d042f9d18fb191c3c8d6dbbf7c5bae54b8ed0fb)

14 months agocrimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run
Samuel Just [Wed, 27 Mar 2024 01:43:06 +0000 (01:43 +0000)]
crimson/.../scrub_events: use PGBackend::list_objects in ScrubFindRange::run

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 9f39124f33e878e321b330e58e97efc0a1cf1242)

14 months agocrimson/.../scrub_events: convert ScrubFindRange::run to a coroutine
Samuel Just [Wed, 27 Mar 2024 01:39:17 +0000 (01:39 +0000)]
crimson/.../scrub_events: convert ScrubFindRange::run to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 5b0416ae100443f43361e1308e2d28e2763ba725)

14 months agocrimson: move snapmapper object into INTERNAL_PG_LOCAL_NS
Samuel Just [Tue, 26 Mar 2024 05:44:37 +0000 (22:44 -0700)]
crimson: move snapmapper object into INTERNAL_PG_LOCAL_NS

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1b7e3c4fb773a6e0a16145114013a07aa85edcc5)

14 months agocrimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects
Samuel Just [Tue, 26 Mar 2024 05:36:07 +0000 (22:36 -0700)]
crimson/osd/pg_backend: introduce INTERNAL_PG_LOCAL_NS, skip in PGBackend::list_objects

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit f32b37a599592d35ea948e66e7cfdfef7113b8ac)

14 months agocrimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine
Samuel Just [Tue, 26 Mar 2024 05:31:08 +0000 (05:31 +0000)]
crimson/osd/pg_backend: convert PGBackend::list_objects to a coroutine

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 4f056a4c1e9f01e4b04c96e85ebcfa29e6469adc)

14 months agocrimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads
Samuel Just [Tue, 26 Mar 2024 05:13:40 +0000 (22:13 -0700)]
crimson/osd/pg_backend: generalize PGBackend::list_objects, add overloads

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 22a628534307d35e3fa90fb36a98f81cbf7cb7f6)

14 months agocrimson/osd/osd_operations/snaptrim_event: update PG's stats 57483/head
junxiang Mu [Fri, 22 Mar 2024 02:44:49 +0000 (22:44 -0400)]
crimson/osd/osd_operations/snaptrim_event: update PG's stats

Fixes: https://tracker.ceph.com/issues/63307
Signed-off-by: junxiang Mu <1948535941@qq.com>
(cherry picked from commit 4a180f559a3fff1615664898603fc832cfd8d72c)

14 months agocrimson/os/seastore/btree: always check the stability of extents within 57482/head
Xuehan Xu [Thu, 21 Mar 2024 01:54:42 +0000 (09:54 +0800)]
crimson/os/seastore/btree: always check the stability of extents within
the current transaction's view

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 7dcd74ac098201d2ea6f9d1c723f9944ade6bf33)

14 months agocrimson/os/seastore/cached_extent: add comments to elaborate why
Xuehan Xu [Mon, 18 Mar 2024 09:24:02 +0000 (17:24 +0800)]
crimson/os/seastore/cached_extent: add comments to elaborate why
MUTATION_PENDING and under-io extents are "stable"

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit b1c59ca077597e14a053d47d4961b12ba7a2a1b0)

14 months agocrimson/os/seastore/btree: check for reserved ptrs when determining
Xuehan Xu [Mon, 18 Mar 2024 02:54:24 +0000 (10:54 +0800)]
crimson/os/seastore/btree: check for reserved ptrs when determining
children stability

Fixes: https://tracker.ceph.com/issues/64957
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 7aa972def9f53fd9ed2f708aa87ce83362765cbf)

14 months agotest/crimson/seastore: add the test case for renaming objects 57481/head
Xuehan Xu [Thu, 7 Mar 2024 09:06:13 +0000 (17:06 +0800)]
test/crimson/seastore: add the test case for renaming objects

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 2520927590ac48783cb35feb570285ec066964a4)

14 months agocrimson/os/seastore: handle OP_COLL_MOVE_RENAME
Xuehan Xu [Tue, 5 Mar 2024 07:58:05 +0000 (15:58 +0800)]
crimson/os/seastore: handle OP_COLL_MOVE_RENAME

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 70245c5b207eca6349920d1877ea237ec687f93d)

14 months agocrimson/osd/osd: handle_scrub_message on any core 57480/head
Matan Breizman [Thu, 7 Mar 2024 09:49:34 +0000 (09:49 +0000)]
crimson/osd/osd: handle_scrub_message on any core

After enabling multicore cluster messenger, there is no
guarantee which core will handle the request.
See: OSDSingletonState::send_to_osd()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit cbad3e4517e91476b19a8766f98240c81df40a36)

14 months agocrimson/osd/main: Enable multicore cluster messenger
Matan Breizman [Thu, 22 Feb 2024 09:35:14 +0000 (09:35 +0000)]
crimson/osd/main: Enable multicore cluster messenger

Taken from: f78e99c

Co-authored-by: Yingxin Cheng <yingxin.cheng@intel.com>
Co-authored-by: Chunmei Liu <chunmei.liu@intel.com>
Co-authored-by: Xinyu Huang <xinyu.huang@intel.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit ab239ad6cc5eb1cb4ca71d5b409bc4001124342f)

14 months agoMerge pull request #57347 from Matan-B/wip-56375-squid
Matan Breizman [Wed, 15 May 2024 14:36:26 +0000 (17:36 +0300)]
Merge pull request #57347 from Matan-B/wip-56375-squid

squid: crimson/.../scrub_events: fix progress lifetime in deep_scan_object

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57328 from Matan-B/wip-55840-squid
Matan Breizman [Wed, 15 May 2024 14:35:59 +0000 (17:35 +0300)]
Merge pull request #57328 from Matan-B/wip-55840-squid

squid: crimson/os/seastore/omap_manager: expand omap tree nodes

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57327 from Matan-B/wip-55887-squid
Matan Breizman [Wed, 15 May 2024 14:35:24 +0000 (17:35 +0300)]
Merge pull request #57327 from Matan-B/wip-55887-squid

squid: crimson/os/seastore: omap_get_values with range [start, "infinite") should return all results from "start"

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57325 from Matan-B/wip-55889-squid
Matan Breizman [Wed, 15 May 2024 14:34:56 +0000 (17:34 +0300)]
Merge pull request #57325 from Matan-B/wip-55889-squid

squid: crimson/os/seastore/async_cleaner: add comments to clarify what kind of extents would be considered dead by the cleaner

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57321 from Matan-B/wip-55770-squid
Matan Breizman [Wed, 15 May 2024 14:34:31 +0000 (17:34 +0300)]
Merge pull request #57321 from Matan-B/wip-55770-squid

squid: CMakeLists: disable Seastar_IO_URING to fix seastar unittest timeout

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57323 from Matan-B/wip-55801-squid
Matan Breizman [Wed, 15 May 2024 13:47:02 +0000 (16:47 +0300)]
Merge pull request #57323 from Matan-B/wip-55801-squid

squid: crimson/os/alienstore: use homebrew counting_semaphore

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57320 from Matan-B/wip-56028-squid
Matan Breizman [Wed, 15 May 2024 13:46:37 +0000 (16:46 +0300)]
Merge pull request #57320 from Matan-B/wip-56028-squid

squid: crimson/os/seastore: cleanups around lba parent-child pointer

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57317 from Matan-B/wip-55888-squid
Matan Breizman [Wed, 15 May 2024 13:46:12 +0000 (16:46 +0300)]
Merge pull request #57317 from Matan-B/wip-55888-squid

squid: crimson/os/seastore: cleanup and use LBAMapping::is_stable()

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57290 from Matan-B/wip-55806-squid
Matan Breizman [Wed, 15 May 2024 13:35:11 +0000 (16:35 +0300)]
Merge pull request #57290 from Matan-B/wip-55806-squid

squid: crimson/os/seastore: copy attrs and omaps when cloning objects

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57324 from Matan-B/wip-52267-squid
Matan Breizman [Wed, 15 May 2024 13:34:32 +0000 (16:34 +0300)]
Merge pull request #57324 from Matan-B/wip-52267-squid

squid: crimson/osd: put snapmapper's key-value pairs into dedicated objs

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57281 from Matan-B/wip-55684-squid
Matan Breizman [Wed, 15 May 2024 13:33:24 +0000 (16:33 +0300)]
Merge pull request #57281 from Matan-B/wip-55684-squid

squid: crimson: fixes for stack-use-after-return on recent clang and gcc versions

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57350 from Matan-B/wip-56217-squid
Matan Breizman [Wed, 15 May 2024 10:41:40 +0000 (13:41 +0300)]
Merge pull request #57350 from Matan-B/wip-56217-squid

squid: test/crimson/sesastore/test_block: add constructor to construct

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57349 from Matan-B/wip-56462-squid
Matan Breizman [Wed, 15 May 2024 10:41:19 +0000 (13:41 +0300)]
Merge pull request #57349 from Matan-B/wip-56462-squid

squid: crimson/os/seastore: fix use after free error

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57348 from Matan-B/wip-56332-squid
Matan Breizman [Wed, 15 May 2024 10:40:56 +0000 (13:40 +0300)]
Merge pull request #57348 from Matan-B/wip-56332-squid

squid: crimson/os/pg_map: allow multiple shards to create new pg mappings at the same time

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57346 from Matan-B/wip-56374-squid
Matan Breizman [Wed, 15 May 2024 10:39:05 +0000 (13:39 +0300)]
Merge pull request #57346 from Matan-B/wip-56374-squid

squid: crimson/osd/osd.h: declare osdmap after OSDSingletonState

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57345 from Matan-B/wip-56250-squid
Matan Breizman [Wed, 15 May 2024 10:38:40 +0000 (13:38 +0300)]
Merge pull request #57345 from Matan-B/wip-56250-squid

squid: crimson/os/seastore: cache metadata during trimming to prevent from disk read

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57344 from Matan-B/wip-55846-squid
Matan Breizman [Wed, 15 May 2024 10:38:12 +0000 (13:38 +0300)]
Merge pull request #57344 from Matan-B/wip-55846-squid

squid: crimson: add coroutine support for errorated and interruptible futures

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57326 from Matan-B/wip-55855-squid
Matan Breizman [Wed, 15 May 2024 10:34:26 +0000 (13:34 +0300)]
Merge pull request #57326 from Matan-B/wip-55855-squid

squid: crimson/os/seastore: adjust SeaStore::_omap_set_kvs() params

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57322 from Matan-B/wip-55981-squid
Matan Breizman [Wed, 15 May 2024 10:27:25 +0000 (13:27 +0300)]
Merge pull request #57322 from Matan-B/wip-55981-squid

squid: crimson/os/seastore/collection_manager: allow empty-delta-buffer collection nodes

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57315 from Matan-B/wip-55977-squid
Matan Breizman [Wed, 15 May 2024 10:21:14 +0000 (13:21 +0300)]
Merge pull request #57315 from Matan-B/wip-55977-squid

squid: crimson/os/seastore: allow to remap the dirty extent

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57314 from Matan-B/wip-55536-squid
Matan Breizman [Wed, 15 May 2024 10:20:41 +0000 (13:20 +0300)]
Merge pull request #57314 from Matan-B/wip-55536-squid

squid: osd/SnapMapper: Crimson - fix OSDriver::get_next

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
14 months agoMerge pull request #57312 from Matan-B/wip-55910-squid
Matan Breizman [Wed, 15 May 2024 10:19:39 +0000 (13:19 +0300)]
Merge pull request #57312 from Matan-B/wip-55910-squid

squid: osd/SnapManager: Crimson - fix std::string constructed from 0

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57311 from Matan-B/wip-55946-squid
Matan Breizman [Wed, 15 May 2024 10:19:09 +0000 (13:19 +0300)]
Merge pull request #57311 from Matan-B/wip-55946-squid

squid: crimson/os/seastore: make less verbose log when testing performance on RBM

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57289 from Matan-B/wip-55772-squid
Matan Breizman [Wed, 15 May 2024 10:18:00 +0000 (13:18 +0300)]
Merge pull request #57289 from Matan-B/wip-55772-squid

squid: crimson/os/seastore: two attrs related bug fixes

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57288 from Matan-B/wip-55705-squid
Matan Breizman [Wed, 15 May 2024 10:17:31 +0000 (13:17 +0300)]
Merge pull request #57288 from Matan-B/wip-55705-squid

squid: crimson/os/seastore: fix wrong-positioned asserts

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57286 from Matan-B/wip-55407-squid
Matan Breizman [Wed, 15 May 2024 10:15:37 +0000 (13:15 +0300)]
Merge pull request #57286 from Matan-B/wip-55407-squid

squid: crimson/osd/shard_services: make sure that only up/acting members can create pgs

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57283 from Matan-B/wip-55641-squid
Matan Breizman [Wed, 15 May 2024 10:15:11 +0000 (13:15 +0300)]
Merge pull request #57283 from Matan-B/wip-55641-squid

squid: crimson/osd/main: Enable multicore client messenger

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #56508 from rosinL/wip-65164-squid
Matan Breizman [Wed, 15 May 2024 10:11:29 +0000 (13:11 +0300)]
Merge pull request #56508 from rosinL/wip-65164-squid

squid: cmake/AddCephTest: bind crimson unittest to different cores

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
14 months agoMerge pull request #57465 from idryomov/wip-51845-squid
Ilya Dryomov [Wed, 15 May 2024 07:32:04 +0000 (09:32 +0200)]
Merge pull request #57465 from idryomov/wip-51845-squid

squid: qa/suites/krbd: drop pre-single-major and move "layering only" coverage

Reviewed-by: Ramana Raja <rraja@redhat.com>
14 months agoMerge pull request #57461 from frittentheke/wip-65838-squid
Aashish Sharma [Wed, 15 May 2024 07:16:03 +0000 (12:46 +0530)]
Merge pull request #57461 from frittentheke/wip-65838-squid

squid: monitoring/ceph-mixin: Cleanup of variables, queries and tests (to fix showMultiCluster=True)

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
14 months agoqa/suites/krbd: rename no-object-map to no-exclusive-lock 57465/head
Ilya Dryomov [Mon, 13 May 2024 11:45:26 +0000 (13:45 +0200)]
qa/suites/krbd: rename no-object-map to no-exclusive-lock

Exclusive lock has always been disabled by this facet, so it might as
well be reflected in its name.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ad6a95d8af42b1c3ed9d244c6161e5269356c7c9)

14 months agoqa/suites/krbd: move "layering only" coverage to fsx
Ilya Dryomov [Mon, 13 May 2024 11:31:17 +0000 (13:31 +0200)]
qa/suites/krbd: move "layering only" coverage to fsx

It makes much more sense there since it's where we actually create
clones and flatten them a lot.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7b9f28e74325aad17c3774d594c3f0e1c4e25dc2)

14 months agoqa/suites/krbd: drop pre-single-major test
Ilya Dryomov [Sun, 12 May 2024 09:15:36 +0000 (11:15 +0200)]
qa/suites/krbd: drop pre-single-major test

Single-major mapping scheme was introduced in 2014 and became the
default in 2017.  It's getting increasingly difficult to build and,
more importantly, to boot a 10 year old kernel with recent userspace
(systemd, etc).  If someone is still running such a kernel, it's
really unlikely that they would have the most recent rbd CLI tool
installed.

Fixes: https://tracker.ceph.com/issues/51845
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 39a579144cd8b97dd338bfcf36c06ad4bcaed526)

14 months agoCleanup of variables, queries and tests to enable showMultiCluster=True 57461/head
Christian Rohmann [Fri, 26 Jan 2024 14:05:52 +0000 (15:05 +0100)]
Cleanup of variables, queries and tests to enable showMultiCluster=True

Rendering the dashboards with showMultiCluster=True allows for
them to work with multiple clusters storing their metrics in a single
Prometheus instance. This works via the cluster label and that functionality
already existed. This just fixes some inconsistencies in applying the label
filters.

Additionally this contains updates to the tests to have them succeed with
with both configurations and avoid the introduction of regressions in
regards to multiCluster in the future.

There also are some consistency cleanups here and there:
 * `datasource` was not used consistently
 * `cluster` label_values are determined from `ceph_health_status`
 * `job` template and filters on this label were removed to align multi cluster
    support solely via the `cluster` label
 * `ceph_hosts` filter now uses label_values from any ceph_metadata metrici
    to now show all instance values, but those of hosts with some Ceph
    component / daemon.
 *  Enable showMultiCluster=True since `cluster` label is now always present,
    via https://github.com/ceph/ceph/pull/54964

Improves: https://tracker.ceph.com/issues/64321
Signed-off-by: Christian Rohmann <christian.rohmann@inovex.de>
(cherry picked from commit 090b8e17f1e84d8b20e05143d5dd7ff107031176)

14 months agoMerge pull request #57426 from zdover23/wip-doc-2024-05-13-backport-57420-to-squid
Anthony D'Atri [Sun, 12 May 2024 21:36:53 +0000 (17:36 -0400)]
Merge pull request #57426 from zdover23/wip-doc-2024-05-13-backport-57420-to-squid

squid: doc/cephfs: edit fs-volumes.rst (1 of x) followup

14 months agodoc/cephfs: edit fs-volumes.rst (1 of x) followup 57426/head
Zac Dover [Sun, 12 May 2024 01:39:34 +0000 (11:39 +1000)]
doc/cephfs: edit fs-volumes.rst (1 of x) followup

Include the suggestions for improving doc/cephfs/fs-volumes.rst made by
Anthony D'Atri here
https://github.com/ceph/ceph/pull/57415#discussion_r1597362110

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit cb700d804b4390fd9f55444dcfc04dfebac3a1bf)

14 months agoMerge pull request #57417 from zdover23/wip-doc-2024-05-12-backport-57415-to-squid
Zac Dover [Sun, 12 May 2024 20:55:15 +0000 (06:55 +1000)]
Merge pull request #57417 from zdover23/wip-doc-2024-05-12-backport-57415-to-squid

squid: doc/cephfs: edit fs-volumes.rst (1 of x)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agoMerge pull request #56519 from rhcs-dashboard/wip-65133-squid
Avan [Sun, 12 May 2024 07:36:18 +0000 (13:06 +0530)]
Merge pull request #56519 from rhcs-dashboard/wip-65133-squid

squid: mgr/dashboard: add support for NFSv3 exports

Reviewed-by: afreen23 <NOT@FOUND>
14 months agoMerge pull request #57422 from zdover23/wip-doc-2024-05-12-backport-57421-to-squid
Anthony D'Atri [Sun, 12 May 2024 05:25:23 +0000 (01:25 -0400)]
Merge pull request #57422 from zdover23/wip-doc-2024-05-12-backport-57421-to-squid

squid: doc/dev: s/to asses/to assess/

14 months agodoc/dev: s/to asses/to assess/ 57422/head
Zac Dover [Sun, 12 May 2024 02:27:23 +0000 (12:27 +1000)]
doc/dev: s/to asses/to assess/

s/to asses/to assess/ in doc/deve/developer_guide/dash-devel.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 832358b072ec5cb4672de8a7a7aebf10a39fd0e5)

14 months agodoc/cephfs: edit fs-volumes.rst (1 of x) 57417/head
Zac Dover [Sat, 11 May 2024 04:57:23 +0000 (14:57 +1000)]
doc/cephfs: edit fs-volumes.rst (1 of x)

Edit doc/cephfs/fs-volumes to the section "FS Subvolumes" (but not
including the section "FS Subvolumes").

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0acbb278fc712c32334fd7340b887d2865dbfb86)

14 months agoMerge pull request #57390 from zdover23/wip-doc-2024-05-10-backport-57372-to-squid
Zac Dover [Sat, 11 May 2024 23:47:27 +0000 (09:47 +1000)]
Merge pull request #57390 from zdover23/wip-doc-2024-05-10-backport-57372-to-squid

squid: doc/cephfs: expand the role of quiesce in consistent backups

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
14 months agoMerge pull request #57364 from zdover23/wip-doc-2024-05-09-radosgw-bucketpolicy-squid
Zac Dover [Fri, 10 May 2024 11:21:17 +0000 (21:21 +1000)]
Merge pull request #57364 from zdover23/wip-doc-2024-05-09-radosgw-bucketpolicy-squid

squid: doc/radosgw: update Squid S3 action list

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agoMerge pull request #57335 from zdover23/wip-doc-2024-05-08-backport-57298-to-squid
Zac Dover [Fri, 10 May 2024 11:12:17 +0000 (21:12 +1000)]
Merge pull request #57335 from zdover23/wip-doc-2024-05-08-backport-57298-to-squid

squid: doc/mgr: edit "Overview" in dashboard.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agoMerge pull request #57295 from zdover23/wip-doc-2024-05-07-backport-57273-to-squid
Zac Dover [Fri, 10 May 2024 11:05:46 +0000 (21:05 +1000)]
Merge pull request #57295 from zdover23/wip-doc-2024-05-07-backport-57273-to-squid

squid: doc/mgr: edit "Resolve IP address to hostname before redirect"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agodoc/cephfs: expand the role of quiesce in consistent backups 57390/head
Leonid Usov [Thu, 9 May 2024 12:29:33 +0000 (15:29 +0300)]
doc/cephfs: expand the role of quiesce in consistent backups

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
Fixes: https://tracker.ceph.com/issues/65881
(cherry picked from commit 34ec2a4c6a065028d593b1e959600a3ce87541a8)

14 months agoMerge pull request #57310 from idryomov/wip-65624-squid
Ilya Dryomov [Fri, 10 May 2024 10:35:11 +0000 (12:35 +0200)]
Merge pull request #57310 from idryomov/wip-65624-squid

squid: pybind/rbd: expose CLONE_FORMAT and FLATTEN image options

Reviewed-by: Ramana Raja <rraja@redhat.com>
14 months agoMerge pull request #57307 from idryomov/wip-65487-squid
Ilya Dryomov [Fri, 10 May 2024 10:35:00 +0000 (12:35 +0200)]
Merge pull request #57307 from idryomov/wip-65487-squid

squid: rbd-mirror: clean up stale pool replayers and callouts better

Reviewed-by: Ramana Raja <rraja@redhat.com>
14 months agoMerge pull request #57031 from idryomov/wip-65481-squid
Ilya Dryomov [Fri, 10 May 2024 10:34:46 +0000 (12:34 +0200)]
Merge pull request #57031 from idryomov/wip-65481-squid

squid: qa: fix krbd_msgr_segments and krbd_rxbounce failing on 8.stream

Reviewed-by: Ramana Raja <rraja@redhat.com>
14 months agoMerge pull request #57370 from weirdwiz/wip-65879-squid
Avan [Fri, 10 May 2024 09:36:49 +0000 (15:06 +0530)]
Merge pull request #57370 from weirdwiz/wip-65879-squid

squid: exporter: handle exceptions gracefully

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
14 months agoexporter: handle exceptions gracefully 57370/head
Divyansh Kamboj [Tue, 30 Apr 2024 07:44:10 +0000 (13:14 +0530)]
exporter: handle exceptions gracefully

ceph exporter crahes, and fails to handle exceptions in
dump_asok_metrics(). add try and catch blocks to handle the exceptions gracefully.

Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
(cherry picked from commit e442ee479987f339afdc333582b6b64c66ba6846)

14 months agoMerge pull request #57303 from ronen-fr/wip-rf-reinstate-q-squid
Laura Flores [Wed, 8 May 2024 23:05:43 +0000 (18:05 -0500)]
Merge pull request #57303 from ronen-fr/wip-rf-reinstate-q-squid

squid:osd/scrub: reinstate scrub reservation queuing

14 months agodoc/radosgw: update Squid S3 action list 57364/head
Zac Dover [Wed, 8 May 2024 22:10:25 +0000 (08:10 +1000)]
doc/radosgw: update Squid S3 action list

Update the list of S3 actions supported in the Squid release of Ceph.

Note: The list of S3 actions supported in Squid is not the list of S3
actions supported in Reef or in Quincy.

Signed-off-by: Zac Dover <zac.dover@proton.me>
14 months agoMerge pull request #57092 from idryomov/wip-65573-squid
Ilya Dryomov [Wed, 8 May 2024 21:28:50 +0000 (23:28 +0200)]
Merge pull request #57092 from idryomov/wip-65573-squid

squid: librbd: make group and group snapshot IDs more random

Reviewed-by: Mykola Golub <mgolub@suse.com>
14 months agoMerge pull request #57242 from cbodley/wip-65767-squid
Casey Bodley [Wed, 8 May 2024 20:23:20 +0000 (21:23 +0100)]
Merge pull request #57242 from cbodley/wip-65767-squid

squid: rgw/multisite-notification: retry storing bucket notification attrs for ECANCELED(ConcurrentModification) errors.

Reviewed-by: Krunal Chheda <kchheda3@bloomberg.net>
14 months agoMerge pull request #57006 from rzarzynski/wip-65185-squid
Laura Flores [Wed, 8 May 2024 17:52:34 +0000 (12:52 -0500)]
Merge pull request #57006 from rzarzynski/wip-65185-squid

squid: osd/PGBackend::be_scan_list: only call stat, getattrs once per object

14 months agoMerge pull request #57300 from cbodley/wip-65821-squid
Casey Bodley [Wed, 8 May 2024 16:47:08 +0000 (17:47 +0100)]
Merge pull request #57300 from cbodley/wip-65821-squid

squid: rgw: fix CompleteMultipart error handling regression

Reviewed-by: Ali Masarwa <amasarwa@redhat.com>
14 months agoMerge pull request #57104 from soumyakoduri/wip-skoduri-squid
Casey Bodley [Wed, 8 May 2024 15:13:35 +0000 (16:13 +0100)]
Merge pull request #57104 from soumyakoduri/wip-skoduri-squid

[squid] rgw/lc: advance head if the current entry doesn't exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
14 months agoMerge pull request #57231 from galsalomon66/wip-65244-squid
Casey Bodley [Wed, 8 May 2024 15:13:23 +0000 (16:13 +0100)]
Merge pull request #57231 from galsalomon66/wip-65244-squid

squid: rgw/s3select: s3select response handler refactor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
14 months agoMerge pull request #57254 from aaSharma14/wip-65790-squid
Aashish Sharma [Wed, 8 May 2024 09:24:47 +0000 (14:54 +0530)]
Merge pull request #57254 from aaSharma14/wip-65790-squid

squid: mgr/dashboard: add prometheus federation config for multi-cluster monitoring

Reviewed-by: Nizamudeen A <nia@redhat.com>
14 months agotest/crimson/sesastore/test_block: add constructor to construct 57350/head
Xuehan Xu [Fri, 15 Mar 2024 11:48:33 +0000 (19:48 +0800)]
test/crimson/sesastore/test_block: add constructor to construct
non-fully-loaded TestBlocks

Before this commit, all TestBlocks are fully-loaded whether they do have
data or not.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 61d7f8821fa06cf48911230228687abdf747dba2)

14 months agocrimson/os/seastore: fix use after free error 57349/head
Zhang Song [Tue, 26 Mar 2024 06:15:35 +0000 (14:15 +0800)]
crimson/os/seastore: fix use after free error

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
(cherry picked from commit 1b5ca67ef8e9de48221d52b6e46f89aad1a731eb)

14 months agocrimson/os/pg_map: allow multiple shards to create new pg mappings at the same time 57348/head
Yingxin Cheng [Wed, 20 Mar 2024 07:25:33 +0000 (15:25 +0800)]
crimson/os/pg_map: allow multiple shards to create new pg mappings at the same time

Also:
* Better detections in case of inconsistent racings, such as:
  * The new mapping is creating towards different cores.
  * Mapping creation is racing with its eracing.
  * Multiple shards are erasing the same mapping at the same time.
* Add more logs to debug in case of unexpected issues.

Fixes: https://tracker.ceph.com/issues/64934
Fixes: https://tracker.ceph.com/issues/64009
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 2422817b7578da44090ea5e2f540c505bbce1412)

14 months agocrimson/osd/shard_services: fix log
Yingxin Cheng [Wed, 20 Mar 2024 07:20:57 +0000 (15:20 +0800)]
crimson/osd/shard_services: fix log

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit de2e66723586d2e9c5fd838e5199547b0d2d5f4f)

14 months agocrimson/osd/pg_map/PGShardMapping: cleanups
Yingxin Cheng [Tue, 19 Mar 2024 03:05:11 +0000 (11:05 +0800)]
crimson/osd/pg_map/PGShardMapping: cleanups

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit b35ed71e23ac5eabfed1c47361ee499bab77cd14)

14 months agocrimson/.../scrub_events: fix progress lifetime in deep_scan_object 57347/head
Samuel Just [Tue, 19 Mar 2024 23:11:48 +0000 (23:11 +0000)]
crimson/.../scrub_events: fix progress lifetime in deep_scan_object

seastar::repeat may or may not move the passed action after the first
invocation, so we can't actually rely on references to variables
captured by value being stable.  Instead, allocate with std::make_unique
and move into a finally lambda.

Fixes: https://tracker.ceph.com/issues/64996
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 3276ad571640bc91fe290832a0001bd96f1f26de)

14 months agocrimson/osd/osd.h: declare osdmap after OSDSingletonState 57346/head
Samuel Just [Mon, 18 Mar 2024 23:26:36 +0000 (23:26 +0000)]
crimson/osd/osd.h: declare osdmap after OSDSingletonState

Otherwise, destructing the osdmap cached_map_t tries to manipulate
OSDSingletonState::osdmaps after it has already been destroyed.

Fixes: https://tracker.ceph.com/issues/64935
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit b623594c44df1c7d75fa7a8d4e8b90f8829d249f)

14 months agocrimson/os/seastore: cache metadata during trimming to prevent from disk read 57345/head
myoungwon oh [Mon, 18 Mar 2024 06:48:07 +0000 (06:48 +0000)]
crimson/os/seastore: cache metadata during trimming to prevent from disk read

I encountered continous disk reads during trimming even though there are sufficient
cache available, in 4K random write test with RBM (RBD).
This is because metadata is note cached if its source is background transaction
within touch_extent(). So, seastore, including the trimming process,  needs to
constantly retrieve metadata (e.g., BACKREF_LEAF).

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 7f253ce5aa98018b5b209fd6e062078d151ad567)

14 months agocrimson/common: add coroutine integration for crimson futures 57344/head
Samuel Just [Wed, 31 Jan 2024 23:34:34 +0000 (15:34 -0800)]
crimson/common: add coroutine integration for crimson futures

Adds coroutine machinery for crimson errorated and interruptible futures.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1126ec3cf6285abbf6a875ab27daff397f09220b)

14 months agocrimson/common: expose unsafe_get0 for interruptible errorated futures
Samuel Just [Sun, 4 Feb 2024 06:47:04 +0000 (22:47 -0800)]
crimson/common: expose unsafe_get0 for interruptible errorated futures

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 11698a6da9d518329003a90777b0ef31454d2bfd)

14 months agocrimson/common: add interruptible_errorator::make_interruptible
Samuel Just [Sun, 4 Feb 2024 06:46:34 +0000 (22:46 -0800)]
crimson/common: add interruptible_errorator::make_interruptible

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 52800dcee08164f9d17d9d293e01cce173670f63)

14 months agocrimson/.../errorator: expose get_exception
Samuel Just [Thu, 8 Feb 2024 01:25:38 +0000 (17:25 -0800)]
crimson/.../errorator: expose get_exception

Useful for intentionally ignoring an exceptional future.  Otherwise,
seastar warns.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 0afd9fa4326d4e7fe5966347a37623379072fd98)

14 months agocrimson/common/interruptible_future: add interruptor::future convenience declaration
Samuel Just [Wed, 31 Jan 2024 23:33:33 +0000 (15:33 -0800)]
crimson/common/interruptible_future: add interruptor::future convenience declaration

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1f152bf9861dd04ace70a62ba8a82c33136b5fb0)

14 months agocrimson/common: expose set_coroutine for crimson future specializations/wrappers
Samuel Just [Wed, 31 Jan 2024 23:32:30 +0000 (15:32 -0800)]
crimson/common: expose set_coroutine for crimson future specializations/wrappers

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 53c4acde1d6b00ed703a71c3137bee92df4be46a)