]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
14 months agotest/crimson/seastore: add the test case for renaming objects 57481/head
Xuehan Xu [Thu, 7 Mar 2024 09:06:13 +0000 (17:06 +0800)]
test/crimson/seastore: add the test case for renaming objects

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 2520927590ac48783cb35feb570285ec066964a4)

14 months agocrimson/os/seastore: handle OP_COLL_MOVE_RENAME
Xuehan Xu [Tue, 5 Mar 2024 07:58:05 +0000 (15:58 +0800)]
crimson/os/seastore: handle OP_COLL_MOVE_RENAME

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 70245c5b207eca6349920d1877ea237ec687f93d)

14 months agoMerge pull request #57347 from Matan-B/wip-56375-squid
Matan Breizman [Wed, 15 May 2024 14:36:26 +0000 (17:36 +0300)]
Merge pull request #57347 from Matan-B/wip-56375-squid

squid: crimson/.../scrub_events: fix progress lifetime in deep_scan_object

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57328 from Matan-B/wip-55840-squid
Matan Breizman [Wed, 15 May 2024 14:35:59 +0000 (17:35 +0300)]
Merge pull request #57328 from Matan-B/wip-55840-squid

squid: crimson/os/seastore/omap_manager: expand omap tree nodes

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57327 from Matan-B/wip-55887-squid
Matan Breizman [Wed, 15 May 2024 14:35:24 +0000 (17:35 +0300)]
Merge pull request #57327 from Matan-B/wip-55887-squid

squid: crimson/os/seastore: omap_get_values with range [start, "infinite") should return all results from "start"

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57325 from Matan-B/wip-55889-squid
Matan Breizman [Wed, 15 May 2024 14:34:56 +0000 (17:34 +0300)]
Merge pull request #57325 from Matan-B/wip-55889-squid

squid: crimson/os/seastore/async_cleaner: add comments to clarify what kind of extents would be considered dead by the cleaner

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57321 from Matan-B/wip-55770-squid
Matan Breizman [Wed, 15 May 2024 14:34:31 +0000 (17:34 +0300)]
Merge pull request #57321 from Matan-B/wip-55770-squid

squid: CMakeLists: disable Seastar_IO_URING to fix seastar unittest timeout

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57323 from Matan-B/wip-55801-squid
Matan Breizman [Wed, 15 May 2024 13:47:02 +0000 (16:47 +0300)]
Merge pull request #57323 from Matan-B/wip-55801-squid

squid: crimson/os/alienstore: use homebrew counting_semaphore

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57320 from Matan-B/wip-56028-squid
Matan Breizman [Wed, 15 May 2024 13:46:37 +0000 (16:46 +0300)]
Merge pull request #57320 from Matan-B/wip-56028-squid

squid: crimson/os/seastore: cleanups around lba parent-child pointer

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57317 from Matan-B/wip-55888-squid
Matan Breizman [Wed, 15 May 2024 13:46:12 +0000 (16:46 +0300)]
Merge pull request #57317 from Matan-B/wip-55888-squid

squid: crimson/os/seastore: cleanup and use LBAMapping::is_stable()

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57290 from Matan-B/wip-55806-squid
Matan Breizman [Wed, 15 May 2024 13:35:11 +0000 (16:35 +0300)]
Merge pull request #57290 from Matan-B/wip-55806-squid

squid: crimson/os/seastore: copy attrs and omaps when cloning objects

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57324 from Matan-B/wip-52267-squid
Matan Breizman [Wed, 15 May 2024 13:34:32 +0000 (16:34 +0300)]
Merge pull request #57324 from Matan-B/wip-52267-squid

squid: crimson/osd: put snapmapper's key-value pairs into dedicated objs

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57281 from Matan-B/wip-55684-squid
Matan Breizman [Wed, 15 May 2024 13:33:24 +0000 (16:33 +0300)]
Merge pull request #57281 from Matan-B/wip-55684-squid

squid: crimson: fixes for stack-use-after-return on recent clang and gcc versions

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57350 from Matan-B/wip-56217-squid
Matan Breizman [Wed, 15 May 2024 10:41:40 +0000 (13:41 +0300)]
Merge pull request #57350 from Matan-B/wip-56217-squid

squid: test/crimson/sesastore/test_block: add constructor to construct

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57349 from Matan-B/wip-56462-squid
Matan Breizman [Wed, 15 May 2024 10:41:19 +0000 (13:41 +0300)]
Merge pull request #57349 from Matan-B/wip-56462-squid

squid: crimson/os/seastore: fix use after free error

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57348 from Matan-B/wip-56332-squid
Matan Breizman [Wed, 15 May 2024 10:40:56 +0000 (13:40 +0300)]
Merge pull request #57348 from Matan-B/wip-56332-squid

squid: crimson/os/pg_map: allow multiple shards to create new pg mappings at the same time

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57346 from Matan-B/wip-56374-squid
Matan Breizman [Wed, 15 May 2024 10:39:05 +0000 (13:39 +0300)]
Merge pull request #57346 from Matan-B/wip-56374-squid

squid: crimson/osd/osd.h: declare osdmap after OSDSingletonState

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57345 from Matan-B/wip-56250-squid
Matan Breizman [Wed, 15 May 2024 10:38:40 +0000 (13:38 +0300)]
Merge pull request #57345 from Matan-B/wip-56250-squid

squid: crimson/os/seastore: cache metadata during trimming to prevent from disk read

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57344 from Matan-B/wip-55846-squid
Matan Breizman [Wed, 15 May 2024 10:38:12 +0000 (13:38 +0300)]
Merge pull request #57344 from Matan-B/wip-55846-squid

squid: crimson: add coroutine support for errorated and interruptible futures

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57326 from Matan-B/wip-55855-squid
Matan Breizman [Wed, 15 May 2024 10:34:26 +0000 (13:34 +0300)]
Merge pull request #57326 from Matan-B/wip-55855-squid

squid: crimson/os/seastore: adjust SeaStore::_omap_set_kvs() params

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57322 from Matan-B/wip-55981-squid
Matan Breizman [Wed, 15 May 2024 10:27:25 +0000 (13:27 +0300)]
Merge pull request #57322 from Matan-B/wip-55981-squid

squid: crimson/os/seastore/collection_manager: allow empty-delta-buffer collection nodes

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57315 from Matan-B/wip-55977-squid
Matan Breizman [Wed, 15 May 2024 10:21:14 +0000 (13:21 +0300)]
Merge pull request #57315 from Matan-B/wip-55977-squid

squid: crimson/os/seastore: allow to remap the dirty extent

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57314 from Matan-B/wip-55536-squid
Matan Breizman [Wed, 15 May 2024 10:20:41 +0000 (13:20 +0300)]
Merge pull request #57314 from Matan-B/wip-55536-squid

squid: osd/SnapMapper: Crimson - fix OSDriver::get_next

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
14 months agoMerge pull request #57312 from Matan-B/wip-55910-squid
Matan Breizman [Wed, 15 May 2024 10:19:39 +0000 (13:19 +0300)]
Merge pull request #57312 from Matan-B/wip-55910-squid

squid: osd/SnapManager: Crimson - fix std::string constructed from 0

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57311 from Matan-B/wip-55946-squid
Matan Breizman [Wed, 15 May 2024 10:19:09 +0000 (13:19 +0300)]
Merge pull request #57311 from Matan-B/wip-55946-squid

squid: crimson/os/seastore: make less verbose log when testing performance on RBM

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57289 from Matan-B/wip-55772-squid
Matan Breizman [Wed, 15 May 2024 10:18:00 +0000 (13:18 +0300)]
Merge pull request #57289 from Matan-B/wip-55772-squid

squid: crimson/os/seastore: two attrs related bug fixes

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57288 from Matan-B/wip-55705-squid
Matan Breizman [Wed, 15 May 2024 10:17:31 +0000 (13:17 +0300)]
Merge pull request #57288 from Matan-B/wip-55705-squid

squid: crimson/os/seastore: fix wrong-positioned asserts

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57286 from Matan-B/wip-55407-squid
Matan Breizman [Wed, 15 May 2024 10:15:37 +0000 (13:15 +0300)]
Merge pull request #57286 from Matan-B/wip-55407-squid

squid: crimson/osd/shard_services: make sure that only up/acting members can create pgs

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #57283 from Matan-B/wip-55641-squid
Matan Breizman [Wed, 15 May 2024 10:15:11 +0000 (13:15 +0300)]
Merge pull request #57283 from Matan-B/wip-55641-squid

squid: crimson/osd/main: Enable multicore client messenger

Reviewed-by: Samuel Just <sjust@redhat.com>
14 months agoMerge pull request #56508 from rosinL/wip-65164-squid
Matan Breizman [Wed, 15 May 2024 10:11:29 +0000 (13:11 +0300)]
Merge pull request #56508 from rosinL/wip-65164-squid

squid: cmake/AddCephTest: bind crimson unittest to different cores

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
14 months agoMerge pull request #57465 from idryomov/wip-51845-squid
Ilya Dryomov [Wed, 15 May 2024 07:32:04 +0000 (09:32 +0200)]
Merge pull request #57465 from idryomov/wip-51845-squid

squid: qa/suites/krbd: drop pre-single-major and move "layering only" coverage

Reviewed-by: Ramana Raja <rraja@redhat.com>
14 months agoMerge pull request #57461 from frittentheke/wip-65838-squid
Aashish Sharma [Wed, 15 May 2024 07:16:03 +0000 (12:46 +0530)]
Merge pull request #57461 from frittentheke/wip-65838-squid

squid: monitoring/ceph-mixin: Cleanup of variables, queries and tests (to fix showMultiCluster=True)

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
14 months agoqa/suites/krbd: rename no-object-map to no-exclusive-lock 57465/head
Ilya Dryomov [Mon, 13 May 2024 11:45:26 +0000 (13:45 +0200)]
qa/suites/krbd: rename no-object-map to no-exclusive-lock

Exclusive lock has always been disabled by this facet, so it might as
well be reflected in its name.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ad6a95d8af42b1c3ed9d244c6161e5269356c7c9)

14 months agoqa/suites/krbd: move "layering only" coverage to fsx
Ilya Dryomov [Mon, 13 May 2024 11:31:17 +0000 (13:31 +0200)]
qa/suites/krbd: move "layering only" coverage to fsx

It makes much more sense there since it's where we actually create
clones and flatten them a lot.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7b9f28e74325aad17c3774d594c3f0e1c4e25dc2)

14 months agoqa/suites/krbd: drop pre-single-major test
Ilya Dryomov [Sun, 12 May 2024 09:15:36 +0000 (11:15 +0200)]
qa/suites/krbd: drop pre-single-major test

Single-major mapping scheme was introduced in 2014 and became the
default in 2017.  It's getting increasingly difficult to build and,
more importantly, to boot a 10 year old kernel with recent userspace
(systemd, etc).  If someone is still running such a kernel, it's
really unlikely that they would have the most recent rbd CLI tool
installed.

Fixes: https://tracker.ceph.com/issues/51845
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 39a579144cd8b97dd338bfcf36c06ad4bcaed526)

14 months agoCleanup of variables, queries and tests to enable showMultiCluster=True 57461/head
Christian Rohmann [Fri, 26 Jan 2024 14:05:52 +0000 (15:05 +0100)]
Cleanup of variables, queries and tests to enable showMultiCluster=True

Rendering the dashboards with showMultiCluster=True allows for
them to work with multiple clusters storing their metrics in a single
Prometheus instance. This works via the cluster label and that functionality
already existed. This just fixes some inconsistencies in applying the label
filters.

Additionally this contains updates to the tests to have them succeed with
with both configurations and avoid the introduction of regressions in
regards to multiCluster in the future.

There also are some consistency cleanups here and there:
 * `datasource` was not used consistently
 * `cluster` label_values are determined from `ceph_health_status`
 * `job` template and filters on this label were removed to align multi cluster
    support solely via the `cluster` label
 * `ceph_hosts` filter now uses label_values from any ceph_metadata metrici
    to now show all instance values, but those of hosts with some Ceph
    component / daemon.
 *  Enable showMultiCluster=True since `cluster` label is now always present,
    via https://github.com/ceph/ceph/pull/54964

Improves: https://tracker.ceph.com/issues/64321
Signed-off-by: Christian Rohmann <christian.rohmann@inovex.de>
(cherry picked from commit 090b8e17f1e84d8b20e05143d5dd7ff107031176)

14 months agoMerge pull request #57426 from zdover23/wip-doc-2024-05-13-backport-57420-to-squid
Anthony D'Atri [Sun, 12 May 2024 21:36:53 +0000 (17:36 -0400)]
Merge pull request #57426 from zdover23/wip-doc-2024-05-13-backport-57420-to-squid

squid: doc/cephfs: edit fs-volumes.rst (1 of x) followup

14 months agodoc/cephfs: edit fs-volumes.rst (1 of x) followup 57426/head
Zac Dover [Sun, 12 May 2024 01:39:34 +0000 (11:39 +1000)]
doc/cephfs: edit fs-volumes.rst (1 of x) followup

Include the suggestions for improving doc/cephfs/fs-volumes.rst made by
Anthony D'Atri here
https://github.com/ceph/ceph/pull/57415#discussion_r1597362110

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit cb700d804b4390fd9f55444dcfc04dfebac3a1bf)

14 months agoMerge pull request #57417 from zdover23/wip-doc-2024-05-12-backport-57415-to-squid
Zac Dover [Sun, 12 May 2024 20:55:15 +0000 (06:55 +1000)]
Merge pull request #57417 from zdover23/wip-doc-2024-05-12-backport-57415-to-squid

squid: doc/cephfs: edit fs-volumes.rst (1 of x)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agoMerge pull request #56519 from rhcs-dashboard/wip-65133-squid
Avan [Sun, 12 May 2024 07:36:18 +0000 (13:06 +0530)]
Merge pull request #56519 from rhcs-dashboard/wip-65133-squid

squid: mgr/dashboard: add support for NFSv3 exports

Reviewed-by: afreen23 <NOT@FOUND>
14 months agoMerge pull request #57422 from zdover23/wip-doc-2024-05-12-backport-57421-to-squid
Anthony D'Atri [Sun, 12 May 2024 05:25:23 +0000 (01:25 -0400)]
Merge pull request #57422 from zdover23/wip-doc-2024-05-12-backport-57421-to-squid

squid: doc/dev: s/to asses/to assess/

14 months agodoc/dev: s/to asses/to assess/ 57422/head
Zac Dover [Sun, 12 May 2024 02:27:23 +0000 (12:27 +1000)]
doc/dev: s/to asses/to assess/

s/to asses/to assess/ in doc/deve/developer_guide/dash-devel.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 832358b072ec5cb4672de8a7a7aebf10a39fd0e5)

14 months agodoc/cephfs: edit fs-volumes.rst (1 of x) 57417/head
Zac Dover [Sat, 11 May 2024 04:57:23 +0000 (14:57 +1000)]
doc/cephfs: edit fs-volumes.rst (1 of x)

Edit doc/cephfs/fs-volumes to the section "FS Subvolumes" (but not
including the section "FS Subvolumes").

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0acbb278fc712c32334fd7340b887d2865dbfb86)

14 months agoMerge pull request #57390 from zdover23/wip-doc-2024-05-10-backport-57372-to-squid
Zac Dover [Sat, 11 May 2024 23:47:27 +0000 (09:47 +1000)]
Merge pull request #57390 from zdover23/wip-doc-2024-05-10-backport-57372-to-squid

squid: doc/cephfs: expand the role of quiesce in consistent backups

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
14 months agoMerge pull request #57364 from zdover23/wip-doc-2024-05-09-radosgw-bucketpolicy-squid
Zac Dover [Fri, 10 May 2024 11:21:17 +0000 (21:21 +1000)]
Merge pull request #57364 from zdover23/wip-doc-2024-05-09-radosgw-bucketpolicy-squid

squid: doc/radosgw: update Squid S3 action list

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agoMerge pull request #57335 from zdover23/wip-doc-2024-05-08-backport-57298-to-squid
Zac Dover [Fri, 10 May 2024 11:12:17 +0000 (21:12 +1000)]
Merge pull request #57335 from zdover23/wip-doc-2024-05-08-backport-57298-to-squid

squid: doc/mgr: edit "Overview" in dashboard.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agoMerge pull request #57295 from zdover23/wip-doc-2024-05-07-backport-57273-to-squid
Zac Dover [Fri, 10 May 2024 11:05:46 +0000 (21:05 +1000)]
Merge pull request #57295 from zdover23/wip-doc-2024-05-07-backport-57273-to-squid

squid: doc/mgr: edit "Resolve IP address to hostname before redirect"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
14 months agodoc/cephfs: expand the role of quiesce in consistent backups 57390/head
Leonid Usov [Thu, 9 May 2024 12:29:33 +0000 (15:29 +0300)]
doc/cephfs: expand the role of quiesce in consistent backups

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
Fixes: https://tracker.ceph.com/issues/65881
(cherry picked from commit 34ec2a4c6a065028d593b1e959600a3ce87541a8)

14 months agoMerge pull request #57310 from idryomov/wip-65624-squid
Ilya Dryomov [Fri, 10 May 2024 10:35:11 +0000 (12:35 +0200)]
Merge pull request #57310 from idryomov/wip-65624-squid

squid: pybind/rbd: expose CLONE_FORMAT and FLATTEN image options

Reviewed-by: Ramana Raja <rraja@redhat.com>
14 months agoMerge pull request #57307 from idryomov/wip-65487-squid
Ilya Dryomov [Fri, 10 May 2024 10:35:00 +0000 (12:35 +0200)]
Merge pull request #57307 from idryomov/wip-65487-squid

squid: rbd-mirror: clean up stale pool replayers and callouts better

Reviewed-by: Ramana Raja <rraja@redhat.com>
14 months agoMerge pull request #57031 from idryomov/wip-65481-squid
Ilya Dryomov [Fri, 10 May 2024 10:34:46 +0000 (12:34 +0200)]
Merge pull request #57031 from idryomov/wip-65481-squid

squid: qa: fix krbd_msgr_segments and krbd_rxbounce failing on 8.stream

Reviewed-by: Ramana Raja <rraja@redhat.com>
14 months agoMerge pull request #57370 from weirdwiz/wip-65879-squid
Avan [Fri, 10 May 2024 09:36:49 +0000 (15:06 +0530)]
Merge pull request #57370 from weirdwiz/wip-65879-squid

squid: exporter: handle exceptions gracefully

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
14 months agoexporter: handle exceptions gracefully 57370/head
Divyansh Kamboj [Tue, 30 Apr 2024 07:44:10 +0000 (13:14 +0530)]
exporter: handle exceptions gracefully

ceph exporter crahes, and fails to handle exceptions in
dump_asok_metrics(). add try and catch blocks to handle the exceptions gracefully.

Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
(cherry picked from commit e442ee479987f339afdc333582b6b64c66ba6846)

14 months agoMerge pull request #57303 from ronen-fr/wip-rf-reinstate-q-squid
Laura Flores [Wed, 8 May 2024 23:05:43 +0000 (18:05 -0500)]
Merge pull request #57303 from ronen-fr/wip-rf-reinstate-q-squid

squid:osd/scrub: reinstate scrub reservation queuing

14 months agodoc/radosgw: update Squid S3 action list 57364/head
Zac Dover [Wed, 8 May 2024 22:10:25 +0000 (08:10 +1000)]
doc/radosgw: update Squid S3 action list

Update the list of S3 actions supported in the Squid release of Ceph.

Note: The list of S3 actions supported in Squid is not the list of S3
actions supported in Reef or in Quincy.

Signed-off-by: Zac Dover <zac.dover@proton.me>
14 months agoMerge pull request #57092 from idryomov/wip-65573-squid
Ilya Dryomov [Wed, 8 May 2024 21:28:50 +0000 (23:28 +0200)]
Merge pull request #57092 from idryomov/wip-65573-squid

squid: librbd: make group and group snapshot IDs more random

Reviewed-by: Mykola Golub <mgolub@suse.com>
14 months agoMerge pull request #57242 from cbodley/wip-65767-squid
Casey Bodley [Wed, 8 May 2024 20:23:20 +0000 (21:23 +0100)]
Merge pull request #57242 from cbodley/wip-65767-squid

squid: rgw/multisite-notification: retry storing bucket notification attrs for ECANCELED(ConcurrentModification) errors.

Reviewed-by: Krunal Chheda <kchheda3@bloomberg.net>
14 months agoMerge pull request #57006 from rzarzynski/wip-65185-squid
Laura Flores [Wed, 8 May 2024 17:52:34 +0000 (12:52 -0500)]
Merge pull request #57006 from rzarzynski/wip-65185-squid

squid: osd/PGBackend::be_scan_list: only call stat, getattrs once per object

14 months agoMerge pull request #57300 from cbodley/wip-65821-squid
Casey Bodley [Wed, 8 May 2024 16:47:08 +0000 (17:47 +0100)]
Merge pull request #57300 from cbodley/wip-65821-squid

squid: rgw: fix CompleteMultipart error handling regression

Reviewed-by: Ali Masarwa <amasarwa@redhat.com>
14 months agoMerge pull request #57104 from soumyakoduri/wip-skoduri-squid
Casey Bodley [Wed, 8 May 2024 15:13:35 +0000 (16:13 +0100)]
Merge pull request #57104 from soumyakoduri/wip-skoduri-squid

[squid] rgw/lc: advance head if the current entry doesn't exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
14 months agoMerge pull request #57231 from galsalomon66/wip-65244-squid
Casey Bodley [Wed, 8 May 2024 15:13:23 +0000 (16:13 +0100)]
Merge pull request #57231 from galsalomon66/wip-65244-squid

squid: rgw/s3select: s3select response handler refactor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
14 months agoMerge pull request #57254 from aaSharma14/wip-65790-squid
Aashish Sharma [Wed, 8 May 2024 09:24:47 +0000 (14:54 +0530)]
Merge pull request #57254 from aaSharma14/wip-65790-squid

squid: mgr/dashboard: add prometheus federation config for multi-cluster monitoring

Reviewed-by: Nizamudeen A <nia@redhat.com>
14 months agotest/crimson/sesastore/test_block: add constructor to construct 57350/head
Xuehan Xu [Fri, 15 Mar 2024 11:48:33 +0000 (19:48 +0800)]
test/crimson/sesastore/test_block: add constructor to construct
non-fully-loaded TestBlocks

Before this commit, all TestBlocks are fully-loaded whether they do have
data or not.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 61d7f8821fa06cf48911230228687abdf747dba2)

14 months agocrimson/os/seastore: fix use after free error 57349/head
Zhang Song [Tue, 26 Mar 2024 06:15:35 +0000 (14:15 +0800)]
crimson/os/seastore: fix use after free error

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
(cherry picked from commit 1b5ca67ef8e9de48221d52b6e46f89aad1a731eb)

14 months agocrimson/os/pg_map: allow multiple shards to create new pg mappings at the same time 57348/head
Yingxin Cheng [Wed, 20 Mar 2024 07:25:33 +0000 (15:25 +0800)]
crimson/os/pg_map: allow multiple shards to create new pg mappings at the same time

Also:
* Better detections in case of inconsistent racings, such as:
  * The new mapping is creating towards different cores.
  * Mapping creation is racing with its eracing.
  * Multiple shards are erasing the same mapping at the same time.
* Add more logs to debug in case of unexpected issues.

Fixes: https://tracker.ceph.com/issues/64934
Fixes: https://tracker.ceph.com/issues/64009
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 2422817b7578da44090ea5e2f540c505bbce1412)

14 months agocrimson/osd/shard_services: fix log
Yingxin Cheng [Wed, 20 Mar 2024 07:20:57 +0000 (15:20 +0800)]
crimson/osd/shard_services: fix log

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit de2e66723586d2e9c5fd838e5199547b0d2d5f4f)

14 months agocrimson/osd/pg_map/PGShardMapping: cleanups
Yingxin Cheng [Tue, 19 Mar 2024 03:05:11 +0000 (11:05 +0800)]
crimson/osd/pg_map/PGShardMapping: cleanups

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit b35ed71e23ac5eabfed1c47361ee499bab77cd14)

14 months agocrimson/.../scrub_events: fix progress lifetime in deep_scan_object 57347/head
Samuel Just [Tue, 19 Mar 2024 23:11:48 +0000 (23:11 +0000)]
crimson/.../scrub_events: fix progress lifetime in deep_scan_object

seastar::repeat may or may not move the passed action after the first
invocation, so we can't actually rely on references to variables
captured by value being stable.  Instead, allocate with std::make_unique
and move into a finally lambda.

Fixes: https://tracker.ceph.com/issues/64996
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 3276ad571640bc91fe290832a0001bd96f1f26de)

14 months agocrimson/osd/osd.h: declare osdmap after OSDSingletonState 57346/head
Samuel Just [Mon, 18 Mar 2024 23:26:36 +0000 (23:26 +0000)]
crimson/osd/osd.h: declare osdmap after OSDSingletonState

Otherwise, destructing the osdmap cached_map_t tries to manipulate
OSDSingletonState::osdmaps after it has already been destroyed.

Fixes: https://tracker.ceph.com/issues/64935
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit b623594c44df1c7d75fa7a8d4e8b90f8829d249f)

14 months agocrimson/os/seastore: cache metadata during trimming to prevent from disk read 57345/head
myoungwon oh [Mon, 18 Mar 2024 06:48:07 +0000 (06:48 +0000)]
crimson/os/seastore: cache metadata during trimming to prevent from disk read

I encountered continous disk reads during trimming even though there are sufficient
cache available, in 4K random write test with RBM (RBD).
This is because metadata is note cached if its source is background transaction
within touch_extent(). So, seastore, including the trimming process,  needs to
constantly retrieve metadata (e.g., BACKREF_LEAF).

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 7f253ce5aa98018b5b209fd6e062078d151ad567)

14 months agocrimson/common: add coroutine integration for crimson futures 57344/head
Samuel Just [Wed, 31 Jan 2024 23:34:34 +0000 (15:34 -0800)]
crimson/common: add coroutine integration for crimson futures

Adds coroutine machinery for crimson errorated and interruptible futures.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1126ec3cf6285abbf6a875ab27daff397f09220b)

14 months agocrimson/common: expose unsafe_get0 for interruptible errorated futures
Samuel Just [Sun, 4 Feb 2024 06:47:04 +0000 (22:47 -0800)]
crimson/common: expose unsafe_get0 for interruptible errorated futures

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 11698a6da9d518329003a90777b0ef31454d2bfd)

14 months agocrimson/common: add interruptible_errorator::make_interruptible
Samuel Just [Sun, 4 Feb 2024 06:46:34 +0000 (22:46 -0800)]
crimson/common: add interruptible_errorator::make_interruptible

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 52800dcee08164f9d17d9d293e01cce173670f63)

14 months agocrimson/.../errorator: expose get_exception
Samuel Just [Thu, 8 Feb 2024 01:25:38 +0000 (17:25 -0800)]
crimson/.../errorator: expose get_exception

Useful for intentionally ignoring an exceptional future.  Otherwise,
seastar warns.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 0afd9fa4326d4e7fe5966347a37623379072fd98)

14 months agocrimson/common/interruptible_future: add interruptor::future convenience declaration
Samuel Just [Wed, 31 Jan 2024 23:33:33 +0000 (15:33 -0800)]
crimson/common/interruptible_future: add interruptor::future convenience declaration

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1f152bf9861dd04ace70a62ba8a82c33136b5fb0)

14 months agocrimson/common: expose set_coroutine for crimson future specializations/wrappers
Samuel Just [Wed, 31 Jan 2024 23:32:30 +0000 (15:32 -0800)]
crimson/common: expose set_coroutine for crimson future specializations/wrappers

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 53c4acde1d6b00ed703a71c3137bee92df4be46a)

14 months agotest/crimson/test_errorator: move tests into TEST_F declarations
Samuel Just [Thu, 16 Mar 2023 20:22:22 +0000 (20:22 +0000)]
test/crimson/test_errorator: move tests into TEST_F declarations

Slightly easier to read this way.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 3f377f013c690d9c17685967f1ebc4dc2adbf838)

14 months agocrimson/.../ops_executor: no need to warn in normal pg op execution path
Samuel Just [Tue, 6 Feb 2024 04:45:04 +0000 (20:45 -0800)]
crimson/.../ops_executor: no need to warn in normal pg op execution path

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit e72f1779f4505616253cba7c41c10aa2b772adaa)

14 months agocrimson/.../errorator: simplify maybe_handle_error_t::handle()
Samuel Just [Wed, 7 Feb 2024 01:19:43 +0000 (17:19 -0800)]
crimson/.../errorator: simplify maybe_handle_error_t::handle()

Using FuturatorT::invoke covers both of the last two cases and also
correctly deals with handlers that return interrupible futures.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 979c1916507092591469f8ea3b0a0440f2b88066)

14 months agodoc/mgr: edit "Overview" in dashboard.rst 57335/head
Zac Dover [Mon, 6 May 2024 15:16:51 +0000 (01:16 +1000)]
doc/mgr: edit "Overview" in dashboard.rst

Edit and update the "Overview" section in doc/mgr/dashboard.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 6cf06c60f7c3f87707c7dc47cde2346cec7cf00f)

14 months agocrimson/os/seastore/omap_manager: expand omap tree nodes 57328/head
Xuehan Xu [Thu, 29 Feb 2024 13:20:56 +0000 (21:20 +0800)]
crimson/os/seastore/omap_manager: expand omap tree nodes

Teuthology tests can issue large omap kvs

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 94803e907abd52f6fdc272098fcfcbc6c44d7a39)

14 months agocrimson/os/seastore: omap_get_values with range [start, "infinite") 57327/head
Xuehan Xu [Sat, 2 Mar 2024 08:44:22 +0000 (16:44 +0800)]
crimson/os/seastore: omap_get_values with range [start, "infinite")
should return all results from "start"

Currently, this range retrieval has a default max result size, which is
wrong

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit e277fbf63cff08538d1e2d3dc99c0c7d69ac5e13)

14 months agocrimson/os/seastore: adjust SeaStore::_omap_set_kvs() params 57326/head
Xuehan Xu [Fri, 1 Mar 2024 02:32:32 +0000 (10:32 +0800)]
crimson/os/seastore: adjust SeaStore::_omap_set_kvs() params

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 9852f4d97e7f4ff6a83b90bf7fa5a15b59e4342a)

14 months agocrimson/os/seastore/async_cleaner: add comments to clarify what kind of 57325/head
Xuehan Xu [Sat, 2 Mar 2024 09:36:33 +0000 (17:36 +0800)]
crimson/os/seastore/async_cleaner: add comments to clarify what kind of
extents would be considered dead by the cleaner

This could prevent future developers from rendering the cleaner abnormal
unexpectedly

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 61c7d6b5483851e90e60d3a82b61740e7df559e6)

14 months agocrimson/osd/pg: filter out snapmapper objects when doing pgls/pgnls 57324/head
Xuehan Xu [Wed, 21 Feb 2024 06:37:13 +0000 (14:37 +0800)]
crimson/osd/pg: filter out snapmapper objects when doing pgls/pgnls

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 8f7921e8fb8a2035fa3f6c2cf6227b8176d6a654)

14 months agocrimson/osd: put snapmapper's key-value pairs into dedicated objs
Xuehan Xu [Fri, 30 Jun 2023 04:33:35 +0000 (12:33 +0800)]
crimson/osd: put snapmapper's key-value pairs into dedicated objs

Otherwise, PG::read_log_and_missing() will meet those key-values and
won't know what to do with them. This is modeling what the classic
osd is doing

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit e56e90857d6849c6045cc3edef896baffe70b6af)

14 months agocrimson/os: add FuturizedStore::Shard::exists() interface
Xuehan Xu [Tue, 16 Jan 2024 08:58:17 +0000 (16:58 +0800)]
crimson/os: add FuturizedStore::Shard::exists() interface

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 6f0803a86af78965eb954d98e6095d96f0c32f1f)

14 months agocrimson/os/alienstore: use homebrew counting_semaphore 57323/head
Xuehan Xu [Tue, 27 Feb 2024 09:06:07 +0000 (17:06 +0800)]
crimson/os/alienstore: use homebrew counting_semaphore

std::counting_semaphore has the bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104928,
go back to our homebrew version

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit c884db34e049716577f7868b76685df182f82678)

14 months agocrimson/os/seastore/collection_manager: allow empty-delta-buffer 57322/head
Xuehan Xu [Wed, 6 Mar 2024 05:49:43 +0000 (13:49 +0800)]
crimson/os/seastore/collection_manager: allow empty-delta-buffer
collection nodes

Fixes: https://tracker.ceph.com/issues/64728
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit e8b9ab73f409d5d9a51100f5dd9922dfcab5dc0e)

14 months agoCMakeLists: disable Seastar_IO_URING to fix seastar unittest timeout 57321/head
luo rixin [Tue, 27 Feb 2024 07:07:17 +0000 (15:07 +0800)]
CMakeLists: disable Seastar_IO_URING to fix seastar unittest timeout

As pr https://github.com/ceph/ceph/pull/55787 bump liburing from 0.7 to 2.5.
with liburing-dev (2.1) installed on ubuntu jammy, Seastar_IO_URING will be set ON,
seastar will be builded with liburing-dev. Ceph bluestore use build_uring in file Builduring.cmake
to build liburing version 2.5 and set URING_INCLUDE_DIR to
/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/liburing/src/include/liburing/,
seastar use URING_INCLUDE_DIR(version is 2.5) to build, but seastar link liburing.so to system
liburing-dev package(version 2.1). The liburing head file seastar building and liburing binary
seastar linked is mismatched.
I have downgraded the liburing version in file 'cmake/modules/Builduring.cmake' to liburing-2.1,
the seastar unittests work fine, no timeout.

Fixes: https://tracker.ceph.com/issues/64789
Signed-off-by: luo rixin <luorixin@huawei.com>
(cherry picked from commit ebedfa7ef0858b52eb7cd652f67bbd8eef6e9674)

14 months agocrimson/os/seastore/transaction_manager: minor refinement to remap_pin() asserts 57320/head
Yingxin Cheng [Fri, 8 Mar 2024 07:48:06 +0000 (15:48 +0800)]
crimson/os/seastore/transaction_manager: minor refinement to remap_pin() asserts

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 2d2df50e24d5a4dc3806b725e8b43cc0fcd1e3c4)

14 months agotest/crimson: drop cout from seastore tests
Yingxin Cheng [Fri, 8 Mar 2024 07:39:40 +0000 (15:39 +0800)]
test/crimson: drop cout from seastore tests

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit bc97b82dcb1b00023a5dca149209f6871fb34a0e)

14 months agocrimson/os/seastore/../fixed_kv_node: add comment to copy-from-source
Yingxin Cheng [Thu, 7 Mar 2024 08:41:25 +0000 (16:41 +0800)]
crimson/os/seastore/../fixed_kv_node: add comment to copy-from-source

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 3df6477809f8aabacbf9850a09527c2887263949)

14 months agocrimson/os/seastore: reuse link_phy_tree_root_node()
Yingxin Cheng [Thu, 7 Mar 2024 08:20:07 +0000 (16:20 +0800)]
crimson/os/seastore: reuse link_phy_tree_root_node()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 8bca062d04e95565c13fdb365f344dceae3a4c1f)

14 months agocrimson/os/cache: get_extent_viewable_by_trans, the extent should be valid
Yingxin Cheng [Thu, 7 Mar 2024 07:46:34 +0000 (15:46 +0800)]
crimson/os/cache: get_extent_viewable_by_trans, the extent should be valid

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 06cf024341284cad62100920958ade131a970902)

14 months agocrimson/os/cache: cleanup get_extent(_by_type)
Yingxin Cheng [Thu, 7 Mar 2024 07:34:06 +0000 (15:34 +0800)]
crimson/os/cache: cleanup get_extent(_by_type)

Distinguish caching vs absent get_extent interfaces, and misc related
cleanups.

After the lba parent-child pointer optimization, the absent path should
be used.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 05c916ba46b9edeb0551e1629373c64c91bcdd5f)

14 months agocrimson/os/seastore: remove unecessary public interfaces
Yingxin Cheng [Thu, 7 Mar 2024 07:30:39 +0000 (15:30 +0800)]
crimson/os/seastore: remove unecessary public interfaces

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 785634262b6c39df44f3e88ab326421f75b06c98)

14 months agocrimson/os/seastore/cache: drop the unnecessary friend fixed_kv_btree
Yingxin Cheng [Thu, 7 Mar 2024 05:42:51 +0000 (13:42 +0800)]
crimson/os/seastore/cache: drop the unnecessary friend fixed_kv_btree

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 0558ef8b1a16060c0cd424635315db25f5c68351)

14 months agotest/crimson/seastore: fixes UT to use LBAMapping::is_stable() 57317/head
myoungwon oh [Sat, 2 Mar 2024 08:58:19 +0000 (08:58 +0000)]
test/crimson/seastore: fixes UT to use LBAMapping::is_stable()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 3c7ef1bbbef45b0901e000ae8b793f5a2279675d)

14 months agocrimson/os/seastore: use LBAMapping::is_stable() wherever appropriate
Yingxin Cheng [Fri, 3 Nov 2023 09:10:55 +0000 (17:10 +0800)]
crimson/os/seastore: use LBAMapping::is_stable() wherever appropriate

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit da0d4048b8249b3038596818545eaf16ca3ee082)