]> git.apps.os.sepia.ceph.com Git - ceph-ci.git/log
ceph-ci.git
16 months agoMerge pull request #57289 from Matan-B/wip-55772-squid
Matan Breizman [Wed, 15 May 2024 10:18:00 +0000 (13:18 +0300)]
Merge pull request #57289 from Matan-B/wip-55772-squid

squid: crimson/os/seastore: two attrs related bug fixes

Reviewed-by: Samuel Just <sjust@redhat.com>
16 months agoMerge pull request #57288 from Matan-B/wip-55705-squid
Matan Breizman [Wed, 15 May 2024 10:17:31 +0000 (13:17 +0300)]
Merge pull request #57288 from Matan-B/wip-55705-squid

squid: crimson/os/seastore: fix wrong-positioned asserts

Reviewed-by: Samuel Just <sjust@redhat.com>
16 months agoMerge pull request #57286 from Matan-B/wip-55407-squid
Matan Breizman [Wed, 15 May 2024 10:15:37 +0000 (13:15 +0300)]
Merge pull request #57286 from Matan-B/wip-55407-squid

squid: crimson/osd/shard_services: make sure that only up/acting members can create pgs

Reviewed-by: Samuel Just <sjust@redhat.com>
16 months agoMerge pull request #57283 from Matan-B/wip-55641-squid
Matan Breizman [Wed, 15 May 2024 10:15:11 +0000 (13:15 +0300)]
Merge pull request #57283 from Matan-B/wip-55641-squid

squid: crimson/osd/main: Enable multicore client messenger

Reviewed-by: Samuel Just <sjust@redhat.com>
16 months agoMerge pull request #56508 from rosinL/wip-65164-squid
Matan Breizman [Wed, 15 May 2024 10:11:29 +0000 (13:11 +0300)]
Merge pull request #56508 from rosinL/wip-65164-squid

squid: cmake/AddCephTest: bind crimson unittest to different cores

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
16 months agoMerge pull request #57465 from idryomov/wip-51845-squid
Ilya Dryomov [Wed, 15 May 2024 07:32:04 +0000 (09:32 +0200)]
Merge pull request #57465 from idryomov/wip-51845-squid

squid: qa/suites/krbd: drop pre-single-major and move "layering only" coverage

Reviewed-by: Ramana Raja <rraja@redhat.com>
16 months agoMerge pull request #57461 from frittentheke/wip-65838-squid
Aashish Sharma [Wed, 15 May 2024 07:16:03 +0000 (12:46 +0530)]
Merge pull request #57461 from frittentheke/wip-65838-squid

squid: monitoring/ceph-mixin: Cleanup of variables, queries and tests (to fix showMultiCluster=True)

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
16 months agoqa/suites/krbd: rename no-object-map to no-exclusive-lock
Ilya Dryomov [Mon, 13 May 2024 11:45:26 +0000 (13:45 +0200)]
qa/suites/krbd: rename no-object-map to no-exclusive-lock

Exclusive lock has always been disabled by this facet, so it might as
well be reflected in its name.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ad6a95d8af42b1c3ed9d244c6161e5269356c7c9)

16 months agoqa/suites/krbd: move "layering only" coverage to fsx
Ilya Dryomov [Mon, 13 May 2024 11:31:17 +0000 (13:31 +0200)]
qa/suites/krbd: move "layering only" coverage to fsx

It makes much more sense there since it's where we actually create
clones and flatten them a lot.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7b9f28e74325aad17c3774d594c3f0e1c4e25dc2)

16 months agoqa/suites/krbd: drop pre-single-major test
Ilya Dryomov [Sun, 12 May 2024 09:15:36 +0000 (11:15 +0200)]
qa/suites/krbd: drop pre-single-major test

Single-major mapping scheme was introduced in 2014 and became the
default in 2017.  It's getting increasingly difficult to build and,
more importantly, to boot a 10 year old kernel with recent userspace
(systemd, etc).  If someone is still running such a kernel, it's
really unlikely that they would have the most recent rbd CLI tool
installed.

Fixes: https://tracker.ceph.com/issues/51845
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 39a579144cd8b97dd338bfcf36c06ad4bcaed526)

16 months agoCleanup of variables, queries and tests to enable showMultiCluster=True
Christian Rohmann [Fri, 26 Jan 2024 14:05:52 +0000 (15:05 +0100)]
Cleanup of variables, queries and tests to enable showMultiCluster=True

Rendering the dashboards with showMultiCluster=True allows for
them to work with multiple clusters storing their metrics in a single
Prometheus instance. This works via the cluster label and that functionality
already existed. This just fixes some inconsistencies in applying the label
filters.

Additionally this contains updates to the tests to have them succeed with
with both configurations and avoid the introduction of regressions in
regards to multiCluster in the future.

There also are some consistency cleanups here and there:
 * `datasource` was not used consistently
 * `cluster` label_values are determined from `ceph_health_status`
 * `job` template and filters on this label were removed to align multi cluster
    support solely via the `cluster` label
 * `ceph_hosts` filter now uses label_values from any ceph_metadata metrici
    to now show all instance values, but those of hosts with some Ceph
    component / daemon.
 *  Enable showMultiCluster=True since `cluster` label is now always present,
    via https://github.com/ceph/ceph/pull/54964

Improves: https://tracker.ceph.com/issues/64321
Signed-off-by: Christian Rohmann <christian.rohmann@inovex.de>
(cherry picked from commit 090b8e17f1e84d8b20e05143d5dd7ff107031176)

16 months agoMerge pull request #57426 from zdover23/wip-doc-2024-05-13-backport-57420-to-squid
Anthony D'Atri [Sun, 12 May 2024 21:36:53 +0000 (17:36 -0400)]
Merge pull request #57426 from zdover23/wip-doc-2024-05-13-backport-57420-to-squid

squid: doc/cephfs: edit fs-volumes.rst (1 of x) followup

16 months agodoc/cephfs: edit fs-volumes.rst (1 of x) followup
Zac Dover [Sun, 12 May 2024 01:39:34 +0000 (11:39 +1000)]
doc/cephfs: edit fs-volumes.rst (1 of x) followup

Include the suggestions for improving doc/cephfs/fs-volumes.rst made by
Anthony D'Atri here
https://github.com/ceph/ceph/pull/57415#discussion_r1597362110

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit cb700d804b4390fd9f55444dcfc04dfebac3a1bf)

16 months agoMerge pull request #57417 from zdover23/wip-doc-2024-05-12-backport-57415-to-squid
Zac Dover [Sun, 12 May 2024 20:55:15 +0000 (06:55 +1000)]
Merge pull request #57417 from zdover23/wip-doc-2024-05-12-backport-57415-to-squid

squid: doc/cephfs: edit fs-volumes.rst (1 of x)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
16 months agoMerge pull request #56519 from rhcs-dashboard/wip-65133-squid
Avan [Sun, 12 May 2024 07:36:18 +0000 (13:06 +0530)]
Merge pull request #56519 from rhcs-dashboard/wip-65133-squid

squid: mgr/dashboard: add support for NFSv3 exports

Reviewed-by: afreen23 <NOT@FOUND>
16 months agoMerge pull request #57422 from zdover23/wip-doc-2024-05-12-backport-57421-to-squid
Anthony D'Atri [Sun, 12 May 2024 05:25:23 +0000 (01:25 -0400)]
Merge pull request #57422 from zdover23/wip-doc-2024-05-12-backport-57421-to-squid

squid: doc/dev: s/to asses/to assess/

16 months agodoc/dev: s/to asses/to assess/
Zac Dover [Sun, 12 May 2024 02:27:23 +0000 (12:27 +1000)]
doc/dev: s/to asses/to assess/

s/to asses/to assess/ in doc/deve/developer_guide/dash-devel.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 832358b072ec5cb4672de8a7a7aebf10a39fd0e5)

16 months agodoc/cephfs: edit fs-volumes.rst (1 of x)
Zac Dover [Sat, 11 May 2024 04:57:23 +0000 (14:57 +1000)]
doc/cephfs: edit fs-volumes.rst (1 of x)

Edit doc/cephfs/fs-volumes to the section "FS Subvolumes" (but not
including the section "FS Subvolumes").

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0acbb278fc712c32334fd7340b887d2865dbfb86)

16 months agoMerge pull request #57390 from zdover23/wip-doc-2024-05-10-backport-57372-to-squid
Zac Dover [Sat, 11 May 2024 23:47:27 +0000 (09:47 +1000)]
Merge pull request #57390 from zdover23/wip-doc-2024-05-10-backport-57372-to-squid

squid: doc/cephfs: expand the role of quiesce in consistent backups

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
16 months agoMerge pull request #57364 from zdover23/wip-doc-2024-05-09-radosgw-bucketpolicy-squid
Zac Dover [Fri, 10 May 2024 11:21:17 +0000 (21:21 +1000)]
Merge pull request #57364 from zdover23/wip-doc-2024-05-09-radosgw-bucketpolicy-squid

squid: doc/radosgw: update Squid S3 action list

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
16 months agoMerge pull request #57335 from zdover23/wip-doc-2024-05-08-backport-57298-to-squid
Zac Dover [Fri, 10 May 2024 11:12:17 +0000 (21:12 +1000)]
Merge pull request #57335 from zdover23/wip-doc-2024-05-08-backport-57298-to-squid

squid: doc/mgr: edit "Overview" in dashboard.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
16 months agoMerge pull request #57295 from zdover23/wip-doc-2024-05-07-backport-57273-to-squid
Zac Dover [Fri, 10 May 2024 11:05:46 +0000 (21:05 +1000)]
Merge pull request #57295 from zdover23/wip-doc-2024-05-07-backport-57273-to-squid

squid: doc/mgr: edit "Resolve IP address to hostname before redirect"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
16 months agodoc/cephfs: expand the role of quiesce in consistent backups
Leonid Usov [Thu, 9 May 2024 12:29:33 +0000 (15:29 +0300)]
doc/cephfs: expand the role of quiesce in consistent backups

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
Fixes: https://tracker.ceph.com/issues/65881
(cherry picked from commit 34ec2a4c6a065028d593b1e959600a3ce87541a8)

16 months agoMerge pull request #57310 from idryomov/wip-65624-squid
Ilya Dryomov [Fri, 10 May 2024 10:35:11 +0000 (12:35 +0200)]
Merge pull request #57310 from idryomov/wip-65624-squid

squid: pybind/rbd: expose CLONE_FORMAT and FLATTEN image options

Reviewed-by: Ramana Raja <rraja@redhat.com>
16 months agoMerge pull request #57307 from idryomov/wip-65487-squid
Ilya Dryomov [Fri, 10 May 2024 10:35:00 +0000 (12:35 +0200)]
Merge pull request #57307 from idryomov/wip-65487-squid

squid: rbd-mirror: clean up stale pool replayers and callouts better

Reviewed-by: Ramana Raja <rraja@redhat.com>
16 months agoMerge pull request #57031 from idryomov/wip-65481-squid
Ilya Dryomov [Fri, 10 May 2024 10:34:46 +0000 (12:34 +0200)]
Merge pull request #57031 from idryomov/wip-65481-squid

squid: qa: fix krbd_msgr_segments and krbd_rxbounce failing on 8.stream

Reviewed-by: Ramana Raja <rraja@redhat.com>
16 months agoMerge pull request #57370 from weirdwiz/wip-65879-squid
Avan [Fri, 10 May 2024 09:36:49 +0000 (15:06 +0530)]
Merge pull request #57370 from weirdwiz/wip-65879-squid

squid: exporter: handle exceptions gracefully

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
16 months agoexporter: handle exceptions gracefully
Divyansh Kamboj [Tue, 30 Apr 2024 07:44:10 +0000 (13:14 +0530)]
exporter: handle exceptions gracefully

ceph exporter crahes, and fails to handle exceptions in
dump_asok_metrics(). add try and catch blocks to handle the exceptions gracefully.

Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
(cherry picked from commit e442ee479987f339afdc333582b6b64c66ba6846)

17 months agoMerge pull request #57303 from ronen-fr/wip-rf-reinstate-q-squid
Laura Flores [Wed, 8 May 2024 23:05:43 +0000 (18:05 -0500)]
Merge pull request #57303 from ronen-fr/wip-rf-reinstate-q-squid

squid:osd/scrub: reinstate scrub reservation queuing

17 months agodoc/radosgw: update Squid S3 action list
Zac Dover [Wed, 8 May 2024 22:10:25 +0000 (08:10 +1000)]
doc/radosgw: update Squid S3 action list

Update the list of S3 actions supported in the Squid release of Ceph.

Note: The list of S3 actions supported in Squid is not the list of S3
actions supported in Reef or in Quincy.

Signed-off-by: Zac Dover <zac.dover@proton.me>
17 months agoMerge pull request #57092 from idryomov/wip-65573-squid
Ilya Dryomov [Wed, 8 May 2024 21:28:50 +0000 (23:28 +0200)]
Merge pull request #57092 from idryomov/wip-65573-squid

squid: librbd: make group and group snapshot IDs more random

Reviewed-by: Mykola Golub <mgolub@suse.com>
17 months agoMerge pull request #57242 from cbodley/wip-65767-squid
Casey Bodley [Wed, 8 May 2024 20:23:20 +0000 (21:23 +0100)]
Merge pull request #57242 from cbodley/wip-65767-squid

squid: rgw/multisite-notification: retry storing bucket notification attrs for ECANCELED(ConcurrentModification) errors.

Reviewed-by: Krunal Chheda <kchheda3@bloomberg.net>
17 months agoMerge pull request #57006 from rzarzynski/wip-65185-squid
Laura Flores [Wed, 8 May 2024 17:52:34 +0000 (12:52 -0500)]
Merge pull request #57006 from rzarzynski/wip-65185-squid

squid: osd/PGBackend::be_scan_list: only call stat, getattrs once per object

17 months agoMerge pull request #57300 from cbodley/wip-65821-squid
Casey Bodley [Wed, 8 May 2024 16:47:08 +0000 (17:47 +0100)]
Merge pull request #57300 from cbodley/wip-65821-squid

squid: rgw: fix CompleteMultipart error handling regression

Reviewed-by: Ali Masarwa <amasarwa@redhat.com>
17 months agoMerge pull request #57104 from soumyakoduri/wip-skoduri-squid
Casey Bodley [Wed, 8 May 2024 15:13:35 +0000 (16:13 +0100)]
Merge pull request #57104 from soumyakoduri/wip-skoduri-squid

[squid] rgw/lc: advance head if the current entry doesn't exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
17 months agoMerge pull request #57231 from galsalomon66/wip-65244-squid
Casey Bodley [Wed, 8 May 2024 15:13:23 +0000 (16:13 +0100)]
Merge pull request #57231 from galsalomon66/wip-65244-squid

squid: rgw/s3select: s3select response handler refactor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
17 months agoMerge pull request #57254 from aaSharma14/wip-65790-squid
Aashish Sharma [Wed, 8 May 2024 09:24:47 +0000 (14:54 +0530)]
Merge pull request #57254 from aaSharma14/wip-65790-squid

squid: mgr/dashboard: add prometheus federation config for multi-cluster monitoring

Reviewed-by: Nizamudeen A <nia@redhat.com>
17 months agodoc/mgr: edit "Overview" in dashboard.rst
Zac Dover [Mon, 6 May 2024 15:16:51 +0000 (01:16 +1000)]
doc/mgr: edit "Overview" in dashboard.rst

Edit and update the "Overview" section in doc/mgr/dashboard.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 6cf06c60f7c3f87707c7dc47cde2346cec7cf00f)

17 months agotest/pybind: drop unnecessary data_pool=None arguments
Ilya Dryomov [Wed, 1 May 2024 13:49:54 +0000 (15:49 +0200)]
test/pybind: drop unnecessary data_pool=None arguments

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 555601796000d83a37324c9f9b065d9c3aa48b8e)

17 months agopybind/rbd: adjust error message in deep_copy()
Ilya Dryomov [Wed, 1 May 2024 13:49:51 +0000 (15:49 +0200)]
pybind/rbd: adjust error message in deep_copy()

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5986217c79f2fe01e21f6910affa896bd2516f02)

17 months agopybind/rbd: expose RBD_IMAGE_OPTION_FLATTEN option
Ilya Dryomov [Wed, 1 May 2024 13:49:47 +0000 (15:49 +0200)]
pybind/rbd: expose RBD_IMAGE_OPTION_FLATTEN option

It takes effect with deep_copy() and migration_prepare().

Fixes: https://tracker.ceph.com/issues/65624
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 39f207ff65a4eb10f67a2d4a8a4af1bad0d2e412)

17 months agopybind/rbd: expose RBD_IMAGE_OPTION_CLONE_FORMAT option
Ilya Dryomov [Sun, 28 Apr 2024 17:19:22 +0000 (19:19 +0200)]
pybind/rbd: expose RBD_IMAGE_OPTION_CLONE_FORMAT option

It takes effect with clone(), deep_copy() and migration_prepare().

Fixes: https://tracker.ceph.com/issues/65624
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 39fafe74625423e5d23a5f1bbd74e58ef9055573)

17 months agoqa/workunits/rbd: wait for replaying status in bootstrap tests
Ilya Dryomov [Mon, 6 May 2024 06:16:01 +0000 (08:16 +0200)]
qa/workunits/rbd: wait for replaying status in bootstrap tests

wait_for_replay_complete() doesn't wait for image status to get
updated.  This didn't matter previously because these tests are run on
two different pools and nothing else was following.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d1d848276f0a33b20482321088c8c7470a84d0b3)

17 months agorbd-mirror: remove callout when destroying pool replayer
Ilya Dryomov [Wed, 24 Apr 2024 10:45:27 +0000 (12:45 +0200)]
rbd-mirror: remove callout when destroying pool replayer

If a pool replayer is removed in an error state (e.g. after failing to
connect to the remote cluster), its callout should be removed as well.
Otherwise, the error would persist causing "daemon health: ERROR"
status to be reported even after a new pool replayer is created and
started successfully.

Fixes: https://tracker.ceph.com/issues/65487
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit b7e79642d53c1802ae1202c3f013677fcfd26b1b)

17 months agorbd-mirror: shut down and remove pool replayer if peer changes
Ilya Dryomov [Wed, 24 Apr 2024 12:42:38 +0000 (14:42 +0200)]
rbd-mirror: shut down and remove pool replayer if peer changes

The code in Mirror::update_pool_replayers() responsible for shutting
down and removing stale pool replayers kicks in only in case the peer
is removed, but not if the peer changes.  However, the code responsible
for (re)starting pool replayers in the same method _does_ create and
start a new pool replayer in that case.  As a result, we can end up
with nearly identical pool replayers running at the same time, hogging
OS resources and confusing instance_id tracking logic and mirror status
reporting at the very least.

The root cause is that PeerSpec is matched normally (i.e. based on all
fields) when it comes to m_pool_replayers, and based only on UUID when
it comes to pool_peers.  This was missed in commit 5463e1a1e1b7
("rbd-mirror: extract optional peer mon_host/key values from MON").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit def93cb0ad93e3e27d7c4534f68c95ef922ab750)

17 months agoMerge pull request #57285 from Matan-B/wip-55699-squid
Rongqi Sun [Tue, 7 May 2024 01:01:53 +0000 (18:01 -0700)]
Merge pull request #57285 from Matan-B/wip-55699-squid

squid: doc/crimson: cleanup duplicate seastore description

17 months agosquid:osd/scrub: reinstate scrub reservation queuing
Ronen Friedman [Mon, 6 May 2024 19:43:41 +0000 (22:43 +0300)]
squid:osd/scrub: reinstate scrub reservation queuing

Re-enabling the Reserver-based scrub queuing (undoing
https://github.com/ceph/ceph/pull/56750), as all known issues
related to the reservation queuing have been fixed and back-ported.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
17 months agorgw: move publish_complete() back to RGWCompleteMultipart::execute()
Casey Bodley [Fri, 3 May 2024 19:43:39 +0000 (15:43 -0400)]
rgw: move publish_complete() back to RGWCompleteMultipart::execute()

move publish_complete() and meta_obj->delete_object() back to execute()
so they only run on success. this allows several member variables to
move back to execute()'s stack as well

Fixes: https://tracker.ceph.com/issues/65746
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ebb37c7bb3aee4663220054c6516164bf046fa8c)

17 months agorgw: CompleteMultipart uses s->object for Notification
Casey Bodley [Fri, 3 May 2024 19:29:00 +0000 (15:29 -0400)]
rgw: CompleteMultipart uses s->object for Notification

get_notification() should be associated with the target object
s->object. the meta_obj has the wrong object name, so required passing
s->object->get_name() as an extra argument

importantly, Notification no longer depends on the lifetime of meta_obj
to avoid a dangling pointer, while the lifetime of s->object is guaranteed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 91dc7f3be945dccd8f59e070e9bc43a2a5df12db)

17 months agorgw: CompleteMultipart uses s->object instead of target_obj
Casey Bodley [Fri, 3 May 2024 19:17:48 +0000 (15:17 -0400)]
rgw: CompleteMultipart uses s->object instead of target_obj

most requests operate directly on s->object. there's no reason to
allocate a separate target_obj for the same purpose

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d09b8ab2e077ceb6a0c6dfb99ce1b45d63a28be4)

17 months agodoc/mgr: edit "Resolve IP address to hostname before redirect"
Zac Dover [Sun, 5 May 2024 19:12:55 +0000 (05:12 +1000)]
doc/mgr: edit "Resolve IP address to hostname before redirect"

Edit the section "Resolve IP address to hostname before redirect" in
doc/mgr/dashboard.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e0838daa9bc1d549a8da2b21608153c77b15455a)

17 months agoMerge pull request #56947 from adk3798/wip-65539-squid
Aashish Sharma [Mon, 6 May 2024 13:45:50 +0000 (19:15 +0530)]
Merge pull request #56947 from adk3798/wip-65539-squid

squid: ceph-mixin: Update mixin to include alerts for the nvmeof gateway(s)

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
17 months agocrimson/os/seastore: also remove xattrs when removing onodes
Xuehan Xu [Tue, 27 Feb 2024 08:27:58 +0000 (16:27 +0800)]
crimson/os/seastore: also remove xattrs when removing onodes

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 8cc25049a897ebafccd2b9bded6d09300575ef2c)

17 months agocrimson/os/seastore: get_attrs should return all attrs
Xuehan Xu [Tue, 27 Feb 2024 08:18:57 +0000 (16:18 +0800)]
crimson/os/seastore: get_attrs should return all attrs

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit abdae14dd017dc2fce6e6c2ed014d7c024864dfc)

17 months agocrimson/os/seastore: attr related debug outputs
Xuehan Xu [Tue, 27 Feb 2024 08:06:02 +0000 (16:06 +0800)]
crimson/os/seastore: attr related debug outputs

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit c841876cb1ec8cf576083cf2e6c076924e9daf4d)

17 months agocrimson/os/seastore: fix wrong-positioned asserts
Xuehan Xu [Thu, 22 Feb 2024 09:03:43 +0000 (17:03 +0800)]
crimson/os/seastore: fix wrong-positioned asserts

Fixes: https://tracker.ceph.com/issues/64535
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit b469cbb507675108cecfcfd9da3bd5144deabcf9)

17 months agocrimson/osd/shard_services: make sure that only up/acting members can
Xuehan Xu [Thu, 1 Feb 2024 05:18:38 +0000 (13:18 +0800)]
crimson/osd/shard_services: make sure that only up/acting members can
create pgs

Fixes: https://tracker.ceph.com/issues/64282
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit ddac5d1cb85f182e5984929b667f4dc56ea97984)

17 months agocrimson/osd: correct variable naming
Xuehan Xu [Thu, 1 Feb 2024 04:55:44 +0000 (12:55 +0800)]
crimson/osd: correct variable naming

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 7000cbf82f40e0997c815a35a3a53f32e1afcef1)

17 months agodoc/crimson: cleanup duplicate seastore description
Rongqi Sun [Thu, 22 Feb 2024 02:41:43 +0000 (10:41 +0800)]
doc/crimson: cleanup duplicate seastore description

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
(cherry picked from commit 8d766c57173cf6bdea0d8fcfa8fa3c1b61fd10ef)

17 months agocommon/buffer_seastar: move to crimson/crimson/buffer_seastar
Matan Breizman [Mon, 19 Feb 2024 16:11:53 +0000 (16:11 +0000)]
common/buffer_seastar: move to crimson/crimson/buffer_seastar

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit c8d8f739f8c3d0b260547e9e788ae2e191b3f446)

17 months agocommon/buffer_seastar: fix alien threads memory
Matan Breizman [Mon, 19 Feb 2024 12:24:52 +0000 (12:24 +0000)]
common/buffer_seastar: fix alien threads memory

The underlying raw_seastar_foreign_ptr::ptr is allocated from seastar.
This ptr is wrapped with seastar::foreign_ptr:
```
/// \c foreign_ptr<> wraps smart pointers -- \ref seastar::shared_ptr<>,
/// or similar, and remembers on what core this happened.
/// When the \c foreign_ptr<> object is destroyed, it sends a message to
/// the original core so that the wrapped object can be safely destroyed.
```

The issue is that once the pointer is de-allocated from an alien thread
it is unable to send a message to the original core.
Fix this issue by making use of seastar::alien integration with non-seastar applications.
In case ~raw_seastar_foreign_ptr() will be called from an alien thread, we will submit *and wait*
for the memory to be released from the origin core.

Fixes: https://tracker.ceph.com/issues/64086
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit c714c3c4f92994413e34a5900492c8987c85669e)

17 months agocrimson/osd/main: enable multicore client msgr
Matan Breizman [Mon, 19 Feb 2024 12:22:16 +0000 (12:22 +0000)]
crimson/osd/main: enable multicore client msgr

Taken from: f78e99c05943ea336a3fe2cc1568fea81e6a9855

Co-authored-by: Yingxin Cheng <yingxin.cheng@intel.com>
Co-authored-by: Chunmei Liu <chunmei.liu@intel.com>
Co-authored-by: Xinyu Huang <xinyu.huang@intel.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 68400ff5450db1fd2c582ef016c9859829dbdf89)

17 months agoMerge pull request #57220 from rhcs-dashboard/wip-65758-squid
Pedro Gonzalez Gomez [Mon, 6 May 2024 09:24:58 +0000 (11:24 +0200)]
Merge pull request #57220 from rhcs-dashboard/wip-65758-squid

squid: mgr/dashboard: exclude cloned-deleted RBD snaps

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
17 months agoMerge pull request #57268 from zdover23/wip-doc-2024-05-05-backport-55380-to-squid-2
Anthony D'Atri [Sun, 5 May 2024 11:55:21 +0000 (07:55 -0400)]
Merge pull request #57268 from zdover23/wip-doc-2024-05-05-backport-55380-to-squid-2

doc/foundation: Updating members for Jan 2024

17 months agodoc/foundation: Updating members for Jan 2024
Mike Perez [Tue, 30 Jan 2024 20:10:22 +0000 (12:10 -0800)]
doc/foundation: Updating members for Jan 2024

Signed-off-by: Mike Perez <thingee@gmail.com>
(cherry picked from commit f979ead661d716a4a1b7bd04a30e8af3691542a0)

17 months agoMerge pull request #57264 from zdover23/wip-doc-2024-05-04-backport-57252-to-squid
Zac Dover [Sat, 4 May 2024 13:02:58 +0000 (23:02 +1000)]
Merge pull request #57264 from zdover23/wip-doc-2024-05-04-backport-57252-to-squid

squid: doc/rados: PR#57022 unfinished business

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
17 months agodoc/rados: PR#57022 unfinished business
Zac Dover [Fri, 3 May 2024 05:32:28 +0000 (15:32 +1000)]
doc/rados: PR#57022 unfinished business

Make the changes suggested by Anthony D'Atri in
https://github.com/ceph/ceph/pull/57022.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 93898d80836ac41041f75b5bf5398f9388abb52f)

17 months agoMerge pull request #57225 from zdover23/wip-doc-2024-05-02-cephadm-services-monitorin...
Zac Dover [Sat, 4 May 2024 12:37:04 +0000 (22:37 +1000)]
Merge pull request #57225 from zdover23/wip-doc-2024-05-02-cephadm-services-monitoring-squid

squid: doc/cephadm: Squid default images procedure

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Adam King <adking@redhat.com>
17 months agomgr/dashboard: add prometheus federation config for mulkti-cluster
Aashish Sharma [Tue, 12 Dec 2023 08:38:26 +0000 (14:08 +0530)]
mgr/dashboard: add prometheus federation config for mulkti-cluster
monitoring

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 82b50b4eac819a71542d766b573f65819046f403)

17 months agoMerge pull request #56964 from adk3798/squid-dashboard-anon-grafana
Nizamudeen A [Fri, 3 May 2024 05:30:10 +0000 (11:00 +0530)]
Merge pull request #56964 from adk3798/squid-dashboard-anon-grafana

squid: mgr/dashboard: fix dashboard not visible on disabled anonymous access

Reviewed-by: Nizamudeen A <nia@redhat.com>
17 months agoMerge pull request #57055 from cbodley/wip-65636-squid
Casey Bodley [Thu, 2 May 2024 20:18:35 +0000 (21:18 +0100)]
Merge pull request #57055 from cbodley/wip-65636-squid

squid: doc/rgw: add release note for changes to rgw_realm init

Reviewed-by: Zac Dover <zac.dover@proton.me>
17 months agoos/: modify getattrs to clear attrs out param before populating
Samuel Just [Thu, 18 Apr 2024 22:19:31 +0000 (15:19 -0700)]
os/: modify getattrs to clear attrs out param before populating

Passing in a non-empty map would otherwise exhibit quite unexpected
behavior.  For the bufferptr overload, any preexisting entries would
not be overwritten due to how std::map::emplace behaves.  For the
bufferlist overload, it would result in appending to any pre-existing
entries.

The prior commit cleans up one such inadvertent caller which resulted
in the below bug.

Fixes: https://tracker.ceph.com/issues/65185
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 915f92ba9dc8e192bdcebcb92e7d06a93a401f5d)

17 months agoMerge pull request #57246 from zdover23/wip-doc-2024-05-03-backport-57207-to-squid
Anthony D'Atri [Thu, 2 May 2024 18:30:34 +0000 (14:30 -0400)]
Merge pull request #57246 from zdover23/wip-doc-2024-05-03-backport-57207-to-squid

squid: doc/rados: s/cepgsqlite/cephsqlite/

17 months agodoc/rados: s/cepgsqlite/cephsqlite/
Zac Dover [Wed, 1 May 2024 16:42:46 +0000 (02:42 +1000)]
doc/rados: s/cepgsqlite/cephsqlite/

As stated in the commit-message line, this corrects the typo "cepg" to
the correct string "ceph".

This typo was discovered by https://github.com/test-erik and this was
brought to our attention way back in
https://github.com/ceph/ceph/pull/50420.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit ddef880947ac6b8b3b08092865f8a9a6fe03839c)

17 months agoMerge pull request #57095 from cbodley/wip-rgw-cache-size-squid
Casey Bodley [Thu, 2 May 2024 17:26:52 +0000 (18:26 +0100)]
Merge pull request #57095 from cbodley/wip-rgw-cache-size-squid

squid: rgw: increase default metadata cache size for accounts

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
17 months agoMerge pull request #57058 from cbodley/wip-65640-squid
Casey Bodley [Thu, 2 May 2024 17:26:36 +0000 (18:26 +0100)]
Merge pull request #57058 from cbodley/wip-65640-squid

squid: rgw/account: add bucket_quota to RGWAccountInfo

Reviewed-by: Oguzhan Ozmen <oozmen@bloomberg.net>
17 months agorgw/multisite-notification: retry storing bucket notification attrs for ECANCELED...
kchheda3 [Fri, 19 Apr 2024 14:59:13 +0000 (10:59 -0400)]
rgw/multisite-notification: retry storing bucket notification attrs for ECANCELED(ConcurrentModification) errors.

An ECANCELED error coming from our write of the bucket instance metadat is a common error for metadata writes on secondary zones, because secondary write races with metadata sync from the write that is forwarded to the master zone

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit 9ca76770434e4dd796afe2c88446f94279919026)

17 months agodoc/cephadm: Squid default images procedure
Zac Dover [Thu, 2 May 2024 08:36:34 +0000 (18:36 +1000)]
doc/cephadm: Squid default images procedure

Address Adam King's request for version-specific
cephadm-container-image-retrieval procedures, which he requested here: https://github.com/ceph/ceph/pull/57208#discussion_r1586614140

Co-authored-by: Adam King <adking@redhat.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
17 months agorefactor of s3select response handler, adding continuation-response to parquet flow...
Gal Salomon [Tue, 19 Mar 2024 11:45:58 +0000 (13:45 +0200)]
refactor of s3select response handler, adding continuation-response to parquet flow, refacor of error response

bug fix (init_success_response).
s3select submodule

fix for json-error-flow

Signed-off-by: Gal Salomon <gal.salomon@gmail.com>
(cherry picked from commit 06eadd66db4eb0a8688ee39ae13093a68f0868b5)

17 months agoMerge pull request #57208 from zdover23/wip-doc-2024-05-02-backport-57148-to-squid
Zac Dover [Thu, 2 May 2024 07:21:47 +0000 (17:21 +1000)]
Merge pull request #57208 from zdover23/wip-doc-2024-05-02-backport-57148-to-squid

squid: doc/cephadm: add default monitor images

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
17 months agomgr/dashboard: exclude cloned-deleted RBD snaps
Ernesto Puerta [Tue, 30 Apr 2024 08:51:23 +0000 (10:51 +0200)]
mgr/dashboard: exclude cloned-deleted RBD snaps

Fixes: https://tracker.ceph.com/issues/65698
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 8b1ef2ee411b7c6fb52825bf577860f3727738c2)

17 months agoMerge pull request #56946 from aaSharma14/wip-64791-squid
Nizamudeen A [Thu, 2 May 2024 05:52:03 +0000 (11:22 +0530)]
Merge pull request #56946 from aaSharma14/wip-64791-squid

squid: mgr/dashboard: Allow the user to add the access/secret key on zone edit and not on zone creation

Reviewed-by: afreen23 <NOT@FOUND>
17 months agodoc/cephadm: add default monitor images
Zac Dover [Tue, 30 Apr 2024 07:09:05 +0000 (17:09 +1000)]
doc/cephadm: add default monitor images

Add a list of default monitor images to the documentation. This commit
is made in response to a request from Eugen Block, and is made using the
information developed by Mr Block here:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/QGC66QIFBKRTPZAQMQEYFXOGZJ7RLWBN/.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d6311d24c277f35fcae5097ed8a19843e62e6149)

17 months agoMerge pull request #56790 from ronen-fr/wip-rf-56389-to-squid
Ronen Friedman [Wed, 1 May 2024 11:26:24 +0000 (14:26 +0300)]
Merge pull request #56790 from ronen-fr/wip-rf-56389-to-squid

squid: osd/scrub: remove detection & handling of reservation timeouts from the code

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
17 months agoMerge pull request #56777 from ronen-fr/wip-rf-56459-squid
Ronen Friedman [Wed, 1 May 2024 10:16:40 +0000 (13:16 +0300)]
Merge pull request #56777 from ronen-fr/wip-rf-56459-squid

squid: osd/scrub:  implement reservation state as a ReplicaActive flag
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
17 months agoMerge pull request #56903 from adk3798/wip-65417-squid
Adam King [Tue, 30 Apr 2024 19:47:19 +0000 (15:47 -0400)]
Merge pull request #56903 from adk3798/wip-65417-squid

squid: cephadm: fix host-maintenance command always exiting with a failure

Reviewed-by: John Mulligan <jmulligan@redhat.com>
17 months agoMerge pull request #56900 from adk3798/wip-65381-squid
Adam King [Tue, 30 Apr 2024 19:44:39 +0000 (15:44 -0400)]
Merge pull request #56900 from adk3798/wip-65381-squid

squid: mgr/cephadm: pass daemon's current image when reconfiguring

Reviewed-by: John Mulligan <jmulligan@redhat.com>
17 months agoMerge pull request #57156 from zdover23/wip-doc-2024-04-30-backport-57154-to-squid
Anthony D'Atri [Tue, 30 Apr 2024 14:05:32 +0000 (10:05 -0400)]
Merge pull request #57156 from zdover23/wip-doc-2024-04-30-backport-57154-to-squid

squid: doc/rados/operations: rephrase OSDs peering

17 months agoMerge pull request #57159 from zdover23/wip-doc-2024-04-30-backport-57153-to-squid
Anthony D'Atri [Tue, 30 Apr 2024 14:04:32 +0000 (10:04 -0400)]
Merge pull request #57159 from zdover23/wip-doc-2024-04-30-backport-57153-to-squid

squid: doc/man/8/radosgw-admin: add get lifecycle command

17 months agodoc/man/8/radosgw-admin: add get lifecycle command
rkhudov [Tue, 30 Apr 2024 10:00:36 +0000 (11:00 +0100)]
doc/man/8/radosgw-admin: add get lifecycle command

Signed-off-by: rkhudov <rhudov@gmail.com>
(cherry picked from commit 04393ec3c1a12d6a9405272ab2cffe9dfa67aa92)

17 months agodoc/rados/operations: rephrase OSDs peering
Piotr Parczewski [Tue, 30 Apr 2024 10:56:44 +0000 (12:56 +0200)]
doc/rados/operations: rephrase OSDs peering

Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
(cherry picked from commit 048f6e539b3035175479a88a4cf6f774cca68ddb)

17 months agoMerge pull request #56412 from zdover23/wip-doc-2024-03-23-backport-56379-to-squid
Zac Dover [Tue, 30 Apr 2024 06:50:53 +0000 (16:50 +1000)]
Merge pull request #56412 from zdover23/wip-doc-2024-03-23-backport-56379-to-squid

squid: doc/install: fix typos in openEuler-installation doc

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
17 months agoMerge pull request #57140 from zdover23/wip-doc-2024-04-30-backport-57136-to-squid
Anthony D'Atri [Mon, 29 Apr 2024 16:42:46 +0000 (12:42 -0400)]
Merge pull request #57140 from zdover23/wip-doc-2024-04-30-backport-57136-to-squid

squid: doc/radosgw: disambiguate version-added remarks

17 months agodoc/radosgw: disambiguate version-added remarks
Zac Dover [Mon, 29 Apr 2024 10:23:13 +0000 (20:23 +1000)]
doc/radosgw: disambiguate version-added remarks

Rewrite two sentences so that they explain precisely what was added to
Ceph and when it was added to Ceph.

This commit is made in response to a request made by Dr Per Mildner on
https://pad.ceph.com/p/Report_Documentation_Bugs.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 586e8e6577c6b62062d6ed60c439088705fbd2c1)

17 months agoMerge pull request #56497 from adk3798/wip-65124-squid
Adam King [Mon, 29 Apr 2024 15:18:43 +0000 (11:18 -0400)]
Merge pull request #56497 from adk3798/wip-65124-squid

squid: mgr/cephadm: add some NVMEof gateway fields to be configurable by cephadm

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
17 months agoMerge pull request #57121 from zdover23/wip-doc-2024-04-29-backport-57119-to-squid
Anthony D'Atri [Mon, 29 Apr 2024 12:36:07 +0000 (08:36 -0400)]
Merge pull request #57121 from zdover23/wip-doc-2024-04-29-backport-57119-to-squid

squid: doc/cephfs: edit front matter in client-auth.rst

17 months agodoc/cephfs: edit front matter in client-auth.rst
Zac Dover [Sun, 28 Apr 2024 13:18:50 +0000 (23:18 +1000)]
doc/cephfs: edit front matter in client-auth.rst

Clarify the front matter in doc/cephfs/client-auth.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 713c7304d15279cbc2031efc05cc9e49911974fe)

17 months agoMerge pull request #57108 from zdover23/wip-doc-2024-04-27-backport-57105-to-squid
Zac Dover [Fri, 26 Apr 2024 23:48:52 +0000 (09:48 +1000)]
Merge pull request #57108 from zdover23/wip-doc-2024-04-27-backport-57105-to-squid

squid: doc/mgr: explain error message - dashboard.rst

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
17 months agodoc/mgr: explain error message - dashboard.rst
Zac Dover [Fri, 26 Apr 2024 11:59:09 +0000 (21:59 +1000)]
doc/mgr: explain error message - dashboard.rst

Explain that an error message received in response to
"redirect_resolve_ip_addr True" might be caused by having an
insufficiently recent release of Ceph running in your cluster.

Fixes: https://tracker.ceph.com/issues/61312
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 78545030761330c44168dbb3f8d1e4c3c9a9a3bd)

17 months agorgw/lc: cleanup duplicate code
Soumya Koduri [Wed, 10 Apr 2024 19:34:17 +0000 (01:04 +0530)]
rgw/lc: cleanup duplicate code

Cleaning up duplicate code around updating head

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit e2b3a3002f404c8e61fd792e773abde1ec60016e)