]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agocommon: `counter dump` command revision
Ali Maredia [Thu, 1 Jun 2023 20:18:26 +0000 (16:18 -0400)]
common: `counter dump` command revision

`counter dump` now emits an array of
<labels,counters> pairs for each individual key.

Commit includes revisions to perf counters unit
test.

Fixes: https://tracker.ceph.com/issues/61587
Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit 78a148869389aa8df57596698c319de807535df8)

2 years agoMerge pull request #52463 from zdover23/wip-doc-2023-07-15-backport-52356-to-reef
Anthony D'Atri [Sat, 15 Jul 2023 02:12:39 +0000 (22:12 -0400)]
Merge pull request #52463 from zdover23/wip-doc-2023-07-15-backport-52356-to-reef

reef: doc/cephfs: add note to isolate metadata pool osds

2 years agodoc/cephfs: add note to isolate metadata pool osds 52463/head
Patrick Donnelly [Fri, 7 Jul 2023 12:42:58 +0000 (08:42 -0400)]
doc/cephfs: add note to isolate metadata pool osds

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4e2e61f16438dcd5eb35854091114d18b6fe9a9e)

2 years agoMerge pull request #52387 from rhcs-dashboard/cleanup-script-reef
Nizamudeen A [Fri, 14 Jul 2023 12:06:38 +0000 (17:36 +0530)]
Merge pull request #52387 from rhcs-dashboard/cleanup-script-reef

reef: mgr/dashboard: move cephadm e2e cleanup to jenkins job config

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2 years agoMerge pull request #52448 from zdover23/wip-doc-2023-07-14-backport-52423-to-reef
Anthony D'Atri [Fri, 14 Jul 2023 11:01:38 +0000 (07:01 -0400)]
Merge pull request #52448 from zdover23/wip-doc-2023-07-14-backport-52423-to-reef

reef: doc/rados: edit ceph-conf.rst

2 years agoMerge pull request #52450 from zdover23/wip-doc-2023-07-14-backport-52402-to-reef
Anthony D'Atri [Fri, 14 Jul 2023 10:59:58 +0000 (06:59 -0400)]
Merge pull request #52450 from zdover23/wip-doc-2023-07-14-backport-52402-to-reef

reef: doc/rados: edit add-or-rm-osds (2 of x)

2 years agodoc/rados: edit add-or-rm-osds (2 of x) 52450/head
Zac Dover [Tue, 11 Jul 2023 21:58:59 +0000 (07:58 +1000)]
doc/rados: edit add-or-rm-osds (2 of x)

Edit doc/rados/operations/add-or-rm-osds.rst (2 of x).

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d83fa5352aa556c4e559e66dd461c200cc86c709)

2 years agodoc/rados: edit ceph-conf.rst 52448/head
Zac Dover [Thu, 13 Jul 2023 08:41:51 +0000 (18:41 +1000)]
doc/rados: edit ceph-conf.rst

Edit doc/rados/configuration/ceph-conf.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit dd8d91b0ded8be70bec3f40f499d7cf9432c9f23)

2 years agoMerge pull request #52425 from zdover23/wip-doc-2023-07-13-backport-52419-to-reef
Anthony D'Atri [Thu, 13 Jul 2023 13:12:31 +0000 (09:12 -0400)]
Merge pull request #52425 from zdover23/wip-doc-2023-07-13-backport-52419-to-reef

reef: docs: fix nfs cluster create syntax

2 years agodocs: fix nfs cluster create syntax 52425/head
Paul Cuzner [Thu, 13 Jul 2023 03:13:42 +0000 (15:13 +1200)]
docs: fix nfs cluster create syntax

The ceph prefix was missing from the command
example.

Signed-off-by: Paul Cuzner <pcuzner@ibm.com>
(cherry picked from commit 43b5f4b4f92341803753fcd3b67132546829e362)

2 years agoMerge pull request #52071 from guits/wip-61339-reef
Guillaume Abrioux [Thu, 13 Jul 2023 08:05:18 +0000 (10:05 +0200)]
Merge pull request #52071 from guits/wip-61339-reef

reef: ceph-volume: drop filestore support

2 years agoMerge pull request #52061 from guits/wip-61152-reef
Guillaume Abrioux [Thu, 13 Jul 2023 08:03:02 +0000 (10:03 +0200)]
Merge pull request #52061 from guits/wip-61152-reef

reef: ceph-volume: fix a bug in `get_lvm_fast_allocs()` (batch)

2 years agoMerge pull request #52344 from adk3798/reef-cephadm-fix-rgw-migration
Yuri Weinstein [Wed, 12 Jul 2023 22:35:54 +0000 (18:35 -0400)]
Merge pull request #52344 from adk3798/reef-cephadm-fix-rgw-migration

reef: mgr/cephadm: fix rgw spec migration with simple specs

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge pull request #52343 from adk3798/reef-default-image
Yuri Weinstein [Wed, 12 Jul 2023 22:30:18 +0000 (18:30 -0400)]
Merge pull request #52343 from adk3798/reef-default-image

reef: mgr/cephadm: set default image for reef

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2 years agoMerge pull request #52318 from mkogan1/wip-59739-reef
Yuri Weinstein [Wed, 12 Jul 2023 13:59:13 +0000 (09:59 -0400)]
Merge pull request #52318 from mkogan1/wip-59739-reef

reef: rgw: D3N cache objects which oid contains slash

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #52316 from mkogan1/wip-61661-reef
Yuri Weinstein [Wed, 12 Jul 2023 13:57:08 +0000 (09:57 -0400)]
Merge pull request #52316 from mkogan1/wip-61661-reef

reef: rgw: d3n: fix valgrind reported leak related to libaio ops

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agoMerge pull request #52297 from cbodley/wip-61888-reef
Yuri Weinstein [Wed, 12 Jul 2023 13:55:54 +0000 (09:55 -0400)]
Merge pull request #52297 from cbodley/wip-61888-reef

reef: rgw: rgwx-skip-decrypt also skips decompression of encrypted objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #51910 from mkogan1/wip-58692-reef
Yuri Weinstein [Wed, 12 Jul 2023 13:54:35 +0000 (09:54 -0400)]
Merge pull request #51910 from mkogan1/wip-58692-reef

reef: rgw: set bulk pool flag when creating data pool

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #50936 from ifed01/wip-ifed-report-min-alloc-size-ree
Yuri Weinstein [Wed, 12 Jul 2023 13:34:04 +0000 (09:34 -0400)]
Merge pull request #50936 from ifed01/wip-ifed-report-min-alloc-size-ree

reef: os/bluestore: report min_alloc_size through "ceph osd metadata"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #52329 from soumyakoduri/wip-skoduri-reef
Yuri Weinstein [Tue, 11 Jul 2023 15:30:25 +0000 (11:30 -0400)]
Merge pull request #52329 from soumyakoduri/wip-skoduri-reef

[reef]rgw: Do not update mtime in `set_attrs`

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #52321 from batrick/wip-61899-reef
Yuri Weinstein [Tue, 11 Jul 2023 14:33:53 +0000 (10:33 -0400)]
Merge pull request #52321 from batrick/wip-61899-reef

reef: pybind: drop GIL during library callouts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoPendingReleaseNotes: note rgw's compress-encrypted zonegroup feature flag 52297/head
Casey Bodley [Mon, 3 Jul 2023 19:06:29 +0000 (15:06 -0400)]
PendingReleaseNotes: note rgw's compress-encrypted zonegroup feature flag

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fd2ec27f538bceadec412a848e2d29b187ed04bb)

2 years agodoc/rgw: document compress-encrypted zonegroup feature flag
Casey Bodley [Mon, 3 Jul 2023 18:59:51 +0000 (14:59 -0400)]
doc/rgw: document compress-encrypted zonegroup feature flag

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d85011a7fc5d3df42a2c046ec7a43fe29f1db08f)

2 years agorgw: require zonegroup feature to compress encrypted objects
Casey Bodley [Mon, 3 Jul 2023 19:00:17 +0000 (15:00 -0400)]
rgw: require zonegroup feature to compress encrypted objects

Reef adds support for combining server-side encryption and compression,
but previous releases would not replicate such objects correctly. gate
the feature behind a zonegroup feature flag to require that all zones
upgrade to reef before enabling. this feature is not enabled by default
in new zonegroups

Fixes: https://tracker.ceph.com/issues/57905
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4bc52954ab7b38a8eac50c116196d6db728cee00)

Conflicts:
src/rgw/rgw_op.cc: SiteConfig -> sal::ZoneGroup::supports

2 years agorgw/sal: add ZoneGroup::supports()
Casey Bodley [Tue, 11 Jul 2023 14:00:24 +0000 (10:00 -0400)]
rgw/sal: add ZoneGroup::supports()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agodoc/rgw: don't capitalize zonegroup feature flags
Casey Bodley [Mon, 3 Jul 2023 18:59:24 +0000 (14:59 -0400)]
doc/rgw: don't capitalize zonegroup feature flags

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 39be699629df5c6a6b82fa956ed4c22fc4a38523)

2 years agorgw: don't enable all zone features by default
Casey Bodley [Fri, 7 Jul 2023 17:00:06 +0000 (13:00 -0400)]
rgw: don't enable all zone features by default

on zonegroup creation, all `supported` features get enabled by default.
however, some features should remain opt-in. add a separate list of
`enabled` features for the subset of features we want enabled by default
on zonegroup creation

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cdb1be836cc1b8c2e8f178bc00bad24d2b8413bf)

Conflicts:
src/rgw/driver/rados/rgw_zone.cc no create_default_zonegroup()

2 years agomgr/dashboard: move cephadm e2e cleanup to jenkins job config 52387/head
Nizamudeen A [Fri, 7 Jul 2023 10:46:00 +0000 (16:16 +0530)]
mgr/dashboard: move cephadm e2e cleanup to jenkins job config

Follows: https://github.com/ceph/ceph-build/pull/2145

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a407f0b7834585c087001059d6cec613e524538b)

2 years agoMerge pull request #52383 from zdover23/wip-doc-2023-07-11-backport-52366-to-reef
Anthony D'Atri [Mon, 10 Jul 2023 23:06:28 +0000 (19:06 -0400)]
Merge pull request #52383 from zdover23/wip-doc-2023-07-11-backport-52366-to-reef

reef: doc/rados: edit add-or-rm-osds (1 of x)

2 years agodoc/rados: edit add-or-rm-osds (1 of x) 52383/head
Zac Dover [Sun, 9 Jul 2023 02:50:11 +0000 (12:50 +1000)]
doc/rados: edit add-or-rm-osds (1 of x)

Edit docs/rados/operations/add-or-rm-osds.

Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a3ab5a7ad8a44c702089294b107d09dca3a98139)

2 years agoMerge pull request #52381 from zdover23/wip-doc-2023-07-11-backport-52371-to-reef
Anthony D'Atri [Mon, 10 Jul 2023 20:22:11 +0000 (16:22 -0400)]
Merge pull request #52381 from zdover23/wip-doc-2023-07-11-backport-52371-to-reef

reef: doc/rados: introduce emdash

2 years agodoc/rados: introduce emdash 52381/head
Zac Dover [Mon, 10 Jul 2023 02:26:04 +0000 (12:26 +1000)]
doc/rados: introduce emdash

Introduce the emdash character in
docs/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a971a3a66a93ce52898f201a41a9aa13e0dd8bfc)

2 years agoMerge pull request #52333 from rhcs-dashboard/wip-61911-reef
Nizamudeen A [Mon, 10 Jul 2023 14:43:12 +0000 (20:13 +0530)]
Merge pull request #52333 from rhcs-dashboard/wip-61911-reef

reef: mgr/dashboard: fix a bug where data would plot wrongly

Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #52367 from zdover23/wip-doc-2023-07-09-backport-52327-to-reef
Anthony D'Atri [Sun, 9 Jul 2023 15:02:16 +0000 (11:02 -0400)]
Merge pull request #52367 from zdover23/wip-doc-2023-07-09-backport-52327-to-reef

reef: doc: add information on expediting MDS recovery

2 years agodoc: add information on expediting MDS recovery 52367/head
Patrick Donnelly [Thu, 6 Jul 2023 01:25:59 +0000 (21:25 -0400)]
doc: add information on expediting MDS recovery

Fixes: https://tracker.ceph.com/issues/61865
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0a15144c584d4445bd037340f053aa64b9c13f32)

2 years agoMerge pull request #52361 from zdover23/wip-doc-2023-07-08-backport-52360-to-reef
Anthony D'Atri [Fri, 7 Jul 2023 23:48:43 +0000 (19:48 -0400)]
Merge pull request #52361 from zdover23/wip-doc-2023-07-08-backport-52360-to-reef

reef: doc/radosgw: add Zonegroup policy explanation

2 years agoMerge pull request #52226 from rzarzynski/wip-59102-reef
Yuri Weinstein [Fri, 7 Jul 2023 21:05:26 +0000 (17:05 -0400)]
Merge pull request #52226 from rzarzynski/wip-59102-reef

reef: msg/async: don't abort when public addrs mismatch bind addrs

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agodoc/radosgw: add Zonegroup policy explanation 52361/head
Zac Dover [Fri, 7 Jul 2023 17:35:15 +0000 (03:35 +1000)]
doc/radosgw: add Zonegroup policy explanation

Add revised Zonegroup policy for "multi-zonegroups". This commit
includes changes that Casey Bodley made in
https://github.com/ceph/ceph/pull/52324#discussion_r1253482258 and that
I have integrated into the docs only now.

Co-authored-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c6400ed79b1233a2238342e4931edf341f1c7b9f)

2 years agoMerge pull request #52190 from ajarr/wip-61706-reef
Yuri Weinstein [Fri, 7 Jul 2023 18:45:27 +0000 (14:45 -0400)]
Merge pull request #52190 from ajarr/wip-61706-reef

reef: mgr/rbd_support: add user-friendly stderr message when module is not ready

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoMerge pull request #52162 from alimaredia/wip-61773-reef
Yuri Weinstein [Fri, 7 Jul 2023 18:44:05 +0000 (14:44 -0400)]
Merge pull request #52162 from alimaredia/wip-61773-reef

reef: Tempest tests Keystone Fix

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agoMerge pull request #52347 from zdover23/wip-doc-2023-07-07-backport-52319-to-reef
zdover23 [Thu, 6 Jul 2023 22:52:13 +0000 (08:52 +1000)]
Merge pull request #52347 from zdover23/wip-doc-2023-07-07-backport-52319-to-reef

reef: doc/foundation: Updating foundation members for July 2023

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #52348 from zdover23/wip-doc-2023-07-07-backport-52324-to-reef
Anthony D'Atri [Thu, 6 Jul 2023 21:10:51 +0000 (17:10 -0400)]
Merge pull request #52348 from zdover23/wip-doc-2023-07-07-backport-52324-to-reef

reef: doc/radosgw: add Zonegroup purpose

2 years agodoc/radosgw: add Zonegroup purpose 52348/head
Zac Dover [Wed, 5 Jul 2023 17:36:52 +0000 (03:36 +1000)]
doc/radosgw: add Zonegroup purpose

Add a note about the telos of zonegroups. The information added in this
commit was originally sent to ceph-users by Casey Bodley on 30 Jun 2023
in response to a question from Yixin Jin. Alexander Patrakov then
suggested that Casey's response should be added to
doc/radsogw/multisite.rst.

Co-authored-by: Casey Bodley <cbodley@redhat.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 5c46a1a00962918c885e434986d0f846c3ee4730)

2 years agodoc/foundation: Updating foundation members for July 2023 52347/head
Mike Perez [Wed, 5 Jul 2023 15:31:04 +0000 (08:31 -0700)]
doc/foundation: Updating foundation members for July 2023

Signed-off-by: Mike Perez <thingee@gmail.com>
(cherry picked from commit 640390906f76da4bfd66bcd8006e964d6e73c98e)

2 years agomgr/cephadm: set default image for reef 52343/head
Adam King [Thu, 6 Jul 2023 15:46:31 +0000 (11:46 -0400)]
mgr/cephadm: set default image for reef

This needs to be a v18 image instead of the
main branch ci build, otherwise those that
bootstrap without providing an image name
will get main instead of reef

Signed-off-by: Adam King <adking@redhat.com>
2 years agomgr/cephadm: fix rgw spec migration with simple specs 52344/head
Adam King [Mon, 3 Jul 2023 18:33:34 +0000 (14:33 -0400)]
mgr/cephadm: fix rgw spec migration with simple specs

The rgw spec migration code, intended to formalize
the rgw_frontend_type spec option, doesn't work with
simple specs i.e.

service_type: rgw
service_id: rgw.1
service_name: rgw.rgw.1
placement:
  label: rgw

because the migration code assumes there will always be
a "spec" section inside the spec. This is the case for
more involved rgw specs such as

service_type: rgw
service_id: foo
placement:
  label: rgw
  count_per_host: 2
spec:
  rgw_realm: myrealm
  rgw_zone: myzone
  rgw_frontend_type: "beast"
  rgw_frontend_port: 5000

which is what the migration is actually concerned about
(verification of the rgw_frontend_type in these specs).

In the case where the spec is more simple, we should
just leave the spec alone and move on. Unfortunately
the current code assumes the field will always be
there and hits an unhandled KeyError when trying to
migrate the more simple specs. This causes the
cephadm module to crash shortly after starting an
upgrade to a version that includes this migration
and it's very difficult to find the root cause. This
can be worked around by adding fields to the rgw
spec before upgrade so the "spec" field exists in
the spec and the migration works as intended.

This commit fixes the migration in the simple
case as well as adding testing for that case to
both the unit tests and orch/cephadm teuthology
upgrade tests

Fixes: https://tracker.ceph.com/issues/61889
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 1860ef83877c76c51a20aac48036bd9590572cc2)

2 years agomgr/dashboard: fix a bug where data would plot on the graphs without converting to... 52333/head
Pedro Gonzalez Gomez [Thu, 16 Mar 2023 12:09:26 +0000 (13:09 +0100)]
mgr/dashboard: fix a bug where data would plot on the graphs without converting to the data units of the graph

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 7aa70a16a54c3387b2e021763ca1d118d6785835)

2 years agorgw: Do not update mtime in `set_attrs` 52329/head
Soumya Koduri [Sat, 24 Jun 2023 06:55:46 +0000 (12:25 +0530)]
rgw: Do not update mtime in `set_attrs`

As per https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html,
the only way for users to modify object metadata is to make a copy of the object and
set the metadata. Hence retain old mtime for any other attr changes

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 31666d71aa5b58a9384938ba31892d48f2ac3f67)

2 years agopybind/rbd: drop GIL when calling into librbd 52321/head
Ilya Dryomov [Mon, 3 Jul 2023 11:34:30 +0000 (13:34 +0200)]
pybind/rbd: drop GIL when calling into librbd

This was missing for rbd_mirror_peer_site_add() and
rbd_get_data_pool_id().

While at it, add a test for data_pool_id().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 1a60f66b0a13089c789e2e056b925ac6fef75021)

2 years agopybind/cephfs: drop gil during cephfs callouts
Patrick Donnelly [Sun, 2 Jul 2023 16:05:08 +0000 (12:05 -0400)]
pybind/cephfs: drop gil during cephfs callouts

This has disastorous consequences including the possibility of deadlock.
In the best case, you have the rmdir holding the GIL until the MDS
responds!

Fixes: https://tracker.ceph.com/issues/61869
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5d249a4c978198f1e8974affac0297b71bebada5)

2 years agoMerge pull request #52291 from zdover23/wip-doc-2023-07-03-backport-52280-to-reef... 52281/head
zdover23 [Wed, 5 Jul 2023 16:47:03 +0000 (02:47 +1000)]
Merge pull request #52291 from zdover23/wip-doc-2023-07-03-backport-52280-to-reef-2nd-attempt

reef: doc/radosgw: refine "Zones" in multisite.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/radosgw: refine "Zones" in multisite.rst 52291/head
Zac Dover [Sat, 1 Jul 2023 11:51:15 +0000 (21:51 +1000)]
doc/radosgw: refine "Zones" in multisite.rst

Describe in greater detail the function of Zones in serving S3 objects,
as requested by Alexander Patrakov here:
https://github.com/ceph/ceph/pull/49981#issuecomment-1615866374.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8dab6c066c4dd3699c7b7c80f7a30a0e18326bba)

2 years agoreef: rgw: D3N cache objects which oid contains slash 52318/head
Mark Kogan [Mon, 29 May 2023 17:36:56 +0000 (17:36 +0000)]
reef: rgw: D3N cache objects which oid contains slash

Fixes: https://tracker.ceph.com/issues/59739
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit fcb91f3e2a6fd8066ac33950d421e0a9b9b4b490)

 Conflicts:
src/rgw/driver/rados/rgw_d3n_datacache.cc
src/rgw/rgw_aio.cc
src/rgw/rgw_d3n_cacherequest.h

2 years agoreef: rgw: d3n: fix valgrind reported leak related to libaio ops 52316/head
Mark Kogan [Wed, 28 Jun 2023 14:06:27 +0000 (14:06 +0000)]
reef: rgw: d3n: fix valgrind reported leak related to libaio ops

Fixes: https://tracker.ceph.com/issues/61661
The valgrind leak indication is a false positive in this case,
it is because the libaio internal thread have not timed out yet
when radosgw is terminated.

```
man aio_init
...
aio_idle_time
This field specifies the amount of time in seconds that a worker thread
should wait for further requests before terminating, after having
completed a previous request.   The
default value is 1.
...
```
for the sake of teuthology reducing the timeout

waiting for 2 minutes for example like below would also prevent the leak report
```
❯ env
LD_LIBRARY_PATH=/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/:$LD_LIBRARY_PATH
PYTHONPATH=$PYTHONPATH:/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/cython_modules/lib.3
RAGWEED_CONF=$(realpath ./ragweed.conf) RAGWEED_STAGES=prepare,check tox
-- -v |& ccze -Aonolookups ;   sleep 2m | pv -t ; pkill radosgw
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit f834e10cfe97c7df8fc5ead682975fe60b435bd7)

2 years agoMerge pull request #52298 from zdover23/wip-doc-2023-07-04-backport-51422-to-reef
zdover23 [Tue, 4 Jul 2023 22:11:55 +0000 (08:11 +1000)]
Merge pull request #52298 from zdover23/wip-doc-2023-07-04-backport-51422-to-reef

reef: doc/rgw/lua: add info uploading a script in cephadm deployment

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rgw/lua: add info uploading a script in cephadm deployment 52298/head
Yuval Lifshitz [Wed, 10 May 2023 09:43:45 +0000 (12:43 +0300)]
doc/rgw/lua: add info uploading a script in cephadm deployment

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit 5cec4f719581e93760c22419e701c563a8688de9)

2 years agoMerge pull request #52311 from zdover23/wip-doc-2023-07-05-backport-52207-to-reef
zdover23 [Tue, 4 Jul 2023 19:35:58 +0000 (05:35 +1000)]
Merge pull request #52311 from zdover23/wip-doc-2023-07-05-backport-52207-to-reef

reef: doc/rados: edit crush-map-edits (2 of x)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2 years agodoc/rados: edit crush-map-edits (2 of x) 52311/head
Zac Dover [Tue, 27 Jun 2023 08:58:32 +0000 (18:58 +1000)]
doc/rados: edit crush-map-edits (2 of x)

Edit doc/rados/operations/crush-map-edits.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit ae04debe9e2fa18c1cb8745b0573ff56576fd3b0)

2 years agoMerge pull request #52309 from zdover23/wip-doc-2023-07-05-backport-52308-to-reef
zdover23 [Tue, 4 Jul 2023 18:25:35 +0000 (04:25 +1000)]
Merge pull request #52309 from zdover23/wip-doc-2023-07-05-backport-52308-to-reef

reef: doc/dev: remove seqdiag assets

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2 years agodoc/dev: remove seqdiag assets 52309/head
Zac Dover [Tue, 4 Jul 2023 17:37:10 +0000 (03:37 +1000)]
doc/dev: remove seqdiag assets

Remove seqdiag assets to determine whether the docs can be built if they
are absent. (Currently they cannot be built when they are present.) If
this works, then these diagrams will be replaced, probably with .png
files.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit ce96ddd4c7da65351deaf14dc4f8b42f5148a352)

2 years agorgw: fetch_remote_obj() preserves RGW_ATTR_COMPRESSION of encrypted objects
Casey Bodley [Wed, 28 Jun 2023 18:57:49 +0000 (14:57 -0400)]
rgw: fetch_remote_obj() preserves RGW_ATTR_COMPRESSION of encrypted objects

if the source object was both compressed and encrypted, preserve its
original compression attribute so it can be decompressed the same way it
was originally compressed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b91ff3fd86f52f7148daf7f9acc9864752186ce0)

2 years agorgw: fetch_remote_obj() will never verify etags of encrypted objects
Casey Bodley [Wed, 28 Jun 2023 18:43:55 +0000 (14:43 -0400)]
rgw: fetch_remote_obj() will never verify etags of encrypted objects

fetch_remote_obj() transfers objects in their encrypted form, so does
not have access to the decrypted data for checksum verification

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ea45da747d5fa898b4aa4ec9223cd7b6983ca177)

2 years agorgw: rgwx-skip-decrypt also skips decompression of encrypted objects
Casey Bodley [Wed, 28 Jun 2023 18:42:16 +0000 (14:42 -0400)]
rgw: rgwx-skip-decrypt also skips decompression of encrypted objects

compression is applied before encryption. so if we skip decryption, we
can't decompress either

Fixes: https://tracker.ceph.com/issues/57905
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8d35759eb8edcc7cd15e6fd67b91da637fc477ae)

2 years agoMerge pull request #52278 from zdover23/wip-doc-2023-07-01-backport-52277-to-reef
Anthony D'Atri [Sat, 1 Jul 2023 11:46:52 +0000 (07:46 -0400)]
Merge pull request #52278 from zdover23/wip-doc-2023-07-01-backport-52277-to-reef

reef: doc/radosgw: s/s3select/S3 Select/

2 years agodoc/radosgw: s/s3select/S3 Select/ 52278/head
Zac Dover [Sat, 1 Jul 2023 11:14:30 +0000 (21:14 +1000)]
doc/radosgw: s/s3select/S3 Select/

Correct several misspelling of "S3 Select". Hat tip to Anthony D'Atri,
who caught this in an earlier PR.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 71c327ef4f56a6ec900a8c5ea071aaf8005a60a5)

2 years agoMerge pull request #52187 from zdover23/wip-doc-2023-06-26-backport-52181-2nd-attempt...
zdover23 [Thu, 29 Jun 2023 23:54:38 +0000 (09:54 +1000)]
Merge pull request #52187 from zdover23/wip-doc-2023-06-26-backport-52181-2nd-attempt-to-reef

reef: doc/radosgw: remove pipes from s3select.rst

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2 years agoMerge pull request #52219 from zdover23/wip-doc-2023-06-28-backport-52194-to-reef
Anthony D'Atri [Thu, 29 Jun 2023 19:11:47 +0000 (15:11 -0400)]
Merge pull request #52219 from zdover23/wip-doc-2023-06-28-backport-52194-to-reef

reef: doc/radosgw: edit "Overview" in s3select.rst

2 years agoMerge pull request #52262 from zdover23/wip-doc-2023-06-30-backport-52250-to-reef
zdover23 [Thu, 29 Jun 2023 18:28:14 +0000 (04:28 +1000)]
Merge pull request #52262 from zdover23/wip-doc-2023-06-30-backport-52250-to-reef

reef: doc/radosgw: edit "Basic Workflow" in s3select.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #52260 from ceph/reef-release
Laura Flores [Thu, 29 Jun 2023 18:10:52 +0000 (13:10 -0500)]
Merge pull request #52260 from ceph/reef-release

v18.1.2

2 years agodoc/radosgw: edit "Basic Workflow" in s3select.rst 52262/head
Zac Dover [Thu, 29 Jun 2023 08:48:00 +0000 (18:48 +1000)]
doc/radosgw: edit "Basic Workflow" in s3select.rst

Edit the "Basic Workflow" section in doc/radosgw/s3select.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4d2c09b683421552cfb4df7f467f2d9a6c9c7c26)

2 years agoMerge pull request #52136 from ljflores/wip-61603-reef
Yuri Weinstein [Wed, 28 Jun 2023 22:17:08 +0000 (18:17 -0400)]
Merge pull request #52136 from ljflores/wip-61603-reef

reef: qa/suites/rados: whitelist POOL_APP_NOT_ENABLED for cls tests

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #52135 from rzarzynski/wip-61742-reef
Yuri Weinstein [Wed, 28 Jun 2023 22:16:21 +0000 (18:16 -0400)]
Merge pull request #52135 from rzarzynski/wip-61742-reef

reef: mon/MonClient: before complete auth with error, reopen session

Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge PR #50921 into reef
Patrick Donnelly [Wed, 28 Jun 2023 17:27:36 +0000 (13:27 -0400)]
Merge PR #50921 into reef

* refs/pull/50921/head:
qa: load file system info if not created

2 years agomsg/async: don't abort when public addrs mismatch bind addrs 52226/head
Radosław Zarzyński [Thu, 16 Mar 2023 17:05:17 +0000 (18:05 +0100)]
msg/async: don't abort when public addrs mismatch bind addrs

Before the 69b47c805fdd2ecd4f58547d58c9f019fc62d447 (PR #50153)
a mismatch (in number or types of stored `entity_addr_t`) between
public addrs and bind addrs vectors was ignored and the former
was taking over anything else -- it was possible to e.g. bind to
both v1 and v2 addresses but expose v2 only. Unfortunately, that's
exactly how Rook configures ceph-mon:

```
debug 2023-03-16T21:01:48.389+0000 7f99822bf8c0  0 starting mon.a rank 0 at public addrs v2:172.30.122.144:3300/0 at bind addrs [v2:10.129.2.21:3300/0,v1:10.129.2.21:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid acc14d1b-fb2b-4f01-8b61-6e7cb26e9200
```

The consequnece is the following abort:

```
ceph version 17.2.5-1338.el9cp (5adce3015143c7c2cc135a71368be194744f5761) quincy (stable)
1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xd3) [0x7ff05392c1a6]
2: /usr/lib64/ceph/libceph-common.so.2(+0x165ac1) [0x7ff05394eac1]
3: (AsyncMessenger::bindv(entity_addrvec_t const&, std::optional<entity_addrvec_t>)+0x1fe) [0x7ff053baa0ce]
4: main()
5: /lib64/libc.so.6(+0x3feb0) [0x7ff053048eb0]
6: __libc_start_main()
7: _start()
debug *** Caught signal (Aborted) **  in thread 7ff052ab18c0 thread_name:ceph-mon 2023-03-16T09:56:35.995+0000 7ff052ab18c0 -1 /builddir/build/BUILD/ceph-17.2.5-1338-g484e8dbb/src/msg/msg_types.h: In function 'void entity_addr_t::set_port(int)' thread 7ff052ab18c0 time 2023-03-16T09:56:35.996339+0000 /builddir/build/BUILD/ceph-17.2.5-1338-g484e8dbb/src/msg/msg_types.h: 359: ceph_abort_msg("abort() called")
```

This commit brings the original logic back but in a way that
preserves the port numbers figured out by. e.g. `Processor::bind`.

Fixes: https://tracker.ceph.com/issues/59100
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit 68fbdcf9b0937e8102b86d0bf84815554b62328d)

2 years agodoc/radosgw: edit "Overview" in s3select.rst 52219/head
Zac Dover [Mon, 26 Jun 2023 11:45:43 +0000 (21:45 +1000)]
doc/radosgw: edit "Overview" in s3select.rst

Edit the "Overview" section in doc/radosgw/s3select.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e051dd1a753614fc829f2054a69eda185f190db6)

2 years ago18.1.2 52260/head v18.1.2
Ceph Release Team [Tue, 27 Jun 2023 19:59:59 +0000 (19:59 +0000)]
18.1.2

Signed-off-by: Ceph Release Team <ceph-maintainers@ceph.io>
2 years agoMerge pull request #51608 from vshankar/tr-61234
Yuri Weinstein [Tue, 27 Jun 2023 14:47:13 +0000 (10:47 -0400)]
Merge pull request #51608 from vshankar/tr-61234

reef: mds: rdlock_path_xlock_dentry supports returning auth target inode

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #51606 from vshankar/tr-59723
Yuri Weinstein [Tue, 27 Jun 2023 14:46:39 +0000 (10:46 -0400)]
Merge pull request #51606 from vshankar/tr-59723

reef: qa: run scrub post file system recovery

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #51515 from dparmar18/wip-61187-reef
Yuri Weinstein [Tue, 27 Jun 2023 14:45:46 +0000 (10:45 -0400)]
Merge pull request #51515 from dparmar18/wip-61187-reef

reef: qa: ignore cluster warning when fs flag refuse_client_session is set

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #51041 from vshankar/tr-59412
Yuri Weinstein [Tue, 27 Jun 2023 14:45:17 +0000 (10:45 -0400)]
Merge pull request #51041 from vshankar/tr-59412

reef: client, mds: update mtime and change attr for snapdir when snaps are created, deleted and renamed

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #51040 from vshankar/tr-59409
Yuri Weinstein [Tue, 27 Jun 2023 14:44:46 +0000 (10:44 -0400)]
Merge pull request #51040 from vshankar/tr-59409

reef: mgr/volumes: avoid returning -ESHUTDOWN back to cli

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #50997 from lxbsz/wip-59404
Yuri Weinstein [Tue, 27 Jun 2023 14:44:08 +0000 (10:44 -0400)]
Merge pull request #50997 from lxbsz/wip-59404

reef: mds: force replay sessionmap version

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #50990 from lxbsz/wip-59399
Yuri Weinstein [Tue, 27 Jun 2023 14:43:33 +0000 (10:43 -0400)]
Merge pull request #50990 from lxbsz/wip-59399

reef: qa: enable kclient test for newop test

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #50987 from lxbsz/wip-59267
Yuri Weinstein [Tue, 27 Jun 2023 14:42:39 +0000 (10:42 -0400)]
Merge pull request #50987 from lxbsz/wip-59267

reef: client: clear the suid/sgid in fallocate path

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #50984 from lxbsz/wip-59384
Yuri Weinstein [Tue, 27 Jun 2023 14:42:04 +0000 (10:42 -0400)]
Merge pull request #50984 from lxbsz/wip-59384

reef: mds: wait for unlink operation to finish

Reviewed-by: Milind Changire <mchangir@redhat.com>
2 years agoMerge pull request #52148 from idryomov/wip-61616-reef
Yuri Weinstein [Mon, 26 Jun 2023 21:32:53 +0000 (17:32 -0400)]
Merge pull request #52148 from idryomov/wip-61616-reef

reef: librbd: avoid object map corruption in snapshots taken under I/O

Reviewed-by: Mykola Golub <mgolub@suse.com>
2 years agoMerge pull request #52116 from cbodley/wip-61729-reef
Yuri Weinstein [Mon, 26 Jun 2023 21:32:10 +0000 (17:32 -0400)]
Merge pull request #52116 from cbodley/wip-61729-reef

reef: librados: aio operate functions can set times

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoMerge pull request #52041 from cbodley/wip-61665-reef
Yuri Weinstein [Mon, 26 Jun 2023 19:16:48 +0000 (15:16 -0400)]
Merge pull request #52041 from cbodley/wip-61665-reef

reef: qa: suppress SyscallParam error during startup on jammy

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agomgr/rbd_support: log number of images 52190/head
Ramana Raja [Wed, 14 Jun 2023 21:41:48 +0000 (17:41 -0400)]
mgr/rbd_support: log number of images

... that have one snapshot request pending when the
mirror_snapshot_schedule handler is shutting down.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit edc3b0e80653c30d0fc318a24673e8a0568912ad)

2 years agomgr/rbd_support: add user-friendly stderr message
Ramana Raja [Wed, 14 Jun 2023 20:17:09 +0000 (16:17 -0400)]
mgr/rbd_support: add user-friendly stderr message

... when the rbd_support module is not ready.

Fixes: https://tracker.ceph.com/issues/61688
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 6351ef5c8e691e359b1bf913dde4dbc8a441be1d)

2 years agodoc/radosgw: remove pipes from s3select.rst 52187/head
Zac Dover [Sun, 25 Jun 2023 09:03:07 +0000 (19:03 +1000)]
doc/radosgw: remove pipes from s3select.rst

Remove strange formatting introduced in
61d6813726fc08980903c6a6c50b0a5797ba7193.

Note to the documentation team: This is not a line-edit. This commit
includes nothing but the removal of pipes added to the left of much of
the text in this file. Several future commits will line-edit this file
and correct its formatting.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a4d302ee1a6e2f71263a898109ebed28e7e8eaca)

2 years agoMerge pull request #52183 from zdover23/wip-doc-2023-06-25-backport-52181-to-reef
zdover23 [Sun, 25 Jun 2023 23:23:10 +0000 (09:23 +1000)]
Merge pull request #52183 from zdover23/wip-doc-2023-06-25-backport-52181-to-reef

reef: doc/radosgw: remove pipes from s3select.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/radosgw: remove pipes from s3select.rst 52183/head
Zac Dover [Sun, 25 Jun 2023 09:03:07 +0000 (19:03 +1000)]
doc/radosgw: remove pipes from s3select.rst

Remove strange formatting introduced in
61d6813726fc08980903c6a6c50b0a5797ba7193.

Note to the documentation team: This is not a line-edit. This commit
includes nothing but the removal of pipes added to the left of much of
the text in this file. Several future commits will line-edit this file
and correct its formatting.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a4d302ee1a6e2f71263a898109ebed28e7e8eaca)

2 years agoMerge pull request #52179 from zdover23/wip-doc-2023-06-24-backport-52167-to-reef
Anthony D'Atri [Sat, 24 Jun 2023 14:06:23 +0000 (10:06 -0400)]
Merge pull request #52179 from zdover23/wip-doc-2023-06-24-backport-52167-to-reef

reef: doc/rados: edit crush-map-edits.rst (1 of x)

2 years agodoc/rados: edit crush-map-edits.rst (1 of x) 52179/head
Zac Dover [Fri, 23 Jun 2023 00:59:10 +0000 (10:59 +1000)]
doc/rados: edit crush-map-edits.rst (1 of x)

Edit doc/rados/operations/crush-map-edits.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit f447c290a877c84cb14f0141ff5330f7147b2f6a)

2 years agoMerge pull request #52108 from mkogan1/wip-58911-reef
Yuri Weinstein [Fri, 23 Jun 2023 16:17:46 +0000 (12:17 -0400)]
Merge pull request #52108 from mkogan1/wip-58911-reef

reef: RGW: Pass and check destination zone trace when fetching object

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agoMerge pull request #52053 from trociny/wip-61432-reef
Yuri Weinstein [Fri, 23 Jun 2023 16:17:16 +0000 (12:17 -0400)]
Merge pull request #52053 from trociny/wip-61432-reef

reef: rgw: multisite data log flag not used

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agoMerge pull request #52119 from cbodley/wip-61380-reef
Yuri Weinstein [Fri, 23 Jun 2023 16:13:00 +0000 (12:13 -0400)]
Merge pull request #52119 from cbodley/wip-61380-reef

reef: Fix FTBFS on gcc 13

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoMerge pull request #52168 from zdover23/wip-doc-2023-06-23-backport-45746-to-reef 52101/head
zdover23 [Fri, 23 Jun 2023 03:26:37 +0000 (13:26 +1000)]
Merge pull request #52168 from zdover23/wip-doc-2023-06-23-backport-45746-to-reef

reef: doc,test: clean up crush rule min/max_size leftovers

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2 years agodoc,test: clean up crush rule min/max_size leftovers 52168/head
Ilya Dryomov [Mon, 28 Mar 2022 18:35:35 +0000 (20:35 +0200)]
doc,test: clean up crush rule min/max_size leftovers

These fields were removed in quincy in commit f95eb04411cd ("crush:
eliminate min_size and max_size").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 1ab943cabb7677f9465e92a00a84d2bfca60beb7)