]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agovalgrind: remove stale openssl suppressions 52574/head
Casey Bodley [Thu, 20 Jul 2023 13:23:52 +0000 (09:23 -0400)]
valgrind: remove stale openssl suppressions

years ago in commit fe97a000990632dbe9734e0d3a96df627f104f7a, the
signature of authenticated_decrypt_update_final() changed and this
suppression no longer matches its mangled form:

-  fun:_ZN4ceph6crypto6onwire25AES128GCM_OnWireRxHandler34authenticated_decrypt_update_finalEONS_6buffer7v*4listEj
+  fun:_ZN4ceph6crypto6onwire25AES128GCM_OnWireRxHandler34authenticated_decrypt_update_finalERNS_6buffer7v15_2_04listE

the fact that we didn't notice must mean that we don't need the
suppressions anymore

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d6d9986315481ca13e8823c8e26c03eeac2d1713)

2 years agovalgrind: suppress uninitialized value in OpenSSL
Casey Bodley [Wed, 19 Jul 2023 17:19:46 +0000 (13:19 -0400)]
valgrind: suppress uninitialized value in OpenSSL

Fixes: https://tracker.ceph.com/issues/61566
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ef61386862a507e993a05e23de75a88d7744f7b4)

2 years agoMerge pull request #52302 from lxbsz/wip-61797
Yuri Weinstein [Thu, 20 Jul 2023 23:12:37 +0000 (19:12 -0400)]
Merge pull request #52302 from lxbsz/wip-61797

reef: client: only wait for write MDS OPs when unmounting

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52525 from cfsnyder/wip-62065-reef
Yuri Weinstein [Thu, 20 Jul 2023 21:43:26 +0000 (17:43 -0400)]
Merge pull request #52525 from cfsnyder/wip-62065-reef

reef: rgw: fix consistency bug with OLH objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #52562 from rhcs-dashboard/reef-make-check-fix
Laura Flores [Thu, 20 Jul 2023 19:39:37 +0000 (14:39 -0500)]
Merge pull request #52562 from rhcs-dashboard/reef-make-check-fix

reef: install-deps: remove the legacy resolver flags

2 years agoinstall-deps: remove the legacy resolver flags 52562/head
Nizamudeen A [Wed, 19 Jul 2023 14:05:05 +0000 (19:35 +0530)]
install-deps: remove the legacy resolver flags

This was a workaround that was introduced long time ago. This will be
something that could be deprectaed at some point [1]. And its preventing some of the dependencies to be
downloaded or stored into the wheelhouse. Deps like jsonschema, parse,
mypy, cryptography etc.

[1] https://pip.pypa.io/en/latest/user_guide/#deprecation-timeline

Fixes: https://tracker.ceph.com/issues/62082
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit d179b95ddf57f8ca48633aed9f6229e39ad83fef)

 Conflicts:
install-deps.sh
  - discard the incoming change and edit the line manually

2 years agoMerge pull request #52469 from idryomov/wip-61587-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:58:40 +0000 (11:58 -0400)]
Merge pull request #52469 from idryomov/wip-61587-reef

reef: common: `counter dump` command revision

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2 years agoMerge pull request #52414 from ljflores/wip-61966-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:58:02 +0000 (11:58 -0400)]
Merge pull request #52414 from ljflores/wip-61966-reef

reef: qa/suites/rados/singleton/all: remove test_envlibrados_for_rocksdb

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #52399 from mchangir/wip-61959-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:53:32 +0000 (11:53 -0400)]
Merge pull request #52399 from mchangir/wip-61959-reef

reef: mon: block osd pool mksnap for fs pools

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52242 from batrick/wip-61424-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:52:37 +0000 (11:52 -0400)]
Merge pull request #52242 from batrick/wip-61424-reef

reef: mon/MDSMonitor: ignore extraneous up:boot messages

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52238 from batrick/wip-61413-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:52:10 +0000 (11:52 -0400)]
Merge pull request #52238 from batrick/wip-61413-reef

reef: mon/MDSMonitor: do not propose on error in prepare_update

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52232 from batrick/wip-61410-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:51:34 +0000 (11:51 -0400)]
Merge pull request #52232 from batrick/wip-61410-reef

reef: mon/MDSMonitor: check fscid in pending exists in current

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52231 from batrick/wip-61759-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:50:09 +0000 (11:50 -0400)]
Merge pull request #52231 from batrick/wip-61759-reef

reef: pybind/rados: keep byte representation if decode fails

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52229 from batrick/wip-61693-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:48:03 +0000 (11:48 -0400)]
Merge pull request #52229 from batrick/wip-61693-reef

reef:  mon/MDSMonitor: batch last_metadata update with pending

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52126 from joscollin/wip-61735-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:40:46 +0000 (11:40 -0400)]
Merge pull request #52126 from joscollin/wip-61735-reef

reef: mds: display sane hex value (0x0) for empty feature bit

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52073 from joscollin/wip-61694-reef
Yuri Weinstein [Wed, 19 Jul 2023 15:40:19 +0000 (11:40 -0400)]
Merge pull request #52073 from joscollin/wip-61694-reef

reef: debian: install cephfs-mirror systemd unit files and man page

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #51684 from lxbsz/wip-61347
Yuri Weinstein [Wed, 19 Jul 2023 15:39:34 +0000 (11:39 -0400)]
Merge pull request #51684 from lxbsz/wip-61347

reef: mds: wait reintegrate to finish when unlinking

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #51607 from vshankar/tr-59724
Yuri Weinstein [Wed, 19 Jul 2023 15:39:00 +0000 (11:39 -0400)]
Merge pull request #51607 from vshankar/tr-59724

reef: mds: allow unlink from lost+found directory

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #51506 from lxbsz/wip-59708
Yuri Weinstein [Wed, 19 Jul 2023 15:37:25 +0000 (11:37 -0400)]
Merge pull request #51506 from lxbsz/wip-59708

reef: mds: do not take the ino which has been used

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #51484 from vshankar/tr-59719
Yuri Weinstein [Wed, 19 Jul 2023 15:36:20 +0000 (11:36 -0400)]
Merge pull request #51484 from vshankar/tr-59719

reef: client: use deep-copy when setting permission during make_request

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agorgw: fix/improve test_rgw_versioning.py tests 52525/head
Cory Snyder [Wed, 12 Jul 2023 14:08:07 +0000 (14:08 +0000)]
rgw: fix/improve test_rgw_versioning.py tests

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit aa1f40e80d78ce08a2f51dad6fadf620866d17a3)

2 years agorgw: apply_olh_log should clear_olh before trimming olh log
Cory Snyder [Wed, 14 Jun 2023 14:01:43 +0000 (14:01 +0000)]
rgw: apply_olh_log should clear_olh before trimming olh log

When pending_removal is true and the log is trimmed prior to
clearing olh, it allows for the possibility that the olh is not
cleared (e.g. due to new pending xattrs on the olh obj), but the
olh log entries are removed.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit dfbe4f9cb078bad9a0244ce433a9728f8bd6b912)

Conflicts:
src/rgw/driver/rados/rgw_rados.cc

Cherry-pick notes:
- conflicts due to https://github.com/ceph/ceph/pull/50206 on main

2 years agorgw: cancel olh modification by removing pending xattrs in set_olh error scenarios
Cory Snyder [Wed, 31 May 2023 18:01:02 +0000 (18:01 +0000)]
rgw: cancel olh modification by removing pending xattrs in set_olh error scenarios

Makes request threads clean up after themselves and remove their pending
xattrs from OLH objects before they return an error from set_olh.

Fixes: https://tracker.ceph.com/issues/59663
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 38133e59d293dc8c16c92666967783d9e08a79cd)

Conflicts:
src/rgw/driver/rados/rgw_rados.cc
src/rgw/driver/rados/rgw_rados.h

Cherry-pick notes:
- conflicts due to https://github.com/ceph/ceph/pull/50206 on main but not reef

2 years agorgw: extract apply_olh logic for clearing olh to a method
Cory Snyder [Mon, 3 Jul 2023 19:01:47 +0000 (19:01 +0000)]
rgw: extract apply_olh logic for clearing olh to a method

Also clears stale pending entries prior to attempting to
clear OLH index entries and remove the OLH object.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 3437897deaec478f231dcad68b05fa38a7f2cb10)

Conflicts:
src/rgw/driver/rados/rgw_rados.cc
src/rgw/driver/rados/rgw_rados.h

Cherry-pick notes:
- conflicts due to https://github.com/ceph/ceph/pull/50206 on main but not reef

2 years agorgw: is_olh() predicate should use user.rgw.olh.ver xattr
Cory Snyder [Tue, 23 May 2023 09:43:38 +0000 (09:43 +0000)]
rgw: is_olh() predicate should use user.rgw.olh.ver xattr

The predicate which determines whether an object is an OLH object was previously based off of the
user.rgw.olh.info xattr, but this xattr is not set when the OLH object is first created. This
means that in cases where an error prevents that xattr from being set later, the OLH object is
incorrectly interpreted as an empty unversioned object. This change modifies the is_olh function
to use the user.rgw.olh.ver xattr. This xattr is set at the time of OLH creation and therefore
ensures consistency.

Fixes: https://tracker.ceph.com/issues/61359
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 74ab3eb94a4177eacdac4fe48528542ba355ecfe)

2 years agoqa/workunits/rgw: add test for PUT response during reshard
Cory Snyder [Tue, 23 May 2023 08:39:32 +0000 (08:39 +0000)]
qa/workunits/rgw: add test for PUT response during reshard

Tests that a PUT succeeds when the bucket is actively resharding.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 38e027a8c3019f12d108563221419bf38ed8ac5b)

2 years agoradosgw-admin: add --inject-delay-at and --inject-delay-ms
Casey Bodley [Wed, 24 May 2023 17:24:52 +0000 (13:24 -0400)]
radosgw-admin: add --inject-delay-at and --inject-delay-ms

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f815439ec59538f1d97ef27993510cace3d4fbf4)

2 years agocommon: FaultInjector supports InjectDelay
Casey Bodley [Wed, 24 May 2023 17:14:55 +0000 (13:14 -0400)]
common: FaultInjector supports InjectDelay

instead of injecting an error code or assertion failure, InjectDelay
just injects an artificial delay before returning success

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 86cb15d3a53cff6f562baa9461f79549c6b82b88)

2 years agoMerge pull request #52486 from zdover23/wip-doc-2023-07-17-backport-52483-to-reef
Anthony D'Atri [Mon, 17 Jul 2023 15:46:01 +0000 (11:46 -0400)]
Merge pull request #52486 from zdover23/wip-doc-2023-07-17-backport-52483-to-reef

reef: doc/mgr/rgw.rst: add missing "ceph" command in cli specification

2 years agodoc/mgr/rgw.rst: add missing "ceph" command in cli specification 52486/head
Ville Ojamo [Mon, 17 Jul 2023 13:05:39 +0000 (20:05 +0700)]
doc/mgr/rgw.rst: add missing "ceph" command in cli specification

The line begins with a prompt so I would
expect it to be a "complete" command,
including the `ceph` binary to run with all
the various listed arguments.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit 37ea3e50f09be5490d7ec614798822aa6f86eaec)

2 years agoMerge pull request #52478 from zdover23/wip-doc-2023-07-17-backport-52476-to-reef
zdover23 [Mon, 17 Jul 2023 07:17:58 +0000 (17:17 +1000)]
Merge pull request #52478 from zdover23/wip-doc-2023-07-17-backport-52476-to-reef

reef: doc/mgr/rgw.rst: multisite typed wrong

Reviewed-by: Rongqi Sun (Svelar) <sunrongqi@huawei.co>
2 years agodoc/mgr/rgw.rst: multisite typed wrong 52478/head
Ville Ojamo [Mon, 17 Jul 2023 03:35:46 +0000 (10:35 +0700)]
doc/mgr/rgw.rst: multisite typed wrong

Multisite is typed wrong as mutlisite so fix the spelling.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit 33b8fdb20329342ef91a391a99065fdb88c16b23)

2 years agoMerge pull request #52470 from zdover23/wip-doc-2023-07-16-backport-52465-to-reef
Anthony D'Atri [Sun, 16 Jul 2023 12:06:23 +0000 (08:06 -0400)]
Merge pull request #52470 from zdover23/wip-doc-2023-07-16-backport-52465-to-reef

reef: doc/rados: edit ceph-conf.rst (2 of x)

2 years agodoc/rados: edit ceph-conf.rst (2 of x) 52470/head
Zac Dover [Sat, 15 Jul 2023 01:09:20 +0000 (11:09 +1000)]
doc/rados: edit ceph-conf.rst (2 of x)

Edit doc/rados/configuration/ceph-conf.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 69b9b20be14c7775e434b25753678533eb163a3f)

2 years agoexporter: adopt counter dump command output 52469/head
Avan Thakkar [Mon, 12 Jun 2023 14:23:05 +0000 (19:53 +0530)]
exporter: adopt counter dump command output

Adopt the counter dump format changes in exporter for extracting the counters.
Removed the condition for `PERFCOUNTER_TIME` as counter dump already does
tranformation internally.

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 3e8ef70336a05892e6635450d98da00cfc94dacb)

2 years agocommon: `counter dump` command revision
Ali Maredia [Thu, 1 Jun 2023 20:18:26 +0000 (16:18 -0400)]
common: `counter dump` command revision

`counter dump` now emits an array of
<labels,counters> pairs for each individual key.

Commit includes revisions to perf counters unit
test.

Fixes: https://tracker.ceph.com/issues/61587
Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit 78a148869389aa8df57596698c319de807535df8)

2 years agoMerge pull request #52463 from zdover23/wip-doc-2023-07-15-backport-52356-to-reef
Anthony D'Atri [Sat, 15 Jul 2023 02:12:39 +0000 (22:12 -0400)]
Merge pull request #52463 from zdover23/wip-doc-2023-07-15-backport-52356-to-reef

reef: doc/cephfs: add note to isolate metadata pool osds

2 years agodoc/cephfs: add note to isolate metadata pool osds 52463/head
Patrick Donnelly [Fri, 7 Jul 2023 12:42:58 +0000 (08:42 -0400)]
doc/cephfs: add note to isolate metadata pool osds

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4e2e61f16438dcd5eb35854091114d18b6fe9a9e)

2 years agoMerge pull request #52387 from rhcs-dashboard/cleanup-script-reef
Nizamudeen A [Fri, 14 Jul 2023 12:06:38 +0000 (17:36 +0530)]
Merge pull request #52387 from rhcs-dashboard/cleanup-script-reef

reef: mgr/dashboard: move cephadm e2e cleanup to jenkins job config

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2 years agoMerge pull request #52448 from zdover23/wip-doc-2023-07-14-backport-52423-to-reef
Anthony D'Atri [Fri, 14 Jul 2023 11:01:38 +0000 (07:01 -0400)]
Merge pull request #52448 from zdover23/wip-doc-2023-07-14-backport-52423-to-reef

reef: doc/rados: edit ceph-conf.rst

2 years agoMerge pull request #52450 from zdover23/wip-doc-2023-07-14-backport-52402-to-reef
Anthony D'Atri [Fri, 14 Jul 2023 10:59:58 +0000 (06:59 -0400)]
Merge pull request #52450 from zdover23/wip-doc-2023-07-14-backport-52402-to-reef

reef: doc/rados: edit add-or-rm-osds (2 of x)

2 years agodoc/rados: edit add-or-rm-osds (2 of x) 52450/head
Zac Dover [Tue, 11 Jul 2023 21:58:59 +0000 (07:58 +1000)]
doc/rados: edit add-or-rm-osds (2 of x)

Edit doc/rados/operations/add-or-rm-osds.rst (2 of x).

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d83fa5352aa556c4e559e66dd461c200cc86c709)

2 years agodoc/rados: edit ceph-conf.rst 52448/head
Zac Dover [Thu, 13 Jul 2023 08:41:51 +0000 (18:41 +1000)]
doc/rados: edit ceph-conf.rst

Edit doc/rados/configuration/ceph-conf.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit dd8d91b0ded8be70bec3f40f499d7cf9432c9f23)

2 years agoMerge pull request #52425 from zdover23/wip-doc-2023-07-13-backport-52419-to-reef
Anthony D'Atri [Thu, 13 Jul 2023 13:12:31 +0000 (09:12 -0400)]
Merge pull request #52425 from zdover23/wip-doc-2023-07-13-backport-52419-to-reef

reef: docs: fix nfs cluster create syntax

2 years agodocs: fix nfs cluster create syntax 52425/head
Paul Cuzner [Thu, 13 Jul 2023 03:13:42 +0000 (15:13 +1200)]
docs: fix nfs cluster create syntax

The ceph prefix was missing from the command
example.

Signed-off-by: Paul Cuzner <pcuzner@ibm.com>
(cherry picked from commit 43b5f4b4f92341803753fcd3b67132546829e362)

2 years agoMerge pull request #52071 from guits/wip-61339-reef
Guillaume Abrioux [Thu, 13 Jul 2023 08:05:18 +0000 (10:05 +0200)]
Merge pull request #52071 from guits/wip-61339-reef

reef: ceph-volume: drop filestore support

2 years agoMerge pull request #52061 from guits/wip-61152-reef
Guillaume Abrioux [Thu, 13 Jul 2023 08:03:02 +0000 (10:03 +0200)]
Merge pull request #52061 from guits/wip-61152-reef

reef: ceph-volume: fix a bug in `get_lvm_fast_allocs()` (batch)

2 years agoPendingReleaseNotes: add a note about deleting files from lost+found directory 51607/head
Venky Shankar [Sat, 6 May 2023 14:54:28 +0000 (10:54 -0400)]
PendingReleaseNotes: add a note about deleting files from lost+found directory

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 85ce1607a36e642be320a2dea5866dd7f26c90b1)

2 years agoqa: add checks that validate removal of entries from lost+found dir
Venky Shankar [Tue, 25 Apr 2023 08:53:58 +0000 (04:53 -0400)]
qa: add checks that validate removal of entries from lost+found dir

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 0252313c8726c74f874436b5f003bb1ee934c571)

2 years agomds: allow unlink operation under lost+found directory
Venky Shankar [Wed, 19 Apr 2023 07:09:04 +0000 (03:09 -0400)]
mds: allow unlink operation under lost+found directory

Deleting entries in lost+found directory is suggested once users
are done with those entries (possibly, by copying them), however,
unlinking is disallowed by the gating the operation via -EROFS.

Remove this constraint.

Fixes: http://tracker.ceph.com/issues/59569
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 241b9396dbc6d506cef297bdb89ab07fe3a7ce02)

2 years agoMerge pull request #52344 from adk3798/reef-cephadm-fix-rgw-migration
Yuri Weinstein [Wed, 12 Jul 2023 22:35:54 +0000 (18:35 -0400)]
Merge pull request #52344 from adk3798/reef-cephadm-fix-rgw-migration

reef: mgr/cephadm: fix rgw spec migration with simple specs

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge pull request #52343 from adk3798/reef-default-image
Yuri Weinstein [Wed, 12 Jul 2023 22:30:18 +0000 (18:30 -0400)]
Merge pull request #52343 from adk3798/reef-default-image

reef: mgr/cephadm: set default image for reef

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2 years agoqa/suites/rados/singleton/all: remove test_envlibrados_for_rocksdb 52414/head
Laura Flores [Thu, 6 Jul 2023 17:14:05 +0000 (12:14 -0500)]
qa/suites/rados/singleton/all: remove test_envlibrados_for_rocksdb

In rocksdb 7.0, all envlibrados files were moved to a separate repository (ref: https://github.com/facebook/rocksdb/pull/9206).
The new repo is temporary and serves as an example before it is finalized where and who to host RADOS support.

Since this new repo is outside of the rocksdb repo and in an unceratin state, we should remove support for it in main
and Reef test suites. Quincy and below still use rocksdb 6.0, so the same does not apply.

Fixes: https://tracker.ceph.com/issues/59057
Signed-off-by: Laura Flores <lflores@redhat.com>
(cherry picked from commit 2fca433c71965ffb13d89043d72029a33b563b06)

2 years agoMerge pull request #52318 from mkogan1/wip-59739-reef
Yuri Weinstein [Wed, 12 Jul 2023 13:59:13 +0000 (09:59 -0400)]
Merge pull request #52318 from mkogan1/wip-59739-reef

reef: rgw: D3N cache objects which oid contains slash

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #52316 from mkogan1/wip-61661-reef
Yuri Weinstein [Wed, 12 Jul 2023 13:57:08 +0000 (09:57 -0400)]
Merge pull request #52316 from mkogan1/wip-61661-reef

reef: rgw: d3n: fix valgrind reported leak related to libaio ops

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agoMerge pull request #52297 from cbodley/wip-61888-reef
Yuri Weinstein [Wed, 12 Jul 2023 13:55:54 +0000 (09:55 -0400)]
Merge pull request #52297 from cbodley/wip-61888-reef

reef: rgw: rgwx-skip-decrypt also skips decompression of encrypted objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #51910 from mkogan1/wip-58692-reef
Yuri Weinstein [Wed, 12 Jul 2023 13:54:35 +0000 (09:54 -0400)]
Merge pull request #51910 from mkogan1/wip-58692-reef

reef: rgw: set bulk pool flag when creating data pool

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #50936 from ifed01/wip-ifed-report-min-alloc-size-ree
Yuri Weinstein [Wed, 12 Jul 2023 13:34:04 +0000 (09:34 -0400)]
Merge pull request #50936 from ifed01/wip-ifed-report-min-alloc-size-ree

reef: os/bluestore: report min_alloc_size through "ceph osd metadata"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoqa: add test to verify blocking of osd pool mksnap for fs pools 52399/head
Milind Changire [Wed, 26 Apr 2023 05:11:24 +0000 (10:41 +0530)]
qa: add test to verify blocking of osd pool mksnap for fs pools

Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit ab64bfaaf978cb9a14ff0520d4dc51ea32e1e93c)

2 years agomon: block osd pool mksnap for fs pools
Milind Changire [Mon, 8 May 2023 07:52:12 +0000 (13:22 +0530)]
mon: block osd pool mksnap for fs pools

Commit 23db15d5c2b disabled pool snaps for the rados mksnap path. But
ceph osd pool mksnap was an alternate way that pool snaps could be
created.
This commit disables pool snaps via this alternate path as well.

NOTE:
Pool-level snaps and fs-level snaps can't co-exist since snap IDs are
likely to clash between the two different mechanisms and can result in
unintentional data loss when either of the snaps are deleted.

Fixes: https://tracker.ceph.com/issues/59552
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit 6203fe15289d27bf02441fd413a40371cb0bc516)

2 years agoMerge pull request #52329 from soumyakoduri/wip-skoduri-reef
Yuri Weinstein [Tue, 11 Jul 2023 15:30:25 +0000 (11:30 -0400)]
Merge pull request #52329 from soumyakoduri/wip-skoduri-reef

[reef]rgw: Do not update mtime in `set_attrs`

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #52321 from batrick/wip-61899-reef
Yuri Weinstein [Tue, 11 Jul 2023 14:33:53 +0000 (10:33 -0400)]
Merge pull request #52321 from batrick/wip-61899-reef

reef: pybind: drop GIL during library callouts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoPendingReleaseNotes: note rgw's compress-encrypted zonegroup feature flag 52297/head
Casey Bodley [Mon, 3 Jul 2023 19:06:29 +0000 (15:06 -0400)]
PendingReleaseNotes: note rgw's compress-encrypted zonegroup feature flag

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fd2ec27f538bceadec412a848e2d29b187ed04bb)

2 years agodoc/rgw: document compress-encrypted zonegroup feature flag
Casey Bodley [Mon, 3 Jul 2023 18:59:51 +0000 (14:59 -0400)]
doc/rgw: document compress-encrypted zonegroup feature flag

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d85011a7fc5d3df42a2c046ec7a43fe29f1db08f)

2 years agorgw: require zonegroup feature to compress encrypted objects
Casey Bodley [Mon, 3 Jul 2023 19:00:17 +0000 (15:00 -0400)]
rgw: require zonegroup feature to compress encrypted objects

Reef adds support for combining server-side encryption and compression,
but previous releases would not replicate such objects correctly. gate
the feature behind a zonegroup feature flag to require that all zones
upgrade to reef before enabling. this feature is not enabled by default
in new zonegroups

Fixes: https://tracker.ceph.com/issues/57905
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4bc52954ab7b38a8eac50c116196d6db728cee00)

Conflicts:
src/rgw/rgw_op.cc: SiteConfig -> sal::ZoneGroup::supports

2 years agorgw/sal: add ZoneGroup::supports()
Casey Bodley [Tue, 11 Jul 2023 14:00:24 +0000 (10:00 -0400)]
rgw/sal: add ZoneGroup::supports()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agodoc/rgw: don't capitalize zonegroup feature flags
Casey Bodley [Mon, 3 Jul 2023 18:59:24 +0000 (14:59 -0400)]
doc/rgw: don't capitalize zonegroup feature flags

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 39be699629df5c6a6b82fa956ed4c22fc4a38523)

2 years agorgw: don't enable all zone features by default
Casey Bodley [Fri, 7 Jul 2023 17:00:06 +0000 (13:00 -0400)]
rgw: don't enable all zone features by default

on zonegroup creation, all `supported` features get enabled by default.
however, some features should remain opt-in. add a separate list of
`enabled` features for the subset of features we want enabled by default
on zonegroup creation

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cdb1be836cc1b8c2e8f178bc00bad24d2b8413bf)

Conflicts:
src/rgw/driver/rados/rgw_zone.cc no create_default_zonegroup()

2 years agomgr/dashboard: move cephadm e2e cleanup to jenkins job config 52387/head
Nizamudeen A [Fri, 7 Jul 2023 10:46:00 +0000 (16:16 +0530)]
mgr/dashboard: move cephadm e2e cleanup to jenkins job config

Follows: https://github.com/ceph/ceph-build/pull/2145

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a407f0b7834585c087001059d6cec613e524538b)

2 years agoMerge pull request #52383 from zdover23/wip-doc-2023-07-11-backport-52366-to-reef
Anthony D'Atri [Mon, 10 Jul 2023 23:06:28 +0000 (19:06 -0400)]
Merge pull request #52383 from zdover23/wip-doc-2023-07-11-backport-52366-to-reef

reef: doc/rados: edit add-or-rm-osds (1 of x)

2 years agodoc/rados: edit add-or-rm-osds (1 of x) 52383/head
Zac Dover [Sun, 9 Jul 2023 02:50:11 +0000 (12:50 +1000)]
doc/rados: edit add-or-rm-osds (1 of x)

Edit docs/rados/operations/add-or-rm-osds.

Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a3ab5a7ad8a44c702089294b107d09dca3a98139)

2 years agoMerge pull request #52381 from zdover23/wip-doc-2023-07-11-backport-52371-to-reef
Anthony D'Atri [Mon, 10 Jul 2023 20:22:11 +0000 (16:22 -0400)]
Merge pull request #52381 from zdover23/wip-doc-2023-07-11-backport-52371-to-reef

reef: doc/rados: introduce emdash

2 years agodoc/rados: introduce emdash 52381/head
Zac Dover [Mon, 10 Jul 2023 02:26:04 +0000 (12:26 +1000)]
doc/rados: introduce emdash

Introduce the emdash character in
docs/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a971a3a66a93ce52898f201a41a9aa13e0dd8bfc)

2 years agoMerge pull request #52333 from rhcs-dashboard/wip-61911-reef
Nizamudeen A [Mon, 10 Jul 2023 14:43:12 +0000 (20:13 +0530)]
Merge pull request #52333 from rhcs-dashboard/wip-61911-reef

reef: mgr/dashboard: fix a bug where data would plot wrongly

Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #52367 from zdover23/wip-doc-2023-07-09-backport-52327-to-reef
Anthony D'Atri [Sun, 9 Jul 2023 15:02:16 +0000 (11:02 -0400)]
Merge pull request #52367 from zdover23/wip-doc-2023-07-09-backport-52327-to-reef

reef: doc: add information on expediting MDS recovery

2 years agodoc: add information on expediting MDS recovery 52367/head
Patrick Donnelly [Thu, 6 Jul 2023 01:25:59 +0000 (21:25 -0400)]
doc: add information on expediting MDS recovery

Fixes: https://tracker.ceph.com/issues/61865
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0a15144c584d4445bd037340f053aa64b9c13f32)

2 years agoMerge pull request #52361 from zdover23/wip-doc-2023-07-08-backport-52360-to-reef
Anthony D'Atri [Fri, 7 Jul 2023 23:48:43 +0000 (19:48 -0400)]
Merge pull request #52361 from zdover23/wip-doc-2023-07-08-backport-52360-to-reef

reef: doc/radosgw: add Zonegroup policy explanation

2 years agoMerge pull request #52226 from rzarzynski/wip-59102-reef
Yuri Weinstein [Fri, 7 Jul 2023 21:05:26 +0000 (17:05 -0400)]
Merge pull request #52226 from rzarzynski/wip-59102-reef

reef: msg/async: don't abort when public addrs mismatch bind addrs

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agodoc/radosgw: add Zonegroup policy explanation 52361/head
Zac Dover [Fri, 7 Jul 2023 17:35:15 +0000 (03:35 +1000)]
doc/radosgw: add Zonegroup policy explanation

Add revised Zonegroup policy for "multi-zonegroups". This commit
includes changes that Casey Bodley made in
https://github.com/ceph/ceph/pull/52324#discussion_r1253482258 and that
I have integrated into the docs only now.

Co-authored-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c6400ed79b1233a2238342e4931edf341f1c7b9f)

2 years agoMerge pull request #52190 from ajarr/wip-61706-reef
Yuri Weinstein [Fri, 7 Jul 2023 18:45:27 +0000 (14:45 -0400)]
Merge pull request #52190 from ajarr/wip-61706-reef

reef: mgr/rbd_support: add user-friendly stderr message when module is not ready

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoMerge pull request #52162 from alimaredia/wip-61773-reef
Yuri Weinstein [Fri, 7 Jul 2023 18:44:05 +0000 (14:44 -0400)]
Merge pull request #52162 from alimaredia/wip-61773-reef

reef: Tempest tests Keystone Fix

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agoMerge pull request #52347 from zdover23/wip-doc-2023-07-07-backport-52319-to-reef
zdover23 [Thu, 6 Jul 2023 22:52:13 +0000 (08:52 +1000)]
Merge pull request #52347 from zdover23/wip-doc-2023-07-07-backport-52319-to-reef

reef: doc/foundation: Updating foundation members for July 2023

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #52348 from zdover23/wip-doc-2023-07-07-backport-52324-to-reef
Anthony D'Atri [Thu, 6 Jul 2023 21:10:51 +0000 (17:10 -0400)]
Merge pull request #52348 from zdover23/wip-doc-2023-07-07-backport-52324-to-reef

reef: doc/radosgw: add Zonegroup purpose

2 years agodoc/radosgw: add Zonegroup purpose 52348/head
Zac Dover [Wed, 5 Jul 2023 17:36:52 +0000 (03:36 +1000)]
doc/radosgw: add Zonegroup purpose

Add a note about the telos of zonegroups. The information added in this
commit was originally sent to ceph-users by Casey Bodley on 30 Jun 2023
in response to a question from Yixin Jin. Alexander Patrakov then
suggested that Casey's response should be added to
doc/radsogw/multisite.rst.

Co-authored-by: Casey Bodley <cbodley@redhat.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 5c46a1a00962918c885e434986d0f846c3ee4730)

2 years agodoc/foundation: Updating foundation members for July 2023 52347/head
Mike Perez [Wed, 5 Jul 2023 15:31:04 +0000 (08:31 -0700)]
doc/foundation: Updating foundation members for July 2023

Signed-off-by: Mike Perez <thingee@gmail.com>
(cherry picked from commit 640390906f76da4bfd66bcd8006e964d6e73c98e)

2 years agomgr/cephadm: set default image for reef 52343/head
Adam King [Thu, 6 Jul 2023 15:46:31 +0000 (11:46 -0400)]
mgr/cephadm: set default image for reef

This needs to be a v18 image instead of the
main branch ci build, otherwise those that
bootstrap without providing an image name
will get main instead of reef

Signed-off-by: Adam King <adking@redhat.com>
2 years agomgr/cephadm: fix rgw spec migration with simple specs 52344/head
Adam King [Mon, 3 Jul 2023 18:33:34 +0000 (14:33 -0400)]
mgr/cephadm: fix rgw spec migration with simple specs

The rgw spec migration code, intended to formalize
the rgw_frontend_type spec option, doesn't work with
simple specs i.e.

service_type: rgw
service_id: rgw.1
service_name: rgw.rgw.1
placement:
  label: rgw

because the migration code assumes there will always be
a "spec" section inside the spec. This is the case for
more involved rgw specs such as

service_type: rgw
service_id: foo
placement:
  label: rgw
  count_per_host: 2
spec:
  rgw_realm: myrealm
  rgw_zone: myzone
  rgw_frontend_type: "beast"
  rgw_frontend_port: 5000

which is what the migration is actually concerned about
(verification of the rgw_frontend_type in these specs).

In the case where the spec is more simple, we should
just leave the spec alone and move on. Unfortunately
the current code assumes the field will always be
there and hits an unhandled KeyError when trying to
migrate the more simple specs. This causes the
cephadm module to crash shortly after starting an
upgrade to a version that includes this migration
and it's very difficult to find the root cause. This
can be worked around by adding fields to the rgw
spec before upgrade so the "spec" field exists in
the spec and the migration works as intended.

This commit fixes the migration in the simple
case as well as adding testing for that case to
both the unit tests and orch/cephadm teuthology
upgrade tests

Fixes: https://tracker.ceph.com/issues/61889
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 1860ef83877c76c51a20aac48036bd9590572cc2)

2 years agomgr/dashboard: fix a bug where data would plot on the graphs without converting to... 52333/head
Pedro Gonzalez Gomez [Thu, 16 Mar 2023 12:09:26 +0000 (13:09 +0100)]
mgr/dashboard: fix a bug where data would plot on the graphs without converting to the data units of the graph

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 7aa70a16a54c3387b2e021763ca1d118d6785835)

2 years agorgw: Do not update mtime in `set_attrs` 52329/head
Soumya Koduri [Sat, 24 Jun 2023 06:55:46 +0000 (12:25 +0530)]
rgw: Do not update mtime in `set_attrs`

As per https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html,
the only way for users to modify object metadata is to make a copy of the object and
set the metadata. Hence retain old mtime for any other attr changes

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 31666d71aa5b58a9384938ba31892d48f2ac3f67)

2 years agopybind/rbd: drop GIL when calling into librbd 52321/head
Ilya Dryomov [Mon, 3 Jul 2023 11:34:30 +0000 (13:34 +0200)]
pybind/rbd: drop GIL when calling into librbd

This was missing for rbd_mirror_peer_site_add() and
rbd_get_data_pool_id().

While at it, add a test for data_pool_id().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 1a60f66b0a13089c789e2e056b925ac6fef75021)

2 years agopybind/cephfs: drop gil during cephfs callouts
Patrick Donnelly [Sun, 2 Jul 2023 16:05:08 +0000 (12:05 -0400)]
pybind/cephfs: drop gil during cephfs callouts

This has disastorous consequences including the possibility of deadlock.
In the best case, you have the rmdir holding the GIL until the MDS
responds!

Fixes: https://tracker.ceph.com/issues/61869
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5d249a4c978198f1e8974affac0297b71bebada5)

2 years agoMerge pull request #52291 from zdover23/wip-doc-2023-07-03-backport-52280-to-reef... 52281/head
zdover23 [Wed, 5 Jul 2023 16:47:03 +0000 (02:47 +1000)]
Merge pull request #52291 from zdover23/wip-doc-2023-07-03-backport-52280-to-reef-2nd-attempt

reef: doc/radosgw: refine "Zones" in multisite.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/radosgw: refine "Zones" in multisite.rst 52291/head
Zac Dover [Sat, 1 Jul 2023 11:51:15 +0000 (21:51 +1000)]
doc/radosgw: refine "Zones" in multisite.rst

Describe in greater detail the function of Zones in serving S3 objects,
as requested by Alexander Patrakov here:
https://github.com/ceph/ceph/pull/49981#issuecomment-1615866374.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8dab6c066c4dd3699c7b7c80f7a30a0e18326bba)

2 years agoreef: rgw: D3N cache objects which oid contains slash 52318/head
Mark Kogan [Mon, 29 May 2023 17:36:56 +0000 (17:36 +0000)]
reef: rgw: D3N cache objects which oid contains slash

Fixes: https://tracker.ceph.com/issues/59739
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit fcb91f3e2a6fd8066ac33950d421e0a9b9b4b490)

 Conflicts:
src/rgw/driver/rados/rgw_d3n_datacache.cc
src/rgw/rgw_aio.cc
src/rgw/rgw_d3n_cacherequest.h

2 years agoreef: rgw: d3n: fix valgrind reported leak related to libaio ops 52316/head
Mark Kogan [Wed, 28 Jun 2023 14:06:27 +0000 (14:06 +0000)]
reef: rgw: d3n: fix valgrind reported leak related to libaio ops

Fixes: https://tracker.ceph.com/issues/61661
The valgrind leak indication is a false positive in this case,
it is because the libaio internal thread have not timed out yet
when radosgw is terminated.

```
man aio_init
...
aio_idle_time
This field specifies the amount of time in seconds that a worker thread
should wait for further requests before terminating, after having
completed a previous request.   The
default value is 1.
...
```
for the sake of teuthology reducing the timeout

waiting for 2 minutes for example like below would also prevent the leak report
```
❯ env
LD_LIBRARY_PATH=/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/:$LD_LIBRARY_PATH
PYTHONPATH=$PYTHONPATH:/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/cython_modules/lib.3
RAGWEED_CONF=$(realpath ./ragweed.conf) RAGWEED_STAGES=prepare,check tox
-- -v |& ccze -Aonolookups ;   sleep 2m | pv -t ; pkill radosgw
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit f834e10cfe97c7df8fc5ead682975fe60b435bd7)

2 years agoMerge pull request #52298 from zdover23/wip-doc-2023-07-04-backport-51422-to-reef
zdover23 [Tue, 4 Jul 2023 22:11:55 +0000 (08:11 +1000)]
Merge pull request #52298 from zdover23/wip-doc-2023-07-04-backport-51422-to-reef

reef: doc/rgw/lua: add info uploading a script in cephadm deployment

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rgw/lua: add info uploading a script in cephadm deployment 52298/head
Yuval Lifshitz [Wed, 10 May 2023 09:43:45 +0000 (12:43 +0300)]
doc/rgw/lua: add info uploading a script in cephadm deployment

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit 5cec4f719581e93760c22419e701c563a8688de9)

2 years agoMerge pull request #52311 from zdover23/wip-doc-2023-07-05-backport-52207-to-reef
zdover23 [Tue, 4 Jul 2023 19:35:58 +0000 (05:35 +1000)]
Merge pull request #52311 from zdover23/wip-doc-2023-07-05-backport-52207-to-reef

reef: doc/rados: edit crush-map-edits (2 of x)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2 years agodoc/rados: edit crush-map-edits (2 of x) 52311/head
Zac Dover [Tue, 27 Jun 2023 08:58:32 +0000 (18:58 +1000)]
doc/rados: edit crush-map-edits (2 of x)

Edit doc/rados/operations/crush-map-edits.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit ae04debe9e2fa18c1cb8745b0573ff56576fd3b0)

2 years agoMerge pull request #52309 from zdover23/wip-doc-2023-07-05-backport-52308-to-reef
zdover23 [Tue, 4 Jul 2023 18:25:35 +0000 (04:25 +1000)]
Merge pull request #52309 from zdover23/wip-doc-2023-07-05-backport-52308-to-reef

reef: doc/dev: remove seqdiag assets

Reviewed-by: Josh Durgin <jdurgin@redhat.com>