]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Casey Bodley [Mon, 3 Jul 2023 19:06:29 +0000 (15:06 -0400)]
PendingReleaseNotes: note rgw's compress-encrypted zonegroup feature flag
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 3 Jul 2023 18:59:51 +0000 (14:59 -0400)]
doc/rgw: document compress-encrypted zonegroup feature flag
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 3 Jul 2023 19:00:17 +0000 (15:00 -0400)]
rgw: require zonegroup feature to compress encrypted objects
Reef adds support for combining server-side encryption and compression,
but previous releases would not replicate such objects correctly. gate
the feature behind a zonegroup feature flag to require that all zones
upgrade to reef before enabling. this feature is not enabled by default
in new zonegroups
Fixes: https://tracker.ceph.com/issues/57905
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 3 Jul 2023 18:59:24 +0000 (14:59 -0400)]
doc/rgw: don't capitalize zonegroup feature flags
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 7 Jul 2023 17:00:06 +0000 (13:00 -0400)]
rgw: don't enable all zone features by default
on zonegroup creation, all `supported` features get enabled by default.
however, some features should remain opt-in. add a separate list of
`enabled` features for the subset of features we want enabled by default
on zonegroup creation
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Adam King [Fri, 7 Jul 2023 15:11:40 +0000 (11:11 -0400)]
Merge pull request #51863 from phlogistonjohn/jjm-cephadm-argument-spec
cephadm: add a new ArgumentSpec type for advanced argument handling
Reviewed-by: Adam King <adking@redhat.com>
Anthony D'Atri [Fri, 7 Jul 2023 13:04:03 +0000 (09:04 -0400)]
Merge pull request #52353 from bluikko/patch-12
doc/man/radosgw: use underscores in configuration var names
Ville Ojamo [Fri, 7 Jul 2023 10:28:08 +0000 (17:28 +0700)]
doc/man/radosgw: more underscores in config var names
Missed one section of `ceph.conf` example where spaces were used instead of underscores.
Change spaces in config var names to underscores.
Ville Ojamo [Fri, 7 Jul 2023 09:49:21 +0000 (16:49 +0700)]
doc/man/radosgw: use underscores in configuration
Most of the documentation already uses underscores instead of spaces in configuration variable names.
Update radosgw(8) man page to follow, changing spaces into underscores.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Ronen Friedman [Fri, 7 Jul 2023 05:10:07 +0000 (08:10 +0300)]
Merge pull request #52337 from ronen-fr/wip-rf-osdtypes-const
osd/osd_types_fmt: mark fmt::formatter<>::format() const
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Yingxin [Fri, 7 Jul 2023 03:36:19 +0000 (11:36 +0800)]
Merge pull request #51179 from aisakaki/wip-lazy-read-r2
crimson/os/seastore: realize lazy read in split overwrite with overwrite refactor
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Zhang Song <zhangsong325@gmail.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Yingxin [Fri, 7 Jul 2023 01:13:15 +0000 (09:13 +0800)]
Merge pull request #51770 from myoungwon/wip-multicore-rbm
crimson/os/seastore/rbm: make rbm support multiple shards
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
zdover23 [Thu, 6 Jul 2023 20:32:52 +0000 (06:32 +1000)]
Merge pull request #52324 from zdover23/wip-doc-2023-07-06-radosgw-multisite
doc/radosgw: add Zonegroup purpose
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
zdover23 [Thu, 6 Jul 2023 20:26:17 +0000 (06:26 +1000)]
Merge pull request #52319 from Thingee/foundation-mem-update-202307
doc/foundation: Updating foundation members for July 2023
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Wed, 5 Jul 2023 17:36:52 +0000 (03:36 +1000)]
doc/radosgw: add Zonegroup purpose
Add a note about the telos of zonegroups. The information added in this
commit was originally sent to ceph-users by Casey Bodley on 30 Jun 2023
in response to a question from Yixin Jin. Alexander Patrakov then
suggested that Casey's response should be added to
doc/radsogw/multisite.rst.
Co-authored-by: Casey Bodley <cbodley@redhat.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
Ali Masarwa [Thu, 6 Jul 2023 15:10:23 +0000 (18:10 +0300)]
Merge pull request #52087 from AliMasarweh/wip-alimasa-persistant-q-observability
RGW: Add observability over the persistent topics queue
Casey Bodley [Thu, 6 Jul 2023 13:59:31 +0000 (09:59 -0400)]
Merge pull request #52216 from cbodley/wip-61838
qa/s3tests: filter on 'sts_tests' and 'webidentity_tests'
Reviewed-by: Ali Maredia <amaredia@redhat.com>
Matan [Thu, 6 Jul 2023 12:42:05 +0000 (15:42 +0300)]
Merge pull request #50458 from Matan-B/wip-matanb-c-perf-basic
qa/suites/crimson-rados: Introduce performance testing
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Mark Nelson <mark.nelson@clyso.com>
Ronen Friedman [Thu, 6 Jul 2023 11:18:55 +0000 (06:18 -0500)]
osd/osd_types_fmt: mark fmt::formatter<>::format() const
For some formatters missed or added after PR#47830 (that fixed
most of the missing 'const's in this file)
Required to allow the use of the formatters in const member functions.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ali Masarwa [Tue, 6 Jun 2023 13:20:40 +0000 (16:20 +0300)]
RGW: Add observability over the persistent topics queue
Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
Ronen Friedman [Thu, 6 Jul 2023 06:28:41 +0000 (09:28 +0300)]
Merge pull request #52293 from ronen-fr/wip-rf-consttime
common: make some common ctors and accessors constexpr
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
myoungwon oh [Sat, 17 Jun 2023 08:27:17 +0000 (08:27 +0000)]
src/test/crimson/seastore/nvmedevice: update a test case based on rbm multi shard change
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
myoungwon oh [Fri, 26 May 2023 08:29:50 +0000 (08:29 +0000)]
crimson/os/seastore/rbm: make rbm support multiple shards
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Patrick Donnelly [Wed, 5 Jul 2023 17:10:13 +0000 (13:10 -0400)]
Merge PR #52290 into main
* refs/pull/52290/head:
pybind/rbd: drop GIL when calling into librbd
pybind/cephfs: drop gil during cephfs callouts
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Mike Perez [Wed, 5 Jul 2023 15:31:04 +0000 (08:31 -0700)]
doc/foundation: Updating foundation members for July 2023
Signed-off-by: Mike Perez <thingee@gmail.com>
Pedro Gonzalez Gomez [Wed, 5 Jul 2023 10:29:20 +0000 (12:29 +0200)]
Merge pull request #50558 from rhcs-dashboard/fix-dataUnits-graph-plotting
mgr/dashboard: fix a bug where data would plot wrongly
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Ronen Friedman [Sun, 2 Jul 2023 17:24:47 +0000 (12:24 -0500)]
common: make some common ctors and accessors constexpr
Add the constexpr qualifier to:
- some utime_t ctors and accessors;
- a shard_id_t ctor;
- ceph_le operator
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Matan [Wed, 5 Jul 2023 08:33:31 +0000 (11:33 +0300)]
Merge pull request #51852 from Matan-B/wip-matanb-crimson-watch-notify-logs
crimson/osd/watch: Add logs around Watch/Notify
Reviewed-by: Samuel Just <sjust@redhat.com>
Ilya Dryomov [Tue, 4 Jul 2023 22:22:53 +0000 (00:22 +0200)]
Merge pull request #52086 from pkalever/fix-mirror-status-desc
rbd-mirror: fix image replayer shut down description on force promote
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
zdover23 [Tue, 4 Jul 2023 19:22:33 +0000 (05:22 +1000)]
Merge pull request #52207 from zdover23/wip-doc-2023-06-27-rados-operations-crush-map-edits-2-of-x-2nd-attempt
doc/rados: edit crush-map-edits (2 of x)
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Tue, 27 Jun 2023 08:58:32 +0000 (18:58 +1000)]
doc/rados: edit crush-map-edits (2 of x)
Edit doc/rados/operations/crush-map-edits.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
zdover23 [Tue, 4 Jul 2023 17:55:15 +0000 (03:55 +1000)]
Merge pull request #52308 from zdover23/wip-doc-2023-07-05-seqdiag-test
doc/dev: remove seqdiag assets
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Zac Dover [Tue, 4 Jul 2023 17:37:10 +0000 (03:37 +1000)]
doc/dev: remove seqdiag assets
Remove seqdiag assets to determine whether the docs can be built if they
are absent. (Currently they cannot be built when they are present.) If
this works, then these diagrams will be replaced, probably with .png
files.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Matan Breizman [Thu, 9 Mar 2023 15:09:12 +0000 (15:09 +0000)]
qa/tasks/cbt.py: Install collectl manually
See:
d4a04809fd7fd8aaf447005f76a1090db99d75c5
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Matan Breizman [Thu, 9 Mar 2023 13:42:54 +0000 (13:42 +0000)]
qa/suites/crimson-rados: Introduce performance testing
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Xinyu Huang [Fri, 26 May 2023 11:05:08 +0000 (11:05 +0000)]
test/crimson/seastore: test if lazy read work correctly
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Xinyu Huang [Fri, 21 Apr 2023 14:22:16 +0000 (14:22 +0000)]
test/crimson/seastore: test overwrite pin
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Xinyu Huang [Fri, 21 Apr 2023 14:19:34 +0000 (14:19 +0000)]
test/crimson/seastore: add test for TransactionManager::remap_pin
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Xinyu Huang [Fri, 21 Apr 2023 14:42:00 +0000 (14:42 +0000)]
crimson/os/seastore: check if length exist in mark_space_used and mark_space_free
to avoid misuse of buffer length and data length when there is a no actual data extent
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Xinyu Huang [Wed, 19 Apr 2023 01:46:31 +0000 (01:46 +0000)]
crimson/os/seastore: deepcopy the buffer of exist clean extent when duplicate_for_write
Beacuse exist clean extent shares buffer with original clean extent.
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Xinyu Huang [Fri, 7 Apr 2023 00:54:36 +0000 (00:54 +0000)]
crimson/os/seastore: realize lazy read in split overwrite with overwrite refactor
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Xinyu Huang [Fri, 21 Apr 2023 15:52:24 +0000 (15:52 +0000)]
crimson/os/seastore/transaction_manager: implement remap_pin
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Xinyu Huang [Fri, 21 Apr 2023 15:10:49 +0000 (15:10 +0000)]
crimson/os/seastore/transaction_manager: implement alloc_remapped_extent
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Xinyu Huang [Sat, 10 Jun 2023 02:16:29 +0000 (10:16 +0800)]
crimson/os/seastore: support no data extent in read path
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
zdover23 [Mon, 3 Jul 2023 17:47:40 +0000 (03:47 +1000)]
Merge pull request #51422 from yuvalif/wip-yuval-cephadm-lua-doc
doc/rgw/lua: add info uploading a script in cephadm deployment
Reviewed-by: Zac Dover <zac.dover@proton.me>
Ilya Dryomov [Mon, 3 Jul 2023 11:34:30 +0000 (13:34 +0200)]
pybind/rbd: drop GIL when calling into librbd
This was missing for rbd_mirror_peer_site_add() and
rbd_get_data_pool_id().
While at it, add a test for data_pool_id().
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Casey Bodley [Mon, 3 Jul 2023 15:19:36 +0000 (11:19 -0400)]
Merge pull request #52247 from cbodley/wip-57905
rgw: rgwx-skip-decrypt also skips decompression of encrypted objects
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Mark Kogan [Mon, 3 Jul 2023 14:06:21 +0000 (17:06 +0300)]
Merge pull request #52249 from mkogan1/wip-t61661-d3n-valg
rgw: d3n: fix valgrind reported leak related to libaio ops
Xinyu Huang [Tue, 21 Feb 2023 06:18:29 +0000 (06:18 +0000)]
crimson/os/seastore: let CachedExtent::ptr optional
separate buffer length and data length, since no buffer extent is allowed
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
Soumya Koduri [Mon, 3 Jul 2023 04:50:48 +0000 (10:20 +0530)]
Merge pull request #52160 from soumyakoduri/wip-skoduri-lc-acl
rgw/set_attrs: Do not modify mtime
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
J. Eric Ivancich [Sun, 2 Jul 2023 20:30:26 +0000 (16:30 -0400)]
Merge pull request #51983 from clwluvw/aws4-date-header
rgw: pick http_date in case of http_x_amz_date absence
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Patrick Donnelly [Sun, 2 Jul 2023 16:05:08 +0000 (12:05 -0400)]
pybind/cephfs: drop gil during cephfs callouts
This has disastorous consequences including the possibility of deadlock.
In the best case, you have the rmdir holding the GIL until the MDS
responds!
Fixes: https://tracker.ceph.com/issues/61869
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Matan Breizman [Wed, 31 May 2023 11:47:45 +0000 (11:47 +0000)]
crimson/osd/watch: don't print bufferlists
The bufferlists clutter the logs and don't infrom of anything useful:
```
DEBUG 2023-05-26 23:36:29,780 [shard 2] osd - send_completion sending notify replies: {notify_reply_t{watcher_gid=4220, watcher_cookie=
139786421930752 , bl=buffer::list(len=0,
)}, notify_reply_t{watcher_gid=4220, watcher_cookie=
139786422032096 , bl=buffer::list(len=0,
)}}
```
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Matan Breizman [Wed, 31 May 2023 08:45:53 +0000 (08:45 +0000)]
crimson/osd/watch: Add logs around Watch/Notify
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Matan [Sun, 2 Jul 2023 11:02:01 +0000 (14:02 +0300)]
Merge pull request #49044 from rzarzynski/wip-crimson-ertr-plain-rets
crimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Matan [Sun, 2 Jul 2023 10:53:25 +0000 (13:53 +0300)]
Merge pull request #52032 from Matan-B/wip-matanb-crimson-no-ignore
crimson/osd: don't ignore start_pg_operation returned future
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan [Sun, 2 Jul 2023 10:15:02 +0000 (13:15 +0300)]
Merge pull request #51147 from liu-chunmei/multicore-osd
Crimson/osd: support multicore osd
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan [Sun, 2 Jul 2023 10:05:04 +0000 (13:05 +0300)]
Merge pull request #51916 from cyx1231st/wip-seastar-msgr-multi-core-2
crimson/net: support connections in multiple shards
Reviewed-by: Samuel Just <sjust@redhat.com>
Anthony D'Atri [Sat, 1 Jul 2023 13:46:19 +0000 (09:46 -0400)]
Merge pull request #52280 from zdover23/wip-doc-2023-07-01-radosgw-multisite
doc/radosgw: refine "Zones" in multisite.rst
Zac Dover [Sat, 1 Jul 2023 11:51:15 +0000 (21:51 +1000)]
doc/radosgw: refine "Zones" in multisite.rst
Describe in greater detail the function of Zones in serving S3 objects,
as requested by Alexander Patrakov here:
https://github.com/ceph/ceph/pull/49981#issuecomment-
1615866374 .
Signed-off-by: Zac Dover <zac.dover@proton.me>
zdover23 [Sat, 1 Jul 2023 11:29:24 +0000 (21:29 +1000)]
Merge pull request #52277 from zdover23/wip-doc-2023-07-01-radosgw-s3select-overview
doc/radosgw: s/s3select/S3 Select/
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Sat, 1 Jul 2023 11:14:30 +0000 (21:14 +1000)]
doc/radosgw: s/s3select/S3 Select/
Correct several misspelling of "S3 Select". Hat tip to Anthony D'Atri,
who caught this in an earlier PR.
Signed-off-by: Zac Dover <zac.dover@proton.me>
J. Eric Ivancich [Fri, 30 Jun 2023 15:33:15 +0000 (11:33 -0400)]
Merge pull request #47595 from 5cs/fix-lc-head-marker
rgwlc: fix lc head marker point to non-exist lc entry
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
J. Eric Ivancich [Fri, 30 Jun 2023 15:30:27 +0000 (11:30 -0400)]
Merge pull request #50667 from ivancich/wip-add-reupload-testing
qa/rgw: test that multipart re-upload does not leave any orphans
Reviewed-by: Cory Snyder <csnyder@iland.com>
Soumya Koduri [Fri, 30 Jun 2023 12:14:02 +0000 (17:44 +0530)]
Merge pull request #52252 from soumyakoduri/wip-skoduri-dbstore
rgw/dbstore: Update README
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yingxin [Fri, 30 Jun 2023 02:32:24 +0000 (10:32 +0800)]
Merge pull request #52217 from zhscn/fix-file-local-variable
crimson/os/seastore/omap_manager: correct editor settings
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
John Mulligan [Mon, 29 May 2023 20:54:20 +0000 (16:54 -0400)]
xxx: DELETEME: hack to find invalid type assignments
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 26 Jun 2023 16:08:34 +0000 (12:08 -0400)]
docs/cephadm: document new extra_{container,entrypoint}_args behavior
Document the new behaviors of extra_container_args and
extra_entrypoint_args. Document that current (previous in code terms)
behavior of splitting strings on spaces and document the recently added
support for ArgumentSpec objects and how they are used.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 29 May 2023 20:52:31 +0000 (16:52 -0400)]
mgr/cephadm: rework how _process_ls_output builds a DaemonDescription
Previously, the _process_ls_output function would create an
DaemonDescription and then assign values to attributes one-by-one. This
prevents the class from doing any data type/value conversions unless we
resort to more complex approaches like descriptors or `__setattr__` for
example. Update the _process_ls_output function to build a
DaemonDescription using keyword args so that we have the opportunity
later to convert data passed to it in `__init__`.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 25 May 2023 23:30:11 +0000 (19:30 -0400)]
mgr/cephadm: update package to use ArgumentSpec based types
Update the cephadm mgr module to match the changes to use
ArgumentSpec based types for extra_container_args and extra_entrypoint
args.
Update orchestrator/_interface.py to match the changes to use
ArgumentSpec based types for extra_container_args and extra_entrypoint args.
The job of splitting up arguments is no longer done here but
lies in a method of the ArgumentSpec type.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 25 May 2023 21:46:58 +0000 (17:46 -0400)]
python-common: extend arguments test to assert splitting behavior
Use the existing arguments test to verify that the ArgumentSpec
objects split the arguments as intended.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 17 May 2023 20:57:24 +0000 (16:57 -0400)]
python-common: extend args handling test for new ArgumentSpec type
Add cases to test_extra_args_handling test to verify new behaviors
brought in by the switch to ArgumentSpec and ArgumentList types.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 17 May 2023 17:50:19 +0000 (13:50 -0400)]
python-common: replace types for extra_*_args with ArgSpec based types
Use the ArgSpec, ArgumentList, and GeneralArgList types throughout
the deployment module to support extra_container_args and
extra_entrypoint_args. The GeneralArgList type supports all possible
input forms while ArgumentList reflects the internal representation
of the argument lists.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
zdover23 [Thu, 29 Jun 2023 16:00:13 +0000 (02:00 +1000)]
Merge pull request #52250 from zdover23/wip-doc-2023-06-29-radosgw-s3select-basic-workflow
doc/radosgw: edit "Basic Workflow" in s3select.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Thu, 29 Jun 2023 08:48:00 +0000 (18:48 +1000)]
doc/radosgw: edit "Basic Workflow" in s3select.rst
Edit the "Basic Workflow" section in doc/radosgw/s3select.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
Soumya Koduri [Thu, 29 Jun 2023 10:30:43 +0000 (16:00 +0530)]
rgw/dbstore: Update README
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Mark Kogan [Wed, 28 Jun 2023 14:06:27 +0000 (14:06 +0000)]
rgw: d3n: fix valgrind reported leak related to libaio ops
Fixes: https://tracker.ceph.com/issues/61661
The valgrind leak indication is a false positive in this case,
it is because the libaio internal thread have not timed out yet
when radosgw is terminated.
```
man aio_init
...
aio_idle_time
This field specifies the amount of time in seconds that a worker thread
should wait for further requests before terminating, after having
completed a previous request. The
default value is 1.
...
```
for the sake of teuthology reducing the timeout
waiting for 2 minutes for example like below would also prevent the leak report
```
❯ env
LD_LIBRARY_PATH=/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/:$LD_LIBRARY_PATH
PYTHONPATH=$PYTHONPATH:/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/cython_modules/lib.3
RAGWEED_CONF=$(realpath ./ragweed.conf) RAGWEED_STAGES=prepare,check tox
-- -v |& ccze -Aonolookups ; sleep 2m | pv -t ; pkill radosgw
```
Signed-off-by: Mark Kogan <mkogan@redhat.com>
Radoslaw Zarzynski [Thu, 29 Jun 2023 07:32:03 +0000 (09:32 +0200)]
Merge pull request #52008 from rishabh-d-dave/improvements-authmon
mon: improvements and cleanup for AuthMonitor
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Yingxin Cheng [Thu, 29 Jun 2023 05:25:28 +0000 (13:25 +0800)]
crimson/osd/heartbeat: relax the order of replacement reset and accept
With the new implementation in messenger, the order of replacement reset
and accept events cannot be determined because they are from different
connections.
Modify the heatbeat logic to tolerate the both cases.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
John Mulligan [Wed, 17 May 2023 17:49:23 +0000 (13:49 -0400)]
python-common: add an ArgumentSpec type for handling args
Add a new ArgumentSpec type to the deployment package. The
ArgumentSpec serves a dual purpose:
* to allow more advanced configuration of extra arguments
* to keep backwards compatibility with string-based args
The previous versions of cephadm supported supplying extra container and
entrypoint arguments for fine-tuning of services and creating custom
containers. However, this mode assumed that spaces in an argument
always meant that the argument should be split into two parts:
"--foo bar" becomes `["--foo", "bar"]`. In some cases there's a good
reason to keep spaces as in "--title=My Little Cluster". When
an argument is expressed as a single string the ArgumentSpec is
designed to retain the existing behavior. When an argument is
expressed as a JSON object then you can explicitly express if
you want the argument split on spaces or not (not split is the default).
The alternative was to keep using strings but add some level of
shell-style quoting. This was discussed but deemed complex and
difficult to read in YAML. Round tripping that data is also challenging.
The JSON object approach also allows for future fields to be added
providing for possible extensibility.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Casey Bodley [Wed, 28 Jun 2023 18:57:49 +0000 (14:57 -0400)]
rgw: fetch_remote_obj() preserves RGW_ATTR_COMPRESSION of encrypted objects
if the source object was both compressed and encrypted, preserve its
original compression attribute so it can be decompressed the same way it
was originally compressed
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 28 Jun 2023 18:43:55 +0000 (14:43 -0400)]
rgw: fetch_remote_obj() will never verify etags of encrypted objects
fetch_remote_obj() transfers objects in their encrypted form, so does
not have access to the decrypted data for checksum verification
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 28 Jun 2023 18:42:16 +0000 (14:42 -0400)]
rgw: rgwx-skip-decrypt also skips decompression of encrypted objects
compression is applied before encryption. so if we skip decryption, we
can't decompress either
Fixes: https://tracker.ceph.com/issues/57905
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Ilya Dryomov [Wed, 28 Jun 2023 09:45:50 +0000 (11:45 +0200)]
Merge pull request #52157 from pkalever/log-improve
rbd-mirror: add some assorted logs
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Soumya Koduri [Sat, 24 Jun 2023 06:55:46 +0000 (12:25 +0530)]
rgw: Do not update mtime in `set_attrs`
As per https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html,
the only way for users to modify object metadata is to make a copy of the object and
set the metadata. Hence retain old mtime for any other attr changes
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
zdover23 [Wed, 28 Jun 2023 06:43:08 +0000 (16:43 +1000)]
Merge pull request #52194 from zdover23/wip-doc-2023-06-26-radosgw-s3select-overview
doc/radosgw: edit "Overview" in s3select.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Mon, 26 Jun 2023 11:45:43 +0000 (21:45 +1000)]
doc/radosgw: edit "Overview" in s3select.rst
Edit the "Overview" section in doc/radosgw/s3select.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
Venky Shankar [Wed, 28 Jun 2023 04:53:54 +0000 (10:23 +0530)]
Merge PR #49971 into main
* refs/pull/49971/head:
doc/cephfs: document MDS_CLIENTS_LAGGY health warning
qa: ignore warnings
qa: add test cases to check client eviction if an OSD is laggy
mds,messages: enable beacon to report clients lagginess
mds: do not evict client on laggy osds
common: add new config option to defer client eviction
osd: add method to check for laggy osds
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yingxin [Wed, 28 Jun 2023 04:53:06 +0000 (12:53 +0800)]
Merge pull request #52090 from xxhdx1985126/wip-fixedkv-btree-merge-fix
crimson/os/seastore/btree: should add left's size when merging levels…
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Prasanna Kumar Kalever [Thu, 22 Jun 2023 14:38:36 +0000 (20:08 +0530)]
rbd-mirror: add some logs
Added some logs as their values where not very clear while parsing though the
log files.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Zhang Song [Wed, 28 Jun 2023 02:01:21 +0000 (10:01 +0800)]
crimson/os/seastore/omap_manager: correct editor settings
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
Casey Bodley [Tue, 27 Jun 2023 21:02:34 +0000 (17:02 -0400)]
qa/rgw/sts: rename tasks, don't duplicate s3tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 27 Jun 2023 20:55:23 +0000 (16:55 -0400)]
qa/rgw/sts: move overrides into overrides.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Tue, 27 Jun 2023 20:45:06 +0000 (16:45 -0400)]
qa/s3tests: filter on 'sts_tests' and 'webidentity_tests'
qa/tasks/s3tests.py was adding `not test_of_sts` and `not
webidentity_test` by default, and the rgw/sts suites were adding
those attrs back in `extra_attrs`
when `extra_attrs` was changed to be additive, this started causing
InvocationError failures
instead of using `extra_attrs` to control these filters,
qa/tasks/s3tests.py now uses the `sts_tests` and `webidentity_tests`
flags from the sts yaml files to control whether or not the `not
test_of_sts`/`not webidentity_test` attrs are added to the pytest
command line
Fixes: https://tracker.ceph.com/issues/61838
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Patrick Donnelly [Tue, 27 Jun 2023 17:43:38 +0000 (13:43 -0400)]
Merge PR #51858 into main
* refs/pull/51858/head:
pybind/mgr/devicehealth: do not crash if db not ready
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
Venky Shankar [Tue, 27 Jun 2023 12:21:31 +0000 (17:51 +0530)]
Merge PR #52170 into main
* refs/pull/52170/head:
qa: assign file system affinity for replaced MDS
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Aashish Sharma [Tue, 27 Jun 2023 11:58:12 +0000 (17:28 +0530)]
Merge pull request #51283 from rhcs-dashboard/disable-delete
mgr/dashboard: disable delete on multisite
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Prasanna Kumar Kalever [Thu, 15 Jun 2023 14:10:16 +0000 (19:40 +0530)]
rbd-mirror: fix image replayer shut down description on force promote
On force promote if the opposite site is down then we currently show
image status description as "local image linked to unknown peer"
Previously:
----------
$ rbd --cluster=site-b mirror image status pool1/img1
img1:
global_id:
a73341a6 -8302-4c97-ac6e-
278083fd347e
state: up+stopping_replay
description: local image linked to unknown peer
service: admin on localhost.localdomain
last_update: 2023-06-15 19:47:45
peer_sites:
name: site-a
state: up+stopped
description: local image is primary
last_update: 2023-06-15 19:47:32
snapshots:
9 .mirror.primary.
a73341a6 -8302-4c97-ac6e-
278083fd347e .
1f101367 -277f-42f0-8308-
e51201d0529a (peer_uuids:[
c46c6d97 -f59b-4591-9d35-
d7ff9d0d72f7 ])
Currently:
---------
$ rbd --cluster=site-b mirror image status pool1/img1
img1:
global_id:
2a6d61e1 -8e76-42c4-af76-
8f61ce65c7e2
state: up+stopped
description: orphan (force promoting)
service: admin on localhost.localdomain
last_update: 2023-06-15 19:29:22
peer_sites:
name: site-a
state: down+stopped
description: local image is primary
last_update: 2023-06-15 19:29:05
snapshots:
9 .mirror.primary.
2a6d61e1 -8e76-42c4-af76-
8f61ce65c7e2 .
99f82a30 -0241-4e51-8428-
7a2376d137f6 (peer_uuids:[
3150c6ef -aeee-45dc-8d0e-
5dc5a53d88eb ])
Fixes: https://tracker.ceph.com/issues/52913
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Venky Shankar [Thu, 22 Jun 2023 10:08:44 +0000 (06:08 -0400)]
qa: assign file system affinity for replaced MDS
Otherwise, the MDS that just got replaced can transition to a rank
for another file system and the test cannot deterministically infer
which MDS needs to checked.
Fixes: http://tracker.ceph.com/issues/61764
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Yingxin Cheng [Tue, 27 Jun 2023 02:31:11 +0000 (10:31 +0800)]
crimson/net: cleanup, rename is_fixed_cpu
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>