]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agoMerge pull request #52353 from bluikko/patch-12
Anthony D'Atri [Fri, 7 Jul 2023 13:04:03 +0000 (09:04 -0400)]
Merge pull request #52353 from bluikko/patch-12

doc/man/radosgw: use underscores in configuration var names

2 years agodoc/man/radosgw: more underscores in config var names 52353/head
Ville Ojamo [Fri, 7 Jul 2023 10:28:08 +0000 (17:28 +0700)]
doc/man/radosgw: more underscores in config var names

Missed one section of `ceph.conf` example where spaces were used instead of underscores.

Change spaces in config var names to underscores.

2 years agodoc/man/radosgw: use underscores in configuration
Ville Ojamo [Fri, 7 Jul 2023 09:49:21 +0000 (16:49 +0700)]
doc/man/radosgw: use underscores in configuration

Most of the documentation already uses underscores instead of spaces in configuration variable names.

Update radosgw(8) man page to follow, changing spaces into underscores.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
2 years agoMerge pull request #52337 from ronen-fr/wip-rf-osdtypes-const
Ronen Friedman [Fri, 7 Jul 2023 05:10:07 +0000 (08:10 +0300)]
Merge pull request #52337 from ronen-fr/wip-rf-osdtypes-const

osd/osd_types_fmt: mark fmt::formatter<>::format() const

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #51179 from aisakaki/wip-lazy-read-r2
Yingxin [Fri, 7 Jul 2023 03:36:19 +0000 (11:36 +0800)]
Merge pull request #51179 from aisakaki/wip-lazy-read-r2

crimson/os/seastore: realize lazy read in split overwrite with overwrite refactor

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Zhang Song <zhangsong325@gmail.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51770 from myoungwon/wip-multicore-rbm
Yingxin [Fri, 7 Jul 2023 01:13:15 +0000 (09:13 +0800)]
Merge pull request #51770 from myoungwon/wip-multicore-rbm

crimson/os/seastore/rbm: make rbm support multiple shards

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2 years agoMerge pull request #52324 from zdover23/wip-doc-2023-07-06-radosgw-multisite
zdover23 [Thu, 6 Jul 2023 20:32:52 +0000 (06:32 +1000)]
Merge pull request #52324 from zdover23/wip-doc-2023-07-06-radosgw-multisite

doc/radosgw: add Zonegroup purpose

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #52319 from Thingee/foundation-mem-update-202307
zdover23 [Thu, 6 Jul 2023 20:26:17 +0000 (06:26 +1000)]
Merge pull request #52319 from Thingee/foundation-mem-update-202307

doc/foundation: Updating foundation members for July 2023

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/radosgw: add Zonegroup purpose 52324/head
Zac Dover [Wed, 5 Jul 2023 17:36:52 +0000 (03:36 +1000)]
doc/radosgw: add Zonegroup purpose

Add a note about the telos of zonegroups. The information added in this
commit was originally sent to ceph-users by Casey Bodley on 30 Jun 2023
in response to a question from Yixin Jin. Alexander Patrakov then
suggested that Casey's response should be added to
doc/radsogw/multisite.rst.

Co-authored-by: Casey Bodley <cbodley@redhat.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #52087 from AliMasarweh/wip-alimasa-persistant-q-observability
Ali Masarwa [Thu, 6 Jul 2023 15:10:23 +0000 (18:10 +0300)]
Merge pull request #52087 from AliMasarweh/wip-alimasa-persistant-q-observability

RGW: Add observability over the persistent topics queue

2 years agoMerge pull request #52216 from cbodley/wip-61838
Casey Bodley [Thu, 6 Jul 2023 13:59:31 +0000 (09:59 -0400)]
Merge pull request #52216 from cbodley/wip-61838

qa/s3tests: filter on 'sts_tests' and 'webidentity_tests'

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2 years agoMerge pull request #50458 from Matan-B/wip-matanb-c-perf-basic
Matan [Thu, 6 Jul 2023 12:42:05 +0000 (15:42 +0300)]
Merge pull request #50458 from Matan-B/wip-matanb-c-perf-basic

qa/suites/crimson-rados: Introduce performance testing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Mark Nelson <mark.nelson@clyso.com>
2 years agoosd/osd_types_fmt: mark fmt::formatter<>::format() const 52337/head
Ronen Friedman [Thu, 6 Jul 2023 11:18:55 +0000 (06:18 -0500)]
osd/osd_types_fmt: mark fmt::formatter<>::format() const

For some formatters missed or added after PR#47830 (that fixed
most of the missing 'const's in this file)

Required to allow the use of the formatters in const member functions.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoRGW: Add observability over the persistent topics queue 52087/head
Ali Masarwa [Tue, 6 Jun 2023 13:20:40 +0000 (16:20 +0300)]
RGW: Add observability over the persistent topics queue

Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2 years agoMerge pull request #52293 from ronen-fr/wip-rf-consttime
Ronen Friedman [Thu, 6 Jul 2023 06:28:41 +0000 (09:28 +0300)]
Merge pull request #52293 from ronen-fr/wip-rf-consttime

common: make some common ctors and accessors constexpr

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2 years agosrc/test/crimson/seastore/nvmedevice: update a test case based on rbm multi shard... 51770/head
myoungwon oh [Sat, 17 Jun 2023 08:27:17 +0000 (08:27 +0000)]
src/test/crimson/seastore/nvmedevice: update a test case based on rbm multi shard change

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agocrimson/os/seastore/rbm: make rbm support multiple shards
myoungwon oh [Fri, 26 May 2023 08:29:50 +0000 (08:29 +0000)]
crimson/os/seastore/rbm: make rbm support multiple shards

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2 years agoMerge PR #52290 into main
Patrick Donnelly [Wed, 5 Jul 2023 17:10:13 +0000 (13:10 -0400)]
Merge PR #52290 into main

* refs/pull/52290/head:
pybind/rbd: drop GIL when calling into librbd
pybind/cephfs: drop gil during cephfs callouts

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agodoc/foundation: Updating foundation members for July 2023 52319/head
Mike Perez [Wed, 5 Jul 2023 15:31:04 +0000 (08:31 -0700)]
doc/foundation: Updating foundation members for July 2023

Signed-off-by: Mike Perez <thingee@gmail.com>
2 years agoMerge pull request #50558 from rhcs-dashboard/fix-dataUnits-graph-plotting
Pedro Gonzalez Gomez [Wed, 5 Jul 2023 10:29:20 +0000 (12:29 +0200)]
Merge pull request #50558 from rhcs-dashboard/fix-dataUnits-graph-plotting

mgr/dashboard: fix a bug where data would plot wrongly

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agocommon: make some common ctors and accessors constexpr 52293/head
Ronen Friedman [Sun, 2 Jul 2023 17:24:47 +0000 (12:24 -0500)]
common: make some common ctors and accessors constexpr

Add the constexpr qualifier to:
- some utime_t ctors and accessors;
- a shard_id_t ctor;
- ceph_le operator

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2 years agoMerge pull request #51852 from Matan-B/wip-matanb-crimson-watch-notify-logs
Matan [Wed, 5 Jul 2023 08:33:31 +0000 (11:33 +0300)]
Merge pull request #51852 from Matan-B/wip-matanb-crimson-watch-notify-logs

crimson/osd/watch: Add logs around Watch/Notify

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #52086 from pkalever/fix-mirror-status-desc
Ilya Dryomov [Tue, 4 Jul 2023 22:22:53 +0000 (00:22 +0200)]
Merge pull request #52086 from pkalever/fix-mirror-status-desc

rbd-mirror: fix image replayer shut down description on force promote

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #52207 from zdover23/wip-doc-2023-06-27-rados-operations-crush...
zdover23 [Tue, 4 Jul 2023 19:22:33 +0000 (05:22 +1000)]
Merge pull request #52207 from zdover23/wip-doc-2023-06-27-rados-operations-crush-map-edits-2-of-x-2nd-attempt

doc/rados: edit crush-map-edits (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: edit crush-map-edits (2 of x) 52207/head
Zac Dover [Tue, 27 Jun 2023 08:58:32 +0000 (18:58 +1000)]
doc/rados: edit crush-map-edits (2 of x)

Edit doc/rados/operations/crush-map-edits.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #52308 from zdover23/wip-doc-2023-07-05-seqdiag-test
zdover23 [Tue, 4 Jul 2023 17:55:15 +0000 (03:55 +1000)]
Merge pull request #52308 from zdover23/wip-doc-2023-07-05-seqdiag-test

doc/dev: remove seqdiag assets

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2 years agodoc/dev: remove seqdiag assets 52308/head
Zac Dover [Tue, 4 Jul 2023 17:37:10 +0000 (03:37 +1000)]
doc/dev: remove seqdiag assets

Remove seqdiag assets to determine whether the docs can be built if they
are absent. (Currently they cannot be built when they are present.) If
this works, then these diagrams will be replaced, probably with .png
files.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoqa/tasks/cbt.py: Install collectl manually 50458/head
Matan Breizman [Thu, 9 Mar 2023 15:09:12 +0000 (15:09 +0000)]
qa/tasks/cbt.py: Install collectl manually

See: d4a04809fd7fd8aaf447005f76a1090db99d75c5

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agoqa/suites/crimson-rados: Introduce performance testing
Matan Breizman [Thu, 9 Mar 2023 13:42:54 +0000 (13:42 +0000)]
qa/suites/crimson-rados: Introduce performance testing

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agotest/crimson/seastore: test if lazy read work correctly 51179/head
Xinyu Huang [Fri, 26 May 2023 11:05:08 +0000 (11:05 +0000)]
test/crimson/seastore: test if lazy read work correctly

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agotest/crimson/seastore: test overwrite pin
Xinyu Huang [Fri, 21 Apr 2023 14:22:16 +0000 (14:22 +0000)]
test/crimson/seastore: test overwrite pin

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agotest/crimson/seastore: add test for TransactionManager::remap_pin
Xinyu Huang [Fri, 21 Apr 2023 14:19:34 +0000 (14:19 +0000)]
test/crimson/seastore: add test for TransactionManager::remap_pin

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agocrimson/os/seastore: check if length exist in mark_space_used and mark_space_free
Xinyu Huang [Fri, 21 Apr 2023 14:42:00 +0000 (14:42 +0000)]
crimson/os/seastore: check if length exist in mark_space_used and mark_space_free

to avoid misuse of buffer length and data length when there is a no actual data extent

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agocrimson/os/seastore: deepcopy the buffer of exist clean extent when duplicate_for_write
Xinyu Huang [Wed, 19 Apr 2023 01:46:31 +0000 (01:46 +0000)]
crimson/os/seastore: deepcopy the buffer of exist clean extent when duplicate_for_write

Beacuse exist clean extent shares buffer with original clean extent.

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agocrimson/os/seastore: realize lazy read in split overwrite with overwrite refactor
Xinyu Huang [Fri, 7 Apr 2023 00:54:36 +0000 (00:54 +0000)]
crimson/os/seastore: realize lazy read in split overwrite with overwrite refactor

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agocrimson/os/seastore/transaction_manager: implement remap_pin
Xinyu Huang [Fri, 21 Apr 2023 15:52:24 +0000 (15:52 +0000)]
crimson/os/seastore/transaction_manager: implement remap_pin

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agocrimson/os/seastore/transaction_manager: implement alloc_remapped_extent
Xinyu Huang [Fri, 21 Apr 2023 15:10:49 +0000 (15:10 +0000)]
crimson/os/seastore/transaction_manager: implement alloc_remapped_extent

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agocrimson/os/seastore: support no data extent in read path
Xinyu Huang [Sat, 10 Jun 2023 02:16:29 +0000 (10:16 +0800)]
crimson/os/seastore: support no data extent in read path

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agoMerge pull request #51422 from yuvalif/wip-yuval-cephadm-lua-doc
zdover23 [Mon, 3 Jul 2023 17:47:40 +0000 (03:47 +1000)]
Merge pull request #51422 from yuvalif/wip-yuval-cephadm-lua-doc

doc/rgw/lua: add info uploading a script in cephadm deployment

Reviewed-by: Zac Dover <zac.dover@proton.me>
2 years agopybind/rbd: drop GIL when calling into librbd 52290/head
Ilya Dryomov [Mon, 3 Jul 2023 11:34:30 +0000 (13:34 +0200)]
pybind/rbd: drop GIL when calling into librbd

This was missing for rbd_mirror_peer_site_add() and
rbd_get_data_pool_id().

While at it, add a test for data_pool_id().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #52247 from cbodley/wip-57905
Casey Bodley [Mon, 3 Jul 2023 15:19:36 +0000 (11:19 -0400)]
Merge pull request #52247 from cbodley/wip-57905

rgw: rgwx-skip-decrypt also skips decompression of encrypted objects

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agoMerge pull request #52249 from mkogan1/wip-t61661-d3n-valg
Mark Kogan [Mon, 3 Jul 2023 14:06:21 +0000 (17:06 +0300)]
Merge pull request #52249 from mkogan1/wip-t61661-d3n-valg

rgw: d3n: fix valgrind reported leak related to libaio ops

2 years agocrimson/os/seastore: let CachedExtent::ptr optional
Xinyu Huang [Tue, 21 Feb 2023 06:18:29 +0000 (06:18 +0000)]
crimson/os/seastore: let CachedExtent::ptr optional

separate buffer length and data length, since no buffer extent is allowed

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2 years agoMerge pull request #52160 from soumyakoduri/wip-skoduri-lc-acl
Soumya Koduri [Mon, 3 Jul 2023 04:50:48 +0000 (10:20 +0530)]
Merge pull request #52160 from soumyakoduri/wip-skoduri-lc-acl

rgw/set_attrs: Do not modify mtime

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agoMerge pull request #51983 from clwluvw/aws4-date-header
J. Eric Ivancich [Sun, 2 Jul 2023 20:30:26 +0000 (16:30 -0400)]
Merge pull request #51983 from clwluvw/aws4-date-header

rgw: pick http_date in case of http_x_amz_date absence

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agopybind/cephfs: drop gil during cephfs callouts
Patrick Donnelly [Sun, 2 Jul 2023 16:05:08 +0000 (12:05 -0400)]
pybind/cephfs: drop gil during cephfs callouts

This has disastorous consequences including the possibility of deadlock.
In the best case, you have the rmdir holding the GIL until the MDS
responds!

Fixes: https://tracker.ceph.com/issues/61869
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agocrimson/osd/watch: don't print bufferlists 51852/head
Matan Breizman [Wed, 31 May 2023 11:47:45 +0000 (11:47 +0000)]
crimson/osd/watch: don't print bufferlists

The bufferlists clutter the logs and don't infrom of anything useful:
```
DEBUG 2023-05-26 23:36:29,780 [shard 2] osd - send_completion sending notify replies: {notify_reply_t{watcher_gid=4220, watcher_cookie=139786421930752, bl=buffer::list(len=0,

)}, notify_reply_t{watcher_gid=4220, watcher_cookie=139786422032096, bl=buffer::list(len=0,

)}}
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agocrimson/osd/watch: Add logs around Watch/Notify
Matan Breizman [Wed, 31 May 2023 08:45:53 +0000 (08:45 +0000)]
crimson/osd/watch: Add logs around Watch/Notify

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agoMerge pull request #49044 from rzarzynski/wip-crimson-ertr-plain-rets
Matan [Sun, 2 Jul 2023 11:02:01 +0000 (14:02 +0300)]
Merge pull request #49044 from rzarzynski/wip-crimson-ertr-plain-rets

crimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #52032 from Matan-B/wip-matanb-crimson-no-ignore
Matan [Sun, 2 Jul 2023 10:53:25 +0000 (13:53 +0300)]
Merge pull request #52032 from Matan-B/wip-matanb-crimson-no-ignore

crimson/osd: don't ignore start_pg_operation returned future

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51147 from liu-chunmei/multicore-osd
Matan [Sun, 2 Jul 2023 10:15:02 +0000 (13:15 +0300)]
Merge pull request #51147 from liu-chunmei/multicore-osd

Crimson/osd: support multicore osd

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51916 from cyx1231st/wip-seastar-msgr-multi-core-2
Matan [Sun, 2 Jul 2023 10:05:04 +0000 (13:05 +0300)]
Merge pull request #51916 from cyx1231st/wip-seastar-msgr-multi-core-2

crimson/net: support connections in multiple shards

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #52280 from zdover23/wip-doc-2023-07-01-radosgw-multisite
Anthony D'Atri [Sat, 1 Jul 2023 13:46:19 +0000 (09:46 -0400)]
Merge pull request #52280 from zdover23/wip-doc-2023-07-01-radosgw-multisite

doc/radosgw: refine "Zones" in multisite.rst

2 years agodoc/radosgw: refine "Zones" in multisite.rst 52280/head
Zac Dover [Sat, 1 Jul 2023 11:51:15 +0000 (21:51 +1000)]
doc/radosgw: refine "Zones" in multisite.rst

Describe in greater detail the function of Zones in serving S3 objects,
as requested by Alexander Patrakov here:
https://github.com/ceph/ceph/pull/49981#issuecomment-1615866374.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #52277 from zdover23/wip-doc-2023-07-01-radosgw-s3select-overview
zdover23 [Sat, 1 Jul 2023 11:29:24 +0000 (21:29 +1000)]
Merge pull request #52277 from zdover23/wip-doc-2023-07-01-radosgw-s3select-overview

doc/radosgw: s/s3select/S3 Select/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/radosgw: s/s3select/S3 Select/ 52277/head
Zac Dover [Sat, 1 Jul 2023 11:14:30 +0000 (21:14 +1000)]
doc/radosgw: s/s3select/S3 Select/

Correct several misspelling of "S3 Select". Hat tip to Anthony D'Atri,
who caught this in an earlier PR.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #47595 from 5cs/fix-lc-head-marker
J. Eric Ivancich [Fri, 30 Jun 2023 15:33:15 +0000 (11:33 -0400)]
Merge pull request #47595 from 5cs/fix-lc-head-marker

rgwlc: fix lc head marker point to non-exist lc entry

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2 years agoMerge pull request #50667 from ivancich/wip-add-reupload-testing
J. Eric Ivancich [Fri, 30 Jun 2023 15:30:27 +0000 (11:30 -0400)]
Merge pull request #50667 from ivancich/wip-add-reupload-testing

qa/rgw: test that multipart re-upload does not leave any orphans

Reviewed-by: Cory Snyder <csnyder@iland.com>
2 years agoMerge pull request #52252 from soumyakoduri/wip-skoduri-dbstore
Soumya Koduri [Fri, 30 Jun 2023 12:14:02 +0000 (17:44 +0530)]
Merge pull request #52252 from soumyakoduri/wip-skoduri-dbstore

rgw/dbstore: Update README

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #52217 from zhscn/fix-file-local-variable
Yingxin [Fri, 30 Jun 2023 02:32:24 +0000 (10:32 +0800)]
Merge pull request #52217 from zhscn/fix-file-local-variable

crimson/os/seastore/omap_manager: correct editor settings

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #52250 from zdover23/wip-doc-2023-06-29-radosgw-s3select-basic...
zdover23 [Thu, 29 Jun 2023 16:00:13 +0000 (02:00 +1000)]
Merge pull request #52250 from zdover23/wip-doc-2023-06-29-radosgw-s3select-basic-workflow

doc/radosgw: edit "Basic Workflow" in s3select.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/radosgw: edit "Basic Workflow" in s3select.rst 52250/head
Zac Dover [Thu, 29 Jun 2023 08:48:00 +0000 (18:48 +1000)]
doc/radosgw: edit "Basic Workflow" in s3select.rst

Edit the "Basic Workflow" section in doc/radosgw/s3select.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agorgw/dbstore: Update README 52252/head
Soumya Koduri [Thu, 29 Jun 2023 10:30:43 +0000 (16:00 +0530)]
rgw/dbstore: Update README

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agorgw: d3n: fix valgrind reported leak related to libaio ops 52249/head
Mark Kogan [Wed, 28 Jun 2023 14:06:27 +0000 (14:06 +0000)]
rgw: d3n: fix valgrind reported leak related to libaio ops

Fixes: https://tracker.ceph.com/issues/61661
The valgrind leak indication is a false positive in this case,
it is because the libaio internal thread have not timed out yet
when radosgw is terminated.

```
man aio_init
...
aio_idle_time
This field specifies the amount of time in seconds that a worker thread
should wait for further requests before terminating, after having
completed a previous request.   The
default value is 1.
...
```
for the sake of teuthology reducing the timeout

waiting for 2 minutes for example like below would also prevent the leak report
```
❯ env
LD_LIBRARY_PATH=/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/:$LD_LIBRARY_PATH
PYTHONPATH=$PYTHONPATH:/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/cython_modules/lib.3
RAGWEED_CONF=$(realpath ./ragweed.conf) RAGWEED_STAGES=prepare,check tox
-- -v |& ccze -Aonolookups ;   sleep 2m | pv -t ; pkill radosgw
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2 years agoMerge pull request #52008 from rishabh-d-dave/improvements-authmon
Radoslaw Zarzynski [Thu, 29 Jun 2023 07:32:03 +0000 (09:32 +0200)]
Merge pull request #52008 from rishabh-d-dave/improvements-authmon

mon: improvements and cleanup for AuthMonitor

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd/heartbeat: relax the order of replacement reset and accept 51916/head
Yingxin Cheng [Thu, 29 Jun 2023 05:25:28 +0000 (13:25 +0800)]
crimson/osd/heartbeat: relax the order of replacement reset and accept

With the new implementation in messenger, the order of replacement reset
and accept events cannot be determined because they are from different
connections.

Modify the heatbeat logic to tolerate the both cases.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agorgw: fetch_remote_obj() preserves RGW_ATTR_COMPRESSION of encrypted objects 52247/head
Casey Bodley [Wed, 28 Jun 2023 18:57:49 +0000 (14:57 -0400)]
rgw: fetch_remote_obj() preserves RGW_ATTR_COMPRESSION of encrypted objects

if the source object was both compressed and encrypted, preserve its
original compression attribute so it can be decompressed the same way it
was originally compressed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agorgw: fetch_remote_obj() will never verify etags of encrypted objects
Casey Bodley [Wed, 28 Jun 2023 18:43:55 +0000 (14:43 -0400)]
rgw: fetch_remote_obj() will never verify etags of encrypted objects

fetch_remote_obj() transfers objects in their encrypted form, so does
not have access to the decrypted data for checksum verification

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agorgw: rgwx-skip-decrypt also skips decompression of encrypted objects
Casey Bodley [Wed, 28 Jun 2023 18:42:16 +0000 (14:42 -0400)]
rgw: rgwx-skip-decrypt also skips decompression of encrypted objects

compression is applied before encryption. so if we skip decryption, we
can't decompress either

Fixes: https://tracker.ceph.com/issues/57905
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #52157 from pkalever/log-improve
Ilya Dryomov [Wed, 28 Jun 2023 09:45:50 +0000 (11:45 +0200)]
Merge pull request #52157 from pkalever/log-improve

rbd-mirror: add some assorted logs

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agorgw: Do not update mtime in `set_attrs` 52160/head
Soumya Koduri [Sat, 24 Jun 2023 06:55:46 +0000 (12:25 +0530)]
rgw: Do not update mtime in `set_attrs`

As per https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html,
the only way for users to modify object metadata is to make a copy of the object and
set the metadata. Hence retain old mtime for any other attr changes

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2 years agoMerge pull request #52194 from zdover23/wip-doc-2023-06-26-radosgw-s3select-overview
zdover23 [Wed, 28 Jun 2023 06:43:08 +0000 (16:43 +1000)]
Merge pull request #52194 from zdover23/wip-doc-2023-06-26-radosgw-s3select-overview

doc/radosgw: edit "Overview" in s3select.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/radosgw: edit "Overview" in s3select.rst 52194/head
Zac Dover [Mon, 26 Jun 2023 11:45:43 +0000 (21:45 +1000)]
doc/radosgw: edit "Overview" in s3select.rst

Edit the "Overview" section in doc/radosgw/s3select.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge PR #49971 into main
Venky Shankar [Wed, 28 Jun 2023 04:53:54 +0000 (10:23 +0530)]
Merge PR #49971 into main

* refs/pull/49971/head:
doc/cephfs: document MDS_CLIENTS_LAGGY health warning
qa: ignore warnings
qa: add test cases to check client eviction if an OSD is laggy
mds,messages: enable beacon to report clients lagginess
mds: do not evict client on laggy osds
common: add new config option to defer client eviction
osd: add method to check for laggy osds

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #52090 from xxhdx1985126/wip-fixedkv-btree-merge-fix
Yingxin [Wed, 28 Jun 2023 04:53:06 +0000 (12:53 +0800)]
Merge pull request #52090 from xxhdx1985126/wip-fixedkv-btree-merge-fix

crimson/os/seastore/btree: should add left's size when merging levels…

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agorbd-mirror: add some logs 52157/head
Prasanna Kumar Kalever [Thu, 22 Jun 2023 14:38:36 +0000 (20:08 +0530)]
rbd-mirror: add some logs

Added some logs as their values where not very clear while parsing though the
log files.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2 years agocrimson/os/seastore/omap_manager: correct editor settings 52217/head
Zhang Song [Wed, 28 Jun 2023 02:01:21 +0000 (10:01 +0800)]
crimson/os/seastore/omap_manager: correct editor settings

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
2 years agoqa/rgw/sts: rename tasks, don't duplicate s3tests 52216/head
Casey Bodley [Tue, 27 Jun 2023 21:02:34 +0000 (17:02 -0400)]
qa/rgw/sts: rename tasks, don't duplicate s3tests

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoqa/rgw/sts: move overrides into overrides.yaml
Casey Bodley [Tue, 27 Jun 2023 20:55:23 +0000 (16:55 -0400)]
qa/rgw/sts: move overrides into overrides.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoqa/s3tests: filter on 'sts_tests' and 'webidentity_tests'
Casey Bodley [Tue, 27 Jun 2023 20:45:06 +0000 (16:45 -0400)]
qa/s3tests: filter on 'sts_tests' and 'webidentity_tests'

qa/tasks/s3tests.py was adding `not test_of_sts` and `not
webidentity_test` by default, and the rgw/sts suites were adding
those attrs back in `extra_attrs`

when `extra_attrs` was changed to be additive, this started causing
InvocationError failures

instead of using `extra_attrs` to control these filters,
qa/tasks/s3tests.py now uses the `sts_tests` and `webidentity_tests`
flags from the sts yaml files to control whether or not the `not
test_of_sts`/`not webidentity_test` attrs are added to the pytest
command line

Fixes: https://tracker.ceph.com/issues/61838
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge PR #51858 into main
Patrick Donnelly [Tue, 27 Jun 2023 17:43:38 +0000 (13:43 -0400)]
Merge PR #51858 into main

* refs/pull/51858/head:
pybind/mgr/devicehealth: do not crash if db not ready

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2 years agoMerge PR #52170 into main
Venky Shankar [Tue, 27 Jun 2023 12:21:31 +0000 (17:51 +0530)]
Merge PR #52170 into main

* refs/pull/52170/head:
qa: assign file system affinity for replaced MDS

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2 years agoMerge pull request #51283 from rhcs-dashboard/disable-delete
Aashish Sharma [Tue, 27 Jun 2023 11:58:12 +0000 (17:28 +0530)]
Merge pull request #51283 from rhcs-dashboard/disable-delete

mgr/dashboard: disable delete on multisite

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2 years agorbd-mirror: fix image replayer shut down description on force promote 52086/head
Prasanna Kumar Kalever [Thu, 15 Jun 2023 14:10:16 +0000 (19:40 +0530)]
rbd-mirror: fix image replayer shut down description on force promote

On force promote if the opposite site is down then we currently show
image status description as "local image linked to unknown peer"

Previously:
----------
$ rbd --cluster=site-b mirror image status pool1/img1
img1:
  global_id:   a73341a6-8302-4c97-ac6e-278083fd347e
  state:       up+stopping_replay
  description: local image linked to unknown peer
  service:     admin on localhost.localdomain
  last_update: 2023-06-15 19:47:45
  peer_sites:
    name: site-a
    state: up+stopped
    description: local image is primary
    last_update: 2023-06-15 19:47:32
  snapshots:
    9 .mirror.primary.a73341a6-8302-4c97-ac6e-278083fd347e.1f101367-277f-42f0-8308-e51201d0529a (peer_uuids:[c46c6d97-f59b-4591-9d35-d7ff9d0d72f7])

Currently:
---------
$ rbd --cluster=site-b mirror image status pool1/img1
img1:
  global_id:   2a6d61e1-8e76-42c4-af76-8f61ce65c7e2
  state:       up+stopped
  description: orphan (force promoting)
  service:     admin on localhost.localdomain
  last_update: 2023-06-15 19:29:22
  peer_sites:
    name: site-a
    state: down+stopped
    description: local image is primary
    last_update: 2023-06-15 19:29:05
  snapshots:
    9 .mirror.primary.2a6d61e1-8e76-42c4-af76-8f61ce65c7e2.99f82a30-0241-4e51-8428-7a2376d137f6 (peer_uuids:[3150c6ef-aeee-45dc-8d0e-5dc5a53d88eb])

Fixes: https://tracker.ceph.com/issues/52913
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2 years agoqa: assign file system affinity for replaced MDS 52170/head
Venky Shankar [Thu, 22 Jun 2023 10:08:44 +0000 (06:08 -0400)]
qa: assign file system affinity for replaced MDS

Otherwise, the MDS that just got replaced can transition to a rank
for another file system and the test cannot deterministically infer
which MDS needs to checked.

Fixes: http://tracker.ceph.com/issues/61764
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2 years agocrimson/net: cleanup, rename is_fixed_cpu
Yingxin Cheng [Tue, 27 Jun 2023 02:31:11 +0000 (10:31 +0800)]
crimson/net: cleanup, rename is_fixed_cpu

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #51928 from samarahu/cmake-cleanup
Samarah Uriarte [Mon, 26 Jun 2023 14:30:40 +0000 (10:30 -0400)]
Merge pull request #51928 from samarahu/cmake-cleanup

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge PR #52149 into main
Venky Shankar [Mon, 26 Jun 2023 12:21:25 +0000 (17:51 +0530)]
Merge PR #52149 into main

* refs/pull/52149/head:
cephfs-journal-tool: disambiguate usage of all keyword (in tool help).

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2 years agoMerge pull request #52155 from nbalacha/poolreplayer-logging
Ilya Dryomov [Mon, 26 Jun 2023 09:49:26 +0000 (11:49 +0200)]
Merge pull request #52155 from nbalacha/poolreplayer-logging

rbd-mirror: add additional logs to PoolReplayer

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge PR #51417 into main
Venky Shankar [Mon, 26 Jun 2023 07:29:12 +0000 (12:59 +0530)]
Merge PR #51417 into main

* refs/pull/51417/head:
client: only wait for write MDS OPs when unmounting
client: flush the mdlog before waiting for unsafe requests

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2 years agoMerge PR #50908 into main
Venky Shankar [Mon, 26 Jun 2023 07:27:43 +0000 (12:57 +0530)]
Merge PR #50908 into main

* refs/pull/50908/head:
mon/MDSMonitor: plug paxos when maybe manipulating the osdmap
mon/MDSMonitor: force immediate propose when evicting an MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge PR #49295 into main
Venky Shankar [Mon, 26 Jun 2023 07:26:33 +0000 (12:56 +0530)]
Merge PR #49295 into main

* refs/pull/49295/head:
mgr/snap_schedule: catch all exceptions for cli

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #51845 from ssdohammer-sl/doc-update-deduplication
zdover23 [Mon, 26 Jun 2023 01:46:07 +0000 (11:46 +1000)]
Merge pull request #51845 from ssdohammer-sl/doc-update-deduplication

doc:dev update how to use deduplication

Reviewed-by: Zac Dover <zac.dover@proton.me>
2 years agodoc:dev update how to use deduplication 51845/head
Sungmin Lee [Fri, 26 May 2023 08:05:46 +0000 (17:05 +0900)]
doc:dev update how to use deduplication

Add prerequisite for deduplication, example outputs, and chunk-repair operation.

Signed-off-by: Sungmin Lee <sung_min.lee@samsung.com>
2 years agocephfs-journal-tool: disambiguate usage of all keyword (in tool help). 52149/head
Manish M Yathnalli [Wed, 21 Jun 2023 07:59:41 +0000 (13:29 +0530)]
cephfs-journal-tool: disambiguate usage of all keyword (in tool help).

        The fs:all for rank option description was confusing. It seemd
like the fs was optional, but it is mandatory. This change modifies the
help message to reflect the correct way to use all in the --rank option.

Fixes: https://tracker.ceph.com/issues/61753
Signed-off-by: Manish M Yathnalli <myathnal@redhat.com>
2 years agoMerge pull request #52181 from zdover23/wip-doc-2023-06-25-radosgw-s3select-formatting
zdover23 [Sun, 25 Jun 2023 13:46:11 +0000 (23:46 +1000)]
Merge pull request #52181 from zdover23/wip-doc-2023-06-25-radosgw-s3select-formatting

doc/radosgw: remove pipes from s3select.rst

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2 years agoMerge pull request #52110 from yuvalif/wip-yuval-fix-61639
Yuval Lifshitz [Sun, 25 Jun 2023 12:54:26 +0000 (15:54 +0300)]
Merge pull request #52110 from yuvalif/wip-yuval-fix-61639

rgw/amqp: remove possible race conditions with the amqp connections

2 years agodoc/radosgw: remove pipes from s3select.rst 52181/head
Zac Dover [Sun, 25 Jun 2023 09:03:07 +0000 (19:03 +1000)]
doc/radosgw: remove pipes from s3select.rst

Remove strange formatting introduced in
61d6813726fc08980903c6a6c50b0a5797ba7193.

Note to the documentation team: This is not a line-edit. This commit
includes nothing but the removal of pipes added to the left of much of
the text in this file. Several future commits will line-edit this file
and correct its formatting.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agorgw/amqp: use "fmt" for converting connection_id to string 52110/head
Yuval Lifshitz [Sun, 25 Jun 2023 08:56:26 +0000 (08:56 +0000)]
rgw/amqp: use "fmt" for converting connection_id to string

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2 years agoMerge pull request #51980 from aisakaki/wip-fix-check_node
Yingxin [Sun, 25 Jun 2023 05:46:17 +0000 (13:46 +0800)]
Merge pull request #51980 from aisakaki/wip-fix-check_node

crimson/os/seastore: fix in check_node

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>