]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Matan Breizman [Tue, 17 Oct 2023 12:46:08 +0000 (15:46 +0300)]
Merge pull request #53946 from Matan-B/wip-52090-reef
reef: crimson/os/seastore/btree: should add left's size when merging levels…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 12:45:48 +0000 (15:45 +0300)]
Merge pull request #53937 from Matan-B/wip-50598-reef
reef: crimson/osd: add embedded suppression ruleset for LSan
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:38:25 +0000 (14:38 +0300)]
Merge pull request #53953 from Matan-B/wip-52369-reef
reef: crimson/os/seastore: add metrics about total invalidated transactions
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:37:59 +0000 (14:37 +0300)]
Merge pull request #53952 from Matan-B/wip-51770-reef
reef: crimson/os/seastore/rbm: make rbm support multiple shards
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:36:49 +0000 (14:36 +0300)]
Merge pull request #53950 from Matan-B/wip-51852-reef
reef: crimson/osd/watch: Add logs around Watch/Notify
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:34:49 +0000 (14:34 +0300)]
Merge pull request #53948 from Matan-B/wip-52032-reef
reef: crimson/osd: don't ignore start_pg_operation returned future
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:29:58 +0000 (14:29 +0300)]
Merge pull request #53945 from Matan-B/wip-51980-reef
reef: crimson/os/seastore: fix in check_node
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:28:09 +0000 (14:28 +0300)]
Merge pull request #53944 from Matan-B/wip-51961-reef
reef: Crimson/osd: Disable concurrent MOSDMap handling
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:27:31 +0000 (14:27 +0300)]
Merge pull request #53943 from Matan-B/wip-51945-reef
reef: crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:21:04 +0000 (14:21 +0300)]
Merge pull request #53941 from Matan-B/wip-51948-reef
reef: crimson/os/seastore/cached_extent: add prepare_commit interface
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:20:38 +0000 (14:20 +0300)]
Merge pull request #53940 from Matan-B/wip-51380-reef
reef: crimson/osd/heartbeat: Improve maybe_share_osdmap behavior
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:19:53 +0000 (14:19 +0300)]
Merge pull request #53939 from Matan-B/wip-51750-reef
reef: crimson/os/seastore/backref_manager: scan backref entries by journal seq
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:19:10 +0000 (14:19 +0300)]
Merge pull request #53938 from Matan-B/wip-51475-reef
reef: crimson/os/seastore: fix ceph_assert in segment_manager.h
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 17 Oct 2023 11:15:39 +0000 (14:15 +0300)]
Merge pull request #53935 from Matan-B/wip-51355-reef
reef: crimson/os/seastore: enable SMR HDD
Reviewed-by: Samuel Just <sjust@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:37:47 +0000 (12:37 -0700)]
Merge pull request #53797 from smanjara/ragweed-test-reef
reef: rgw/upgrade: point upgrade suites to ragweed ceph-reef branch
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:37:03 +0000 (12:37 -0700)]
Merge pull request #53773 from cbodley/wip-63057-reef
reef: rgw: improve buffer list utilization in the chunkupload scenario
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:35:12 +0000 (12:35 -0700)]
Merge pull request #53770 from cbodley/wip-63054-reef
reef: rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:34:05 +0000 (12:34 -0700)]
Merge pull request #53767 from cbodley/wip-63051-reef
reef: rgw: s3website doesn't prefetch for web_dir() check
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:33:06 +0000 (12:33 -0700)]
Merge pull request #53765 from cbodley/wip-63048-reef
reef: rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_rea…
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:32:15 +0000 (12:32 -0700)]
Merge pull request #53760 from cbodley/wip-63060-reef
reef: rgw: fix unwatch crash at radosgw startup
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:31:29 +0000 (12:31 -0700)]
Merge pull request #53756 from cbodley/wip-63042-reef
reef: [CVE-2023-43040] rgw: Fix bucket validation against POST policies
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Yuri Weinstein [Mon, 16 Oct 2023 19:30:28 +0000 (12:30 -0700)]
Merge pull request #53681 from mkogan1/wip-t62141-reef
reef: valgrind: UninitCondition under __run_exit_handlers suppression
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Anthony D'Atri [Sun, 15 Oct 2023 23:54:09 +0000 (19:54 -0400)]
Merge pull request #54026 from zdover23/wip-doc-2023-10-16-backport-54022-to-reef
reef: doc/rados: Edit troubleshooting-osd (3 of x)
Zac Dover [Sun, 15 Oct 2023 12:43:39 +0000 (22:43 +1000)]
doc/rados: Edit troubleshooting-osd (3 of x)
Edit doc/rados/troubleshooting/troubleshooting-osd.rst (3 of x).
Follows https://github.com/ceph/ceph/pull/53986.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
01b5aa5d27a97a02cef03b9eb0ee9dbcd55684f1 )
Anthony D'Atri [Fri, 13 Oct 2023 20:31:28 +0000 (16:31 -0400)]
Merge pull request #54008 from zdover23/wip-doc-2023-10-14-backport-54006-to-reef
doc/rados: remove FileStore material (in Reef)
Zac Dover [Fri, 13 Oct 2023 18:40:26 +0000 (04:40 +1000)]
doc/rados: remove FileStore material (in Reef)
Remove FileStore-related material because FileStore is not supported in
Reef.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
eab0be5ca513ca2dfaeb31911c175e88bcf4f9e4 )
Anthony D'Atri [Fri, 13 Oct 2023 20:15:24 +0000 (16:15 -0400)]
Merge pull request #54007 from zdover23/wip-doc-2023-10-14-backport-54005-to-reef-2
reef: doc/dev: remove cache-pool
Zac Dover [Fri, 13 Oct 2023 18:17:44 +0000 (04:17 +1000)]
doc/dev: remove cache-pool
Remove cache-pool, which is deprecated in Reef.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
073b0919ed9e413a04a7e12b7a0c23b5e058cd52 )
zdover23 [Fri, 13 Oct 2023 14:01:03 +0000 (00:01 +1000)]
Merge pull request #54000 from zdover23/wip-doc-2023-10-13-backport-53986-to-reef
reef: doc/rados: Edit troubleshooting-osd (2 of x)
Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Zac Dover [Thu, 12 Oct 2023 08:33:58 +0000 (18:33 +1000)]
doc/rados: Edit troubleshooting-osd (2 of x)
Edit doc/rados/troubleshooting/troubleshooting.rst (2 of x).
Follows https://github.com/ceph/ceph/pull/53936.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
37e7099267996a3075b4902a10a19d94fc738c08 )
Nizamudeen A [Fri, 13 Oct 2023 05:35:08 +0000 (11:05 +0530)]
Merge pull request #53458 from rhcs-dashboard/wip-62602-reef
reef: mgr/dashboard: remove unnecessary failing hosts e2e
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Anthony D'Atri [Thu, 12 Oct 2023 09:02:48 +0000 (05:02 -0400)]
Merge pull request #53982 from zdover23/wip-doc-2023-10-12-backport-53936-to-reef
reef: doc/rados: edit troubleshooting-osd (1 of x)
Zac Dover [Wed, 11 Oct 2023 10:38:55 +0000 (20:38 +1000)]
doc/rados: edit troubleshooting-osd (1 of x)
Edit doc/rados/troubleshooting/troubleshooting-osd.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
0fc663d793fe34e42e19b251fcc79b5af52581c6 )
Zhang Song [Sun, 9 Jul 2023 04:22:58 +0000 (12:22 +0800)]
crimson/os/seastore: add metrics about total invalidated transactions
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit
6265452c50c2bbd0283a66d03d6a8538c2285293 )
myoungwon oh [Sat, 17 Jun 2023 08:27:17 +0000 (08:27 +0000)]
src/test/crimson/seastore/nvmedevice: update a test case based on rbm multi shard change
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
7a15d484c552a177585954cdb4e2e226a43c1035 )
myoungwon oh [Fri, 26 May 2023 08:29:50 +0000 (08:29 +0000)]
crimson/os/seastore/rbm: make rbm support multiple shards
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit
1d6beef975144210ae29626e61aeb284aff30d53 )
Matan Breizman [Wed, 31 May 2023 11:47:45 +0000 (11:47 +0000)]
crimson/osd/watch: don't print bufferlists
The bufferlists clutter the logs and don't infrom of anything useful:
```
DEBUG 2023-05-26 23:36:29,780 [shard 2] osd - send_completion sending notify replies: {notify_reply_t{watcher_gid=4220, watcher_cookie=
139786421930752 , bl=buffer::list(len=0,
)}, notify_reply_t{watcher_gid=4220, watcher_cookie=
139786422032096 , bl=buffer::list(len=0,
)}}
```
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
3548489f622c50ccaa1d8c351e420873743ed980 )
Matan Breizman [Wed, 31 May 2023 08:45:53 +0000 (08:45 +0000)]
crimson/osd/watch: Add logs around Watch/Notify
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
b1c05f9acecfd2dffb57d63f42fec9da156ff366 )
Matan Breizman [Tue, 13 Jun 2023 09:58:34 +0000 (09:58 +0000)]
crimson/osd: don't ignore start_pg_operation returned future
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
43bf04e6e9cb82fa7cfe5644786d91e6265bcc8a )
Xuehan Xu [Fri, 16 Jun 2023 06:42:38 +0000 (14:42 +0800)]
crimson/os/seastore/btree: should add left's size when merging levels if donor is left
Fixes: https://tracker.ceph.com/issues/61709
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
918d26bf979814c158744979e16fdfaf1ac4ed3d )
Xinyu Huang [Fri, 9 Jun 2023 07:25:48 +0000 (15:25 +0800)]
crimson/os/seastore: fix bug in check_node
EXIST_CLEAN and EXIST_MUTATION_PENDING shuold not be treated as
CLEAN in check_node because they are transaction private and the
leafnode has been duplicated for write.
fix: https://tracker.ceph.com/issues/61626
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit
f02f20c2aecfbcb24d4a2d49db159a15b63487f4 )
Matan Breizman [Tue, 20 Jun 2023 17:51:52 +0000 (17:51 +0000)]
crimson/osd/osd_operations/pg_advance_map: 'from' epoch may vary between PGS explanation
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
ba519f5e72350e86647b7d9bc40fc2d26d908da3 )
Matan Breizman [Sun, 28 May 2023 10:31:37 +0000 (10:31 +0000)]
crimson/osd/osdmap_gate: got_map() assert increase of current epoch
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
06bf66154386662cd7bcccbef6cd7a2bdeb6e539 )
Matan Breizman [Wed, 24 May 2023 15:36:10 +0000 (15:36 +0000)]
crimson/osd/shard_services: get_or_create_pg() remove unused variable
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
a8800c7e700f0ed262885906ee59dcaa15230932 )
Matan Breizman [Wed, 10 May 2023 13:54:17 +0000 (13:54 +0000)]
crimson/osd/shard_services: Refactor OSDSingletonState::load_map
Added logs
If case reconstructed - epoch is unsigned.
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
ecac14a70031299d744fecb7b6c3fb464a346a80 )
Matan Breizman [Wed, 10 May 2023 13:53:18 +0000 (13:53 +0000)]
crimson/osd/osd_meta: Abort instead of throwing runtime_error
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
558758f81e0e8f954de91b6b04dd3e33a29019a4 )
Matan Breizman [Wed, 10 May 2023 13:52:23 +0000 (13:52 +0000)]
crimson/osd: Add logs around handling of osdmaps
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
6dfb8f2fcbe73dc3f1291f729fc7457679fc4c3c )
Matan Breizman [Wed, 24 May 2023 12:52:40 +0000 (12:52 +0000)]
crimson/osd/pg: Fix assert of osd_debug_verify_cached_snaps
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
569fdda551f02f0c97f244d1fef260b3500e655c )
Matan Breizman [Wed, 7 Jun 2023 14:36:37 +0000 (14:36 +0000)]
crimson/osd/osd: handle_osd_map() remove unused ConnectionRef conn
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
3bae740a2e3a5561cbc75c75ab1e52c450045731 )
Matan Breizman [Wed, 7 Jun 2023 12:35:59 +0000 (12:35 +0000)]
crimson/osd/osd: serialize MOSDMap message handling
See included comment for rationale.
Fixes: https://tracker.ceph.com/issues/59165
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
63fa59cbd285da369755412d93022393aad5aa36 )
Radoslaw Zarzynski [Fri, 9 Jun 2023 17:32:27 +0000 (17:32 +0000)]
crimson/osd: suppress the assertion in Watch::notify_ack
Replaced the assert with an error log entry but ultimately
this commit should be reverted.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
76bfae9491134671a40976b4e6fc06d6f7e01306 )
Radosław Zarzyński [Tue, 6 Jun 2023 15:43:21 +0000 (17:43 +0200)]
crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack
Fixes: https://tracker.ceph.com/issues/61504
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
83d1634e192d864630ce927f89c61d8d626923e5 )
Xuehan Xu [Wed, 7 Jun 2023 01:55:38 +0000 (09:55 +0800)]
crimson/os/seastore/cached_extent: add prepare_commit interface
move new fixedkv-btree nodes' inter-node pointer adjustments into
the prepare_commit interface
Fixes: https://tracker.ceph.com/issues/61608
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
f179c72a61bcda48a54efee48cc3975e493d064d )
Samuel Just [Sat, 20 May 2023 00:34:45 +0000 (17:34 -0700)]
crimson/osd/heartbeat: maintain Session::projected_epoch correctly in maybe_share_osdmap
Previously, we only updated projected_map to the value sent by the peer.
This patch simplifies the end by removing an unnecessary guard and
updating projected_epoch before call,ing send_incremental_map_to_osd.
Fixes: https://tracker.ceph.com/issues/61304
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
29a1deb50488b3a1c32040804296e6619d605133 )
Samuel Just [Fri, 19 May 2023 22:36:16 +0000 (15:36 -0700)]
crimson/osd/heartbeat: clarify logging in Heartbeat::maybe_share_osdmap
Reduce verbosity and ensure that all log lines have the peer id.
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
f89b5beaa576c5869ffa565faf7eb846a28aa3ba )
Samuel Just [Fri, 19 May 2023 22:29:25 +0000 (15:29 -0700)]
crimson/osd/heartbeat: clarify variable names in Heartbeat::handle_ping
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
004fc660496a881319c432e443453c64b1ce6524 )
Samuel Just [Fri, 19 May 2023 22:48:03 +0000 (15:48 -0700)]
crimson/osd/heartbeat: rename Session::last_epoch_sent (and accessors) to projected_epoch
last_epoch_sent was a confusing name since we also update it when an
MOSDPing from the peer reflects a higher epoch.
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
a5d20275f2bd9349177f3106bd1d921a10cf8dde )
Zhang Song [Thu, 25 May 2023 08:30:44 +0000 (16:30 +0800)]
crimson/os/seastore/backref_manager: scan backref entries by journal seq
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit
1a6d3c4c4c64a0d86d57651c8b906c67ee7aa8c5 )
Aravind Ramesh [Mon, 15 May 2023 09:22:13 +0000 (11:22 +0200)]
crimson/os/seastore: fix ceph_assert in segment_manager.h
Assert is seen when crimson-osd is compiled with -DWITH_ZNS=ON and
crimson-osd is started with a regular SSD.
ceph/src/crimson/os/seastore/segment_manager.h:77 : In function 'void crimson::os::seastore::block_sm_superblock_t::validate() const', ceph_assert(%s)
get_default_backend_of_device(config.spec.dtype) == backend_type_t::SEGMENTED
The device type "dtype" is not getting propogated properly
to BlockSegmentManager instantiation causing the assert at a
later point.
Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
(cherry picked from commit
cb73d3a9b7a3d6ff5b44e9fd1285aadf7ae787d7 )
Radoslaw Zarzynski [Wed, 15 Mar 2023 09:40:15 +0000 (09:40 +0000)]
crimson/osd: add embedded suppression ruleset for LSan
This commit, basing the idea from the Chromium browser,
embdeds the suppression rules directly into the crimson
executable. The benefit is simplicity and no need to modify
the teuthology code to ship a file across all involved nodes
like already happens for `valgrind.supp`.
From `teuthology/task/install/util.py`:
```
def _ship_utilities(ctx):
"""
Write a copy of valgrind.supp to each of the remote sites. Set executables
used by Ceph in /usr/local/bin. When finished (upon exit of the teuthology
run), remove these files.
:param ctx: Context
"""
testdir = teuthology.get_testdir(ctx)
filenames = []
log.info('Shipping valgrind.supp...')
assert 'suite_path' in ctx.config
try:
with open(
os.path.join(ctx.config['suite_path'], 'valgrind.supp'),
'rb'
) as f:
fn = os.path.join(testdir, 'valgrind.supp')
filenames.append(fn)
for rem in ctx.cluster.remotes.keys():
teuthology.sudo_write_file(
remote=rem,
path=fn,
data=f,
)
f.seek(0)
except IOError as e:
log.info('Cannot ship supression file for valgrind: %s...', e.strerror)
```
With these suppressions `--mkfs` with crimson, BlueStore and tcmalloc
starts returning the proper exit code:
```
[rzarzynski@o06 build]$ /home/rzarzynski/ceph1/build/bin/crimson-osd -i 0 -c /home/rzarzynski/ceph1/build/ceph.conf --mkfs --key AQCZsRBkM5CPIBAACmCbEiP3DPh+x9iiaRDZmA== --osd-uuid
c3d4aea0 -02de-459b-ad00-
fe943906a9a8 --cpuset 0-0
...
created object store /home/rzarzynski/ceph1/build/dev/osd0 for osd.0 fsid
8c9aa852 -1ba2-4800-9695-
e0be4bf877dc
-----------------------------------------------------
Suppressions used:
count bytes template
1 8 InitModule
-----------------------------------------------------
[rzarzynski@o06 build]$ echo $?
0
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
6ed8d839b421442a64410444ca8f88f157ae28b3 )
Aravind Ramesh [Wed, 3 May 2023 09:01:00 +0000 (11:01 +0200)]
crimson/os/seastore: Enable SMR HDD.
SMR HDDs are also zoned block devices similar to ZNS SSDs. SMR
devices have 2 types of zones named as conventional zones which are
random writeable and sequentially-write-required zones, which are
sequentially writable and lend themselves to zone operations like
ZNS SSD zones.
Conventional zones do not respond to zone operations, so to enable
crimson-osd to use SMR HDDs, for now, we don't consider the
conventional zones which generally make up a very small percentage
of the drive capacity.
Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
(cherry picked from commit
355ffa4ef8ea11c57d6498973859683aa67e9729 )
Aravind Ramesh [Tue, 2 May 2023 05:55:37 +0000 (07:55 +0200)]
crimson/os/seastore: change zoned device interface name from ZNS to ZBD
We can essentially support SMR devices(HDD) and ZNS device(SSDs)
which are both subsets of Zoned Block Devices under the same interface
ZNSSegmentManager with small changes in design. Hence changing the
interface name to a more contextual name ZBDSegmentManager.
This also helps to add SMR device support for crimson.
Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
(cherry picked from commit
7fbdeabe38c361a3628e906ab1b9df2c98d62741 )
zdover23 [Wed, 11 Oct 2023 04:10:34 +0000 (14:10 +1000)]
Merge pull request #53932 from zdover23/wip-doc-2023-10-11-backport-53908-to-reef
reef: doc/rados: edit memory-profiling.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Tue, 10 Oct 2023 08:26:53 +0000 (18:26 +1000)]
doc/rados: edit memory-profiling.rst
Edit doc/rados/troubleshooting/memory-profiling.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
3be932425e3020085db5abff6f914938015d8590 )
Yuri Weinstein [Tue, 10 Oct 2023 23:09:50 +0000 (16:09 -0700)]
Merge pull request #53786 from idryomov/wip-63010-reef
reef: qa/suites/krbd: stress test for recovering from watch errors
Reviewed-by: Mykola Golub <mgolub@suse.com>
Adam King [Tue, 10 Oct 2023 18:51:30 +0000 (14:51 -0400)]
Merge pull request #53870 from adk3798/wip-63114-reef
reef: mgr/cephadm: ceph orch add fails when ipv6 address is surrounded by square brackets.
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Adam King [Tue, 10 Oct 2023 18:50:48 +0000 (14:50 -0400)]
Merge pull request #53866 from adk3798/wip-62801-reef
reef: cephadm: run tcmu-runner through script to do restart on failure
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Adam King [Tue, 10 Oct 2023 18:50:02 +0000 (14:50 -0400)]
Merge pull request #53865 from adk3798/wip-62795-reef
reef: mgr/cephadm: don't use image tag in orch upgrade ls
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Adam King [Tue, 10 Oct 2023 18:49:20 +0000 (14:49 -0400)]
Merge pull request #53404 from adk3798/reef-tcmu-custom-configs
reef: cephadm: make custom_configs work for tcmu-runner container
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Rishabh Dave [Tue, 10 Oct 2023 09:36:12 +0000 (15:06 +0530)]
Merge pull request #52819 from rishabh-d-dave/wip-62335-reef
reef: mds: use variable g_ceph_context directly in MDSAuthCaps
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Juan Miguel Olmo [Tue, 10 Oct 2023 09:04:53 +0000 (11:04 +0200)]
Merge pull request #53033 from rhcs-dashboard/wip-62025-reef
reef: msgr: AsyncMessenger add faulted connections metrics
Anthony D'Atri [Mon, 9 Oct 2023 20:48:06 +0000 (16:48 -0400)]
Merge pull request #53897 from zdover23/wip-doc-2023-10-10-backport-53890-to-reef
reef: doc/rados: edit troubleshooting-mon.rst (4 of x)
Adam King [Mon, 9 Oct 2023 20:06:28 +0000 (16:06 -0400)]
Merge pull request #53868 from adk3798/wip-62960-reef
reef: mgr/cephadm: removing double quotes from the generated nvmeof config
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Adam King [Mon, 9 Oct 2023 20:05:35 +0000 (16:05 -0400)]
Merge pull request #53867 from adk3798/wip-62910-reef
reef: mgr/cephadm: Adding sort-by support for ceph orch ps
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Adam King [Mon, 9 Oct 2023 20:04:33 +0000 (16:04 -0400)]
Merge pull request #53751 from adk3798/reef-haproxy-version
reef: cephadm: fix haproxy version with certain containers
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Adam King [Mon, 9 Oct 2023 20:03:37 +0000 (16:03 -0400)]
Merge pull request #53741 from adk3798/wip-63032-reef
reef: mgr/cephadm: fix REFRESHED column of orch ps being unpopulated
Reviewed-by: Michael Fritch <mfritch@suse.com>
Adam King [Mon, 9 Oct 2023 20:02:17 +0000 (16:02 -0400)]
Merge pull request #53159 from ljflores/wip-62590-reef
reef: python-common: drive_selection: fix KeyError when osdspec_affinity is not set
Reviewed-by: Adam King <adking@redhat.com>
Zac Dover [Mon, 9 Oct 2023 08:55:07 +0000 (18:55 +1000)]
doc/rados: edit troubleshooting-mon.rst (4 of x)
Edit doc/rados/troubleshooting/troubleshooting-mon.rst.
Follows https://github.com/ceph/ceph/pull/53875
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
99e92fb94b2c29e565e3f4e04658e103a09ef125 )
Yuri Weinstein [Mon, 9 Oct 2023 15:48:12 +0000 (08:48 -0700)]
Merge pull request #53820 from ljflores/wip-63083-reef
reef: osd: fix read balancer logic to avoid redundant primary assignment
Reviewed-by: Josh Salomon <josh.salomon@gmail.com>
Yuri Weinstein [Mon, 9 Oct 2023 15:44:55 +0000 (08:44 -0700)]
Merge pull request #52767 from kamoltat/wip-ksirivad-backport-reef-52633
reef: pybind/mgr/pg_autoscaler: Cut back osdmap.get_pools calls
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
Yuri Weinstein [Mon, 9 Oct 2023 15:43:50 +0000 (08:43 -0700)]
Merge pull request #52737 from ljflores/wip-62275-reef
reef: test/TestOSDMap: don't use the deprecated std::random_shuffle method
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Yuri Weinstein [Mon, 9 Oct 2023 15:40:20 +0000 (08:40 -0700)]
Merge pull request #52721 from NitzanMordhai/wip-62251-reef
reef: qa/standalone/osd/divergent-prior.sh: Divergent test 3 with pg_autoscale_mode on pick divergent osd
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
Yuri Weinstein [Mon, 9 Oct 2023 15:39:30 +0000 (08:39 -0700)]
Merge pull request #52466 from k0ste/wip-62029-reef
reef: mon/ConfigMonitor: update crush_location from osd entity
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Adam King [Fri, 22 Sep 2023 23:30:26 +0000 (19:30 -0400)]
mgr/cephadm: add unit test for _process_ls_output
This is a weird function to make a unit test for
since it's essentially just moving data from a
list of dicts into a list of DaemonDescriptions,
but wanted to have some coverage to lower the
chance of breaking something again.
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit
47c8ad452423f4bf78885ec911c3b21a52cd90ae )
Conflicts:
src/pybind/mgr/cephadm/tests/test_cephadm.py
Adam King [Fri, 22 Sep 2023 22:34:59 +0000 (18:34 -0400)]
mgr/cephadm: fix REFRESHED column of orch ps being unpopulated
The way the daemon ls data was processed was changed in
https://github.com/ceph/ceph/commit/
1fd4132c7c03602719f29230732b12c8afa04779
and it seems that commit removed a line that set the
last_refresh field. This commit just adds it back
in the new location after the change.
Without this in "ceph orch ps" the REFRESHED column
for every daemon just reports "-"
Fixes: https://tracker.ceph.com/issues/62954
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit
06cf5bb95205965ab6b625b9069a11fa89ed4fb9 )
Anthony D'Atri [Sun, 8 Oct 2023 12:53:04 +0000 (08:53 -0400)]
Merge pull request #53879 from zdover23/wip-doc-2023-10-08-backport-53875-to-reef
reef: doc/rados: edit troubleshooting-mon.rst (3 of x)
Anthony D'Atri [Sun, 8 Oct 2023 12:51:57 +0000 (08:51 -0400)]
Merge pull request #53881 from zdover23/wip-doc-2023-10-08-backport-53874-to-reef
reef: doc/rados: edit troubleshooting/community.rst
Zac Dover [Sat, 7 Oct 2023 20:39:23 +0000 (06:39 +1000)]
doc/rados: edit troubleshooting/community.rst
Edit doc/rados/troubleshooting/community.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
fabfec273474a85cf3687ddf89183f0607c2ad71 )
Zac Dover [Sat, 7 Oct 2023 21:33:17 +0000 (07:33 +1000)]
doc/rados: edit troubleshooting-mon.rst (3 of x)
Edit doc/rados/troubleshooting/troubleshooting-mon.rst.
Follows https://github.com/ceph/ceph/pull/52827
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
fc45a0c4dd0247f472a940d6c92e7acdb0cdc354 )
Anthony D'Atri [Sun, 8 Oct 2023 01:55:25 +0000 (21:55 -0400)]
Merge pull request #53877 from zdover23/wip-doc-2023-10-08-backport-53876-to-reef
reef: doc/architecture: repair RBD sentence
Zac Dover [Sat, 7 Oct 2023 21:43:43 +0000 (07:43 +1000)]
doc/architecture: repair RBD sentence
Improve an ambiguous sentence in doc/architecture.rst.
The problem presented by the original sentence is that the phrasal verb
"to provide with" is implicated in one of its possible readings.
Interpreted in that way, the sentence seems to express the incorrect
idea that RBD furnishes block devices with snapshotting and cloning, as
though snapshotting and cloning are being delivered to the block
devices. In fact, snapshotting and cloning are just features of RBD, and
are features that are described on this page:
https://docs.ceph.com/en/quincy/rbd/rbd-snapshot/.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
5abd530460e3d5a60142d9350d93544402b6dec4 )
zdover23 [Sat, 7 Oct 2023 05:33:19 +0000 (16:33 +1100)]
Merge pull request #53871 from zdover23/wip-doc-2023-10-07-backport-53790-to-reef
reef: doc/architecture: edit "Peering and Sets"
Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Zac Dover [Tue, 3 Oct 2023 12:11:50 +0000 (22:11 +1000)]
doc/architecture: edit "Peering and Sets"
Edit the English in the section "Peering and Sets" in the file
doc/architecture.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
c69b111966f190ac4a1ebd5244a9cf9f61970dd7 )
zdover23 [Sat, 7 Oct 2023 01:54:33 +0000 (12:54 +1100)]
Merge pull request #53837 from zdover23/wip-doc-2023-10-05-backport-53819-to-reef
reef: doc/rados: edit troubleshooting.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Anthony D'Atri [Fri, 6 Oct 2023 22:15:58 +0000 (18:15 -0400)]
Merge pull request #53869 from adk3798/wip-62973-reef
reef: mgr/cephadm: add ability to zap OSDs' devices while draining host
Teoman ONAY [Mon, 3 Jul 2023 14:00:20 +0000 (16:00 +0200)]
ceph orch add fails when ipv6 address is surrounded by square brackets.
fixes: https://tracker.ceph.com/issues/61885
fixes: https://bugzilla.redhat.com/show_bug.cgi?id=
2153448
Signed-off-by: Teoman ONAY <tonay@ibm.com>
(cherry picked from commit
1ea71bee6197ed0357b586498a43d9d726160a43 )
Adam King [Mon, 5 Jun 2023 19:05:55 +0000 (15:05 -0400)]
mgr/cephadm: add ability to zap OSDs' devices while draining host
Currently, when cephadm drains a host, it will remove all OSDs on
the host, but provides no option to zap the OSD's devices afterwards.
Given users are draining the host likely to remove it from the cluster,
it makes sense some users would want to clean up the devices on the
host that were being used for OSDs. Cephadm already supports zapping
devices outside of host draining, so it makes shouldn't take much to
add that functionality to the host drain as well.
Fixes: https://tracker.ceph.com/issues/61593
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit
85043ff4cee108c152f5aa8af267c85e353c475a )
Redouane Kachach [Thu, 21 Sep 2023 08:13:13 +0000 (10:13 +0200)]
mgr/cephadm: removing double quotes from the generated nvmeof config
Fixes: https://tracker.ceph.com/issues/62838
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit
6b8187b2ec8a3bf9092da72c7a8b00dc7e5e1ce3 )
Redouane Kachach [Wed, 13 Sep 2023 11:46:42 +0000 (13:46 +0200)]
Adding service_name and daemon_type and fixing mypy issues
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit
72e9345faa725bd2aaba41e06ed34927badd7622 )
Redouane Kachach [Thu, 29 Jun 2023 10:02:21 +0000 (12:02 +0200)]
mgr/cephadm: Adding sort-by support for ceph orch ps
Fixes: https://tracker.ceph.com/issues/61856
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit
b347106af329e238a927f47436acd54999c4d9af )