]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
21 months agoMerge pull request #53950 from Matan-B/wip-51852-reef
Matan Breizman [Tue, 17 Oct 2023 11:36:49 +0000 (14:36 +0300)]
Merge pull request #53950 from Matan-B/wip-51852-reef

reef: crimson/osd/watch: Add logs around Watch/Notify

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53948 from Matan-B/wip-52032-reef
Matan Breizman [Tue, 17 Oct 2023 11:34:49 +0000 (14:34 +0300)]
Merge pull request #53948 from Matan-B/wip-52032-reef

reef: crimson/osd: don't ignore start_pg_operation returned future

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53945 from Matan-B/wip-51980-reef
Matan Breizman [Tue, 17 Oct 2023 11:29:58 +0000 (14:29 +0300)]
Merge pull request #53945 from Matan-B/wip-51980-reef

reef: crimson/os/seastore: fix in check_node

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53944 from Matan-B/wip-51961-reef
Matan Breizman [Tue, 17 Oct 2023 11:28:09 +0000 (14:28 +0300)]
Merge pull request #53944 from Matan-B/wip-51961-reef

reef: Crimson/osd: Disable concurrent MOSDMap handling

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53943 from Matan-B/wip-51945-reef
Matan Breizman [Tue, 17 Oct 2023 11:27:31 +0000 (14:27 +0300)]
Merge pull request #53943 from Matan-B/wip-51945-reef

reef: crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53941 from Matan-B/wip-51948-reef
Matan Breizman [Tue, 17 Oct 2023 11:21:04 +0000 (14:21 +0300)]
Merge pull request #53941 from Matan-B/wip-51948-reef

reef: crimson/os/seastore/cached_extent: add prepare_commit interface

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53940 from Matan-B/wip-51380-reef
Matan Breizman [Tue, 17 Oct 2023 11:20:38 +0000 (14:20 +0300)]
Merge pull request #53940 from Matan-B/wip-51380-reef

reef: crimson/osd/heartbeat: Improve maybe_share_osdmap behavior

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53939 from Matan-B/wip-51750-reef
Matan Breizman [Tue, 17 Oct 2023 11:19:53 +0000 (14:19 +0300)]
Merge pull request #53939 from Matan-B/wip-51750-reef

reef: crimson/os/seastore/backref_manager: scan backref entries by journal seq

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53938 from Matan-B/wip-51475-reef
Matan Breizman [Tue, 17 Oct 2023 11:19:10 +0000 (14:19 +0300)]
Merge pull request #53938 from Matan-B/wip-51475-reef

reef: crimson/os/seastore: fix ceph_assert in segment_manager.h

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53935 from Matan-B/wip-51355-reef
Matan Breizman [Tue, 17 Oct 2023 11:15:39 +0000 (14:15 +0300)]
Merge pull request #53935 from Matan-B/wip-51355-reef

reef: crimson/os/seastore: enable SMR HDD

Reviewed-by: Samuel Just <sjust@redhat.com>
21 months agoMerge pull request #53797 from smanjara/ragweed-test-reef
Yuri Weinstein [Mon, 16 Oct 2023 19:37:47 +0000 (12:37 -0700)]
Merge pull request #53797 from smanjara/ragweed-test-reef

reef: rgw/upgrade: point upgrade suites to ragweed ceph-reef branch

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53773 from cbodley/wip-63057-reef
Yuri Weinstein [Mon, 16 Oct 2023 19:37:03 +0000 (12:37 -0700)]
Merge pull request #53773 from cbodley/wip-63057-reef

reef: rgw: improve buffer list utilization in the chunkupload scenario

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53770 from cbodley/wip-63054-reef
Yuri Weinstein [Mon, 16 Oct 2023 19:35:12 +0000 (12:35 -0700)]
Merge pull request #53770 from cbodley/wip-63054-reef

reef: rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53767 from cbodley/wip-63051-reef
Yuri Weinstein [Mon, 16 Oct 2023 19:34:05 +0000 (12:34 -0700)]
Merge pull request #53767 from cbodley/wip-63051-reef

reef: rgw: s3website doesn't prefetch for web_dir() check

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53765 from cbodley/wip-63048-reef
Yuri Weinstein [Mon, 16 Oct 2023 19:33:06 +0000 (12:33 -0700)]
Merge pull request #53765 from cbodley/wip-63048-reef

reef: rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_rea…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53760 from cbodley/wip-63060-reef
Yuri Weinstein [Mon, 16 Oct 2023 19:32:15 +0000 (12:32 -0700)]
Merge pull request #53760 from cbodley/wip-63060-reef

reef: rgw: fix unwatch crash at radosgw startup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53756 from cbodley/wip-63042-reef
Yuri Weinstein [Mon, 16 Oct 2023 19:31:29 +0000 (12:31 -0700)]
Merge pull request #53756 from cbodley/wip-63042-reef

reef: [CVE-2023-43040] rgw: Fix bucket validation against POST policies

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
21 months agoMerge pull request #53681 from mkogan1/wip-t62141-reef
Yuri Weinstein [Mon, 16 Oct 2023 19:30:28 +0000 (12:30 -0700)]
Merge pull request #53681 from mkogan1/wip-t62141-reef

reef: valgrind: UninitCondition under __run_exit_handlers suppression

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #54026 from zdover23/wip-doc-2023-10-16-backport-54022-to-reef
Anthony D'Atri [Sun, 15 Oct 2023 23:54:09 +0000 (19:54 -0400)]
Merge pull request #54026 from zdover23/wip-doc-2023-10-16-backport-54022-to-reef

reef: doc/rados: Edit troubleshooting-osd (3 of x)

21 months agodoc/rados: Edit troubleshooting-osd (3 of x) 54026/head
Zac Dover [Sun, 15 Oct 2023 12:43:39 +0000 (22:43 +1000)]
doc/rados: Edit troubleshooting-osd (3 of x)

Edit doc/rados/troubleshooting/troubleshooting-osd.rst (3 of x).

Follows https://github.com/ceph/ceph/pull/53986.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 01b5aa5d27a97a02cef03b9eb0ee9dbcd55684f1)

22 months agoMerge pull request #54008 from zdover23/wip-doc-2023-10-14-backport-54006-to-reef
Anthony D'Atri [Fri, 13 Oct 2023 20:31:28 +0000 (16:31 -0400)]
Merge pull request #54008 from zdover23/wip-doc-2023-10-14-backport-54006-to-reef

doc/rados: remove FileStore material (in Reef)

22 months agodoc/rados: remove FileStore material (in Reef) 54008/head
Zac Dover [Fri, 13 Oct 2023 18:40:26 +0000 (04:40 +1000)]
doc/rados: remove FileStore material (in Reef)

Remove FileStore-related material because FileStore is not supported in
Reef.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit eab0be5ca513ca2dfaeb31911c175e88bcf4f9e4)

22 months agoMerge pull request #54007 from zdover23/wip-doc-2023-10-14-backport-54005-to-reef-2
Anthony D'Atri [Fri, 13 Oct 2023 20:15:24 +0000 (16:15 -0400)]
Merge pull request #54007 from zdover23/wip-doc-2023-10-14-backport-54005-to-reef-2

reef: doc/dev: remove cache-pool

22 months agodoc/dev: remove cache-pool 54007/head
Zac Dover [Fri, 13 Oct 2023 18:17:44 +0000 (04:17 +1000)]
doc/dev: remove cache-pool

Remove cache-pool, which is deprecated in Reef.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 073b0919ed9e413a04a7e12b7a0c23b5e058cd52)

22 months agoMerge pull request #54000 from zdover23/wip-doc-2023-10-13-backport-53986-to-reef
zdover23 [Fri, 13 Oct 2023 14:01:03 +0000 (00:01 +1000)]
Merge pull request #54000 from zdover23/wip-doc-2023-10-13-backport-53986-to-reef

reef: doc/rados: Edit troubleshooting-osd (2 of x)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
22 months agodoc/rados: Edit troubleshooting-osd (2 of x) 54000/head
Zac Dover [Thu, 12 Oct 2023 08:33:58 +0000 (18:33 +1000)]
doc/rados: Edit troubleshooting-osd (2 of x)

Edit doc/rados/troubleshooting/troubleshooting.rst (2 of x).

Follows https://github.com/ceph/ceph/pull/53936.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 37e7099267996a3075b4902a10a19d94fc738c08)

22 months agoMerge pull request #53458 from rhcs-dashboard/wip-62602-reef
Nizamudeen A [Fri, 13 Oct 2023 05:35:08 +0000 (11:05 +0530)]
Merge pull request #53458 from rhcs-dashboard/wip-62602-reef

reef: mgr/dashboard: remove unnecessary failing hosts e2e

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
22 months agoMerge pull request #53982 from zdover23/wip-doc-2023-10-12-backport-53936-to-reef
Anthony D'Atri [Thu, 12 Oct 2023 09:02:48 +0000 (05:02 -0400)]
Merge pull request #53982 from zdover23/wip-doc-2023-10-12-backport-53936-to-reef

reef: doc/rados: edit troubleshooting-osd (1 of x)

22 months agodoc/rados: edit troubleshooting-osd (1 of x) 53982/head
Zac Dover [Wed, 11 Oct 2023 10:38:55 +0000 (20:38 +1000)]
doc/rados: edit troubleshooting-osd (1 of x)

Edit doc/rados/troubleshooting/troubleshooting-osd.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0fc663d793fe34e42e19b251fcc79b5af52581c6)

22 months agocrimson/osd/watch: don't print bufferlists 53950/head
Matan Breizman [Wed, 31 May 2023 11:47:45 +0000 (11:47 +0000)]
crimson/osd/watch: don't print bufferlists

The bufferlists clutter the logs and don't infrom of anything useful:
```
DEBUG 2023-05-26 23:36:29,780 [shard 2] osd - send_completion sending notify replies: {notify_reply_t{watcher_gid=4220, watcher_cookie=139786421930752, bl=buffer::list(len=0,

)}, notify_reply_t{watcher_gid=4220, watcher_cookie=139786422032096, bl=buffer::list(len=0,

)}}
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 3548489f622c50ccaa1d8c351e420873743ed980)

22 months agocrimson/osd/watch: Add logs around Watch/Notify
Matan Breizman [Wed, 31 May 2023 08:45:53 +0000 (08:45 +0000)]
crimson/osd/watch: Add logs around Watch/Notify

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit b1c05f9acecfd2dffb57d63f42fec9da156ff366)

22 months agocrimson/osd: don't ignore start_pg_operation returned future 53948/head
Matan Breizman [Tue, 13 Jun 2023 09:58:34 +0000 (09:58 +0000)]
crimson/osd: don't ignore start_pg_operation returned future

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 43bf04e6e9cb82fa7cfe5644786d91e6265bcc8a)

22 months agocrimson/os/seastore: fix bug in check_node 53945/head
Xinyu Huang [Fri, 9 Jun 2023 07:25:48 +0000 (15:25 +0800)]
crimson/os/seastore: fix bug in check_node

EXIST_CLEAN and EXIST_MUTATION_PENDING shuold not be treated as
CLEAN in check_node because they are transaction private and the
leafnode has been duplicated for write.

fix: https://tracker.ceph.com/issues/61626

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
(cherry picked from commit f02f20c2aecfbcb24d4a2d49db159a15b63487f4)

22 months agocrimson/osd/osd_operations/pg_advance_map: 'from' epoch may vary between PGS explanation 53944/head
Matan Breizman [Tue, 20 Jun 2023 17:51:52 +0000 (17:51 +0000)]
crimson/osd/osd_operations/pg_advance_map: 'from' epoch may vary between PGS explanation

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit ba519f5e72350e86647b7d9bc40fc2d26d908da3)

22 months agocrimson/osd/osdmap_gate: got_map() assert increase of current epoch
Matan Breizman [Sun, 28 May 2023 10:31:37 +0000 (10:31 +0000)]
crimson/osd/osdmap_gate: got_map() assert increase of current epoch

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 06bf66154386662cd7bcccbef6cd7a2bdeb6e539)

22 months agocrimson/osd/shard_services: get_or_create_pg() remove unused variable
Matan Breizman [Wed, 24 May 2023 15:36:10 +0000 (15:36 +0000)]
crimson/osd/shard_services: get_or_create_pg() remove unused variable

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit a8800c7e700f0ed262885906ee59dcaa15230932)

22 months agocrimson/osd/shard_services: Refactor OSDSingletonState::load_map
Matan Breizman [Wed, 10 May 2023 13:54:17 +0000 (13:54 +0000)]
crimson/osd/shard_services: Refactor OSDSingletonState::load_map

Added logs
If case reconstructed - epoch is unsigned.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit ecac14a70031299d744fecb7b6c3fb464a346a80)

22 months agocrimson/osd/osd_meta: Abort instead of throwing runtime_error
Matan Breizman [Wed, 10 May 2023 13:53:18 +0000 (13:53 +0000)]
crimson/osd/osd_meta: Abort instead of throwing runtime_error

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 558758f81e0e8f954de91b6b04dd3e33a29019a4)

22 months agocrimson/osd: Add logs around handling of osdmaps
Matan Breizman [Wed, 10 May 2023 13:52:23 +0000 (13:52 +0000)]
crimson/osd: Add logs around handling of osdmaps

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 6dfb8f2fcbe73dc3f1291f729fc7457679fc4c3c)

22 months agocrimson/osd/pg: Fix assert of osd_debug_verify_cached_snaps
Matan Breizman [Wed, 24 May 2023 12:52:40 +0000 (12:52 +0000)]
crimson/osd/pg: Fix assert of osd_debug_verify_cached_snaps

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 569fdda551f02f0c97f244d1fef260b3500e655c)

22 months agocrimson/osd/osd: handle_osd_map() remove unused ConnectionRef conn
Matan Breizman [Wed, 7 Jun 2023 14:36:37 +0000 (14:36 +0000)]
crimson/osd/osd: handle_osd_map() remove unused ConnectionRef conn

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 3bae740a2e3a5561cbc75c75ab1e52c450045731)

22 months agocrimson/osd/osd: serialize MOSDMap message handling
Matan Breizman [Wed, 7 Jun 2023 12:35:59 +0000 (12:35 +0000)]
crimson/osd/osd: serialize MOSDMap message handling

See included comment for rationale.

Fixes: https://tracker.ceph.com/issues/59165
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 63fa59cbd285da369755412d93022393aad5aa36)

22 months agocrimson/osd: suppress the assertion in Watch::notify_ack 53943/head
Radoslaw Zarzynski [Fri, 9 Jun 2023 17:32:27 +0000 (17:32 +0000)]
crimson/osd: suppress the assertion in Watch::notify_ack

Replaced the assert with an error log entry but ultimately
this commit should be reverted.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 76bfae9491134671a40976b4e6fc06d6f7e01306)

22 months agocrimson/osd: fix Notify life-time mismanagement in Watch::notify_ack
Radosław Zarzyński [Tue, 6 Jun 2023 15:43:21 +0000 (17:43 +0200)]
crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack

Fixes: https://tracker.ceph.com/issues/61504
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 83d1634e192d864630ce927f89c61d8d626923e5)

22 months agocrimson/os/seastore/cached_extent: add prepare_commit interface 53941/head
Xuehan Xu [Wed, 7 Jun 2023 01:55:38 +0000 (09:55 +0800)]
crimson/os/seastore/cached_extent: add prepare_commit interface

move new fixedkv-btree nodes' inter-node pointer adjustments into
the prepare_commit interface

Fixes: https://tracker.ceph.com/issues/61608
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit f179c72a61bcda48a54efee48cc3975e493d064d)

22 months agocrimson/osd/heartbeat: maintain Session::projected_epoch correctly in maybe_share_osdmap 53940/head
Samuel Just [Sat, 20 May 2023 00:34:45 +0000 (17:34 -0700)]
crimson/osd/heartbeat: maintain Session::projected_epoch correctly in maybe_share_osdmap

Previously, we only updated projected_map to the value sent by the peer.
This patch simplifies the end by removing an unnecessary guard and
updating projected_epoch before call,ing send_incremental_map_to_osd.

Fixes: https://tracker.ceph.com/issues/61304
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 29a1deb50488b3a1c32040804296e6619d605133)

22 months agocrimson/osd/heartbeat: clarify logging in Heartbeat::maybe_share_osdmap
Samuel Just [Fri, 19 May 2023 22:36:16 +0000 (15:36 -0700)]
crimson/osd/heartbeat: clarify logging in Heartbeat::maybe_share_osdmap

Reduce verbosity and ensure that all log lines have the peer id.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit f89b5beaa576c5869ffa565faf7eb846a28aa3ba)

22 months agocrimson/osd/heartbeat: clarify variable names in Heartbeat::handle_ping
Samuel Just [Fri, 19 May 2023 22:29:25 +0000 (15:29 -0700)]
crimson/osd/heartbeat: clarify variable names in Heartbeat::handle_ping

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 004fc660496a881319c432e443453c64b1ce6524)

22 months agocrimson/osd/heartbeat: rename Session::last_epoch_sent (and accessors) to projected_epoch
Samuel Just [Fri, 19 May 2023 22:48:03 +0000 (15:48 -0700)]
crimson/osd/heartbeat: rename Session::last_epoch_sent (and accessors) to projected_epoch

last_epoch_sent was a confusing name since we also update it when an
MOSDPing from the peer reflects a higher epoch.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit a5d20275f2bd9349177f3106bd1d921a10cf8dde)

22 months agocrimson/os/seastore/backref_manager: scan backref entries by journal seq 53939/head
Zhang Song [Thu, 25 May 2023 08:30:44 +0000 (16:30 +0800)]
crimson/os/seastore/backref_manager: scan backref entries by journal seq

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 1a6d3c4c4c64a0d86d57651c8b906c67ee7aa8c5)

22 months agocrimson/os/seastore: fix ceph_assert in segment_manager.h 53938/head
Aravind Ramesh [Mon, 15 May 2023 09:22:13 +0000 (11:22 +0200)]
crimson/os/seastore: fix ceph_assert in segment_manager.h

Assert is seen when crimson-osd is compiled with -DWITH_ZNS=ON and
crimson-osd is started with a regular SSD.

ceph/src/crimson/os/seastore/segment_manager.h:77 : In function 'void crimson::os::seastore::block_sm_superblock_t::validate() const', ceph_assert(%s)
get_default_backend_of_device(config.spec.dtype) == backend_type_t::SEGMENTED

The device type "dtype" is not getting propogated properly
to BlockSegmentManager instantiation causing the assert at a
later point.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
(cherry picked from commit cb73d3a9b7a3d6ff5b44e9fd1285aadf7ae787d7)

22 months agocrimson/os/seastore: Enable SMR HDD. 53935/head
Aravind Ramesh [Wed, 3 May 2023 09:01:00 +0000 (11:01 +0200)]
crimson/os/seastore: Enable SMR HDD.

SMR HDDs are also zoned block devices similar to ZNS SSDs. SMR
devices have 2 types of zones named as conventional zones which are
random writeable and sequentially-write-required zones, which are
sequentially writable and lend themselves to zone operations like
ZNS SSD zones.

Conventional zones do not respond to zone operations, so to enable
crimson-osd to use SMR HDDs, for now, we don't consider the
conventional zones which generally make up a very small percentage
of the drive capacity.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
(cherry picked from commit 355ffa4ef8ea11c57d6498973859683aa67e9729)

22 months agocrimson/os/seastore: change zoned device interface name from ZNS to ZBD
Aravind Ramesh [Tue, 2 May 2023 05:55:37 +0000 (07:55 +0200)]
crimson/os/seastore: change zoned device interface name from ZNS to ZBD

We can essentially support SMR devices(HDD) and ZNS device(SSDs)
which are both subsets of Zoned Block Devices under the same interface
ZNSSegmentManager with small changes in design. Hence changing the
interface name to a more contextual name ZBDSegmentManager.
This also helps to add SMR device support for crimson.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
(cherry picked from commit 7fbdeabe38c361a3628e906ab1b9df2c98d62741)

22 months agoMerge pull request #53932 from zdover23/wip-doc-2023-10-11-backport-53908-to-reef
zdover23 [Wed, 11 Oct 2023 04:10:34 +0000 (14:10 +1000)]
Merge pull request #53932 from zdover23/wip-doc-2023-10-11-backport-53908-to-reef

reef: doc/rados: edit memory-profiling.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agodoc/rados: edit memory-profiling.rst 53932/head
Zac Dover [Tue, 10 Oct 2023 08:26:53 +0000 (18:26 +1000)]
doc/rados: edit memory-profiling.rst

Edit doc/rados/troubleshooting/memory-profiling.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 3be932425e3020085db5abff6f914938015d8590)

22 months agoMerge pull request #53786 from idryomov/wip-63010-reef
Yuri Weinstein [Tue, 10 Oct 2023 23:09:50 +0000 (16:09 -0700)]
Merge pull request #53786 from idryomov/wip-63010-reef

reef: qa/suites/krbd: stress test for recovering from watch errors

Reviewed-by: Mykola Golub <mgolub@suse.com>
22 months agoMerge pull request #53870 from adk3798/wip-63114-reef
Adam King [Tue, 10 Oct 2023 18:51:30 +0000 (14:51 -0400)]
Merge pull request #53870 from adk3798/wip-63114-reef

reef: mgr/cephadm: ceph orch add fails when ipv6 address is surrounded by square brackets.

Reviewed-by: John Mulligan <jmulligan@redhat.com>
22 months agoMerge pull request #53866 from adk3798/wip-62801-reef
Adam King [Tue, 10 Oct 2023 18:50:48 +0000 (14:50 -0400)]
Merge pull request #53866 from adk3798/wip-62801-reef

reef: cephadm: run tcmu-runner through script to do restart on failure

Reviewed-by: John Mulligan <jmulligan@redhat.com>
22 months agoMerge pull request #53865 from adk3798/wip-62795-reef
Adam King [Tue, 10 Oct 2023 18:50:02 +0000 (14:50 -0400)]
Merge pull request #53865 from adk3798/wip-62795-reef

reef: mgr/cephadm: don't use image tag in orch upgrade ls

Reviewed-by: John Mulligan <jmulligan@redhat.com>
22 months agoMerge pull request #53404 from adk3798/reef-tcmu-custom-configs
Adam King [Tue, 10 Oct 2023 18:49:20 +0000 (14:49 -0400)]
Merge pull request #53404 from adk3798/reef-tcmu-custom-configs

reef: cephadm: make custom_configs work for tcmu-runner container

Reviewed-by: John Mulligan <jmulligan@redhat.com>
22 months agoMerge pull request #52819 from rishabh-d-dave/wip-62335-reef
Rishabh Dave [Tue, 10 Oct 2023 09:36:12 +0000 (15:06 +0530)]
Merge pull request #52819 from rishabh-d-dave/wip-62335-reef

reef: mds: use variable g_ceph_context directly in MDSAuthCaps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
22 months agoMerge pull request #53033 from rhcs-dashboard/wip-62025-reef
Juan Miguel Olmo [Tue, 10 Oct 2023 09:04:53 +0000 (11:04 +0200)]
Merge pull request #53033 from rhcs-dashboard/wip-62025-reef

reef: msgr: AsyncMessenger add faulted connections metrics

22 months agoMerge pull request #53897 from zdover23/wip-doc-2023-10-10-backport-53890-to-reef
Anthony D'Atri [Mon, 9 Oct 2023 20:48:06 +0000 (16:48 -0400)]
Merge pull request #53897 from zdover23/wip-doc-2023-10-10-backport-53890-to-reef

reef: doc/rados: edit troubleshooting-mon.rst (4 of x)

22 months agoMerge pull request #53868 from adk3798/wip-62960-reef
Adam King [Mon, 9 Oct 2023 20:06:28 +0000 (16:06 -0400)]
Merge pull request #53868 from adk3798/wip-62960-reef

reef: mgr/cephadm: removing double quotes from the generated nvmeof config

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
22 months agoMerge pull request #53867 from adk3798/wip-62910-reef
Adam King [Mon, 9 Oct 2023 20:05:35 +0000 (16:05 -0400)]
Merge pull request #53867 from adk3798/wip-62910-reef

reef: mgr/cephadm: Adding sort-by support for ceph orch ps

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
22 months agoMerge pull request #53751 from adk3798/reef-haproxy-version
Adam King [Mon, 9 Oct 2023 20:04:33 +0000 (16:04 -0400)]
Merge pull request #53751 from adk3798/reef-haproxy-version

reef: cephadm: fix haproxy version with certain containers

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
22 months agoMerge pull request #53741 from adk3798/wip-63032-reef
Adam King [Mon, 9 Oct 2023 20:03:37 +0000 (16:03 -0400)]
Merge pull request #53741 from adk3798/wip-63032-reef

reef: mgr/cephadm: fix REFRESHED column of orch ps being unpopulated

Reviewed-by: Michael Fritch <mfritch@suse.com>
22 months agoMerge pull request #53159 from ljflores/wip-62590-reef
Adam King [Mon, 9 Oct 2023 20:02:17 +0000 (16:02 -0400)]
Merge pull request #53159 from ljflores/wip-62590-reef

reef: python-common: drive_selection: fix KeyError when osdspec_affinity is not set

Reviewed-by: Adam King <adking@redhat.com>
22 months agodoc/rados: edit troubleshooting-mon.rst (4 of x) 53897/head
Zac Dover [Mon, 9 Oct 2023 08:55:07 +0000 (18:55 +1000)]
doc/rados: edit troubleshooting-mon.rst (4 of x)

Edit doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/53875

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 99e92fb94b2c29e565e3f4e04658e103a09ef125)

22 months agoMerge pull request #53820 from ljflores/wip-63083-reef
Yuri Weinstein [Mon, 9 Oct 2023 15:48:12 +0000 (08:48 -0700)]
Merge pull request #53820 from ljflores/wip-63083-reef

reef: osd: fix read balancer logic to avoid redundant primary assignment

Reviewed-by: Josh Salomon <josh.salomon@gmail.com>
22 months agoMerge pull request #52767 from kamoltat/wip-ksirivad-backport-reef-52633
Yuri Weinstein [Mon, 9 Oct 2023 15:44:55 +0000 (08:44 -0700)]
Merge pull request #52767 from kamoltat/wip-ksirivad-backport-reef-52633

reef: pybind/mgr/pg_autoscaler: Cut back osdmap.get_pools calls

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
22 months agoMerge pull request #52737 from ljflores/wip-62275-reef
Yuri Weinstein [Mon, 9 Oct 2023 15:43:50 +0000 (08:43 -0700)]
Merge pull request #52737 from ljflores/wip-62275-reef

reef: test/TestOSDMap: don't use the deprecated std::random_shuffle method

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
22 months agoMerge pull request #52721 from NitzanMordhai/wip-62251-reef
Yuri Weinstein [Mon, 9 Oct 2023 15:40:20 +0000 (08:40 -0700)]
Merge pull request #52721 from NitzanMordhai/wip-62251-reef

reef: qa/standalone/osd/divergent-prior.sh: Divergent test 3 with pg_autoscale_mode on pick divergent osd

Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
22 months agoMerge pull request #52466 from k0ste/wip-62029-reef
Yuri Weinstein [Mon, 9 Oct 2023 15:39:30 +0000 (08:39 -0700)]
Merge pull request #52466 from k0ste/wip-62029-reef

reef: mon/ConfigMonitor: update crush_location from osd entity

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
22 months agomgr/cephadm: add unit test for _process_ls_output 53741/head
Adam King [Fri, 22 Sep 2023 23:30:26 +0000 (19:30 -0400)]
mgr/cephadm: add unit test for _process_ls_output

This is a weird function to make a unit test for
since it's essentially just moving data from a
list of dicts into a list of DaemonDescriptions,
but wanted to have some coverage to lower the
chance of breaking something again.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 47c8ad452423f4bf78885ec911c3b21a52cd90ae)

Conflicts:
src/pybind/mgr/cephadm/tests/test_cephadm.py

22 months agomgr/cephadm: fix REFRESHED column of orch ps being unpopulated
Adam King [Fri, 22 Sep 2023 22:34:59 +0000 (18:34 -0400)]
mgr/cephadm: fix REFRESHED column of orch ps being unpopulated

The way the daemon ls data was processed was changed in
https://github.com/ceph/ceph/commit/1fd4132c7c03602719f29230732b12c8afa04779
and it seems that commit removed a line that set the
last_refresh field. This commit just adds it back
in the new location after the change.

Without this in "ceph orch ps" the REFRESHED column
for every daemon just reports "-"

Fixes: https://tracker.ceph.com/issues/62954
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 06cf5bb95205965ab6b625b9069a11fa89ed4fb9)

22 months agoMerge pull request #53879 from zdover23/wip-doc-2023-10-08-backport-53875-to-reef
Anthony D'Atri [Sun, 8 Oct 2023 12:53:04 +0000 (08:53 -0400)]
Merge pull request #53879 from zdover23/wip-doc-2023-10-08-backport-53875-to-reef

reef: doc/rados: edit troubleshooting-mon.rst (3 of x)

22 months agoMerge pull request #53881 from zdover23/wip-doc-2023-10-08-backport-53874-to-reef
Anthony D'Atri [Sun, 8 Oct 2023 12:51:57 +0000 (08:51 -0400)]
Merge pull request #53881 from zdover23/wip-doc-2023-10-08-backport-53874-to-reef

reef: doc/rados: edit troubleshooting/community.rst

22 months agodoc/rados: edit troubleshooting/community.rst 53881/head
Zac Dover [Sat, 7 Oct 2023 20:39:23 +0000 (06:39 +1000)]
doc/rados: edit troubleshooting/community.rst

Edit doc/rados/troubleshooting/community.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit fabfec273474a85cf3687ddf89183f0607c2ad71)

22 months agodoc/rados: edit troubleshooting-mon.rst (3 of x) 53879/head
Zac Dover [Sat, 7 Oct 2023 21:33:17 +0000 (07:33 +1000)]
doc/rados: edit troubleshooting-mon.rst (3 of x)

Edit doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/52827

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit fc45a0c4dd0247f472a940d6c92e7acdb0cdc354)

22 months agoMerge pull request #53877 from zdover23/wip-doc-2023-10-08-backport-53876-to-reef
Anthony D'Atri [Sun, 8 Oct 2023 01:55:25 +0000 (21:55 -0400)]
Merge pull request #53877 from zdover23/wip-doc-2023-10-08-backport-53876-to-reef

reef: doc/architecture: repair RBD sentence

22 months agodoc/architecture: repair RBD sentence 53877/head
Zac Dover [Sat, 7 Oct 2023 21:43:43 +0000 (07:43 +1000)]
doc/architecture: repair RBD sentence

Improve an ambiguous sentence in doc/architecture.rst.

The problem presented by the original sentence is that the phrasal verb
"to provide with" is implicated in one of its possible readings.
Interpreted in that way, the sentence seems to express the incorrect
idea that RBD furnishes block devices with snapshotting and cloning, as
though snapshotting and cloning are being delivered to the block
devices. In fact, snapshotting and cloning are just features of RBD, and
are features that are described on this page:
https://docs.ceph.com/en/quincy/rbd/rbd-snapshot/.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 5abd530460e3d5a60142d9350d93544402b6dec4)

22 months agoMerge pull request #53871 from zdover23/wip-doc-2023-10-07-backport-53790-to-reef
zdover23 [Sat, 7 Oct 2023 05:33:19 +0000 (16:33 +1100)]
Merge pull request #53871 from zdover23/wip-doc-2023-10-07-backport-53790-to-reef

reef: doc/architecture: edit "Peering and Sets"

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
22 months agodoc/architecture: edit "Peering and Sets" 53871/head
Zac Dover [Tue, 3 Oct 2023 12:11:50 +0000 (22:11 +1000)]
doc/architecture: edit "Peering and Sets"

Edit the English in the section "Peering and Sets" in the file
doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c69b111966f190ac4a1ebd5244a9cf9f61970dd7)

22 months agoMerge pull request #53837 from zdover23/wip-doc-2023-10-05-backport-53819-to-reef
zdover23 [Sat, 7 Oct 2023 01:54:33 +0000 (12:54 +1100)]
Merge pull request #53837 from zdover23/wip-doc-2023-10-05-backport-53819-to-reef

reef: doc/rados: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agoMerge pull request #53869 from adk3798/wip-62973-reef
Anthony D'Atri [Fri, 6 Oct 2023 22:15:58 +0000 (18:15 -0400)]
Merge pull request #53869 from adk3798/wip-62973-reef

reef: mgr/cephadm: add ability to zap OSDs' devices while draining host

22 months agoceph orch add fails when ipv6 address is surrounded by square brackets. 53870/head
Teoman ONAY [Mon, 3 Jul 2023 14:00:20 +0000 (16:00 +0200)]
ceph orch add fails when ipv6 address is surrounded by square brackets.

fixes: https://tracker.ceph.com/issues/61885
fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2153448

Signed-off-by: Teoman ONAY <tonay@ibm.com>
(cherry picked from commit 1ea71bee6197ed0357b586498a43d9d726160a43)

22 months agomgr/cephadm: add ability to zap OSDs' devices while draining host 53869/head
Adam King [Mon, 5 Jun 2023 19:05:55 +0000 (15:05 -0400)]
mgr/cephadm: add ability to zap OSDs' devices while draining host

Currently, when cephadm drains a host, it will remove all OSDs on
the host, but provides no option to zap the OSD's devices afterwards.
Given users are draining the host likely to remove it from the cluster,
it makes sense some users would want to clean up the devices on the
host that were being used for OSDs. Cephadm already supports zapping
devices outside of host draining, so it makes shouldn't take much to
add that functionality to the host drain as well.

Fixes: https://tracker.ceph.com/issues/61593
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 85043ff4cee108c152f5aa8af267c85e353c475a)

22 months agomgr/cephadm: removing double quotes from the generated nvmeof config 53868/head
Redouane Kachach [Thu, 21 Sep 2023 08:13:13 +0000 (10:13 +0200)]
mgr/cephadm: removing double quotes from the generated nvmeof config
Fixes: https://tracker.ceph.com/issues/62838
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 6b8187b2ec8a3bf9092da72c7a8b00dc7e5e1ce3)

22 months agoAdding service_name and daemon_type and fixing mypy issues 53867/head
Redouane Kachach [Wed, 13 Sep 2023 11:46:42 +0000 (13:46 +0200)]
Adding service_name and daemon_type and fixing mypy issues

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 72e9345faa725bd2aaba41e06ed34927badd7622)

22 months agomgr/cephadm: Adding sort-by support for ceph orch ps
Redouane Kachach [Thu, 29 Jun 2023 10:02:21 +0000 (12:02 +0200)]
mgr/cephadm: Adding sort-by support for ceph orch ps
Fixes: https://tracker.ceph.com/issues/61856
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit b347106af329e238a927f47436acd54999c4d9af)

22 months agocephadm: run tcmu-runner through script to do restart on failure 53866/head
Adam King [Tue, 13 Jun 2023 23:54:30 +0000 (19:54 -0400)]
cephadm: run tcmu-runner through script to do restart on failure

Currently, cephadm runs tcmu-runner as a background
process inside the unit file deployed for iscsi
(rbd-target-api is the primary process). This means
if tcmu-runner crashes for whatever reason, systemd
will not attempt to restart it. This commits sets
up a script to serve as the container entrypoint
for the tcmu-runner container that will run
tcmu-runner and also restart it on failure
(unless there are too many failures in a short
period, at which point it gives up).

The hope is to eventually drop use of this script
for a better solution in squid onward, but this
should be helpful on older releases (quincy and
pacific at least) where we won't be able to
bring that better solution

Fixes: https://tracker.ceph.com/issues/61667
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 47eb6b3f62afe993073429b02051ae0343d7aea3)

Conflicts:
src/cephadm/tests/test_cephadm.py

22 months agomgr/cephadm: don't use image tag in orch upgrade ls 53865/head
Adam King [Fri, 1 Sep 2023 13:05:04 +0000 (09:05 -0400)]
mgr/cephadm: don't use image tag in orch upgrade ls

Using the tag seems to screw up the auth URL generated
and is unnecessary since we're trying to get a list
of tags for the image anyway.

Fixes: https://tracker.ceph.com/issues/62679
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 0f4426d3a085fc2f5137593029469c3b20ace77e)

22 months agoMerge pull request #53835 from rhcs-dashboard/wip-63074-reef
Nizamudeen A [Fri, 6 Oct 2023 05:38:26 +0000 (11:08 +0530)]
Merge pull request #53835 from rhcs-dashboard/wip-63074-reef

reef: mgr/dashboard: fix the landing page layout issues

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
22 months agoMerge pull request #53853 from zdover23/wip-doc-2023-10-06-backport-53834-to-reef
Anthony D'Atri [Fri, 6 Oct 2023 00:41:26 +0000 (20:41 -0400)]
Merge pull request #53853 from zdover23/wip-doc-2023-10-06-backport-53834-to-reef

reef: doc: remove egg fragment from dev/developer_guide/running-tests-locally

22 months agodoc: remove egg fragment from dev/developer_guide/running-tests-locally 53853/head
Dhairya Parmar [Thu, 5 Oct 2023 08:12:31 +0000 (13:42 +0530)]
doc: remove egg fragment from dev/developer_guide/running-tests-locally

DEPRECATION: git+https://github.com/ceph/teuthology#egg=teuthology
[test] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce
this behaviour change. A possible replacement is to use the req @ url syntax,
and remove the egg fragment. Discussion can be found at
https://github.com/pypa/pip/issues/11617

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 1f78a32a1822fea1a56a88215e452b730b7672c7)

22 months agoMerge pull request #53412 from trociny/wip-62352-reef
Shilpa Jagannath [Thu, 5 Oct 2023 21:02:34 +0000 (17:02 -0400)]
Merge pull request #53412 from trociny/wip-62352-reef

reef: qa: fix keystone in rgw/crypt/barbican.yaml

22 months agodoc/rados: edit troubleshooting.rst 53837/head
Zac Dover [Wed, 4 Oct 2023 11:14:13 +0000 (21:14 +1000)]
doc/rados: edit troubleshooting.rst

Edit doc/rads/troubleshooting.rst to remove some language that sounds
quite close to marketing language.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 9d9d92837e44e3af233bc228850be09e8f64eb7c)

22 months agoMerge pull request #53811 from zdover23/wip-doc-2023-10-04-backport-53791-to-reef
zdover23 [Thu, 5 Oct 2023 11:31:56 +0000 (22:31 +1100)]
Merge pull request #53811 from zdover23/wip-doc-2023-10-04-backport-53791-to-reef

reef: doc/rados: edit ops/control.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
22 months agomgr/dashboard: fix rgw inventory card and broken shadows 53835/head
Nizamudeen A [Tue, 3 Oct 2023 12:03:13 +0000 (17:33 +0530)]
mgr/dashboard: fix rgw inventory card and broken shadows

Mess up of the dashboard landing page layout fixes PR

Fixes: http://tracker.ceph.com/issues/62961
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 4c6c5787cb338995955b4a30fed9c348337e721c)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.html
  - Accept incoming changes
src/pybind/mgr/dashboard/frontend/src/app/shared/components/card-row/card-row.component.html
  - Accept incoming changes