]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
10 months agoMerge pull request #59456 from xxhdx1985126/wip-67731
Matan Breizman [Thu, 12 Sep 2024 10:45:55 +0000 (13:45 +0300)]
Merge pull request #59456 from xxhdx1985126/wip-67731

crimson/osd/pg: clear backfill_state when the PG goes clean

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
10 months agoMerge pull request #59652 from xxhdx1985126/wip-67945
Matan Breizman [Thu, 12 Sep 2024 10:44:15 +0000 (13:44 +0300)]
Merge pull request #59652 from xxhdx1985126/wip-67945

crimson/osd/osd_operations/snaptrim_event: only increase osd_osd_param_t::at_version after clone adjustments

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
10 months agoMerge pull request #59564 from Matan-B/wip-matanb-crimson-osd-release
Matan Breizman [Thu, 12 Sep 2024 10:42:11 +0000 (13:42 +0300)]
Merge pull request #59564 from Matan-B/wip-matanb-crimson-osd-release

crimson/osd: write require_osd_release only when needed

Reviewed-by: Samuel Just <sjust@redhat.com>
10 months agoMerge pull request #59506 from Matan-B/wip-matanb-crimson-seastore-logs
Matan Breizman [Thu, 12 Sep 2024 10:40:53 +0000 (13:40 +0300)]
Merge pull request #59506 from Matan-B/wip-matanb-crimson-seastore-logs

qa/config/seastore: enable all subsystem debug

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
10 months agoMerge pull request #59753 from wodeshijie33941/main 59759/head
Zac Dover [Thu, 12 Sep 2024 09:40:14 +0000 (19:40 +1000)]
Merge pull request #59753 from wodeshijie33941/main

doc/install: Keep the name field of the created user consistent with …

Reviewed-by: Zac Dover <zac.dover@proton.me>
10 months agoMerge pull request #59741 from yuvalif/wip-yuval-68033
Yuval Lifshitz [Thu, 12 Sep 2024 08:36:14 +0000 (11:36 +0300)]
Merge pull request #59741 from yuvalif/wip-yuval-68033

rgw/kafka: refactor topic creation to avoid rd_kafka_topic_name()

Reviewed-By: Casey Bodley <cbodley@ibm.com>
10 months agodoc/install: Keep the name field of the created user consistent with the node name... 59753/head
hejindong [Thu, 12 Sep 2024 07:14:21 +0000 (15:14 +0800)]
doc/install: Keep the name field of the created user consistent with the node name in the Start RADOSGW service command

If the user name does not match the name of the node that started the RADOSGW service, this will cause confusion for those who are new to ceph. Because they can't start the radosgw service normally as shown in the tutorial.

Signed-off-by: hejindong <wodeshijie3394@126.com>
10 months agoMerge pull request #59482 from rhcs-dashboard/nvmeof-api-group-support
Nizamudeen A [Thu, 12 Sep 2024 05:52:23 +0000 (11:22 +0530)]
Merge pull request #59482 from rhcs-dashboard/nvmeof-api-group-support

mgr/dashboard: add gw_groups support to nvmeof api

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: caroav <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
10 months agoMerge pull request #59576 from rhcs-dashboard/hosts-selection-show-only-10-entries-fix
Nizamudeen A [Thu, 12 Sep 2024 05:47:01 +0000 (11:17 +0530)]
Merge pull request #59576 from rhcs-dashboard/hosts-selection-show-only-10-entries-fix

mgr/dashboard: service form hosts selection only show up to 10 entries

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
10 months agoMerge pull request #59738 from gbregman/main
Gil Bregman [Wed, 11 Sep 2024 21:17:05 +0000 (00:17 +0300)]
Merge pull request #59738 from gbregman/main

Make the discovery and gateway IPs configurable in NVMEof configuration

10 months agoMerge pull request #59681 from adk3798/cephadm-enable-module-bootstrap-retry
Adam King [Wed, 11 Sep 2024 15:21:31 +0000 (11:21 -0400)]
Merge pull request #59681 from adk3798/cephadm-enable-module-bootstrap-retry

cephadm: bootstrap: verify orch module is running before setting cephadm backend

Reviewed-by: John Mulligan <jmulligan@redhat.com>
10 months agorgw/kafka: refactor topic creation to avoid rd_kafka_topic_name() 59741/head
Yuval Lifshitz [Wed, 11 Sep 2024 15:12:22 +0000 (15:12 +0000)]
rgw/kafka: refactor topic creation to avoid rd_kafka_topic_name()

also, use unique_ptr for safer memory management.

Fixes: https://tracker.ceph.com/issues/68033
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
10 months agomgr/cephadm: Make the discovery and gateway IPs configurable in NVMEof configuration 59738/head
Gil Bregman [Wed, 11 Sep 2024 14:06:19 +0000 (17:06 +0300)]
mgr/cephadm: Make the discovery and gateway IPs configurable in NVMEof configuration
Fixes https://tracker.ceph.com/issues/68030

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
10 months agopython-common/ceph/deployment: Make the discovery and gateway IPs configurable in...
Gil Bregman [Wed, 11 Sep 2024 14:05:43 +0000 (17:05 +0300)]
python-common/ceph/deployment: Make the discovery and gateway IPs configurable in NVMEof configuration
Fixes https://tracker.ceph.com/issues/68030

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
10 months agoMerge pull request #59635 from zdover23/wip-doc-2024-09-06-rados-ops-health-checks
Zac Dover [Wed, 11 Sep 2024 12:52:48 +0000 (22:52 +1000)]
Merge pull request #59635 from zdover23/wip-doc-2024-09-06-rados-ops-health-checks

doc/rados: add "pgs not deep scrubbed in time" info

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
10 months agomgr/dashboard: expose gw_groups list api 59482/head
Nizamudeen A [Tue, 3 Sep 2024 12:29:47 +0000 (17:59 +0530)]
mgr/dashboard: expose gw_groups list api

also if the cephadm is available, make the api smart enough to
understand the running daemons and get its gateway_addr instead of
taking the first one in the list so that the HA will be there for the UI
and API as well

Fixes: https://tracker.ceph.com/issues/67774
Signed-off-by: Nizamudeen A <nia@redhat.com>
10 months agomgr/dashboard: add gw_groups support to nvmeof api
Nizamudeen A [Wed, 28 Aug 2024 13:03:19 +0000 (18:33 +0530)]
mgr/dashboard: add gw_groups support to nvmeof api

Introduces a gw_group query-param to gateway and subsystem requests.

Modified the dashboard config to support adding more than one gateway to
the list.

Fixes: https://tracker.ceph.com/issues/67774
Signed-off-by: Nizamudeen A <nia@redhat.com>
10 months agoMerge pull request #59066 from xxhdx1985126/wip-67380
Matan Breizman [Wed, 11 Sep 2024 06:13:59 +0000 (09:13 +0300)]
Merge pull request #59066 from xxhdx1985126/wip-67380

crimson/osd: cancel ongoing pglog-based recoveries on recovery defering

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
10 months agoMerge pull request #59485 from mcv21/cephadm_image_warn
Adam King [Tue, 10 Sep 2024 19:51:47 +0000 (15:51 -0400)]
Merge pull request #59485 from mcv21/cephadm_image_warn

cephadm: emit warning if daemon's image is not to be used

Reviewed-by: Adam King <adking@redhat.com>
10 months agoMerge pull request #59599 from rhcs-dashboard/fix-67893-main
afreen23 [Tue, 10 Sep 2024 14:40:52 +0000 (20:10 +0530)]
Merge pull request #59599 from rhcs-dashboard/fix-67893-main

mgr/dashboard: Rephrase dedicated pool helper in rbd create form

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>
10 months agoMerge pull request #59447 from afreen23/wip-nvmeof-microcopy
afreen23 [Tue, 10 Sep 2024 14:14:58 +0000 (19:44 +0530)]
Merge pull request #59447 from afreen23/wip-nvmeof-microcopy

mgr/dashboard: Update nvmeof microcopies

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>
10 months agoMerge pull request #59569 from afreen23/wip-nvme-ns
afreen23 [Tue, 10 Sep 2024 14:14:09 +0000 (19:44 +0530)]
Merge pull request #59569 from afreen23/wip-nvme-ns

mgr/dashboard: Increase maximum namespace count to 1024

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>
10 months agoMerge pull request #56858 from rhcs-dashboard/fix-grafana-server-depreciation
Adam King [Tue, 10 Sep 2024 13:59:39 +0000 (09:59 -0400)]
Merge pull request #56858 from rhcs-dashboard/fix-grafana-server-depreciation

mgr/dashboard: use grafana server instead of grafana-server in grafana 10.4.0

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@ibm.com>
10 months agoMerge pull request #59610 from adk3798/cephadm-ssh-configurable-keepalive
Adam King [Tue, 10 Sep 2024 13:42:16 +0000 (09:42 -0400)]
Merge pull request #59610 from adk3798/cephadm-ssh-configurable-keepalive

mgr/cephadm: make ssh keepalive settings configurable

Reviewed-by: John Mulligan <jmulligan@redhat.com>
10 months agoMerge pull request #59643 from rkachach/fix_issue_67939
Adam King [Tue, 10 Sep 2024 13:37:59 +0000 (09:37 -0400)]
Merge pull request #59643 from rkachach/fix_issue_67939

mgr/cephadm: changing the nginx upstream image used by mgmt-gateway

Reviewed-by: Adam King <adking@redhat.com>
10 months agoMerge pull request #59658 from phlogistonjohn/jjm-remove-object-deleting-func
Adam King [Tue, 10 Sep 2024 13:36:51 +0000 (09:36 -0400)]
Merge pull request #59658 from phlogistonjohn/jjm-remove-object-deleting-func

mgr/smb: stop trying to clean external store during cluster sync

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
10 months agoMerge pull request #58994 from phlogistonjohn/jjm-smb-vfs-ceph-new
Adam King [Tue, 10 Sep 2024 13:35:31 +0000 (09:35 -0400)]
Merge pull request #58994 from phlogistonjohn/jjm-smb-vfs-ceph-new

mgr/smb: add support for new ceph vfs module

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
10 months agoMerge pull request #58880 from aclamk/wip-aclamk-bs-faster-objectstore-test
Adam Kupczyk [Tue, 10 Sep 2024 13:14:23 +0000 (15:14 +0200)]
Merge pull request #58880 from aclamk/wip-aclamk-bs-faster-objectstore-test

test/store_test: Make ceph_test_objectstore run faster

10 months agoMerge pull request #59107 from nbalacha/wip-nbalacha-async-sorted-snaps
Ilya Dryomov [Tue, 10 Sep 2024 11:54:57 +0000 (13:54 +0200)]
Merge pull request #59107 from nbalacha/wip-nbalacha-async-sorted-snaps

librbd: make "group snap list" async and optionally sorted by snap creation time

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
10 months agocrimson/osd/ops_executer: cleanup snap_map_* 59652/head
Matan Breizman [Tue, 10 Sep 2024 11:51:32 +0000 (11:51 +0000)]
crimson/osd/ops_executer: cleanup snap_map_*

Since https://github.com/ceph/ceph/pull/58868, snap map updates
are now maintained via PG::update_snap_map.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agocrimson/osd/osd_operations/snaptrim_event: only increase
Xuehan Xu [Sun, 8 Sep 2024 04:55:23 +0000 (12:55 +0800)]
crimson/osd/osd_operations/snaptrim_event: only increase
osd_osd_param_t::at_version after clone adjustments

Fixes: https://tracker.ceph.com/issues/67945
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
10 months agocrimson/osd/osd_operations/snaptrim_event: we do SnapMapper updates in
Xuehan Xu [Sun, 8 Sep 2024 04:31:28 +0000 (12:31 +0800)]
crimson/osd/osd_operations/snaptrim_event: we do SnapMapper updates in
ReplicatedBackend::submit_transaction(), drop all SnapMapper updates
before that

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
10 months agoMerge pull request #59675 from zdover23/wip-doc-2024-09-10-README-md-vstart
Zac Dover [Tue, 10 Sep 2024 09:10:42 +0000 (19:10 +1000)]
Merge pull request #59675 from zdover23/wip-doc-2024-09-10-README-md-vstart

doc/README.md: improve formatting

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
10 months agoMerge pull request #58986 from NitzanMordhai/wip-nitzan-crimson-op_gate-alianstore...
Matan Breizman [Tue, 10 Sep 2024 06:45:08 +0000 (09:45 +0300)]
Merge pull request #58986 from NitzanMordhai/wip-nitzan-crimson-op_gate-alianstore-multi-core

crimson: use gate per shard for AlienStore and OSD

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
10 months agoMerge pull request #58868 from xxhdx1985126/wip-crimson-clean_region-based-clone...
Matan Breizman [Tue, 10 Sep 2024 06:30:55 +0000 (09:30 +0300)]
Merge pull request #58868 from xxhdx1985126/wip-crimson-clean_region-based-clone-recovery

crimson/osd/pg: properly propagate snap mapper updates and do clean-region-based clone objects recovery

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
10 months agomgr/dashboard: service form hosts selection only show up to 10 entries 59576/head
Super User [Tue, 3 Sep 2024 08:46:45 +0000 (14:16 +0530)]
mgr/dashboard: service form hosts selection only show up to 10 entries

exposed new UIRouter API to get all hosts

Fixes: https://tracker.ceph.com/issues/67413
Signed-off-by: Naman Munet <nmunet@redhat.com>
10 months agoMerge pull request #59677 from zdover23/wip-doc-2024-09-10-man-ceph-bluestore-tool
Anthony D'Atri [Tue, 10 Sep 2024 03:47:00 +0000 (23:47 -0400)]
Merge pull request #59677 from zdover23/wip-doc-2024-09-10-man-ceph-bluestore-tool

doc/man: edit ceph-bluestore-tool.rst

10 months agocephadm: bootstrap: verify orch module is running before setting cephadm backend 59681/head
Adam King [Mon, 9 Sep 2024 22:51:17 +0000 (18:51 -0400)]
cephadm: bootstrap: verify orch module is running before setting cephadm backend

There have been occasional failures when bootstrapping with error
messages like

```
Module 'orchestrator' is not enabled/loaded (required by command 'orch set backend'): use `ceph mgr module enable orchestrator` to enable it
```

despite the orchestrator being meant to be an always on module.

Since trying to enable an already enabled module causes no issues,
trying to enable it ourselves seems like a sensible workaround.

Fixes: https://tracker.ceph.com/issues/67969
Signed-off-by: Adam King <adking@redhat.com>
10 months agodoc/man: edit ceph-bluestore-tool.rst 59677/head
Zac Dover [Mon, 9 Sep 2024 19:39:34 +0000 (05:39 +1000)]
doc/man: edit ceph-bluestore-tool.rst

Edit the section "bluefs-bdev-migrate" in
doc/man/8/ceph-bluestore-tool.rst to add the information that this
operation expands the target storage by updating its size label, making
"bluefs-bdev-expand" unnecessary.

Improve the subject-verb agreement in this section, and supply some
absent definite articles.

Co-authored-by: Peter Gervai <grin@drop.grin.hu>
Signed-off-by: Zac Dover <zac.dover@proton.me>
10 months agoMerge pull request #58603 from cbodley/wip-rgw-sal-concrete-lc
Casey Bodley [Mon, 9 Sep 2024 18:02:38 +0000 (14:02 -0400)]
Merge pull request #58603 from cbodley/wip-rgw-sal-concrete-lc

rgw/sal: LCHead and LCEntry don't need abstraction

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agodoc/README.md: improve formatting 59675/head
Zac Dover [Mon, 9 Sep 2024 15:21:18 +0000 (01:21 +1000)]
doc/README.md: improve formatting

Improve the formatting of the section "Building Ceph" in the file
README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
10 months agomgr/cephadm: changing the nginx upstream image used by mgmt-gateway 59643/head
Redouane Kachach [Mon, 9 Sep 2024 14:29:43 +0000 (16:29 +0200)]
mgr/cephadm: changing the nginx upstream image used by mgmt-gateway

pointing mgmt-gateway nginx image to a different upstream repository

Fixes: https://tracker.ceph.com/issues/67939
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
10 months agoMerge pull request #59297 from ivoalmeida/carbon-datatable-performance
Ivo Almeida [Mon, 9 Sep 2024 14:05:02 +0000 (15:05 +0100)]
Merge pull request #59297 from ivoalmeida/carbon-datatable-performance

mgr/dashboard: datatable performance improvement

Reviewed-by: Nizamudeen A <nia@redhat.com>
10 months agomgr/smb: stop trying to clean external store during cluster sync 59658/head
John Mulligan [Sun, 8 Sep 2024 14:42:36 +0000 (10:42 -0400)]
mgr/smb: stop trying to clean external store during cluster sync

It was found during testing that a sequence of commands like:
```
ceph smb cluster create slow1 user --define-user-pass=user1%badf00d --clustering=always
--placement=3
sleep 0.5
ceph smb share create slow1 share1 cephfs --subvolume=g1/sv1 --path=/
sleep 0.5
ceph smb share create slow1 share2 cephfs --subvolume=g1/sv2 --path=/
```
would create a CTDB enabled cluster that would fail to start up
correctly. The issue was due to the call to `external.rm_other_in_ns`
during the cluster sync operation. In the CTDB enabled mode, objects are
written to the pool outside of the smb mgr module's direct control, in
particular `cluster.meta.json`, and this function, intended to keep the
pool & namespace tidy, was removing objects needed by CTDB-enabled mode.
The failure is somewhat timing sensitive due to the ctdb enablement
sidecars coming up before or after the object was deleted.

Remove this function call so that these objects stop getting deleted at
inopportune times. While we could have tried making this function
"smarter" and only deleting some unexpected objects, in this case I feel
that keeping it simple is better. If we find this pool getting cluttered
in the future we can add a smarter pool-tidying-up function later.

Fixes: https://tracker.ceph.com/issues/67946
Signed-off-by: John Mulligan <jmulligan@redhat.com>
10 months agoMerge pull request #58815 from synarete/samba-metrics-exporter
Adam King [Mon, 9 Sep 2024 13:30:47 +0000 (09:30 -0400)]
Merge pull request #58815 from synarete/samba-metrics-exporter

cephadm: samba metrics exporter

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
10 months agoMerge pull request #59534 from cbodley/wip-rgw-listing-max-entries
Casey Bodley [Mon, 9 Sep 2024 12:51:48 +0000 (08:51 -0400)]
Merge pull request #59534 from cbodley/wip-rgw-listing-max-entries

rgw: raise default rgw_max_listing_results=5000

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #57635 from clwluvw/trans-id
Casey Bodley [Mon, 9 Sep 2024 12:51:30 +0000 (08:51 -0400)]
Merge pull request #57635 from clwluvw/trans-id

rgw: log x-amz-request-id along with the request

Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
10 months agoMerge pull request #58082 from jzhu116-bloomberg/wip-66516
Casey Bodley [Mon, 9 Sep 2024 12:51:09 +0000 (08:51 -0400)]
Merge pull request #58082 from jzhu116-bloomberg/wip-66516

rgw/multipart: use cls_version to avoid racing between part upload and multipart complete

Reviewed-by: Casey Bodley <cbodley@redhat.com>
10 months agoMerge pull request #59014 from liangmingyuanneo/wip-bugfix-beast-socket-accept
Casey Bodley [Mon, 9 Sep 2024 12:50:16 +0000 (08:50 -0400)]
Merge pull request #59014 from liangmingyuanneo/wip-bugfix-beast-socket-accept

rgw/beast: optimize for accept when meeting error in listenning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
10 months agocrimson/osd: check_osdmap_features - add log 59564/head
Matan Breizman [Mon, 2 Sep 2024 14:03:43 +0000 (14:03 +0000)]
crimson/osd: check_osdmap_features - add log

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agocrimson/osd/ops_executer: pass MutFunc by value 58868/head
Matan Breizman [Sun, 8 Sep 2024 13:49:23 +0000 (13:49 +0000)]
crimson/osd/ops_executer: pass MutFunc by value

Since we now co_await mut_func, we should not pass it by rvalue ref.

```
DEBUG 2024-09-01 15:54:46,212 [shard 0:main] osd - do_osd_ops_execute: object 2:c4c92e5a:::rbd_trash:head submitting txn
=================================================================
==17416==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f590008a430 at pc 0x0000040a367a bp 0x7ffc0b1d5ff0 sp 0x7ffc0b1d5fe0
Address 0x7f590008a430 is located in stack of thread T0 at offset 48 in frame
    #0 0x40b0a2b in crimson::osd::PG::do_osd_ops_execute ... lambda(std::error_code const&)#1}&&)::{lambda()#1}::operator()() const (/usr/bin/ceph-osd+0x40b0a2b)
```

Co-authored-by: Xuehan Xu <xuxuehan@qianxin.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agocrimson/osd/ops_executer: avoid seastar::async comment
Matan Breizman [Mon, 26 Aug 2024 10:10:35 +0000 (10:10 +0000)]
crimson/osd/ops_executer: avoid seastar::async comment

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agocrimson/osd/osd_operations/snaptrim_event: make use of process_and_submit
Matan Breizman [Mon, 26 Aug 2024 10:18:08 +0000 (10:18 +0000)]
crimson/osd/osd_operations/snaptrim_event: make use of process_and_submit

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agocrimson/osd/osd_operations/snaptrim_event: introduce process_and_submit
Matan Breizman [Mon, 26 Aug 2024 10:07:36 +0000 (10:07 +0000)]
crimson/osd/osd_operations/snaptrim_event: introduce process_and_submit

We should be carful with using capturing lambdas with coroutines.
Avoid capturing [this] by passing process_and_submit instead.

See: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rcoro-capture

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agocrimson/osd/ops_executer: flush_changes_n_do_ops_effects to use
Matan Breizman [Wed, 21 Aug 2024 13:37:13 +0000 (13:37 +0000)]
crimson/osd/ops_executer: flush_changes_n_do_ops_effects to use
coroutines

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agocrimson/osd/osd_operations/snaptrim_event: start to use coroutines
Matan Breizman [Tue, 20 Aug 2024 14:11:25 +0000 (14:11 +0000)]
crimson/osd/osd_operations/snaptrim_event: start to use coroutines

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agocrimson/osd/pg: make use of SnapMapper::update_snap_map
Matan Breizman [Tue, 20 Aug 2024 13:08:35 +0000 (13:08 +0000)]
crimson/osd/pg: make use of SnapMapper::update_snap_map

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
10 months agomgr/dashboard: datatable performance improvement 59297/head
Ivo Almeida [Wed, 14 Aug 2024 15:19:05 +0000 (16:19 +0100)]
mgr/dashboard: datatable performance improvement

Fixes: https://tracker.ceph.com/issues/67796:
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
10 months agoMerge pull request #59657 from eriksjolund/fix-sign-of-errno
Ilya Dryomov [Mon, 9 Sep 2024 09:15:10 +0000 (11:15 +0200)]
Merge pull request #59657 from eriksjolund/fix-sign-of-errno

tools/rbd_ggate: fix sign of errno

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
10 months agoMerge pull request #59000 from connorfawcett/rbd-random
Ilya Dryomov [Mon, 9 Sep 2024 08:51:49 +0000 (10:51 +0200)]
Merge pull request #59000 from connorfawcett/rbd-random

rbd/bench: Add ability to write a random string of bytes or a specific byte with --pattern-byte option

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
10 months agoMerge pull request #59607 from yuvalif/wip-yuval-67658
Yuval Lifshitz [Mon, 9 Sep 2024 08:37:20 +0000 (11:37 +0300)]
Merge pull request #59607 from yuvalif/wip-yuval-67658

rgw/notifications: free completion pointer using unique_ptr

Reviewed-By: Adam Emerson <aemerson@ibm.com>, Casey Bodley <cbodley@ibm.com>
10 months agoMerge pull request #59642 from zdover23/wip-doc-2024-09-06-cephadm-upgrade
Zac Dover [Mon, 9 Sep 2024 02:44:33 +0000 (12:44 +1000)]
Merge pull request #59642 from zdover23/wip-doc-2024-09-06-cephadm-upgrade

doc/cephadm: add malformed-JSON removal instructions

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
10 months agoMerge pull request #59553 from cyx1231st/wip-seastore-more-reports3
Yingxin [Mon, 9 Sep 2024 02:10:22 +0000 (10:10 +0800)]
Merge pull request #59553 from cyx1231st/wip-seastore-more-reports3

crimson/os/seastore/cache: report cache access stats

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
10 months agodoc/cephadm: add malformed-JSON removal instructions 59642/head
Zac Dover [Fri, 6 Sep 2024 13:31:21 +0000 (23:31 +1000)]
doc/cephadm: add malformed-JSON removal instructions

Add instructions for removing malformed JSON that causes the cephadm
module to fail to start, which in turn makes the orchestrator fail to
start.

Signed-off-by: Zac Dover <zac.dover@proton.me>
10 months agoMerge pull request #59590 from ronen-fr/wip-rf-delay-conf
Ronen Friedman [Sun, 8 Sep 2024 13:59:22 +0000 (16:59 +0300)]
Merge pull request #59590 from ronen-fr/wip-rf-delay-conf

osd/scrub: add configuration parameters to control delay duration

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
10 months agotools/rbd_ggate: fix sign of errno 59657/head
Erik Sjölund [Sun, 8 Sep 2024 12:24:09 +0000 (14:24 +0200)]
tools/rbd_ggate: fix sign of errno

Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
10 months agoMerge pull request #59628 from zdover23/wip-doc-2024-09-06-dev-release-checklist...
Zac Dover [Sun, 8 Sep 2024 10:54:06 +0000 (20:54 +1000)]
Merge pull request #59628 from zdover23/wip-doc-2024-09-06-dev-release-checklist-rtd-activation

doc/dev: add "activate latest release" RTD step

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
10 months agoMerge pull request #59592 from leonidc/wip-leonidc0409-blocklist-fix
Aviv Caro [Sun, 8 Sep 2024 07:23:59 +0000 (10:23 +0300)]
Merge pull request #59592 from leonidc/wip-leonidc0409-blocklist-fix

leonidc0409 blocklist fix

10 months agoMerge pull request #44640 from effi-ofer/live-migration-nbd
Ilya Dryomov [Sat, 7 Sep 2024 10:07:27 +0000 (12:07 +0200)]
Merge pull request #44640 from effi-ofer/live-migration-nbd

librbd/migration: add NBD stream

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
10 months agoMerge pull request #57388 from mchangir/mgr-snap_schedule-correctly-fetch-mds_max_sna...
Venky Shankar [Fri, 6 Sep 2024 14:03:52 +0000 (19:33 +0530)]
Merge pull request #57388 from mchangir/mgr-snap_schedule-correctly-fetch-mds_max_snaps_per_dir-from-mds

mgr/snap_schedule: correctly fetch mds_max_snaps_per_dir from mds

Reviewed-by: Venky Shankar <vshankar@redhat.com>
10 months agoMerge pull request #59579 from leonidc/wip-leonidc0309-gw-deleting-state
Aviv Caro [Fri, 6 Sep 2024 13:22:26 +0000 (16:22 +0300)]
Merge pull request #59579 from leonidc/wip-leonidc0309-gw-deleting-state

 gw deleting state

10 months agomgr/cephadm: stop running "true" before every command 59610/head
Adam King [Fri, 6 Sep 2024 13:04:37 +0000 (09:04 -0400)]
mgr/cephadm: stop running "true" before every command

It turns out this is completely unnecessary. With the keepalive
settings on the connection, we no longer need to run an
additional command with a timeout on it to quickly detect a
host is offline

Signed-off-by: Adam King <adking@redhat.com>
10 months agoMerge pull request #44359 from mchangir/mds-uninline-file-during-scrub
Rishabh Dave [Fri, 6 Sep 2024 13:01:13 +0000 (18:31 +0530)]
Merge pull request #44359 from mchangir/mds-uninline-file-during-scrub

mds: un-inline data on scrub

Reviewed-by: Xiubo Li <xiubli@redhat.com>
10 months agodoc/rados: add "pgs not deep scrubbed in time" info 59635/head
Zac Dover [Fri, 6 Sep 2024 12:43:59 +0000 (22:43 +1000)]
doc/rados: add "pgs not deep scrubbed in time" info

Add a procedure to doc/rados/operations/health-warnings.rst that
explains how to remedy the "X PGs not deep-scrubbed in time" health
warning.

This procedure was developed by Eugen Block, and is at the time of this
commit available on his blog at
https://heiterbiswolkig.blogs.nde.ag/2024/09/06/pgs-not-deep-scrubbed-in-time/

Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
10 months agolibrbd/migration/NBDStream: abstract out libnbd and add unit tests 44640/head
Ilya Dryomov [Thu, 5 Sep 2024 13:43:07 +0000 (15:43 +0200)]
librbd/migration/NBDStream: abstract out libnbd and add unit tests

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
10 months agoqa/suites/rbd: add snippets for migration import from NBD
Ilya Dryomov [Tue, 3 Sep 2024 15:03:55 +0000 (17:03 +0200)]
qa/suites/rbd: add snippets for migration import from NBD

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
10 months agolibrbd/migration/NBDStream: call nbd_shutdown() in close()
Ilya Dryomov [Mon, 2 Sep 2024 20:21:23 +0000 (22:21 +0200)]
librbd/migration/NBDStream: call nbd_shutdown() in close()

Avoid warnings like

  qemu-nbd: Disconnect client, due to: Failed to read request:
  Unexpected end-of-file before all bytes were read

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
10 months agolibrbd/migration/NBDStream: switch to NBD URIs
Ilya Dryomov [Mon, 2 Sep 2024 20:17:40 +0000 (22:17 +0200)]
librbd/migration/NBDStream: switch to NBD URIs

This removes the constraint on the transport being TCP, allowing to
use a Unix domain socket or other options.  It also allows specifying
export names which a) are needed in case of serving different content
on different exports and b) some servers may require regardless.

Additionally, NBD URIs are future proof as all that NBDStream needs to
do is forward the string to libnbd.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
10 months agolibrbd/migration/NBDStream: check for nbd_get_size() errors
Ilya Dryomov [Mon, 2 Sep 2024 20:14:27 +0000 (22:14 +0200)]
librbd/migration/NBDStream: check for nbd_get_size() errors

nbd_get_size() can fail with EOVERFLOW and possibly other errors.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
10 months agolibrbd/migration/NBDStream: introduce from_nbd_errno()
Ilya Dryomov [Mon, 2 Sep 2024 20:11:29 +0000 (22:11 +0200)]
librbd/migration/NBDStream: introduce from_nbd_errno()

Errors returned by nbd_get_errno() can't be used to complete Contexts
directly because a) these errors are positive while complete() in most
cases expects a negative error and b) nbd_get_errno() can return 0 even
after libnbd call fails (i.e. returns -1).

Introduce a helper with EIO as a default/fallback error.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
10 months agolibrbd/migration/NBDStream: be more thorough around nbd_block_status()
Ilya Dryomov [Sun, 25 Aug 2024 11:10:58 +0000 (13:10 +0200)]
librbd/migration/NBDStream: be more thorough around nbd_block_status()

nbd_block_status() is specified to be really loose: it can return
more status (go beyond the requested range), less status (cover only
some part of the requested range) or nothing at all.  The former would
likely violate some assumptions in ObjectCopyRequest, while the latter
would lead to straight data corruption -- parent blocks that weren't
covered would never get copied up into the destination image.

On top of that, the current implementation is very suboptimal.
Because only "nr_entries == 2" responses are effectively processed,
any extent which has a small amount of data and is otherwise zeroed
(e.g. 8K of data in a 4M object) gets listed as DATA.  A copyup for
such an object would write 4M worth of data instead of 8K, consuming
space in the destination image unnecessarily.

Also, it's unclear whether zero-sized entries can be returned.
Currently, if it happens we would hit an assert in interval_map.

This fixes all of these issues.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
10 months agolibrbd/migration: add nbd stream
Effi Ofer [Tue, 18 Jan 2022 12:43:32 +0000 (14:43 +0200)]
librbd/migration: add nbd stream

Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Effi Ofer <effio@il.ibm.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
10 months agoMerge pull request #59630 from zdover23/wip-doc-2024-09-06-README-md-build-prerequisites
Anthony D'Atri [Fri, 6 Sep 2024 11:49:13 +0000 (07:49 -0400)]
Merge pull request #59630 from zdover23/wip-doc-2024-09-06-README-md-build-prerequisites

doc/README.md: edit "Build Prerequisites"

10 months agoMerge pull request #59575 from juan-ferrer-toribio/tracker-67849
Aashish Sharma [Fri, 6 Sep 2024 10:24:21 +0000 (15:54 +0530)]
Merge pull request #59575 from juan-ferrer-toribio/tracker-67849

mgr/dashboard: Fix variable capitalization in embedded rbd-details panel

Reviewed-by: Nizamudeen A <nia@redhat.com>
10 months agorbd/bench: Add ability to write a random string of bytes or a specific byte with... 59000/head
Connor Fawcett [Tue, 6 Aug 2024 17:29:56 +0000 (18:29 +0100)]
rbd/bench: Add ability to write a random string of bytes or a specific byte with --pattern-bytes option

Fixes: https://tracker.ceph.com/issues/67319
Signed-off-by: Connor Fawcett <connorfa@uk.ibm.com>
10 months agodoc/README.md: edit "Build Prerequisites" 59630/head
Zac Dover [Fri, 6 Sep 2024 07:56:27 +0000 (17:56 +1000)]
doc/README.md: edit "Build Prerequisites"

Make the sentences in the "Build Prerequisites" section of README.md
more direct.

Signed-off-by: Zac Dover <zac.dover@proton.me>
10 months ago mon/NVMeofGw*: 59592/head
Leonid Chernin [Wed, 4 Sep 2024 08:54:59 +0000 (08:54 +0000)]
 mon/NVMeofGw*:
 1. fix blocklist bug - blockist was not called
 2. originally monitor only bloklisted specific ana groups but since we allow
    the changing of ns ana grp on the fly for the sake of ns load balance,
    it is not good enough and we need to blocklist all the cluster contexts
    of the failing gateway
Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
10 months agodoc/dev: add "activate latest release" RTD step 59628/head
Zac Dover [Thu, 5 Sep 2024 23:31:47 +0000 (09:31 +1000)]
doc/dev: add "activate latest release" RTD step

Add a step to the section called "First Stable Release" that explains
how to activate the latest release in Read the Docs.

Signed-off-by: Zac Dover <zac.dover@proton.me>
10 months agoMerge pull request #56597 from liangmingyuanneo/optimize-reshard
Casey Bodley [Thu, 5 Sep 2024 17:21:47 +0000 (13:21 -0400)]
Merge pull request #56597 from liangmingyuanneo/optimize-reshard

rgw reshard: optimize reshard process to minimum blocking time

Reviewed-by: Casey Bodley <cbodley@redhat.com>
10 months agoMerge pull request #59170 from guits/remove-legacy-disk-sorting
Guillaume Abrioux [Thu, 5 Sep 2024 15:04:51 +0000 (17:04 +0200)]
Merge pull request #59170 from guits/remove-legacy-disk-sorting

ceph-volume: switch over to new disk sorting behavior

10 months agoMerge pull request #59604 from guits/wip-osd-id-raw
Guillaume Abrioux [Thu, 5 Sep 2024 15:00:04 +0000 (17:00 +0200)]
Merge pull request #59604 from guits/wip-osd-id-raw

ceph-volume: pass self.osd_id to create_id() call

10 months agorgw/notifications: free completion pointer using unique_ptr 59607/head
Yuval Lifshitz [Tue, 27 Aug 2024 20:05:21 +0000 (20:05 +0000)]
rgw/notifications: free completion pointer using unique_ptr

also fix access to possible dpp dangling pointer

Fixes: https://tracker.ceph.com/issues/67658
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
10 months agoMerge pull request #59386 from mkogan1/wip-qlen_qactive
Mark Kogan [Thu, 5 Sep 2024 14:23:30 +0000 (17:23 +0300)]
Merge pull request #59386 from mkogan1/wip-qlen_qactive

rgw: decrement qlen/qactive perf counters on error

10 months agoMerge pull request #59613 from zdover23/wip-doc-2024-09-05-rados-config-mon-lookup-dns
Anthony D'Atri [Thu, 5 Sep 2024 13:54:59 +0000 (09:54 -0400)]
Merge pull request #59613 from zdover23/wip-doc-2024-09-05-rados-config-mon-lookup-dns

doc/rados: add link to messenger v2 info in mon-lookup-dns.rst

10 months agoMerge PR #59414 into main
Patrick Donnelly [Thu, 5 Sep 2024 12:37:24 +0000 (08:37 -0400)]
Merge PR #59414 into main

* refs/pull/59414/head:
mds: move fscrypt_last_block to vector in mds_co mempool
mds: move fscrypt_file inode_t metadata to mds_co mempool
mds: move fscrypt_auth inode_t metadata to mds_co mempool
include/buffer: add helpers for uint8_t vectors

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
10 months agomgr/dashboard: Increase maximum namespace count to 1024 59569/head
Afreen Misbah [Tue, 3 Sep 2024 07:26:42 +0000 (12:56 +0530)]
mgr/dashboard: Increase maximum namespace count to 1024

Fixes https://tracker.ceph.com/issues/67871

The maximum namespace count will be 1024 for subsystems now.

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
10 months agocrimson/common/gated: Comment out ceph_assert for shard ID check 58986/head
Nitzan Mordechai [Wed, 28 Aug 2024 12:49:27 +0000 (12:49 +0000)]
crimson/common/gated: Comment out ceph_assert for shard ID check

After adding sid (shard ID) to Gated, each shard should have its own gate.
However, some users of Gated had a single instance, which either failed
these assertions or created the Gated object on the wrong shard.
A tracker issue #67773 was opened for this, and this commit comments out
the assertions until the issue is resolved.

Signed-off-by: NitzanMordhai <nmordech@redhat.com>
10 months agocrimson/net/io_handler: ctor add gate ctor with shard id
Nitzan Mordechai [Wed, 28 Aug 2024 09:08:26 +0000 (09:08 +0000)]
crimson/net/io_handler: ctor add gate ctor with shard id

IOHandler::to_new_sid provides a mechanism to switch to a new core from the current one used.
This behavior is tricky for making sure that we keep the IOHandler::shard_states_t::gate instance to be used on the same core that it was created.
In order to resolve this, we should be able to construct Gated out of specified core_id (and not from this_shard_id).

See: https://github.com/ceph/ceph/pull/58986#discussion_r1733782952

Signed-off-by: NitzanMordhai <nmordech@redhat.com>
10 months agodoc/rados: add link to messenger v2 info in mon-lookup-dns.rst 59613/head
Zac Dover [Thu, 5 Sep 2024 07:59:15 +0000 (17:59 +1000)]
doc/rados: add link to messenger v2 info in mon-lookup-dns.rst

Add a link to the page about Messenger v2 to the end of
doc/rados/configuration/mon-lookup-dns.rst.

Fixes: https://tracker.ceph.com/issues/58752
Signed-off-by: Zac Dover <zac.dover@proton.me>
10 months agoqa: test fetching foreign config item 57388/head
Milind Changire [Fri, 10 May 2024 09:52:51 +0000 (15:22 +0530)]
qa: test fetching foreign config item

Signed-off-by: Milind Changire <mchangir@redhat.com>