]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
11 months agodoc: add clustering related items to smb docs 59086/head
John Mulligan [Wed, 14 Aug 2024 18:19:17 +0000 (14:19 -0400)]
doc: add clustering related items to smb docs

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>
11 months agoqa/suites/orch: add a pair of teuthology tests for ctdb smb clusters
John Mulligan [Sat, 10 Aug 2024 18:42:16 +0000 (14:42 -0400)]
qa/suites/orch: add a pair of teuthology tests for ctdb smb clusters

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>
11 months agoqa/suites/orch: old smb tests need placement count 1 to avoid using clustering
John Mulligan [Sat, 10 Aug 2024 16:49:24 +0000 (12:49 -0400)]
qa/suites/orch: old smb tests need placement count 1 to avoid using clustering

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>
11 months agomgr/cephadm: enable the smb service to prevent stray ctdb services
John Mulligan [Mon, 12 Aug 2024 14:56:51 +0000 (10:56 -0400)]
mgr/cephadm: enable the smb service to prevent stray ctdb services

Tell cephadm that any `ctdb` services are "owned" by the smb service
and should be ignored as not a stray.
Ideally, we do this on a per service basis but the info that the ctdb
lock helper provides to its registration function is pretty generic.
Future versions of samba may improve upon this.

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>
11 months agomgr/cephadm: extend stray service detection with a general ignore hook
John Mulligan [Mon, 12 Aug 2024 14:56:36 +0000 (10:56 -0400)]
mgr/cephadm: extend stray service detection with a general ignore hook

Extend the system's current stray service detection with a new method on
the service classes so that new classes can hook into the stray services
in the case that ceph services and cephadm services have differing names
or use subsystems that call into ceph with different names (my use
case).

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>
11 months agomgr/cephadm: move logic determining name in stray func
John Mulligan [Mon, 12 Aug 2024 13:52:20 +0000 (09:52 -0400)]
mgr/cephadm: move logic determining name in stray func

Encapsulate the logic determining the name of a stray service into a
method reducing the length and levels of indent in the stray checker
function.

Signed-off-by: John Mulligan <phlogistonjohn@asynchrono.us>
11 months agomgr/smb: enable clustering when setting up a cluster
John Mulligan [Mon, 15 Jul 2024 19:41:56 +0000 (15:41 -0400)]
mgr/smb: enable clustering when setting up a cluster

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agomgr/smb: add a cluster resource field to manage clustering
John Mulligan [Mon, 15 Jul 2024 19:41:43 +0000 (15:41 -0400)]
mgr/smb: add a cluster resource field to manage clustering

Add a new `clustering` field to the smb cluster resource. This field can
be used to select either automatic clustering with ctdb, or disable it,
or require it. The default is automatic and is based on the count value
in the placement spec. A count of 1 disables clustering and any other
value it is enabled.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agomgr/cephadm: configure ctdb cluster metadata from cephadm smb service
John Mulligan [Thu, 15 Aug 2024 20:40:47 +0000 (16:40 -0400)]
mgr/cephadm: configure ctdb cluster metadata from cephadm smb service

Add support to the smb service module so that cephadm will provide
information about the layout of the smb daemons to the clustermeta
module that, in turn, will provide the information sambacc needs to
configure ctdb.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agomgr/smb: add a python module to help manage the ctdb cluster
John Mulligan [Mon, 15 Jul 2024 19:39:19 +0000 (15:39 -0400)]
mgr/smb: add a python module to help manage the ctdb cluster

Add a new module clustermeta that implements a JSON based interface
compatible with sambacc. This module will be called directly by cephadm
as it places the daemons on the cluster nodes.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agomgr/smb: add function to parse rados psuedo-uri values
John Mulligan [Mon, 15 Jul 2024 19:22:43 +0000 (15:22 -0400)]
mgr/smb: add function to parse rados psuedo-uri values

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agomgr/smb: add support for rados locks to rados store
John Mulligan [Mon, 15 Jul 2024 19:22:22 +0000 (15:22 -0400)]
mgr/smb: add support for rados locks to rados store

Add support for using rados object locks to the rados store classes.
Callers directly using the rados store outside the store interface will
be able to make use of locking.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agomgr/cephadm: improve key management of smb service
John Mulligan [Mon, 15 Jul 2024 19:38:12 +0000 (15:38 -0400)]
mgr/cephadm: improve key management of smb service

The clustered mode of a logical smb cluster needs certain additional
capabilities in the rados pool. Improve, reorganize the key
configuration functions, and add the new caps.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agopython-common: add clustering related params to SMBSpec
John Mulligan [Mon, 15 Jul 2024 19:16:56 +0000 (15:16 -0400)]
python-common: add clustering related params to SMBSpec

Add parameters related to ctdb clustering to the smb service
deployment spec.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agocephadm: add ctdb support to smb daemon type
John Mulligan [Mon, 15 Jul 2024 19:16:04 +0000 (15:16 -0400)]
cephadm: add ctdb support to smb daemon type

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agocephadm: allow longer subcomponent names
John Mulligan [Mon, 15 Jul 2024 19:14:37 +0000 (15:14 -0400)]
cephadm: allow longer subcomponent names

Allow subcomponent names up to 32 chars long.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agocephadm: add a new context getter for rank
John Mulligan [Mon, 15 Jul 2024 19:14:13 +0000 (15:14 -0400)]
cephadm: add a new context getter for rank

Add a new context getter function to fetch a daemon's rank and rank
generation value.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
11 months agoMerge pull request #58460 from rkachach/fix_issue_oauth2_support
Adam King [Tue, 20 Aug 2024 12:35:44 +0000 (08:35 -0400)]
Merge pull request #58460 from rkachach/fix_issue_oauth2_support

adding support for SSO based on auth2-proxy

Reviewed-by: Adam King <adking@redhat.com>
11 months agoMerge pull request #58860 from adk3798/cephadm-nvmeof-require-group
Adam King [Tue, 20 Aug 2024 12:20:02 +0000 (08:20 -0400)]
Merge pull request #58860 from adk3798/cephadm-nvmeof-require-group

mgr/cephadm: require "group" parameter in nvmeof specs

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
11 months agoMerge pull request #59165 from NitzanMordhai/wip-nitzan-test-rados-tools-newline...
NitzanMordhai [Tue, 20 Aug 2024 12:07:21 +0000 (15:07 +0300)]
Merge pull request #59165 from NitzanMordhai/wip-nitzan-test-rados-tools-newline-trim

test: test_rados_tools compare output without trimming newline

11 months agoMerge pull request #59153 from ajarr/wip-67436
Ilya Dryomov [Tue, 20 Aug 2024 10:19:23 +0000 (12:19 +0200)]
Merge pull request #59153 from ajarr/wip-67436

rbd: fix CLI output of `rbd group snap info` command when a group snapshot with no member images

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Sunil Angadi <Sunil.Angadi@ibm.com>
11 months agoMerge pull request #59292 from cyx1231st/wip-seastore-revert-decouple-ool-writes
Yingxin [Tue, 20 Aug 2024 08:30:57 +0000 (16:30 +0800)]
Merge pull request #59292 from cyx1231st/wip-seastore-revert-decouple-ool-writes

Revert "crimson/os/seastore: wait ool writes in DeviceSubmission phase"

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
11 months agoMerge pull request #59241 from tobias-urdin/openstack-upperconstraints
Casey Bodley [Mon, 19 Aug 2024 17:10:57 +0000 (13:10 -0400)]
Merge pull request #59241 from tobias-urdin/openstack-upperconstraints

qa: barbican: restrict python packages with upper-constraints

Reviewed-by: Casey Bodley <cbodley@redhat.com>
11 months agoMerge pull request #58961 from NitzanMordhai/wip-nitzan-dencoder-test-forward-incompa...
Yuri Weinstein [Mon, 19 Aug 2024 14:25:47 +0000 (07:25 -0700)]
Merge pull request #58961 from NitzanMordhai/wip-nitzan-dencoder-test-forward-incompat-fix

workunit/dencoder: dencoder test forward incompat fix

Reviewed-by: Casey Bodley <cbodley@redhat.com>
11 months agoMerge pull request #58594 from jamiepryde/isa-xor-raid
Yuri Weinstein [Mon, 19 Aug 2024 14:24:56 +0000 (07:24 -0700)]
Merge pull request #58594 from jamiepryde/isa-xor-raid

erasure-code/isa: Use isa/raid's xor_gen() instead of the region_xor(…

Reviewed-by: Mark Nelson <mnelson@redhat.com>
11 months agoqa: barbican: restrict python packages with upper-constraints 59241/head
Tobias Urdin [Thu, 15 Aug 2024 15:17:14 +0000 (17:17 +0200)]
qa: barbican: restrict python packages with upper-constraints

We install barbican by doing a pip install directly on the
cloned git repository but we don't honor the upper-constraints
from the OpenStack Requirements project that handles what
versions is supported.

This changes the pip install command that we issue when
installing barbican to honor the requirements for the
version (derived from the branch) that we use, in
this case it's the 2023.1 release upper-constraints [1].

This prevents us from pulling in untested Python packages.

This only updates Barbican because for the Keystone job
we dont directly issue pip but install using tox using the
`venv` environment which already by default sets the
constraints as you can see in [2].

[1] https://releases.openstack.org/constraints/upper/2023.1
[2] https://github.com/openstack/keystone/blob/stable/2023.1/tox.ini#L12

Fixes: https://tracker.ceph.com/issues/67444
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
11 months agoMerge pull request #59239 from yuvalif/wip-yuval-67513
Yuval Lifshitz [Mon, 19 Aug 2024 10:37:07 +0000 (13:37 +0300)]
Merge pull request #59239 from yuvalif/wip-yuval-67513

Reviewed-By: Casey Bodley <cbodley@ibm.com>
test/rgw/notification: use real ip address instead of localhost

based on that comment:
https://tracker.ceph.com/issues/67206#note-6
the address used by the endpoint is taken as the real IP address of the
host where the test script is running and not localhost.

we also changed the rabbitmq-server conf to allow "guest"
user to connect over non localhost address

Fixes: https://tracker.ceph.com/issues/67206
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
11 months agoMerge pull request #59200 from ifed01/wip-ifed-fix-store-test-col-ref
Igor Fedotov [Mon, 19 Aug 2024 09:47:40 +0000 (12:47 +0300)]
Merge pull request #59200 from ifed01/wip-ifed-fix-store-test-col-ref

test/store_test: fix assertions due to unclosed collection refs.

Reviewd-by: Pere Diaz Bou <pere-altea@hotmail.com>
11 months agoMerge pull request #59256 from zdover23/wip-doc-2024-08-17-cephfs-ceph-dokan-mount...
Zac Dover [Mon, 19 Aug 2024 07:21:51 +0000 (17:21 +1000)]
Merge pull request #59256 from zdover23/wip-doc-2024-08-17-cephfs-ceph-dokan-mount-point

doc/cephfs: s/mountpoint/mount point/

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
11 months agoMerge pull request #58995 from rhcs-dashboard/fix-66844-main
Nizamudeen A [Mon, 19 Aug 2024 05:49:52 +0000 (11:19 +0530)]
Merge pull request #58995 from rhcs-dashboard/fix-66844-main

qa/mgr/dashboard: fix test race condition

Reviewed-by: Nizamudeen A <nia@redhat.com>
11 months agoMerge pull request #59212 from cyx1231st/wip-seastore-more-reports
Yingxin [Mon, 19 Aug 2024 02:18:32 +0000 (10:18 +0800)]
Merge pull request #59212 from cyx1231st/wip-seastore-more-reports

crimson/os/seastore/cache: report lru usage/in/out with trans and extent type

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
11 months agoRevert "crimson/os/seastore: wait ool writes in DeviceSubmission phase" 59292/head
Yingxin Cheng [Mon, 19 Aug 2024 01:48:28 +0000 (09:48 +0800)]
Revert "crimson/os/seastore: wait ool writes in DeviceSubmission phase"

This reverts commit c9e423facea79d42f0496264f267adee5d911b87.

The commit starts to submit OOL writes before submitting the journal
write, true, but it cannot guarantee that OOL writes finish before the
journal write.

Thus it is possible that during SeaStore restart, a journal record
appears valid but its dependent OOL records are partial written, which
leads to corruption.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agoMerge pull request #59290 from anthonyeleven/mountpoint
Anthony D'Atri [Sun, 18 Aug 2024 15:43:00 +0000 (08:43 -0700)]
Merge pull request #59290 from anthonyeleven/mountpoint

doc: Harmonize 'mountpoint'

11 months agodoc: Harmonize 'mountpoint' 59290/head
Anthony D'Atri [Sun, 18 Aug 2024 15:23:39 +0000 (11:23 -0400)]
doc: Harmonize 'mountpoint'

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
11 months agoMerge pull request #59257 from zdover23/wip-doc-2024-08-17-cephfs-mount-point
Zac Dover [Sat, 17 Aug 2024 20:00:23 +0000 (06:00 +1000)]
Merge pull request #59257 from zdover23/wip-doc-2024-08-17-cephfs-mount-point

doc/cephfs: s/mountpoint/mount point/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
11 months agodoc/cephfs: s/mountpoint/mount point/ 59257/head
Zac Dover [Sat, 17 Aug 2024 03:44:30 +0000 (13:44 +1000)]
doc/cephfs: s/mountpoint/mount point/

Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).

cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486 in
which page 345 of The IBM Style Guide is referenced to back up this
change.

This commit alters only English-language text and example commands in
which the string "{mount point}" is meant to be replaced. No commands
meant for cutting-and-pasting have been altered in this commit.

Signed-off-by: Zac Dover <zac.dover@proton.me>
11 months agodoc/cephfs: s/mountpoint/mount point/ 59256/head
Zac Dover [Sat, 17 Aug 2024 03:37:58 +0000 (13:37 +1000)]
doc/cephfs: s/mountpoint/mount point/

Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).

cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486
in which page 345 of The IBM Style Guide is referenced to back up this
change.

Signed-off-by: Zac Dover <zac.dover@proton.me>
11 months agoMerge pull request #58355 from batrick/ceph-backport-fetchhead
Venky Shankar [Fri, 16 Aug 2024 16:14:21 +0000 (21:44 +0530)]
Merge pull request #58355 from batrick/ceph-backport-fetchhead

script/ceph-backport: robustness adjustments for local git repo quirks

Reviewed-by: Venky Shankar <vshankar@redhat.com>
11 months agotest/rgw/notification: use real ip address instead of localhost 59239/head
Yuval Lifshitz [Thu, 15 Aug 2024 14:34:57 +0000 (14:34 +0000)]
test/rgw/notification: use real ip address instead of localhost

based on that comment:
https://tracker.ceph.com/issues/67206#note-6
the address used by the endpoint is taken as the real IP address of the
host where the test script is running and not localhost.

we also changed the rabbitmq-server conf to allow "guest"
user to connect over non localhost address

Fixes: https://tracker.ceph.com/issues/67206
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
11 months agoMerge pull request #59167 from zdover23/wip-doc-2024-08-12-cephfs-file-layouts
Zac Dover [Fri, 16 Aug 2024 09:20:01 +0000 (19:20 +1000)]
Merge pull request #59167 from zdover23/wip-doc-2024-08-12-cephfs-file-layouts

doc/cephfs: improve "layout fields" text

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
11 months agoMerge PR #58896 into main
Venky Shankar [Fri, 16 Aug 2024 06:03:26 +0000 (11:33 +0530)]
Merge PR #58896 into main

* refs/pull/58896/head:
client: flush the caps release in filesystem sync

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
11 months agoMerge pull request #59205 from xxhdx1985126/wip-seastore-find-pending-version
Yingxin [Fri, 16 Aug 2024 05:48:27 +0000 (13:48 +0800)]
Merge pull request #59205 from xxhdx1985126/wip-seastore-find-pending-version

crimson/os/seastore/btree: fix minor corner case issue

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agoclient: flush the caps release in filesystem sync 58896/head
Xiubo Li [Mon, 29 Jul 2024 06:20:41 +0000 (14:20 +0800)]
client: flush the caps release in filesystem sync

We have hit a race between cap releases and cap revoke request
that will cause the check_caps() to miss sending a cap revoke ack
to MDS. And the client will depend on the cap release to release
that revoking caps, which could be delayed for some unknown reasons.

In Kclient we have figured out the RCA about race and we need
a way to explictly trigger this manually could help to get rid
of the caps revoke stuck issue.

Fixes: https://tracker.ceph.com/issues/67221
Signed-off-by: Xiubo Li <xiubli@redhat.com>
11 months agoMerge pull request #58415 from ljflores/wip-tracker-66809
Laura Flores [Thu, 15 Aug 2024 19:02:36 +0000 (14:02 -0500)]
Merge pull request #58415 from ljflores/wip-tracker-66809

qa/suites/upgrade: ignore PG_AVAILABILITY and MON_DOWN for quincy-x and reef-x upgrade suites

11 months agoMerge pull request #59220 from ivoalmeida/carbon-datatable-cleanups
Ivo Almeida [Thu, 15 Aug 2024 17:07:47 +0000 (18:07 +0100)]
Merge pull request #59220 from ivoalmeida/carbon-datatable-cleanups

mgr/dashboard: carbon datatables impr and cleanups

Reviewed-by: Nizamudeen A <nia@redhat.com>
11 months agoMerge pull request #59218 from yuvalif/wip-yuval-67525
J. Eric Ivancich [Thu, 15 Aug 2024 14:22:09 +0000 (10:22 -0400)]
Merge pull request #59218 from yuvalif/wip-yuval-67525

rgw/notifications: fixing radosgw-admin notification json

Reviewed-by: Casey Bodley <cbodley@redhat.com>
11 months agoMerge pull request #59118 from xxhdx1985126/wip-crimson-backfill-cancellation
Matan Breizman [Thu, 15 Aug 2024 11:02:38 +0000 (14:02 +0300)]
Merge pull request #59118 from xxhdx1985126/wip-crimson-backfill-cancellation

crimson/osd/backfill_state: support backfill cancellation

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
11 months agocrimson/osd/backfill_state: support backfilling cancellation 59118/head
Xuehan Xu [Sat, 10 Aug 2024 06:22:09 +0000 (14:22 +0800)]
crimson/osd/backfill_state: support backfilling cancellation

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
11 months agocrimson/osd/pg_recovery: reset backfill_state when backfill finished
Xuehan Xu [Sat, 10 Aug 2024 06:22:52 +0000 (14:22 +0800)]
crimson/osd/pg_recovery: reset backfill_state when backfill finished

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
11 months agoMerge pull request #57966 from xxhdx1985126/wip-crimson-concurrent-recover-missing
Matan Breizman [Thu, 15 Aug 2024 08:09:23 +0000 (11:09 +0300)]
Merge pull request #57966 from xxhdx1985126/wip-crimson-concurrent-recover-missing

crimson/osd/osd_operations: make the "recover_missing" phase concurrent

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
11 months agoMerge pull request #53151 from xxhdx1985126/wip-crimson-backfill-fixes
Matan Breizman [Thu, 15 Aug 2024 08:08:25 +0000 (11:08 +0300)]
Merge pull request #53151 from xxhdx1985126/wip-crimson-backfill-fixes

crimson/osd/backfill_state: fixes two corner cases in backfilling

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
11 months agoMerge pull request #58911 from yuvalif/wip-yuval-67229
Yuval Lifshitz [Thu, 15 Aug 2024 08:00:20 +0000 (11:00 +0300)]
Merge pull request #58911 from yuvalif/wip-yuval-67229

test/cls_2pc_queue: prevent list+remove race between consumers

Reviewed-By: Casey Bodley <cbodley@ibm.com>
11 months agoMerge pull request #59219 from yuvalif/wip-yuval-50610
Yuval Lifshitz [Thu, 15 Aug 2024 07:58:50 +0000 (10:58 +0300)]
Merge pull request #59219 from yuvalif/wip-yuval-50610

doc/rgw/notification: persistent notification queue full behavior

Reviewed-By: Anthony D'Atri <anthony.datri@gmail.com>
11 months agocrimson/os/seastore/cache: report lru usage/in/out with trans and extent type 59212/head
Yingxin Cheng [Wed, 14 Aug 2024 05:22:10 +0000 (13:22 +0800)]
crimson/os/seastore/cache: report lru usage/in/out with trans and extent type

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore: cleanup periodical reporting
Yingxin Cheng [Wed, 14 Aug 2024 05:20:30 +0000 (13:20 +0800)]
crimson/os/seastore: cleanup periodical reporting

Consolidate time into a single place per SeaStore::Shard.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore/cache/lru: renames
Yingxin Cheng [Fri, 9 Aug 2024 08:55:41 +0000 (16:55 +0800)]
crimson/os/seastore/cache/lru: renames

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore/cache: refine lru logics
Yingxin Cheng [Fri, 9 Aug 2024 08:13:48 +0000 (16:13 +0800)]
crimson/os/seastore/cache: refine lru logics

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore: move counter_by_extent_t definition
Yingxin Cheng [Fri, 9 Aug 2024 08:01:39 +0000 (16:01 +0800)]
crimson/os/seastore: move counter_by_extent_t definition

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore/seastore_types: unify checks to the extent types
Yingxin Cheng [Fri, 9 Aug 2024 06:08:38 +0000 (14:08 +0800)]
crimson/os/seastore/seastore_types: unify checks to the extent types

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agoMerge pull request #59225 from zdover23/wip-doc-2024-08-15-glossary-flapping-osd
Anthony D'Atri [Thu, 15 Aug 2024 02:38:51 +0000 (19:38 -0700)]
Merge pull request #59225 from zdover23/wip-doc-2024-08-15-glossary-flapping-osd

doc/glossary: add "flapping OSD"

11 months agoMerge pull request #59028 from cbodley/wip-67326
Casey Bodley [Wed, 14 Aug 2024 18:16:05 +0000 (14:16 -0400)]
Merge pull request #59028 from cbodley/wip-67326

rgw/notify: visit() returns copy of owner string

Reviewed-by: Adam Emerson <aemerson@redhat.com>
11 months agodoc/glossary: add "flapping OSD" 59225/head
Zac Dover [Wed, 14 Aug 2024 18:08:14 +0000 (04:08 +1000)]
doc/glossary: add "flapping OSD"

Add an entry for "Flapping OSD" to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>
11 months agoMerge pull request #58448 from cbodley/wip-rgw-lc-async
Casey Bodley [Wed, 14 Aug 2024 17:47:54 +0000 (13:47 -0400)]
Merge pull request #58448 from cbodley/wip-rgw-lc-async

cls/rgw: define lc ops in terms of ObjectOperation instead of IoCtx

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
11 months agomgr/dashboard: carbon datatables impr and cleanups 59220/head
Ivo Almeida [Wed, 14 Aug 2024 11:15:36 +0000 (12:15 +0100)]
mgr/dashboard: carbon datatables impr and cleanups

Fixes: https://tracker.ceph.com/issues/67544,
https://tracker.ceph.com/issues/67538,
https://tracker.ceph.com/issues/67542, https://tracker.ceph.com/issues/67545, https://tracker.ceph.com/issues/67546

Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
11 months agoMerge pull request #58965 from linuxbox2/wip-lcgt-typo
Casey Bodley [Wed, 14 Aug 2024 14:57:44 +0000 (10:57 -0400)]
Merge pull request #58965 from linuxbox2/wip-lcgt-typo

rgwlc: fix typo in getlc (ObjectSizeGreaterThan)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
11 months agodoc/rgw/notification: persistent notification queue full behavior 59219/head
Yuval Lifshitz [Wed, 14 Aug 2024 11:02:09 +0000 (11:02 +0000)]
doc/rgw/notification: persistent notification queue full behavior

Fixes: https://tracker.ceph.com/issues/50610
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
11 months agoMerge pull request #59169 from cbodley/wip-67464
Casey Bodley [Wed, 14 Aug 2024 13:14:13 +0000 (09:14 -0400)]
Merge pull request #59169 from cbodley/wip-67464

rgw: revert account-related changes to get_iam_policy_from_attr()

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
11 months agoMerge pull request #57888 from liangmingyuanneo/wip-standalone-test-pg-repair
Ronen Friedman [Wed, 14 Aug 2024 13:12:28 +0000 (16:12 +0300)]
Merge pull request #57888 from liangmingyuanneo/wip-standalone-test-pg-repair

qa/standalone: bugfix for latecy repair after scrub

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
11 months agoMerge pull request #59210 from rhcs-dashboard/multi-cluster-overview-usage-template-fix
naman munet [Wed, 14 Aug 2024 12:42:39 +0000 (18:12 +0530)]
Merge pull request #59210 from rhcs-dashboard/multi-cluster-overview-usage-template-fix

mgr/dashboard: fix multi-cluster usage bar error after carbon changes

11 months agomgr/cephadm: adding oauth2-proxy cephadm service 58460/head
Redouane Kachach [Tue, 2 Jul 2024 15:28:40 +0000 (17:28 +0200)]
mgr/cephadm: adding oauth2-proxy cephadm service

adding new oauth2-proxy service. The enable_auth flag enables SSO
authentication via the oauth2-proxy service. The user must ensure the
oauth2-proxy service is deployed before enabling this flag in the
mgmt-gateway service.

FQDN related changes: previously, we were obtaining the FQDN using a
call to the Python socket library run inside the container. While this
generally works, the FQDN returned inside a container can sometimes
differ from the one obtained outside the container. This discrepancy
could cause some issues. To ensure consistency, we now use the FQDN
from the inventory, which provides the correct value as recognized on the host.

Signed-off-by: Redouane Kachach <rkachach@ibm.com>
11 months agorgw/notifications: fixing radosgw-admin notification json 59218/head
Yuval Lifshitz [Wed, 14 Aug 2024 10:41:18 +0000 (10:41 +0000)]
rgw/notifications: fixing radosgw-admin notification json

Fixes: https://tracker.ceph.com/issues/67525
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
11 months agoMerge pull request #59168 from zdover23/wip-doc-2024-08-12-cephfs-cache-configuration
Zac Dover [Wed, 14 Aug 2024 10:09:38 +0000 (20:09 +1000)]
Merge pull request #59168 from zdover23/wip-doc-2024-08-12-cephfs-cache-configuration

doc/cephfs: improve cache-configuration.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
11 months agocrimson/os/seastore/cache: pass missing src to touch_extent()
Yingxin Cheng [Mon, 5 Aug 2024 03:09:14 +0000 (11:09 +0800)]
crimson/os/seastore/cache: pass missing src to touch_extent()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore/cache: cleanup add_extent()
Yingxin Cheng [Fri, 2 Aug 2024 07:06:38 +0000 (15:06 +0800)]
crimson/os/seastore/cache: cleanup add_extent()

Move add_to_dirty() and touch_extent() out of add_extent(), this removes
duplicated calls to touch_extent() from the on_cache callback.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore/cache: cleanup remove_from_dirty()
Yingxin Cheng [Thu, 1 Aug 2024 08:46:39 +0000 (16:46 +0800)]
crimson/os/seastore/cache: cleanup remove_from_dirty()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore: drop duplicated calls to touch_extent()
Yingxin Cheng [Thu, 1 Aug 2024 08:44:48 +0000 (16:44 +0800)]
crimson/os/seastore: drop duplicated calls to touch_extent()

The extent is already PRESENT, which means it was already touched in
this transaction.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agocrimson/os/seastore/cached_extent: rename primary_ref_list
Yingxin Cheng [Thu, 1 Aug 2024 08:43:41 +0000 (16:43 +0800)]
crimson/os/seastore/cached_extent: rename primary_ref_list

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
11 months agomgr/dashboard: fix multi-cluster usage bar error after carbon changes 59210/head
Naman Munet [Wed, 14 Aug 2024 07:13:22 +0000 (12:43 +0530)]
mgr/dashboard: fix multi-cluster usage bar error after carbon changes

fixes: https://tracker.ceph.com/issues/67536

Signed-off-by: Naman Munet <nmunet@redhat.com>
11 months agoMerge pull request #59186 from rhcs-dashboard/replace-cluster-capacity-with-usage-bar
naman munet [Wed, 14 Aug 2024 05:57:02 +0000 (11:27 +0530)]
Merge pull request #59186 from rhcs-dashboard/replace-cluster-capacity-with-usage-bar

mgr/dashboard: replace individual cluster's capacity info with Usage bar in Multi-Cluster

11 months agoMerge PR #59025 into main
Venky Shankar [Wed, 14 Aug 2024 04:58:09 +0000 (10:28 +0530)]
Merge PR #59025 into main

* refs/pull/59025/head:
tools/rados: Fix extra NL in getxattr

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Gabriel Benhanokh <gbenhano@redhat.com>
11 months agoMerge pull request #58485 from ivoalmeida/carbon-datatable
Nizamudeen A [Wed, 14 Aug 2024 04:18:04 +0000 (09:48 +0530)]
Merge pull request #58485 from ivoalmeida/carbon-datatable

mgr/dashboard: replace ngx-datatable by carbon datatable

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
11 months agocrimson/os/seastore/btree: fix minor corner case issue 59205/head
Xuehan Xu [Wed, 14 Aug 2024 03:00:00 +0000 (11:00 +0800)]
crimson/os/seastore/btree: fix minor corner case issue

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
11 months agoqa/suites/upgrade: ignore MON_DOWN and PG_AVAILABILITY warnings in upgrade tests 58415/head
Laura Flores [Tue, 13 Aug 2024 22:58:17 +0000 (17:58 -0500)]
qa/suites/upgrade: ignore MON_DOWN and PG_AVAILABILITY warnings in upgrade tests

Signed-off-by: Laura Flores <lflores@ibm.com>
11 months agorbd: fix CLI output of `rbd group snap info` command 59153/head
Ramana Raja [Sun, 11 Aug 2024 02:18:07 +0000 (22:18 -0400)]
rbd: fix CLI output of `rbd group snap info` command

... when a group snapshot has no member images.

A group snapshot can be created with no member images. For such a group
snapshot, omit the 'image snap' and 'images' fields from the
unformatted CLI output of `rbd group snap info` command so as to not
confuse the user. In the librbd C/C++ data structures representing a
group snapshot with no member images, set the 'image_snap_name' data
member to an empty string.

Fixes: https://tracker.ceph.com/issues/67436
Signed-off-by: Ramana Raja <rraja@redhat.com>
11 months agoMerge pull request #58140 from guits/cv-tpm2-support
Guillaume Abrioux [Tue, 13 Aug 2024 19:27:42 +0000 (21:27 +0200)]
Merge pull request #58140 from guits/cv-tpm2-support

ceph-volume: add TPM2 token enrollment support for encrypted OSDs

11 months agoMerge pull request #58956 from ThomasLamprecht/ceph-volume-debian-dependency
Guillaume Abrioux [Tue, 13 Aug 2024 19:18:33 +0000 (21:18 +0200)]
Merge pull request #58956 from ThomasLamprecht/ceph-volume-debian-dependency

debian pkg: record python3-packaging dependency for ceph-volume

11 months agoqa/s3tests: configure tenant name for 's3 tenant' section 59169/head
Casey Bodley [Tue, 13 Aug 2024 17:06:32 +0000 (13:06 -0400)]
qa/s3tests: configure tenant name for 's3 tenant' section

Signed-off-by: Casey Bodley <cbodley@redhat.com>
11 months agomgr/dashboard: replace ngx-datatable by carbon 58485/head
Ivo Almeida [Wed, 26 Jun 2024 14:42:12 +0000 (15:42 +0100)]
mgr/dashboard: replace ngx-datatable by carbon

Fixes: https://tracker.ceph.com/issues/66965
* replaced ngx-datatable by carbon datatable
* created carbon themes for content and tables
* redesigned table actions to render as kebab menu options per data row
* keep only primary actions on datatable toolbar
* implemented carbon batch actions

Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
11 months agotest/store_test: fix assertions due to unclosed collection refs. 59200/head
Igor Fedotov [Tue, 13 Aug 2024 16:15:36 +0000 (19:15 +0300)]
test/store_test: fix assertions due to unclosed collection refs.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
11 months agoqa/cephadm: add group param when applying nvmeof 58860/head
Adam King [Fri, 9 Aug 2024 17:16:30 +0000 (13:16 -0400)]
qa/cephadm: add group param when applying nvmeof

Since it will now be required

Signed-off-by: Adam King <adking@redhat.com>
11 months agomgr/cephadm: make nvme-gw adds be able to handle multiple services/groups
Adam King [Wed, 31 Jul 2024 17:30:02 +0000 (13:30 -0400)]
mgr/cephadm: make nvme-gw adds be able to handle multiple services/groups

Before this was grabbing the service spec for the first daemon
description in the list. This meant every daemon would be added
with the pool/group of whatever that spec happened to specify.
This patch grabs the spec, and therefore also the pool/group
individually for each nvmeof daemon

Signed-off-by: Adam King <adking@redhat.com>
11 months agomgr/cephadm: migrate nvmeof specs without group field
Adam King [Thu, 25 Jul 2024 17:36:07 +0000 (13:36 -0400)]
mgr/cephadm: migrate nvmeof specs without group field

As we have added the group field as a requirement for new
nvmeof specs and check for it in spec validation, we need
a migration to populate this field for specs we find that
don't have it.

Signed-off-by: Adam King <adking@redhat.com>
11 months agomgr/orchestrator: require "group" field for nvmeof specs
Adam King [Thu, 25 Jul 2024 17:35:35 +0000 (13:35 -0400)]
mgr/orchestrator: require "group" field for nvmeof specs

Signed-off-by: Adam King <adking@redhat.com>
11 months agoMerge pull request #59112 from idryomov/wip-67353
Ilya Dryomov [Tue, 13 Aug 2024 14:39:05 +0000 (16:39 +0200)]
Merge pull request #59112 from idryomov/wip-67353

qa: adjust expected io_opt in krbd_discard_granularity.t

Reviewed-by: Ramana Raja <rraja@redhat.com>
11 months agoMerge pull request #54504 from aclamk/wip-aclamk-bs-refactor-write-path
Adam Kupczyk [Tue, 13 Aug 2024 13:15:50 +0000 (15:15 +0200)]
Merge pull request #54504 from aclamk/wip-aclamk-bs-refactor-write-path

os/bluestore: Recompression, part 2. New write path.

11 months agoMerge pull request #59001 from cbodley/wip-rgw-async-inline-gc
Casey Bodley [Tue, 13 Aug 2024 12:25:53 +0000 (08:25 -0400)]
Merge pull request #59001 from cbodley/wip-rgw-async-inline-gc

rgw/rados: delete_objs_inline() uses rgw_rados_operate()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
11 months agomgr/dashboard: replace individual cluster's capacity info with Usage bar in Multi... 59186/head
Naman Munet [Tue, 13 Aug 2024 07:38:32 +0000 (13:08 +0530)]
mgr/dashboard: replace individual cluster's capacity info with Usage bar in Multi-Cluster

fixes: https://tracker.ceph.com/issues/67193

Signed-off-by: Naman Munet <nmunet@redhat.com>
11 months agoMerge pull request #58410 from bill-scales/erasure_coding_design_document
Zac Dover [Tue, 13 Aug 2024 02:34:05 +0000 (12:34 +1000)]
Merge pull request #58410 from bill-scales/erasure_coding_design_document

doc: doc/dev/osd_interals/erasure_coding/enhancements.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Sam Just <sjust@redhat.com>
11 months agoMerge pull request #59177 from anthonyeleven/aad-qat-followup
Zac Dover [Tue, 13 Aug 2024 02:17:25 +0000 (12:17 +1000)]
Merge pull request #59177 from anthonyeleven/aad-qat-followup

doc/radosgw: improve qat-accel.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>
11 months agodoc/radosgw: improve qat-accel.rst 59177/head
Anthony D'Atri [Tue, 13 Aug 2024 01:38:52 +0000 (21:38 -0400)]
doc/radosgw: improve qat-accel.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>