]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
19 months agoMerge pull request #54679 from Suyashd999/add-rgw
Adam King [Mon, 4 Dec 2023 12:26:13 +0000 (07:26 -0500)]
Merge pull request #54679 from Suyashd999/add-rgw

src/cephadm: Added rgw for listing in Daemons

Reviewed-by: Adam King <adking@redhat.com>
19 months agoMerge pull request #54076 from rhcs-dashboard/subvolume-snapshot
Nizamudeen A [Mon, 4 Dec 2023 08:42:51 +0000 (14:12 +0530)]
Merge pull request #54076 from rhcs-dashboard/subvolume-snapshot

mgr/dashboard: cephfs subvolume list snapshots

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: ivoalmeida <NOT@FOUND>
19 months agoMerge pull request #54758 from zdover23/wip-doc-2023-12-03-rados-stretch-mode-repair...
Anthony D'Atri [Sun, 3 Dec 2023 14:40:46 +0000 (09:40 -0500)]
Merge pull request #54758 from zdover23/wip-doc-2023-12-03-rados-stretch-mode-repair-of-54689

doc/rados: repair stretch-mode.rst

19 months agodoc/rados: repair stretch-mode.rst 54758/head
Zac Dover [Sun, 3 Dec 2023 12:17:46 +0000 (13:17 +0100)]
doc/rados: repair stretch-mode.rst

Remove a section of doc/rados/operations/stretch-mode.rst that I wrongly
re-included after its removal. The request for this (re)-removal is
here: https://github.com/ceph/ceph/pull/54689#discussion_r1413007655.

Signed-off-by: Zac Dover <zac.dover@proton.me>
19 months agoMerge pull request #54437 from Matan-B/wip-matanb-crimson-osdmap-trimming
Matan Breizman [Sun, 3 Dec 2023 08:06:33 +0000 (10:06 +0200)]
Merge pull request #54437 from Matan-B/wip-matanb-crimson-osdmap-trimming

crimson/osd: introduce osdmap trimming

Reviewed-by: Samuel Just <sjust@redhat.com>
19 months agoMerge pull request #53712 from ronen-fr/wip-rf-move-1
Ronen Friedman [Sun, 3 Dec 2023 07:51:41 +0000 (09:51 +0200)]
Merge pull request #53712 from ronen-fr/wip-rf-move-1

osd: fix a missing 'noexcept' on a move ctor

Reviewed-by: Samuel Just <sjust@redhat.com>
19 months agoMerge pull request #54482 from ronen-fr/wip-rf-repl-hp
Ronen Friedman [Sun, 3 Dec 2023 07:45:18 +0000 (09:45 +0200)]
Merge pull request #54482 from ronen-fr/wip-rf-repl-hp

osd/scrub: decouple being reserved from handling scrub requests

Reviewed-by: Samuel Just <sjust@redhat.com>
19 months agoMerge pull request #54689 from PC-Admin/update-stretch-mode-docs
zdover23 [Sun, 3 Dec 2023 05:56:59 +0000 (06:56 +0100)]
Merge pull request #54689 from PC-Admin/update-stretch-mode-docs

doc/rados: Update stretch mode docs.

Reviewed-by: Zac Dover <zac.dover@gmail.com>
19 months agodoc/rados: update "stretch mode" 54689/head
PC-Admin [Tue, 28 Nov 2023 09:45:52 +0000 (17:45 +0800)]
doc/rados: update "stretch mode"

Update stretch mode docs, min_size and max_size are no longer defined in
the CRUSH map and the example rule given will fail to compile.

Specify that the tiebreaker data centre cannot be defined in CRUSH as
this produces an error.

Signed-off-by: Michael Collins <perthserverplus@gmail.com>
19 months agoosd: fix a missing 'noexcept' on a move ctor 53712/head
Ronen Friedman [Thu, 28 Sep 2023 12:04:06 +0000 (07:04 -0500)]
osd: fix a missing 'noexcept' on a move ctor

as a non-default, non-noexcept move ctor is ignored by
stl containers.

See clang-tidy's performance-noexcept-move-constructor

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
19 months agoMerge pull request #54752 from zdover23/wip-doc-2023-12-02-radosgw-multisite-formatting
zdover23 [Sat, 2 Dec 2023 06:22:12 +0000 (07:22 +0100)]
Merge pull request #54752 from zdover23/wip-doc-2023-12-02-radosgw-multisite-formatting

doc/radosgw: fix formatting

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
19 months agodoc/radosgw: fix formatting 54752/head
Zac Dover [Sat, 2 Dec 2023 05:38:28 +0000 (06:38 +0100)]
doc/radosgw: fix formatting

Repair the formatting of a string that had a string inside backticks
that itself was inside double asterisks. The presence of the asterisks
around the entire string caused the backticks to appear in the rendered
documentation.

Signed-off-by: Zac Dover <zac.dover@proton.me>
19 months agoMerge pull request #54526 from cbodley/wip-rgw-acl-cleanup
Casey Bodley [Fri, 1 Dec 2023 16:46:33 +0000 (16:46 +0000)]
Merge pull request #54526 from cbodley/wip-rgw-acl-cleanup

rgw/acl: clean up RGWAccessControlPolicy and friends

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
19 months agoMerge pull request #54488 from cbodley/wip-versioned-variant
Casey Bodley [Fri, 1 Dec 2023 16:02:20 +0000 (16:02 +0000)]
Merge pull request #54488 from cbodley/wip-versioned-variant

common: add versioned encodings for std::variant

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
19 months agotests/scrub: deactivate osd-scrub-dump stand-alone test 54482/head
Ronen Friedman [Fri, 1 Dec 2023 14:48:49 +0000 (08:48 -0600)]
tests/scrub: deactivate osd-scrub-dump stand-alone test

as the scrub reservation changes had made it obsolete.
Note - it is not an issue of fixing the test, but rather
that the tested functionality is no longer there.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
19 months agocommon: add generate_test_instances() for std::variant 54488/head
Casey Bodley [Tue, 14 Nov 2023 19:43:25 +0000 (14:43 -0500)]
common: add generate_test_instances() for std::variant

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agocommon: add versioned encodings for std::variant
Casey Bodley [Tue, 14 Nov 2023 01:05:47 +0000 (20:05 -0500)]
common: add versioned encodings for std::variant

adds two encoding strategies for `std::variant<>` under the namespaces
`ceph::versioned_variant` and `ceph::converted_variant`

these versioned encodings allow the variant to be extended with new
types, provided that they're always added to the end without changing
or removing existing types. because of this requirement, no default
encoding is provided for `std::variant`. callers must opt in to one
namespace or the other

the `converted_variant` encoding requires the variant's first type T
to use versioned encoding, and guarantees that the variant's encoding
is backward-compatible with T's

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agomgr/dashboard: fs snapshots e2e 54076/head
Nizamudeen A [Fri, 27 Oct 2023 08:15:44 +0000 (13:45 +0530)]
mgr/dashboard: fs snapshots e2e

Fixes: https://tracker.ceph.com/issues/63237
Signed-off-by: Nizamudeen A <nia@redhat.com>
19 months agomgr/dashboard: cephfs subvolume list snapshots
Nizamudeen A [Wed, 18 Oct 2023 17:46:09 +0000 (23:16 +0530)]
mgr/dashboard: cephfs subvolume list snapshots

Added a tab for displaying the subvolume snapshots
- this tab will show an info alert when there are no subvolumes present
- if the subvolume is present, then it'll be auto-selected by default

Implemented a filter to search the groups and subvolumes by its name.
Also added a scrollbar when there are too many items in the nav list

Modified the REST APIs to fetch only the names of the resources and
fetch the info when an API call is requesting for it.

Added unit tests

Fixes: https://tracker.ceph.com/issues/63237
Signed-off-by: Nizamudeen A <nia@redhat.com>
19 months agoMerge pull request #54648 from rhcs-dashboard/fs-rename-fix
Nizamudeen A [Fri, 1 Dec 2023 12:51:49 +0000 (18:21 +0530)]
Merge pull request #54648 from rhcs-dashboard/fs-rename-fix

mgr/dashboard: fs rename only when fs is offline

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: ivoalmeida <NOT@FOUND>
19 months agoMerge pull request #54731 from zdover23/wip-doc-2023-11-30-rados-tshooting-log-and...
zdover23 [Fri, 1 Dec 2023 10:43:24 +0000 (11:43 +0100)]
Merge pull request #54731 from zdover23/wip-doc-2023-11-30-rados-tshooting-log-and-debug-note

doc/rados: improve formatting of log-and-debug.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
19 months agoMerge pull request #54730 from zdover23/wip-doc-2023-11-30-glossary-monitor-store
zdover23 [Fri, 1 Dec 2023 09:44:41 +0000 (10:44 +0100)]
Merge pull request #54730 from zdover23/wip-doc-2023-11-30-glossary-monitor-store

doc/glossary.rst: add "Monitor Store"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
19 months agodoc/rados: improve formatting of log-and-debug.rst 54731/head
Zac Dover [Thu, 30 Nov 2023 08:16:38 +0000 (09:16 +0100)]
doc/rados: improve formatting of log-and-debug.rst

Improve the arrangment of information in the section "Ceph Subsystems"
in doc/rados/troubleshooting/log-and-debug.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
19 months agoMerge pull request #54724 from ajarr/wip-63673
Ilya Dryomov [Fri, 1 Dec 2023 09:33:10 +0000 (10:33 +0100)]
Merge pull request #54724 from ajarr/wip-63673

qa/workunits/rbd/cli_generic.sh: narrow race window

... when checking that rbd_support module command fails after blocklisting the module's client.

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
19 months agomgr/dashboard: generalized code-block component 54648/head
Nizamudeen A [Thu, 30 Nov 2023 09:04:41 +0000 (14:34 +0530)]
mgr/dashboard: generalized code-block component

Fixes: https://tracker.ceph.com/issues/63608
Signed-off-by: Nizamudeen A <nia@redhat.com>
19 months agorgw/acl: remove virtual destructor of ACLGranteeType 54526/head
Casey Bodley [Tue, 28 Nov 2023 13:51:41 +0000 (08:51 -0500)]
rgw/acl: remove virtual destructor of ACLGranteeType

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl: ACLGrant uses variant for grantee types
Casey Bodley [Sat, 18 Nov 2023 16:27:50 +0000 (11:27 -0500)]
rgw/acl: ACLGrant uses variant for grantee types

use of `ACLGrant::get_id()` was awkward because most grantee types
returned nothing, but emails were returned as `struct rgw_user`. change
the internal representation into a variant, and expose getters for each
grantee type so callers can handle each type specifically. the encoded
format of `ACLGrant` remains unchanged

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl: req_state stores ACLs by value instead of unique_ptr
Casey Bodley [Sat, 18 Nov 2023 15:22:20 +0000 (10:22 -0500)]
rgw/acl: req_state stores ACLs by value instead of unique_ptr

we no longer rely on polymorphism for the s3/swift variants of
`RGWAccessControlPolicy`, so `req_state` can store `bucket_acl`,
`object_acl` and `user_acl` by value

most functions now take these acls by const- or mutable reference
instead of pointers since they won't be nullptr

some code paths won't initialize some of these bucket/object/user acl
variables, and we rely on `RGWAccessControlPolicy::verify_permissions()`
to return false for those because we won't match an empty owner or
array of grants

in only one case, `verify_user_permissions()` has to return true when
`user_acl` is uninitialized, because S3 doesn't have user acls so
uninitialized user acls should not deny access

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/s3: parse_policy() as free function
Casey Bodley [Sat, 18 Nov 2023 02:29:25 +0000 (21:29 -0500)]
rgw/acl/s3: parse_policy() as free function

s3 acl parsing classes no longer inherit from the acl classes
themselves, and are all encapsulated in rgw_acl_s3.cc behind a single
rgw::s3::parse_policy() function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/s3: write_policy_xml() as free function
Casey Bodley [Fri, 17 Nov 2023 20:31:17 +0000 (15:31 -0500)]
rgw/acl/s3: write_policy_xml() as free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/s3: create_policy_from_headers() as free function
Casey Bodley [Thu, 16 Nov 2023 21:38:31 +0000 (16:38 -0500)]
rgw/acl/s3: create_policy_from_headers() as free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/s3: create_canned_acl() as free function
Casey Bodley [Thu, 16 Nov 2023 20:13:25 +0000 (15:13 -0500)]
rgw/acl/s3: create_canned_acl() as free function

adds canned acls to base class RGWAccessControlPolicy

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl: use create_default() instead of empty canned acls
Casey Bodley [Thu, 16 Nov 2023 20:11:33 +0000 (15:11 -0500)]
rgw/acl: use create_default() instead of empty canned acls

create_default() adds the same default acl, without the relation to s3
canned acls

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: remove RGWAccessControlPolicy subclasses
Casey Bodley [Thu, 16 Nov 2023 19:05:36 +0000 (14:05 -0500)]
rgw/acl/swift: remove RGWAccessControlPolicy subclasses

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: format_account_acl() as free function
Casey Bodley [Thu, 16 Nov 2023 18:59:57 +0000 (13:59 -0500)]
rgw/acl/swift: format_account_acl() as free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: create_account_policy() as free function
Casey Bodley [Thu, 16 Nov 2023 18:59:18 +0000 (13:59 -0500)]
rgw/acl/swift: create_account_policy() as free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: add_grants() as static free function
Casey Bodley [Thu, 16 Nov 2023 18:43:19 +0000 (13:43 -0500)]
rgw/acl/swift: add_grants() as static free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: add_grants() uses user_to_grant() helper
Casey Bodley [Thu, 16 Nov 2023 18:42:21 +0000 (13:42 -0500)]
rgw/acl/swift: add_grants() uses user_to_grant() helper

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: format_container_acls() as free function
Casey Bodley [Thu, 16 Nov 2023 18:33:30 +0000 (13:33 -0500)]
rgw/acl/swift: format_container_acls() as free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: merge_policy() as free function
Casey Bodley [Thu, 16 Nov 2023 18:25:36 +0000 (13:25 -0500)]
rgw/acl/swift: merge_policy() as free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: create_container_policy() as free function
Casey Bodley [Thu, 16 Nov 2023 18:08:45 +0000 (13:08 -0500)]
rgw/acl/swift: create_container_policy() as free function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: parse_grant() as static function
Casey Bodley [Thu, 16 Nov 2023 18:00:25 +0000 (13:00 -0500)]
rgw/acl/swift: parse_grant() as static function

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl/swift: parse acl header with ceph::split()
Casey Bodley [Thu, 16 Nov 2023 17:44:21 +0000 (12:44 -0500)]
rgw/acl/swift: parse acl header with ceph::split()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl: add_grant() takes const ref
Casey Bodley [Thu, 16 Nov 2023 15:46:35 +0000 (10:46 -0500)]
rgw/acl: add_grant() takes const ref

that also required fixing some const-correctness issues

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl: remove CephContext members
Casey Bodley [Thu, 16 Nov 2023 02:07:59 +0000 (21:07 -0500)]
rgw/acl: remove CephContext members

remove the CephContext member variables and pass in
DoutPrefixProvider for logging where it's necessary

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw: remove unused policy_bl_to_stream()
Casey Bodley [Thu, 16 Nov 2023 02:01:27 +0000 (21:01 -0500)]
rgw: remove unused policy_bl_to_stream()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl: remove unused virtual compare_group_name()
Casey Bodley [Thu, 16 Nov 2023 01:17:54 +0000 (20:17 -0500)]
rgw/acl: remove unused virtual compare_group_name()

that was the only virtual function, so remove virtual destructors too

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw: ACLOwner as raw struct
Casey Bodley [Mon, 13 Nov 2023 18:14:13 +0000 (13:14 -0500)]
rgw: ACLOwner as raw struct

this was a class with two protected members, but with a getter/setter
for each. this 'encapsulation' only added unnecessary boilerplate

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw: reorder rgw_user members for default operator<=>
Casey Bodley [Sat, 18 Nov 2023 16:43:58 +0000 (11:43 -0500)]
rgw: reorder rgw_user members for default operator<=>

the default operator<=> does a memberwise comparison the same that
rgw_user::compare() did, except that it compared `ns` before `id`

reorder the rgw_user members so that the default operator<=> can
replace compare() and the related comparison operators

replaces uses of rgw_user::compare() with operator== and !=

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agorgw/acl: remove empty base class ACLGrantee
Casey Bodley [Fri, 3 Nov 2023 20:08:28 +0000 (16:08 -0400)]
rgw/acl: remove empty base class ACLGrantee

an unnecessary base class of ACLGrantee_S3 which is just for s3
xml parsing

Signed-off-by: Casey Bodley <cbodley@redhat.com>
19 months agoMerge pull request #54304 from cyx1231st/wip-crimson-make-crosscore-send-ordered
Matan Breizman [Thu, 30 Nov 2023 12:39:31 +0000 (14:39 +0200)]
Merge pull request #54304 from cyx1231st/wip-crimson-make-crosscore-send-ordered

crimson/osd: support to send messages concurrently with exclusive phases

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
19 months agoMerge pull request #54697 from yuvalif/wip-yuval-63314
Yuval Lifshitz [Thu, 30 Nov 2023 12:23:28 +0000 (14:23 +0200)]
Merge pull request #54697 from yuvalif/wip-yuval-63314

rgw/kafka/amqp: fix race conditionn in async completion handlers

reviewed-by: cbodley

19 months agoosd/scrub: handling unexpected scrub requests
Ronen Friedman [Thu, 30 Nov 2023 06:35:29 +0000 (00:35 -0600)]
osd/scrub: handling unexpected scrub requests

if arriving while still handling a previous chunk request,
the handling of of the previous chunk request will be aborted.
No response is sent. The scrubber resets, then handles the new request.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
19 months agoosd/scrub: separate counters for primary vs. replica scrubs
Ronen Friedman [Mon, 20 Nov 2023 13:31:31 +0000 (07:31 -0600)]
osd/scrub: separate counters for primary vs. replica scrubs

The OSD limits the number of concurrent scrubs performed on its PGs.
This limit is now enforced separately for primary and replica scrubs.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
19 months agoosd/scrub: skip reserving replicas for high-priority scrubs
Ronen Friedman [Tue, 14 Nov 2023 12:01:08 +0000 (06:01 -0600)]
osd/scrub: skip reserving replicas for high-priority scrubs

(The primary side of the required changes)

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
19 months agoosd/scrub: decouple being reserved from handling scrub requests
Ronen Friedman [Mon, 13 Nov 2023 07:09:45 +0000 (01:09 -0600)]
osd/scrub: decouple being reserved from handling scrub requests

For a replica, following this change:

* 'ReplicaActive' captures the state of the scrubber when
  acting as a replica, from peering to interval change;

* "being reserved" is just a flag maintained by ReplicaActive, and
  is no longer a prerequisite for handling scrub requests.

* each scrub request is now associated with its own 'token' value.

and the following minor simplification:

* the 'should we wait for pushes' decision is now part of the
  code executed on the transition from ReplicaIdle into ReplicaActiveOp.
  StartReplicaNoWait can now be discarded.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
19 months agoMerge pull request #54305 from rhcs-dashboard/add-tags
Nizamudeen A [Thu, 30 Nov 2023 09:07:29 +0000 (14:37 +0530)]
Merge pull request #54305 from rhcs-dashboard/add-tags

mgr/dashboard: add tags to edit bucket

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
19 months agoMerge pull request #54355 from nobuto-m/info-rbd-stats-pools
Nizamudeen A [Thu, 30 Nov 2023 08:08:54 +0000 (13:38 +0530)]
Merge pull request #54355 from nobuto-m/info-rbd-stats-pools

mgr/dashboard: info on why RBD graphs are empty

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
19 months agodoc/glossary.rst: add "Monitor Store" 54730/head
Zac Dover [Thu, 30 Nov 2023 07:08:00 +0000 (08:08 +0100)]
doc/glossary.rst: add "Monitor Store"

Add the term "Monitor Store" to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>
19 months agoMerge PR #53431 into main
Venky Shankar [Thu, 30 Nov 2023 05:58:10 +0000 (11:28 +0530)]
Merge PR #53431 into main

* refs/pull/53431/head:
qa: add test cases to verify error reporting works as expected
mgr: fix some doc strings in object_format.py
mgr/tests: test returning error status works as expected
mgr: make object_format's Responder class capable of responding err status
mgr/nfs: report proper errno with err status

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
19 months agoMerge pull request #54631 from rishabh-d-dave/mgr-vol-tests
Rishabh Dave [Thu, 30 Nov 2023 05:17:42 +0000 (10:47 +0530)]
Merge pull request #54631 from rishabh-d-dave/mgr-vol-tests

qa/cephfs: improvements for name generators in test_volumes.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
19 months agoMerge pull request #50359 from adamemerson/wip-remove-rgwsi_rados
Casey Bodley [Wed, 29 Nov 2023 21:54:36 +0000 (21:54 +0000)]
Merge pull request #50359 from adamemerson/wip-remove-rgwsi_rados

rgw: Remove `RGWSI_RADOS`

Reviewed-by: Casey Bodley <cbodley@redhat.com>
19 months agoMerge pull request #54616 from rishabh-d-dave/mgr-vol-improvements
Rishabh Dave [Wed, 29 Nov 2023 19:20:26 +0000 (00:50 +0530)]
Merge pull request #54616 from rishabh-d-dave/mgr-vol-improvements

mgr/vol: make some improvements

Reviewed-by: Venky Shankar <vshankar@redhat.com>
19 months agoMerge pull request #54308 from rishabh-d-dave/mgr-vol-flake-warnings
Rishabh Dave [Wed, 29 Nov 2023 19:19:28 +0000 (00:49 +0530)]
Merge pull request #54308 from rishabh-d-dave/mgr-vol-flake-warnings

mgr/vol: fix flake8 warnings

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
19 months agoqa/workunits/rbd/cli_generic.sh: narrow race window 54724/head
Ramana Raja [Wed, 29 Nov 2023 16:25:30 +0000 (11:25 -0500)]
qa/workunits/rbd/cli_generic.sh: narrow race window

... when checking whether a rbd_support module command fails after
blocklisting the module's client.

In tests that check the recovery of the rbd_support module after its
client is blocklisted, the rbd_support module's client is
blocklisted using the `osd blocklist add` command. Next,
`osd blocklist ls` command is issued to confirm that the client is
blocklisted. A rbd_support module command is then issued and expected
to fail in order to verify that the blocklisting has affected the
rbd_support module's operations. Sometimes it was observed that before
this rbd_support module command reached the ceph-mgr, the rbd_support
module detected the blocklisting, recovered from it, and was able to
serve the command. To reduce the race window that occurs when trying to
verify that the rbd_support module's operation is affected by client
blocklisting, get rid of the `osd blocklist ls` command.

Fixes: https://tracker.ceph.com/issues/63673
Signed-off-by: Ramana Raja <rraja@redhat.com>
19 months agoMerge pull request #54424 from phlogistonjohn/jjm-cephadm-daemon-to-container
Adam King [Wed, 29 Nov 2023 18:48:46 +0000 (13:48 -0500)]
Merge pull request #54424 from phlogistonjohn/jjm-cephadm-daemon-to-container

cephadm: add a more generic daemon_to_container function

Reviewed-by: Adam King <adking@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` 50359/head
Adam C. Emerson [Tue, 6 Dec 2022 00:51:25 +0000 (19:51 -0500)]
rgw: Remove `RGWSI_RADOS`

And remove the last places that reference it.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` from `RGWSI_SysObj*`
Adam C. Emerson [Mon, 5 Dec 2022 22:10:51 +0000 (17:10 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_SysObj*`

Simply use the RADOS handle and `rgw_rados_ref` directly.

Also make `rgw::AccessListFilter` a std::function.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` from `RGWSI_User_RADOS`
Adam C. Emerson [Fri, 2 Dec 2022 07:59:11 +0000 (02:59 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_User_RADOS`

Simply use the RADOS handle and `rgw_rados_ref` directly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` from `RGWSI_Zone_Tools`
Adam C. Emerson [Fri, 2 Dec 2022 07:49:09 +0000 (02:49 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_Zone_Tools`

Simply use the RADOS handle directly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` from `RGWSI_Zone`
Adam C. Emerson [Fri, 2 Dec 2022 07:30:39 +0000 (02:30 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_Zone`

Simply use the RADOS handle directly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` from `RGWSI_Notify`
Adam C. Emerson [Fri, 2 Dec 2022 07:17:44 +0000 (02:17 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_Notify`

Simply use the RADOS handle directly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` from `RGWSI_MDLog`
Adam C. Emerson [Fri, 2 Dec 2022 06:55:47 +0000 (01:55 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_MDLog`

Simply use the RADOS handle and `rgw_rados_ref` directly.

Also move `async_processor` out from `RGWSI_RADOS` and into
`RGWServices_Def`. This is as good a place as it for any, for now, as
it's reachable by everyone who needs it and exists through the
lifetime of the process.

Eventually it's going to go away due to coroutinization, anyway.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` from `RGWSI_ConfigKey_RADOS`
Adam C. Emerson [Fri, 2 Dec 2022 06:14:02 +0000 (01:14 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_ConfigKey_RADOS`

Simply use the RADOS handle directly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove `RGWSI_RADOS` from `RGWSI_Cls`
Adam C. Emerson [Fri, 2 Dec 2022 05:55:48 +0000 (00:55 -0500)]
rgw: Remove `RGWSI_RADOS` from `RGWSI_Cls`

Simply use the RADOS handle and `rgw_rados_ref` directly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Remove RGWSI_RADOS from RGWSI_BI_RADOS
Adam C. Emerson [Fri, 2 Dec 2022 05:26:00 +0000 (00:26 -0500)]
rgw: Remove RGWSI_RADOS from RGWSI_BI_RADOS

Simply use the RADOS handle and `rgw_rados_ref` directly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agorgw: Distentangle `rgw_rados_ref` from `RGWSI_RADOS`
Adam C. Emerson [Fri, 2 Dec 2022 04:25:54 +0000 (23:25 -0500)]
rgw: Distentangle `rgw_rados_ref` from `RGWSI_RADOS`

`RGWSI_RADOS` has no reason to exist. `rgw_rados_ref`, on the other
hand, is fairly useful. Move `rgw_rados_ref` to
`store/rados/rgw_tools.h` along with a function to create one.

Then clean up the fallout.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
19 months agoMerge pull request #54619 from cbodley/wip-rgw-sal-get-stats-async
Casey Bodley [Wed, 29 Nov 2023 16:24:26 +0000 (16:24 +0000)]
Merge pull request #54619 from cbodley/wip-rgw-sal-get-stats-async

rgw/sal: clean up read_stats_async() callbacks

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
19 months agoMerge pull request #54582 from phlogistonjohn/jjm-cephadm-more-template-error
Adam King [Wed, 29 Nov 2023 15:55:54 +0000 (10:55 -0500)]
Merge pull request #54582 from phlogistonjohn/jjm-cephadm-more-template-error

cephadm: add a custom template not found exception with diagnostic info

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
19 months agoqa/cephfs: improvements for name generators in test_volumes.py 54631/head
Rishabh Dave [Thu, 23 Nov 2023 10:40:00 +0000 (16:10 +0530)]
qa/cephfs: improvements for name generators in test_volumes.py

Generate a name that is shorter and easier to remember.

Also, write a simpler, faster & better helper method for generating
unique names. This method will also have shorter and more concise name,
so this will be easier to type and easier to read.

Fixes: https://tracker.ceph.com/issues/63680
Signed-off-by: Rishabh Dave <ridave@redhat.com>
19 months agoMerge pull request #54618 from alimaredia/wip-remove-vstart-runner-radosgw-admin...
Casey Bodley [Wed, 29 Nov 2023 15:18:57 +0000 (15:18 +0000)]
Merge pull request #54618 from alimaredia/wip-remove-vstart-runner-radosgw-admin-task

qa: remove vstart runner from radosgw_admin task

Reviewed-by: Casey Bodley <cbodley@redhat.com>
19 months agoMerge pull request #54578 from ronen-fr/wip-rf-dedup-clang
Ronen Friedman [Wed, 29 Nov 2023 13:39:37 +0000 (15:39 +0200)]
Merge pull request #54578 from ronen-fr/wip-rf-dedup-clang

tools: modify ceph_dedup_tool to maintain Clang 15 compatibility

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
19 months agoosd/scrub: use OpCarryingEvent for scrub events carrying common data
Ronen Friedman [Wed, 29 Nov 2023 13:16:24 +0000 (07:16 -0600)]
osd/scrub: use OpCarryingEvent for scrub events carrying common data

Copied from a similar implementation in Crimson.
The payload includes the MOSDScrub* op, and the 'from' id.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Split from "osd/scrub: move responsibility for clearing the 'being reserved' state"

19 months agoPeeringState: introduce optional on_replica_activate notification
Samuel Just [Tue, 19 Sep 2023 22:05:07 +0000 (15:05 -0700)]
PeeringState: introduce optional on_replica_activate notification

The OSD will use it to activate the scrubber in replica mode.

Signed-off-by: Samuel Just <sjust@redhat.com>
19 months agoMerge pull request #48483 from batrick/i48673
Venky Shankar [Wed, 29 Nov 2023 10:13:54 +0000 (15:43 +0530)]
Merge pull request #48483 from batrick/i48673

mds: revert standby-replay trimming changes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
19 months agorgw/kafka/amqp: fix race conditionn in async completion handlers 54697/head
Yuval Lifshitz [Tue, 28 Nov 2023 17:49:03 +0000 (17:49 +0000)]
rgw/kafka/amqp: fix race conditionn in async completion handlers

Fixes: https://tracker.ceph.com/issues/63314
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
19 months agocrimson/osd: update osd_beacon_report_interval on conf change 54437/head
Matan Breizman [Thu, 23 Nov 2023 15:15:58 +0000 (15:15 +0000)]
crimson/osd: update osd_beacon_report_interval on conf change

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
19 months agocrimson/osd: OSDBeacon to report lec
Matan Breizman [Sun, 12 Nov 2023 13:31:40 +0000 (13:31 +0000)]
crimson/osd: OSDBeacon to report lec

See: https://github.com/ceph/ceph/pull/14504

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
19 months agocrimson/osd: update TODO comments
Matan Breizman [Sun, 12 Nov 2023 08:29:50 +0000 (08:29 +0000)]
crimson/osd: update TODO comments

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
19 months agoosd/crimson/pg: support cluster_osdmap_trim_lower_bound()
Matan Breizman [Thu, 9 Nov 2023 16:34:44 +0000 (16:34 +0000)]
osd/crimson/pg: support cluster_osdmap_trim_lower_bound()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
19 months agocrimson/osd: update superblock managment
Matan Breizman [Thu, 23 Nov 2023 14:42:28 +0000 (14:42 +0000)]
crimson/osd: update superblock managment

introduce PerShardState::superblock to be updated on
each shard once the OSDSingletonState instance is updated.

Note: PerShardState::superblock is a dependency for
      PG::cluster_osdmap_trim_lower_bound().

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
19 months agocrimson/osd: introduce osdmap trimming
Matan Breizman [Thu, 9 Nov 2023 12:44:42 +0000 (12:44 +0000)]
crimson/osd: introduce osdmap trimming

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
19 months agocrimson/osd/osd_meta: add OSDMeta::remove_map
Matan Breizman [Sun, 12 Nov 2023 08:15:23 +0000 (08:15 +0000)]
crimson/osd/osd_meta: add OSDMeta::remove_map

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
19 months agocrimson/common/shared_lru: introduce cached_key_lower_bound()
Matan Breizman [Thu, 9 Nov 2023 16:51:20 +0000 (16:51 +0000)]
crimson/common/shared_lru: introduce cached_key_lower_bound()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
19 months agoMerge pull request #54287 from Matan-B/wip-matanb-crimson-do_osd_ops_execute-v3
Matan Breizman [Wed, 29 Nov 2023 08:13:41 +0000 (10:13 +0200)]
Merge pull request #54287 from Matan-B/wip-matanb-crimson-do_osd_ops_execute-v3

crimson/osd: execute `submit_error_log` as an ExclusivePhase

Reviewed-by: Samuel Just <sjust@redhat.com>
19 months agoMerge pull request #54209 from VallariAg/wip-xml-scanner
Vallari Agrawal [Wed, 29 Nov 2023 06:51:02 +0000 (12:21 +0530)]
Merge pull request #54209 from VallariAg/wip-xml-scanner

qa: use Remote.run_unit_test and ValgrindScanner

19 months agoMerge pull request #54685 from zdover23/wip-doc-2023-11-28-rados-tshooting-log-and...
zdover23 [Wed, 29 Nov 2023 04:39:10 +0000 (05:39 +0100)]
Merge pull request #54685 from zdover23/wip-doc-2023-11-28-rados-tshooting-log-and-debug-small-repair

doc/rados: improve "Ceph Subsystems"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
19 months agoMerge pull request #54684 from zdover23/wip-doc-2023-11-28-rados-ops-pgs-typo
zdover23 [Wed, 29 Nov 2023 04:34:43 +0000 (05:34 +0100)]
Merge pull request #54684 from zdover23/wip-doc-2023-11-28-rados-ops-pgs-typo

doc/rados: ops/pgs: s/power of 2/power of two

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
19 months agodoc/rados: improve "Ceph Subsystems" 54685/head
Zac Dover [Tue, 28 Nov 2023 05:08:48 +0000 (06:08 +0100)]
doc/rados: improve "Ceph Subsystems"

Improve the English in the subsection "Ceph Subsystems" in the section
"Subsystem, Log and Debug Settings" [sic] in
doc/rados/troubleshooting/log-and-debug.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
19 months agodoc/rados: ops/pgs: s/power of 2/power of two 54684/head
Zac Dover [Tue, 28 Nov 2023 04:29:50 +0000 (05:29 +0100)]
doc/rados: ops/pgs: s/power of 2/power of two

Change "power of 2" to "power of two" in
doc/rados/operations/placement-groups.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>