]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
17 months agoUpdate libkmip submodule to pull in some portability changes. 53668/head
Marcus Watts [Tue, 26 Sep 2023 07:04:35 +0000 (03:04 -0400)]
Update libkmip submodule to pull in some portability changes.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
17 months agoMerge pull request #55217 from ronen-fr/wip-rf-old-reserv
Ronen Friedman [Tue, 23 Jan 2024 19:39:35 +0000 (21:39 +0200)]
Merge pull request #55217 from ronen-fr/wip-rf-old-reserv

osd/scrub: check reservation replies for relevance

Reviewed-by: Samuel Just <sjust@redhat.com>-
17 months agoMerge pull request #55067 from yaarith/telemetry-pool-flags
Laura Flores [Tue, 23 Jan 2024 18:22:32 +0000 (12:22 -0600)]
Merge pull request #55067 from yaarith/telemetry-pool-flags

mgr/telemetry: add pool flags

17 months agoMerge pull request #55240 from rosinL/wip-fix-64032
Laura Flores [Tue, 23 Jan 2024 16:38:46 +0000 (10:38 -0600)]
Merge pull request #55240 from rosinL/wip-fix-64032

install-deps: Force remove ceph-libboost* packages

17 months agoMerge pull request #55278 from Himura2la/patch-2
zdover23 [Tue, 23 Jan 2024 15:58:23 +0000 (01:58 +1000)]
Merge pull request #55278 from Himura2la/patch-2

Docs: Specify correct fs type for mkfs on volume creation

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
17 months agoMerge pull request #55277 from yuvalif/wip-yuval-63578
Casey Bodley [Tue, 23 Jan 2024 14:18:22 +0000 (14:18 +0000)]
Merge pull request #55277 from yuvalif/wip-yuval-63578

rgw/lua: fix compilation issue when lua packages are disabled

Reviewed-by: Casey Bodley <cbodley@redhat.com>
17 months agodoc: specify correct fs type for mkfs 55278/head
Himura Kazuto [Tue, 23 Jan 2024 12:59:10 +0000 (12:59 +0000)]
doc: specify correct fs type for mkfs

The default value is ext2, which is not supported (anymore?).

Signed-off-by: Vladislav Glagolev <vladislav.glagolev@devexpress.com>
17 months agorgw/lua: fix compilation issue when lua packages are disabled 55277/head
Yuval Lifshitz [Tue, 23 Jan 2024 11:09:26 +0000 (11:09 +0000)]
rgw/lua: fix compilation issue when lua packages are disabled

Fixes: https://tracker.ceph.com/issues/63578#change-253102
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
17 months agoosd/scrub: check reservation replies for relevance 55217/head
Ronen Friedman [Wed, 17 Jan 2024 15:36:16 +0000 (09:36 -0600)]
osd/scrub: check reservation replies for relevance

Compare a token (nonce) carried in the reservation reply with the remembered
token of the reservation request.  If they don't match, the reply is
stale and should be ignored (and logged).

Fixes: https://tracker.ceph.com/issues/64052
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
17 months agoMerge pull request #55107 from ronen-fr/wip-rf-rm-penaltyq
Ronen Friedman [Tue, 23 Jan 2024 05:57:39 +0000 (07:57 +0200)]
Merge pull request #55107 from ronen-fr/wip-rf-rm-penaltyq

osd/scrub: remove the 'penalty queue' from the scrubber

Reviewed-by: Samuel Just <sjust@redhat.com>-
17 months agoMerge pull request #55269 from zdover23/wip-doc-2024-01-23-radosgw-admin-read-write...
zdover23 [Tue, 23 Jan 2024 02:31:46 +0000 (12:31 +1000)]
Merge pull request #55269 from zdover23/wip-doc-2024-01-23-radosgw-admin-read-write-global-rate-limit-config

doc/radosgw: edit "read/write global rate limit" admin.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
17 months agodoc/radosgw: edit "read/write global rate limit" admin.rst 55269/head
Zac Dover [Tue, 23 Jan 2024 02:13:10 +0000 (12:13 +1000)]
doc/radosgw: edit "read/write global rate limit" admin.rst

Edit "Reading/Writing Global Rate Limit Configuration" in
doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
17 months agoMerge pull request #55223 from athanatos/sjust/wip-64055
Samuel Just [Mon, 22 Jan 2024 21:26:19 +0000 (13:26 -0800)]
Merge pull request #55223 from athanatos/sjust/wip-64055

crimson: clear obc_registry on interval change

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
17 months agoMerge pull request #55070 from pdvian/wip-fix-progressevent
Yuri Weinstein [Mon, 22 Jan 2024 16:18:41 +0000 (08:18 -0800)]
Merge pull request #55070 from pdvian/wip-fix-progressevent

mon: initialize ProgressEvent::add_to_ceph_s

Reviewed-by: Laura Flores <lflores@redhat.com>
17 months agoMerge pull request #53154 from ifed01/wip-ifed-no-death-tests
Yuri Weinstein [Mon, 22 Jan 2024 16:17:35 +0000 (08:17 -0800)]
Merge pull request #53154 from ifed01/wip-ifed-no-death-tests

test/store_test: get rid off assert_death.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
17 months agoMerge pull request #49415 from ljflores/wip-update-telemetry-upgrade
Yuri Weinstein [Mon, 22 Jan 2024 16:13:17 +0000 (08:13 -0800)]
Merge pull request #49415 from ljflores/wip-update-telemetry-upgrade

qa/workunits: update telemetry quincy workunits with `basic_pool_options_bluestore` collection

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
17 months agoosd/scrub: update job's NB on failure 55107/head
Ronen Friedman [Tue, 2 Jan 2024 16:09:06 +0000 (10:09 -0600)]
osd/scrub: update job's NB on failure

When a scrub job fails, update its NB to the current time plus a
fixed delay.  This prevents the job from being scheduled again
immediately.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
17 months agoosd/scrub: fix set_last_deep_scrub_stamp()
Ronen Friedman [Tue, 9 Jan 2024 14:15:33 +0000 (08:15 -0600)]
osd/scrub: fix set_last_deep_scrub_stamp()

The call should update last_scrub_stamp, too, without
requiring an extra call to on_scrub_schedule_input_change()

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
17 months agoosd/scrub: introduce a 'not before' attribute for scrub jobs
Ronen Friedman [Sun, 31 Dec 2023 16:18:09 +0000 (10:18 -0600)]
osd/scrub: introduce a 'not before' attribute for scrub jobs

The NB enables the OSD to delay the next attempt to schedule a specific
scrub job.  This is useful for jobs that have failed for whatever
reason, especially if the primary has failed to acquire the replicas.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
17 months agoosd/scrub: remove the 'penalized jobs' queue
Ronen Friedman [Sat, 30 Dec 2023 12:36:26 +0000 (06:36 -0600)]
osd/scrub: remove the 'penalized jobs' queue

The 'penalized jobs' queue was used to track scrub jobs that had failed
to acquire their replicas, and to prevent those jobs from being retried
too quickly.  This functionality will be replaced by a
simple 'not before' delay (see the next commits).

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
17 months agoMerge pull request #55262 from Matan-B/wip-matanb-crimson-bluestore-submit
Matan Breizman [Mon, 22 Jan 2024 08:44:23 +0000 (10:44 +0200)]
Merge pull request #55262 from Matan-B/wip-matanb-crimson-bluestore-submit

crimson/os/alienstore/alien_log: _flush concurrently

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
17 months agocrimson/os/alienstore/alien_log: _flush concurrently 55262/head
Matan Breizman [Sun, 21 Jan 2024 09:33:59 +0000 (09:33 +0000)]
crimson/os/alienstore/alien_log: _flush concurrently

In continuation to c15e56e386251403a876454f6a4aa186284565e1

Authored-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
17 months agoMerge pull request #55190 from zdover23/wip-doc-2024-01-16-radosgw-admin-enable-disab...
zdover23 [Sun, 21 Jan 2024 09:47:10 +0000 (19:47 +1000)]
Merge pull request #55190 from zdover23/wip-doc-2024-01-16-radosgw-admin-enable-disable-bucket-rate-limit

doc/radosgw: edit "Enable/Disable Bucket Rate Limit"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
17 months agoMerge pull request #53288 from rzarzynski/wip-crimson-dont-shadow-store-in-ecbackend
Matan Breizman [Sun, 21 Jan 2024 08:44:29 +0000 (10:44 +0200)]
Merge pull request #53288 from rzarzynski/wip-crimson-dont-shadow-store-in-ecbackend

crimson: drop store from ECBackend to not shadow PGBackend::store

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
17 months agoMerge pull request #54813 from amathuria/wip-crimson-amat-fix-config-set-cmd
Matan Breizman [Sun, 21 Jan 2024 08:43:59 +0000 (10:43 +0200)]
Merge pull request #54813 from amathuria/wip-crimson-amat-fix-config-set-cmd

src/crimson: Add support for the OSD to receive config changes

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
17 months agoMerge pull request #55127 from idryomov/wip-63341
Ilya Dryomov [Sat, 20 Jan 2024 17:43:35 +0000 (18:43 +0100)]
Merge pull request #55127 from idryomov/wip-63341

librbd: improve rbd_diff_iterate2() performance in fast-diff mode

Reviewed-by: Mykola Golub <mgolub@suse.com>
17 months agoPendingReleaseNotes: add rbd_diff_iterate2 note 55127/head
Ilya Dryomov [Sat, 20 Jan 2024 15:00:46 +0000 (16:00 +0100)]
PendingReleaseNotes: add rbd_diff_iterate2 note

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd: try to preserve object map for diff-iterate in fast-diff mode
Ilya Dryomov [Sat, 6 Jan 2024 16:08:04 +0000 (17:08 +0100)]
librbd: try to preserve object map for diff-iterate in fast-diff mode

As an optimization, try to ensure that the object map for the end
version is preloaded through the acquisition of exclusive lock and
as a consequence remains around until exclusive lock is released.
If it's not around, DiffRequest would (re)load it on each call.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: potentially use in-memory object map in DiffRequest
Ilya Dryomov [Sat, 6 Jan 2024 16:05:39 +0000 (17:05 +0100)]
librbd/object_map: potentially use in-memory object map in DiffRequest

If the object map for the end version is around (already loaded in
memory, either due to the end version being a snapshot or due to
exclusive lock being held), use it to run diff-iterate against the
beginning of time.  Since it's the only object map needed in that
case, such calls would be satisfied locally.

Fixes: https://tracker.ceph.com/issues/63341
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: decouple object map processing in DiffRequest
Ilya Dryomov [Fri, 5 Jan 2024 12:15:54 +0000 (13:15 +0100)]
librbd/object_map: decouple object map processing in DiffRequest

In preparation for potentially using in-memory object map, decouple
object map processing from loading object maps and place the logic in
prepare_for_object_map() and process_object_map().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agocommon/bit_vector: fix iterator vs reference constness confusion
Ilya Dryomov [Fri, 5 Jan 2024 11:23:24 +0000 (12:23 +0100)]
common/bit_vector: fix iterator vs reference constness confusion

T (ConstIterator or Iterator) is confused with const T here:
IteratorImpl dereference operator is wrongly overloaded on const
and returns Reference instead of ConstReference for ConstIterator.
This then fails inside bufferlist bowels because Reference is
incompatible with bufferlist::const_iterator.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: make object map in handle_load_object_map() local
Ilya Dryomov [Thu, 4 Jan 2024 10:44:46 +0000 (11:44 +0100)]
librbd/object_map: make object map in handle_load_object_map() local

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: don't resize object map in handle_load_object_map()
Ilya Dryomov [Thu, 4 Jan 2024 10:39:20 +0000 (11:39 +0100)]
librbd/object_map: don't resize object map in handle_load_object_map()

Currently it's done in two cases:

- if the loaded object map is larger than expected based on byte size,
  it's truncated to expected number of objects
- in case of deep-copy, if the loaded object map is smaller than diff
  state, it's expanded to get "track the largest of all versions in the
  set" semantics

Both of these cases can be easily dealt with without modifying the
object map.  Being able to process a const object map is needed for
working on in-memory object map which is external to DiffRequest.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agocommon/bit_vector: fix IteratorImpl post-increment operator
Ilya Dryomov [Sat, 6 Jan 2024 11:22:35 +0000 (12:22 +0100)]
common/bit_vector: fix IteratorImpl post-increment operator

It's totally broken: instead of returning the current position and
moving to the next position, it returns the next position and doesn't
move anywhere.  Luckily it hasn't been used until now.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd: drop DiffIterate::diff_object_map() declaration
Ilya Dryomov [Thu, 28 Dec 2023 09:52:11 +0000 (10:52 +0100)]
librbd: drop DiffIterate::diff_object_map() declaration

This is a leftover from commit 2b3a46801d39 ("librbd: switch
diff-iterate API to use new object-map diff helper").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd: propagate diff-iterate range to parent in fast-diff mode
Ilya Dryomov [Thu, 28 Dec 2023 09:14:18 +0000 (10:14 +0100)]
librbd: propagate diff-iterate range to parent in fast-diff mode

When getting parent diff, pass the overlap-reduced image extent instead
of the entire 0..overlap range to avoid a similar quadratic slowdown on
cloned images.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: add support for ranged diff-iterate
Ilya Dryomov [Wed, 27 Dec 2023 17:07:05 +0000 (18:07 +0100)]
librbd/object_map: add support for ranged diff-iterate

Currently diff-iterate in fast-diff mode is performed on the entire
image no matter what image extent is passed to the API.  Then, unused
diff just gets discarded as DiffIterate ends up querying only objects
that the passed image extent maps to.  This hasn't been an issue for
internal consumers ("rbd du", "rbd diff", etc) because they work on the
entire image, but turns out to lead to quadratic slowdown in some QEMU
use cases.

0..UINT64_MAX range is carved out for deep-copy which is unranged by
definition.  To get effectively unranged diff-iterate, 0..UINT64_MAX-1
range can be used.

Fixes: https://tracker.ceph.com/issues/63341
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agoinclude/intarith: introduce round_down_to()
Ilya Dryomov [Wed, 27 Dec 2023 15:18:50 +0000 (16:18 +0100)]
include/intarith: introduce round_down_to()

Same as with round_up_to(), d isn't required to be a power of two.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agotest/librbd: expand TestMockObjectMapDiffRequest edge case coverage
Ilya Dryomov [Sat, 23 Dec 2023 14:19:09 +0000 (15:19 +0100)]
test/librbd: expand TestMockObjectMapDiffRequest edge case coverage

For each covered edge case or error, run through the following
scenarios:

- where the edge case concerns snap_id_start
- where the edge case concerns snap_id_end
- where the edge case concerns intermediate snapshot and
  snap_id_start == 0 (diff against the beginning of time)
- where the edge case concerns intermediate snapshot and
  snap_id_start != 0 (diff from snapshot)

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: allow intermediate snaps to be skipped on diff-iterate
Ilya Dryomov [Sat, 23 Dec 2023 13:47:54 +0000 (14:47 +0100)]
librbd/object_map: allow intermediate snaps to be skipped on diff-iterate

In case of diff-iterate against the beginning of time, the result
depends only on the end version.  Loading and processing object maps
or intermediate snapshots is redundant and can be skipped.

This optimization is made possible by commit be507aaed15f ("librbd:
diff-iterate shouldn't ever report "new hole" against a hole") and, to
a lesser extent, the previous commit.

Getting FastDiffInvalid, LoadObjectMapError and ObjectMapTooSmall to
pass required tweaking not just expectations, but also start/end snap
ids and thus also the meaning of these tests.  This is addressed in the
next commit.

Fixes: https://tracker.ceph.com/issues/63341
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: resurrect diff-iterate behavior when image is shrunk
Ilya Dryomov [Fri, 22 Dec 2023 17:50:20 +0000 (18:50 +0100)]
librbd/object_map: resurrect diff-iterate behavior when image is shrunk

The new "track the largest of all versions in the set, diff state is
only ever grown" semantics introduced in commit 330f2a7bb94f ("librbd:
helper state machine for computing diffs between object-maps") don't
make sense for diff-iterate.  It's a waste because DiffIterate won't
query beyond the end version size -- this is baked into the API.

Limit this behavior to deep-copy and resurrect the original behavior
from 2015 for diff-iterate.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: fix diff from snapshot when image is grown
Ilya Dryomov [Fri, 22 Dec 2023 15:10:12 +0000 (16:10 +0100)]
librbd/object_map: fix diff from snapshot when image is grown

Commit 399a45e11332 ("librbd/object_map: rbd diff between two
snapshots lists entire image content") fixed most of the damage caused
by commit b81cd2460de7 ("librbd/object_map: diff state machine should
track object existence"), but the case of a "resize diff" when diffing
from snapshot was missed.  An area that was freshly allocated in image
resize is the same in principle as a freshly created image and objects
marked OBJECT_EXISTS_CLEAN are no exception.  Diff for such objects in
such an area should be set to DIFF_STATE_DATA_UPDATED, however
currently when diffing from snapshot, it's set to DIFF_STATE_DATA.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agolibrbd/object_map: drop bogus if in handle_load_object_map()
Ilya Dryomov [Wed, 20 Dec 2023 11:22:17 +0000 (12:22 +0100)]
librbd/object_map: drop bogus if in handle_load_object_map()

It became redundant with commit b81cd2460de7 ("librbd/object_map: diff
state machine should track object existence") -- it != end_it condition
in the loop is sufficient.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agotest/librbd: refactor TestMockObjectMapDiffRequest tests
Ilya Dryomov [Thu, 21 Dec 2023 15:50:31 +0000 (16:50 +0100)]
test/librbd: refactor TestMockObjectMapDiffRequest tests

In preparation for multiple similarly configured MockTestImageCtx
objects being used in a single test, centralize their creation and add
a couple of helpers for setting expectations from a callback.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agotest/librbd: improve TestMockObjectMapDiffRequest.InvalidStartSnap
Ilya Dryomov [Thu, 21 Dec 2023 17:08:21 +0000 (18:08 +0100)]
test/librbd: improve TestMockObjectMapDiffRequest.InvalidStartSnap

Use a range where only snap_id_start is invalid.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
17 months agoMerge pull request #55177 from zdover23/wip-doc-2024-01-15-radosgw-admin-get-bucket...
zdover23 [Sat, 20 Jan 2024 07:34:44 +0000 (17:34 +1000)]
Merge pull request #55177 from zdover23/wip-doc-2024-01-15-radosgw-admin-get-bucket-rate-limit

doc/radosgw: edit admin.rst "Get Bucket Rate Limit"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
17 months agodoc/radosgw: edit admin.rst "Get Bucket Rate Limit" 55177/head
Zac Dover [Sun, 14 Jan 2024 23:29:23 +0000 (09:29 +1000)]
doc/radosgw: edit admin.rst "Get Bucket Rate Limit"

Edit "Get Bucket Rate Limit" in doc/radosgw/admin.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
17 months agocrimson/osd/ops_executor: simplify prepare_clone, use set_clone_state 55223/head
Samuel Just [Sat, 20 Jan 2024 01:41:01 +0000 (17:41 -0800)]
crimson/osd/ops_executor: simplify prepare_clone, use set_clone_state

- Remove static_snap_oi name -- left over from old classic implementation,
  misleading.
- OpsExecutor::prepare_clone can only be invoked on a primary, remove
  the branch.
- Create the obs directly and pass to obc via set_clone_state, which
  sets fully_loaded.

Signed-off-by: Samuel Just <sjust@redhat.com>
17 months agocrimson/osd: invalidate contents of obc_registry on interval_change
Samuel Just [Thu, 18 Jan 2024 05:07:15 +0000 (21:07 -0800)]
crimson/osd: invalidate contents of obc_registry on interval_change

We can't guarantee that all continuations will complete before on_flush,
so we unhook and invalidate all live obcs from the registry upon
interval_change.  As long as any lingering continuations simply note the
interval change and terminate, this should be safe.

Signed-off-by: Samuel Just <sjust@redhat.com>
17 months agoMerge PR #54869 into main
Venky Shankar [Fri, 19 Jan 2024 07:49:21 +0000 (13:19 +0530)]
Merge PR #54869 into main

* refs/pull/54869/head:
mds: dump sr objects in debug output

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
17 months agoMerge PR #54915 into main
Venky Shankar [Fri, 19 Jan 2024 07:46:19 +0000 (13:16 +0530)]
Merge PR #54915 into main

* refs/pull/54915/head:
doc/cephfs/client-auth.rst: correct `This so because` to  `This is because`
doc/cephfs/client-auth.rst: correct ``fs authorize cephfs1 /dir1 client.x rw``

Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge PR #54972 into main
Venky Shankar [Fri, 19 Jan 2024 07:34:18 +0000 (13:04 +0530)]
Merge PR #54972 into main

* refs/pull/54972/head:
src: kernel mount command returning misleading error message

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
17 months agoMerge pull request #55170 from zdover23/wip-doc-2024-01-14-radosgw-admin-set-bucket...
zdover23 [Fri, 19 Jan 2024 05:54:08 +0000 (15:54 +1000)]
Merge pull request #55170 from zdover23/wip-doc-2024-01-14-radosgw-admin-set-bucket-rate-limit

doc/radosgw: edit admin.rst "set bucket rate limit"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
17 months agoinstall-deps: Force remove ceph-libboost* packages 55240/head
luo rixin [Fri, 19 Jan 2024 02:53:04 +0000 (10:53 +0800)]
install-deps: Force remove ceph-libboost* packages

As `apt-get remove ceph-libboost*` show the error and cause the make check failed.
`E: The package ceph-libboost1.79-dev needs to be reinstalled, but I can't find an archive for it.`
The ceph-libboost* packages may be not removed, so use `dpkg` to force remove ceph-libboost*.

Fixes: https://tracker.ceph.com/issues/64032
Signed-off-by: luo rixin <luorixin@huawei.com>
18 months agocommon/intrusive_lru: add clear() mechanism and state for removed items with live...
Samuel Just [Thu, 18 Jan 2024 04:26:07 +0000 (20:26 -0800)]
common/intrusive_lru: add clear() mechanism and state for removed items with live references

We need to be able to clear the lru without waiting for all outstanding
references to be released.  Items with such references enter an
invalidated state where references can still be added and removed with
deletion at use_count == 0, but can't be accessed any longer via the lru.

Signed-off-by: Samuel Just <sjust@redhat.com>
18 months agoMerge pull request #54990 from ifed01/wip-ifed-fix-expand-with-ncb
Igor Fedotov [Thu, 18 Jan 2024 18:31:00 +0000 (21:31 +0300)]
Merge pull request #54990 from ifed01/wip-ifed-fix-expand-with-ncb

os/bluestore: fix free space update after bdev-expand in NCB mode.

Reviewed-by: Pere Diaz Bou <pere-altea@hotmail.com>
18 months agoMerge pull request #55188 from cyx1231st/wip-crimson-increase-obc-lru
Samuel Just [Thu, 18 Jan 2024 16:52:15 +0000 (08:52 -0800)]
Merge pull request #55188 from cyx1231st/wip-crimson-increase-obc-lru

common/options/crimson: increase crimson_osd_obc_lru_size to 512

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
18 months agoMerge pull request #54759 from jzhu116-bloomberg/wip-63178
Casey Bodley [Thu, 18 Jan 2024 16:13:19 +0000 (16:13 +0000)]
Merge pull request #54759 from jzhu116-bloomberg/wip-63178

rgw/lc: do not add datalog/bilog for some lc actions

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
18 months agoMerge pull request #54973 from Matan-B/wip-matanb-should-whiteout-helper
Yuri Weinstein [Thu, 18 Jan 2024 15:39:25 +0000 (07:39 -0800)]
Merge pull request #54973 from Matan-B/wip-matanb-should-whiteout-helper

osd/osd/PrimaryLogPG: use should_whiteout helper in _delete_oid()

Reviewed-by: Samuel Just <sjust@redhat.com>
18 months agoMerge pull request #54780 from jianwei1216/log_to_syslog_main
Yuri Weinstein [Thu, 18 Jan 2024 15:38:17 +0000 (07:38 -0800)]
Merge pull request #54780 from jianwei1216/log_to_syslog_main

common: LogClient do not output meaningless logs by default

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
18 months agoMerge pull request #54116 from EdwardVitor/fix-warning-mon
Yuri Weinstein [Thu, 18 Jan 2024 15:37:34 +0000 (07:37 -0800)]
Merge pull request #54116 from EdwardVitor/fix-warning-mon

mon: resolve warning about inconsistent variable types

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
18 months agoMerge pull request #55161 from tobydarling/patch-2
zdover23 [Thu, 18 Jan 2024 10:41:38 +0000 (20:41 +1000)]
Merge pull request #55161 from tobydarling/patch-2

doc/rados/operations: Fix off-by-one erors in control.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
18 months agoMerge pull request #55172 from Matan-B/wip-matanb-crimson-neorados
Matan Breizman [Thu, 18 Jan 2024 10:22:49 +0000 (12:22 +0200)]
Merge pull request #55172 from Matan-B/wip-matanb-crimson-neorados

rados_api_tests: Crimson Skip unsupported

Reviewed-by: Samuel Just <sjust@redhat.com>
18 months agoMerge pull request #53281 from xxhdx1985126/wip-crimson-recover-snapset
Matan Breizman [Thu, 18 Jan 2024 10:06:49 +0000 (12:06 +0200)]
Merge pull request #53281 from xxhdx1985126/wip-crimson-recover-snapset

crimson/osd/pg: set obc's ssc when handling pull responses

18 months agoMerge pull request #55202 from zhscn/fix-ssc
Matan Breizman [Thu, 18 Jan 2024 10:05:04 +0000 (12:05 +0200)]
Merge pull request #55202 from zhscn/fix-ssc

crimson/osd: get SnapSetContext from head_obc

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
18 months agosrc: kernel mount command returning misleading error message 54972/head
neeraj pratap singh [Wed, 20 Dec 2023 08:45:35 +0000 (14:15 +0530)]
src: kernel mount command returning misleading error message

Fixes: https://tracker.ceph.com/issues/63866
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
18 months agoMerge pull request #53814 from rhcs-dashboard/fix-61720-main
Aashish Sharma [Thu, 18 Jan 2024 08:03:49 +0000 (13:33 +0530)]
Merge pull request #53814 from rhcs-dashboard/fix-61720-main

mgr/dashboard: upgrade from old 'graph' type panels to the new 'timeseries' panel

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
18 months agoMerge pull request #55181 from rkachach/fix_issue_64027
Redouane Kachach [Thu, 18 Jan 2024 05:55:38 +0000 (06:55 +0100)]
Merge pull request #55181 from rkachach/fix_issue_64027

mgr/rook: always recreate kvm default network + fix groups refresh

18 months agocommon/intrusive_lru: generalize evict() slightly to allow other target values
Samuel Just [Thu, 18 Jan 2024 04:22:10 +0000 (20:22 -0800)]
common/intrusive_lru: generalize evict() slightly to allow other target values

Signed-off-by: Samuel Just <sjust@redhat.com>
18 months agocommon/intrusive_lru: clarify invariants
Samuel Just [Tue, 16 Jan 2024 23:38:51 +0000 (23:38 +0000)]
common/intrusive_lru: clarify invariants

Signed-off-by: Samuel Just <sjust@redhat.com>
18 months agoMerge PR #53080 into main
Venky Shankar [Thu, 18 Jan 2024 05:02:24 +0000 (10:32 +0530)]
Merge PR #53080 into main

* refs/pull/53080/head:
mds/journal: make used_preallocated_ino crash more graceful

Reviewed-by: Venky Shankar <vshankar@redhat.com>
18 months agoMerge PR #53238 into main
Venky Shankar [Thu, 18 Jan 2024 05:00:00 +0000 (10:30 +0530)]
Merge PR #53238 into main

* refs/pull/53238/head:
mds: just wait the client flushes the snap and dirty buffer

Reviewed-by: Venky Shankar <vshankar@redhat.com>
18 months agoMerge PR #54511 into main
Venky Shankar [Thu, 18 Jan 2024 04:59:04 +0000 (10:29 +0530)]
Merge PR #54511 into main

* refs/pull/54511/head:
test/libcephfs: move the snapdiff to a dedicated binary

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
18 months agoMerge PR #54958 into main
Venky Shankar [Thu, 18 Jan 2024 04:57:38 +0000 (10:27 +0530)]
Merge PR #54958 into main

* refs/pull/54958/head:
mds: func name fixup, plus, and additional log message

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
18 months agoMerge PR #54682 into main
Venky Shankar [Thu, 18 Jan 2024 04:55:47 +0000 (10:25 +0530)]
Merge PR #54682 into main

* refs/pull/54682/head:
qa: drop mon_host check from test_cephfs_mirror_peer_bootstrap
pybind/mgr/mirroring: drop mon_host from peer_list

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
18 months agomgr/rook: always recreate kvm default network + fix groups refresh 55181/head
Redouane Kachach [Mon, 15 Jan 2024 12:41:06 +0000 (13:41 +0100)]
mgr/rook: always recreate kvm default network + fix groups refresh
Fixes: https://tracker.ceph.com/issues/64027
This change also includes:
- adding ~/.local/bin to path so behave binary can be found
- adding requirements.txt file for testing dependencies
- increasing timeout used to wait for tools deployment to 90s
- increasing timeout used to wait for kvm network to 20s

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
18 months agoMerge pull request #55168 from anthonyeleven/ec-table
Anthony D'Atri [Wed, 17 Jan 2024 20:04:43 +0000 (15:04 -0500)]
Merge pull request #55168 from anthonyeleven/ec-table

doc/rados/operations: add EC overhead table to erasure-code.rst

18 months agodoc/rados/operations: add EC overhead table to erasure-code.rst 55168/head
Anthony D'Atri [Sat, 13 Jan 2024 17:48:58 +0000 (12:48 -0500)]
doc/rados/operations: add EC overhead table to erasure-code.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
18 months agoMerge PR #54041 into main
Venky Shankar [Wed, 17 Jan 2024 14:29:22 +0000 (19:59 +0530)]
Merge PR #54041 into main

* refs/pull/54041/head:
qa: fix error reporting string in assert_cluster_log

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
18 months agoMerge pull request #55105 from ronen-fr/wip-rf-nonorecovery
Ronen Friedman [Wed, 17 Jan 2024 12:02:26 +0000 (14:02 +0200)]
Merge pull request #55105 from ronen-fr/wip-rf-nonorecovery

osd/scrub: allow replicas to scrub even if their OSDs have ongoing backfills

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
18 months agoqa/config/crimson_qa_overrides.yaml: Decrease lru_size for testing 55188/head
Matan Breizman [Wed, 17 Jan 2024 11:51:50 +0000 (11:51 +0000)]
qa/config/crimson_qa_overrides.yaml:  Decrease lru_size for testing

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
18 months agotest/neorados/misc: Crimson - Skip NeoRadosMisc.WriteSame 55172/head
Matan Breizman [Tue, 16 Jan 2024 11:00:37 +0000 (11:00 +0000)]
test/neorados/misc: Crimson - Skip NeoRadosMisc.WriteSame

See: https://tracker.ceph.com/issues/64040

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
18 months agotest/librados/aio_cxx: Crimson - Skip LibRadosAio.MultiReads
Matan Breizman [Mon, 15 Jan 2024 14:50:15 +0000 (14:50 +0000)]
test/librados/aio_cxx: Crimson - Skip LibRadosAio.MultiReads

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
18 months agotest/neorados/list: Seperate NeoRadosECTest from NeoRadosTest
Matan Breizman [Mon, 15 Jan 2024 14:47:00 +0000 (14:47 +0000)]
test/neorados/list: Seperate NeoRadosECTest from NeoRadosTest

otherwise SKIP_IF_CRIMSON won't work in this case

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
18 months agotest/neorados/aio_cxx: Seperate NeoRadosECTest from NeoRadosTest
Matan Breizman [Sun, 14 Jan 2024 14:05:39 +0000 (14:05 +0000)]
test/neorados/aio_cxx: Seperate NeoRadosECTest from NeoRadosTest

SKIP_IF_CRIMSON won't work here since we try to create EC pools
prior to the test being run.
Skip if the entire test instead by seperating EC tests.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
18 months agotest/neorados/io: Crimson - Skip NeoRadosIo.Limits
Matan Breizman [Wed, 17 Jan 2024 11:04:43 +0000 (11:04 +0000)]
test/neorados/io: Crimson - Skip NeoRadosIo.Limits

See: https://tracker.ceph.com/issues/64040

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
18 months agotest/neorados/cls: Crimson - Skip NeoRadosCls.RemoteReads
Matan Breizman [Mon, 15 Jan 2024 14:53:17 +0000 (14:53 +0000)]
test/neorados/cls: Crimson - Skip NeoRadosCls.RemoteReads

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
18 months agocrimson/osd: fix dangling reference bug 55202/head
Zhang Song [Wed, 17 Jan 2024 02:07:46 +0000 (10:07 +0800)]
crimson/osd: fix dangling reference bug

Cherry-picked from: 3fad54e1547c2e5d8b5c54cfcdc48f2a520c3292

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
18 months agoMerge pull request #54070 from rhcs-dashboard/add-multisite-sync-status-unit-tests
Pedro Gonzalez Gomez [Wed, 17 Jan 2024 10:03:03 +0000 (11:03 +0100)]
Merge pull request #54070 from rhcs-dashboard/add-multisite-sync-status-unit-tests

mgr/dashboard: add frontend unit tests for rgw multisite sync status card

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
18 months agoMerge pull request #54765 from Matan-B/wip-matanb-crimson-submit-error-cleanup
Matan Breizman [Wed, 17 Jan 2024 09:39:38 +0000 (11:39 +0200)]
Merge pull request #54765 from Matan-B/wip-matanb-crimson-submit-error-cleanup

crimson/osd: submit_error_log cleanup

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
18 months agoMerge pull request #55171 from Matan-B/wip-matanb-crimson-return-no-supp
Matan Breizman [Wed, 17 Jan 2024 08:51:37 +0000 (10:51 +0200)]
Merge pull request #55171 from Matan-B/wip-matanb-crimson-return-no-supp

crimson/osd/osd_operations/client_request: return OpInfo errors

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
18 months agoMerge pull request #55175 from ronen-fr/wip-rf-max-test-options
Ronen Friedman [Wed, 17 Jan 2024 08:48:33 +0000 (10:48 +0200)]
Merge pull request #55175 from ronen-fr/wip-rf-max-test-options

qa/suites/rados/thrash: modify selection of max-scrubs configuration values

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
18 months agoMerge pull request #55189 from zhscn/fix-clone-indiect-mapping
Yingxin [Wed, 17 Jan 2024 08:20:30 +0000 (16:20 +0800)]
Merge pull request #55189 from zhscn/fix-clone-indiect-mapping

crimson/os/seastore: correct the parameter passed to LBAManager::clone_mapping

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
18 months agocrimson/osd: get SnapSetContext from head_obc
Zhang Song [Wed, 17 Jan 2024 03:39:53 +0000 (11:39 +0800)]
crimson/osd: get SnapSetContext from head_obc

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
18 months agocrimson/osd/object_context: rename ObjectContext::list_hook to obc_accessing_hook
Xuehan Xu [Wed, 23 Aug 2023 01:58:47 +0000 (09:58 +0800)]
crimson/osd/object_context: rename ObjectContext::list_hook to obc_accessing_hook

There's already a "list_hook" in the base class "intrusive_lru_base"

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
18 months agoMerge pull request #55115 from ljflores/wip-tracker-63967
Laura Flores [Tue, 16 Jan 2024 19:07:10 +0000 (13:07 -0600)]
Merge pull request #55115 from ljflores/wip-tracker-63967

qa/tasks: fix syntax for deep-scrub command

18 months agoMerge pull request #54966 from yuvalif/wip-yuval-63855
Yuval Lifshitz [Tue, 16 Jan 2024 14:12:31 +0000 (16:12 +0200)]
Merge pull request #54966 from yuvalif/wip-yuval-63855

rgw/notifications: dont send notifications if op has failed

reviewed-by: cbodley

18 months agoMerge pull request #55169 from zdover23/wip-doc-2024-01-14-radosgw-admin-enable-disab...
zdover23 [Tue, 16 Jan 2024 13:51:29 +0000 (23:51 +1000)]
Merge pull request #55169 from zdover23/wip-doc-2024-01-14-radosgw-admin-enable-disable-user-rate-limit

docs/radosgw: edit admin.rst "enable/disable user rate limit"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
18 months agoMerge pull request #55148 from cbodley/wip-63994
Casey Bodley [Tue, 16 Jan 2024 13:22:30 +0000 (13:22 +0000)]
Merge pull request #55148 from cbodley/wip-63994

rgw: fix use of creds in forward_iam_request()

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
18 months agoMerge pull request #54803 from rhcs-dashboard/increase-plottable-graphs-dashboard
Pedro Gonzalez Gomez [Tue, 16 Jan 2024 13:15:48 +0000 (14:15 +0100)]
Merge pull request #54803 from rhcs-dashboard/increase-plottable-graphs-dashboard

mgr/dashboard: increase the number of plottable graphs in charts

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>