]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
12 months agoqa/tasks/qemu: Fix OS version comparison 58171/head
Zack Cerza [Fri, 14 Jun 2024 19:37:16 +0000 (13:37 -0600)]
qa/tasks/qemu: Fix OS version comparison

See: https://sentry.ceph.com/share/issue/21ed88d705854238bdafbf6711e795ee/
They're strings, not floats.
This surfaced as a result of https://github.com/ceph/teuthology/pull/1953

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit 260a45a03148ba4d2bbfcac9485bcf9a8743cade)

12 months agoMerge pull request #58130 from zdover23/wip-doc-2024-06-19-backport-58123-to-squid
Zac Dover [Thu, 20 Jun 2024 13:48:42 +0000 (23:48 +1000)]
Merge pull request #58130 from zdover23/wip-doc-2024-06-19-backport-58123-to-squid

squid: doc: clarify availability vs integrity

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
12 months agoMerge pull request #57571 from idryomov/wip-65813-squid
Ilya Dryomov [Thu, 20 Jun 2024 10:49:27 +0000 (12:49 +0200)]
Merge pull request #57571 from idryomov/wip-65813-squid

squid: librbd: don't crash on a zero-length read if buffer is NULL

12 months agoMerge pull request #57602 from ajarr/wip-66117-squid
Ilya Dryomov [Thu, 20 Jun 2024 10:48:34 +0000 (12:48 +0200)]
Merge pull request #57602 from ajarr/wip-66117-squid

squid: librbd: create rbd_trash object during pool initialization and namespace creation

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
12 months agoMerge pull request #57766 from idryomov/wip-66232-squid
Ilya Dryomov [Thu, 20 Jun 2024 10:47:10 +0000 (12:47 +0200)]
Merge pull request #57766 from idryomov/wip-66232-squid

squid: qa/suites/rbd: override extra_system_packages directly on install task

Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agoMerge pull request #58044 from idryomov/wip-64662-squid
Ilya Dryomov [Thu, 20 Jun 2024 10:44:10 +0000 (12:44 +0200)]
Merge pull request #58044 from idryomov/wip-64662-squid

squid: librbd: allow cloning from non-user snapshots

Reviewed-by: Ramana Raja <rraja@redhat.com>
12 months agoMerge pull request #56883 from guits/wip-65480-squid
Guillaume Abrioux [Thu, 20 Jun 2024 07:26:12 +0000 (09:26 +0200)]
Merge pull request #56883 from guits/wip-65480-squid

squid: ceph-volume: osd objectstore refactor

12 months agoMerge pull request #56931 from adk3798/wip-65515-squid
Avan [Thu, 20 Jun 2024 06:15:00 +0000 (11:45 +0530)]
Merge pull request #56931 from adk3798/wip-65515-squid

squid: mgr/dashboard: fix duplicate grafana panels when on mgr failover

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
13 months agoMerge pull request #58122 from ljflores/wip-66554-squid
Laura Flores [Wed, 19 Jun 2024 16:48:23 +0000 (11:48 -0500)]
Merge pull request #58122 from ljflores/wip-66554-squid

squid: qa/rados: don't pass --max_attr_len to ceph_test_rados unconditionally

13 months agoMerge pull request #58127 from petrutlucian94/wip-66547-squid
Ilya Dryomov [Wed, 19 Jun 2024 12:03:34 +0000 (14:03 +0200)]
Merge pull request #58127 from petrutlucian94/wip-66547-squid

squid: ceph_test_rados_api_misc: adjust LibRadosMiscConnectFailure.ConnectTimeout timeout

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
13 months agoMerge pull request #58116 from zdover23/wip-doc-2024-06-18-backport-57352-to-squid
Zac Dover [Wed, 19 Jun 2024 10:23:55 +0000 (20:23 +1000)]
Merge pull request #58116 from zdover23/wip-doc-2024-06-18-backport-57352-to-squid

squid: doc/dev/crimson: CPU cores allocation

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
13 months agoMerge pull request #58105 from zdover23/wip-doc-2024-06-18-backport-58901-to-squid
Zac Dover [Wed, 19 Jun 2024 10:21:52 +0000 (20:21 +1000)]
Merge pull request #58105 from zdover23/wip-doc-2024-06-18-backport-58901-to-squid

squid: doc/dev: link to ceph.io leads list

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
13 months agoMerge pull request #58067 from Matan-B/wip-57788-squid
Matan Breizman [Wed, 19 Jun 2024 10:15:04 +0000 (13:15 +0300)]
Merge pull request #58067 from Matan-B/wip-57788-squid

squid: crimson/os/seastore: implement disk and writer level stats reporting

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58066 from Matan-B/wip-57654-squid
Matan Breizman [Wed, 19 Jun 2024 10:14:44 +0000 (13:14 +0300)]
Merge pull request #58066 from Matan-B/wip-57654-squid

squid: crimson/os/seastore: fix compile error in release build

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58032 from Matan-B/wip-57279-squid
Matan Breizman [Wed, 19 Jun 2024 10:14:22 +0000 (13:14 +0300)]
Merge pull request #58032 from Matan-B/wip-57279-squid

squid: crimson/osd/pg: trigger wait_for_active_blocker on replica osds when the activate event is committed

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agodoc: clarify availability vs integrity 58130/head
Gregory O'Neill [Tue, 18 Jun 2024 21:17:00 +0000 (17:17 -0400)]
doc: clarify availability vs integrity

(cherry picked from commit 3888cf77075eceb079b1d5a8ce19092d12fd82bd)

13 months agoceph_test_rados_api_misc: adjust LibRadosMiscConnectFailure.ConnectTimeout timeout 58127/head
Lucian Petrut [Mon, 17 Jun 2024 18:37:11 +0000 (18:37 +0000)]
ceph_test_rados_api_misc: adjust LibRadosMiscConnectFailure.ConnectTimeout timeout

The test sets a 2 second connection timeout and asserts that the
attempt takes less than 4 seconds.

Under load, the asserted timeout can be exceeded. In particular,
this affects the Windows CI job, which uses a VM with a reduced
amount of resources.

    [ RUN      ] LibRadosMiscConnectFailure.ConnectTimeout
    /home/ubuntu/ceph/src/test/librados/misc.cc:87: Failure
    Expected: (dur) < (utime_t(4, 0)), actual: 5.987712 vs 4.000000

We'll adjust the timeouts, using a 5s configured timeout, asserting
that the connection attempt doesn't take more than 15s.

Fixes: https://tracker.ceph.com/issues/66534
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 3a2b4a58898477ae9f62b506bd5be56ff9abacf0)

13 months agoqa/rados: don't pass --max_attr_len to ceph_test_rados unconditionally 58122/head
Radosław Zarzyński [Mon, 3 Jun 2024 22:00:38 +0000 (00:00 +0200)]
qa/rados: don't pass --max_attr_len to ceph_test_rados unconditionally

Fixes: https://tracker.ceph.com/issues/66321
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit 8a6d56e62fc03062e60828a3bc9f37ed7232b3e7)

13 months agodoc/dev/crimson: CPU cores allocation 58116/head
Matan Breizman [Wed, 8 May 2024 08:23:30 +0000 (11:23 +0300)]
doc/dev/crimson: CPU cores allocation

CPU cores allocation and
add crimson-smp vstart flag

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 35d257bfaa5ed7cf91c8a8b523ed4494fdd27667)

13 months agoMerge pull request #58093 from Matan-B/wip-55831-squid
Matan Breizman [Tue, 18 Jun 2024 10:53:44 +0000 (13:53 +0300)]
Merge pull request #58093 from Matan-B/wip-55831-squid

squid: crimson/os/seastore: lba pointer consolidation

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58065 from Matan-B/wip-57383-squid
Matan Breizman [Tue, 18 Jun 2024 10:52:04 +0000 (13:52 +0300)]
Merge pull request #58065 from Matan-B/wip-57383-squid

squid: crimson: fix user_version handling

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58030 from Matan-B/wip-57561-squid
Matan Breizman [Tue, 18 Jun 2024 10:49:38 +0000 (13:49 +0300)]
Merge pull request #58030 from Matan-B/wip-57561-squid

squid: crimson/osd/ops_executer: LIST_SNAPS only on CEPH_SNAPDIR

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #57862 from rhcs-dashboard/wip-66345-squid
Pedro Gonzalez Gomez [Tue, 18 Jun 2024 10:16:28 +0000 (12:16 +0200)]
Merge pull request #57862 from rhcs-dashboard/wip-66345-squid

squid: mgr/dashboard: add smb service management support

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
13 months agoMerge pull request #57861 from rhcs-dashboard/wip-66037-squid
Pedro Gonzalez Gomez [Tue, 18 Jun 2024 10:16:05 +0000 (12:16 +0200)]
Merge pull request #57861 from rhcs-dashboard/wip-66037-squid

squid: mgr/dashboard: add upgrade notification

13 months agocrimson/osd: remove osd_op_params_t::user_at_version 58065/head
Samuel Just [Thu, 9 May 2024 03:06:58 +0000 (20:06 -0700)]
crimson/osd: remove osd_op_params_t::user_at_version

osd_op_params_t::user_at_version was populated from
osd_op_params_t::at_version before the call to prepare_transaction,
which incremented osd_op_params_t::at_version.version.  As a result,
the value stored in object_info_t::user_version ended up one version
behind object_info_t::version.  The log entry, on the other hand,
ended up with the correct version as OpsExecutor::prepare_transaction
populates it directly from at_version.  As a result, the primary could
return different versions to the client depending on whether the IO was
already in the log.

This commit eliminates osd_op_params_t::user_at_version and updates
PGBackend::mutate_object to behave like prepare_transaction.  Because
the prior commit removes the prepare_transaction increment, this isn't
strictly necessary, but it is simpler.

Fixes: https://tracker.ceph.com/issues/65857
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 85fdadab2adcfbec83a82c6f1bae1f4f31a96f54)

13 months agocrimson/osd/ops_executor: only increment osd_op_params_t::at_version for clone
Samuel Just [Thu, 9 May 2024 03:39:18 +0000 (20:39 -0700)]
crimson/osd/ops_executor: only increment osd_op_params_t::at_version for clone

Previously, we incremented prior to usage in both prepare_transaction
and execute_clone.  Because at_version is initialized from
PG::get_next_version(), this results in log entries skipping
values.  Instead, only increment after populating clone
object_info.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 7e0dde721d5203623b06351bb4b261b3c9fe2c33)

13 months agocrimson/osd/ops_executor: use clone oi values in log entry
Samuel Just [Thu, 9 May 2024 03:01:49 +0000 (20:01 -0700)]
crimson/osd/ops_executor: use clone oi values in log entry

The fields from the clone_obc oi are what we're actually
trying to fill in here -- we might as well use them
directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 207e5a6e9ba54deb2fb18d5a5ad432d42072453d)

13 months agocrimson/osd/ops_executor: update osd_op_params_t::at_version in execute_clone
Samuel Just [Thu, 9 May 2024 02:45:45 +0000 (02:45 +0000)]
crimson/osd/ops_executor: update osd_op_params_t::at_version in execute_clone

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1ecf3d64853c2fa92bf467692b98a58b4670b212)

13 months agocrimson/osd/pg: remove slightly confusing assert from PG::submit_transaction
Samuel Just [Thu, 9 May 2024 02:31:35 +0000 (19:31 -0700)]
crimson/osd/pg: remove slightly confusing assert from PG::submit_transaction

This assert should always hold, so it's not wrong.  However,
osd_op_p.at_version is chosen during transaction construction time
using the current epoch via PG::get_next_version.  Asserting it here
is more confusing than helpful.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 7a04f6230d9376d155ac92a6c3bf1ab5eeca644f)

13 months agocrimson/osd/pg_backend: remove uninformative #if 0 block
Samuel Just [Thu, 9 May 2024 02:27:59 +0000 (19:27 -0700)]
crimson/osd/pg_backend: remove uninformative #if 0 block

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 2bd7c38381bc22bc73130e6d138e001096e66faf)

13 months agoMerge pull request #58064 from Matan-B/wip-57056-squid
Matan Breizman [Tue, 18 Jun 2024 07:13:12 +0000 (10:13 +0300)]
Merge pull request #58064 from Matan-B/wip-57056-squid

squid:  crimson/osd: make osd_op_params::at_version coherent with last log entry

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58063 from Matan-B/wip-57245-squid
Matan Breizman [Tue, 18 Jun 2024 07:11:41 +0000 (10:11 +0300)]
Merge pull request #58063 from Matan-B/wip-57245-squid

squid: qa/config/crimson_qa_overrides: adjust mgr_stats_period

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58024 from Matan-B/wip-57251-squid
Matan Breizman [Tue, 18 Jun 2024 07:10:38 +0000 (10:10 +0300)]
Merge pull request #58024 from Matan-B/wip-57251-squid

squid: crimson/osd/pg_backend: only create new onode when it doesn't exist

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #57471 from guits/wip-66021-squid
Guillaume Abrioux [Tue, 18 Jun 2024 06:40:27 +0000 (08:40 +0200)]
Merge pull request #57471 from guits/wip-66021-squid

squid: ceph-volume: use os.makedirs for mkdir_p

13 months agoMerge pull request #57473 from guits/wip-66023-squid
Guillaume Abrioux [Tue, 18 Jun 2024 06:39:55 +0000 (08:39 +0200)]
Merge pull request #57473 from guits/wip-66023-squid

squid: cephadm: change loki/promtail default image tags

13 months agoMerge pull request #58004 from zdover23/wip-doc-2024-06-13-backport-57650-to-squid
Guillaume Abrioux [Tue, 18 Jun 2024 06:38:17 +0000 (08:38 +0200)]
Merge pull request #58004 from zdover23/wip-doc-2024-06-13-backport-57650-to-squid

squid: ceph-volume: use importlib from stdlib on Python 3.8 and up

13 months agodoc/dev: link to ceph.io leads list 58105/head
Zac Dover [Mon, 17 Jun 2024 14:39:10 +0000 (00:39 +1000)]
doc/dev: link to ceph.io leads list

Remove the docs.ceph.com list of team leads, and link to the list of
leads on ceph.io.

Fixes: https://tracker.ceph.com/issues/66392
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 5134aeb78bb1d9d956442333ad372e1f397ff7b3)

13 months agocrimson/os/seastore/btree: drop FixedKVNode::child_pos_for_key() 58093/head
Xuehan Xu [Thu, 16 May 2024 01:21:45 +0000 (09:21 +0800)]
crimson/os/seastore/btree: drop FixedKVNode::child_pos_for_key()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit bfc2abae90dc0cd2a3a0ef7b62006435b305f51d)

13 months agocrimson/os/seastore/btree: better UT checks
Xuehan Xu [Wed, 15 May 2024 08:15:01 +0000 (16:15 +0800)]
crimson/os/seastore/btree: better UT checks

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit dca239f84ca1cd0607be337b452673418036d0bc)

13 months agocrimson/os/seastore/btree: correct lba pointer comments
Xuehan Xu [Wed, 15 May 2024 07:46:01 +0000 (15:46 +0800)]
crimson/os/seastore/btree: correct lba pointer comments

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 9bb054b30f66d1b03f0ea6937b08806c4b64f5b6)

13 months agocrimson/os/seastore/cached_extent: EXIST_MUTATION_PENDING also
Xuehan Xu [Thu, 29 Feb 2024 06:35:31 +0000 (14:35 +0800)]
crimson/os/seastore/cached_extent: EXIST_MUTATION_PENDING also
considered as mutatione_pending

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 211ca3792e80c4319a23596a5f439bab886c200f)

13 months agocrimson/os/seastore/cached_extent: adjust
Xuehan Xu [Thu, 29 Feb 2024 06:16:37 +0000 (14:16 +0800)]
crimson/os/seastore/cached_extent: adjust
CachedExtent::on_replace_prior()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit d4c78f1524bebfa3f370f5a4a019800dfeaf04f1)

13 months agocrimson/os/seastore/btree: misc consolidations
Xuehan Xu [Thu, 29 Feb 2024 06:06:35 +0000 (14:06 +0800)]
crimson/os/seastore/btree: misc consolidations

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 72d721ad72d15296b069752985f12e173f9690da)

13 months agocrimson/os/seastore/cache: drop unnecessary friend declarations
Xuehan Xu [Wed, 28 Feb 2024 10:26:24 +0000 (18:26 +0800)]
crimson/os/seastore/cache: drop unnecessary friend declarations

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit fecf501c41471e132427f2a40f26298a38ba4003)

13 months agocrimson/os/seastore/btree: add CachedExtent::on_rewrite() and
Xuehan Xu [Wed, 28 Feb 2024 09:59:41 +0000 (17:59 +0800)]
crimson/os/seastore/btree: add CachedExtent::on_rewrite() and
CachedExtent::is_rewrite()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 1b8a781f665df06b056511debec46deac604a79c)

13 months agocrimson/os/seastore/lba_manager: extents pass to LBALeafNode::update()
Xuehan Xu [Wed, 28 Feb 2024 08:00:23 +0000 (16:00 +0800)]
crimson/os/seastore/lba_manager: extents pass to LBALeafNode::update()
must either has a null parent tracker or point to the current leaf node

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit 4b155f3ee5ab4fbd212e117d2cd40dd78d846d3a)

13 months agoMerge pull request #58034 from Matan-B/wip-57593-squid
Matan Breizman [Mon, 17 Jun 2024 15:25:09 +0000 (18:25 +0300)]
Merge pull request #58034 from Matan-B/wip-57593-squid

squid: Crimson: Support basic deployments

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58027 from Matan-B/wip-57455-squid
Matan Breizman [Mon, 17 Jun 2024 15:24:18 +0000 (18:24 +0300)]
Merge pull request #58027 from Matan-B/wip-57455-squid

squid: crimson/osd/replicated_backend: no need to set_rollback_to for repops

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58022 from Matan-B/wip-57069-squid
Matan Breizman [Mon, 17 Jun 2024 15:19:33 +0000 (18:19 +0300)]
Merge pull request #58022 from Matan-B/wip-57069-squid

squid: crimson/common/operation: detach blockers from blocking events when they are destroyed

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58015 from Matan-B/wip-57474-squid
Matan Breizman [Mon, 17 Jun 2024 15:18:34 +0000 (18:18 +0300)]
Merge pull request #58015 from Matan-B/wip-57474-squid

squid: crimson/os/seastore/transaction_manager: correct the offset of the data copied from the original extents

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58013 from Matan-B/wip-57432-squid
Matan Breizman [Mon, 17 Jun 2024 15:18:11 +0000 (18:18 +0300)]
Merge pull request #58013 from Matan-B/wip-57432-squid

squid: crimson/os/seastore/object_data_handler: cleanup read()

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58011 from Matan-B/wip-57368-squid
Matan Breizman [Mon, 17 Jun 2024 15:17:24 +0000 (18:17 +0300)]
Merge pull request #58011 from Matan-B/wip-57368-squid

squid: crimson/os/seastore: add is_data_stable() to allow delta-overwrite on EXIST_CLEAN

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58029 from Matan-B/wip-57692-squid
Matan Breizman [Mon, 17 Jun 2024 13:30:13 +0000 (16:30 +0300)]
Merge pull request #58029 from Matan-B/wip-57692-squid

squid: crimson/osd/pg_shard_manager: remove the unnecessary "std::move" call

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58021 from Matan-B/wip-57725-squid
Matan Breizman [Mon, 17 Jun 2024 13:29:30 +0000 (16:29 +0300)]
Merge pull request #58021 from Matan-B/wip-57725-squid

squid: crimson/osd/object_context_loader: Fix obc cache existence usage

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58016 from Matan-B/wip-56114-squid
Matan Breizman [Mon, 17 Jun 2024 13:28:58 +0000 (16:28 +0300)]
Merge pull request #58016 from Matan-B/wip-56114-squid

squid: src/test/TestRados: add max-attr-len to control the max length of attributes sent to OSDs

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58009 from Matan-B/wip-55488-squid
Matan Breizman [Mon, 17 Jun 2024 13:28:18 +0000 (16:28 +0300)]
Merge pull request #58009 from Matan-B/wip-55488-squid

squid: crimson/osd/osd_operations/client_request: make loading-obc concurrent

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58075 from idryomov/wip-cephadm-flake8-fix-squid
Pedro Gonzalez Gomez [Mon, 17 Jun 2024 07:19:18 +0000 (09:19 +0200)]
Merge pull request #58075 from idryomov/wip-cephadm-flake8-fix-squid

squid: mgr/cephadm: fix flake8 test failures

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
13 months agoMerge pull request #57840 from rishabh-d-dave/wip-66330-squid
Jos Collin [Mon, 17 Jun 2024 05:08:15 +0000 (10:38 +0530)]
Merge pull request #57840 from rishabh-d-dave/wip-66330-squid

squid: mon,cephfs: require confirmation flag to bring down unhealthy MDS

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
13 months agoMerge pull request #57682 from batrick/wip-66196-squid
Jos Collin [Mon, 17 Jun 2024 04:54:25 +0000 (10:24 +0530)]
Merge pull request #57682 from batrick/wip-66196-squid

squid: mds: use regular dispatch for processing beacons

Reviewed-by: Jos Collin <jcollin@redhat.com>
13 months agomgr/cephadm: fix flake8 test failures 58075/head
Nizamudeen A [Sun, 16 Jun 2024 09:11:04 +0000 (14:41 +0530)]
mgr/cephadm: fix flake8 test failures

after a recent update on flake8, some tests in cephadm started failing
```
flake8: commands[0]> flake8 --config=tox.ini alerts balancer cephadm cli_api crash devicehealth diskprediction_local hello iostat localpool nfs orchestrator prometheus rbd_support rgw selftest smb
cephadm/configchecks.py:94:6: E204 whitespace after decorator '@'
cephadm/module.py:3017:10: E204 whitespace after decorator '@'
cephadm/module.py:3344:144: E226 missing whitespace around arithmetic operator
cephadm/tests/test_cephadm.py:982:6: E204 whitespace after decorator '@'
3     E204 whitespace after decorator '@'
1     E226 missing whitespace around arithmetic operator
```

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a9563d91c2ac405ae9101bd4e63fa9b749d54432)

13 months agocrimson/os/seastore: implement disk and writer level stats reporting 58067/head
Yingxin Cheng [Fri, 24 May 2024 06:20:49 +0000 (14:20 +0800)]
crimson/os/seastore: implement disk and writer level stats reporting

At SeaStore level, to understand the device balances:
* IOPS
* per-writer depth
* bandwidth
* IO size

Picking a particular SeaStore shard, to understand:
* Per device tier write status
* Per writer status in a device tier
* Per transaction type status
* How each type of transactions write through the writers

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 8f67346015f0917cf56f0534fc316af8560921de)

13 months agocrimson/os/seastore/../record_submitter: account transaction related stats
Yingxin Cheng [Fri, 24 May 2024 06:54:09 +0000 (14:54 +0800)]
crimson/os/seastore/../record_submitter: account transaction related stats

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit b68a0aec64ba4dccc6b4ff4533f4d77bbb523ec4)

13 months agocrimson/os/seastore/extent_placement_manager: pass trans src to the submitter
Yingxin Cheng [Fri, 17 May 2024 09:26:45 +0000 (17:26 +0800)]
crimson/os/seastore/extent_placement_manager: pass trans src to the submitter

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 61e31e32bad1540c0a5b84fefec10321b9ee4b90)

13 months agocrimson/os/seastore/../record_submitter: drop a duplicated metric
Yingxin Cheng [Fri, 17 May 2024 09:05:57 +0000 (17:05 +0800)]
crimson/os/seastore/../record_submitter: drop a duplicated metric

The original "record group num" is identical to "io num".

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit f1e7ca6f9c18d05726ff6beed871152849d6c9a4)

13 months agocrimson/os/seastore/../record_submitter: account submission from record_group_t
Yingxin Cheng [Fri, 24 May 2024 06:52:55 +0000 (14:52 +0800)]
crimson/os/seastore/../record_submitter: account submission from record_group_t

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit c245900cf3f7784d9622799c2dbd6189e5d52258)

13 months agocrimson/os/seastore: move stats to seastore_types.h
Yingxin Cheng [Fri, 24 May 2024 06:42:28 +0000 (14:42 +0800)]
crimson/os/seastore: move stats to seastore_types.h

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 1d2b33e619308acecc6745d73fb742d0881d0ce8)

13 months agocrimson/os/seastore/extent_placement_manager: minor cleanups around get_writer()
Yingxin Cheng [Fri, 24 May 2024 05:48:29 +0000 (13:48 +0800)]
crimson/os/seastore/extent_placement_manager: minor cleanups around get_writer()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit db5e077f31ce8ce8cf6f9f200288bf5a12223986)

13 months agocrimson/os/seastore/seastore_types: replace the deprecated journal_type_t
Yingxin Cheng [Fri, 24 May 2024 03:39:04 +0000 (11:39 +0800)]
crimson/os/seastore/seastore_types: replace the deprecated journal_type_t

Replace by backend_type_t.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 7800ad515af6774821d4cbe3ee69e105b3a8bcf6)

13 months agocrimson/osd: move stats_timer after store is mounted
Yingxin Cheng [Tue, 21 May 2024 06:00:28 +0000 (14:00 +0800)]
crimson/osd: move stats_timer after store is mounted

To report stats after the store is initialized.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit f183adb469ceb647fd2c2dad25d10adbc891b711)

13 months agocrimson/osd: gate stats reporting
Yingxin Cheng [Fri, 17 May 2024 09:03:14 +0000 (17:03 +0800)]
crimson/osd: gate stats reporting

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit e3d60f0ccf614d77fde46fd4c7b9a184d52f1618)

13 months agocrimson/os/seastore/seastore_types: use utility to compare incompatible types
Yingxin Cheng [Tue, 21 May 2024 05:56:12 +0000 (13:56 +0800)]
crimson/os/seastore/seastore_types: use utility to compare incompatible types

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 3af14e29654ebd5a671986073b37504e441ab4a1)

13 months agocrimson/os/seastore: fix compile error in release build 58066/head
Zhang Song [Thu, 23 May 2024 09:12:06 +0000 (17:12 +0800)]
crimson/os/seastore: fix compile error in release build

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
(cherry picked from commit 36c7eb73fedfe5814ce4a2424a391c84339eaddb)

13 months agocrimson/osd: make osd_op_params::at_version coherent with last log entry 58064/head
Radoslaw Zarzynski [Tue, 23 Apr 2024 14:10:22 +0000 (14:10 +0000)]
crimson/osd: make osd_op_params::at_version coherent with last log entry

Before this commit we were doing something like:

1. initialize `at_version` with PG::projected_last_update`
   **incremented by one**.
2. produce a log entry at such version.
3. increment `at_version` for the sake of a further production
   that may never come.

The problem is `osd_op_params::at_version` is higher by one
than the last log entry which hurts at later stages of
`osd_op_params` processing (I was hit in the shared EC code
by the assertion in `PG::op_applied`).

This patch changes the algorithm to:

A. initialize `at_version` with PG::projected_last_update`
   **incremented by one**.
B. increment `at_version` for the sake of the very next production.
C. produce a log entry at this version.

Co-authored-by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 5e787fda2e6186e3963e29225b4fa0ac145d2c45)

13 months agocrimson/osd: s/fill_op_params_bump_pg_version/fill_op_params/ in OpsExecuter
Radoslaw Zarzynski [Tue, 23 Apr 2024 14:04:02 +0000 (14:04 +0000)]
crimson/osd: s/fill_op_params_bump_pg_version/fill_op_params/ in OpsExecuter

Since https://github.com/ceph/ceph/pull/53054 it doesn't
bump `PG::projected_last_update` anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit eca3d26124cdbe5eff0d2e356417c977a37e115a)

13 months agocrimson/osd: on the cloning path bump obj version closer to its usage
Radoslaw Zarzynski [Tue, 23 Apr 2024 13:54:23 +0000 (13:54 +0000)]
crimson/osd: on the cloning path bump obj version closer to its usage

My understanding at the time of writing is this change is a pure
refactoring.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 99fdc0cc478c169c30627d4436bb7aebd3335c8d)

13 months agoMerge pull request #58033 from Matan-B/wip-57313-squid
Matan Breizman [Sun, 16 Jun 2024 10:12:54 +0000 (13:12 +0300)]
Merge pull request #58033 from Matan-B/wip-57313-squid

squid: crimson/osd/ops_executor: calculation of clone_overlap shouldn't consider snap contexts

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58031 from Matan-B/wip-57908-squid
Matan Breizman [Sun, 16 Jun 2024 10:11:40 +0000 (13:11 +0300)]
Merge pull request #58031 from Matan-B/wip-57908-squid

squid: crimson/osd/osd_operations: correct connection pipelines for osd operations

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58028 from Matan-B/wip-57416-squid
Matan Breizman [Sun, 16 Jun 2024 10:09:57 +0000 (13:09 +0300)]
Merge pull request #58028 from Matan-B/wip-57416-squid

squid: crimson/osd/pg: hold PGs' references to the last minute of snap trim events executions

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58026 from Matan-B/wip-57386-squid
Matan Breizman [Sun, 16 Jun 2024 10:08:59 +0000 (13:08 +0300)]
Merge pull request #58026 from Matan-B/wip-57386-squid

squid: crimson/osd/recovery_backend: change recovery waiters' promises into optional ones

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58025 from Matan-B/wip-57276-squid
Matan Breizman [Sun, 16 Jun 2024 10:08:39 +0000 (13:08 +0300)]
Merge pull request #58025 from Matan-B/wip-57276-squid

squid: crimson/osd/pg_backend: DONOT modify OSDOp::indata when handling CEPH_OSD_OP_CHECKSUM

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58023 from Matan-B/wip-57204-squid
Matan Breizman [Sun, 16 Jun 2024 10:07:43 +0000 (13:07 +0300)]
Merge pull request #58023 from Matan-B/wip-57204-squid

squid: crimson/osd: clamp reads to object size and bring full read trimming

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58020 from Matan-B/wip-57711-squid
Matan Breizman [Sun, 16 Jun 2024 10:05:20 +0000 (13:05 +0300)]
Merge pull request #58020 from Matan-B/wip-57711-squid

squid: crimson/os/seastore/async_cleaner: fix incorrect get_num_rolls()

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58019 from Matan-B/wip-57694-squid
Matan Breizman [Sun, 16 Jun 2024 10:04:56 +0000 (13:04 +0300)]
Merge pull request #58019 from Matan-B/wip-57694-squid

squid: crimson/os/seastore/rbm/avlallocator: return enough regions when request size is greater than max_alloc_size

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58018 from Matan-B/wip-57656-squid
Matan Breizman [Sun, 16 Jun 2024 10:04:28 +0000 (13:04 +0300)]
Merge pull request #58018 from Matan-B/wip-57656-squid

squid: test/crimson/seastore/transaction_manager_test_state: fix compilation…

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58017 from Matan-B/wip-57262-squid
Matan Breizman [Sun, 16 Jun 2024 10:04:00 +0000 (13:04 +0300)]
Merge pull request #58017 from Matan-B/wip-57262-squid

squid: crimson/os/seastore/lba_manager: don't increase intermediate mappings' refcount if LBAManager::clone_mapping() is called to remap mappings

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58014 from Matan-B/wip-55735-squid
Matan Breizman [Sun, 16 Jun 2024 09:56:54 +0000 (12:56 +0300)]
Merge pull request #58014 from Matan-B/wip-55735-squid

squid: crimson/common/errorator: disallow void-returning error handlers

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58012 from Matan-B/wip-57476-squid
Matan Breizman [Sun, 16 Jun 2024 09:54:55 +0000 (12:54 +0300)]
Merge pull request #58012 from Matan-B/wip-57476-squid

squid: crimson/os/seastore/transaction_manager: drop unused code

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58010 from Matan-B/wip-57434-squid
Matan Breizman [Sun, 16 Jun 2024 09:53:53 +0000 (12:53 +0300)]
Merge pull request #58010 from Matan-B/wip-57434-squid

squid: crimson/os/seastore/record_scanner: replace [=] capturing

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoMerge pull request #58008 from Matan-B/wip-57102-squid
Matan Breizman [Sun, 16 Jun 2024 09:52:34 +0000 (12:52 +0300)]
Merge pull request #58008 from Matan-B/wip-57102-squid

squid: crimson/osd/osdop_params:Unify OpsExecuter::user_modify and osd_op_params_t::user_modify

Reviewed-by: Samuel Just <sjust@redhat.com>
13 months agoqa/config/crimson_qa_overrides: adjust mgr_stats_period 58063/head
Matan Breizman [Thu, 2 May 2024 13:54:18 +0000 (13:54 +0000)]
qa/config/crimson_qa_overrides: adjust mgr_stats_period

Some test jobs fail with `local_shared_foreign_ptr: Assertion `ptr && *ptr' failed`
It seems that we attempt to use a connection which is not yet ready to use
after setting up the daemons on boot.
Adjust the mgr_stats_period to allow more time for the daemons to set up.
See: https://tracker.ceph.com/issues/62162#note-10

Note: This is not a fix but more of a temporary solution to avoid noise
      in the testing suite (Tracker stays open).

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 752b4c6110590cdd85b04e6a01e6f1affcdee97c)

13 months agoMerge pull request #58059 from zdover23/wip-doc-2024-06-16-backport-58057-to-squid
Anthony D'Atri [Sun, 16 Jun 2024 06:16:22 +0000 (02:16 -0400)]
Merge pull request #58059 from zdover23/wip-doc-2024-06-16-backport-58057-to-squid

squid: doc/rados: explain replaceable parts of command

13 months agodoc/rados: explain replaceable parts of command 58059/head
Zac Dover [Sat, 15 Jun 2024 11:55:18 +0000 (21:55 +1000)]
doc/rados: explain replaceable parts of command

Add an explanation that directs the reader to replace the "X" part of
the command "ceph tell mon.X mon_status" with the value specific to the
reader's Ceph cluster (which is (probably) not "X").

In the future, such replaceable strings in commands may be bounded by
angle brackets ("<" and ">").

This improvement to the documentation was suggested on the [ceph-users]
email list by Joel Davidow. This email, an absolute model of user
engagement with an upstream project, can be reviewed here:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/KF67F5TXFSSTPXV7EKL6JKLA5KZQDLDQ/

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d071ad2575c86f300a9ba39df3c4949e5dc9c47d)

13 months agoceph-volume: fix regression 58004/head
Guillaume Abrioux [Mon, 3 Jun 2024 12:00:10 +0000 (14:00 +0200)]
ceph-volume: fix regression

This fixes a regression introduced by: 24f8e5c61b19deab7397b0237f8376c6c03a5dcb

`iter_entry_points` from `pkg_resources` takes one argument whereas
`entry_points` from `importlib.metadata` does not.

The call to `entry_points(group=group)` makes ceph-volume fail.

Fixes: https://tracker.ceph.com/issues/66328
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 6e3b0b93055538cad234018ef8700bfacec03076)

13 months agoMerge pull request #57998 from ljflores/wip-tracker-66460
Laura Flores [Fri, 14 Jun 2024 17:52:42 +0000 (12:52 -0500)]
Merge pull request #57998 from ljflores/wip-tracker-66460

squid: qa/suites/rados/thrash-old-clients: update supported releases and distro

13 months agoMerge pull request #58049 from zdover23/wip-doc-2024-06-14-backport-58007-to-squid
Anthony D'Atri [Fri, 14 Jun 2024 15:14:23 +0000 (11:14 -0400)]
Merge pull request #58049 from zdover23/wip-doc-2024-06-14-backport-58007-to-squid

squid: doc/rados: add pg-states and pg-concepts to tree

13 months agoMerge pull request #57525 from batrick/wip-66044-squid
Jos Collin [Fri, 14 Jun 2024 11:33:46 +0000 (17:03 +0530)]
Merge pull request #57525 from batrick/wip-66044-squid

squid: qa: unmount clients before damaging the fs

Reviewed-by: Jos Collin <jcollin@redhat.com>
13 months agoMerge pull request #57762 from joscollin/wip-66270-squid
Jos Collin [Fri, 14 Jun 2024 11:20:23 +0000 (16:50 +0530)]
Merge pull request #57762 from joscollin/wip-66270-squid

squid: pybind/mgr/mirroring: Fix KeyError: 'directory_count' in daemon status

Reviewed-by: Venky Shankar <vshankar@redhat.com>
13 months agoMerge pull request #57760 from joscollin/wip-66277-squid
Jos Collin [Fri, 14 Jun 2024 11:19:09 +0000 (16:49 +0530)]
Merge pull request #57760 from joscollin/wip-66277-squid

squid: cephfs-journal-tool: Add preventive measures to avoid fs corruption

Reviewed-by: Venky Shankar <vshankar@redhat.com>
13 months agoMerge pull request #57795 from rishabh-d-dave/wip-65920-squid
Jos Collin [Fri, 14 Jun 2024 11:12:26 +0000 (16:42 +0530)]
Merge pull request #57795 from rishabh-d-dave/wip-65920-squid

squid: mds: don't add counters in warning for standby-replay MDS

Reviewed-by: Jos Collin <jcollin@redhat.com>
13 months agoMerge pull request #57945 from ceph/wip-lusov-qdb-exclude-or-cancel-squid
Jos Collin [Fri, 14 Jun 2024 11:00:51 +0000 (16:30 +0530)]
Merge pull request #57945 from ceph/wip-lusov-qdb-exclude-or-cancel-squid

squid: mds: QuiesceDbRequest: update the internal encoding of ops

Reviewed-by: Jos Collin <jcollin@redhat.com>