]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Radosław Zarzyński [Mon, 3 Jun 2024 22:00:38 +0000 (00:00 +0200)]
qa/rados: don't pass --max_attr_len to ceph_test_rados unconditionally
Fixes: https://tracker.ceph.com/issues/66321
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit
8a6d56e62fc03062e60828a3bc9f37ed7232b3e7 )
Matan Breizman [Tue, 18 Jun 2024 10:53:44 +0000 (13:53 +0300)]
Merge pull request #58093 from Matan-B/wip-55831-squid
squid: crimson/os/seastore: lba pointer consolidation
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 18 Jun 2024 10:52:04 +0000 (13:52 +0300)]
Merge pull request #58065 from Matan-B/wip-57383-squid
squid: crimson: fix user_version handling
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 18 Jun 2024 10:49:38 +0000 (13:49 +0300)]
Merge pull request #58030 from Matan-B/wip-57561-squid
squid: crimson/osd/ops_executer: LIST_SNAPS only on CEPH_SNAPDIR
Reviewed-by: Samuel Just <sjust@redhat.com>
Pedro Gonzalez Gomez [Tue, 18 Jun 2024 10:16:28 +0000 (12:16 +0200)]
Merge pull request #57862 from rhcs-dashboard/wip-66345-squid
squid: mgr/dashboard: add smb service management support
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Pedro Gonzalez Gomez [Tue, 18 Jun 2024 10:16:05 +0000 (12:16 +0200)]
Merge pull request #57861 from rhcs-dashboard/wip-66037-squid
squid: mgr/dashboard: add upgrade notification
Samuel Just [Thu, 9 May 2024 03:06:58 +0000 (20:06 -0700)]
crimson/osd: remove osd_op_params_t::user_at_version
osd_op_params_t::user_at_version was populated from
osd_op_params_t::at_version before the call to prepare_transaction,
which incremented osd_op_params_t::at_version.version. As a result,
the value stored in object_info_t::user_version ended up one version
behind object_info_t::version. The log entry, on the other hand,
ended up with the correct version as OpsExecutor::prepare_transaction
populates it directly from at_version. As a result, the primary could
return different versions to the client depending on whether the IO was
already in the log.
This commit eliminates osd_op_params_t::user_at_version and updates
PGBackend::mutate_object to behave like prepare_transaction. Because
the prior commit removes the prepare_transaction increment, this isn't
strictly necessary, but it is simpler.
Fixes: https://tracker.ceph.com/issues/65857
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
85fdadab2adcfbec83a82c6f1bae1f4f31a96f54 )
Samuel Just [Thu, 9 May 2024 03:39:18 +0000 (20:39 -0700)]
crimson/osd/ops_executor: only increment osd_op_params_t::at_version for clone
Previously, we incremented prior to usage in both prepare_transaction
and execute_clone. Because at_version is initialized from
PG::get_next_version(), this results in log entries skipping
values. Instead, only increment after populating clone
object_info.
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
7e0dde721d5203623b06351bb4b261b3c9fe2c33 )
Samuel Just [Thu, 9 May 2024 03:01:49 +0000 (20:01 -0700)]
crimson/osd/ops_executor: use clone oi values in log entry
The fields from the clone_obc oi are what we're actually
trying to fill in here -- we might as well use them
directly.
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
207e5a6e9ba54deb2fb18d5a5ad432d42072453d )
Samuel Just [Thu, 9 May 2024 02:45:45 +0000 (02:45 +0000)]
crimson/osd/ops_executor: update osd_op_params_t::at_version in execute_clone
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
1ecf3d64853c2fa92bf467692b98a58b4670b212 )
Samuel Just [Thu, 9 May 2024 02:31:35 +0000 (19:31 -0700)]
crimson/osd/pg: remove slightly confusing assert from PG::submit_transaction
This assert should always hold, so it's not wrong. However,
osd_op_p.at_version is chosen during transaction construction time
using the current epoch via PG::get_next_version. Asserting it here
is more confusing than helpful.
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
7a04f6230d9376d155ac92a6c3bf1ab5eeca644f )
Samuel Just [Thu, 9 May 2024 02:27:59 +0000 (19:27 -0700)]
crimson/osd/pg_backend: remove uninformative #if 0 block
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
2bd7c38381bc22bc73130e6d138e001096e66faf )
Matan Breizman [Tue, 18 Jun 2024 07:13:12 +0000 (10:13 +0300)]
Merge pull request #58064 from Matan-B/wip-57056-squid
squid: crimson/osd: make osd_op_params::at_version coherent with last log entry
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 18 Jun 2024 07:11:41 +0000 (10:11 +0300)]
Merge pull request #58063 from Matan-B/wip-57245-squid
squid: qa/config/crimson_qa_overrides: adjust mgr_stats_period
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 18 Jun 2024 07:10:38 +0000 (10:10 +0300)]
Merge pull request #58024 from Matan-B/wip-57251-squid
squid: crimson/osd/pg_backend: only create new onode when it doesn't exist
Reviewed-by: Samuel Just <sjust@redhat.com>
Guillaume Abrioux [Tue, 18 Jun 2024 06:40:27 +0000 (08:40 +0200)]
Merge pull request #57471 from guits/wip-66021-squid
squid: ceph-volume: use os.makedirs for mkdir_p
Guillaume Abrioux [Tue, 18 Jun 2024 06:39:55 +0000 (08:39 +0200)]
Merge pull request #57473 from guits/wip-66023-squid
squid: cephadm: change loki/promtail default image tags
Guillaume Abrioux [Tue, 18 Jun 2024 06:38:17 +0000 (08:38 +0200)]
Merge pull request #58004 from zdover23/wip-doc-2024-06-13-backport-57650-to-squid
squid: ceph-volume: use importlib from stdlib on Python 3.8 and up
Xuehan Xu [Thu, 16 May 2024 01:21:45 +0000 (09:21 +0800)]
crimson/os/seastore/btree: drop FixedKVNode::child_pos_for_key()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
bfc2abae90dc0cd2a3a0ef7b62006435b305f51d )
Xuehan Xu [Wed, 15 May 2024 08:15:01 +0000 (16:15 +0800)]
crimson/os/seastore/btree: better UT checks
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
dca239f84ca1cd0607be337b452673418036d0bc )
Xuehan Xu [Wed, 15 May 2024 07:46:01 +0000 (15:46 +0800)]
crimson/os/seastore/btree: correct lba pointer comments
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
9bb054b30f66d1b03f0ea6937b08806c4b64f5b6 )
Xuehan Xu [Thu, 29 Feb 2024 06:35:31 +0000 (14:35 +0800)]
crimson/os/seastore/cached_extent: EXIST_MUTATION_PENDING also
considered as mutatione_pending
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
211ca3792e80c4319a23596a5f439bab886c200f )
Xuehan Xu [Thu, 29 Feb 2024 06:16:37 +0000 (14:16 +0800)]
crimson/os/seastore/cached_extent: adjust
CachedExtent::on_replace_prior()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
d4c78f1524bebfa3f370f5a4a019800dfeaf04f1 )
Xuehan Xu [Thu, 29 Feb 2024 06:06:35 +0000 (14:06 +0800)]
crimson/os/seastore/btree: misc consolidations
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
72d721ad72d15296b069752985f12e173f9690da )
Xuehan Xu [Wed, 28 Feb 2024 10:26:24 +0000 (18:26 +0800)]
crimson/os/seastore/cache: drop unnecessary friend declarations
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
fecf501c41471e132427f2a40f26298a38ba4003 )
Xuehan Xu [Wed, 28 Feb 2024 09:59:41 +0000 (17:59 +0800)]
crimson/os/seastore/btree: add CachedExtent::on_rewrite() and
CachedExtent::is_rewrite()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
1b8a781f665df06b056511debec46deac604a79c )
Xuehan Xu [Wed, 28 Feb 2024 08:00:23 +0000 (16:00 +0800)]
crimson/os/seastore/lba_manager: extents pass to LBALeafNode::update()
must either has a null parent tracker or point to the current leaf node
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
4b155f3ee5ab4fbd212e117d2cd40dd78d846d3a )
Matan Breizman [Mon, 17 Jun 2024 15:25:09 +0000 (18:25 +0300)]
Merge pull request #58034 from Matan-B/wip-57593-squid
squid: Crimson: Support basic deployments
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:24:18 +0000 (18:24 +0300)]
Merge pull request #58027 from Matan-B/wip-57455-squid
squid: crimson/osd/replicated_backend: no need to set_rollback_to for repops
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:19:33 +0000 (18:19 +0300)]
Merge pull request #58022 from Matan-B/wip-57069-squid
squid: crimson/common/operation: detach blockers from blocking events when they are destroyed
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:18:34 +0000 (18:18 +0300)]
Merge pull request #58015 from Matan-B/wip-57474-squid
squid: crimson/os/seastore/transaction_manager: correct the offset of the data copied from the original extents
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:18:11 +0000 (18:18 +0300)]
Merge pull request #58013 from Matan-B/wip-57432-squid
squid: crimson/os/seastore/object_data_handler: cleanup read()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:17:24 +0000 (18:17 +0300)]
Merge pull request #58011 from Matan-B/wip-57368-squid
squid: crimson/os/seastore: add is_data_stable() to allow delta-overwrite on EXIST_CLEAN
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 13:30:13 +0000 (16:30 +0300)]
Merge pull request #58029 from Matan-B/wip-57692-squid
squid: crimson/osd/pg_shard_manager: remove the unnecessary "std::move" call
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 13:29:30 +0000 (16:29 +0300)]
Merge pull request #58021 from Matan-B/wip-57725-squid
squid: crimson/osd/object_context_loader: Fix obc cache existence usage
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 13:28:58 +0000 (16:28 +0300)]
Merge pull request #58016 from Matan-B/wip-56114-squid
squid: src/test/TestRados: add max-attr-len to control the max length of attributes sent to OSDs
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 13:28:18 +0000 (16:28 +0300)]
Merge pull request #58009 from Matan-B/wip-55488-squid
squid: crimson/osd/osd_operations/client_request: make loading-obc concurrent
Reviewed-by: Samuel Just <sjust@redhat.com>
Pedro Gonzalez Gomez [Mon, 17 Jun 2024 07:19:18 +0000 (09:19 +0200)]
Merge pull request #58075 from idryomov/wip-cephadm-flake8-fix-squid
squid: mgr/cephadm: fix flake8 test failures
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Jos Collin [Mon, 17 Jun 2024 05:08:15 +0000 (10:38 +0530)]
Merge pull request #57840 from rishabh-d-dave/wip-66330-squid
squid: mon,cephfs: require confirmation flag to bring down unhealthy MDS
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Jos Collin [Mon, 17 Jun 2024 04:54:25 +0000 (10:24 +0530)]
Merge pull request #57682 from batrick/wip-66196-squid
squid: mds: use regular dispatch for processing beacons
Reviewed-by: Jos Collin <jcollin@redhat.com>
Nizamudeen A [Sun, 16 Jun 2024 09:11:04 +0000 (14:41 +0530)]
mgr/cephadm: fix flake8 test failures
after a recent update on flake8, some tests in cephadm started failing
```
flake8: commands[0]> flake8 --config=tox.ini alerts balancer cephadm cli_api crash devicehealth diskprediction_local hello iostat localpool nfs orchestrator prometheus rbd_support rgw selftest smb
cephadm/configchecks.py:94:6: E204 whitespace after decorator '@'
cephadm/module.py:3017:10: E204 whitespace after decorator '@'
cephadm/module.py:3344:144: E226 missing whitespace around arithmetic operator
cephadm/tests/test_cephadm.py:982:6: E204 whitespace after decorator '@'
3 E204 whitespace after decorator '@'
1 E226 missing whitespace around arithmetic operator
```
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit
a9563d91c2ac405ae9101bd4e63fa9b749d54432 )
Radoslaw Zarzynski [Tue, 23 Apr 2024 14:10:22 +0000 (14:10 +0000)]
crimson/osd: make osd_op_params::at_version coherent with last log entry
Before this commit we were doing something like:
1. initialize `at_version` with PG::projected_last_update`
**incremented by one**.
2. produce a log entry at such version.
3. increment `at_version` for the sake of a further production
that may never come.
The problem is `osd_op_params::at_version` is higher by one
than the last log entry which hurts at later stages of
`osd_op_params` processing (I was hit in the shared EC code
by the assertion in `PG::op_applied`).
This patch changes the algorithm to:
A. initialize `at_version` with PG::projected_last_update`
**incremented by one**.
B. increment `at_version` for the sake of the very next production.
C. produce a log entry at this version.
Co-authored-by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
5e787fda2e6186e3963e29225b4fa0ac145d2c45 )
Radoslaw Zarzynski [Tue, 23 Apr 2024 14:04:02 +0000 (14:04 +0000)]
crimson/osd: s/fill_op_params_bump_pg_version/fill_op_params/ in OpsExecuter
Since https://github.com/ceph/ceph/pull/53054 it doesn't
bump `PG::projected_last_update` anymore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
eca3d26124cdbe5eff0d2e356417c977a37e115a )
Radoslaw Zarzynski [Tue, 23 Apr 2024 13:54:23 +0000 (13:54 +0000)]
crimson/osd: on the cloning path bump obj version closer to its usage
My understanding at the time of writing is this change is a pure
refactoring.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
99fdc0cc478c169c30627d4436bb7aebd3335c8d )
Matan Breizman [Sun, 16 Jun 2024 10:12:54 +0000 (13:12 +0300)]
Merge pull request #58033 from Matan-B/wip-57313-squid
squid: crimson/osd/ops_executor: calculation of clone_overlap shouldn't consider snap contexts
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:11:40 +0000 (13:11 +0300)]
Merge pull request #58031 from Matan-B/wip-57908-squid
squid: crimson/osd/osd_operations: correct connection pipelines for osd operations
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:09:57 +0000 (13:09 +0300)]
Merge pull request #58028 from Matan-B/wip-57416-squid
squid: crimson/osd/pg: hold PGs' references to the last minute of snap trim events executions
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:08:59 +0000 (13:08 +0300)]
Merge pull request #58026 from Matan-B/wip-57386-squid
squid: crimson/osd/recovery_backend: change recovery waiters' promises into optional ones
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:08:39 +0000 (13:08 +0300)]
Merge pull request #58025 from Matan-B/wip-57276-squid
squid: crimson/osd/pg_backend: DONOT modify OSDOp::indata when handling CEPH_OSD_OP_CHECKSUM
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:07:43 +0000 (13:07 +0300)]
Merge pull request #58023 from Matan-B/wip-57204-squid
squid: crimson/osd: clamp reads to object size and bring full read trimming
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:05:20 +0000 (13:05 +0300)]
Merge pull request #58020 from Matan-B/wip-57711-squid
squid: crimson/os/seastore/async_cleaner: fix incorrect get_num_rolls()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:04:56 +0000 (13:04 +0300)]
Merge pull request #58019 from Matan-B/wip-57694-squid
squid: crimson/os/seastore/rbm/avlallocator: return enough regions when request size is greater than max_alloc_size
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:04:28 +0000 (13:04 +0300)]
Merge pull request #58018 from Matan-B/wip-57656-squid
squid: test/crimson/seastore/transaction_manager_test_state: fix compilation…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:04:00 +0000 (13:04 +0300)]
Merge pull request #58017 from Matan-B/wip-57262-squid
squid: crimson/os/seastore/lba_manager: don't increase intermediate mappings' refcount if LBAManager::clone_mapping() is called to remap mappings
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 09:56:54 +0000 (12:56 +0300)]
Merge pull request #58014 from Matan-B/wip-55735-squid
squid: crimson/common/errorator: disallow void-returning error handlers
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 09:54:55 +0000 (12:54 +0300)]
Merge pull request #58012 from Matan-B/wip-57476-squid
squid: crimson/os/seastore/transaction_manager: drop unused code
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 09:53:53 +0000 (12:53 +0300)]
Merge pull request #58010 from Matan-B/wip-57434-squid
squid: crimson/os/seastore/record_scanner: replace [=] capturing
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 09:52:34 +0000 (12:52 +0300)]
Merge pull request #58008 from Matan-B/wip-57102-squid
squid: crimson/osd/osdop_params:Unify OpsExecuter::user_modify and osd_op_params_t::user_modify
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 2 May 2024 13:54:18 +0000 (13:54 +0000)]
qa/config/crimson_qa_overrides: adjust mgr_stats_period
Some test jobs fail with `local_shared_foreign_ptr: Assertion `ptr && *ptr' failed`
It seems that we attempt to use a connection which is not yet ready to use
after setting up the daemons on boot.
Adjust the mgr_stats_period to allow more time for the daemons to set up.
See: https://tracker.ceph.com/issues/62162#note-10
Note: This is not a fix but more of a temporary solution to avoid noise
in the testing suite (Tracker stays open).
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
752b4c6110590cdd85b04e6a01e6f1affcdee97c )
Anthony D'Atri [Sun, 16 Jun 2024 06:16:22 +0000 (02:16 -0400)]
Merge pull request #58059 from zdover23/wip-doc-2024-06-16-backport-58057-to-squid
squid: doc/rados: explain replaceable parts of command
Zac Dover [Sat, 15 Jun 2024 11:55:18 +0000 (21:55 +1000)]
doc/rados: explain replaceable parts of command
Add an explanation that directs the reader to replace the "X" part of
the command "ceph tell mon.X mon_status" with the value specific to the
reader's Ceph cluster (which is (probably) not "X").
In the future, such replaceable strings in commands may be bounded by
angle brackets ("<" and ">").
This improvement to the documentation was suggested on the [ceph-users]
email list by Joel Davidow. This email, an absolute model of user
engagement with an upstream project, can be reviewed here:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/KF67F5TXFSSTPXV7EKL6JKLA5KZQDLDQ/
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
d071ad2575c86f300a9ba39df3c4949e5dc9c47d )
Guillaume Abrioux [Mon, 3 Jun 2024 12:00:10 +0000 (14:00 +0200)]
ceph-volume: fix regression
This fixes a regression introduced by:
24f8e5c61b19deab7397b0237f8376c6c03a5dcb
`iter_entry_points` from `pkg_resources` takes one argument whereas
`entry_points` from `importlib.metadata` does not.
The call to `entry_points(group=group)` makes ceph-volume fail.
Fixes: https://tracker.ceph.com/issues/66328
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit
6e3b0b93055538cad234018ef8700bfacec03076 )
Laura Flores [Fri, 14 Jun 2024 17:52:42 +0000 (12:52 -0500)]
Merge pull request #57998 from ljflores/wip-tracker-66460
squid: qa/suites/rados/thrash-old-clients: update supported releases and distro
Anthony D'Atri [Fri, 14 Jun 2024 15:14:23 +0000 (11:14 -0400)]
Merge pull request #58049 from zdover23/wip-doc-2024-06-14-backport-58007-to-squid
squid: doc/rados: add pg-states and pg-concepts to tree
Jos Collin [Fri, 14 Jun 2024 11:33:46 +0000 (17:03 +0530)]
Merge pull request #57525 from batrick/wip-66044-squid
squid: qa: unmount clients before damaging the fs
Reviewed-by: Jos Collin <jcollin@redhat.com>
Jos Collin [Fri, 14 Jun 2024 11:20:23 +0000 (16:50 +0530)]
Merge pull request #57762 from joscollin/wip-66270-squid
squid: pybind/mgr/mirroring: Fix KeyError: 'directory_count' in daemon status
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Jos Collin [Fri, 14 Jun 2024 11:19:09 +0000 (16:49 +0530)]
Merge pull request #57760 from joscollin/wip-66277-squid
squid: cephfs-journal-tool: Add preventive measures to avoid fs corruption
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Jos Collin [Fri, 14 Jun 2024 11:12:26 +0000 (16:42 +0530)]
Merge pull request #57795 from rishabh-d-dave/wip-65920-squid
squid: mds: don't add counters in warning for standby-replay MDS
Reviewed-by: Jos Collin <jcollin@redhat.com>
Jos Collin [Fri, 14 Jun 2024 11:00:51 +0000 (16:30 +0530)]
Merge pull request #57945 from ceph/wip-lusov-qdb-exclude-or-cancel-squid
squid: mds: QuiesceDbRequest: update the internal encoding of ops
Reviewed-by: Jos Collin <jcollin@redhat.com>
Zac Dover [Thu, 13 Jun 2024 11:29:47 +0000 (21:29 +1000)]
doc/rados: add pg-states and pg-concepts to tree
Add "pg-states" and "pg-concepts" to the left tree pane on
docs.ceph.com.
This commit has been made in response to a request from the upstream
made in https://pad.ceph.com/p/Report_Documentation_Bugs.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
0629f47faf73a5b88adbeceaf022ee23111bae7d )
Laura Flores [Mon, 1 Apr 2024 17:27:54 +0000 (12:27 -0500)]
qa/suites: add "mon down" log variations to ignorelist
Fixes: https://tracker.ceph.com/issues/64864
Signed-off-by: Laura Flores <lflores@ibm.com>
(cherry picked from commit
d475ac3e6ab86a4913e1d318989c617031978bc2 )
Conflicts:
qa/suites/orch/cephadm/smoke/start.yaml
qa/suites/orch/cephadm/workunits/task/test_host_drain.yaml
qa/suites/orch/cephadm/workunits/task/test_monitoring_stack_basic.yaml
qa/suites/orch/cephadm/workunits/task/test_rgw_multisite.yaml
qa/suites/orch/cephadm/workunits/task/test_set_mon_crush_locations.yaml
The log-only-match entry was backported to squid before the ignorelist changes,
but in main it was introduced after the ignorelist changes.
See https://github.com/ceph/ceph/commit/
b4522dd332d40a54b9e0be58bd96aeaa345f8977 .
Matan Breizman [Thu, 30 May 2024 09:12:33 +0000 (09:12 +0000)]
crimson/osd: warn cluster when optimal deployment is not used
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
64a9915ecbafadc5afb16b04366a69fc4e548568 )
Matan Breizman [Tue, 21 May 2024 09:56:01 +0000 (09:56 +0000)]
crimson: introduce crimson_seastar_num_threads
This new option is there to allow *basic* cluster deployments.
crimson_seastar_num_threads can be used globally for all the OSDs in the cluster:
```
osd:
crimson_seastar_num_threads: <n>
crimson_alien_op_num_threads: <m>
```
As a result, all the available CPUs will be allocated to *both* seastar reactor threads
and to Alienstore threads - without any exclusion.
Notes:
* The core allocation will most likely overlap between OSDS and/or Seastar and Alienstore.
An optiomal deployment requires `crimson_alien_thread_cpu_cores`
and `crimson_seastar_cpu_cores` to be set for each OSD based on the host its located at.
* Seastar reactor number (smp::count) will be deduced directly from crimson_seastar_num_threads
---
Documentation followup: https://github.com/ceph/ceph/pull/57352
---
### Option 1 (Optimal)
* CPUs are pinned to the provided sets (seastar and alien cpu sets)
* thread-affinity = 1 (seastar's default).
* smp (smp::count / reactor count): `seastar_cpu_set.size()`
* Seastar reactor threads: `seastar_cpu_set.size()`
* Alienstore threads num: `crimson_alien_op_num_threads`
---
### Option 2 (Basic):
* No CPU pinning.
* thread-affinity = 0
* smp (smp::count / reactor count) = `crimson_seastar_num_threads`
* Seastar reactor threads num: `crimson_seastar_num_threads`
* Alienstore threads num: `crimson_alien_op_num_threads`
---
Fixes: https://tracker.ceph.com/issues/65752
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
e0b6fff4f3bf9495fc0fdc17e5ea1f53ffa1d423 )
Xuehan Xu [Tue, 7 May 2024 10:05:27 +0000 (18:05 +0800)]
crimson/osd/ops_executor: calculation of clone_overlap shouldn't consider
snap contexts
Fixes: https://tracker.ceph.com/issues/65832
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
213ae3f42cbb478b503212d704161b16e90cf878 )
Xuehan Xu [Thu, 6 Jun 2024 03:55:38 +0000 (11:55 +0800)]
crimson/osd/osd_operations: correct connection pipelines for osd
operations
Fixes: https://tracker.ceph.com/issues/66374
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
cb379bbdb7526c8858f64e035a676e23a82770a6 )
junxiang Mu [Mon, 20 May 2024 10:00:53 +0000 (06:00 -0400)]
crimson/osd/ops_executer: LIST_SNAPS only on CEPH_SNAPDIR
fixes: https://tracker.ceph.com/issues/65663
Signed-off-by: junxiang Mu <1948535941@qq.com>
(cherry picked from commit
2e7482ebcf19ae066e2d7c2d297d4f0d3250dc63 )
Xuehan Xu [Fri, 24 May 2024 09:35:47 +0000 (17:35 +0800)]
crimson/osd/pg_shard_manager: remove the unnecessary "std::move" call
Fixing the following compilation issue:
/home/xuxuehan/src/ceph/src/crimson/osd/pg_shard_manager.h:236:7: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
236 | );
| ^
/home/xuxuehan/src/ceph/src/crimson/osd/pg_shard_manager.h:236:7: note: remove ‘std::move’ call
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
3d77b5c7e2ed95d424cdc9b97f9b41cea1a2648a )
Xuehan Xu [Sat, 11 May 2024 07:01:54 +0000 (15:01 +0800)]
crimson/osd/pg: clear PG_STATE_SNAPTRIM on interval changes
This is to avoid the scenarios in which pgs get activated before
PG_STATE_SNAPTRIM of the last pg interval gets cleared
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
2c216b95ece4a0ec7ad723bdd2b00efce3135b2d )
Xuehan Xu [Sat, 11 May 2024 07:00:58 +0000 (15:00 +0800)]
crimson/osd/pg: hold PGs' references to the last minute of snap trim
events executions
Fixes: https://tracker.ceph.com/issues/65972
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
1116c418b4fed0c8b6ab42d4f6c23cdc1afbef57 )
Xuehan Xu [Tue, 14 May 2024 03:05:17 +0000 (11:05 +0800)]
crimson/osd/replicated_backend: no need to set_rollback_to for repops
Fixes: https://tracker.ceph.com/issues/66002
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
aeb43ba413d0c8fb6f8066107d48ac8f45cb2bfa )
Xuehan Xu [Fri, 10 May 2024 06:29:34 +0000 (14:29 +0800)]
crimson/osd/recovery_backend: change recovery waiters' promises into
optional ones
Fixes: https://tracker.ceph.com/issues/65894
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
422c94251d1ff1ecb38170afc9db16f6c5f7072c )
Xuehan Xu [Mon, 6 May 2024 02:11:40 +0000 (10:11 +0800)]
crimson/osd/pg_backend: DONOT modify OSDOp::indata when handling
CEPH_OSD_OP_CHECKSUM
So that crimson's handling of CEPH_OSD_OP_CHECKSUM is idempotent
Fixes: https://tracker.ceph.com/issues/65804
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
899afd3472abb8055924278e7ee29c609ec4eb54 )
Xuehan Xu [Fri, 3 May 2024 04:36:23 +0000 (12:36 +0800)]
crimson/osd/pg_backend: only create new onode when it doesn't exist
Fixes: https://tracker.ceph.com/issues/65773
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
814ee076d2721c52491f7677deae6df5b7366334 )
Radoslaw Zarzynski [Wed, 1 May 2024 14:06:29 +0000 (14:06 +0000)]
crimson/osd: clamp reads to object size and bring full read trimming
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
788c364de79787df5d6d808e423cb36b579f4fa7 )
Xuehan Xu [Wed, 24 Apr 2024 07:58:19 +0000 (15:58 +0800)]
crimson/osd/osd_operations/client_request: also dump pg_tracking_events
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
d11eddfb01e67a052116c22b863d92d37f21b362 )
Xuehan Xu [Wed, 24 Apr 2024 02:56:14 +0000 (10:56 +0800)]
crimson/common/operation: detach blockers from blocking events when they are destroyed
Fixes: https://tracker.ceph.com/issues/65632
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
89f23accb4a8d53492cb0a9d169c08177e6e1848 )
Xuehan Xu [Wed, 24 Apr 2024 02:39:05 +0000 (10:39 +0800)]
crimson/common/operation.h: move functionality of OrderedConcurrentPhaseT<T>::BlockingEvent into BlockerT
This makes compliers' instantiation of crimson::BlockerT<T>::BlockingEvent complete when instantiating
crimson::BlockerT<T>
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
01aa611e71a5ac8affaeaed2a2211c6bdc0d76ce )
Matan Breizman [Thu, 6 Jun 2024 09:48:09 +0000 (09:48 +0000)]
crimson/osd/object_context_loader: get_or_load to support atomicity
make use of try_lock in order to support atomicity when called in
ObjectContext::_with_lock()
Co-authored-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
1675ce8c1b5347ad13b65389686cd45853a4149e )
Yingxin Cheng [Mon, 3 Jun 2024 06:33:26 +0000 (14:33 +0800)]
crimson/common/tri_mutex: make lock() atomic if doesn't need wait
Otherwise, promotion cannot be atomic with the 1st locker.
Identified by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
f63d76a2ae348e1419dab20fab9557ce483080ca )
Yingxin Cheng [Mon, 3 Jun 2024 06:31:35 +0000 (14:31 +0800)]
crimson/common/tri_mutex: minor cleanup to be consistent
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
251b9d4bca14c314b467fd77202e76a70ed90c0f )
Matan Breizman [Sun, 2 Jun 2024 15:17:36 +0000 (15:17 +0000)]
crimson/common/tri_mutex: add waiter_t::waiter_name
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
71eef36644e50fdf269f898e707182e72e083eeb )
Matan Breizman [Sun, 2 Jun 2024 15:00:37 +0000 (15:00 +0000)]
crimson/common/tri_mutex: add debug logs
to be used only for testing
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
6c5106d134c7a4c308253b8ba04aa66416bf1689 )
Matan Breizman [Thu, 30 May 2024 11:48:46 +0000 (11:48 +0000)]
crimson/osd/object_context_loader: with_head_obc to log `existed`
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
583cb17a308c9c56851719371aac75d08e3f35c8 )
Matan Breizman [Tue, 28 May 2024 13:52:08 +0000 (13:52 +0000)]
crimson/osd/object_context_loader: cleanup `loaded`
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
5b5d2ea4aebcbbe3fec349377337fe38f5a2f27a )
Matan Breizman [Tue, 28 May 2024 13:23:10 +0000 (13:23 +0000)]
crimson/osd/object_context: await in-progress loading (per-obc)
```
// obc loading is a concurrent phase. In case this obc is being loaded,
// make other useres of this obc to await for the loading to complete.
```
Since we now await for loading to finish (in-case in progress), we can
also assert is_loaded().
Fixes: https://tracker.ceph.com/issues/65451
Co-authored-by: Xuehan Xu <xuxuehan@qianxin.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
e456bd7cdd1ad1cd68026cebedd5b1d9d01001f7 )
Matan Breizman [Mon, 27 May 2024 15:24:00 +0000 (15:24 +0000)]
crimson/osd/object_context: cleanup is_loaded_and_valid
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
ccc0001a5dc54aef5b4d4b3f85952f1edaf17fb1 )
Matan Breizman [Mon, 27 May 2024 13:21:41 +0000 (13:21 +0000)]
crimson/osd/object_context_loader: Fix obc cache existence usage
with_head_obc() uses get_cached_obc() to get_or_create obc instances.
If the obc exists in cache, get_or_load_obc is called with `existed`=true.
The assumption above is wrong.
Cache existence (`existed`) only guarantees that the obc instance was created (and inserted) in the obc_registery.
However, it does **not** assure that the obc was actually loaded.
As obc-loading is now concurrent, it's possible for the first user to only create the obc in
cache (without loading yet) and the second concurrent user to assume it was already loaded.
With this patch, we verify that the obc was loaded in get_or_load_obc.
* make loading-obc concurrent PR: https://github.com/ceph/ceph/pull/55488
Fixes: https://tracker.ceph.com/issues/64206
Fixes: https://tracker.ceph.com/issues/66214
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
0b90ee74e06cc5a3a30088fa39fde5634148bda1 )
Yingxin Cheng [Mon, 27 May 2024 02:16:41 +0000 (10:16 +0800)]
crimson/os/seastore/async_cleaner: fix incorrect get_num_rolls()
The number of journal segments should not be based on the committed
journal_head. Otherwise, if a new journal segment is just opened and the
committed journal_head hasn't been updated, the result will be wrong.
This causes ceph_assert(get_segments_reclaimable() == 0) in
SegmentCleaner::get_next_reclaim_segment().
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
5ae59cf52159bdc1ad3637da8654524cb7b4b4b6 )
Zhang Song [Fri, 24 May 2024 09:56:48 +0000 (17:56 +0800)]
crimson/os/seastore/rbm/avlallocator: return enough regions when request size is greater than max_alloc_size
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
(cherry picked from commit
8ff2a31c51dc3259d56764ddae11bf2fcd544d68 )
Matan Breizman [Thu, 23 May 2024 11:06:41 +0000 (11:06 +0000)]
test/crimson/seastore/transaction_manager_test_state: fix compilation error
Introduced in: https://github.com/ceph/ceph/commit/
898efa78e2cde90a2bdf8e6f7eda1d4def736705
Followup: https://github.com/ceph/ceph/pull/53433
```
FAILED: src/crimson/tools/CMakeFiles/perf-staged-fltree.dir/perf_staged_fltree.cc.o
error: ‘GetParam’ was not declared in this scope
287 | if (std::get<1>(GetParam()) == integrity_check_t::FULL_CHECK) {
| ^~~~~~~~
/home/yogisha/ceph/src/test/crimson/seastore/transaction_manager_test_state.h: In member function ‘virtual seastar::future<> SeaStoreTestState::_init()’:
/home/yogisha/ceph/src/test/crimson/seastore/transaction_manager_test_state.h:439:21: error: ‘GetParam’ was not declared in this scope
439 | if (std::get<1>(GetParam()) == integrity_check_t::FULL_CHECK) {
| ^~~~~~~~
```
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
8a182dffc1ad2571f53a4990c19cb4a329722101 )