]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Samuel Just [Thu, 9 May 2024 03:06:58 +0000 (20:06 -0700)]
crimson/osd: remove osd_op_params_t::user_at_version
osd_op_params_t::user_at_version was populated from
osd_op_params_t::at_version before the call to prepare_transaction,
which incremented osd_op_params_t::at_version.version. As a result,
the value stored in object_info_t::user_version ended up one version
behind object_info_t::version. The log entry, on the other hand,
ended up with the correct version as OpsExecutor::prepare_transaction
populates it directly from at_version. As a result, the primary could
return different versions to the client depending on whether the IO was
already in the log.
This commit eliminates osd_op_params_t::user_at_version and updates
PGBackend::mutate_object to behave like prepare_transaction. Because
the prior commit removes the prepare_transaction increment, this isn't
strictly necessary, but it is simpler.
Fixes: https://tracker.ceph.com/issues/65857
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
85fdadab2adcfbec83a82c6f1bae1f4f31a96f54 )
Samuel Just [Thu, 9 May 2024 03:39:18 +0000 (20:39 -0700)]
crimson/osd/ops_executor: only increment osd_op_params_t::at_version for clone
Previously, we incremented prior to usage in both prepare_transaction
and execute_clone. Because at_version is initialized from
PG::get_next_version(), this results in log entries skipping
values. Instead, only increment after populating clone
object_info.
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
7e0dde721d5203623b06351bb4b261b3c9fe2c33 )
Samuel Just [Thu, 9 May 2024 03:01:49 +0000 (20:01 -0700)]
crimson/osd/ops_executor: use clone oi values in log entry
The fields from the clone_obc oi are what we're actually
trying to fill in here -- we might as well use them
directly.
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
207e5a6e9ba54deb2fb18d5a5ad432d42072453d )
Samuel Just [Thu, 9 May 2024 02:45:45 +0000 (02:45 +0000)]
crimson/osd/ops_executor: update osd_op_params_t::at_version in execute_clone
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
1ecf3d64853c2fa92bf467692b98a58b4670b212 )
Samuel Just [Thu, 9 May 2024 02:31:35 +0000 (19:31 -0700)]
crimson/osd/pg: remove slightly confusing assert from PG::submit_transaction
This assert should always hold, so it's not wrong. However,
osd_op_p.at_version is chosen during transaction construction time
using the current epoch via PG::get_next_version. Asserting it here
is more confusing than helpful.
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
7a04f6230d9376d155ac92a6c3bf1ab5eeca644f )
Samuel Just [Thu, 9 May 2024 02:27:59 +0000 (19:27 -0700)]
crimson/osd/pg_backend: remove uninformative #if 0 block
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
2bd7c38381bc22bc73130e6d138e001096e66faf )
Matan Breizman [Tue, 18 Jun 2024 07:13:12 +0000 (10:13 +0300)]
Merge pull request #58064 from Matan-B/wip-57056-squid
squid: crimson/osd: make osd_op_params::at_version coherent with last log entry
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 18 Jun 2024 07:11:41 +0000 (10:11 +0300)]
Merge pull request #58063 from Matan-B/wip-57245-squid
squid: qa/config/crimson_qa_overrides: adjust mgr_stats_period
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Tue, 18 Jun 2024 07:10:38 +0000 (10:10 +0300)]
Merge pull request #58024 from Matan-B/wip-57251-squid
squid: crimson/osd/pg_backend: only create new onode when it doesn't exist
Reviewed-by: Samuel Just <sjust@redhat.com>
Guillaume Abrioux [Tue, 18 Jun 2024 06:40:27 +0000 (08:40 +0200)]
Merge pull request #57471 from guits/wip-66021-squid
squid: ceph-volume: use os.makedirs for mkdir_p
Guillaume Abrioux [Tue, 18 Jun 2024 06:39:55 +0000 (08:39 +0200)]
Merge pull request #57473 from guits/wip-66023-squid
squid: cephadm: change loki/promtail default image tags
Guillaume Abrioux [Tue, 18 Jun 2024 06:38:17 +0000 (08:38 +0200)]
Merge pull request #58004 from zdover23/wip-doc-2024-06-13-backport-57650-to-squid
squid: ceph-volume: use importlib from stdlib on Python 3.8 and up
Matan Breizman [Mon, 17 Jun 2024 15:25:09 +0000 (18:25 +0300)]
Merge pull request #58034 from Matan-B/wip-57593-squid
squid: Crimson: Support basic deployments
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:24:18 +0000 (18:24 +0300)]
Merge pull request #58027 from Matan-B/wip-57455-squid
squid: crimson/osd/replicated_backend: no need to set_rollback_to for repops
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:19:33 +0000 (18:19 +0300)]
Merge pull request #58022 from Matan-B/wip-57069-squid
squid: crimson/common/operation: detach blockers from blocking events when they are destroyed
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:18:34 +0000 (18:18 +0300)]
Merge pull request #58015 from Matan-B/wip-57474-squid
squid: crimson/os/seastore/transaction_manager: correct the offset of the data copied from the original extents
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:18:11 +0000 (18:18 +0300)]
Merge pull request #58013 from Matan-B/wip-57432-squid
squid: crimson/os/seastore/object_data_handler: cleanup read()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 15:17:24 +0000 (18:17 +0300)]
Merge pull request #58011 from Matan-B/wip-57368-squid
squid: crimson/os/seastore: add is_data_stable() to allow delta-overwrite on EXIST_CLEAN
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 13:30:13 +0000 (16:30 +0300)]
Merge pull request #58029 from Matan-B/wip-57692-squid
squid: crimson/osd/pg_shard_manager: remove the unnecessary "std::move" call
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 13:29:30 +0000 (16:29 +0300)]
Merge pull request #58021 from Matan-B/wip-57725-squid
squid: crimson/osd/object_context_loader: Fix obc cache existence usage
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 13:28:58 +0000 (16:28 +0300)]
Merge pull request #58016 from Matan-B/wip-56114-squid
squid: src/test/TestRados: add max-attr-len to control the max length of attributes sent to OSDs
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Mon, 17 Jun 2024 13:28:18 +0000 (16:28 +0300)]
Merge pull request #58009 from Matan-B/wip-55488-squid
squid: crimson/osd/osd_operations/client_request: make loading-obc concurrent
Reviewed-by: Samuel Just <sjust@redhat.com>
Pedro Gonzalez Gomez [Mon, 17 Jun 2024 07:19:18 +0000 (09:19 +0200)]
Merge pull request #58075 from idryomov/wip-cephadm-flake8-fix-squid
squid: mgr/cephadm: fix flake8 test failures
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Jos Collin [Mon, 17 Jun 2024 05:08:15 +0000 (10:38 +0530)]
Merge pull request #57840 from rishabh-d-dave/wip-66330-squid
squid: mon,cephfs: require confirmation flag to bring down unhealthy MDS
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Jos Collin [Mon, 17 Jun 2024 04:54:25 +0000 (10:24 +0530)]
Merge pull request #57682 from batrick/wip-66196-squid
squid: mds: use regular dispatch for processing beacons
Reviewed-by: Jos Collin <jcollin@redhat.com>
Nizamudeen A [Sun, 16 Jun 2024 09:11:04 +0000 (14:41 +0530)]
mgr/cephadm: fix flake8 test failures
after a recent update on flake8, some tests in cephadm started failing
```
flake8: commands[0]> flake8 --config=tox.ini alerts balancer cephadm cli_api crash devicehealth diskprediction_local hello iostat localpool nfs orchestrator prometheus rbd_support rgw selftest smb
cephadm/configchecks.py:94:6: E204 whitespace after decorator '@'
cephadm/module.py:3017:10: E204 whitespace after decorator '@'
cephadm/module.py:3344:144: E226 missing whitespace around arithmetic operator
cephadm/tests/test_cephadm.py:982:6: E204 whitespace after decorator '@'
3 E204 whitespace after decorator '@'
1 E226 missing whitespace around arithmetic operator
```
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit
a9563d91c2ac405ae9101bd4e63fa9b749d54432 )
Radoslaw Zarzynski [Tue, 23 Apr 2024 14:10:22 +0000 (14:10 +0000)]
crimson/osd: make osd_op_params::at_version coherent with last log entry
Before this commit we were doing something like:
1. initialize `at_version` with PG::projected_last_update`
**incremented by one**.
2. produce a log entry at such version.
3. increment `at_version` for the sake of a further production
that may never come.
The problem is `osd_op_params::at_version` is higher by one
than the last log entry which hurts at later stages of
`osd_op_params` processing (I was hit in the shared EC code
by the assertion in `PG::op_applied`).
This patch changes the algorithm to:
A. initialize `at_version` with PG::projected_last_update`
**incremented by one**.
B. increment `at_version` for the sake of the very next production.
C. produce a log entry at this version.
Co-authored-by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
5e787fda2e6186e3963e29225b4fa0ac145d2c45 )
Radoslaw Zarzynski [Tue, 23 Apr 2024 14:04:02 +0000 (14:04 +0000)]
crimson/osd: s/fill_op_params_bump_pg_version/fill_op_params/ in OpsExecuter
Since https://github.com/ceph/ceph/pull/53054 it doesn't
bump `PG::projected_last_update` anymore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
eca3d26124cdbe5eff0d2e356417c977a37e115a )
Radoslaw Zarzynski [Tue, 23 Apr 2024 13:54:23 +0000 (13:54 +0000)]
crimson/osd: on the cloning path bump obj version closer to its usage
My understanding at the time of writing is this change is a pure
refactoring.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
99fdc0cc478c169c30627d4436bb7aebd3335c8d )
Matan Breizman [Sun, 16 Jun 2024 10:12:54 +0000 (13:12 +0300)]
Merge pull request #58033 from Matan-B/wip-57313-squid
squid: crimson/osd/ops_executor: calculation of clone_overlap shouldn't consider snap contexts
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:11:40 +0000 (13:11 +0300)]
Merge pull request #58031 from Matan-B/wip-57908-squid
squid: crimson/osd/osd_operations: correct connection pipelines for osd operations
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:09:57 +0000 (13:09 +0300)]
Merge pull request #58028 from Matan-B/wip-57416-squid
squid: crimson/osd/pg: hold PGs' references to the last minute of snap trim events executions
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:08:59 +0000 (13:08 +0300)]
Merge pull request #58026 from Matan-B/wip-57386-squid
squid: crimson/osd/recovery_backend: change recovery waiters' promises into optional ones
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:08:39 +0000 (13:08 +0300)]
Merge pull request #58025 from Matan-B/wip-57276-squid
squid: crimson/osd/pg_backend: DONOT modify OSDOp::indata when handling CEPH_OSD_OP_CHECKSUM
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:07:43 +0000 (13:07 +0300)]
Merge pull request #58023 from Matan-B/wip-57204-squid
squid: crimson/osd: clamp reads to object size and bring full read trimming
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:05:20 +0000 (13:05 +0300)]
Merge pull request #58020 from Matan-B/wip-57711-squid
squid: crimson/os/seastore/async_cleaner: fix incorrect get_num_rolls()
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:04:56 +0000 (13:04 +0300)]
Merge pull request #58019 from Matan-B/wip-57694-squid
squid: crimson/os/seastore/rbm/avlallocator: return enough regions when request size is greater than max_alloc_size
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:04:28 +0000 (13:04 +0300)]
Merge pull request #58018 from Matan-B/wip-57656-squid
squid: test/crimson/seastore/transaction_manager_test_state: fix compilation…
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 10:04:00 +0000 (13:04 +0300)]
Merge pull request #58017 from Matan-B/wip-57262-squid
squid: crimson/os/seastore/lba_manager: don't increase intermediate mappings' refcount if LBAManager::clone_mapping() is called to remap mappings
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 09:56:54 +0000 (12:56 +0300)]
Merge pull request #58014 from Matan-B/wip-55735-squid
squid: crimson/common/errorator: disallow void-returning error handlers
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 09:54:55 +0000 (12:54 +0300)]
Merge pull request #58012 from Matan-B/wip-57476-squid
squid: crimson/os/seastore/transaction_manager: drop unused code
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 09:53:53 +0000 (12:53 +0300)]
Merge pull request #58010 from Matan-B/wip-57434-squid
squid: crimson/os/seastore/record_scanner: replace [=] capturing
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Sun, 16 Jun 2024 09:52:34 +0000 (12:52 +0300)]
Merge pull request #58008 from Matan-B/wip-57102-squid
squid: crimson/osd/osdop_params:Unify OpsExecuter::user_modify and osd_op_params_t::user_modify
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan Breizman [Thu, 2 May 2024 13:54:18 +0000 (13:54 +0000)]
qa/config/crimson_qa_overrides: adjust mgr_stats_period
Some test jobs fail with `local_shared_foreign_ptr: Assertion `ptr && *ptr' failed`
It seems that we attempt to use a connection which is not yet ready to use
after setting up the daemons on boot.
Adjust the mgr_stats_period to allow more time for the daemons to set up.
See: https://tracker.ceph.com/issues/62162#note-10
Note: This is not a fix but more of a temporary solution to avoid noise
in the testing suite (Tracker stays open).
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
752b4c6110590cdd85b04e6a01e6f1affcdee97c )
Anthony D'Atri [Sun, 16 Jun 2024 06:16:22 +0000 (02:16 -0400)]
Merge pull request #58059 from zdover23/wip-doc-2024-06-16-backport-58057-to-squid
squid: doc/rados: explain replaceable parts of command
Zac Dover [Sat, 15 Jun 2024 11:55:18 +0000 (21:55 +1000)]
doc/rados: explain replaceable parts of command
Add an explanation that directs the reader to replace the "X" part of
the command "ceph tell mon.X mon_status" with the value specific to the
reader's Ceph cluster (which is (probably) not "X").
In the future, such replaceable strings in commands may be bounded by
angle brackets ("<" and ">").
This improvement to the documentation was suggested on the [ceph-users]
email list by Joel Davidow. This email, an absolute model of user
engagement with an upstream project, can be reviewed here:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/KF67F5TXFSSTPXV7EKL6JKLA5KZQDLDQ/
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
d071ad2575c86f300a9ba39df3c4949e5dc9c47d )
Guillaume Abrioux [Mon, 3 Jun 2024 12:00:10 +0000 (14:00 +0200)]
ceph-volume: fix regression
This fixes a regression introduced by:
24f8e5c61b19deab7397b0237f8376c6c03a5dcb
`iter_entry_points` from `pkg_resources` takes one argument whereas
`entry_points` from `importlib.metadata` does not.
The call to `entry_points(group=group)` makes ceph-volume fail.
Fixes: https://tracker.ceph.com/issues/66328
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit
6e3b0b93055538cad234018ef8700bfacec03076 )
Laura Flores [Fri, 14 Jun 2024 17:52:42 +0000 (12:52 -0500)]
Merge pull request #57998 from ljflores/wip-tracker-66460
squid: qa/suites/rados/thrash-old-clients: update supported releases and distro
Anthony D'Atri [Fri, 14 Jun 2024 15:14:23 +0000 (11:14 -0400)]
Merge pull request #58049 from zdover23/wip-doc-2024-06-14-backport-58007-to-squid
squid: doc/rados: add pg-states and pg-concepts to tree
Jos Collin [Fri, 14 Jun 2024 11:33:46 +0000 (17:03 +0530)]
Merge pull request #57525 from batrick/wip-66044-squid
squid: qa: unmount clients before damaging the fs
Reviewed-by: Jos Collin <jcollin@redhat.com>
Jos Collin [Fri, 14 Jun 2024 11:20:23 +0000 (16:50 +0530)]
Merge pull request #57762 from joscollin/wip-66270-squid
squid: pybind/mgr/mirroring: Fix KeyError: 'directory_count' in daemon status
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Jos Collin [Fri, 14 Jun 2024 11:19:09 +0000 (16:49 +0530)]
Merge pull request #57760 from joscollin/wip-66277-squid
squid: cephfs-journal-tool: Add preventive measures to avoid fs corruption
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Jos Collin [Fri, 14 Jun 2024 11:12:26 +0000 (16:42 +0530)]
Merge pull request #57795 from rishabh-d-dave/wip-65920-squid
squid: mds: don't add counters in warning for standby-replay MDS
Reviewed-by: Jos Collin <jcollin@redhat.com>
Jos Collin [Fri, 14 Jun 2024 11:00:51 +0000 (16:30 +0530)]
Merge pull request #57945 from ceph/wip-lusov-qdb-exclude-or-cancel-squid
squid: mds: QuiesceDbRequest: update the internal encoding of ops
Reviewed-by: Jos Collin <jcollin@redhat.com>
Zac Dover [Thu, 13 Jun 2024 11:29:47 +0000 (21:29 +1000)]
doc/rados: add pg-states and pg-concepts to tree
Add "pg-states" and "pg-concepts" to the left tree pane on
docs.ceph.com.
This commit has been made in response to a request from the upstream
made in https://pad.ceph.com/p/Report_Documentation_Bugs.
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit
0629f47faf73a5b88adbeceaf022ee23111bae7d )
Laura Flores [Mon, 1 Apr 2024 17:27:54 +0000 (12:27 -0500)]
qa/suites: add "mon down" log variations to ignorelist
Fixes: https://tracker.ceph.com/issues/64864
Signed-off-by: Laura Flores <lflores@ibm.com>
(cherry picked from commit
d475ac3e6ab86a4913e1d318989c617031978bc2 )
Conflicts:
qa/suites/orch/cephadm/smoke/start.yaml
qa/suites/orch/cephadm/workunits/task/test_host_drain.yaml
qa/suites/orch/cephadm/workunits/task/test_monitoring_stack_basic.yaml
qa/suites/orch/cephadm/workunits/task/test_rgw_multisite.yaml
qa/suites/orch/cephadm/workunits/task/test_set_mon_crush_locations.yaml
The log-only-match entry was backported to squid before the ignorelist changes,
but in main it was introduced after the ignorelist changes.
See https://github.com/ceph/ceph/commit/
b4522dd332d40a54b9e0be58bd96aeaa345f8977 .
Matan Breizman [Thu, 30 May 2024 09:12:33 +0000 (09:12 +0000)]
crimson/osd: warn cluster when optimal deployment is not used
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
64a9915ecbafadc5afb16b04366a69fc4e548568 )
Matan Breizman [Tue, 21 May 2024 09:56:01 +0000 (09:56 +0000)]
crimson: introduce crimson_seastar_num_threads
This new option is there to allow *basic* cluster deployments.
crimson_seastar_num_threads can be used globally for all the OSDs in the cluster:
```
osd:
crimson_seastar_num_threads: <n>
crimson_alien_op_num_threads: <m>
```
As a result, all the available CPUs will be allocated to *both* seastar reactor threads
and to Alienstore threads - without any exclusion.
Notes:
* The core allocation will most likely overlap between OSDS and/or Seastar and Alienstore.
An optiomal deployment requires `crimson_alien_thread_cpu_cores`
and `crimson_seastar_cpu_cores` to be set for each OSD based on the host its located at.
* Seastar reactor number (smp::count) will be deduced directly from crimson_seastar_num_threads
---
Documentation followup: https://github.com/ceph/ceph/pull/57352
---
### Option 1 (Optimal)
* CPUs are pinned to the provided sets (seastar and alien cpu sets)
* thread-affinity = 1 (seastar's default).
* smp (smp::count / reactor count): `seastar_cpu_set.size()`
* Seastar reactor threads: `seastar_cpu_set.size()`
* Alienstore threads num: `crimson_alien_op_num_threads`
---
### Option 2 (Basic):
* No CPU pinning.
* thread-affinity = 0
* smp (smp::count / reactor count) = `crimson_seastar_num_threads`
* Seastar reactor threads num: `crimson_seastar_num_threads`
* Alienstore threads num: `crimson_alien_op_num_threads`
---
Fixes: https://tracker.ceph.com/issues/65752
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
e0b6fff4f3bf9495fc0fdc17e5ea1f53ffa1d423 )
Xuehan Xu [Tue, 7 May 2024 10:05:27 +0000 (18:05 +0800)]
crimson/osd/ops_executor: calculation of clone_overlap shouldn't consider
snap contexts
Fixes: https://tracker.ceph.com/issues/65832
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
213ae3f42cbb478b503212d704161b16e90cf878 )
Xuehan Xu [Thu, 6 Jun 2024 03:55:38 +0000 (11:55 +0800)]
crimson/osd/osd_operations: correct connection pipelines for osd
operations
Fixes: https://tracker.ceph.com/issues/66374
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
cb379bbdb7526c8858f64e035a676e23a82770a6 )
Xuehan Xu [Fri, 24 May 2024 09:35:47 +0000 (17:35 +0800)]
crimson/osd/pg_shard_manager: remove the unnecessary "std::move" call
Fixing the following compilation issue:
/home/xuxuehan/src/ceph/src/crimson/osd/pg_shard_manager.h:236:7: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
236 | );
| ^
/home/xuxuehan/src/ceph/src/crimson/osd/pg_shard_manager.h:236:7: note: remove ‘std::move’ call
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
3d77b5c7e2ed95d424cdc9b97f9b41cea1a2648a )
Xuehan Xu [Sat, 11 May 2024 07:01:54 +0000 (15:01 +0800)]
crimson/osd/pg: clear PG_STATE_SNAPTRIM on interval changes
This is to avoid the scenarios in which pgs get activated before
PG_STATE_SNAPTRIM of the last pg interval gets cleared
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
2c216b95ece4a0ec7ad723bdd2b00efce3135b2d )
Xuehan Xu [Sat, 11 May 2024 07:00:58 +0000 (15:00 +0800)]
crimson/osd/pg: hold PGs' references to the last minute of snap trim
events executions
Fixes: https://tracker.ceph.com/issues/65972
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
1116c418b4fed0c8b6ab42d4f6c23cdc1afbef57 )
Xuehan Xu [Tue, 14 May 2024 03:05:17 +0000 (11:05 +0800)]
crimson/osd/replicated_backend: no need to set_rollback_to for repops
Fixes: https://tracker.ceph.com/issues/66002
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
aeb43ba413d0c8fb6f8066107d48ac8f45cb2bfa )
Xuehan Xu [Fri, 10 May 2024 06:29:34 +0000 (14:29 +0800)]
crimson/osd/recovery_backend: change recovery waiters' promises into
optional ones
Fixes: https://tracker.ceph.com/issues/65894
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
422c94251d1ff1ecb38170afc9db16f6c5f7072c )
Xuehan Xu [Mon, 6 May 2024 02:11:40 +0000 (10:11 +0800)]
crimson/osd/pg_backend: DONOT modify OSDOp::indata when handling
CEPH_OSD_OP_CHECKSUM
So that crimson's handling of CEPH_OSD_OP_CHECKSUM is idempotent
Fixes: https://tracker.ceph.com/issues/65804
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
899afd3472abb8055924278e7ee29c609ec4eb54 )
Xuehan Xu [Fri, 3 May 2024 04:36:23 +0000 (12:36 +0800)]
crimson/osd/pg_backend: only create new onode when it doesn't exist
Fixes: https://tracker.ceph.com/issues/65773
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
814ee076d2721c52491f7677deae6df5b7366334 )
Radoslaw Zarzynski [Wed, 1 May 2024 14:06:29 +0000 (14:06 +0000)]
crimson/osd: clamp reads to object size and bring full read trimming
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit
788c364de79787df5d6d808e423cb36b579f4fa7 )
Xuehan Xu [Wed, 24 Apr 2024 07:58:19 +0000 (15:58 +0800)]
crimson/osd/osd_operations/client_request: also dump pg_tracking_events
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
d11eddfb01e67a052116c22b863d92d37f21b362 )
Xuehan Xu [Wed, 24 Apr 2024 02:56:14 +0000 (10:56 +0800)]
crimson/common/operation: detach blockers from blocking events when they are destroyed
Fixes: https://tracker.ceph.com/issues/65632
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
89f23accb4a8d53492cb0a9d169c08177e6e1848 )
Xuehan Xu [Wed, 24 Apr 2024 02:39:05 +0000 (10:39 +0800)]
crimson/common/operation.h: move functionality of OrderedConcurrentPhaseT<T>::BlockingEvent into BlockerT
This makes compliers' instantiation of crimson::BlockerT<T>::BlockingEvent complete when instantiating
crimson::BlockerT<T>
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit
01aa611e71a5ac8affaeaed2a2211c6bdc0d76ce )
Matan Breizman [Thu, 6 Jun 2024 09:48:09 +0000 (09:48 +0000)]
crimson/osd/object_context_loader: get_or_load to support atomicity
make use of try_lock in order to support atomicity when called in
ObjectContext::_with_lock()
Co-authored-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
1675ce8c1b5347ad13b65389686cd45853a4149e )
Yingxin Cheng [Mon, 3 Jun 2024 06:33:26 +0000 (14:33 +0800)]
crimson/common/tri_mutex: make lock() atomic if doesn't need wait
Otherwise, promotion cannot be atomic with the 1st locker.
Identified by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
f63d76a2ae348e1419dab20fab9557ce483080ca )
Yingxin Cheng [Mon, 3 Jun 2024 06:31:35 +0000 (14:31 +0800)]
crimson/common/tri_mutex: minor cleanup to be consistent
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
251b9d4bca14c314b467fd77202e76a70ed90c0f )
Matan Breizman [Sun, 2 Jun 2024 15:17:36 +0000 (15:17 +0000)]
crimson/common/tri_mutex: add waiter_t::waiter_name
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
71eef36644e50fdf269f898e707182e72e083eeb )
Matan Breizman [Sun, 2 Jun 2024 15:00:37 +0000 (15:00 +0000)]
crimson/common/tri_mutex: add debug logs
to be used only for testing
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
6c5106d134c7a4c308253b8ba04aa66416bf1689 )
Matan Breizman [Thu, 30 May 2024 11:48:46 +0000 (11:48 +0000)]
crimson/osd/object_context_loader: with_head_obc to log `existed`
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
583cb17a308c9c56851719371aac75d08e3f35c8 )
Matan Breizman [Tue, 28 May 2024 13:52:08 +0000 (13:52 +0000)]
crimson/osd/object_context_loader: cleanup `loaded`
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
5b5d2ea4aebcbbe3fec349377337fe38f5a2f27a )
Matan Breizman [Tue, 28 May 2024 13:23:10 +0000 (13:23 +0000)]
crimson/osd/object_context: await in-progress loading (per-obc)
```
// obc loading is a concurrent phase. In case this obc is being loaded,
// make other useres of this obc to await for the loading to complete.
```
Since we now await for loading to finish (in-case in progress), we can
also assert is_loaded().
Fixes: https://tracker.ceph.com/issues/65451
Co-authored-by: Xuehan Xu <xuxuehan@qianxin.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
e456bd7cdd1ad1cd68026cebedd5b1d9d01001f7 )
Matan Breizman [Mon, 27 May 2024 15:24:00 +0000 (15:24 +0000)]
crimson/osd/object_context: cleanup is_loaded_and_valid
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
ccc0001a5dc54aef5b4d4b3f85952f1edaf17fb1 )
Matan Breizman [Mon, 27 May 2024 13:21:41 +0000 (13:21 +0000)]
crimson/osd/object_context_loader: Fix obc cache existence usage
with_head_obc() uses get_cached_obc() to get_or_create obc instances.
If the obc exists in cache, get_or_load_obc is called with `existed`=true.
The assumption above is wrong.
Cache existence (`existed`) only guarantees that the obc instance was created (and inserted) in the obc_registery.
However, it does **not** assure that the obc was actually loaded.
As obc-loading is now concurrent, it's possible for the first user to only create the obc in
cache (without loading yet) and the second concurrent user to assume it was already loaded.
With this patch, we verify that the obc was loaded in get_or_load_obc.
* make loading-obc concurrent PR: https://github.com/ceph/ceph/pull/55488
Fixes: https://tracker.ceph.com/issues/64206
Fixes: https://tracker.ceph.com/issues/66214
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
0b90ee74e06cc5a3a30088fa39fde5634148bda1 )
Yingxin Cheng [Mon, 27 May 2024 02:16:41 +0000 (10:16 +0800)]
crimson/os/seastore/async_cleaner: fix incorrect get_num_rolls()
The number of journal segments should not be based on the committed
journal_head. Otherwise, if a new journal segment is just opened and the
committed journal_head hasn't been updated, the result will be wrong.
This causes ceph_assert(get_segments_reclaimable() == 0) in
SegmentCleaner::get_next_reclaim_segment().
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit
5ae59cf52159bdc1ad3637da8654524cb7b4b4b6 )
Zhang Song [Fri, 24 May 2024 09:56:48 +0000 (17:56 +0800)]
crimson/os/seastore/rbm/avlallocator: return enough regions when request size is greater than max_alloc_size
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
(cherry picked from commit
8ff2a31c51dc3259d56764ddae11bf2fcd544d68 )
Matan Breizman [Thu, 23 May 2024 11:06:41 +0000 (11:06 +0000)]
test/crimson/seastore/transaction_manager_test_state: fix compilation error
Introduced in: https://github.com/ceph/ceph/commit/
898efa78e2cde90a2bdf8e6f7eda1d4def736705
Followup: https://github.com/ceph/ceph/pull/53433
```
FAILED: src/crimson/tools/CMakeFiles/perf-staged-fltree.dir/perf_staged_fltree.cc.o
error: ‘GetParam’ was not declared in this scope
287 | if (std::get<1>(GetParam()) == integrity_check_t::FULL_CHECK) {
| ^~~~~~~~
/home/yogisha/ceph/src/test/crimson/seastore/transaction_manager_test_state.h: In member function ‘virtual seastar::future<> SeaStoreTestState::_init()’:
/home/yogisha/ceph/src/test/crimson/seastore/transaction_manager_test_state.h:439:21: error: ‘GetParam’ was not declared in this scope
439 | if (std::get<1>(GetParam()) == integrity_check_t::FULL_CHECK) {
| ^~~~~~~~
```
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit
8a182dffc1ad2571f53a4990c19cb4a329722101 )
Xuehan Xu [Mon, 20 May 2024 02:45:25 +0000 (10:45 +0800)]
crimson/os/seastore/lba_manager: hide incref_extent(t, addr, delta)
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
4b4d04bef29f82d07f667570f758f7993e429dd3 )
Xuehan Xu [Thu, 16 May 2024 06:56:20 +0000 (14:56 +0800)]
crimson/os/seastore: add comments to highlight necessary invariants
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
ac4981160e790368cbb49b469937bee83edd26fb )
Xuehan Xu [Mon, 13 May 2024 03:48:28 +0000 (11:48 +0800)]
crimson/os/seastore/cache: add a TODO comment to recommand dropping
`Cache::retire_extent_addr` in the future
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
cf49ef60755e3201b9126e46c4f0208416acc41c )
Xuehan Xu [Fri, 10 May 2024 08:34:02 +0000 (16:34 +0800)]
crimson/os/seastore/object_data_handler: minor bug fix
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
ab312f3e2b5db784afc4427c0511afbebb8838f8 )
Xuehan Xu [Wed, 8 May 2024 03:14:33 +0000 (11:14 +0800)]
crimson/os/seastore/lba_manager: clean up LBAManager::decref_extent()
interface
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
502d922bdd45b6177080a6da8466420c8de563ec )
Xuehan Xu [Thu, 9 May 2024 12:02:07 +0000 (20:02 +0800)]
crimson/os/seastore/transaction_manager: refactor extent remapping
dec_ref/alloc/clone/inc_ref of lba mappings caused by extent remappings
are now integrated into a single LBAManager::remap_lba_mappings()
interface
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
a2e244388c4822d81474bf0e45caada6c97e6e24 )
Xuehan Xu [Thu, 9 May 2024 11:59:19 +0000 (19:59 +0800)]
crimson/os/seastore/transaction_manager: add get_extent_if_linked() interface
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
a6f90bc27ec62c6595028aa97524abffd682dcf3 )
Xuehan Xu [Thu, 9 May 2024 12:01:29 +0000 (20:01 +0800)]
crimson/os/seastore/cache: add retire_absent_extent_addr() interface
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
6e583ccf458c740d6da86a19722b3e484aaf285c )
Xuehan Xu [Sat, 4 May 2024 04:21:13 +0000 (12:21 +0800)]
crimson/os/seastore/lba_manager: don't increase intermediate mappings'
refcount if LBAManager::clone_mapping() is called to remap mappings
TransactionManager::remap_pin() will increase the refcount itself
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
0cac72adc175404a013e86e450c58e9fb3f94af4 )
Xuehan Xu [Mon, 11 Mar 2024 08:41:21 +0000 (16:41 +0800)]
qa/suites/crimson-rados(-experimental): set max-attr-len to 8192 for crimson thrash
tests
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
45943555ea170e630218eb1ae107b606ba06f0a7 )
Xuehan Xu [Mon, 11 Mar 2024 08:38:54 +0000 (16:38 +0800)]
test/osd/TestRados: add parameter max-attr-len
SeaStore can't handle too big attributes
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
45a6d3f2c7fd5281a398b08762849fd37488ac21 )
Xuehan Xu [Wed, 15 May 2024 07:58:24 +0000 (15:58 +0800)]
crimson/os/seastore: set last_committed_crc for rewritten extents
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
e7ccb91c1d1c6b0048edab96cb70b0cc671d6124 )
Xuehan Xu [Wed, 15 May 2024 07:31:14 +0000 (15:31 +0800)]
crimson/os/seastore/transaction_manager: correct the offset of the data
copied from the original extents
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
a136cc62d6878b124c41700e2f55ca214cdf041a )
Xuehan Xu [Tue, 27 Feb 2024 04:49:41 +0000 (12:49 +0800)]
crimson/common/errorator: drop discard_all
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
0043fa92ab2435b4237d0d79112ab397730a8e91 )
Xuehan Xu [Mon, 26 Feb 2024 02:27:55 +0000 (10:27 +0800)]
crimson: adjust all void-returning handlers to either return values or
switch to errorator::asserts
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
2f2b4f6170dbbd24c404d2bf58170e0818df8c80 )
Xuehan Xu [Sat, 2 Mar 2024 07:33:55 +0000 (15:33 +0800)]
crimson/common/errorator: make assert_all() interfaces accept functors
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
(cherry picked from commit
a543db27108003fa4ace41a559efc5fb35555d23 )