Samuel Just [Fri, 13 Dec 2024 01:37:28 +0000 (17:37 -0800)]
doc/dev/crimson/pipeline.rst: simplify and update to reflect new stages
This commit updates pipeline.rst to include some basic information about
how the pipeline stages now work. I've removed the explicit listing of
the different stages as I'd rather readers refer to the actual
implementation for those details to avoid them getting out of date.
I also removed the comparison to classic as the approach has now diverged
quite a bit and I feel that the ordering part is more important to focus
on than the points at which processing might block.
Samuel Just [Wed, 27 Nov 2024 02:22:16 +0000 (18:22 -0800)]
crimson: introduce and use repop stage
Repops previously used PGPipeline::await_map. This is actually
important as we need them to be processed in order. However, using
await_map was confusing and using a single exclusive stage is decidedly
unoptimal as we could allow pipelineing on write commit. For now, move
them over to their own pipeline stage so we can remove the PGPipeline
struct entirely. Later, we'll improve replica write pipelining for
better replica-side write concurrency.
We want to emplace and initialize osd_op_params upon first write,
but we don't want to fill at_version, pg_trim_to, pg_committed_to,
or last_complete until prepare_transaction because we don't want to
require a particular commit order any earlier than we have to.
That the log entry's verison matches the object_info on the actual
object is a pretty core invariant. This commit moves creating the
log entry for head and populating the metadata into
OpsExecuter::prepare_head_update.
As a side effect, flush_clone_metadata and CloningCtx::apply_to
were removed and split between prepare_head_update (portions
related to the head's ssc) and flush_changes_and_submit.
Samuel Just [Thu, 24 Oct 2024 23:33:56 +0000 (16:33 -0700)]
crimson: expose CommonOBCPipeline via ObjectContextLoader::Orderer
- adds ObjectContext::obc_pipeline
- exposes ObjectContext::obc_pipeline via ObjectContextLoader::Orderer
- allows obcs to be in the registry without being loaded
- adds ObjectContext::loading bool to signal that loading has begun
Samuel Just [Thu, 24 Oct 2024 23:39:18 +0000 (16:39 -0700)]
crimson: manage log submission atomicity independently of pipeline stages
It's important to construct and submit log entries atomically because
the submission order needs to match the entry numbering. Previously,
this occurred under the pg-wide exclusive process stage. Shortly, each
obc will have its own process stage, so we need to ensure that atomicity
seperately from the pipeline stages. Introduce a simple mutex.
Samuel Just [Fri, 22 Nov 2024 02:34:12 +0000 (18:34 -0800)]
crimson: clear obcs whether referenced or not upon replica write
It's always possible for there to be an in-progress replica-read
when the replica processes a repop. It's rare in our tests because
the read and write submitted by the test client would need to
overlap in time. This makes the results non-deterministic and
thus a somewhat less sensitive test. Note, the space of valid
results is well defined -- it would have to be state before or
after any of the outstanding writes. Any other result or a torn
read would be wrong. It's probably worth updating RadosModel
to add such a pattern, but we can do that later.
This branch makes this race much more likely and even observable
with the existing RadosModel implementation as it extends the
obc lifetime past the point of returning the result to the client
in order to ensure that it outlives the handle.
Fixes: https://tracker.ceph.com/issues/69013 Signed-off-by: Samuel Just <sjust@redhat.com>
These tests assert on the number of keys on the group header object,
which changed with commits ec6224db3e55 ("cls/rbd: save group snapshot
creation order in a new key") and 3662bc7b3dc3 ("cls/rbd: save max
group snap order").
These tests assert on the number of keys on the group header object,
which changed with commits ec6224db3e55 ("cls/rbd: save group snapshot
creation order in a new key") and 3662bc7b3dc3 ("cls/rbd: save max
group snap order").
Zac Dover [Wed, 4 Dec 2024 02:13:05 +0000 (03:13 +0100)]
doc/rados: fix sentences in health-checks (3 of x)
Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.
This commit edits the second third of
doc/rados/operations/health-checks.rst.
Note to (I hope soon) future Zac: There are a a couple of places near
the end of this file where the sentences are ungrammatical. Update these
in a separate PR (in isolation, so that the grammar and technical
accuracy of these sentences can be the primary focus of the reviewers).
Dan Mick [Thu, 21 Nov 2024 02:18:59 +0000 (18:18 -0800)]
container/Containerfile: purge .repo files with secrets before commit
ceph.repo had creds in it for download.ceph.com/prerelease.
Remove the .repo files we construct, since they're not necessary
once the container is built (no one should be dnf'ing anything
in the container).
Laura Flores [Tue, 3 Dec 2024 22:15:19 +0000 (16:15 -0600)]
qa/workunits/mon: ensure election strategy is "connectivity" for stretch mode
The election strategy is randomly chosen for this type of test. Sometimes,
the test passes if the "connectivity" election strategy happens to be picked.
But if a different strategy, i.e. "classic", is picked, then the test will fail.
We can ensure that the election strategy is "connectivity" by setting it in the
workunit with the ceph CLI command. Although connectivity was specified in
stretch-mode-5-mons-8-osds.yaml, that config ultimately gets overridden by
the "qa/mon_config" yaml.
Fixes: https://tracker.ceph.com/issues/69107 Signed-off-by: Laura Flores <lflores@ibm.com>
Dan Mick [Fri, 1 Nov 2024 02:55:36 +0000 (19:55 -0700)]
container/make-manifest-list.py
- don't print command failure in worker; let the caller print them
if desired (allow silent failure)
- allow for empty tags list
- look for CEPH_SHA1. GIT_COMMIT was the sha1 of the ceph-container.git
commit
- change default paths to prerelease
- add --dry-run to avoid final push
- rename 'HOST' to 'CONTAINER_HOST'
- Use ARCH_SPECIFIC_HOST instead of CONTAINER_HOST (which is used by podman)
Zac Dover [Tue, 3 Dec 2024 11:02:43 +0000 (12:02 +0100)]
doc/rados: fix sentences in health-checks (2 of x)
Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.
This commit edits the second third of
doc/rados/operations/health-checks.rst.
Zac Dover [Tue, 3 Dec 2024 08:28:09 +0000 (09:28 +0100)]
doc/rados: make sentences agree in health-checks.rst
Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.
This commit edits the first third of
doc/rados/operations/health-checks.rst.
projects / ceph.git / log