Ronen Friedman [Sat, 5 Oct 2024 12:33:49 +0000 (07:33 -0500)]
osd/scrub: modify ScrubStore contents retrieval
A separate commit added a simple test to verify the new
store implementation (creating both shallow & deep errors),
scrubbing (step 1), deep scrubbing (step 2), then shallow
scrubbing again (step 3). The test verifies that
the results after step 2 include all shallow errors data (*),
and that the results after step 3 include all deep errors
data.
The test highlighted the need to correctly partition and
retrieve the "shards inconsistencies" and the "selected
shard" data, which was not fully implemented in the
previous commit. Thus, this commit adds the following:
- add_object_error() no longer filters out data saved during
deep scrubbing; it also filters less of the shallow scrubs
"shards inconsistencies" data;
- merge_encoded_error_wrappers() now merges the "shards
inconsistencies" data correctly, handling the multiple
scenarios possible.
(*) note the special case of not being able to read the
object's version during deep scrubbing (due to a read
error). In this case - the data collected during the
shallow scrub will not be reported.
common/scrub,osd/scrub: minor cleanups to ScrubStore
Including:
- introducing 'no out param' encode() for the inconsistent wrappers;
- renaming the ambiguous 'empty()' to 'is_empty()';
- removing unused code;
- a few other minor cleanups.
osd/scrub: add dout() capability to the ScrubStore
now that the ScrubSTore object is directly created by the
scrubber, (and has a lifetime that does not extend beyond
the scrubber object), we can add the same dout()
mechanism used by the other scrubber sub-objects.
Note: that mechanism will be changed shortly, so that the
sub-objects would use one prefix() creator supplied by
the Scrubber object.
osd/scrub: directly create or reinit the ScrubStore
The ScrubStore is now directly created or reinitialized by the
Scrubber. Note that the store object is not identical to the
errors DB: the errors DB is an entity in the OSD store (a
collection of OMap entries in a uniquely-named object(s)),
while the ScrubSTore object is a cacher and accessor for
that entity. That one can be recreated or disposed of at
will.
We now do not recreate the ScrubStore object for every scrub.
JonBailey1993 [Wed, 9 Oct 2024 10:28:42 +0000 (11:28 +0100)]
common/io_exerciser: Modify is_locked_by_me call in ceph_test_rados_io_sequence
is_locked_by_me() is a function of ceph::mutex which is only used in debug builds. By using the ceph_mutex_is_locked_by_me macro, we can neatly make sure we only run this function in debug mode, allowing compilation to no longer be affected when running in release mode.
Signed-off-by: Jon Bailey <jonathan.bailey1@ibm.com>
* refs/pull/60037/head:
test/common: add death test for double !recursive lock
common/test: do not test exception raised from recursive lock
test/common: fix invalid vim mode
common,osdc: remove obsolete ceph::mutex_debugging
common: assert debug mutex lock is not held if !recursive
Revert the commit (manually, by restoring the file by hand to the state
it was in prior to d7c144c) to the state that it was in before I added
the Executive Council Responsibilities document to governance.rst. This
document cannot be edited at will, but must be voted on by the
Leadership Team.
Aashish Sharma [Thu, 3 Oct 2024 08:28:14 +0000 (13:58 +0530)]
mgr/dashboard: fix gateways section error:”404 - Not Found RGW Daemon not found: None”
A case was missed here where we do have a default realm created but no default_zonegorup, in that case, the existing behavior should prevail and that's not being handled. If a default_realm is created but no default_zonegroup is there, weshould continue getting the keys from daemon_name = next(iter(daemon_keys))
mgr/dashboard: show non default realm sync status in rgw overview page
Currently, we just show the sync status of the default realm in rgw
overview page. This PR is to show the sync status of non-default realms
as well. Multisite sync status can be viewed for any of the active daemon
which runs in default/non-default realm.
Yuval Lifshitz [Tue, 1 Oct 2024 15:19:46 +0000 (15:19 +0000)]
common: missing std include with GCC 14
In file included from src/rgw/driver/posix/bucket_cache.h:19,
from src/test/rgw/test_posix_bucket_cache.cc:4:
src/common/cohort_lru.h: In member function _void cohort::lru::TreeX<T, TTree, CLT, CEQ, K, LK>::lock()_:
src/common/cohort_lru.h:334:14: error: _for_each_ is not a member of _std_
334 | std::for_each(locks.begin(), locks.end(),
| ^~~~~~~~
src/common/cohort_lru.h: In member function _void cohort::lru::TreeX<T, TTree, CLT, CEQ, K, LK>::unlock()_:
/home/yuvalif/ceph5/src/common/cohort_lru.h:339:14: error: _for_each_ is not a member of _std_
339 | std::for_each(locks.begin(), locks.end(),
| ^~~~~~~~
Aashish Sharma [Fri, 4 Oct 2024 10:54:02 +0000 (16:24 +0530)]
mgr/dashboard: increase timeout to detect replication user in the secondary cluster
Increase timeout to detect replication user in the secondary cluster in rgw multisite automation wizard. Currently its set to 2 mins, increase it to 5 minutes.
when you import realm token to the secondary cluster, we wait for the replication/system user we created in the primary cluster to be present in the secondary cluster and when we find that user we set the credentials in the secondary cluster using ceph dashboard set-rgw-crdentials . The timeout for this is set to 2 minutes and sometimes it takes more than 2 minutes for the user to be replicated in the secondary cluster
When libcephfs aio tests (src/test/client) are run
with objectcacher disabled (ceph_test_client --client_oc=false),
the TestClient.LlreadvLlwritev fails and core dumps. The client
hits the assert 'caps_ref[c]<0'.
This patch fixes the same. There is no need to give out cap_ref
and take it again between multiple read because of short reads.
In some cases, the get_caps used to fail in C_Read_Sync_NonBlocking::finish
causing cap_ref to go negative when put_cap_ref is done at last in
C_Read_Finish::finish_io
Cause:
In aio path, the client_lock was not being held
in the internal callback after the io is done where
it's expected to be taken leading to corruption.
Zac Dover [Fri, 4 Oct 2024 13:21:32 +0000 (23:21 +1000)]
doc/governance: add exec council responsibilites
Add the Ceph Executive Council's responsibilties to the
doc/governance.rst document. It was decided during the weekly CLT
meeting on 30 Sep 2024 to add this to the ceph/ceph git repository.
mgr/smb: fix condition for smb earmark when cluster_id doesn't match
This commit resolves an issue where accessing `earmark.split('.')[2]` would cause a
"list index out of range" error when the earmark is set to just "smb" without additional scopes.
The fix introduces a parsing function to safely handle earmarks, ensuring proper behavior
even when no cluster ID or additional scopes are present.
Ilya Dryomov [Thu, 3 Oct 2024 15:54:07 +0000 (17:54 +0200)]
librbd/crypto/LoadRequest: clone format for migration source image
Migration source and migration target images naturally have the same
encryption format, but the user shouldn't have to need to specify it
for the image that they can't even immediately see -- migration source
image gets moved to the RBD trash to avoid mistaken usage while
migration is in progress.
Formats must also be cloned if the image is under migration, so
rename m_is_current_format_cloned to m_is_current_format_assumed to
avoid potential confusion with clone() being called in two places but
m_is_current_format_cloned being set in only one place.
John Mulligan [Tue, 1 Oct 2024 15:27:44 +0000 (11:27 -0400)]
cephadm: use a shared smb.conf for clustered smb container sets
Use a shared smb.conf when deploying ctdb enabled containers. There was
a problem updating configs on the ctdb enabled clusters and the issue
was that the configwatch sidecar was not using CTDB, rather it had a
"default" copy of smb.conf that enabled only registry config, but not
CTDB. Examining the cluster this problem was found to be general to all
sidecars that are either sambacc based (not starting smbd, winbindd,
etc) and the smbmetrics sidecar.
Fixes: https://tracker.ceph.com/issues/68322 Signed-off-by: John Mulligan <jmulligan@redhat.com>
RGW: Cloud Restore cli and its corresponding response for user.
* For first and repititive request 202 Accepted will be corresponding response code.
* For CloudRestored status 200 OK will be corresponding response code.
* For conflicting requests 409 Conflict corresponding response code.
Also Fixed storage class update while listing objects.
Earlier while restoring object temporarily list-objects (s3api) and
radosgw-admin bucket list didn't have updated storage class. With this
fixed it now has the cloudtier storage class.
* It allows read-through for cloud-tiered objects via restore_obj_from_cloud
* New tier config options user need to set allow_read_through to true and
read_through_restore_days more than 1 for this feature to work, also
objects with retain_head_object will be available for this feature.
* First get request will fail with restoring in progress error, objects
are downloaded asynchronously.
* The objects restore are temporary.
* Tested `aws s3api get-object`, `aws s3api head-object` and `aws s3 cp`
In addition send timeout errors for first readthrough request
Also addressed lint warning and other cleanup(review comments)
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
projects / ceph.git / log