]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Adam Kupczyk [Mon, 17 Jul 2023 07:31:05 +0000 (07:31 +0000)]
os/bluestore: Fix split_cache
Fix cache was broken as it moved blobs and extents between cache shards
without changing atomic num_blobs, num_extent counters.
Now tracking counters correctly.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 4 May 2023 08:50:10 +0000 (08:50 +0000)]
os/bluestore: Add UT for ExtentMap::dup_esb()
Add unit tests for ExtentMap::dup_esb() to test_bluestore_types.
Extra hooks added to BlueStore:
1) bluestore_extent_ref_map_t::debug_peek inspects how many times specific AU is used
2) ExtentMap::debug_list_disk_layout to extract how onode maps to AUs
3) BlueStore::debug_punch_hole to use logic from _wctx_finish internal function
Unit tests are limited to operations aligned to allocation unit.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Igor Fedotov [Wed, 19 Apr 2023 15:00:10 +0000 (18:00 +0300)]
test/bluestore_types: introduce basic UT for new ExtentMap::dup
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
Adam Kupczyk [Mon, 20 Feb 2023 08:52:48 +0000 (08:52 +0000)]
os/store_test: Change test limits
Test ReproNoBlobMultiTest required very narrow value of l_bluestore_gc_merged.
After ExtentMap::dup changes it can no longer reach it.
Generally that perf counter is useless; as we have now less allocated space
912798515 vs
9160228864 before.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Mon, 13 Feb 2023 14:04:21 +0000 (14:04 +0000)]
os/store_test: New SyntheticShardingLimited tests operate on limited objects
Iterate over few objects with limited onode shard size.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 9 Feb 2023 09:25:48 +0000 (09:25 +0000)]
os/store_test: New SyntheticLimited tests operate on limited objects
New SyntheticLimited tests operate on restiricted amount of objects.
This helps to more easily replicate problems that occur when we
manipulate objects that have in-flight "writing" buffers.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Mon, 16 Jan 2023 13:29:15 +0000 (13:29 +0000)]
tests/ceph_test_objectstore: Fix clone_range
It was possible to get srcoff and dstoff different.
It is untested and error-prone.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Wed, 9 Aug 2023 07:30:03 +0000 (07:30 +0000)]
os/bluestore: Simplify
Remove unnecessary arguments from functions.
Reduce unneeded indirections for accessing extent_map.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Fri, 31 Mar 2023 09:41:45 +0000 (09:41 +0000)]
os/bluestore: Enable selection of old/new ExtentMap::dup
Modify _do_clone_range to select variant of ExtentMap::dup depending on elastic_shared_blob.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Fri, 31 Mar 2023 09:04:40 +0000 (09:04 +0000)]
os/bluestore: Add conf option elastic_shared_blobs
Having new ExtentMap::dup that introduces heavy changes to blob processing
seems risky. We will enable it only on demand. In future, once the feature
is tested in production, the choice should be removed (and feature always on).
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 11 May 2023 10:53:24 +0000 (10:53 +0000)]
os/bluestore: Resurrect ExtentMap::dup
1. Rename ExtentMap::dup to ExtentMap::dup_esb
2. Ressurect ExtentMap::dup
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Mon, 16 Jan 2023 12:46:39 +0000 (12:46 +0000)]
os/bluestore: Fix copying Buffers
Only copy Buffers that were not copied before.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Wed, 10 May 2023 12:20:10 +0000 (12:20 +0000)]
os/bluestore: Modify ExtentMap::dup to reuse blobs
Make ExtentMap::dup able to reuse some already existing shared blobs,
when a regular blob has to be transformed to shared blob.
Make ExtentMap::dup() use make_range_shared_maybe_merge a primary tool for cloning.
Modify ExtentMap::dup to make a diligent merge of blobs.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Wed, 10 May 2023 12:40:38 +0000 (12:40 +0000)]
os/bluestore: Add make_range_shared_maybe_merge
Adds function that converts a specified range in object to shared blobs,
possibly merging them with other shared blobs.
Modify make_range_shared_maybe_merge to allow for merging blobs.
It now uses can_merge_blobs and merge_blobs.
Add discard_unused_buffers() to make_range_shared.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Wed, 10 May 2023 11:22:16 +0000 (11:22 +0000)]
os/bluestore: Introduce reblob_extents
Func reblob_extents is used to modify extents.
It is final step of melding 2 blobs together.
It removes reference to blob that is to be phased out,
and replaces it with reference to the blob that is the sum of them.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Wed, 10 May 2023 12:18:38 +0000 (12:18 +0000)]
os/bluestore: Add scan_shared_blobs and find_best_companion
Introduce function that will scan through relevant range that is to be cloned,
and function that will find best matching blob to attach to.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Wed, 10 May 2023 09:30:23 +0000 (09:30 +0000)]
os/bluestore: Add discard_unused_buffers function
When we punch_hole in blobs we leave Buffers unchanged.
Normally it is not a problem, but when we merge blobs there is a collision.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Wed, 10 May 2023 09:53:06 +0000 (09:53 +0000)]
os/bluestore: Add can_merge_blob and merge_blob
Two new functions:
1) can_marge_blob checks if 2 blobs are compatible to be merged together
2) merge_blob merges 2 blobs into 1, emptying source and putting all to destination
Modify merge_blob() to return logical length of produced blob.
Clear "unused" bitmap
When make blob shared or merge it with other blob, clear unused.
It drops some potential optimizations for writing into large blobs,
but it is unlikely to be useful.
Such info can be useful only after it reverts from Shared to regular blob.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Mon, 16 Jan 2023 11:16:43 +0000 (11:16 +0000)]
os/bluestore: Add functions for partial blob copy
Add functions that can copy parts of blobs.
It is necessary for merging blobs together,
which happens on cloning (ExtentMap::dup).
Fixed:
Blob::copy_extents_over_empty was faulty when insertion was targetting
last extent and that extent was invalid(empty).
Add dup() for bluestore_blob_t and bluestore_blob_use_tracker.
Changed:
Modified Blob::copy_from for better readability.
Added bluestore_blob_t::adjust_to initization that conforms to
other blob specifics.
Move assert for is_mutable() out of bluestore_blob_t::add_tail,
so it can be used in blobs that are shared.
Modify bluestore_blob_use_tracker_t::get to automatically expand
when accessing more AUs then originally declared.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Mon, 21 Nov 2022 11:25:40 +0000 (11:25 +0000)]
os/bluestore: Remove inject_21040.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Thu, 25 May 2023 19:36:02 +0000 (19:36 +0000)]
os/bluestore: Add wrappers to access Blob::bc (BufferSpace)
The reason was historical - it was to give one access method to bc regardles where it was defined SharedBlob or Blob.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 25 May 2023 19:26:28 +0000 (19:26 +0000)]
os/bluestore: Fix tracking of num_blobs
By moving BufferSpace from SharedBlob to Blob tracking of num_blobs get broken.
Fixed that and reinforced by adding asserts to BufferCacheShard destructor.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 25 May 2023 19:20:51 +0000 (19:20 +0000)]
os/bluestore: Add lock in ExtentMap::dup
_dup_writing requires locking of BufferCacheShard
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Sun, 27 Nov 2022 11:44:10 +0000 (11:44 +0000)]
os/bluestore: Refactor, add _dup_writing when cloning
With BufferSpace now attached to Blob (was SharedBlob inside it),
on-the-fly 'writing' buffers must be copied to clones.
Otherwise those objects will read data from disk before it is written there.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Fri, 24 Mar 2023 20:49:30 +0000 (20:49 +0000)]
os/bluestore: Fix BlueStore::_do_remove
Fix blobs having the same empty shared blob.
Each blob on creation gets its own unique (empty) SharedBlob object.
ExtentMap::dup() sometimes merges blobs together, so 2 different blobs
get the same SharedBlob object.
Function _do_remove() tries to convert shared blobs into regular ones.
If it succeeds we could get 2 blobs having the same EMPTY SharedBlob object.
The solution is to create detached SharedBlob if necessary.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Sun, 27 Nov 2022 11:26:27 +0000 (11:26 +0000)]
os/bluestore: Adapt split_cache
Adapt split_cache to new situation.
Now buffers are attached to Blob, and we need always move them,
regardless that we already moved relevant SharedBlob.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Wed, 23 Nov 2022 15:07:23 +0000 (15:07 +0000)]
os/bluestore: Move BufferSpace from SharedBlob to Blob
This is necessary to enable adding more Buffers to Blobs that are shared.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Wed, 23 Nov 2022 11:49:47 +0000 (11:49 +0000)]
os/bluestore: Refactor finish_write
Move finish_write from SharedBlob to Blob.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Wed, 23 Nov 2022 11:20:25 +0000 (11:20 +0000)]
os/bluestore: Refactor TransContext
Modified TransContext.
Changed
std::set<SharedBlobRef> shared_blobs_written
to
std::set<BlobRef> blobs_written
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Mon, 16 Jan 2023 13:32:59 +0000 (13:32 +0000)]
os/bluestore: Better dout for bluestore_blob_t
Add printing of len to operator<< for const bluestore_blob_t.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Mon, 16 Jan 2023 09:28:47 +0000 (09:28 +0000)]
os/bluestore: Add operator<< for BufferSpace
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 19 Jan 2023 13:22:10 +0000 (13:22 +0000)]
os/bluestore: Added printing of csum size
Modifed bluestore_blob_t to include current size of csum_data.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Wed, 10 May 2023 11:37:33 +0000 (11:37 +0000)]
os/bluestore: Debugs before ceph_assert in resharding.
When encode_some fails twice ceph aborts.
Now we print object details just before.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 16 Feb 2023 12:46:33 +0000 (12:46 +0000)]
os/bluestore: add ExtentMap::maybe_reshard
maybe_reshard is created to filter out unnecessary calls to request_reshard.
The intended use is to let just request maybe_reshard, and delegate check
if the action is really necessary to the implementation detail level.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 16 Feb 2023 12:21:53 +0000 (12:21 +0000)]
os/bluestore: Fix ExtentMap::reshard
When reshard is applied for the first time, expand reshard range to encompas whole object.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Thu, 16 Feb 2023 12:40:25 +0000 (12:40 +0000)]
os/bluestore: Improve ExtentMap::fault_range
Make faster exit when sharding not enabled.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Tue, 10 Jan 2023 17:40:57 +0000 (17:40 +0000)]
os/bluestore: Add printing of broken shared blobs
When we do fsck with non-repair mode, we do not get any info about shared blobs
that actually were corrupted. Now we print them.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Tue, 10 Jan 2023 10:50:54 +0000 (10:50 +0000)]
os/bluestore: Refactor foreach_shared_blob in fsck
Upgrade local foreach_shared_blob into _fsck_foreach_shared_blob
that can be used on entire BlueStore scope.
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adam Kupczyk [Fri, 27 Jan 2023 15:13:10 +0000 (15:13 +0000)]
os/bluestore: Expand TwoQBufferCacheShard::_audit
Add more checks on consistency.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Tue, 24 Jan 2023 13:05:52 +0000 (13:05 +0000)]
os/bluestore: Fix DEBUG_CACHE
Fix it, so it can be enabled and work.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Adam Kupczyk [Fri, 17 Feb 2023 14:51:55 +0000 (14:51 +0000)]
os/store_test: Retune tests to current code
After introduction of lazy statfs updates and mechanism to store them at exit,
some tests required tune-up.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Anthony D'Atri [Fri, 24 Mar 2023 17:54:06 +0000 (13:54 -0400)]
Merge pull request #50660 from zdover23/wip-doc-2023-03-25-rados-operations-bluestore-migration-prompt-fix
doc/operations: fix prompt in bluestore-migration
Zac Dover [Fri, 24 Mar 2023 17:47:10 +0000 (03:47 +1000)]
doc/operations: fix prompt in bluestore-migration
Fix a single prompt in bluestore-migration.rst.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Ilya Dryomov [Fri, 24 Mar 2023 12:00:10 +0000 (13:00 +0100)]
Merge pull request #50302 from weirdwiz/rbd-perf-counters
rbd-mirror: switch to labeled perf counters
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Rishabh Dave [Fri, 24 Mar 2023 10:58:38 +0000 (16:28 +0530)]
Merge pull request #50497 from rishabh-d-dave/fs-qa-caps-helper
qa/cephfs: add more helper methods to caps_helper.py
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Ilya Dryomov [Fri, 24 Mar 2023 10:41:04 +0000 (11:41 +0100)]
Merge pull request #49302 from petrutlucian94/adapter_resets
rbd-wnbd: optionally handle wnbd adapter restart events
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Avan [Fri, 24 Mar 2023 09:26:28 +0000 (14:56 +0530)]
Merge pull request #50369 from rhcs-dashboard/exporter-labeled-counters
exporter: add support for exposing labeled perf counters
Lucian Petrut [Thu, 23 Mar 2023 08:24:01 +0000 (08:24 +0000)]
rbd-wnbd: consistently use negative error codes in rbd-wnbd
The rbd-wnbd iterators return positive errors, which is why
in certain cases we may end up with both positive and negative
error codes.
This change ensures that we'll consistently use negative
error codes.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Lucian Petrut [Wed, 7 Dec 2022 13:18:48 +0000 (15:18 +0200)]
common, rbd-wnbd: bump Windows log level
We're increasing the log level for certain Windows operational log
messages.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Lucian Petrut [Wed, 16 Nov 2022 11:24:34 +0000 (13:24 +0200)]
rbd-wnbd: optionally handle wnbd adapter restart events
The WNBD adapter may be reset in certain situations (e.g. driver
upgrade, MS WHQL tests, etc).
We're going to monitor the WNBD adapter using WMI[1] events, restarting
the rbd-wnbd disk mappings whenever necessary. Adapter monitoring can be
enabled by passing the --adapter-monitoring-enabled flag to the service.
This feature is optional for the following reasons:
* it's mainly used during development / driver certification
* we had to use a relatively small polling interval, which might imply
additional resource usage. WMI quotas also have to be considered.
While at it, we're updating two lambdas that are submitted to thread pools,
avoiding default reference capturing and explicitly specifying the variables
that get copied.
[1] https://learn.microsoft.com/en-us/windows/win32/wmisdk/wmi-start-page
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Rishabh Dave [Mon, 13 Mar 2023 10:46:13 +0000 (16:16 +0530)]
qa/cephfs: add more helper methods to caps_helper.py
Add methods that will accept read/write permissions, CephFS names and
CephFS mount point and in return will generate string form of MON, OSD
and MDS caps exactly as it is reported in Ceph keyrings.
Replace similar code in test_multifs_auth.py with calls to these helper
methods.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Anthony D'Atri [Fri, 24 Mar 2023 00:11:04 +0000 (20:11 -0400)]
Merge pull request #50649 from Thingee/foundation-mem-update-
20230323
doc/foundation: Update Foundation members
Mike Perez [Thu, 23 Mar 2023 22:51:17 +0000 (15:51 -0700)]
doc/foundation: Update Foundation members
Removing EasyStack, Vexxhost and adding 42on
Signed-off-by: Mike Perez <thingee@gmail.com>
J. Eric Ivancich [Thu, 23 Mar 2023 19:29:19 +0000 (15:29 -0400)]
Merge pull request #50545 from ivancich/wip-fix-bi-restore-script-installation
rgw: install rgw scripts with common files rather than radosgw files
Reviewed-by: Casey Bodley <cbodley@redhat.com>
J. Eric Ivancich [Thu, 23 Mar 2023 19:17:34 +0000 (15:17 -0400)]
Merge pull request #50617 from ivancich/wip-add-unordered-list-restore-index
rgw: add unordered listing to reindex to force stats update
Reviewed-by: Casey Bodley <cbodley@redhat.com>
avanthakkar [Fri, 3 Mar 2023 13:20:00 +0000 (18:50 +0530)]
exporter: export labeled perf-counters as prometheus metrics
Fixes: https://tracker.ceph.com/issues/59063
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
zdover23 [Thu, 23 Mar 2023 13:05:28 +0000 (23:05 +1000)]
Merge pull request #50634 from zdover23/wip-doc-2023-03-23-rados-operations-user-management-1-of-x
doc/rados: edit user-management.rst (1 of x)
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Thu, 23 Mar 2023 02:21:06 +0000 (12:21 +1000)]
doc/rados: edit user-management.rst (1 of x)
Edit doc/rados/operations/user-management.rst. (1 of x)
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
Yingxin [Thu, 23 Mar 2023 09:23:01 +0000 (17:23 +0800)]
Merge pull request #50555 from myoungwon/wip-rbm-get-stat
crimson/os/seastore: implement get_stat() for RBM
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
myoungwon oh [Thu, 16 Mar 2023 06:35:59 +0000 (15:35 +0900)]
crimson/os/seastore/rbm: implement get_stat()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Yuri Weinstein [Wed, 22 Mar 2023 23:24:51 +0000 (16:24 -0700)]
Merge pull request #50334 from kamoltat/wip-ksirivad-fix-autoscale-warn
pybind/mgr/pg_autoscaler: Reorderd if statement for the func: _maybe_adjust
Reviewed-by: Laura Flores <lflores@redhat.com>
Yuri Weinstein [Wed, 22 Mar 2023 23:23:56 +0000 (16:23 -0700)]
Merge pull request #49748 from ifed01/wip-ifed-bound-rm-range-keys
kv/RocksDBStore: use bounded iterators in rm_range_keys
Reviewed-by: Cory Snyder <csnyder@iland.com>
Yuri Weinstein [Wed, 22 Mar 2023 23:23:10 +0000 (16:23 -0700)]
Merge pull request #49380 from Matan-B/wip-matanb-lrod-logs
osd/PeeringState: Add logs around can_serve_replica_read() / last_complete_ondisk()
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Casey Bodley [Wed, 22 Mar 2023 17:24:44 +0000 (13:24 -0400)]
Merge pull request #50510 from cbodley/wip-rgw-server-name
rgw: add default Server response header
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Casey Bodley [Wed, 22 Mar 2023 17:22:33 +0000 (13:22 -0400)]
Merge pull request #50504 from cbodley/wip-59048
rgw/s3: DeleteObjects response uses correct delete_marker flag
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Lucian Petrut [Wed, 16 Nov 2022 08:50:14 +0000 (10:50 +0200)]
common: add win32/wstring.h
Windows APIs heavily use wchar. ceph-dokan and rbd-wnbd
have some duplicated helpers that convert wstrings to/from
utf8 strings.
To avoid duplication and allow reusing those helpers, we're moving
them to common/win32/wstring.h.
We're using the "win32" subfolder because it's unlikely that this
will ever be used on other platforms.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Matan [Wed, 22 Mar 2023 14:26:53 +0000 (16:26 +0200)]
Merge pull request #50492 from Matan-B/wip-matanb-c-logging-levels
crimson/common/log: Fix debug logging macros
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Matan [Wed, 22 Mar 2023 14:25:18 +0000 (16:25 +0200)]
Merge pull request #50491 from Matan-B/wip-matanb-c-snaps-tests
test/librados/snapshots*: Enable Crimson supported tests
Reviewed-by: Samuel Just <sjust@redhat.com>
Matan [Wed, 22 Mar 2023 14:24:02 +0000 (16:24 +0200)]
Merge pull request #50546 from Matan-B/wip-matanb-crimson-only-order-snap
crimson/osd: Support librados::OPERATION_ORDERSNAP
Reviewed-by: Samuel Just <sjust@redhat.com>
Casey Bodley [Wed, 22 Mar 2023 12:59:19 +0000 (08:59 -0400)]
Merge pull request #49328 from BryceCao/wip-fix-python-script-using-s3cmd-with-error-code-403
rgw : fix python script using s3cmd with error code 403
Reviewed-by: Casey Bodley <cbodley@redhat.com>
zdover23 [Tue, 21 Mar 2023 22:09:20 +0000 (08:09 +1000)]
Merge pull request #50606 from zdover23/wip-doc-2023-03-21-rados-operations-erasure-code
doc/rados: line-edit erasure-code.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
J. Eric Ivancich [Sat, 18 Mar 2023 18:35:39 +0000 (14:35 -0400)]
rgw: add unordered listing to reindex to force stats update
By including an unordered listing in the script, we will complete
placing objects in the bucket index and allow stats to be updated
rather than waiting for this to happen organically at a user's
request. Unordered is preferred as it can run more efficiently.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Casey Bodley [Tue, 21 Mar 2023 21:52:28 +0000 (17:52 -0400)]
Merge pull request #50486 from linuxbox2/wip-dedup-opsqs
Do not duplicate query-string in ops-log
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Laura Flores [Tue, 21 Mar 2023 20:42:52 +0000 (15:42 -0500)]
Merge pull request #50468 from aclamk/wip-aclamk-harmonize-bluefs-log
os/bluestore: BlueFS: harmonize log read and writes modes
Mark Kogan [Tue, 21 Mar 2023 16:30:12 +0000 (18:30 +0200)]
rgw : fix python script using s3cmd with error code 403 w/https
Fixes: https://tracker.ceph.com/issues/54104
Signed-off-by: Mark Kogan <mkogan@redhat.com>
Zac Dover [Tue, 21 Mar 2023 12:27:15 +0000 (22:27 +1000)]
doc/rados: line-edit erasure-code.rst
Line-edit doc/rados/operations/erasure-code.rst.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Divyansh Kamboj [Fri, 17 Mar 2023 09:19:14 +0000 (14:49 +0530)]
rbd-mirror: add and rename perf counters for journal and snapshot mirroring
This commit renames the existing snapshot and journal based performance
counters in the rbd-mirror daemon to better reflect their purpose.
Additionally, new perf counters have been added to provide more detailed
information about the synchronization of snapshots between the source
and destination clusters.
This commit also switches to use labels instead of having the image spec
in the name of the counter.
Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
Yingxin [Tue, 21 Mar 2023 07:55:27 +0000 (15:55 +0800)]
Merge pull request #50602 from xxhdx1985126/wip-invalid-clean-pending-extents
crimson/os/seastore: don't set INVALID extents to CLEAN when reading extents
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Xuehan Xu [Tue, 21 Mar 2023 05:52:43 +0000 (05:52 +0000)]
crimson/os/seastore: don't set INVALID extents to CLEAN when reading
extents
CLEAN_PENDING extents may be invalidated before read completes:
1. transaction A retired an laddr, which lead to a RetirePlaceHolder in
Cache
2. transaction B try to read that extent, and replace A's
RetirePlaceHolder with it;
3. transaction A commits and invalidate that extent;
4. transaction B complete reading that extent;
In this case, we shouldn't set the extent's state to CLEAN
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
zdover23 [Mon, 20 Mar 2023 00:59:21 +0000 (10:59 +1000)]
Merge pull request #50488 from zdover23/wip-doc-2023-03-13-rados-bluestore-migration-2-of-x
doc/rados: edit operations/bs-migration (2 of x)
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
zdover23 [Mon, 20 Mar 2023 00:58:55 +0000 (10:58 +1000)]
Merge pull request #50487 from zdover23/wip-doc-2023-03-12-rados-bluestore-migration
doc/rados: edit operations/bs-migration (1 of x)
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
zdover23 [Sun, 19 Mar 2023 07:56:22 +0000 (17:56 +1000)]
Merge pull request #50579 from zdover23/wip-doc-2023-03-18-rados-configuration-mon-lookup-dns
doc/rados: line edit mon-lookup-dns top matter
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Sat, 18 Mar 2023 05:27:54 +0000 (15:27 +1000)]
doc/rados: line edit mon-lookup-dns top matter
Improve the syntax of the top matter of
doc/rados/operations/mon-lookup-dns.rst.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Casey Bodley [Fri, 17 Mar 2023 19:36:41 +0000 (15:36 -0400)]
Merge pull request #49443 from cbodley/wip-qa-supported-distros
qa/distros: add ubuntu 22 as supported distro
Reviewed-by: Yuri Weinstein <yuriw@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Rishabh Dave [Fri, 17 Mar 2023 18:23:27 +0000 (23:53 +0530)]
Merge pull request #50498 from rishabh-d-dave/fs-qa-caps-helper-minor
qa/cephfs: minor corrections in caps_helper.py
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Yingxin [Fri, 17 Mar 2023 14:07:40 +0000 (22:07 +0800)]
Merge pull request #50556 from aravind-wdc/fix-header
crimson: fix build error
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Ilya Dryomov [Fri, 17 Mar 2023 12:30:12 +0000 (13:30 +0100)]
Merge pull request #50559 from petrutlucian94/improve_win32_clock_check
common: avoid redefining clock type on Windows
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Venky Shankar [Fri, 17 Mar 2023 08:55:42 +0000 (14:25 +0530)]
Merge PR #50544 into main
* refs/pull/50544/head:
test/libcephfs: skip flaky timestamp assertion on Windows
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Nizamudeen A [Fri, 17 Mar 2023 08:31:29 +0000 (14:01 +0530)]
Merge pull request #50106 from rhcs-dashboard/new-landing-page-and-toggle-button
mgr/dashboard: add button toggle to switch between landing pages
Lucian Petrut [Thu, 16 Mar 2023 14:25:50 +0000 (14:25 +0000)]
common: avoid redefining clock type on Windows
mingw >= 8.0.1 defines CLOCK_REALTIME_COARSE, so we'll avoid
overriding it if already set.
Clock precision [1]:
mingw < 8.0.1:
* CLOCK_REALTIME: ~10-55ms (GetSystemTimeAsFileTime)
mingw >= 8.0.1:
* CLOCK_REALTIME: <1us (GetSystemTimePreciseAsFileTime)
* CLOCK_REALTIME_COARSE: ~10-55ms (GetSystemTimeAsFileTime)
* CLOCK_MONOTONIC: <1us if TSC is usable, ~10-55ms otherwise
(QueryPerformanceCounter)
[1] https://github.com/mirror/mingw-w64/commit/
dcd990ed423381cf35702df9495d44f1979ebe50
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
zdover23 [Fri, 17 Mar 2023 05:54:32 +0000 (15:54 +1000)]
Merge pull request #50563 from zdover23/wip-doc-2023-03-17-rados-operations-upmap
doc/rados: line-edit upmap.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Thu, 16 Mar 2023 17:41:08 +0000 (03:41 +1000)]
doc/rados: line-edit upmap.rst
Edit all of doc/rados/upmap.rst.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Anthony D'Atri [Thu, 16 Mar 2023 22:32:42 +0000 (18:32 -0400)]
Merge pull request #50564 from Thingee/update-foundation-202303
docs: Update Foundation members for March 2023
Mike Perez [Thu, 16 Mar 2023 22:23:55 +0000 (15:23 -0700)]
docs: Update Foundation members for March 2023
Signed-off-by: Mike Perez <thingee@gmail.com>
Aravind Ramesh [Thu, 16 Mar 2023 07:02:39 +0000 (12:32 +0530)]
crimson: fix build error
With -DWITH_SEASTAR=ON, build is failing.
src/crimson/osd/osd_operations/snaptrim_event.cc:174:29: error: ‘sleep’ is not a member of ‘seastar’
Signed-off-by: Aravind Ramesh <Aravind.Ramesh@wdc.com>
Lucian Petrut [Wed, 15 Mar 2023 09:04:40 +0000 (09:04 +0000)]
test/libcephfs: skip flaky timestamp assertion on Windows
There's a new libcephfs test that creates a snapshot and
compares ctime/mtime. The issue is that one of the assertion
fails on Windows, potentially due to reduced timestamp
precision.
For now, we'll skip those assertions on Windows.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Pedro Gonzalez Gomez [Tue, 14 Feb 2023 10:21:57 +0000 (11:21 +0100)]
mgr/dashboard: add button toggle to switch between new and old landing page
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Neha Ojha [Wed, 15 Mar 2023 19:50:42 +0000 (12:50 -0700)]
Merge pull request #50534 from neha-ojha/wip-release-cl
doc/dev/release-checklists.rst: add after dev freeze section
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Matan Breizman [Wed, 15 Mar 2023 16:04:42 +0000 (16:04 +0000)]
test/librados/snapshots_cxx: Enable OrderSnap for Crimson
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Matan Breizman [Wed, 15 Mar 2023 15:52:16 +0000 (15:52 +0000)]
crimson/osd/osd_operations/client_request: Support librados::OPERATION_ORDERSNAP
Signed-off-by: Matan Breizman <mbreizma@redhat.com>