]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Ronen Friedman [Tue, 19 Sep 2023 12:16:03 +0000 (07:16 -0500)]
osd/scrub: handle configuration changes in OsdScrub
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Tue, 19 Sep 2023 11:55:25 +0000 (06:55 -0500)]
osd/scrub: move initiate_a_scrub() to OsdScrub
Scrub initiation is now fully owned by OsdScrub.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Mon, 18 Sep 2023 16:31:37 +0000 (11:31 -0500)]
osd/scrub: add ScrubQueue::ready_to_scrub()
At this phase of the refactoring:
this is the main interface from the scrub scheduler in OsdScrub
to the ScrubQueue. The ScrubQueue provides the ordered list of
all targets (for now - PGs) that are ready for scrubbing.
Scrub initiation code is modified to use the new interface.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Mon, 18 Sep 2023 15:11:21 +0000 (10:11 -0500)]
osd/scrub: move scrub_sleep_time() to OsdScrub
also scrub_time_permit().
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Mon, 18 Sep 2023 11:00:34 +0000 (06:00 -0500)]
osd/scrub: extract restrictions_on_scrubbing()
from ScrubQueue::select_pg_and_scrub().
Clearing the path to moving some ScrubQueue methods into
OscScrub. Starting here with the CPU load tracker.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Sat, 16 Sep 2023 16:53:35 +0000 (11:53 -0500)]
osd/scrub: introduce OsdService::get_locked_pg()
which returns an RAII-wrapper around a locked PG.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Fri, 15 Sep 2023 14:03:09 +0000 (09:03 -0500)]
osd/scrub: move OSD::sched_scrub() to OsdScrub
... (as OsdScrub::initiate_scrub()).
The random backoff dice roller (scrub_random_backoff())
is moved as well.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Thu, 14 Sep 2023 14:09:43 +0000 (09:09 -0500)]
osd/scrub: declare OsdScrub, an OSD subobject
for all OSD scrub things.
For now: OsdScrub is mostly a forwarder to the ScrubQueue object
(which it now owns).
The resource counters moved into a separate object within OsdScrub.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Wed, 13 Sep 2023 07:09:14 +0000 (02:09 -0500)]
osd/scrub: unify dout macros across scrub scheduling code
to facilitate easy migration of code fragments between
related classes.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Mon, 11 Sep 2023 06:56:33 +0000 (01:56 -0500)]
osd/scrub: complete ScrubJob transition from within ScrubQueue
ScrubJob is now in the Scrub namespace.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Sun, 10 Sep 2023 19:44:33 +0000 (14:44 -0500)]
osd/scrub: set_reserving_now() signature modified
set_reserving_now() can now return a failure status, indicating
a race between two PGs to start scrubbing on the same OSD.
The scrubber FSM is modified to handle the failure.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Sun, 10 Sep 2023 18:50:35 +0000 (13:50 -0500)]
osd/scrub: select_pg_and_scrub() moved into osd_scrub.cc
No code changes.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Sun, 10 Sep 2023 18:32:53 +0000 (13:32 -0500)]
osd/scrub: renaming & fmt support for restrictions structure
Renaming ScrubPreconds, the collection of "environmental"
restrictions on possible scrubs, to OSDRestrictions.
Also - providing fmtlib support for that structure.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Sun, 10 Sep 2023 18:07:31 +0000 (13:07 -0500)]
osd/scrub: introducing random_bool_with_probability()
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Sun, 10 Sep 2023 17:55:59 +0000 (12:55 -0500)]
osd/scrub: moving the resources counters code into a separate file
No code changes in this commit.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Sun, 10 Sep 2023 15:14:39 +0000 (10:14 -0500)]
osd/scrub: moving code as-is into osd_scrub.cc
Code from OSD.cc & osd_scrub_sched.cc moved as-is,
to be modified in followup commits.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Ronen Friedman [Sat, 9 Sep 2023 19:44:00 +0000 (14:44 -0500)]
osd/scrub: moving ScrubJob declaration as-is
from osd_scrub_sched.h into scrub_job.h.
A purely mechanical step. No code is changed.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Kamoltat (Junior) Sirivadhna [Tue, 19 Sep 2023 18:12:35 +0000 (14:12 -0400)]
Merge pull request #51921 from kamoltat/wip-ksirivad-fix-54136
pybind/mgr/pg_autoscaler: Use bytes_used for actual_raw_used
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
Ilya Dryomov [Tue, 19 Sep 2023 14:35:18 +0000 (16:35 +0200)]
Merge pull request #53532 from zdover23/wip-doc-2023-09-19-man-ceph-monstore-tool
doc/man: s/kvstore-tool/monstore-tool/
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Zac Dover [Tue, 19 Sep 2023 13:12:34 +0000 (23:12 +1000)]
doc/man: s/kvstore-tool/monstore-tool/
s/kvstore-tool/monstore-tool/ in accordance with Ilya's remark here:
https://github.com/ceph/ceph/pull/53450#discussion_r1329804085
Signed-off-by: Zac Dover <zac.dover@proton.me>
Nizamudeen A [Tue, 19 Sep 2023 12:38:29 +0000 (18:08 +0530)]
Merge pull request #53411 from rhcs-dashboard/align-charts
mgr/dashboard: align charts of landing page
Reviewed-by: Nizamudeen A <nia@redhat.com>
Yingxin [Tue, 19 Sep 2023 08:03:34 +0000 (16:03 +0800)]
Merge pull request #53230 from myoungwon/fix-cbj-overflow-bug
crimson/os/seastore/cbj: fix a potential overflow bug on segment_seq
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Milind Changire [Tue, 19 Sep 2023 07:51:09 +0000 (13:21 +0530)]
Merge PR #52631 into main
* refs/pull/52631/head:
mds: add debug logs to monitor ceph.dir.subvolume management
mds: dump subvolume flag for inode
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Milind Changire [Tue, 19 Sep 2023 07:23:29 +0000 (12:53 +0530)]
Merge PR #52892 into main
* refs/pull/52892/head:
qa: add test to validate periodic checks by async threads
mgr/volumes: periodically check for async work
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Milind Changire [Tue, 19 Sep 2023 07:22:48 +0000 (12:52 +0530)]
Merge PR #52755 into main
* refs/pull/52755/head:
mds: adjust pre_segments_size for MDLog when trimming segments for standby-replay
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Milind Changire [Tue, 19 Sep 2023 07:21:52 +0000 (12:51 +0530)]
Merge PR #52692 into main
* refs/pull/52692/head:
qa/tasks/cephfs: reset the client_inject_fixed_oldest_tid after test
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
zdover23 [Mon, 18 Sep 2023 23:04:02 +0000 (09:04 +1000)]
Merge pull request #53518 from likid0/patch-2
doc/dev: Fix typos in cephfs-mirroring.rst and deduplication.rst
Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Daniel Parkes [Mon, 18 Sep 2023 21:03:28 +0000 (23:03 +0200)]
doc/dev: Fix typos in files cephfs-mirroring.rst and deduplication.rst
Typo Error in Doc cephfs-mirroring.rst , replace RAODS with RADOS
Typo Error in Doc deduplication.rst , replace RAODS with RADOS
Signed-off-by: Daniel Parkes <dparkes@redhat.com>
Casey Bodley [Mon, 18 Sep 2023 22:01:31 +0000 (23:01 +0100)]
Merge pull request #53502 from dang/wip-dang-cls-test
RGW - Fix cls test build on new gcc
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Daniel Gryniewicz [Mon, 18 Sep 2023 18:36:58 +0000 (14:36 -0400)]
Merge pull request #53508 from dang/wip-dang-posix-cache
RGW - Add wait backoff to posix bucket cache test
Reviewed-by Matt Benjamin <mbenjamin@redhat.com>
Daniel Gryniewicz [Mon, 18 Sep 2023 15:47:54 +0000 (11:47 -0400)]
RGW - Add wait backoff to posix bucket cache test
The CI appears to be really slow, and even a second of wait for inotify
sometimes fails. Add an exponential backoff wait of up to ~25 seconds
to hopefully make the test pass reliably.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Casey Bodley [Mon, 18 Sep 2023 13:46:55 +0000 (14:46 +0100)]
Merge pull request #53478 from EdwardVitor/cuiming_chinamobile
auth:rectify a cmake compilation warning
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Daniel Gryniewicz [Mon, 18 Sep 2023 12:50:53 +0000 (08:50 -0400)]
RGW - Fix cls test build on new gcc
The new encoder types broke building the cls test on newer gcc (13+) due
to undefined encoder/decoder. Add the file that defines those to the
test.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Patrick Donnelly [Mon, 18 Sep 2023 12:37:16 +0000 (08:37 -0400)]
Merge PR #53175 into main
* refs/pull/53175/head:
qa: increase the http postBuffer size and disable sslVerify
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
cuiming_yewu [Fri, 15 Sep 2023 06:14:39 +0000 (14:14 +0800)]
auth:rectify a cmake compilation warning
rectify src/auth/cephx/CephxProtocol.cc 1 warning
with the variable 'ch' Used before initialized
auth/cephx/CephxProtocol.cc:595:57: warning: '*((void*)& ch +8)' may be used uninitialized in this function [-Wmaybe-uninitialized]
msg.server_challenge_plus_one = ch.server_challenge + 1;
~~~~~~~~~~~~~~~~~~~~^~~
Signed-off-by: cuiming <cuiming_yewu@cmss.chinamobile.com>
Anthony D'Atri [Sun, 17 Sep 2023 14:40:26 +0000 (10:40 -0400)]
Merge pull request #53490 from zdover23/wip-doc-2023-09-17-architecture-6-of-x
doc/architecture: "Edit HA Auth" (one of several)
Zac Dover [Sun, 17 Sep 2023 08:56:40 +0000 (18:56 +1000)]
doc/architecture: "Edit HA Auth" (one of several)
Edit "High Availability Authentication" in doc/architecture.rst.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Anthony D'Atri [Sat, 16 Sep 2023 16:46:58 +0000 (12:46 -0400)]
Merge pull request #53487 from zdover23/wip-doc-2023-09-16-architecture-5-of-x
doc/architecture: Edit "HA Auth"
Zac Dover [Sat, 16 Sep 2023 12:27:29 +0000 (22:27 +1000)]
doc/architecture: Edit "HA Auth"
Edit "High Availability Authentication" in doc/architecture.rst.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Patrick Donnelly [Fri, 15 Sep 2023 18:38:02 +0000 (14:38 -0400)]
Merge PR #52638 into main
* refs/pull/52638/head:
mds: flush monc log before abort
qa: check for expected cluster log message
qa: ignore expected cluster warning from damage tests
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Patrick Donnelly [Fri, 15 Sep 2023 16:11:37 +0000 (12:11 -0400)]
Merge PR #52199 into main
* refs/pull/52199/head:
mds: continue linking if targeti is temporarily located in stray dir
Revert "mds: wait unlink to finish to avoid conflict when creating same dentries"
Revert "mds: clear the STATE_UNLINKING state when the unlink fails"
Revert "mds: wait reintegrate to finish when unlinking"
Revert "mds: notify the waiters in replica MDSs"
Revert "mds: wait the linkmerge/migrate to finish after unlink"
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Daniel Gryniewicz [Fri, 15 Sep 2023 12:58:52 +0000 (08:58 -0400)]
Merge pull request #52933 from dang/wip-dang-posix-driver
RGW - Add POSIX Driver
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Adam King [Thu, 14 Sep 2023 17:10:16 +0000 (13:10 -0400)]
Merge pull request #53296 from phlogistonjohn/jjm-fix-git-ls-files
cephadm: remove duplicates when listing files in scan
Reviewed-by: Adam King <adking@redhat.com>
Daniel Gryniewicz [Tue, 24 Jan 2023 15:02:07 +0000 (10:02 -0500)]
RGW - Add POSIX Driver
This is the MVP for a driver for RGW that operates on top of a POSIX
filesystem. It supports get, put, list, copy, multipart, external
access via the filesystem itself, and ordered bucket listings via an
LRU-based cache.
Note that this is currently a Filter, indended to run on top of dbstore.
This is because it currently doesn't have any User implementation, so it
depends on dbstore's User. Everything else is implemented in
POSIXDriver. Once there is a User implementation, this will become a
Store, instead of a Filter.
Commit messages from bucket listing cache:
rgw/posixdriver: recycle lmdb database handles as required
While LMDB workflows often do not close/return database handles,
ours continually reuses them. This requires us to close each
handle (atomically) when a cache entry is recycled.
rgw/posixdriver: don't instantiate bucket cache entries from notify events
rgw/posixdriver: incorporate lmdb-safe for now
The current inclusion is based on https://github.com/Martchus/lmdb-safe,
which is actively maintained but currently has some packaging issues the
author has agreed to accept fixes for.
For now, skip the submodule to save time and remove an external dependency.
rgw/posixdriver: fix listing of cached, empty bucket
* check lmdb enumeration result in all cases and w/better style
* add unit test for enumeration of an empty cached directory
rgw/posixdriver: nest lmdbs in a directory under the dbroot path to avoid cleanup issues
rgw/posixdriver: refactor for posix integration
* Derive BucketCache types as templates on a SAL driver and SAL
bucket pair.
* Integrate cache fills as callbacks into SAL layer (or mock, for
tests)
* Renaming and cleanups
rgw/posixdriver: add bucket cache implementation and tests
Adds free-standing cache of buckets and object names, with
bucket names (and listing attributes, upcoming) managed in
a hashed set of lmdb databases, which provides ordering and
a high-performance listing cache.
An framework for notification on new object creation (e.g.,
outside S3 workflow) is provided, and a Linux implementation
using inotify.
FindLMDB.cmake taken with attribution and license.
rgw/posixdriver: add zpp_bits serialization (FAST)
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam King [Thu, 14 Sep 2023 15:23:38 +0000 (11:23 -0400)]
Merge pull request #52690 from phlogistonjohn/jjm-cephadm-daemon-ident
cephadm: refactor code to make use of DaemonIdentity type
Reviewed-by: Adam King <adking@redhat.com>
zdover23 [Thu, 14 Sep 2023 11:44:15 +0000 (21:44 +1000)]
Merge pull request #53450 from zdover23/wip-doc-2023-09-14-man-ceph-monstore-tool
doc/man: edit ceph-monstore-tool.rst
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Ronen Friedman [Thu, 14 Sep 2023 10:33:46 +0000 (13:33 +0300)]
Merge pull request #53426 from ronen-fr/wip-rf-fshut
osd: cleaning stop_for_fast_shutdown()
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Venky Shankar [Thu, 14 Sep 2023 10:09:07 +0000 (15:39 +0530)]
Merge PR #53070 into main
* refs/pull/53070/head:
PendingReleaseNotes: add note about 'm' and 'M' periods
doc/cephfs: add note about monthly scheduling and retention period
qa: update minute period multiplier to m
qa: change minute duration specifier from M to m
mgr/snap_schedule: change minute duration specifier from M to m
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yingxin [Thu, 14 Sep 2023 07:15:24 +0000 (15:15 +0800)]
Merge pull request #53433 from myoungwon/fix-53232
crimson/tools/perf_staged_fltree: fix compile error
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Aravind Ramesh <aravind.ramesh@wdc.com>
Venky Shankar [Thu, 14 Sep 2023 03:59:07 +0000 (09:29 +0530)]
Merge PR #51891 into main
* refs/pull/51891/head:
qa: fix cephfs-top tests for the new fields
cephfs-top: include the missing fields in --dump output
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
zdover23 [Thu, 14 Sep 2023 03:27:52 +0000 (13:27 +1000)]
Merge pull request #53444 from zdover23/wip-doc-2023-09-14-architecture-4-of-x
doc/architecture: edit "High Avail. Monitors"
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Zac Dover [Thu, 14 Sep 2023 03:24:11 +0000 (13:24 +1000)]
doc/man: edit ceph-monstore-tool.rst
Clean up the English in doc/man/8/ceph-monstore-tool.rst. Put all
commands in the imperative mood; make sentences more direct.
Follows https://github.com/ceph/ceph/pull/52860.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Zac Dover [Wed, 13 Sep 2023 14:09:45 +0000 (00:09 +1000)]
doc/architecture: edit "High Avail. Monitors"
Improve the sentence structure in the "High Availability Monitors"
section of doc/architecture.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
Radoslaw Zarzynski [Wed, 13 Sep 2023 19:45:47 +0000 (21:45 +0200)]
Merge pull request #53178 from aclamk/wip-aclamk-bs-esb-5-enabled
bluestore: Elastic Shared Blobs 5 - enabled, final
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com
John Mulligan [Wed, 13 Sep 2023 19:44:40 +0000 (15:44 -0400)]
cephadm: fix an error running iscsi services
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Matt Benjamin [Wed, 13 Sep 2023 16:57:52 +0000 (12:57 -0400)]
Merge pull request #52793 from linuxbox2/wip-fix-test-librgw-xattr
rgw/file: fix build of librgw_file_xattr.cc unit test
Daniel Gryniewicz [Wed, 13 Sep 2023 13:35:39 +0000 (09:35 -0400)]
Merge pull request #51368 from dang/wip-dang-zipper-motr
Fix MOTR build, and move MOTR and DAOS to proper location
Ali Maredia [Wed, 13 Sep 2023 11:46:19 +0000 (07:46 -0400)]
Merge pull request #53391 from shreyanshjain7174/bucket_quota
RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage
Reviewed-by: Ali Maredia <amaredia@redhat.com>
Myoungwon Oh [Wed, 13 Sep 2023 05:16:16 +0000 (14:16 +0900)]
crimson/tools/perf_staged_fltree: fix compile error
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
zdover23 [Wed, 13 Sep 2023 07:07:26 +0000 (17:07 +1000)]
Merge pull request #53423 from zdover23/wip-doc-2023-09-12-architecture-3-of-x
doc/architecture: edit "Cluster Map"
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Venky Shankar [Wed, 13 Sep 2023 05:40:59 +0000 (11:10 +0530)]
Merge PR #53153 into main
* refs/pull/53153/head:
Revert "mds: align quota.max_bytes to 4MB or 4KB"
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Casey Bodley [Tue, 12 Sep 2023 18:27:00 +0000 (19:27 +0100)]
Merge pull request #53370 from wowly/rgw-cache-nginx-if-match-header
examples: fixed errors with the if-match header in the examples of rgw-cache nginx configs
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Zac Dover [Tue, 12 Sep 2023 11:17:40 +0000 (21:17 +1000)]
doc/architecture: edit "Cluster Map"
Edit the section "Cluster Map" in doc/architecture.rst.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
Anthony D'Atri [Tue, 12 Sep 2023 16:00:08 +0000 (12:00 -0400)]
Merge pull request #53422 from zdover23/wip-doc-2023-09-12-glossary-primary-affinity
doc/glossary: add "primary affinity" to glossary
Ronen Friedman [Tue, 12 Sep 2023 13:15:40 +0000 (08:15 -0500)]
osd: cleaning stop_for_fast_shutdown()
Removed unsued variables to prevent compiler warnings.
Protected the shard lock.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Adam Kupczyk [Tue, 12 Sep 2023 15:47:52 +0000 (17:47 +0200)]
Merge pull request #52407 from rhcs-dashboard/osd-op-scraper
os/bluestore: osd scrape read/write operations
Pere Diaz Bou [Wed, 12 Jul 2023 11:05:17 +0000 (13:05 +0200)]
os/bluestore: osd scrape read/write operations
To run:
```bash
../src/test/objectstore/allocsim/ops_scraper.py --ceph_bin_path bin/ceph
```
Example:
```bash
$ ../src/test/objectstore/allocsim/ops_scraper.py --ceph_bin_path bin/ceph [13:06:39]
collecting from osd. 0
op_type | offset/extent | name | pgid | initiated_at | who
-------------------------------------------------------
write 0~
4194304 4:
c4b97d2c :::
be151868021c187609fb97e4c22ae1f9 :head 4.3 2023-07-12T13:06:34.959082+0200 client.4831.0:351
write 0~
4194304 4:
d392075e :::
b068dce954dca8ca56d72e02e87f39c8 :head 4.b 2023-07-12T13:06:35.093439+0200 client.4831.0:358
write 0~
4194304 4:
bc1ac643 :::
7b815bd25a40695c891dc06bdf2b65be :head 4.1d 2023-07-12T13:06:35.128097+0200 client.4831.0:360
write 0~
4194304 4:
d0d21bd5 :::
cc531199dd2707370bb461cce82bdef6 :head 4.b 2023-07-12T13:06:36.023082+0200 client.4831.0:367
write 0~
4194304 4:
a9abc92c :::
0703d6f43d14bc33d69f94ecb96c522c :head 4.15 2023-07-12T13:06:36.367886+0200 client.4831.0:369
write 0~
4194304 4:
9b4684ea :::
d5856dc0ecad96f811012e0937d5e0ac :head 4.19 2023-07-12T13:06:36.654584+0200 client.4831.0:370
write 0~
4194304 4:
d15b8c87 :::
c3344e5de8c8936876965d9fb056ad89 :head 4.b 2023-07-12T13:06:36.835690+0200 client.4831.0:371
write 0~
4194304 4:
ad41de2a :::
8a1e2743b577a67d6bc8a1d514391b53 :head 4.15 2023-07-12T13:06:37.102741+0200 client.4831.0:373
write 0~
4194304 4:
fc35d54f :::
39cd961e7e33680f4bfd1ca0fe43dda4 :head 4.1f 2023-07-12T13:06:38.047612+0200 client.4831.0:377
collecting from osd. 1
op_type | offset/extent | name | pgid | initiated_at | who
-------------------------------------------------------
write 0~
4194304 4:
4c9ed2e8 :::
86206ccc35d8aceaf78670a751a6b550 :head 4.12 2023-07-12T13:06:34.771423+0200 client.4831.0:340
write 0~
4194304 4:
106d9524 :::
fe7534cc604b0eb50498cb23aaca9161 :head 4.8 2023-07-12T13:06:35.128105+0200 client.4831.0:359
write 0~
4194304 4:
a6e3fdb1 :::
2692491bfc4c584ca5b1efcb896936c2 :head 4.5 2023-07-12T13:06:35.567070+0200 client.4831.0:363
write 0~
4194304 4:
2d9a471a :::
ba51255646d00c496c2638bad03e0869 :head 4.14 2023-07-12T13:06:35.568290+0200 client.4831.0:364
write 0~
4194304 4:
2b1381b1 :::
c58fa31e5d3a4f5550ee143bcb07a49f :head 4.14 2023-07-12T13:06:35.764699+0200 client.4831.0:365
write 0~
4194304 4:
a22e158f :::
b2116e08700da18595b96d58120fe81c :head 4.5 2023-07-12T13:06:36.023101+0200 client.4831.0:366
```
Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
Ali Masarwa [Tue, 12 Sep 2023 13:51:25 +0000 (16:51 +0300)]
Merge pull request #53419 from AliMasarweh/wip-alimasa-persistent-q-test
RGW | bucket notification: adding more sleeping time to prevent test from failing
Reviewed-by: yuvalif <ylifshit@redhat.com>
Jos Collin [Fri, 2 Jun 2023 11:09:49 +0000 (16:39 +0530)]
qa: fix cephfs-top tests for the new fields
Fixes: https://tracker.ceph.com/issues/61397
Signed-off-by: Jos Collin <jcollin@redhat.com>
Jos Collin [Fri, 2 Jun 2023 11:08:29 +0000 (16:38 +0530)]
cephfs-top: include the missing fields in --dump output
Fixes: https://tracker.ceph.com/issues/61397
Signed-off-by: Jos Collin <jcollin@redhat.com>
Zac Dover [Tue, 12 Sep 2023 10:56:03 +0000 (20:56 +1000)]
doc/glossary: add "primary affinity" to glossary
Add "Primary Affinity" to the glossary and link to the information in
the RADOS documentation that (more or less) explains it.
Signed-off-by: Zac Dover <zac.dover@proton.me>
Ali Masarwa [Tue, 12 Sep 2023 10:31:31 +0000 (13:31 +0300)]
RGW | bucket notification: adding more sleeping time to prevent test from failing
Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
Pedro Gonzalez Gomez [Tue, 12 Sep 2023 08:08:38 +0000 (10:08 +0200)]
mgr/dashboard: align charts of landing page
Fixes: https://tracker.ceph.com/issues/62813
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Milind Changire [Tue, 12 Sep 2023 08:39:39 +0000 (14:09 +0530)]
PendingReleaseNotes: add note about 'm' and 'M' periods
Signed-off-by: Milind Changire <mchangir@redhat.com>
Milind Changire [Thu, 24 Aug 2023 02:19:43 +0000 (07:49 +0530)]
doc/cephfs: add note about monthly scheduling and retention period
Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
Milind Changire [Tue, 12 Sep 2023 08:30:33 +0000 (14:00 +0530)]
qa: update minute period multiplier to m
Signed-off-by: Milind Changire <mchangir@redhat.com>
Milind Changire [Thu, 24 Aug 2023 02:18:17 +0000 (07:48 +0530)]
qa: change minute duration specifier from M to m
Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
Milind Changire [Thu, 24 Aug 2023 02:17:46 +0000 (07:47 +0530)]
mgr/snap_schedule: change minute duration specifier from M to m
Problem:
As per the issue tracker, the period spec specifier 'M' is not
consistent with what is used elsewhere, like the period specifiers
displayed in the 'ceph status' command output.
The 'M' period specifier is used as a 'minute' level period specifier by
the cephfs team.
The issue reporter suggests to use 'M' as a 'month' period specifier.
Solution:
Since the 'minute' level period specifer, 'M', is used internally by
the development team, it is failrly easy to swap the 'minute' ('M')
level and 'month' ('m') level period specifers to finally mean that 'm'
implies 'minute' level period and 'M' implies 'month' level period.
Also, since this is the first time that somebody has ever reported that
neither the 'M' nor the 'm' level specifiers work in production, it is a
good idea to fix them once and for all.
Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
Roy Sahar [Tue, 12 Sep 2023 07:43:24 +0000 (10:43 +0300)]
Merge pull request #53408 from roysahar/nvmeof_fix_omap_state_object_contains_None
nvmeof-gw: omap object name contains None as string due to generated template to ceph-nvmeof.conf
Liu-Chunmei [Tue, 12 Sep 2023 06:25:24 +0000 (23:25 -0700)]
Merge pull request #53232 from myoungwon/wip-enable-rbm-tests
test/crimson/seastore/rbm: add sub-tests regarding RBM to the existing tests
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
Myoungwon Oh [Fri, 25 Aug 2023 08:55:36 +0000 (17:55 +0900)]
crimson/os/seastore/object_data_handler: consider a RBM case when checking if write can be merged
RBM's paddr always indicates physical address, which means it doesn't have the dealayed.
So, this commit adds a condition that checks if given paddr is used for ongoing write.
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Myoungwon Oh [Thu, 31 Aug 2023 02:43:24 +0000 (11:43 +0900)]
crimson/os/seastore/cbj: fix a potential overflow bug
CircularBoundedJournal is similar to circular queue, so segment_seq
increases after rolling. However, current implementation always
increases segment_seq_t when rolling occurs, resulting in
the overflow if segment_seq_t hits MAX_SEG_SEQ.
To mitigate this, this commit changes the type of the segment_seq_t
to uint64_t.
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Xiubo Li [Wed, 30 Aug 2023 05:24:01 +0000 (13:24 +0800)]
mds: continue linking if targeti is temporarily located in stray dir
If one inode has more than one hardlink and after the primary dentry
is unlinked it will located the inode in the stray dir temporarily,
which is pending reintegration.
Just before the linkmerge/migrate is triggered a link request comes
it will fail with -EXDEV.
Just skip it and continue the linking.
Fixes: https://tracker.ceph.com/issues/56695
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Tue, 12 Sep 2023 02:25:35 +0000 (10:25 +0800)]
Revert "mds: wait unlink to finish to avoid conflict when creating same dentries"
This reverts commit
d4b9431dfe23a97075766fa2a2b76d7554543b0c .
Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Tue, 12 Sep 2023 02:24:59 +0000 (10:24 +0800)]
Revert "mds: clear the STATE_UNLINKING state when the unlink fails"
This reverts commit
417f24782cfcf45827dff74adf8df2283b2a2c72 .
Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Wed, 30 Aug 2023 04:56:59 +0000 (12:56 +0800)]
Revert "mds: wait reintegrate to finish when unlinking"
This reverts commit
c3b36723defae32b0a3ef1ac37b58ec57d356b65 .
Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Wed, 30 Aug 2023 04:55:10 +0000 (12:55 +0800)]
Revert "mds: notify the waiters in replica MDSs"
This reverts commit
478db14a658c39ecbfa107923d9ad44bfdd89368 .
Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Xiubo Li [Wed, 30 Aug 2023 04:51:57 +0000 (12:51 +0800)]
Revert "mds: wait the linkmerge/migrate to finish after unlink"
This reverts commit
48f9a8990b402daa0f0f9109bc4b0087c01b33c6 .
Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Yingxin [Tue, 12 Sep 2023 01:21:29 +0000 (09:21 +0800)]
Merge pull request #53305 from xxhdx1985126/wip-seastore-onode-erase-replay
crimson/os/seastore/onode_manager: populate value recorders of onodes to be erased
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Roy Sahar [Mon, 11 Sep 2023 22:52:53 +0000 (01:52 +0300)]
nvmeof-gw: omap object name contains None due to generated template contains string None
Signed-off-by: Roy Sahar <royswi@gmail.com>
Laura Flores [Mon, 11 Sep 2023 22:22:03 +0000 (18:22 -0400)]
Merge pull request #53344 from ljflores/wip-tracker-62761
common: add CephContext parameter to tracing::Tracer::init() in !HAVE_JAEGER branch
John Mulligan [Thu, 27 Jul 2023 00:08:23 +0000 (20:08 -0400)]
cephadm: convert _write_iscsi_unit_poststop_commands to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 27 Jul 2023 00:05:03 +0000 (20:05 -0400)]
cephadm: convert _write_osd_unit_poststop_commands to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 27 Jul 2023 00:01:26 +0000 (20:01 -0400)]
cephadm: convert _write_iscsi_unit_run_commands to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 27 Jul 2023 00:00:01 +0000 (20:00 -0400)]
cephadm: convert _write_osd_unit_run_commands to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 26 Jul 2023 23:53:58 +0000 (19:53 -0400)]
cephadm: convert _dispatch_deploy to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 26 Jul 2023 23:50:16 +0000 (19:50 -0400)]
cephadm: convert get_deployment_type to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 26 Jul 2023 23:40:30 +0000 (19:40 -0400)]
cephadm: convert get_deployment_container to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 26 Jul 2023 23:26:47 +0000 (19:26 -0400)]
cephadm: convert get_container to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 26 Jul 2023 23:25:07 +0000 (19:25 -0400)]
cephadm: add daemon_name property to DaemonIdentity
Add a convenience property to DaemonIdentity and DaemonSubIdentity that
produces a name in the ceph daemon naming convention: <type>.<id>.
Signed-off-by: John Mulligan <jmulligan@redhat.com>