]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Matt Benjamin [Thu, 16 Sep 2021 13:57:06 +0000 (09:57 -0400)]
rgw:adminops: add test case for 'info' section
Add 'info' section test case to the radosgw_admin_test.py qa
task.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Tue, 29 Jun 2021 17:54:35 +0000 (13:54 -0400)]
rgw: introduce new /{admin}/info api resource
The new resource returns an array of informational data, which
currently consists of the RADOS cluster fsid as "cluster_fsid."
Fixes: https://tracker.ceph.com/issues/51432
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Tue, 29 Jun 2021 13:23:40 +0000 (09:23 -0400)]
rgw:sal: expose cluster fsid in RGWStore
Adds a get_cluster_id() method to RGWStore, provided librados
cluster_fsid in the RADOS implementation.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Alfonso Martínez [Thu, 9 Dec 2021 15:27:03 +0000 (16:27 +0100)]
Merge pull request #44001 from votdev/issue_53317_fix_proxy_conf
mgr/dashboard: API docs UI does not work with Angular dev server
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Matt Benjamin [Thu, 9 Dec 2021 15:05:16 +0000 (10:05 -0500)]
Merge pull request #44237 from pritha-srivastava/wip-rgw-abort-multipart-bypass-gc
rgw: deleting objects inline in case bypass_gc is specified
Samuel Just [Thu, 9 Dec 2021 07:51:30 +0000 (23:51 -0800)]
Merge pull request #44127 from cyx1231st/wip-seastore-merge-journal-headers-2
crimson/os/seastore: make JournalSubmitter aware of header merging
Reviewed-by: Samuel Just <sjust@redhat.com>
Yingxin Cheng [Mon, 29 Nov 2021 02:17:05 +0000 (10:17 +0800)]
crimson/os/seastore/journal: make JournalSubmitter aware of header merging
Introduce an option seastore_journal_batch_preferred_fullness for
JournalSubmitter to control the preferred fullness to trigger batch
flushes.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Mon, 29 Nov 2021 02:11:09 +0000 (10:11 +0800)]
crimson/os/seastore: refactor get_encoded_length_after()
To return a more informative record_group_size_t.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 26 Nov 2021 05:13:56 +0000 (13:13 +0800)]
crimson/os/seastore: debug print journal write details
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Thu, 25 Nov 2021 03:06:05 +0000 (11:06 +0800)]
crimson/os/seastore: measure records write in journal
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Thu, 25 Nov 2021 01:23:53 +0000 (09:23 +0800)]
crimson/os/seastore: fix record metrics in cache
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Thu, 9 Dec 2021 03:09:27 +0000 (11:09 +0800)]
crimson/os/seastore/cache: misc cleanup
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Mon, 22 Nov 2021 08:55:02 +0000 (16:55 +0800)]
crimson/os/seastore: add debug logs to print transaction name
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Mon, 22 Nov 2021 02:55:22 +0000 (10:55 +0800)]
crimson/os/seastore: don't print detail if the extent is not ready
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Mon, 22 Nov 2021 02:51:38 +0000 (10:51 +0800)]
crimson/os/seastore: print extent length
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Mon, 22 Nov 2021 02:50:06 +0000 (10:50 +0800)]
crimson/os/seastore: move implementation to seastore_types.cc
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 19 Nov 2021 05:17:43 +0000 (13:17 +0800)]
crimson/os/seastore: merge records metadata if they are grouped
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 19 Nov 2021 05:30:06 +0000 (13:30 +0800)]
crimson/os/seastore: mark out empty transactions
TODO: avoid write if the transaction is empty.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 19 Nov 2021 05:17:16 +0000 (13:17 +0800)]
crimson/os/seastore: refactor, introduce record_t and record_group_t with sizes
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Tue, 16 Nov 2021 08:47:12 +0000 (16:47 +0800)]
crimson/os/seastore: scan records based on record_locator_t
Record may not have its own base if headers are merged.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Tue, 16 Nov 2021 08:15:42 +0000 (16:15 +0800)]
crimson/os/seastore: add more checks when read record_header_t
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Tue, 16 Nov 2021 06:23:15 +0000 (14:23 +0800)]
crimson/os/seastore: misc cleanup and reformat
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Tue, 16 Nov 2021 02:14:05 +0000 (10:14 +0800)]
crimson/os/seastore: add logs in ExtentReader
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yuri Weinstein [Wed, 8 Dec 2021 21:04:57 +0000 (13:04 -0800)]
Merge pull request #43919 from ronen-fr/wip-rf-test-nodeep
osd/scrub (& qa/standalone): test for scrub behavior when no-scrub is set but no-deep-scrub is not
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>
Yuri Weinstein [Wed, 8 Dec 2021 21:04:14 +0000 (13:04 -0800)]
Merge pull request #43305 from heylinn/ceph_rundir_sysvinit
init-ceph: create /var/run/ceph for sysvinit
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Wed, 8 Dec 2021 21:02:50 +0000 (13:02 -0800)]
Merge pull request #44090 from sseshasa/wip-fix-require-osd-release
osd/OSDMap: Add health warning if 'require-osd-release' != current release
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Samuel Just [Wed, 8 Dec 2021 20:02:45 +0000 (12:02 -0800)]
Merge pull request #44244 from cyx1231st/wip-seastore-refine-metrics
crimson/os/seastore: refine transaction metrics
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Wed, 8 Dec 2021 20:02:04 +0000 (12:02 -0800)]
Merge pull request #44242 from liu-chunmei/crimson-fix-heartbeat-addrs
crimson/osd: fix heartbeat front and back blank ip
Reviewed-by: Samuel Just <sjust@redhat.com>
Igor Fedotov [Wed, 8 Dec 2021 19:21:40 +0000 (22:21 +0300)]
Merge pull request #44098 from ifed01/wip-ifed-dump-alloc-unit
os/bluestore: dump bluestore/bluefs alloc unit sizes with perf dump
Reviewed-by: Laura Flores <lflores@redhat.com>
Igor Fedotov [Wed, 8 Dec 2021 19:19:22 +0000 (22:19 +0300)]
Merge pull request #43840 from ifed01/wip-ifed-verbose-open-col
osd,bluestore: gracefully handle a failure during meta collection load
Reviewed-by: jdurgin@redhat.com
Reviewed-by: nojha@redhat.com
Igor Fedotov [Wed, 8 Dec 2021 19:17:38 +0000 (22:17 +0300)]
Merge pull request #39691 from aclamk/wip-bs-compression-blob-64k
os/bluestore: Set new compression blob size to 64K
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
zdover23 [Wed, 8 Dec 2021 19:14:37 +0000 (05:14 +1000)]
Merge pull request #44213 from zdover23/wip-doc-2021-12-05-hardware-recommendations-removing-journal
doc/start: remove journal info from hardware recs
Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
Daniel Gryniewicz [Wed, 8 Dec 2021 16:34:41 +0000 (11:34 -0500)]
Merge pull request #44232 from Huber-ming/admin_fixlogs
radosgw-admin: fix some error logs
Daniel Gryniewicz [Wed, 8 Dec 2021 16:22:28 +0000 (11:22 -0500)]
Merge pull request #43915 from qiuxinyidian/dev-rgw
rgw: when radosgw-admin stating user, add user exists judging
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Daniel Gryniewicz [Wed, 8 Dec 2021 16:19:12 +0000 (11:19 -0500)]
Merge pull request #43834 from Huber-ming/admin_bilog
radosgw-admin: supplement help documents with 'bilog autotrim'
Zac Dover [Sun, 5 Dec 2021 14:44:09 +0000 (00:44 +1000)]
doc/start: remove journal info from hardware recs
This PR removes mentions of journaling from the hardware
recommendations.
Journaling was a FileStore-related practice. BlueStore is
the default backend for Ceph OSDs and has been since
Luminous. The documentation should reflect that.
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Sebastian Wagner [Wed, 8 Dec 2021 14:51:36 +0000 (15:51 +0100)]
Merge pull request #44093 from melissa-kun-li/ssh-non-root-user
mgr/cephadm: support bootstrap with non-root ssh-user
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Sage Weil [Wed, 8 Dec 2021 12:11:53 +0000 (07:11 -0500)]
Merge PR #44162 into master
* refs/pull/44162/head:
mgr: only queue notify events that modules ask for
pybind/mgr: annotate which events modules consume
pybind/mgr: introduce NotifyType enum
mgr: stop issuing events that no modules consume
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Sage Weil [Wed, 8 Dec 2021 12:10:21 +0000 (07:10 -0500)]
Merge PR #44196 into master
* refs/pull/44196/head:
mon/MgrStatMonitor: do not spam subscribers (mgr) with service_map
Reviewed-by: Neha Ojha <nojha@redhat.com>
Sage Weil [Wed, 8 Dec 2021 12:10:08 +0000 (07:10 -0500)]
Merge PR #44207 into master
* refs/pull/44207/head:
mgr/ActivePyModule: avoid with_gil where possible
mgr/ActivePyModules: push without_gil_t down into blocks
Reviewed-by: Kefu Chai <kchai@redhat.com>
Radoslaw Zarzynski [Wed, 8 Dec 2021 10:24:24 +0000 (11:24 +0100)]
Merge pull request #43542 from rzarzynski/wip-crimson-net-ms_learn_from_peer
crimson/net: add support for ms_learn_addr_from_peer.
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Sebastian Wagner [Wed, 8 Dec 2021 09:25:49 +0000 (10:25 +0100)]
Merge pull request #44104 from guits/guits-cephadm-skip-cv-restorecon
cephadm: pass `CEPH_VOLUME_SKIP_RESTORECON=yes`
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Sébastien Han [Wed, 8 Dec 2021 08:56:49 +0000 (09:56 +0100)]
Merge pull request #44239 from guits/guits-add-skip-needs-root
ceph-volume: make it possible to skip needs_root()
Kefu Chai [Wed, 8 Dec 2021 08:50:20 +0000 (16:50 +0800)]
Merge pull request #44245 from tchaikov/wip-crimson-seastar
seastar: pick up change to fix FTBFS with old cryptopp
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Thu, 14 Oct 2021 13:55:23 +0000 (13:55 +0000)]
crimson/net: add support for ms_learn_addr_from_peer.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Samuel Just [Wed, 8 Dec 2021 07:43:17 +0000 (23:43 -0800)]
Merge pull request #44179 from athanatos/sjust/wip-pin-race
crimson/os/seastore: initialize logical pins before exposing to cache
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Kefu Chai [Wed, 8 Dec 2021 07:30:10 +0000 (15:30 +0800)]
crimson/osd: s/seastar::fprint()/fmt::print()/
otherwise, we'd have warnings like:
./src/crimson/osd/main.cc:106:16: error: 'fprint<const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > &>' is deprecated: use std::format_to() or fmt::print() [-Werror,-Wdeprecated-declarations]
seastar::fprint(std::cerr, "already have key in keyring: %s\n", path);
^
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Kefu Chai [Wed, 8 Dec 2021 06:03:41 +0000 (14:03 +0800)]
seastar: pick up change to fix FTBFS with old cryptopp
Signed-off-by: Kefu Chai <kefu@xsky.com>
Yuval Lifshitz [Wed, 8 Dec 2021 06:05:32 +0000 (08:05 +0200)]
Merge pull request #44186 from qiuxinyidian/rgw-de
rgw: add object null point judging when listing pubsub topics
Yingxin Cheng [Tue, 7 Dec 2021 08:48:03 +0000 (16:48 +0800)]
crimson/os/seastore: measure the number of conflicting transactions by srcs
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Mon, 6 Dec 2021 08:33:47 +0000 (16:33 +0800)]
crimson/os/seastore: differentiate cleaner trim/reclaim transactions
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
chunmei-liu [Tue, 7 Dec 2021 23:31:24 +0000 (15:31 -0800)]
crimson/osd: fix heartbeat front and back blank ip
when ceph.conf not set public ip & cluster ip, heartbeat will get blank ip address. when osd::_send_boot , classic osd will check if heartbeat front and back addrs are blank ip, if they are blank ip, will use public ip which is learned from mon to set into them. So implement them in crimson osd.
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Kefu Chai [Wed, 8 Dec 2021 00:56:11 +0000 (08:56 +0800)]
Merge pull request #44147 from rzarzynski/wip-crimson-new-seastar
crimson: bump up Seastar to recent master and fix FTBFS
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Neha Ojha [Tue, 7 Dec 2021 21:34:15 +0000 (13:34 -0800)]
Merge pull request #44095 from Matan-B/wip-matanb-local-workunits
doc/dev: Running workunits locally
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuval Lifshitz [Tue, 7 Dec 2021 19:20:33 +0000 (21:20 +0200)]
Merge pull request #43940 from TRYTOBE8TME/wip-rgw-empty-config
src/rgw: Empty configuration support
Yuval Lifshitz [Tue, 7 Dec 2021 19:19:27 +0000 (21:19 +0200)]
Merge pull request #43665 from zenomri/wip-omri-multipart-trace
rgw/tracer: Multipart upload trace
Samuel Just [Tue, 7 Dec 2021 15:53:21 +0000 (07:53 -0800)]
Merge pull request #44156 from rzarzynski/wip-crimson-fix-process_op-sequencing
crimson/osd: fix sequencing issues in ClientRequest::process_op.
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Tue, 7 Dec 2021 15:52:41 +0000 (07:52 -0800)]
Merge pull request #44223 from rzarzynski/wip-crimson-fix-pullinfo-on-push
crimson/osd: don't assume a pull must happen if there is no push.
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Samuel Just [Tue, 7 Dec 2021 15:52:02 +0000 (07:52 -0800)]
Merge pull request #44224 from rzarzynski/wip-crimson-clean-msghs
crimson/osd: clean the recovery message-related header inclusion.
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Tue, 7 Dec 2021 15:49:08 +0000 (07:49 -0800)]
Merge pull request #44184 from rzarzynski/wip-crimson-internal_client_request-fix-hobj
crimson/osd: fix assertion failure in InternalClientRequest.
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Pritha Srivastava [Tue, 7 Dec 2021 11:01:25 +0000 (16:31 +0530)]
rgw: deleting objects inline in case bypass_gc is specified
for bucket remove command.
fixes: https://tracker.ceph.com/issues/53512
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
Guillaume Abrioux [Tue, 7 Dec 2021 14:18:10 +0000 (15:18 +0100)]
ceph-volume: make it possible to skip needs_root()
Add the possibility to skip the `needs_root()` decorator.
See linked tracker for details.
Fixes: https://tracker.ceph.com/issues/53511
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Alfonso Martínez [Tue, 7 Dec 2021 14:02:42 +0000 (15:02 +0100)]
Merge pull request #44145 from rhcs-dashboard/fix-frontend-vulnerabilities
mgr/dashboard: fix frontend deps' vulnerabilities
Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Radoslaw Zarzynski [Tue, 30 Nov 2021 19:30:32 +0000 (19:30 +0000)]
crimson/osd: fix sequencing issues in ClientRequest::process_op.
The following crash has been observed in one of the runs at Sepia:
```
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/
17.0.0-8898-ge57ad63c /rpm/el8/BUILD/
ceph-17.0.0-8898-ge57ad63c /src/crimson/osd/osd_operation_sequencer.h:123: void crimson::osd::OpSequencer::finish_op_in_order(crimson::osd::ClientRequest&): Assertion `op.get_id() > last_completed_id' failed.
Aborting on shard 0.
Backtrace:
Reactor stalled for 1807 ms on shard 0. Backtrace: 0xb14ab 0x46e57428 0x46bc450d 0x46be03bd 0x46be0782 0x46be0946 0x46be0bf6 0x12b1f 0x137341 0x3fdd6a92 0x3fddccdb 0x3fdde1ee 0x3fdde8b3 0x3fdd3f2b 0x3fdd4442 0x3fdd4c3a 0x12b1f 0x3737e 0x21db4 0x21c88 0x2fa75 0x3b769527 0x3b8418af 0x3b8423cb 0x3b842ce0 0x3b84383d 0x3a116220 0x3a143f31 0x3a144bcd 0x46b96271 0x46bde51a 0x46d6891b 0x46d6a8f0 0x4681a7d2 0x4681f03b 0x39fd50f2 0x23492 0x39b7a7dd
0# gsignal in /lib64/libc.so.6
1# abort in /lib64/libc.so.6
2# 0x00007FB9FB946C89 in /lib64/libc.so.6
3# 0x00007FB9FB954A76 in /lib64/libc.so.6
4# 0x00005595E98E6528 in ceph-osd
5# 0x00005595E99BE8B0 in ceph-osd
6# 0x00005595E99BF3CC in ceph-osd
7# 0x00005595E99BFCE1 in ceph-osd
8# 0x00005595E99C083E in ceph-osd
9# 0x00005595E8293221 in ceph-osd
10# 0x00005595E82C0F32 in ceph-osd
11# 0x00005595E82C1BCE in ceph-osd
12# 0x00005595F4D13272 in ceph-osd
13# 0x00005595F4D5B51B in ceph-osd
14# 0x00005595F4EE591C in ceph-osd
15# 0x00005595F4EE78F1 in ceph-osd
16# 0x00005595F49977D3 in ceph-osd
17# 0x00005595F499C03C in ceph-osd
18# main in ceph-osd
19# __libc_start_main in /lib64/libc.so.6
20# _start in ceph-osd
```
The sequence of events provides at least two clues:
- the op no. 32 finished before the op no. 29 which was waiting
for `ObjectContext`,
- the op no. 29 was a short-living one -- it wasn't waiting even
on `obc`.
```
rzarzynski@teuthology:/home/teuthworker/archive/rzarzynski-2021-11-22_22:01:32-rados-master-distro-basic-smithi$ less ./
6520106 /remote/smithi115/log/ceph-osd.3.log.gz
...
DEBUG 2021-11-22 22:32:24,531 [shard 0] osd - client_request(id=29, detail=m=[osd_op(client.4371.0:36 4.d 4.
f0fb5e1d (undecoded) ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e23) v8]): start
DEBUG 2021-11-22 22:32:24,531 [shard 0] osd - client_request(id=29, detail=m=[osd_op(client.4371.0:36 4.d 4.
f0fb5e1d (undecoded) ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e23) v8]): in repeat
...
DEBUG 2021-11-22 22:32:24,546 [shard 0] osd - client_request(id=29, detail=m=[osd_op(client.4371.0:36 4.d 4.
f0fb5e1d (undecoded) ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e23) v8]) same_interval_since: 21
DEBUG 2021-11-22 22:32:24,546 [shard 0] osd - OpSequencer::start_op: op=29, last_started=27, last_unblocked=27, last_completed=27
...
DEBUG 2021-11-22 22:32:24,621 [shard 0] osd - client_request(id=32, detail=m=[osd_op(client.4371.0:49 4.d 4.
81addbad (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e23) v8]): start
DEBUG 2021-11-22 22:32:24,621 [shard 0] osd - client_request(id=32, detail=m=[osd_op(client.4371.0:49 4.d 4.
81addbad (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e23) v8]): in repeat
...
DEBUG 2021-11-22 22:32:24,626 [shard 0] osd - client_request(id=32, detail=m=[osd_op(client.4371.0:49 4.d 4.
81addbad (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e23) v8]) same_interval_s
ince: 21
DEBUG 2021-11-22 22:32:24,626 [shard 0] osd - OpSequencer::start_op: op=32, last_started=29, last_unblocked=29, last_completed=27
<note that op 32 is very short living>
DEBUG 2021-11-22 22:32:24,669 [shard 0] osd - OpSequencer::finish_op_in_order: op=32, last_started=32, last_unblocked=32, last_completed=27
...
DEBUG 2021-11-22 22:32:24,671 [shard 0] osd - client_request(id=32, detail=m=[osd_op(client.4371.0:49 4.d 4:
b5dbb581 :::smithi11538976-13:head {write 601684~619341 in=
619341b , stat} snapc 0={} RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e23) v8]): destroying
...
DEBUG 2021-11-22 22:32:24,722 [shard 0] osd - client_request(id=29, detail=m=[osd_op(client.4371.0:36 4.d 4:
b87adf0f :::smithi11538976-9:head {read 0~1} snapc 0={} RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e23) v8]): got obc lock
...
INFO 2021-11-22 22:32:24,723 [shard 0] osd - client_request(id=29, detail=m=[osd_op(client.4371.0:36 4.d 4:
b87adf0f :::smithi11538976-9:head {read 0~1} snapc 0={} RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e23) v8]) obc.get()=0x6190000d5780
...
DEBUG 2021-11-22 22:32:24,753 [shard 0] osd - OpSequencer::finish_op_in_order: op=29, last_started=32, last_unblocked=32, last_completed=32
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/
17.0.0-8898-ge57ad63c /rpm/el8/BUILD/
ceph-17.0.0-8898-ge57ad63c /src/crimson/osd/osd_operation_sequencer.h:123: void crimson::osd::OpSequencer::finish_op_in_order(crimson::osd::ClientRequest&): Assertion `op.get_id() > last_completed_id' failed.
Aborting on shard 0.
```
This could be explained in a scenario where:
- op no. 29 skipped stages of the execution pipeline while
- it wrongly informed `OpSequencer` the execution was in-order.
Static analysis shows there are multiple problems of this genre
in the `ClientRequest::process_op()` and its callees with the most
recently merged one being the path for `PG::already_complete()`.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Samuel Just [Thu, 2 Dec 2021 21:45:47 +0000 (21:45 +0000)]
crimson/os/seastore/lba_manager: initialize lba node pins using get_extent
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 2 Dec 2021 03:25:15 +0000 (19:25 -0800)]
crimson/os/seastore: initialize logical pins before exposing to cache
Otherwise, another task may get a reference to the extent before
we've set the pin.
Fixes: https://tracker.ceph.com/issues/53267
Signed-off-by: Samuel Just <sjust@redhat.com>
Samuel Just [Tue, 7 Dec 2021 06:21:55 +0000 (22:21 -0800)]
Merge pull request #44231 from xxhdx1985126/wip-cpu-profile
crimson/os/seastore: fix compiler error for gcc > 9 and clang13
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Huber-ming [Tue, 7 Dec 2021 05:59:47 +0000 (13:59 +0800)]
radosgw-admin: fix some error logs
Signed-off-by: Huber-ming <zhangsm01@inspur.com>
Xuehan Xu [Tue, 7 Dec 2021 04:27:14 +0000 (12:27 +0800)]
crimson/os/seastore: fix compiler error for gcc > 9 and clang13
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
David Galloway [Mon, 6 Dec 2021 17:58:22 +0000 (12:58 -0500)]
Merge pull request #44222 from ceph/wip-m2r
doc: Use older mistune
Radoslaw Zarzynski [Mon, 6 Dec 2021 17:29:39 +0000 (17:29 +0000)]
crimson/osd: clean the recovery message-related header inclusion.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Mon, 6 Dec 2021 16:54:41 +0000 (16:54 +0000)]
crimson/osd: don't assume a pull must happen if there is no push.
In the classical OSD the `ReplicatedRecoveryBackend::recover_object()`
divides into two main flows: pull and push:
```cpp
int ReplicatedBackend::recover_object(
const hobject_t &hoid,
// ...
)
{
dout(10) << __func__ << ": " << hoid << dendl;
RPGHandle *h = static_cast<RPGHandle *>(_h);
if (get_parent()->get_local_missing().is_missing(hoid)) {
ceph_assert(!obc);
// pull
prepare_pull(
v,
hoid,
head,
h);
} else {
ceph_assert(obc);
int started = start_pushes(
hoid,
obc,
h);
// ...
}
return 0;
}
```
Pulls may also enter the push path (`C_ReplicatedBackend_OnPullComplete`)
but push handling doesn't draw any assumption on that. What's important,
`recover_object()` may result in no pulls and pushes.
This isn't the case of crimson as its implementation of the push path
asserts that, if no push is scheduled, `PullInfo` must be allocated.
This patch reworks this logic to reflects the classical one and to avoid
crashes like the following one:
```
DEBUG 2021-12-01 18:43:00,220 [shard 0] osd - recover_object: loaded obc: 3:
4e058a2e :::smithi13839607-45:head
WARN 2021-12-01 18:43:00,220 [shard 0] none - intrusive_ptr_add_ref(p=0x6190000d7f80, use_count=3)
WARN 2021-12-01 18:43:00,220 [shard 0] none - intrusive_ptr_release(p=0x6190000d7f80, use_count=4)
TRACE 2021-12-01 18:43:00,220 [shard 0] osd - call_with_interruption_impl clearing interrupt_cond: 0x60300012b210,N7crimson3osd20IOInterruptConditionE
TRACE 2021-12-01 18:43:00,220 [shard 0] osd - call_with_interruption_impl: may_interrupt: false, local interrupt_condintion: 0x60300012b210, global interrupt_cond: 0x0,N7crimson3osd20IOInterruptConditionE
TRACE 2021-12-01 18:43:00,220 [shard 0] osd - set: interrupt_cond: 0x60300012b210, ref_count: 1
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/
17.0.0-8902-g52fd47fe /rpm/el8/BUILD/ceph-17.0.
0-8902-g52fd47fe /src/crimson/osd/replicated_recovery_backend.cc:84: ReplicatedRecoveryBackend::maybe_push_shards(const hobject_t&, eversion_t)::<lambda()>: Assertion `recovery.pi' failed.
Aborting on shard 0.
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
David Galloway [Mon, 6 Dec 2021 15:32:56 +0000 (10:32 -0500)]
doc: Use older mistune
https://github.com/miyakogi/m2r/issues/66
Signed-off-by: David Galloway <dgallowa@redhat.com>
benhanokh [Sun, 5 Dec 2021 07:47:49 +0000 (09:47 +0200)]
Merge pull request #43870 from benhanokh/restore_alloc_file
NCB::refresh allocation-file after FSCK remove
Gabriel BenHanokh [Mon, 8 Nov 2021 17:12:40 +0000 (19:12 +0200)]
BlueStore: Fix a bug when FSCK is invoked in mount()/umount()/mkfs() with DEEP option
Fixes: https://tracker.ceph.com/issues/53185
NCB mishandles fsck DEEP in mount()/umount()/mkfs() case causing it to remove the allocation-file without destaging a new copy (which will cost us a full rebuild on startup)
There are also few confiliting calls to open_db()/close_db() passing inconsistent read-only flag
We fix both issues by storing open-db type (read-only/read-write) and using it for close-db (which won't pass read-only flag anymore)
We also move allocation-file destage to close-db so it will be refreshed after being removed by fsck and such
Signed-off-by: Gabriel Benhanokh <gbenhano@redhat.com>
Sage Weil [Sat, 4 Dec 2021 17:52:45 +0000 (12:52 -0500)]
Merge PR #44155 into master
* refs/pull/44155/head:
mgr: limit changes to pg_num
Reviewed-by: Neha Ojha <nojha@redhat.com>
Sage Weil [Sat, 4 Dec 2021 17:52:09 +0000 (12:52 -0500)]
Merge PR #44108 into master
* refs/pull/44108/head:
mgr: fix locking for MetadataUpdate::finish
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Mykola Golub [Sat, 4 Dec 2021 16:25:02 +0000 (18:25 +0200)]
Merge pull request #44204 from songtongshuai/sts_ceph
test/librbd: add get_group test
Reviewed-by: Mykola Golub <mgolub@suse.com>
zdover23 [Sat, 4 Dec 2021 05:53:42 +0000 (15:53 +1000)]
Merge pull request #44189 from zdover23/wip-doc-2021-12-02-documenting-ceph
doc/start: update documenting-ceph.rst (1 of x)
Reviewed-by: Laura Flores <lflores@redhat.com>
Sage Weil [Fri, 3 Dec 2021 17:42:57 +0000 (12:42 -0500)]
Merge PR #44017 into master
* refs/pull/44017/head:
mgr/cephadm: Do not propogate access logs from cherrypy
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Matt Benjamin [Fri, 3 Dec 2021 17:22:20 +0000 (12:22 -0500)]
Merge pull request #44139 from linuxbox2/wip-rgw-lcselect
rgwlc: permit lifecycle processing for a single bucket
Sage Weil [Fri, 3 Dec 2021 17:05:13 +0000 (12:05 -0500)]
Merge PR #44132 into master
* refs/pull/44132/head:
mgr/prometheus: define module options for standby
Reviewed-by: Laura Flores <lflores@redhat.com>
songtongshuai_yewu [Fri, 3 Dec 2021 14:29:03 +0000 (09:29 -0500)]
test/librbd: add get_group test
Signed-off-by: songtongshuai_yewu <songtongshuai_yewu@cmss.chinamobile.com>
Zac Dover [Thu, 2 Dec 2021 14:15:44 +0000 (00:15 +1000)]
doc/start: update documenting-ceph.rst (1 of x)
This PR updates the content on documenting-ceph,
which is, as of December 2021, in need of an
update.
This is the first of what I estimate to be three
to five PRs against this .rst file.
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Sridhar Seshasayee [Fri, 3 Dec 2021 09:55:32 +0000 (15:25 +0530)]
PendingReleaseNotes: Release note 'require-osd-release' health warning
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Sridhar Seshasayee [Mon, 22 Nov 2021 15:16:02 +0000 (20:46 +0530)]
osd/OSDMap: Add health warning if 'require-osd-release' != current release
After all OSDs are upgraded to a new release, generate a health warning if
the 'require-osd-release' flag doesn't match the the new release version.
This will result in the cluster showing a warning in the health state until
the flag is set properly.
Fixes: https://tracker.ceph.com/issues/51984
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Deepika Upadhyay [Fri, 3 Dec 2021 09:35:12 +0000 (15:05 +0530)]
Merge pull request #44103 from majianpeng/librbd-pwl-flush-by-blockgurad
librbd/cache/pwl: Using BlockGuard control overlap ops order when flu…
Reviewed-by: Mykola Golub mgolub@suse.com
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Deepika Upadhyay [Fri, 3 Dec 2021 06:21:08 +0000 (11:51 +0530)]
Merge pull request #44144 from majianpeng/librbd-fix-discard-granularity
librbd: fix discard granularity for pwl cache
Reviewed-by: mgolub@suse.com
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Sage Weil [Thu, 2 Dec 2021 15:23:45 +0000 (10:23 -0500)]
mgr: only queue notify events that modules ask for
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Thu, 2 Dec 2021 15:23:22 +0000 (10:23 -0500)]
pybind/mgr: annotate which events modules consume
Signed-off-by: Sage Weil <sage@newdream.net>
Sage Weil [Thu, 2 Dec 2021 15:22:48 +0000 (10:22 -0500)]
pybind/mgr: introduce NotifyType enum
Note that we don't annotate the dashboard NotificationQueue because it is
used internally by the dashboard with other events.
Signed-off-by: Sage Weil <sage@newdream.net>
zdover23 [Thu, 2 Dec 2021 23:37:51 +0000 (09:37 +1000)]
Merge pull request #43848 from dvanders/doc_bench
doc: add disk benchmarking and cache tuning recommendations
Reviewed-by: Zac Dover <zac.dover@gmail.com>
Yuri Weinstein [Thu, 2 Dec 2021 23:28:17 +0000 (15:28 -0800)]
Merge pull request #43691 from curtbruns/use_optimal_for_min_alloc
os/bluestore: Set min_alloc_size to optimal io size
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Yuri Weinstein [Thu, 2 Dec 2021 23:27:18 +0000 (15:27 -0800)]
Merge pull request #43578 from ronen-fr/wip-rf-log-options
common: hide internal logger configuration strings from clients
Reviewed-by: Laura Flores <lflores@redhat.com>
Yuri Weinstein [Thu, 2 Dec 2021 23:26:24 +0000 (15:26 -0800)]
Merge pull request #43493 from myoungwon/wip-52872
test: increase retry duration when calculating manifest ref. count
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Yuri Weinstein [Thu, 2 Dec 2021 23:25:10 +0000 (15:25 -0800)]
Merge pull request #43944 from aclamk/wip-aclamk-fix-crush-location-hook-50659
crush: Fix segfault in update_from_hook
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Sage Weil [Thu, 2 Dec 2021 22:46:26 +0000 (17:46 -0500)]
mon/MgrStatMonitor: do not spam subscribers (mgr) with service_map
We are comparing the monmap epoch to the service_map epoch!
Fixes: https://tracker.ceph.com/issues/53479
Signed-off-by: Sage Weil <sage@newdream.net>
Igor Fedotov [Mon, 8 Nov 2021 12:18:11 +0000 (15:18 +0300)]
os/bluestore: report amount of loaded collections.
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
Igor Fedotov [Mon, 8 Nov 2021 12:14:46 +0000 (15:14 +0300)]
osd: report error and fail if store returns no meta collection
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
Yuri Weinstein [Thu, 2 Dec 2021 20:23:28 +0000 (12:23 -0800)]
Merge pull request #44194 from ceph/wip-yuriw-crontab-master
qa/tests: switch all gibba machines to smithi
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>