]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Sebastian Wagner [Thu, 15 Jul 2021 14:23:38 +0000 (16:23 +0200)]
Merge pull request #42220 from dsavineau/issue_51546
mgr/cephadm/iscsi: check if dashboard is enabled
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Casey Bodley [Thu, 15 Jul 2021 13:25:47 +0000 (09:25 -0400)]
Merge pull request #42339 from cbodley/wip-revert-vstart-system
Revert "src/vstart: create rgw user with system flag"
Reviewed-by: Varsha Rao <varao@redhat.com>
Ernesto Puerta [Thu, 15 Jul 2021 13:24:51 +0000 (15:24 +0200)]
Merge pull request #41300 from rhcs-dashboard/force-maintenance-test-error
mgr/dashboard: Fix test_error force maintenance dashboard check
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Kefu Chai [Thu, 15 Jul 2021 03:06:34 +0000 (11:06 +0800)]
Merge pull request #42334 from tchaikov/wip-crimson-logging
crimson/common/log: print out logger.debug() when log level >=6
Reviewed-by: Mark Nelson <mnelson@readhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Kefu Chai [Thu, 15 Jul 2021 03:04:52 +0000 (11:04 +0800)]
Merge pull request #42337 from rzarzynski/wip-crimson-alienstore-dedicated-subsys
crimson/os: give AlienStore its own debug subsystem.
Reviewed-by: Mark Nelson <mnelson@readhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Adam C. Emerson [Wed, 14 Jul 2021 20:49:52 +0000 (16:49 -0400)]
Merge pull request #42251 from adamemerson/wip-robustly-invalidate
rgw: Robustly Invalidate
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Sage Weil [Wed, 14 Jul 2021 18:38:59 +0000 (14:38 -0400)]
Merge PR #42041 into master
* refs/pull/42041/head:
mgr/restful: ignore min/max_size
test/crush: drop min/max_size refs
qa/workunits/mon/pool_ops: remove test for min/max_size check
qa: scrub a few remaining mentions of ruleset
qa/standalone/mon/osd-*: fix tests
PendingReleaseNotes: note min/max_size removal
mgr/dashboard: remove max/min_size and ruleset
mon/OSDMonitor: fix calls to CrushTester
crush: eliminate min_size and max_size
test/cli/crushtool: reunumber rulesets in test maps
crushtool: require min/max or num-rep for --test
crush: remove last traces of 'ruleset'
test/cli/crushtool: use 'id' instead of 'ruleset' in crush inputs
crushtool: take --min-rep and --max-rep explicitly
crush/CrushTester: drop --ruleset
doc: scrub 'ruleset' from docs
src/erasure-code: rule, not ruleset
mon/OSDMonitor: remove check_crush_rule() callers
mon/OSDMonitor: rule, not ruleset
crushtool: remove check for overlapped ruels
crush/CrushWrapper: get_osd_pool_default_crush_replicated_ruleset -> rule
crush: remove find_rule()
mon/OSDMonitor: use pool's crush rule directly
osd/OSDMap: drop checks for ruleset == ruleid
osd/OSDMap: use pool's crush rule_id directly
mon/PGMap: use pool's crush_rule directly
mon/OSDMonitor: remove crush ruleset->rule rewrite
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 17:12:20 +0000 (01:12 +0800)]
Merge pull request #42332 from rzarzynski/wip-crimson-remotepeeringevent-sends-when-active
crimson/osd: send peering messages only when OSD is active.
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Adam C. Emerson [Wed, 14 Jul 2021 15:02:21 +0000 (11:02 -0400)]
rgw: Robust notify invalidates on cache timeout
This avoids a potential race condition in which updates are delayed.
Fixes: https://tracker.ceph.com/issues/51674
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Wed, 14 Jul 2021 14:57:02 +0000 (10:57 -0400)]
rgw: Rename REMOVE_OBJ to INVALIDATE_OBJ
Also rename ObjectCache::remove to ObjectCache::invalidate_remove
Since we're depending on these message types/functions having
invalidate semantics but NOT caching a negative result, rename and
leave a comment for clarity.
Fixes: https://tracker.ceph.com/issues/51674
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Adam C. Emerson [Wed, 7 Jul 2021 22:47:00 +0000 (18:47 -0400)]
rgw: distribute() takes RGWCacheNotifyInfo
So we don't have to parse the bufferlist back out to find what object
to throw out of the cache.
Fixes: https://tracker.ceph.com/issues/51674
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Casey Bodley [Wed, 14 Jul 2021 16:37:48 +0000 (12:37 -0400)]
Revert "src/vstart: create rgw user with system flag"
This reverts commit
0d0cd4c6b0e8bbeaa71f54c831d67ef880e58016 .
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Radoslaw Zarzynski [Wed, 14 Jul 2021 15:37:37 +0000 (15:37 +0000)]
crimson/os: give AlienStore its own debug subsystem.
For the sake of decoupling it from the BlueStore's logging.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
zdover23 [Wed, 14 Jul 2021 15:35:14 +0000 (01:35 +1000)]
Merge pull request #42327 from zdover23/wip-doc-cephadm-troubleshooting-1-of-x-2021-07-15
doc/cephadm: rewrite troubleshooting 1 of x
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Adam C. Emerson [Wed, 14 Jul 2021 14:40:22 +0000 (10:40 -0400)]
Merge pull request #42324 from adamemerson/wip-51661
rgw: Don't segfault on datalog trim
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Dimitri Savineau [Wed, 7 Jul 2021 14:11:41 +0000 (10:11 -0400)]
mgr/cephadm/iscsi: check if dashboard is enabled
When the mgr dashboard module isn't enabled then the iSCSI service deletion
is stuck and the cluster state goes ERR.
The `ceph dashboard` commands aren't available when the mgr dashboard module
isnt' enabled.
Closes: https://tracker.ceph.com/issues/51546
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 13:24:46 +0000 (21:24 +0800)]
crimson/common/log: print out logger.debug() when log level >=6
less chatty when debug level is 5. this matches the behavior of classic
osd.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Adam C. Emerson [Tue, 13 Jul 2021 20:05:47 +0000 (16:05 -0400)]
rgw: Don't segfault on datalog trim
Synchronous (or yielded, basically other-than AioCompletion trim)
would try to dereference the past-the-end iterator if we were trimming
to a point in the most recent generation.
https://tracker.ceph.com/issues/51661
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Radoslaw Zarzynski [Tue, 13 Jul 2021 12:54:55 +0000 (12:54 +0000)]
crimson/osd: send peering messages only when OSD is active.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Fri, 9 Jul 2021 14:43:08 +0000 (14:43 +0000)]
crimson/osd: assert OSD in active when dispatching peering msgs.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Nizamudeen A [Wed, 12 May 2021 10:07:40 +0000 (15:37 +0530)]
mgr/dashboard: Fix test_error force maintenance dashboard check
Fixes: https://tracker.ceph.com/issues/50771
Signed-off-by: Nizamudeen A <nia@redhat.com>
Ilya Dryomov [Wed, 14 Jul 2021 09:54:44 +0000 (11:54 +0200)]
Merge pull request #41490 from idryomov/wip-rbd-pwl-ssd-tailp
librbd/cache/pwl/ssd: fix first_free_entry and m_first_free_entry corruption
Reviewed-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Ilya Dryomov [Wed, 14 Jul 2021 09:44:41 +0000 (11:44 +0200)]
Merge pull request #42145 from hualongfeng/fix_get_context_error
librbd/cache/pwl/ssd: fix use-after-free on C_BlockIORequest
Reviewed-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Sebastian Wagner [Wed, 14 Jul 2021 09:16:45 +0000 (11:16 +0200)]
Merge pull request #42244 from zdover23/wip-doc-cephadm-operations-data-location-and-health-checks-2021-07-08
doc/cephadm: operations: Data location & ...
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Sebastian Wagner [Wed, 14 Jul 2021 09:15:39 +0000 (11:15 +0200)]
Merge pull request #42288 from mgfritch/cephadm-test-fixup-mock
cephadm: use CephadmContext rather than MagicMock
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 08:23:33 +0000 (16:23 +0800)]
Merge pull request #42310 from rzarzynski/wip-crimson-remotepeeringevent-waits-on-active
crimson/osd: remote peering requests wait for OSD activation.
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Hualong Feng [Fri, 2 Jul 2021 01:49:39 +0000 (09:49 +0800)]
librbd/cache/pwl/ssd: fix use-after-free on C_BlockIORequest
In setup_schedule_append() function, its first expression
will cause the req to be deleted, and subsequent use of
the variable req becomes an illegal operation. And due to
delete, rep->m_image_ctx will be empty, so it lead to
segfault in AbstractWriteLog::get_context().
So pass the `req` into `schedule_append()` function.
Fixes: https://tracker.ceph.com/issues/50951
Signed-off-by: Hualong Feng <hualong.feng@intel.com>
Radoslaw Zarzynski [Tue, 13 Jul 2021 12:39:02 +0000 (12:39 +0000)]
crimson/osd: move the wait-for-active logic into OSDState.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 06:41:35 +0000 (14:41 +0800)]
Merge pull request #42314 from rzarzynski/wip-crimson-single-do_peering_event
crimson/osd: consider epoch_sent when ignoring old peering event.
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 06:33:57 +0000 (14:33 +0800)]
Merge pull request #42271 from xxhdx1985126/wip-seastore-writev
crimson/os/seastore: use writev to avoid extra memory copy
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 06:27:13 +0000 (14:27 +0800)]
Merge pull request #42311 from rzarzynski/wip-crimson-fix-premature-activation
crimson/osd: prevent premature OSD activation.
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 06:24:35 +0000 (14:24 +0800)]
Merge pull request #42202 from cyx1231st/wip-seastore-metrics
crimson/os/seastore: implement label-based profiling with metrics
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Zac Dover [Wed, 14 Jul 2021 06:17:19 +0000 (16:17 +1000)]
doc/cephadm: rewrite troubleshooting 1 of x
This PR improves the readability and format
of the troubleshooting.rst file. This also
makes a change to the markdown of one of the
sub-subsections so that it is made of tildes
(~) instead of carets (^), because that's
the RST standard.
Signed-off-by: Zac Dover <zac.dover@gmail.com>
Xuehan Xu [Sat, 10 Jul 2021 04:42:18 +0000 (12:42 +0800)]
crimson/os/seastore: use writev to avoid extra memory copy
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Yingxin Cheng [Mon, 12 Jul 2021 08:11:26 +0000 (16:11 +0800)]
crimson/os/seastore/cache: improve logs for transaction conflict/reset
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Wed, 7 Jul 2021 01:46:32 +0000 (09:46 +0800)]
crimson/os/seastore/cache: count invalidated transactions
Labeled by source and extent-type.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Wed, 7 Jul 2021 01:45:33 +0000 (09:45 +0800)]
crimson/os/seastore/cache: count committed transactions
Labeled by source.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Wed, 7 Jul 2021 01:42:23 +0000 (09:42 +0800)]
crimson/os/seastore/cache: count created transactions
Labeled by source.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Wed, 7 Jul 2021 01:37:24 +0000 (09:37 +0800)]
crimson/os/seastore: introduce src_t to classify transactions
* INIT - READ => INIT
* INIT - MUTATE => INIT
* SEASTORE/TEST - READ => READ
* SEASTORE/TEST - MUTATE => MUTATE
* CLEANER - READ => CLEANER
* CLEANER - MUTATE => CLEANER
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Yingxin Cheng [Fri, 9 Jul 2021 07:07:49 +0000 (15:07 +0800)]
crimson/os/seastore: consolidate the static member CachedExtent::TYPE
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Kefu Chai [Wed, 14 Jul 2021 01:40:12 +0000 (09:40 +0800)]
Merge pull request #42279 from myoungwon/wip-fix-51627
osd: fix to recover adjacent clone when set_chunk is called
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 01:36:31 +0000 (09:36 +0800)]
Merge pull request #42285 from orozery/append-frame-claim-append
msg/async/ProtocolV2: optimize append_frame
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 01:18:22 +0000 (09:18 +0800)]
Merge pull request #42218 from ifed01/wip-ifed-compact-after-upgrade
os/bluestore: compact db after bulk omap naming upgrade.
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 01:16:53 +0000 (09:16 +0800)]
Merge pull request #42197 from ifed01/wip-ifed-fix-spurious-read-errors-warn
os/bluestore: respect bluestore_warn_on_spurious_read_errors setting
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 01:02:34 +0000 (09:02 +0800)]
Merge pull request #42291 from dsavineau/issue_51620
cephadm: ensure sysctl_dir exist
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Wed, 14 Jul 2021 01:00:54 +0000 (09:00 +0800)]
Merge pull request #42276 from tchaikov/wip-common-LogEntry
common/LogEntry: drop support of LogSummary v2 encoding scheme
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Sage Weil [Tue, 13 Jul 2021 22:20:21 +0000 (18:20 -0400)]
Merge PR #42319 into master
* refs/pull/42319/head:
qa/tasks/rebuild_mondb: fix rebuild vs logmonitor external_log_to
Reviewed-by: Neha Ojha <nojha@redhat.com>
Neha Ojha [Tue, 13 Jul 2021 21:10:43 +0000 (14:10 -0700)]
Merge pull request #42294 from neha-ojha/wip-51101
qa/*/test_envlibrados_for_rocksdb.sh: install libarchive-3.3.3
Reviewed-by: Kefu Chai <kchai@redhat.com>
Sage Weil [Tue, 13 Jul 2021 20:26:33 +0000 (16:26 -0400)]
Merge PR #42278 into master
* refs/pull/42278/head:
doc/mgr/nfs: update about RGW exports
doc/cephfs/nfs: update about nfs module
doc/mgr/nfs: update cephfs export create command about client and squash arguments
doc/mgr/nfs: update nfs links
doc/mgr/nfs: add missing cluster_id to export info command
doc/cephfs: move nfs doc under mgr docs
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Sage Weil [Tue, 13 Jul 2021 18:44:57 +0000 (14:44 -0400)]
qa/tasks/rebuild_mondb: fix rebuild vs logmonitor external_log_to
Signed-off-by: Sage Weil <sage@newdream.net>
Kefu Chai [Mon, 12 Jul 2021 04:40:50 +0000 (12:40 +0800)]
common/LogEntry: drop support of LogSummary v2 encoding scheme
LogSummary's v3 encoding scheme was introduced in
648aaf271cb02c647f046288656c11f15a7799b2 , which was in turn included
by Ceph v13.1.0 and all newer releases. since LogSummary is persistented
by monitor, and it is trimmed regularly by monitor, there is no need
to read a LogSummary encoded by 2 releases older monitor.
in this change, the support of LogSummary v2 encoding scheme is dropped.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 17:04:07 +0000 (01:04 +0800)]
Merge pull request #42301 from tchaikov/wip-doc-crimson-nbd
doc/dev/crimson: add a section for testing with crimson-store-nbd
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Ernesto Puerta [Tue, 13 Jul 2021 16:54:37 +0000 (18:54 +0200)]
Merge pull request #42305 from rhcs-dashboard/remove-rgw-frontend-ssl-key
mgr/dashboard: remove usage of 'rgw_frontend_ssl_key'
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 10:01:37 +0000 (18:01 +0800)]
doc/dev/crimson: add a section for testing with crimson-store-nbd
Signed-off-by: Kefu Chai <kchai@redhat.com>
Radoslaw Zarzynski [Tue, 13 Jul 2021 16:26:20 +0000 (16:26 +0000)]
crimson/osd: consider epoch_sent when ignoring old peering event.
The classical OSD does:
```
bool old_peering_evt(PGPeeringEventRef evt) {
return old_peering_msg(evt->get_epoch_sent(), evt->get_epoch_requested());
}
```
```
bool PG::old_peering_msg(epoch_t reply_epoch, epoch_t query_epoch)
{
if (auto last_reset = get_last_peering_reset();
last_reset > reply_epoch || last_reset > query_epoch) {
dout(10) << "old_peering_msg reply_epoch " << reply_epoch << " query_epoch "
<< query_epoch << " last_peering_reset " << last_reset << dendl;
return true;
}
return false;
}
```
See also: https://gist.github.com/rzarzynski/
e60750c279ae9d234728b0b199562c40 #gistcomment-
3808289 .
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Tue, 13 Jul 2021 16:24:04 +0000 (16:24 +0000)]
crimson/osd: flip the negation in PG::do_peering_event().
Less operations, lower mental load on reader's mind.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Tue, 13 Jul 2021 16:15:14 +0000 (16:15 +0000)]
crimson/osd: unify the variants of PG::do_peering_event().
Simpler and more comprehensible that way.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Sage Weil [Tue, 13 Jul 2021 16:04:55 +0000 (12:04 -0400)]
Merge PR #42227 into master
* refs/pull/42227/head:
doc/man/8/cephadm: add --log-to-file (and --single-host-defaults)
cephadm: add bootstrap --log-to-file option
Reviewed-by: Adam King <adking@redhat.com>
Sage Weil [Tue, 13 Jul 2021 16:04:38 +0000 (12:04 -0400)]
Merge PR #42014 into master
* refs/pull/42014/head:
mon/LogMonitor: make 'ceph log last' work with new storage
mon/LogMonitor: transition log entries on upgrade to quincy
mon/LogMonitor: store logentries in separate keys
mon/LogMonitor: replace semi-broken pending_summary with pending_keys
mon/LogMonitor: externally log events from when we were out of quorum
mon/LogMonitor: factor logging to file out of update_from_paxos
mon: funnel SIGHUP handler through handle_mon_signal
common/LogEntry: add some constness
mon/LogMonitor: log mkfs in cluster channel
common/LRUSet: combine lru and hash-based lookup
mon/LogMonitor: remove stray derr
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 12:06:00 +0000 (20:06 +0800)]
doc/dev/crimson: add a missing not
Signed-off-by: Kefu Chai <kchai@redhat.com>
Radoslaw Zarzynski [Mon, 12 Jul 2021 16:51:54 +0000 (16:51 +0000)]
crimson/osd: improve debugs and clean up OSD activation.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 15:39:11 +0000 (23:39 +0800)]
Merge pull request #42312 from rzarzynski/wip-crimson-boot_epoch-in-MOSDBoot
crimson/osd: use boot_epoch for MOSDBoot.
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 15:36:15 +0000 (23:36 +0800)]
Merge pull request #42313 from rzarzynski/wip-crimson-drop-temporary-from-pg
crimson/osd: drop a temporary in PG::handle_initialize().
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Neha Ojha [Mon, 12 Jul 2021 22:25:55 +0000 (22:25 +0000)]
qa/*/test_envlibrados_for_rocksdb.sh: install libarchive-3.3.3
To workaround the libarchive dependency issue seen with centos 8, which
has been causing consistent failures like
```
2021-06-04T04:52:51.147 INFO:tasks.workunit.client.0.smithi071.stdout:Installed:
2021-06-04T04:52:51.148 INFO:tasks.workunit.client.0.smithi071.stdout: cmake-3.18.2-9.el8.x86_64 cmake-data-3.18.2-9.el8.noarch
...
2021-06-04T04:52:57.554 INFO:tasks.workunit.client.0.smithi071.stderr:+ cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_TESTS=ON -DWITH_LIBRADOS=ON -DWITH_SNAPPY=ON -DWITH_GFLAGS=OFF -DFAIL_ON_WARNINGS=OFF ..
2021-06-04T04:52:57.579 DEBUG:teuthology.orchestra.run:got remote process result: 127
2021-06-04T04:52:57.580 INFO:tasks.workunit.client.0.smithi071.stderr:cmake: symbol lookup error: cmake: undefined symbol: archive_write_add_filter_zstd
```
More details in https://tracker.ceph.com/issues/51101#note-5
Fixes: https://tracker.ceph.com/issues/51101
Signed-off-by: Neha Ojha <nojha@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 15:08:45 +0000 (23:08 +0800)]
Merge pull request #42303 from tchaikov/wip-crimson-nbd-graceful-shutdown
crimson/tools/store_nbd: fixes to implement graceful shutdown
Reviewed-by: Samuel Just <sjust@redhat.com>
Patrick Donnelly [Tue, 13 Jul 2021 15:07:22 +0000 (08:07 -0700)]
Merge PR #42029 into master
* refs/pull/42029/head:
vstart_runner: use FileNotFoundError when os.stat() fails
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Patrick Donnelly [Tue, 13 Jul 2021 15:05:25 +0000 (08:05 -0700)]
Merge PR #42030 into master
* refs/pull/42030/head:
vstart_runner: maintain log level when --debug is passed
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 13 Jul 2021 15:03:18 +0000 (08:03 -0700)]
Merge PR #42033 into master
* refs/pull/42033/head:
vstart_runner: add log messages to vstart_runner.py
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Radoslaw Zarzynski [Tue, 13 Jul 2021 12:09:39 +0000 (12:09 +0000)]
crimson/osd: remote peering requests wait for OSD activation.
Before the patch `RemotePeeringRequest` instances were not
waiting for OSD activation. This was eluding the protection
from handling old, outdated peering events the `MOSDBoot`
machinery offers. The net results are crashes like this one
(`OSDState is booting` has been produced by a custom debug):
```
2021-07-07T18:20:23.293 INFO:journalctl@ceph.osd.2.smithi145.stdout:Jul 07 18:16:30 smithi145 conmon[71083]: DEBUG 2021-07-07 18:16:30,535 [shard 0] ms - [osd.2(cluster) v2:172.21.15.145:6802/2@62336 >> osd
.1 v2:172.21.15.145:6809/2] <== #19 === pg_lease(4.9 pg_lease(ru 60.120281219s ub 68.121276855s int 16.000000000s) e86/86) v1 (133)
2021-07-07T18:20:23.293 INFO:journalctl@ceph.osd.2.smithi145.stdout:Jul 07 18:16:30 smithi145 conmon[71083]: DEBUG 2021-07-07 18:16:30,536 [shard 0] osd - handle_peering_op on 4.9 from 1
2021-07-07T18:20:23.293 INFO:journalctl@ceph.osd.2.smithi145.stdout:Jul 07 18:16:30 smithi145 conmon[71083]: DEBUG 2021-07-07 18:16:30,536 [shard 0] osd - peering_event(id=125, detail=PeeringEvent(from=1 pg
id=4.9 sent=86 requested=86 evt=epoch_sent: 86 epoch_requested: 86 MLease epoch 86 from osd.1 pg_lease(ru 60.120281219s ub 68.121276855s int 16.000000000s))): start
2021-07-07T18:20:23.293 INFO:journalctl@ceph.osd.2.smithi145.stdout:Jul 07 18:16:30 smithi145 conmon[71083]: DEBUG 2021-07-07 18:16:30,536 [shard 0] osd - peering_event(id=125, detail=PeeringEvent(from=1 pg
id=4.9 sent=86 requested=86 evt=epoch_sent: 86 epoch_requested: 86 MLease epoch 86 from osd.1 pg_lease(ru 60.120281219s ub 68.121276855s int 16.000000000s))): got map 93
2021-07-07T18:20:23.294 INFO:journalctl@ceph.osd.2.smithi145.stdout:Jul 07 18:16:30 smithi145 conmon[71083]: DEBUG 2021-07-07 18:16:30,536 [shard 0] osd - peering_event(id=125, detail=PeeringEvent(from=1 pgid=4.9 sent=86 requested=86 evt=epoch_sent: 86 epoch_requested: 86 MLease epoch 86 from osd.1 pg_lease(ru 60.120281219s ub 68.121276855s int 16.000000000s))): OSDState is booting
2021-07-07T18:20:23.294 INFO:journalctl@ceph.osd.2.smithi145.stdout:Jul 07 18:16:30 smithi145 conmon[71083]: ERROR 2021-07-07 18:16:30,536 [shard 0] none - /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/
17.0.0-5007-g3a9abb02 /rpm/el8/BUILD/
ceph-17.0.0-5007-g3a9abb02 /src/crimson/osd/osd_operations/peering_event.cc:165 : In function 'crimson::osd::RemotePeeringEvent::get_pg()::<lambda()>', ceph_assert(%s)
2021-07-07T18:20:23.294 INFO:journalctl@ceph.osd.2.smithi145.stdout:Jul 07 18:16:30 smithi145 conmon[71083]: osd.state.is_active()
2021-07-07T18:20:23.294 INFO:journalctl@ceph.osd.2.smithi145.stdout:Jul 07 18:16:30 smithi145 conmon[71083]: Aborting on shard 0.
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Mon, 12 Jul 2021 14:42:36 +0000 (14:42 +0000)]
crimson/osd: drop a temporary in PG::handle_initialize().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Wed, 7 Jul 2021 14:46:37 +0000 (14:46 +0000)]
crimson/osd: use boot_epoch for MOSDBoot.
For the sake of compliance and similarity with the classcial OSD.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Mon, 12 Jul 2021 14:26:48 +0000 (14:26 +0000)]
crimson/osd: prevent premature OSD activation.
In contrast to the classical OSD:
```
int OSD::init()
{
// ...
{
epoch_t bind_epoch = osdmap->get_epoch();
service.set_epochs(NULL, NULL, &bind_epoch);
}
// ...
// load up pgs (as they previously existed)
load_pgs();
```
crimson doesn't set the `bind_epoch` when initializing. The net
result is going active prematurely which happens because the 3rd
condition (`bind_epoch < osdmap->get_up_from(whoami)`) is always
true.
```
if (osdmap->is_up(whoami) &&
osdmap->get_addrs(whoami) == public_msgr->get_myaddrs() &&
bind_epoch < osdmap->get_up_from(whoami)) {
if (state.is_booting()) {
logger().info("osd.{}: activating...", whoami);
```
Nullifying it translates the "is it activated?" check basically
into "is it up?" verification. This is problematic in a situation
like:
1. Primary got new OSDMap but replica has not.
2. Replica restarts, sends `MOSDBoot` and receives the newer map
from the previous point.
3. Primary sends a message that is unexpected by replica.
4. Monitor publishes a new OSDMap diven by the `MOSDBoot`.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 14:15:34 +0000 (22:15 +0800)]
Merge pull request #42267 from ifed01/wip-fix-missing-shared-blob
os/bluestore: fix erroneous SharedBlob record removal during repair.
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Sebastian Wagner [Tue, 13 Jul 2021 14:06:01 +0000 (16:06 +0200)]
Merge pull request #42209 from sebastian-philipp/doc-dev-cephadm-define-vars
doc/dev/cephadm: Define variables
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Avan Thakkar [Tue, 13 Jul 2021 12:41:10 +0000 (18:11 +0530)]
mgr/dashboard: remove usage of 'rgw_frontend_ssl_key'
Fixes: https://tracker.ceph.com/issues/51643
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Removing the usage of rgw_frontend_ssl_key from the rgw service form.
Kefu Chai [Tue, 13 Jul 2021 13:29:29 +0000 (21:29 +0800)]
Merge pull request #42274 from liewegas/cleanup-blkdev
common/blkdev: remove stray debug output
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 13:23:43 +0000 (21:23 +0800)]
Merge pull request #42206 from ronen-fr/wip-ronenf-list-object
common/hobject: a minor fix and performance gain to hobjects listing
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Or Ozeri [Sun, 6 Jun 2021 13:55:15 +0000 (16:55 +0300)]
msg/async/ProtocolV2: optimize append_frame
The commonly used append_frame function currently copies
frame data, incurring expensive heap allocation and data copying.
Instead, switch to claiming the frame data, re-using it without copying.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Kefu Chai [Tue, 13 Jul 2021 12:25:15 +0000 (20:25 +0800)]
Merge pull request #42210 from sebastian-philipp/options-ms-bind-port-max
common/options: global.yaml: change ms_bind_port_max to 7568
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 12:24:31 +0000 (20:24 +0800)]
Merge pull request #42268 from neha-ojha/wip-health-cleanup
mon/PGMap: remove get_stuck_counts because there are no callers
Reviewed-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 12:23:07 +0000 (20:23 +0800)]
Merge pull request #42235 from tchaikov/tools/kvstore-tool
tools/kvstore_tool: add "std::" before ostream and string
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 11:42:19 +0000 (19:42 +0800)]
crimson/tools/store_nbd: handle ECONNABORTED returned by accept()
if we abort accept() call, an ECONNABORTED is expected. and we should
handle it, otherwise unhandled exception will be noticed by seastar's
reactor. and it complains in that case.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Tue, 13 Jul 2021 11:22:57 +0000 (19:22 +0800)]
crimson/tools/store_nbd: call segment_manager->close() after tm->close()
TransactionManager::close() calls into journal->close(), which in turn
calls BlockSegmentManager::segment_close(). and
SegmentStateTracker::write_out() is then called by
BlockSegmentManager::segment_close().
but BlockSegmentManager::close() closes the underlying seastar::file,
we are not able to write to the file after closing it.
in this change, to ensure that we can close a segment correctly in
TMDriver::close(), tm->close() is called before
segment_manager->close().
Signed-off-by: Kefu Chai <kchai@redhat.com>
myoungwon oh [Mon, 12 Jul 2021 05:20:45 +0000 (14:20 +0900)]
osd: fix to recover adjacent clone when set_chunk is called
set_chunk needs adjacent clones to calculate reference count
fixes: https://tracker.ceph.com/issues/51627
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Varsha Rao [Mon, 12 Jul 2021 07:31:49 +0000 (13:01 +0530)]
doc/mgr/nfs: update about RGW exports
This patch just moves the RGW exports created using nfs module to mgr/nfs
document. The RGW requirements will be updated in a different PR.
Signed-off-by: Varsha Rao <varao@redhat.com>
Varsha Rao [Mon, 12 Jul 2021 07:08:36 +0000 (12:38 +0530)]
doc/cephfs/nfs: update about nfs module
Signed-off-by: Varsha Rao <varao@redhat.com>
Varsha Rao [Mon, 12 Jul 2021 06:29:59 +0000 (11:59 +0530)]
doc/mgr/nfs: update cephfs export create command about client and squash arguments
Signed-off-by: Varsha Rao <varao@redhat.com>
Varsha Rao [Tue, 29 Jun 2021 13:39:44 +0000 (19:09 +0530)]
doc/mgr/nfs: update nfs links
Signed-off-by: Varsha Rao <varao@redhat.com>
Varsha Rao [Tue, 29 Jun 2021 13:08:03 +0000 (18:38 +0530)]
doc/mgr/nfs: add missing cluster_id to export info command
Signed-off-by: Varsha Rao <varao@redhat.com>
Varsha Rao [Tue, 29 Jun 2021 13:03:36 +0000 (18:33 +0530)]
doc/cephfs: move nfs doc under mgr docs
Fixes: https://tracker.ceph.com/issues/51428
Signed-off-by: Varsha Rao <varao@redhat.com>
zdover23 [Mon, 12 Jul 2021 20:20:29 +0000 (06:20 +1000)]
Merge pull request #42269 from zdover23/wip-doc-dev-essentials-irc-2021-07-10
doc/dev: add IRC information to dev guide
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
zdover23 [Mon, 12 Jul 2021 20:19:30 +0000 (06:19 +1000)]
Merge pull request #42113 from zdover23/wip-doc-upgrading-ceph-potential-problems-2021-06-30
doc/cephadm: improve "Potential Problems"
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
zdover23 [Mon, 12 Jul 2021 20:18:56 +0000 (06:18 +1000)]
Merge pull request #42092 from zdover23/wip-doc-upgrading-ceph-starting-the-upgrade-2021-06-29
doc/cephadm: improving "Starting the Upgrade"
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Sage Weil [Mon, 12 Jul 2021 17:45:35 +0000 (13:45 -0400)]
doc/man/8/cephadm: add --log-to-file (and --single-host-defaults)
Signed-off-by: Sage Weil <sage@newdream.net>
Dimitri Savineau [Mon, 12 Jul 2021 14:39:36 +0000 (10:39 -0400)]
cephadm: ensure sysctl_dir exist
For some reason, the sysctl directory could not exist if no packages dropping
a custom sysctl file is installed on the host.
Instead we create the directory if it doesn't exist.
Closes: https://tracker.ceph.com/issues/51620
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
Ernesto Puerta [Mon, 12 Jul 2021 15:30:19 +0000 (17:30 +0200)]
Merge pull request #42217 from clwluvw/osd-device-details-grafana
monitoring: fix Physical Device Latency unit
Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: mykaul <NOT@FOUND>
Reviewed-by: p-se <NOT@FOUND>
Ernesto Puerta [Mon, 12 Jul 2021 15:03:30 +0000 (17:03 +0200)]
Merge pull request #41034 from nSedrickm/auth-storage-directive
mgr/dashboard: create directive for AuthStorage service
Reviewed-by: Alfonso MartÃnez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Ngwa Sedrick Meh [Mon, 26 Apr 2021 20:17:08 +0000 (21:17 +0100)]
mgr/dashboard: create directive for AuthStorage service
This commit adds a directive that can be used to conditionally display elements based on authorization/scopes criteria
Fixes: https://tracker.ceph.com/issues/47355
Signed-off-by: Ngwa Sedrick Meh <nsedrick101@gmail.com>
Sage Weil [Sun, 11 Jul 2021 17:24:00 +0000 (13:24 -0400)]
common/blkdev: remove stray debug output
Signed-off-by: Sage Weil <sage@newdream.net>
Gal Salomon [Sun, 11 Jul 2021 20:03:02 +0000 (23:03 +0300)]
Merge pull request #41834 from grajoria/master
doc: Correction and improvisation for Timestamp part of the doc