]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
xie xingguo [Sun, 5 May 2019 05:17:32 +0000 (13:17 +0800)]
mgr/DaemonServer: safe-to-destroy - do not consider irrelevant pgs
otherwise ceph osd safe-to-destroy would say NO even if we want to
destroy an down+out osd from an actually active+clean pool!
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Sage Weil [Thu, 25 Apr 2019 01:40:17 +0000 (20:40 -0500)]
Merge PR #27626 into master
* refs/pull/27626/head:
mgr/dashboard: Do not default to 'admin' as Admin Resource
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Gabriel Brascher <gabriel@apache.org>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Sage Weil [Thu, 25 Apr 2019 01:38:19 +0000 (20:38 -0500)]
Merge PR #26184 into master
* refs/pull/26184/head:
mgr/dashboard: Replace IP address validation with Python standard library functions
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Sage Weil [Thu, 25 Apr 2019 01:38:02 +0000 (20:38 -0500)]
Merge PR #26730 into master
* refs/pull/26730/head:
src/test: remove unused objects to calculate accurate dedup ratio.
src/tools: fix counting total objects
src/tools: fix printing wrong offset.
src/tools: fix dedup tool's name
qa/workunit: add rabin chunk test
src/tool: add rabin fingerprint option to cephdeduptool
src/common: add rabin fingerprint class
Initail work for rabin fingerprint
Reviewed-by: Sage Weil <sage@redhat.com>
Sage Weil [Wed, 24 Apr 2019 22:00:57 +0000 (17:00 -0500)]
Merge PR #27566 into master
* refs/pull/27566/head:
auth: kill AuthUnknownSessionHandler
auth: kill AuthUnknown* (except AuthUnknownSessionHandler)
msg: remove old ms_* auth methods from Dispatcher interface
mon/MonClient: discard old challenge if authorizer is bad
msg/async/ProtocolV1: use AuthServer and AuthClient
mon/Monitor: handle v1 call into handle_auth_request
msg/Connection: add is_msgr2()
mon/MonClient: tolerate lack of authorizer for some dispatchers
Reviewed-by: Ricardo Dias <rdias@suse.com>
Casey Bodley [Wed, 24 Apr 2019 20:07:33 +0000 (16:07 -0400)]
Merge pull request #27579 from cbodley/wip-rgw-split-log-trim
rgw: split mdlog/datalog trimming into separate files
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Casey Bodley [Wed, 24 Apr 2019 19:47:59 +0000 (15:47 -0400)]
Merge pull request #26173 from cbodley/wip-rgw-async-putobjprocessor
rgw: add YieldingAioThrottle for async PutObj/GetObj
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Sage Weil [Fri, 12 Apr 2019 21:22:04 +0000 (16:22 -0500)]
auth: kill AuthUnknownSessionHandler
The only user is ProtocolV1, which passes in an in-use auth_method...
which must be either AUTH_NONE or AUTH_CEPHX since we successfully
authenticated at this point.
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 12 Apr 2019 21:20:45 +0000 (16:20 -0500)]
auth: kill AuthUnknown* (except AuthUnknownSessionHandler)
These are unreferenced by any other code.
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 12 Apr 2019 19:21:20 +0000 (14:21 -0500)]
msg: remove old ms_* auth methods from Dispatcher interface
In the mon there is an opportunity to further clean up the code around
get_authorizer().
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Tue, 23 Apr 2019 18:52:07 +0000 (13:52 -0500)]
mon/MonClient: discard old challenge if authorizer is bad
If we are a server/accepter, and we get a bad authorizer, discard our old
challenge, since the previous exchange has failed. This way if the client
retries, their new authorize attempt won't conflict with our old challenge.
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 12 Apr 2019 19:19:38 +0000 (14:19 -0500)]
msg/async/ProtocolV1: use AuthServer and AuthClient
Stop using the old ms_* auth methods and instead use the new interfaces
(like V2).
Signed-off-by: Sage Weil <sage@redhat.com>
David Zafman [Wed, 24 Apr 2019 17:58:11 +0000 (10:58 -0700)]
Merge pull request #27279 from dzafman/wip-divergent
Improvements to standalone tests
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kefu Chai [Wed, 24 Apr 2019 16:18:48 +0000 (00:18 +0800)]
Merge pull request #27750 from tchaikov/wip-silence-gcc-warning
test/objectstore: silence -Wsign-compare warning
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Sage Weil [Wed, 24 Apr 2019 15:48:17 +0000 (10:48 -0500)]
Merge PR #27721 into master
* refs/pull/27721/head:
tools: pin the version of breathe that works with Python2
Reviewed-by: Kefu Chai <kchai@redhat.com>
Casey Bodley [Thu, 18 Apr 2019 13:18:37 +0000 (09:18 -0400)]
test/rgw: rgw throttle tests use mock operations
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 17 Apr 2019 21:04:55 +0000 (17:04 -0400)]
rgw: add optional_yield to Aio::librados_op()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 17 Apr 2019 21:04:37 +0000 (17:04 -0400)]
rgw: add optional_yield to get_obj_data
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 17 Apr 2019 21:03:54 +0000 (17:03 -0400)]
rgw: add optional_yield to PutObjProcessors
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Nathan Cutler [Wed, 24 Apr 2019 13:49:00 +0000 (15:49 +0200)]
Merge pull request #27221 from LenzGr/update-backportscripts
script: Obtain milestones via github API
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Casey Bodley [Mon, 28 Jan 2019 17:41:41 +0000 (12:41 -0500)]
rgw: read ops use make_throttle() with optional_yield
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 28 Jan 2019 18:12:11 +0000 (13:12 -0500)]
rgw: write ops use make_throttle() with optional_yield
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Mon, 28 Jan 2019 18:11:17 +0000 (13:11 -0500)]
rgw: add make_throttle() for optionally-async AioThrottle
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 10 Oct 2018 12:04:31 +0000 (08:04 -0400)]
rgw: add YieldingAioThrottle
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 10 Oct 2018 12:03:55 +0000 (08:03 -0400)]
rgw: add BlockingAioThrottle
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Apr 2019 13:42:28 +0000 (09:42 -0400)]
Merge pull request #25692 from Liuchang0812/fix-26835
rgw: returns LimitExceeded when user creates too many ACLs
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Apr 2019 13:41:52 +0000 (09:41 -0400)]
Merge pull request #25044 from tianshan/fix_36763
rgw: set null version object acl issues
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Apr 2019 13:41:31 +0000 (09:41 -0400)]
Merge pull request #21154 from mslovy/rgw-admin-api
rgw: update op_mask of user via admin rest api
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Apr 2019 13:40:36 +0000 (09:40 -0400)]
Merge pull request #17338 from tianshan/fix_roles_pool
rgw: add roles_pool in RGWZoneParams dump/decode json
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Wed, 24 Apr 2019 13:40:16 +0000 (09:40 -0400)]
Merge pull request #17591 from shashalu/fix-bucket-list-versions
rgw: list_bucket versions return NextVersionIdMarker = "null" if next_marker.instance is empty
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Sage Weil [Wed, 24 Apr 2019 13:37:49 +0000 (08:37 -0500)]
Merge PR #27472 into master
* refs/pull/27472/head:
doc/rados/operations/devices: document device failure prediction
Reviewed-by: Rick Chen <rick.chen@prophetstor.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Kefu Chai [Wed, 24 Apr 2019 13:34:26 +0000 (21:34 +0800)]
Merge pull request #27573 from iotcg/corpus
import-generated.sh: use PATH to get ceph-dencoder
Reviewed-by: Kefu Chai <kchai@redhat.com>
Sage Weil [Wed, 24 Apr 2019 13:15:19 +0000 (08:15 -0500)]
Merge PR #27696 into master
* refs/pull/27696/head:
osd: make use of pg history and past_intervals in pg_create2 messages
mon/OSDMonitor: track history and past_intervals for creating pgs
osd/osd_types: make PastIntervals pi_compact_rep print participants
osd/osd_types: take bare const OSDMap * to check_new_interval
osd/osd_types: add pg_history_t ctor that takes creation epoch+stamp
Reviewed-by: Neha Ojha <nojha@redhat.com>
Sage Weil [Wed, 24 Apr 2019 13:14:55 +0000 (08:14 -0500)]
Merge PR #27656 into master
* refs/pull/27656/head:
doc/dev/erasure-coded-pool: update
doc/rados/operations/erasure-code*: update default ec profile references
common/options: change default erasure-code-profile to k=2 m=2
Reviewed-by: Neha Ojha <nojha@redhat.com>
Kefu Chai [Wed, 24 Apr 2019 12:58:35 +0000 (20:58 +0800)]
test/objectstore: silence -Wsign-compare warning
Signed-off-by: Kefu Chai <kchai@redhat.com>
Mykola Golub [Wed, 24 Apr 2019 12:40:43 +0000 (15:40 +0300)]
Merge pull request #27683 from dillaman/wip-24668
qa/workunits/rbd: use more recent qemu-iotests that support Bionic
Reviewed-by: Mykola Golub <mgolub@suse.com>
Lenz Grimmer [Thu, 18 Apr 2019 11:33:22 +0000 (13:33 +0200)]
script: Obtain milestones via github API
Replace the hard-coded list of milestones with an API request
that obtains this data from github directly.
Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
Changcheng Liu [Sat, 13 Apr 2019 08:15:35 +0000 (16:15 +0800)]
import-generated.sh: correct access to ceph-dencoder
align with instructions in doc/dev/corpus.rst.
doc/dev/corpus.rst is under change by kchai@redhat.com
in PR 27552.
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
Kefu Chai [Wed, 24 Apr 2019 11:49:54 +0000 (19:49 +0800)]
Merge pull request #27704 from tchaikov/wip-denc-dump
denc: allow DencDumper to dump OOB buffer
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Lenz Grimmer [Wed, 24 Apr 2019 11:41:03 +0000 (13:41 +0200)]
mgr/dashboard: iSCSI: Limit members to 1 group (#27520)
mgr/dashboard: iSCSI: Limit members to 1 group
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Lenz Grimmer [Wed, 24 Apr 2019 11:38:14 +0000 (13:38 +0200)]
Merge pull request #27722 from rhcs-dashboard/39434-prometheus-replace-whitespaces
mgr/prometheus: replace whitespaces in metrics' names
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Lenz Grimmer [Wed, 24 Apr 2019 11:31:50 +0000 (13:31 +0200)]
Merge pull request #27678 from tspmelo/wip-iscsi-discovery-read
mgr/dashboard: Fix iSCSI Discovery user permissions
Lenz Grimmer [Wed, 24 Apr 2019 11:29:46 +0000 (13:29 +0200)]
Merge pull request #27609 from tspmelo/wip-iscsi-form-warn
mgr/dashboard: Fix iSCSI target form warning
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Mykola Golub [Wed, 24 Apr 2019 10:00:52 +0000 (13:00 +0300)]
Merge pull request #27720 from dillaman/wip-39407
rbd-mirror: clear out bufferlist prior to listing mirror images
Reviewed-by: Mykola Golub <mgolub@suse.com>
myoungwon oh [Wed, 24 Apr 2019 05:58:09 +0000 (14:58 +0900)]
src/test: remove unused objects to calculate accurate dedup ratio.
Signed-off-by: Myoungwon Oh <ommw@sk.com>
Kefu Chai [Tue, 23 Apr 2019 11:23:05 +0000 (19:23 +0800)]
denc: use scope_guard to close(fd)
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 22 Apr 2019 14:08:15 +0000 (22:08 +0800)]
crush,msg: silence -Wsign-compare warnings
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 22 Apr 2019 13:27:16 +0000 (21:27 +0800)]
denc: allow DencDumper to dump OOB buffer
if encoder encodes buffers into the bufferlist directly, the
contiguous_appender's out_of_band_offset would be greater than zero, in
that case, we should dump the bufferlist passed to appender instead of
just looking at the [pre_pos, post_pos).
Signed-off-by: Kefu Chai <kchai@redhat.com>
Kefu Chai [Mon, 22 Apr 2019 13:13:03 +0000 (21:13 +0800)]
denc: rewrite DENC_DUMP_{PRE,POST} using template
* for better readability
* for better debugging experience
* so we can `friend` it in a class in a follow-up change
Signed-off-by: Kefu Chai <kchai@redhat.com>
Xie Xingguo [Wed, 24 Apr 2019 00:44:08 +0000 (08:44 +0800)]
Merge pull request #27719 from xiexingguo/wip-crush-node-flags
mon/OSDMonitor: crush node flags - two fixes; add tests
Reviewed-by: Sage Weil <sage@redhat.com>
Casey Bodley [Tue, 23 Apr 2019 19:50:07 +0000 (15:50 -0400)]
Merge pull request #26107 from alimaredia/wip-dpp-rgw-put-obj-processor
rgw: log refactoring for putobj_processor
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Ali Maredia [Mon, 4 Mar 2019 20:55:11 +0000 (15:55 -0500)]
rgw: log refactoring for AtomicObjProcessor complete
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Sage Weil [Tue, 23 Apr 2019 18:08:29 +0000 (13:08 -0500)]
Merge PR #27628 into master
* refs/pull/27628/head:
common: optimize check_utf8
Reviewed-by: Kefu Chai <kchai@redhat.com>
Sage Weil [Tue, 23 Apr 2019 18:06:52 +0000 (13:06 -0500)]
Merge PR #27702 into master
* refs/pull/27702/head:
osd/PG: fix last_complete re-calculation on splitting
Reviewed-by: Neha Ojha <nojha@redhat.com>
Sage Weil [Tue, 23 Apr 2019 18:06:15 +0000 (13:06 -0500)]
Merge PR #27691 into master
* refs/pull/27691/head:
os/bluestore: fix out-of-bound access in bmap allocator.
os/bluestore: reproduce out-of-bound access for bmap allocator
Reviewed-by: Sage Weil <sage@redhat.com>
Sage Weil [Tue, 23 Apr 2019 18:05:44 +0000 (13:05 -0500)]
Merge PR #27707 into master
* refs/pull/27707/head:
common/util: handle long lines in /proc/cpuinfo
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2019 17:53:52 +0000 (10:53 -0700)]
Merge PR #27537 into master
* refs/pull/27537/head:
mds: better output of 'ceph health detail'
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2019 17:52:41 +0000 (10:52 -0700)]
Merge PR #27511 into master
* refs/pull/27511/head:
mds: fix SnapRealm::resolve_snapname for long name
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Tue, 23 Apr 2019 17:50:49 +0000 (10:50 -0700)]
Merge PR #27077 into master
* refs/pull/27077/head:
test: check listattr for snapshot btime entry
test: extend LibCephFS.Xattrs test
client: remove unused vxattr length helpers
client: fix _listxattr() vxattr buffer length calculation
test: add libcephfs snap.btime xattr test
client: add ceph.snap.btime vxattr
mds: carry snapshot creation time with InodeStat
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Casey Bodley [Tue, 23 Apr 2019 16:48:55 +0000 (12:48 -0400)]
Merge pull request #27725 from theanalyst/perf-counter-names
rgw: sync counters: drop spaces from counter names
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Abhishek Lekshmanan [Tue, 23 Apr 2019 15:27:08 +0000 (17:27 +0200)]
rgw: sync counters: drop spaces from counter names
Since this might break modules like prometheus and general json processing tools
aren't too happy with spaces.
Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
alfonsomthd [Tue, 23 Apr 2019 14:20:47 +0000 (16:20 +0200)]
mgr/prometheus: replace whitespaces in metrics' names
Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Jason Dillaman [Tue, 23 Apr 2019 14:07:26 +0000 (10:07 -0400)]
Merge pull request #27703 from tchaikov/wip-rbd-replay-denc
rbd_replay: call the member decode() explicitly
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Lenz Grimmer [Tue, 23 Apr 2019 13:54:23 +0000 (15:54 +0200)]
mgr/dashboard: Clean up TableComponent tests and code (#26784)
mgr/dashboard: Clean up TableComponent tests and code
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Alfredo Deza [Tue, 23 Apr 2019 13:09:09 +0000 (09:09 -0400)]
tools: pin the version of breathe that works with Python2
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Kefu Chai [Mon, 22 Apr 2019 11:26:54 +0000 (19:26 +0800)]
rbd_replay: call the member decode() explicitly
otherwise, the one defined using WRITE_RAW_ENCODER is called instead.
so in this change, rename the the member function which happens to have
the same signature with
decode(type &v, ::ceph::bufferlist::const_iterator& p)
where `type` is `__u8`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Jason Dillaman [Tue, 23 Apr 2019 12:24:52 +0000 (08:24 -0400)]
rbd-mirror: clear out bufferlist prior to listing mirror images
The second call to list mirrored images will fail deep within the
msgr code due to a "bad crc in data" error.
Fixes: http://tracker.ceph.com/issues/39407
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Tue, 23 Apr 2019 12:11:38 +0000 (08:11 -0400)]
Merge pull request #27521 from trociny/wip-rbd-remove-clone_v2-parent
librbd: optionally move parent image to trash on remove
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Tue, 23 Apr 2019 12:11:07 +0000 (08:11 -0400)]
Merge pull request #27484 from majianpeng/rbd-nbd
rbd-nbd: sscanf return 0 mean not-match
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Sage Weil [Tue, 9 Apr 2019 21:45:47 +0000 (16:45 -0500)]
doc/rados/operations/devices: document device failure prediction
Signed-off-by: Sage Weil <sage@redhat.com>
Casey Bodley [Tue, 23 Apr 2019 12:07:13 +0000 (08:07 -0400)]
Merge pull request #27697 from cbodley/wip-rgw-bucket-list-unordered
rgw: cls_bucket_list_unordered lists a single shard
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Sage Weil [Fri, 19 Apr 2019 21:41:04 +0000 (16:41 -0500)]
osd: make use of pg history and past_intervals in pg_create2 messages
If we get a mismatched epoch and past_intervals, error out early, or else
we'll end up asserting later in the PastIntervals code.
Signed-off-by: Sage Weil <sage@redhat.com>
Ashish Singh [Tue, 29 Jan 2019 09:33:18 +0000 (15:03 +0530)]
mgr/dashboard: Replace IP address validation with Python standard library functions
Instead of self-written validation methods to validate IPv4 and IPv6 addresses.
Use Python's standard library functions `ipaddress`.
Signed-off-by: Ashish Singh <assingh@redhat.com>
Kefu Chai [Tue, 23 Apr 2019 11:14:55 +0000 (19:14 +0800)]
Merge pull request #27713 from tchaikov/wip-24842
doc/rbd/rbd-cloudstack: update disk offering URL to new docs
Reviewed-by: Wido den Hollander <wido@42on.com>
xie xingguo [Tue, 23 Apr 2019 06:20:17 +0000 (14:20 +0800)]
qa: add crush-node-flags test
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Kefu Chai [Tue, 23 Apr 2019 05:26:45 +0000 (13:26 +0800)]
doc/rbd/rbd-cloudstack: update disk offering URL to new docs
point hyperlinks to latest
Signed-off-by: Kefu Chai <kchai@redhat.com>
xie xingguo [Tue, 23 Apr 2019 03:24:07 +0000 (11:24 +0800)]
mon/OSDMonitor: remove crush node flags too on "crush rm"
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
xie xingguo [Tue, 23 Apr 2019 03:03:04 +0000 (11:03 +0800)]
mon/OSDMonitor: make per-OSD no{out,down,in,out} flags prior to CRUSH nodes
This way we 'll be more compatible with older versions, and can effectively
reduce the map size for large clusters.
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
David Zafman [Wed, 3 Apr 2019 03:21:35 +0000 (20:21 -0700)]
test: Divergent testing of _merge_object_divergent_entries() cases
Case 1: A more recent update exists
Case 2: The first entry in the divergent sequence is a create
Case 3 NOT TESTED - Ohject currently missing
Case 4: We can rollback all of the entries
Case 5: We cannot rollback at least 1 of the entries
Support starting OSDs even when "noup" is set (don't wait for up).
Move create_ec_pool() to ceph-helpers.sh
Fixes: https://tracker.ceph.com/issues/39162
Signed-off-by: David Zafman <dzafman@redhat.com>
xie xingguo [Sat, 20 Apr 2019 08:34:12 +0000 (16:34 +0800)]
osd/PG: fix last_complete re-calculation on splitting
We add hard-limit for pg_logs now, which means we might keep trimming
old log entries irrespective of pg's current missing_set. This as a
result can cause the last_complete pointer moving far ahead of the real
on-disk version (the oldest need of missing_set, for instance) the
corresponding pg should have on splitting:
```
2019-04-19 06:41:52.559247
7efd4725c700 10 osd.2 271 Splitting pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 les/251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 unknown NOTIFY m=518 mbc={}] into 5.16
2019-04-19 06:41:52.561413
7efd4725c700 10 osd.2 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 c/f 251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 lcod 0'0 unknown NOTIFY m=261 mbc={}] release_backoffs [MIN,MAX)
```
For the above example, parent's last_complete cursor changed from **0'0** to
**238'300** directly due to the effort of trying to catch up the oldest
log entry changing when splitting was done. However, back into v12.2.9 primary
would still reference shard's last_complete field when trying to figure out all
possible locations of a currently missing object (see PG::MissingLoc::add_source_info):
```c++
if (oinfo.last_complete < need) {
if (omissing.is_missing(soid)) {
ldout(pg->cct, 10) << "search_for_missing " << soid << " " << need
<< " also missing on osd." << fromosd << dendl;
continue;
}
}
```
Hence a wrongly calculated last_complete could then make primary mis-consider
that a specific shard might have the authoritative object it currently
looking for:
```
2019-04-19 06:41:52.904163
7fd4cfb5a700 10 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}] proc_replica_log for osd.2: 5.6( v 270'943 lc 238'30
0 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/c/f 251/224/0 250/271/229) log((249'563,270'943], crt=270'943) missing(261 may_include_del
etes = 1)
2019-04-19 06:41:52.904645
7fd4cfb5a700 20 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}] after missing 5:
624c3a7a :::benchmark_data_smithi190
_39968_object1382:head need 226'110 have 0'0
2019-04-19 06:41:53.567820
7fd4d035b700 10 osd.5 pg_epoch: 272 pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=271/272 n=471 ec=223/223 lis/c 250/223 les/c/f
251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 unknown m=16 u=13 mbc={255={(1+0)=220,(2+0)=28}}] search_for_missing 5:624c3a
7a:::benchmark_data_smithi190_39968_object1382:head 226'110 is on osd.2
```
note that ```5:
624c3a7a :::benchmark_data_smithi190_39968_object1382:head 226'110```
was actually missing on both primary and shard osd.2 whereas primary insisted that
object should exist on shard osd.2!
https://github.com/ceph/ceph/pull/26175 posted an indirect fix
for the above problem by ignoring last_complete when checking the missing set,
but it should generally make more sense to fill in the last_complete field correctly
whenever possible.
Hence coming this additional fix.
Fixes: http://tracker.ceph.com/issues/26958
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Sage Weil [Mon, 22 Apr 2019 22:07:27 +0000 (17:07 -0500)]
Merge PR #27708 into master
* refs/pull/27708/head:
doc/governance: add cbodey
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Sage Weil [Mon, 22 Apr 2019 22:06:42 +0000 (17:06 -0500)]
Merge PR #27693 into master
* refs/pull/27693/head:
mgr/telemetry: default to reports every 24h; lower minimum
mgr/telemetry: exclude hostname field in crash reports
Reviewed-by: Dan Mick <dmick@redhat.com>
Sage Weil [Mon, 22 Apr 2019 20:37:11 +0000 (15:37 -0500)]
doc/governance: add cbodey
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Mon, 22 Apr 2019 17:51:41 +0000 (12:51 -0500)]
common/util: handle long lines in /proc/cpuinfo
Fixes: http://tracker.ceph.com/issues/38296
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Mon, 22 Apr 2019 17:36:29 +0000 (12:36 -0500)]
doc/dev/erasure-coded-pool: update
Signed-off-by: Sage Weil <sage@redhat.com>
Jason Dillaman [Thu, 18 Apr 2019 18:02:45 +0000 (14:02 -0400)]
qa/workunits/rbd: use more recent qemu-iotests that support Bionic
Fixes: http://tracker.ceph.com/issues/24668
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Sage Weil [Mon, 22 Apr 2019 16:20:55 +0000 (11:20 -0500)]
doc/rados/operations/erasure-code*: update default ec profile references
Signed-off-by: Sage Weil <sage@redhat.com>
Ali Maredia [Mon, 4 Mar 2019 18:42:37 +0000 (13:42 -0500)]
rgw: thread DoutPrefixProvider into fetch_remote_obj
This is for the AtomicObjProcessor declared there
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Ali Maredia [Thu, 24 Jan 2019 04:53:53 +0000 (23:53 -0500)]
rgw: log refactoring for putobj_processor
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Casey Bodley [Fri, 19 Apr 2019 22:38:47 +0000 (18:38 -0400)]
rgw: cls_bucket_list_unordered lists a single shard
CLSRGWIssueBucketList sends the request to every shard, but this loop
intended to list only the current_shard
Fixes: http://tracker.ceph.com/issues/39393
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Casey Bodley [Fri, 19 Apr 2019 22:37:35 +0000 (18:37 -0400)]
cls/rgw: expose cls_rgw_bucket_list_op for single shard
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Mykola Golub [Mon, 22 Apr 2019 15:13:02 +0000 (18:13 +0300)]
Merge pull request #27694 from dillaman/wip-39386
qa/suites/rbd: added writearound cache test permutations
Reviewed-by: Mykola Golub <mgolub@suse.com>
Mykola Golub [Mon, 22 Apr 2019 15:12:30 +0000 (18:12 +0300)]
Merge pull request #27682 from dillaman/wip-39031
librbd: async open/close should free ImageCtx before issuing callback
Reviewed-by: Mykola Golub <mgolub@suse.com>
Sage Weil [Mon, 22 Apr 2019 14:58:24 +0000 (09:58 -0500)]
Merge pull request #24744 from liewegas/wip-stale-prs
.github/stale.yml: warn at 60, close at 90; adjust message
Sage Weil [Fri, 19 Apr 2019 17:24:48 +0000 (12:24 -0500)]
mgr/telemetry: default to reports every 24h; lower minimum
Allow more frequent telemetry reports.
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 19 Apr 2019 17:09:36 +0000 (12:09 -0500)]
mgr/telemetry: exclude hostname field in crash reports
On some systems the hostname is a fully-qualified domain name and
(even when not a fqdn) may inadvertantly allow the cluster to be
identified.
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 19 Apr 2019 21:33:04 +0000 (16:33 -0500)]
mon/OSDMonitor: track history and past_intervals for creating pgs
PG create messages from mons are the last case where the OSD may have to
scan an unbounded number of old maps in order to construct a valid
pg_history_t and PastIntervals. Try to avoid making that a difficult
case by maintaining those structures on the monitor.
It is still possible that the mon may send a pg create message to the OSD
and it sits in a message queue for a very long time, but this would be a
very difficult situation to get into, and is no different from inter-OSD
messages that include history and past_intervals.
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 19 Apr 2019 21:01:20 +0000 (16:01 -0500)]
osd/osd_types: make PastIntervals pi_compact_rep print participants
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 19 Apr 2019 19:52:14 +0000 (14:52 -0500)]
osd/osd_types: take bare const OSDMap * to check_new_interval
Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 19 Apr 2019 18:56:25 +0000 (13:56 -0500)]
osd/osd_types: add pg_history_t ctor that takes creation epoch+stamp
Signed-off-by: Sage Weil <sage@redhat.com>