]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agomgr/DaemonServer: safe-to-destroy - do not consider irrelevant pgs 27962/head
xie xingguo [Sun, 5 May 2019 05:17:32 +0000 (13:17 +0800)]
mgr/DaemonServer: safe-to-destroy - do not consider irrelevant pgs

otherwise ceph osd safe-to-destroy would say NO even if we want to
destroy an down+out osd from an actually active+clean pool!

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #27626 into master
Sage Weil [Thu, 25 Apr 2019 01:40:17 +0000 (20:40 -0500)]
Merge PR #27626 into master

* refs/pull/27626/head:
mgr/dashboard: Do not default to 'admin' as Admin Resource

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Gabriel Brascher <gabriel@apache.org>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
6 years agoMerge PR #26184 into master
Sage Weil [Thu, 25 Apr 2019 01:38:19 +0000 (20:38 -0500)]
Merge PR #26184 into master

* refs/pull/26184/head:
mgr/dashboard: Replace IP address validation with Python standard library functions

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agoMerge PR #26730 into master
Sage Weil [Thu, 25 Apr 2019 01:38:02 +0000 (20:38 -0500)]
Merge PR #26730 into master

* refs/pull/26730/head:
src/test: remove unused objects to calculate accurate dedup ratio.
src/tools: fix counting total objects
src/tools: fix printing wrong offset.
src/tools: fix dedup tool's name
qa/workunit: add rabin chunk test
src/tool: add rabin fingerprint option to cephdeduptool
src/common: add rabin fingerprint class
Initail work for rabin fingerprint

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27566 into master
Sage Weil [Wed, 24 Apr 2019 22:00:57 +0000 (17:00 -0500)]
Merge PR #27566 into master

* refs/pull/27566/head:
auth: kill AuthUnknownSessionHandler
auth: kill AuthUnknown* (except AuthUnknownSessionHandler)
msg: remove old ms_* auth methods from Dispatcher interface
mon/MonClient: discard old challenge if authorizer is bad
msg/async/ProtocolV1: use AuthServer and AuthClient
mon/Monitor: handle v1 call into handle_auth_request
msg/Connection: add is_msgr2()
mon/MonClient: tolerate lack of authorizer for some dispatchers

Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge pull request #27579 from cbodley/wip-rgw-split-log-trim
Casey Bodley [Wed, 24 Apr 2019 20:07:33 +0000 (16:07 -0400)]
Merge pull request #27579 from cbodley/wip-rgw-split-log-trim

rgw: split mdlog/datalog trimming into separate files

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
6 years agoMerge pull request #26173 from cbodley/wip-rgw-async-putobjprocessor
Casey Bodley [Wed, 24 Apr 2019 19:47:59 +0000 (15:47 -0400)]
Merge pull request #26173 from cbodley/wip-rgw-async-putobjprocessor

rgw: add YieldingAioThrottle for async PutObj/GetObj

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
6 years agoauth: kill AuthUnknownSessionHandler 27566/head
Sage Weil [Fri, 12 Apr 2019 21:22:04 +0000 (16:22 -0500)]
auth: kill AuthUnknownSessionHandler

The only user is ProtocolV1, which passes in an in-use auth_method...
which must be either AUTH_NONE or AUTH_CEPHX since we successfully
authenticated at this point.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoauth: kill AuthUnknown* (except AuthUnknownSessionHandler)
Sage Weil [Fri, 12 Apr 2019 21:20:45 +0000 (16:20 -0500)]
auth: kill AuthUnknown* (except AuthUnknownSessionHandler)

These are unreferenced by any other code.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomsg: remove old ms_* auth methods from Dispatcher interface
Sage Weil [Fri, 12 Apr 2019 19:21:20 +0000 (14:21 -0500)]
msg: remove old ms_* auth methods from Dispatcher interface

In the mon there is an opportunity to further clean up the code around
get_authorizer().

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/MonClient: discard old challenge if authorizer is bad
Sage Weil [Tue, 23 Apr 2019 18:52:07 +0000 (13:52 -0500)]
mon/MonClient: discard old challenge if authorizer is bad

If we are a server/accepter, and we get a bad authorizer, discard our old
challenge, since the previous exchange has failed.  This way if the client
retries, their new authorize attempt won't conflict with our old challenge.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomsg/async/ProtocolV1: use AuthServer and AuthClient
Sage Weil [Fri, 12 Apr 2019 19:19:38 +0000 (14:19 -0500)]
msg/async/ProtocolV1: use AuthServer and AuthClient

Stop using the old ms_* auth methods and instead use the new interfaces
(like V2).

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27279 from dzafman/wip-divergent
David Zafman [Wed, 24 Apr 2019 17:58:11 +0000 (10:58 -0700)]
Merge pull request #27279 from dzafman/wip-divergent

Improvements to standalone tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27750 from tchaikov/wip-silence-gcc-warning
Kefu Chai [Wed, 24 Apr 2019 16:18:48 +0000 (00:18 +0800)]
Merge pull request #27750 from tchaikov/wip-silence-gcc-warning

test/objectstore: silence -Wsign-compare warning

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge PR #27721 into master
Sage Weil [Wed, 24 Apr 2019 15:48:17 +0000 (10:48 -0500)]
Merge PR #27721 into master

* refs/pull/27721/head:
tools: pin the version of breathe that works with Python2

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agotest/rgw: rgw throttle tests use mock operations 26173/head
Casey Bodley [Thu, 18 Apr 2019 13:18:37 +0000 (09:18 -0400)]
test/rgw: rgw throttle tests use mock operations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add optional_yield to Aio::librados_op()
Casey Bodley [Wed, 17 Apr 2019 21:04:55 +0000 (17:04 -0400)]
rgw: add optional_yield to Aio::librados_op()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add optional_yield to get_obj_data
Casey Bodley [Wed, 17 Apr 2019 21:04:37 +0000 (17:04 -0400)]
rgw: add optional_yield to get_obj_data

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add optional_yield to PutObjProcessors
Casey Bodley [Wed, 17 Apr 2019 21:03:54 +0000 (17:03 -0400)]
rgw: add optional_yield to PutObjProcessors

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27221 from LenzGr/update-backportscripts
Nathan Cutler [Wed, 24 Apr 2019 13:49:00 +0000 (15:49 +0200)]
Merge pull request #27221 from LenzGr/update-backportscripts

script: Obtain milestones via github API

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agorgw: read ops use make_throttle() with optional_yield
Casey Bodley [Mon, 28 Jan 2019 17:41:41 +0000 (12:41 -0500)]
rgw: read ops use make_throttle() with optional_yield

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: write ops use make_throttle() with optional_yield
Casey Bodley [Mon, 28 Jan 2019 18:12:11 +0000 (13:12 -0500)]
rgw: write ops use make_throttle() with optional_yield

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add make_throttle() for optionally-async AioThrottle
Casey Bodley [Mon, 28 Jan 2019 18:11:17 +0000 (13:11 -0500)]
rgw: add make_throttle() for optionally-async AioThrottle

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add YieldingAioThrottle
Casey Bodley [Wed, 10 Oct 2018 12:04:31 +0000 (08:04 -0400)]
rgw: add YieldingAioThrottle

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add BlockingAioThrottle
Casey Bodley [Wed, 10 Oct 2018 12:03:55 +0000 (08:03 -0400)]
rgw: add BlockingAioThrottle

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25692 from Liuchang0812/fix-26835
Casey Bodley [Wed, 24 Apr 2019 13:42:28 +0000 (09:42 -0400)]
Merge pull request #25692 from Liuchang0812/fix-26835

 rgw: returns LimitExceeded when user creates too many ACLs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25044 from tianshan/fix_36763
Casey Bodley [Wed, 24 Apr 2019 13:41:52 +0000 (09:41 -0400)]
Merge pull request #25044 from tianshan/fix_36763

rgw: set null version object acl issues

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #21154 from mslovy/rgw-admin-api
Casey Bodley [Wed, 24 Apr 2019 13:41:31 +0000 (09:41 -0400)]
Merge pull request #21154 from mslovy/rgw-admin-api

rgw:  update op_mask of user via admin rest api

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #17338 from tianshan/fix_roles_pool
Casey Bodley [Wed, 24 Apr 2019 13:40:36 +0000 (09:40 -0400)]
Merge pull request #17338 from tianshan/fix_roles_pool

rgw: add roles_pool in RGWZoneParams dump/decode json

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #17591 from shashalu/fix-bucket-list-versions
Casey Bodley [Wed, 24 Apr 2019 13:40:16 +0000 (09:40 -0400)]
Merge pull request #17591 from shashalu/fix-bucket-list-versions

rgw: list_bucket versions return NextVersionIdMarker = "null" if next_marker.instance is empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #27472 into master
Sage Weil [Wed, 24 Apr 2019 13:37:49 +0000 (08:37 -0500)]
Merge PR #27472 into master

* refs/pull/27472/head:
doc/rados/operations/devices: document device failure prediction

Reviewed-by: Rick Chen <rick.chen@prophetstor.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #27573 from iotcg/corpus
Kefu Chai [Wed, 24 Apr 2019 13:34:26 +0000 (21:34 +0800)]
Merge pull request #27573 from iotcg/corpus

import-generated.sh: use PATH to get ceph-dencoder

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #27696 into master
Sage Weil [Wed, 24 Apr 2019 13:15:19 +0000 (08:15 -0500)]
Merge PR #27696 into master

* refs/pull/27696/head:
osd: make use of pg history and past_intervals in pg_create2 messages
mon/OSDMonitor: track history and past_intervals for creating pgs
osd/osd_types: make PastIntervals pi_compact_rep print participants
osd/osd_types: take bare const OSDMap * to check_new_interval
osd/osd_types: add pg_history_t ctor that takes creation epoch+stamp

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27656 into master
Sage Weil [Wed, 24 Apr 2019 13:14:55 +0000 (08:14 -0500)]
Merge PR #27656 into master

* refs/pull/27656/head:
doc/dev/erasure-coded-pool: update
doc/rados/operations/erasure-code*: update default ec profile references
common/options: change default erasure-code-profile to k=2 m=2

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agotest/objectstore: silence -Wsign-compare warning 27750/head
Kefu Chai [Wed, 24 Apr 2019 12:58:35 +0000 (20:58 +0800)]
test/objectstore: silence -Wsign-compare warning

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27683 from dillaman/wip-24668
Mykola Golub [Wed, 24 Apr 2019 12:40:43 +0000 (15:40 +0300)]
Merge pull request #27683 from dillaman/wip-24668

qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoscript: Obtain milestones via github API 27221/head
Lenz Grimmer [Thu, 18 Apr 2019 11:33:22 +0000 (13:33 +0200)]
script: Obtain milestones via github API

Replace the hard-coded list of milestones with an API request
that obtains this data from github directly.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
6 years agoimport-generated.sh: correct access to ceph-dencoder 27573/head
Changcheng Liu [Sat, 13 Apr 2019 08:15:35 +0000 (16:15 +0800)]
import-generated.sh: correct access to ceph-dencoder

align with instructions in doc/dev/corpus.rst.
doc/dev/corpus.rst is under change by kchai@redhat.com
in PR 27552.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agoMerge pull request #27704 from tchaikov/wip-denc-dump
Kefu Chai [Wed, 24 Apr 2019 11:49:54 +0000 (19:49 +0800)]
Merge pull request #27704 from tchaikov/wip-denc-dump

denc: allow DencDumper to dump OOB buffer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: iSCSI: Limit members to 1 group (#27520)
Lenz Grimmer [Wed, 24 Apr 2019 11:41:03 +0000 (13:41 +0200)]
mgr/dashboard: iSCSI: Limit members to 1 group (#27520)

mgr/dashboard: iSCSI: Limit members to 1 group

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #27722 from rhcs-dashboard/39434-prometheus-replace-whitespaces
Lenz Grimmer [Wed, 24 Apr 2019 11:38:14 +0000 (13:38 +0200)]
Merge pull request #27722 from rhcs-dashboard/39434-prometheus-replace-whitespaces

mgr/prometheus: replace whitespaces in metrics' names

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #27678 from tspmelo/wip-iscsi-discovery-read
Lenz Grimmer [Wed, 24 Apr 2019 11:31:50 +0000 (13:31 +0200)]
Merge pull request #27678 from tspmelo/wip-iscsi-discovery-read

mgr/dashboard: Fix iSCSI Discovery user permissions

6 years agoMerge pull request #27609 from tspmelo/wip-iscsi-form-warn
Lenz Grimmer [Wed, 24 Apr 2019 11:29:46 +0000 (13:29 +0200)]
Merge pull request #27609 from tspmelo/wip-iscsi-form-warn

mgr/dashboard: Fix iSCSI target form warning

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #27720 from dillaman/wip-39407
Mykola Golub [Wed, 24 Apr 2019 10:00:52 +0000 (13:00 +0300)]
Merge pull request #27720 from dillaman/wip-39407

rbd-mirror: clear out bufferlist prior to listing mirror images

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agosrc/test: remove unused objects to calculate accurate dedup ratio. 26730/head
myoungwon oh [Wed, 24 Apr 2019 05:58:09 +0000 (14:58 +0900)]
src/test: remove unused objects to calculate accurate dedup ratio.

Signed-off-by: Myoungwon Oh <ommw@sk.com>
6 years agodenc: use scope_guard to close(fd) 27704/head
Kefu Chai [Tue, 23 Apr 2019 11:23:05 +0000 (19:23 +0800)]
denc: use scope_guard to close(fd)

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrush,msg: silence -Wsign-compare warnings
Kefu Chai [Mon, 22 Apr 2019 14:08:15 +0000 (22:08 +0800)]
crush,msg: silence -Wsign-compare warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodenc: allow DencDumper to dump OOB buffer
Kefu Chai [Mon, 22 Apr 2019 13:27:16 +0000 (21:27 +0800)]
denc: allow DencDumper to dump OOB buffer

if encoder encodes buffers into the bufferlist directly, the
contiguous_appender's out_of_band_offset would be greater than zero, in
that case, we should dump the bufferlist passed to appender instead of
just looking at the [pre_pos, post_pos).

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodenc: rewrite DENC_DUMP_{PRE,POST} using template
Kefu Chai [Mon, 22 Apr 2019 13:13:03 +0000 (21:13 +0800)]
denc: rewrite DENC_DUMP_{PRE,POST} using template

* for better readability
* for better debugging experience
* so we can `friend` it in a class in a follow-up change

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27719 from xiexingguo/wip-crush-node-flags
Xie Xingguo [Wed, 24 Apr 2019 00:44:08 +0000 (08:44 +0800)]
Merge pull request #27719 from xiexingguo/wip-crush-node-flags

mon/OSDMonitor: crush node flags - two fixes; add tests

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26107 from alimaredia/wip-dpp-rgw-put-obj-processor
Casey Bodley [Tue, 23 Apr 2019 19:50:07 +0000 (15:50 -0400)]
Merge pull request #26107 from alimaredia/wip-dpp-rgw-put-obj-processor

rgw: log refactoring for putobj_processor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: log refactoring for AtomicObjProcessor complete 26107/head
Ali Maredia [Mon, 4 Mar 2019 20:55:11 +0000 (15:55 -0500)]
rgw: log refactoring for AtomicObjProcessor complete

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agoMerge PR #27628 into master
Sage Weil [Tue, 23 Apr 2019 18:08:29 +0000 (13:08 -0500)]
Merge PR #27628 into master

* refs/pull/27628/head:
common: optimize check_utf8

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #27702 into master
Sage Weil [Tue, 23 Apr 2019 18:06:52 +0000 (13:06 -0500)]
Merge PR #27702 into master

* refs/pull/27702/head:
osd/PG: fix last_complete re-calculation on splitting

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27691 into master
Sage Weil [Tue, 23 Apr 2019 18:06:15 +0000 (13:06 -0500)]
Merge PR #27691 into master

* refs/pull/27691/head:
os/bluestore: fix out-of-bound access in bmap allocator.
os/bluestore: reproduce out-of-bound access for bmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27707 into master
Sage Weil [Tue, 23 Apr 2019 18:05:44 +0000 (13:05 -0500)]
Merge PR #27707 into master

* refs/pull/27707/head:
common/util: handle long lines in /proc/cpuinfo

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge PR #27537 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:53:52 +0000 (10:53 -0700)]
Merge PR #27537 into master

* refs/pull/27537/head:
mds: better output of 'ceph health detail'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27511 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:52:41 +0000 (10:52 -0700)]
Merge PR #27511 into master

* refs/pull/27511/head:
mds: fix SnapRealm::resolve_snapname for long name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27077 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:50:49 +0000 (10:50 -0700)]
Merge PR #27077 into master

* refs/pull/27077/head:
test: check listattr for snapshot btime entry
test: extend LibCephFS.Xattrs test
client: remove unused vxattr length helpers
client: fix _listxattr() vxattr buffer length calculation
test: add libcephfs snap.btime xattr test
client: add ceph.snap.btime vxattr
mds: carry snapshot creation time with InodeStat

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #27725 from theanalyst/perf-counter-names
Casey Bodley [Tue, 23 Apr 2019 16:48:55 +0000 (12:48 -0400)]
Merge pull request #27725 from theanalyst/perf-counter-names

rgw: sync counters: drop spaces from counter names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
6 years agorgw: sync counters: drop spaces from counter names 27725/head
Abhishek Lekshmanan [Tue, 23 Apr 2019 15:27:08 +0000 (17:27 +0200)]
rgw: sync counters: drop spaces from counter names

Since this might break modules like prometheus and general json processing tools
aren't too happy with spaces.

Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agomgr/prometheus: replace whitespaces in metrics' names 27722/head
alfonsomthd [Tue, 23 Apr 2019 14:20:47 +0000 (16:20 +0200)]
mgr/prometheus: replace whitespaces in metrics' names

Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
6 years agoMerge pull request #27703 from tchaikov/wip-rbd-replay-denc
Jason Dillaman [Tue, 23 Apr 2019 14:07:26 +0000 (10:07 -0400)]
Merge pull request #27703 from tchaikov/wip-rbd-replay-denc

rbd_replay: call the member decode() explicitly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agomgr/dashboard: Clean up TableComponent tests and code (#26784)
Lenz Grimmer [Tue, 23 Apr 2019 13:54:23 +0000 (15:54 +0200)]
mgr/dashboard: Clean up TableComponent tests and code (#26784)

mgr/dashboard: Clean up TableComponent tests and code

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agotools: pin the version of breathe that works with Python2 27721/head
Alfredo Deza [Tue, 23 Apr 2019 13:09:09 +0000 (09:09 -0400)]
tools: pin the version of breathe that works with Python2

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agorbd_replay: call the member decode() explicitly 27703/head
Kefu Chai [Mon, 22 Apr 2019 11:26:54 +0000 (19:26 +0800)]
rbd_replay: call the member decode() explicitly

otherwise, the one defined using WRITE_RAW_ENCODER is called instead.
so in this change, rename the the member function which happens to have
the same signature with

decode(type &v, ::ceph::bufferlist::const_iterator& p)

where `type` is `__u8`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorbd-mirror: clear out bufferlist prior to listing mirror images 27720/head
Jason Dillaman [Tue, 23 Apr 2019 12:24:52 +0000 (08:24 -0400)]
rbd-mirror: clear out bufferlist prior to listing mirror images

The second call to list mirrored images will fail deep within the
msgr code due to a "bad crc in data" error.

Fixes: http://tracker.ceph.com/issues/39407
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27521 from trociny/wip-rbd-remove-clone_v2-parent
Jason Dillaman [Tue, 23 Apr 2019 12:11:38 +0000 (08:11 -0400)]
Merge pull request #27521 from trociny/wip-rbd-remove-clone_v2-parent

librbd: optionally move parent image to trash on remove

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27484 from majianpeng/rbd-nbd
Jason Dillaman [Tue, 23 Apr 2019 12:11:07 +0000 (08:11 -0400)]
Merge pull request #27484 from majianpeng/rbd-nbd

rbd-nbd: sscanf return 0 mean not-match

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agodoc/rados/operations/devices: document device failure prediction 27472/head
Sage Weil [Tue, 9 Apr 2019 21:45:47 +0000 (16:45 -0500)]
doc/rados/operations/devices: document device failure prediction

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27697 from cbodley/wip-rgw-bucket-list-unordered
Casey Bodley [Tue, 23 Apr 2019 12:07:13 +0000 (08:07 -0400)]
Merge pull request #27697 from cbodley/wip-rgw-bucket-list-unordered

 rgw: cls_bucket_list_unordered lists a single shard

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoosd: make use of pg history and past_intervals in pg_create2 messages 27696/head
Sage Weil [Fri, 19 Apr 2019 21:41:04 +0000 (16:41 -0500)]
osd: make use of pg history and past_intervals in pg_create2 messages

If we get a mismatched epoch and past_intervals, error out early, or else
we'll end up asserting later in the PastIntervals code.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: Replace IP address validation with Python standard library functions 26184/head
Ashish Singh [Tue, 29 Jan 2019 09:33:18 +0000 (15:03 +0530)]
mgr/dashboard: Replace IP address validation with Python standard library functions

Instead of self-written validation methods to validate IPv4 and IPv6 addresses.
Use Python's standard library functions `ipaddress`.

Signed-off-by: Ashish Singh <assingh@redhat.com>
6 years agoMerge pull request #27713 from tchaikov/wip-24842
Kefu Chai [Tue, 23 Apr 2019 11:14:55 +0000 (19:14 +0800)]
Merge pull request #27713 from tchaikov/wip-24842

doc/rbd/rbd-cloudstack: update disk offering URL to new docs

Reviewed-by: Wido den Hollander <wido@42on.com>
6 years agoqa: add crush-node-flags test 27719/head
xie xingguo [Tue, 23 Apr 2019 06:20:17 +0000 (14:20 +0800)]
qa: add crush-node-flags test

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agodoc/rbd/rbd-cloudstack: update disk offering URL to new docs 27713/head
Kefu Chai [Tue, 23 Apr 2019 05:26:45 +0000 (13:26 +0800)]
doc/rbd/rbd-cloudstack: update disk offering URL to new docs

point hyperlinks to latest

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomon/OSDMonitor: remove crush node flags too on "crush rm"
xie xingguo [Tue, 23 Apr 2019 03:24:07 +0000 (11:24 +0800)]
mon/OSDMonitor: remove crush node flags too on "crush rm"

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agomon/OSDMonitor: make per-OSD no{out,down,in,out} flags prior to CRUSH nodes
xie xingguo [Tue, 23 Apr 2019 03:03:04 +0000 (11:03 +0800)]
mon/OSDMonitor: make per-OSD no{out,down,in,out} flags prior to CRUSH nodes

This way we 'll be more compatible with older versions, and can effectively
reduce the map size for large clusters.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agotest: Divergent testing of _merge_object_divergent_entries() cases 27279/head
David Zafman [Wed, 3 Apr 2019 03:21:35 +0000 (20:21 -0700)]
test: Divergent testing of _merge_object_divergent_entries() cases

Case 1: A more recent update exists
Case 2: The first entry in the divergent sequence is a create
Case 3  NOT TESTED - Ohject currently missing
Case 4: We can rollback all of the entries
Case 5: We cannot rollback at least 1 of the entries

Support starting OSDs even when "noup" is set (don't wait for up).
Move create_ec_pool() to ceph-helpers.sh

Fixes: https://tracker.ceph.com/issues/39162
Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agoosd/PG: fix last_complete re-calculation on splitting 27702/head
xie xingguo [Sat, 20 Apr 2019 08:34:12 +0000 (16:34 +0800)]
osd/PG: fix last_complete re-calculation on splitting

We add hard-limit for pg_logs now, which means we might keep trimming
old log entries irrespective of pg's current missing_set. This as a
result can cause the last_complete pointer moving far ahead of the real
on-disk version (the oldest need of missing_set, for instance) the
corresponding pg should have on splitting:

```
2019-04-19 06:41:52.559247 7efd4725c700 10 osd.2 271 Splitting pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 les/251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 unknown NOTIFY m=518 mbc={}] into 5.16
2019-04-19 06:41:52.561413 7efd4725c700 10 osd.2 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 c/f 251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 lcod 0'0 unknown NOTIFY m=261 mbc={}] release_backoffs [MIN,MAX)
```

For the above example, parent's last_complete cursor changed from **0'0** to
**238'300** directly due to the effort of trying to catch up the oldest
log entry changing when splitting was done. However, back into v12.2.9 primary
would still reference shard's last_complete field when trying to figure out all
possible locations of a currently missing object (see PG::MissingLoc::add_source_info):

```c++
  if (oinfo.last_complete < need) {
    if (omissing.is_missing(soid)) {
      ldout(pg->cct, 10) << "search_for_missing " << soid << " " << need
                         << " also missing on osd." << fromosd << dendl;
      continue;
    }
  }
```

Hence a wrongly calculated last_complete could then make primary mis-consider
that a specific shard might have the authoritative object it currently
looking for:

```
2019-04-19 06:41:52.904163 7fd4cfb5a700 10 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}] proc_replica_log for osd.2: 5.6( v 270'943 lc 238'30
0 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/c/f 251/224/0 250/271/229) log((249'563,270'943], crt=270'943) missing(261 may_include_del
etes = 1)
2019-04-19 06:41:52.904645 7fd4cfb5a700 20 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}]  after missing 5:624c3a7a:::benchmark_data_smithi190
_39968_object1382:head need 226'110 have 0'0
2019-04-19 06:41:53.567820 7fd4d035b700 10 osd.5 pg_epoch: 272 pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=271/272 n=471 ec=223/223 lis/c 250/223 les/c/f
251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 unknown m=16 u=13 mbc={255={(1+0)=220,(2+0)=28}}] search_for_missing 5:624c3a
7a:::benchmark_data_smithi190_39968_object1382:head 226'110 is on osd.2
```

note that ```5:624c3a7a:::benchmark_data_smithi190_39968_object1382:head 226'110```
was actually missing on both primary and shard osd.2 whereas primary insisted that
object should exist on shard osd.2!

https://github.com/ceph/ceph/pull/26175 posted an indirect fix
for the above problem by ignoring last_complete when checking the missing set,
but it should generally make more sense to fill in the last_complete field correctly
whenever possible.
Hence coming this additional fix.

Fixes: http://tracker.ceph.com/issues/26958
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #27708 into master
Sage Weil [Mon, 22 Apr 2019 22:07:27 +0000 (17:07 -0500)]
Merge PR #27708 into master

* refs/pull/27708/head:
doc/governance: add cbodey

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #27693 into master
Sage Weil [Mon, 22 Apr 2019 22:06:42 +0000 (17:06 -0500)]
Merge PR #27693 into master

* refs/pull/27693/head:
mgr/telemetry: default to reports every 24h; lower minimum
mgr/telemetry: exclude hostname field in crash reports

Reviewed-by: Dan Mick <dmick@redhat.com>
6 years agodoc/governance: add cbodey 27708/head
Sage Weil [Mon, 22 Apr 2019 20:37:11 +0000 (15:37 -0500)]
doc/governance: add cbodey

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/util: handle long lines in /proc/cpuinfo 27707/head
Sage Weil [Mon, 22 Apr 2019 17:51:41 +0000 (12:51 -0500)]
common/util: handle long lines in /proc/cpuinfo

Fixes: http://tracker.ceph.com/issues/38296
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/dev/erasure-coded-pool: update 27656/head
Sage Weil [Mon, 22 Apr 2019 17:36:29 +0000 (12:36 -0500)]
doc/dev/erasure-coded-pool: update

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/workunits/rbd: use more recent qemu-iotests that support Bionic 27683/head
Jason Dillaman [Thu, 18 Apr 2019 18:02:45 +0000 (14:02 -0400)]
qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Fixes: http://tracker.ceph.com/issues/24668
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agodoc/rados/operations/erasure-code*: update default ec profile references
Sage Weil [Mon, 22 Apr 2019 16:20:55 +0000 (11:20 -0500)]
doc/rados/operations/erasure-code*: update default ec profile references

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agorgw: thread DoutPrefixProvider into fetch_remote_obj
Ali Maredia [Mon, 4 Mar 2019 18:42:37 +0000 (13:42 -0500)]
rgw: thread DoutPrefixProvider into fetch_remote_obj

This is for the AtomicObjProcessor declared there

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agorgw: log refactoring for putobj_processor
Ali Maredia [Thu, 24 Jan 2019 04:53:53 +0000 (23:53 -0500)]
rgw: log refactoring for putobj_processor

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agorgw: cls_bucket_list_unordered lists a single shard 27697/head
Casey Bodley [Fri, 19 Apr 2019 22:38:47 +0000 (18:38 -0400)]
rgw: cls_bucket_list_unordered lists a single shard

CLSRGWIssueBucketList sends the request to every shard, but this loop
intended to list only the current_shard

Fixes: http://tracker.ceph.com/issues/39393
Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agocls/rgw: expose cls_rgw_bucket_list_op for single shard
Casey Bodley [Fri, 19 Apr 2019 22:37:35 +0000 (18:37 -0400)]
cls/rgw: expose cls_rgw_bucket_list_op for single shard

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27694 from dillaman/wip-39386
Mykola Golub [Mon, 22 Apr 2019 15:13:02 +0000 (18:13 +0300)]
Merge pull request #27694 from dillaman/wip-39386

qa/suites/rbd: added writearound cache test permutations

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #27682 from dillaman/wip-39031
Mykola Golub [Mon, 22 Apr 2019 15:12:30 +0000 (18:12 +0300)]
Merge pull request #27682 from dillaman/wip-39031

librbd: async open/close should free ImageCtx before issuing callback

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #24744 from liewegas/wip-stale-prs
Sage Weil [Mon, 22 Apr 2019 14:58:24 +0000 (09:58 -0500)]
Merge pull request #24744 from liewegas/wip-stale-prs

.github/stale.yml: warn at 60, close at 90; adjust message

6 years agomgr/telemetry: default to reports every 24h; lower minimum 27693/head
Sage Weil [Fri, 19 Apr 2019 17:24:48 +0000 (12:24 -0500)]
mgr/telemetry: default to reports every 24h; lower minimum

Allow more frequent telemetry reports.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/telemetry: exclude hostname field in crash reports
Sage Weil [Fri, 19 Apr 2019 17:09:36 +0000 (12:09 -0500)]
mgr/telemetry: exclude hostname field in crash reports

On some systems the hostname is a fully-qualified domain name and
(even when not a fqdn) may inadvertantly allow the cluster to be
identified.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/OSDMonitor: track history and past_intervals for creating pgs
Sage Weil [Fri, 19 Apr 2019 21:33:04 +0000 (16:33 -0500)]
mon/OSDMonitor: track history and past_intervals for creating pgs

PG create messages from mons are the last case where the OSD may have to
scan an unbounded number of old maps in order to construct a valid
pg_history_t and PastIntervals.  Try to avoid making that a difficult
case by maintaining those structures on the monitor.

It is still possible that the mon may send a pg create message to the OSD
and it sits in a message queue for a very long time, but this would be a
very difficult situation to get into, and is no different from inter-OSD
messages that include history and past_intervals.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd/osd_types: make PastIntervals pi_compact_rep print participants
Sage Weil [Fri, 19 Apr 2019 21:01:20 +0000 (16:01 -0500)]
osd/osd_types: make PastIntervals pi_compact_rep print participants

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd/osd_types: take bare const OSDMap * to check_new_interval
Sage Weil [Fri, 19 Apr 2019 19:52:14 +0000 (14:52 -0500)]
osd/osd_types: take bare const OSDMap * to check_new_interval

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd/osd_types: add pg_history_t ctor that takes creation epoch+stamp
Sage Weil [Fri, 19 Apr 2019 18:56:25 +0000 (13:56 -0500)]
osd/osd_types: add pg_history_t ctor that takes creation epoch+stamp

Signed-off-by: Sage Weil <sage@redhat.com>