]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agodoc: changelog: drop redundant nautilus and component names 27793/head
Abhishek Lekshmanan [Mon, 29 Apr 2019 18:34:22 +0000 (20:34 +0200)]
doc: changelog: drop redundant nautilus and component names

Also sort the entries

Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>
6 years agodoc: release notes for 14.2.1
Abhishek Lekshmanan [Thu, 25 Apr 2019 18:01:20 +0000 (20:01 +0200)]
doc: release notes for 14.2.1

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agoMerge pull request #27574 from iotcg/iwarp_rocev2
Kefu Chai [Thu, 25 Apr 2019 08:35:07 +0000 (16:35 +0800)]
Merge pull request #27574 from iotcg/iwarp_rocev2

msg/async/rdma: fix memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agomsg/async/rdma: use ibv_device_attr object type in Device class 27574/head
Changcheng Liu [Thu, 25 Apr 2019 05:10:54 +0000 (13:10 +0800)]
msg/async/rdma: use ibv_device_attr object type in Device class

1. Avoid to do memory management without using pointer to operate the
allocated space.
2. Since member type has been changed in class Device, it use member
domain operator "." to access to the sub-member in object.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agomsg/async/rdma: free memory when Infiniband obj is freed
Changcheng Liu [Thu, 25 Apr 2019 05:08:11 +0000 (13:08 +0800)]
msg/async/rdma: free memory when Infiniband obj is freed

device_list is allocated in Infiniband::init.
When Infiniband obj is destructed, free the allocated
space in case of memory leak.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agomsg/async/rdma: check allocated memory immediately before using it
Changcheng Liu [Thu, 25 Apr 2019 05:02:29 +0000 (13:02 +0800)]
msg/async/rdma: check allocated memory immediately before using it

The new "allocation functions" will through exception if it failes to
allocate space. There's no need to assert(devices) here for every
get_device operation.
Refer to: https://en.cppreference.com/w/cpp/memory/new/bad_alloc

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agomsg/async/rdma: free member hold memory when destructing obj
Changcheng Liu [Thu, 25 Apr 2019 05:00:45 +0000 (13:00 +0800)]
msg/async/rdma: free member hold memory when destructing obj

device_context_list hold the memory allocated through
ibv_get_device_list. release the memory through calling
ibv_free_device_list.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agomsg/sync/rdma: abort rdma operation when ibv_query_device failed
Changcheng Liu [Thu, 25 Apr 2019 04:58:30 +0000 (12:58 +0800)]
msg/sync/rdma: abort rdma operation when ibv_query_device failed

ibv_query_device return none zero when failing to query rdma
device. run ceph_abort in any case when ibv_query_device return
none zero.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agoMerge pull request #27741 from tchaikov/doc/dev/wip-corpus.rst
Kefu Chai [Thu, 25 Apr 2019 05:30:10 +0000 (13:30 +0800)]
Merge pull request #27741 from tchaikov/doc/dev/wip-corpus.rst

doc/dev/corpus.rst: correct instructions

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27626 into master
Sage Weil [Thu, 25 Apr 2019 01:40:17 +0000 (20:40 -0500)]
Merge PR #27626 into master

* refs/pull/27626/head:
mgr/dashboard: Do not default to 'admin' as Admin Resource

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Gabriel Brascher <gabriel@apache.org>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
6 years agoMerge PR #26184 into master
Sage Weil [Thu, 25 Apr 2019 01:38:19 +0000 (20:38 -0500)]
Merge PR #26184 into master

* refs/pull/26184/head:
mgr/dashboard: Replace IP address validation with Python standard library functions

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agoMerge PR #26730 into master
Sage Weil [Thu, 25 Apr 2019 01:38:02 +0000 (20:38 -0500)]
Merge PR #26730 into master

* refs/pull/26730/head:
src/test: remove unused objects to calculate accurate dedup ratio.
src/tools: fix counting total objects
src/tools: fix printing wrong offset.
src/tools: fix dedup tool's name
qa/workunit: add rabin chunk test
src/tool: add rabin fingerprint option to cephdeduptool
src/common: add rabin fingerprint class
Initail work for rabin fingerprint

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27566 into master
Sage Weil [Wed, 24 Apr 2019 22:00:57 +0000 (17:00 -0500)]
Merge PR #27566 into master

* refs/pull/27566/head:
auth: kill AuthUnknownSessionHandler
auth: kill AuthUnknown* (except AuthUnknownSessionHandler)
msg: remove old ms_* auth methods from Dispatcher interface
mon/MonClient: discard old challenge if authorizer is bad
msg/async/ProtocolV1: use AuthServer and AuthClient
mon/Monitor: handle v1 call into handle_auth_request
msg/Connection: add is_msgr2()
mon/MonClient: tolerate lack of authorizer for some dispatchers

Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge pull request #27579 from cbodley/wip-rgw-split-log-trim
Casey Bodley [Wed, 24 Apr 2019 20:07:33 +0000 (16:07 -0400)]
Merge pull request #27579 from cbodley/wip-rgw-split-log-trim

rgw: split mdlog/datalog trimming into separate files

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
6 years agoMerge pull request #26173 from cbodley/wip-rgw-async-putobjprocessor
Casey Bodley [Wed, 24 Apr 2019 19:47:59 +0000 (15:47 -0400)]
Merge pull request #26173 from cbodley/wip-rgw-async-putobjprocessor

rgw: add YieldingAioThrottle for async PutObj/GetObj

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
6 years agoauth: kill AuthUnknownSessionHandler 27566/head
Sage Weil [Fri, 12 Apr 2019 21:22:04 +0000 (16:22 -0500)]
auth: kill AuthUnknownSessionHandler

The only user is ProtocolV1, which passes in an in-use auth_method...
which must be either AUTH_NONE or AUTH_CEPHX since we successfully
authenticated at this point.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoauth: kill AuthUnknown* (except AuthUnknownSessionHandler)
Sage Weil [Fri, 12 Apr 2019 21:20:45 +0000 (16:20 -0500)]
auth: kill AuthUnknown* (except AuthUnknownSessionHandler)

These are unreferenced by any other code.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomsg: remove old ms_* auth methods from Dispatcher interface
Sage Weil [Fri, 12 Apr 2019 19:21:20 +0000 (14:21 -0500)]
msg: remove old ms_* auth methods from Dispatcher interface

In the mon there is an opportunity to further clean up the code around
get_authorizer().

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/MonClient: discard old challenge if authorizer is bad
Sage Weil [Tue, 23 Apr 2019 18:52:07 +0000 (13:52 -0500)]
mon/MonClient: discard old challenge if authorizer is bad

If we are a server/accepter, and we get a bad authorizer, discard our old
challenge, since the previous exchange has failed.  This way if the client
retries, their new authorize attempt won't conflict with our old challenge.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomsg/async/ProtocolV1: use AuthServer and AuthClient
Sage Weil [Fri, 12 Apr 2019 19:19:38 +0000 (14:19 -0500)]
msg/async/ProtocolV1: use AuthServer and AuthClient

Stop using the old ms_* auth methods and instead use the new interfaces
(like V2).

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27279 from dzafman/wip-divergent
David Zafman [Wed, 24 Apr 2019 17:58:11 +0000 (10:58 -0700)]
Merge pull request #27279 from dzafman/wip-divergent

Improvements to standalone tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27750 from tchaikov/wip-silence-gcc-warning
Kefu Chai [Wed, 24 Apr 2019 16:18:48 +0000 (00:18 +0800)]
Merge pull request #27750 from tchaikov/wip-silence-gcc-warning

test/objectstore: silence -Wsign-compare warning

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge PR #27721 into master
Sage Weil [Wed, 24 Apr 2019 15:48:17 +0000 (10:48 -0500)]
Merge PR #27721 into master

* refs/pull/27721/head:
tools: pin the version of breathe that works with Python2

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agotest/rgw: rgw throttle tests use mock operations 26173/head
Casey Bodley [Thu, 18 Apr 2019 13:18:37 +0000 (09:18 -0400)]
test/rgw: rgw throttle tests use mock operations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add optional_yield to Aio::librados_op()
Casey Bodley [Wed, 17 Apr 2019 21:04:55 +0000 (17:04 -0400)]
rgw: add optional_yield to Aio::librados_op()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add optional_yield to get_obj_data
Casey Bodley [Wed, 17 Apr 2019 21:04:37 +0000 (17:04 -0400)]
rgw: add optional_yield to get_obj_data

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add optional_yield to PutObjProcessors
Casey Bodley [Wed, 17 Apr 2019 21:03:54 +0000 (17:03 -0400)]
rgw: add optional_yield to PutObjProcessors

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27221 from LenzGr/update-backportscripts
Nathan Cutler [Wed, 24 Apr 2019 13:49:00 +0000 (15:49 +0200)]
Merge pull request #27221 from LenzGr/update-backportscripts

script: Obtain milestones via github API

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agorgw: read ops use make_throttle() with optional_yield
Casey Bodley [Mon, 28 Jan 2019 17:41:41 +0000 (12:41 -0500)]
rgw: read ops use make_throttle() with optional_yield

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: write ops use make_throttle() with optional_yield
Casey Bodley [Mon, 28 Jan 2019 18:12:11 +0000 (13:12 -0500)]
rgw: write ops use make_throttle() with optional_yield

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add make_throttle() for optionally-async AioThrottle
Casey Bodley [Mon, 28 Jan 2019 18:11:17 +0000 (13:11 -0500)]
rgw: add make_throttle() for optionally-async AioThrottle

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add YieldingAioThrottle
Casey Bodley [Wed, 10 Oct 2018 12:04:31 +0000 (08:04 -0400)]
rgw: add YieldingAioThrottle

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: add BlockingAioThrottle
Casey Bodley [Wed, 10 Oct 2018 12:03:55 +0000 (08:03 -0400)]
rgw: add BlockingAioThrottle

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25692 from Liuchang0812/fix-26835
Casey Bodley [Wed, 24 Apr 2019 13:42:28 +0000 (09:42 -0400)]
Merge pull request #25692 from Liuchang0812/fix-26835

 rgw: returns LimitExceeded when user creates too many ACLs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25044 from tianshan/fix_36763
Casey Bodley [Wed, 24 Apr 2019 13:41:52 +0000 (09:41 -0400)]
Merge pull request #25044 from tianshan/fix_36763

rgw: set null version object acl issues

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #21154 from mslovy/rgw-admin-api
Casey Bodley [Wed, 24 Apr 2019 13:41:31 +0000 (09:41 -0400)]
Merge pull request #21154 from mslovy/rgw-admin-api

rgw:  update op_mask of user via admin rest api

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #17338 from tianshan/fix_roles_pool
Casey Bodley [Wed, 24 Apr 2019 13:40:36 +0000 (09:40 -0400)]
Merge pull request #17338 from tianshan/fix_roles_pool

rgw: add roles_pool in RGWZoneParams dump/decode json

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #17591 from shashalu/fix-bucket-list-versions
Casey Bodley [Wed, 24 Apr 2019 13:40:16 +0000 (09:40 -0400)]
Merge pull request #17591 from shashalu/fix-bucket-list-versions

rgw: list_bucket versions return NextVersionIdMarker = "null" if next_marker.instance is empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #27472 into master
Sage Weil [Wed, 24 Apr 2019 13:37:49 +0000 (08:37 -0500)]
Merge PR #27472 into master

* refs/pull/27472/head:
doc/rados/operations/devices: document device failure prediction

Reviewed-by: Rick Chen <rick.chen@prophetstor.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #27573 from iotcg/corpus
Kefu Chai [Wed, 24 Apr 2019 13:34:26 +0000 (21:34 +0800)]
Merge pull request #27573 from iotcg/corpus

import-generated.sh: use PATH to get ceph-dencoder

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #27696 into master
Sage Weil [Wed, 24 Apr 2019 13:15:19 +0000 (08:15 -0500)]
Merge PR #27696 into master

* refs/pull/27696/head:
osd: make use of pg history and past_intervals in pg_create2 messages
mon/OSDMonitor: track history and past_intervals for creating pgs
osd/osd_types: make PastIntervals pi_compact_rep print participants
osd/osd_types: take bare const OSDMap * to check_new_interval
osd/osd_types: add pg_history_t ctor that takes creation epoch+stamp

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27656 into master
Sage Weil [Wed, 24 Apr 2019 13:14:55 +0000 (08:14 -0500)]
Merge PR #27656 into master

* refs/pull/27656/head:
doc/dev/erasure-coded-pool: update
doc/rados/operations/erasure-code*: update default ec profile references
common/options: change default erasure-code-profile to k=2 m=2

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agotest/objectstore: silence -Wsign-compare warning 27750/head
Kefu Chai [Wed, 24 Apr 2019 12:58:35 +0000 (20:58 +0800)]
test/objectstore: silence -Wsign-compare warning

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27683 from dillaman/wip-24668
Mykola Golub [Wed, 24 Apr 2019 12:40:43 +0000 (15:40 +0300)]
Merge pull request #27683 from dillaman/wip-24668

qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoscript: Obtain milestones via github API 27221/head
Lenz Grimmer [Thu, 18 Apr 2019 11:33:22 +0000 (13:33 +0200)]
script: Obtain milestones via github API

Replace the hard-coded list of milestones with an API request
that obtains this data from github directly.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
6 years agoimport-generated.sh: correct access to ceph-dencoder 27573/head
Changcheng Liu [Sat, 13 Apr 2019 08:15:35 +0000 (16:15 +0800)]
import-generated.sh: correct access to ceph-dencoder

align with instructions in doc/dev/corpus.rst.
doc/dev/corpus.rst is under change by kchai@redhat.com
in PR 27552.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agodoc/dev/corpus.rst: correct instructions 27741/head
Kefu Chai [Mon, 22 Apr 2019 13:26:39 +0000 (21:26 +0800)]
doc/dev/corpus.rst: correct instructions

* should keep all OSD up and running, otherwise some tests will wait for
  healthy cluster for ever.
* should start 2 MGR for an active-standby setting -- better coverage
this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27704 from tchaikov/wip-denc-dump
Kefu Chai [Wed, 24 Apr 2019 11:49:54 +0000 (19:49 +0800)]
Merge pull request #27704 from tchaikov/wip-denc-dump

denc: allow DencDumper to dump OOB buffer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: iSCSI: Limit members to 1 group (#27520)
Lenz Grimmer [Wed, 24 Apr 2019 11:41:03 +0000 (13:41 +0200)]
mgr/dashboard: iSCSI: Limit members to 1 group (#27520)

mgr/dashboard: iSCSI: Limit members to 1 group

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #27722 from rhcs-dashboard/39434-prometheus-replace-whitespaces
Lenz Grimmer [Wed, 24 Apr 2019 11:38:14 +0000 (13:38 +0200)]
Merge pull request #27722 from rhcs-dashboard/39434-prometheus-replace-whitespaces

mgr/prometheus: replace whitespaces in metrics' names

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #27678 from tspmelo/wip-iscsi-discovery-read
Lenz Grimmer [Wed, 24 Apr 2019 11:31:50 +0000 (13:31 +0200)]
Merge pull request #27678 from tspmelo/wip-iscsi-discovery-read

mgr/dashboard: Fix iSCSI Discovery user permissions

6 years agoMerge pull request #27609 from tspmelo/wip-iscsi-form-warn
Lenz Grimmer [Wed, 24 Apr 2019 11:29:46 +0000 (13:29 +0200)]
Merge pull request #27609 from tspmelo/wip-iscsi-form-warn

mgr/dashboard: Fix iSCSI target form warning

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #27720 from dillaman/wip-39407
Mykola Golub [Wed, 24 Apr 2019 10:00:52 +0000 (13:00 +0300)]
Merge pull request #27720 from dillaman/wip-39407

rbd-mirror: clear out bufferlist prior to listing mirror images

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agosrc/test: remove unused objects to calculate accurate dedup ratio. 26730/head
myoungwon oh [Wed, 24 Apr 2019 05:58:09 +0000 (14:58 +0900)]
src/test: remove unused objects to calculate accurate dedup ratio.

Signed-off-by: Myoungwon Oh <ommw@sk.com>
6 years agodenc: use scope_guard to close(fd) 27704/head
Kefu Chai [Tue, 23 Apr 2019 11:23:05 +0000 (19:23 +0800)]
denc: use scope_guard to close(fd)

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrush,msg: silence -Wsign-compare warnings
Kefu Chai [Mon, 22 Apr 2019 14:08:15 +0000 (22:08 +0800)]
crush,msg: silence -Wsign-compare warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodenc: allow DencDumper to dump OOB buffer
Kefu Chai [Mon, 22 Apr 2019 13:27:16 +0000 (21:27 +0800)]
denc: allow DencDumper to dump OOB buffer

if encoder encodes buffers into the bufferlist directly, the
contiguous_appender's out_of_band_offset would be greater than zero, in
that case, we should dump the bufferlist passed to appender instead of
just looking at the [pre_pos, post_pos).

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodenc: rewrite DENC_DUMP_{PRE,POST} using template
Kefu Chai [Mon, 22 Apr 2019 13:13:03 +0000 (21:13 +0800)]
denc: rewrite DENC_DUMP_{PRE,POST} using template

* for better readability
* for better debugging experience
* so we can `friend` it in a class in a follow-up change

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27719 from xiexingguo/wip-crush-node-flags
Xie Xingguo [Wed, 24 Apr 2019 00:44:08 +0000 (08:44 +0800)]
Merge pull request #27719 from xiexingguo/wip-crush-node-flags

mon/OSDMonitor: crush node flags - two fixes; add tests

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26107 from alimaredia/wip-dpp-rgw-put-obj-processor
Casey Bodley [Tue, 23 Apr 2019 19:50:07 +0000 (15:50 -0400)]
Merge pull request #26107 from alimaredia/wip-dpp-rgw-put-obj-processor

rgw: log refactoring for putobj_processor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: log refactoring for AtomicObjProcessor complete 26107/head
Ali Maredia [Mon, 4 Mar 2019 20:55:11 +0000 (15:55 -0500)]
rgw: log refactoring for AtomicObjProcessor complete

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agoMerge PR #27628 into master
Sage Weil [Tue, 23 Apr 2019 18:08:29 +0000 (13:08 -0500)]
Merge PR #27628 into master

* refs/pull/27628/head:
common: optimize check_utf8

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #27702 into master
Sage Weil [Tue, 23 Apr 2019 18:06:52 +0000 (13:06 -0500)]
Merge PR #27702 into master

* refs/pull/27702/head:
osd/PG: fix last_complete re-calculation on splitting

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27691 into master
Sage Weil [Tue, 23 Apr 2019 18:06:15 +0000 (13:06 -0500)]
Merge PR #27691 into master

* refs/pull/27691/head:
os/bluestore: fix out-of-bound access in bmap allocator.
os/bluestore: reproduce out-of-bound access for bmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27707 into master
Sage Weil [Tue, 23 Apr 2019 18:05:44 +0000 (13:05 -0500)]
Merge PR #27707 into master

* refs/pull/27707/head:
common/util: handle long lines in /proc/cpuinfo

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge PR #27537 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:53:52 +0000 (10:53 -0700)]
Merge PR #27537 into master

* refs/pull/27537/head:
mds: better output of 'ceph health detail'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27511 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:52:41 +0000 (10:52 -0700)]
Merge PR #27511 into master

* refs/pull/27511/head:
mds: fix SnapRealm::resolve_snapname for long name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27077 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:50:49 +0000 (10:50 -0700)]
Merge PR #27077 into master

* refs/pull/27077/head:
test: check listattr for snapshot btime entry
test: extend LibCephFS.Xattrs test
client: remove unused vxattr length helpers
client: fix _listxattr() vxattr buffer length calculation
test: add libcephfs snap.btime xattr test
client: add ceph.snap.btime vxattr
mds: carry snapshot creation time with InodeStat

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #27725 from theanalyst/perf-counter-names
Casey Bodley [Tue, 23 Apr 2019 16:48:55 +0000 (12:48 -0400)]
Merge pull request #27725 from theanalyst/perf-counter-names

rgw: sync counters: drop spaces from counter names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
6 years agorgw: sync counters: drop spaces from counter names 27725/head
Abhishek Lekshmanan [Tue, 23 Apr 2019 15:27:08 +0000 (17:27 +0200)]
rgw: sync counters: drop spaces from counter names

Since this might break modules like prometheus and general json processing tools
aren't too happy with spaces.

Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agomgr/prometheus: replace whitespaces in metrics' names 27722/head
alfonsomthd [Tue, 23 Apr 2019 14:20:47 +0000 (16:20 +0200)]
mgr/prometheus: replace whitespaces in metrics' names

Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
6 years agoMerge pull request #27703 from tchaikov/wip-rbd-replay-denc
Jason Dillaman [Tue, 23 Apr 2019 14:07:26 +0000 (10:07 -0400)]
Merge pull request #27703 from tchaikov/wip-rbd-replay-denc

rbd_replay: call the member decode() explicitly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agomgr/dashboard: Clean up TableComponent tests and code (#26784)
Lenz Grimmer [Tue, 23 Apr 2019 13:54:23 +0000 (15:54 +0200)]
mgr/dashboard: Clean up TableComponent tests and code (#26784)

mgr/dashboard: Clean up TableComponent tests and code

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agotools: pin the version of breathe that works with Python2 27721/head
Alfredo Deza [Tue, 23 Apr 2019 13:09:09 +0000 (09:09 -0400)]
tools: pin the version of breathe that works with Python2

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agorbd_replay: call the member decode() explicitly 27703/head
Kefu Chai [Mon, 22 Apr 2019 11:26:54 +0000 (19:26 +0800)]
rbd_replay: call the member decode() explicitly

otherwise, the one defined using WRITE_RAW_ENCODER is called instead.
so in this change, rename the the member function which happens to have
the same signature with

decode(type &v, ::ceph::bufferlist::const_iterator& p)

where `type` is `__u8`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorbd-mirror: clear out bufferlist prior to listing mirror images 27720/head
Jason Dillaman [Tue, 23 Apr 2019 12:24:52 +0000 (08:24 -0400)]
rbd-mirror: clear out bufferlist prior to listing mirror images

The second call to list mirrored images will fail deep within the
msgr code due to a "bad crc in data" error.

Fixes: http://tracker.ceph.com/issues/39407
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27521 from trociny/wip-rbd-remove-clone_v2-parent
Jason Dillaman [Tue, 23 Apr 2019 12:11:38 +0000 (08:11 -0400)]
Merge pull request #27521 from trociny/wip-rbd-remove-clone_v2-parent

librbd: optionally move parent image to trash on remove

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27484 from majianpeng/rbd-nbd
Jason Dillaman [Tue, 23 Apr 2019 12:11:07 +0000 (08:11 -0400)]
Merge pull request #27484 from majianpeng/rbd-nbd

rbd-nbd: sscanf return 0 mean not-match

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agodoc/rados/operations/devices: document device failure prediction 27472/head
Sage Weil [Tue, 9 Apr 2019 21:45:47 +0000 (16:45 -0500)]
doc/rados/operations/devices: document device failure prediction

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27697 from cbodley/wip-rgw-bucket-list-unordered
Casey Bodley [Tue, 23 Apr 2019 12:07:13 +0000 (08:07 -0400)]
Merge pull request #27697 from cbodley/wip-rgw-bucket-list-unordered

 rgw: cls_bucket_list_unordered lists a single shard

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoosd: make use of pg history and past_intervals in pg_create2 messages 27696/head
Sage Weil [Fri, 19 Apr 2019 21:41:04 +0000 (16:41 -0500)]
osd: make use of pg history and past_intervals in pg_create2 messages

If we get a mismatched epoch and past_intervals, error out early, or else
we'll end up asserting later in the PastIntervals code.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: Replace IP address validation with Python standard library functions 26184/head
Ashish Singh [Tue, 29 Jan 2019 09:33:18 +0000 (15:03 +0530)]
mgr/dashboard: Replace IP address validation with Python standard library functions

Instead of self-written validation methods to validate IPv4 and IPv6 addresses.
Use Python's standard library functions `ipaddress`.

Signed-off-by: Ashish Singh <assingh@redhat.com>
6 years agoMerge pull request #27713 from tchaikov/wip-24842
Kefu Chai [Tue, 23 Apr 2019 11:14:55 +0000 (19:14 +0800)]
Merge pull request #27713 from tchaikov/wip-24842

doc/rbd/rbd-cloudstack: update disk offering URL to new docs

Reviewed-by: Wido den Hollander <wido@42on.com>
6 years agoqa: add crush-node-flags test 27719/head
xie xingguo [Tue, 23 Apr 2019 06:20:17 +0000 (14:20 +0800)]
qa: add crush-node-flags test

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agodoc/rbd/rbd-cloudstack: update disk offering URL to new docs 27713/head
Kefu Chai [Tue, 23 Apr 2019 05:26:45 +0000 (13:26 +0800)]
doc/rbd/rbd-cloudstack: update disk offering URL to new docs

point hyperlinks to latest

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomon/OSDMonitor: remove crush node flags too on "crush rm"
xie xingguo [Tue, 23 Apr 2019 03:24:07 +0000 (11:24 +0800)]
mon/OSDMonitor: remove crush node flags too on "crush rm"

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agomon/OSDMonitor: make per-OSD no{out,down,in,out} flags prior to CRUSH nodes
xie xingguo [Tue, 23 Apr 2019 03:03:04 +0000 (11:03 +0800)]
mon/OSDMonitor: make per-OSD no{out,down,in,out} flags prior to CRUSH nodes

This way we 'll be more compatible with older versions, and can effectively
reduce the map size for large clusters.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agotest: Divergent testing of _merge_object_divergent_entries() cases 27279/head
David Zafman [Wed, 3 Apr 2019 03:21:35 +0000 (20:21 -0700)]
test: Divergent testing of _merge_object_divergent_entries() cases

Case 1: A more recent update exists
Case 2: The first entry in the divergent sequence is a create
Case 3  NOT TESTED - Ohject currently missing
Case 4: We can rollback all of the entries
Case 5: We cannot rollback at least 1 of the entries

Support starting OSDs even when "noup" is set (don't wait for up).
Move create_ec_pool() to ceph-helpers.sh

Fixes: https://tracker.ceph.com/issues/39162
Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agoosd/PG: fix last_complete re-calculation on splitting 27702/head
xie xingguo [Sat, 20 Apr 2019 08:34:12 +0000 (16:34 +0800)]
osd/PG: fix last_complete re-calculation on splitting

We add hard-limit for pg_logs now, which means we might keep trimming
old log entries irrespective of pg's current missing_set. This as a
result can cause the last_complete pointer moving far ahead of the real
on-disk version (the oldest need of missing_set, for instance) the
corresponding pg should have on splitting:

```
2019-04-19 06:41:52.559247 7efd4725c700 10 osd.2 271 Splitting pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 les/251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 unknown NOTIFY m=518 mbc={}] into 5.16
2019-04-19 06:41:52.561413 7efd4725c700 10 osd.2 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 c/f 251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 lcod 0'0 unknown NOTIFY m=261 mbc={}] release_backoffs [MIN,MAX)
```

For the above example, parent's last_complete cursor changed from **0'0** to
**238'300** directly due to the effort of trying to catch up the oldest
log entry changing when splitting was done. However, back into v12.2.9 primary
would still reference shard's last_complete field when trying to figure out all
possible locations of a currently missing object (see PG::MissingLoc::add_source_info):

```c++
  if (oinfo.last_complete < need) {
    if (omissing.is_missing(soid)) {
      ldout(pg->cct, 10) << "search_for_missing " << soid << " " << need
                         << " also missing on osd." << fromosd << dendl;
      continue;
    }
  }
```

Hence a wrongly calculated last_complete could then make primary mis-consider
that a specific shard might have the authoritative object it currently
looking for:

```
2019-04-19 06:41:52.904163 7fd4cfb5a700 10 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}] proc_replica_log for osd.2: 5.6( v 270'943 lc 238'30
0 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/c/f 251/224/0 250/271/229) log((249'563,270'943], crt=270'943) missing(261 may_include_del
etes = 1)
2019-04-19 06:41:52.904645 7fd4cfb5a700 20 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}]  after missing 5:624c3a7a:::benchmark_data_smithi190
_39968_object1382:head need 226'110 have 0'0
2019-04-19 06:41:53.567820 7fd4d035b700 10 osd.5 pg_epoch: 272 pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=271/272 n=471 ec=223/223 lis/c 250/223 les/c/f
251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 unknown m=16 u=13 mbc={255={(1+0)=220,(2+0)=28}}] search_for_missing 5:624c3a
7a:::benchmark_data_smithi190_39968_object1382:head 226'110 is on osd.2
```

note that ```5:624c3a7a:::benchmark_data_smithi190_39968_object1382:head 226'110```
was actually missing on both primary and shard osd.2 whereas primary insisted that
object should exist on shard osd.2!

https://github.com/ceph/ceph/pull/26175 posted an indirect fix
for the above problem by ignoring last_complete when checking the missing set,
but it should generally make more sense to fill in the last_complete field correctly
whenever possible.
Hence coming this additional fix.

Fixes: http://tracker.ceph.com/issues/26958
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #27708 into master
Sage Weil [Mon, 22 Apr 2019 22:07:27 +0000 (17:07 -0500)]
Merge PR #27708 into master

* refs/pull/27708/head:
doc/governance: add cbodey

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #27693 into master
Sage Weil [Mon, 22 Apr 2019 22:06:42 +0000 (17:06 -0500)]
Merge PR #27693 into master

* refs/pull/27693/head:
mgr/telemetry: default to reports every 24h; lower minimum
mgr/telemetry: exclude hostname field in crash reports

Reviewed-by: Dan Mick <dmick@redhat.com>
6 years agodoc/governance: add cbodey 27708/head
Sage Weil [Mon, 22 Apr 2019 20:37:11 +0000 (15:37 -0500)]
doc/governance: add cbodey

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/util: handle long lines in /proc/cpuinfo 27707/head
Sage Weil [Mon, 22 Apr 2019 17:51:41 +0000 (12:51 -0500)]
common/util: handle long lines in /proc/cpuinfo

Fixes: http://tracker.ceph.com/issues/38296
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/dev/erasure-coded-pool: update 27656/head
Sage Weil [Mon, 22 Apr 2019 17:36:29 +0000 (12:36 -0500)]
doc/dev/erasure-coded-pool: update

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/workunits/rbd: use more recent qemu-iotests that support Bionic 27683/head
Jason Dillaman [Thu, 18 Apr 2019 18:02:45 +0000 (14:02 -0400)]
qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Fixes: http://tracker.ceph.com/issues/24668
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agodoc/rados/operations/erasure-code*: update default ec profile references
Sage Weil [Mon, 22 Apr 2019 16:20:55 +0000 (11:20 -0500)]
doc/rados/operations/erasure-code*: update default ec profile references

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agorgw: thread DoutPrefixProvider into fetch_remote_obj
Ali Maredia [Mon, 4 Mar 2019 18:42:37 +0000 (13:42 -0500)]
rgw: thread DoutPrefixProvider into fetch_remote_obj

This is for the AtomicObjProcessor declared there

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agorgw: log refactoring for putobj_processor
Ali Maredia [Thu, 24 Jan 2019 04:53:53 +0000 (23:53 -0500)]
rgw: log refactoring for putobj_processor

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agorgw: cls_bucket_list_unordered lists a single shard 27697/head
Casey Bodley [Fri, 19 Apr 2019 22:38:47 +0000 (18:38 -0400)]
rgw: cls_bucket_list_unordered lists a single shard

CLSRGWIssueBucketList sends the request to every shard, but this loop
intended to list only the current_shard

Fixes: http://tracker.ceph.com/issues/39393
Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agocls/rgw: expose cls_rgw_bucket_list_op for single shard
Casey Bodley [Fri, 19 Apr 2019 22:37:35 +0000 (18:37 -0400)]
cls/rgw: expose cls_rgw_bucket_list_op for single shard

Signed-off-by: Casey Bodley <cbodley@redhat.com>