]> git.apps.os.sepia.ceph.com Git - ceph-ci.git/log
ceph-ci.git
6 years agoMerge pull request #17338 from tianshan/fix_roles_pool
Casey Bodley [Wed, 24 Apr 2019 13:40:36 +0000 (09:40 -0400)]
Merge pull request #17338 from tianshan/fix_roles_pool

rgw: add roles_pool in RGWZoneParams dump/decode json

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #17591 from shashalu/fix-bucket-list-versions
Casey Bodley [Wed, 24 Apr 2019 13:40:16 +0000 (09:40 -0400)]
Merge pull request #17591 from shashalu/fix-bucket-list-versions

rgw: list_bucket versions return NextVersionIdMarker = "null" if next_marker.instance is empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #27472 into master
Sage Weil [Wed, 24 Apr 2019 13:37:49 +0000 (08:37 -0500)]
Merge PR #27472 into master

* refs/pull/27472/head:
doc/rados/operations/devices: document device failure prediction

Reviewed-by: Rick Chen <rick.chen@prophetstor.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #27573 from iotcg/corpus
Kefu Chai [Wed, 24 Apr 2019 13:34:26 +0000 (21:34 +0800)]
Merge pull request #27573 from iotcg/corpus

import-generated.sh: use PATH to get ceph-dencoder

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #27696 into master
Sage Weil [Wed, 24 Apr 2019 13:15:19 +0000 (08:15 -0500)]
Merge PR #27696 into master

* refs/pull/27696/head:
osd: make use of pg history and past_intervals in pg_create2 messages
mon/OSDMonitor: track history and past_intervals for creating pgs
osd/osd_types: make PastIntervals pi_compact_rep print participants
osd/osd_types: take bare const OSDMap * to check_new_interval
osd/osd_types: add pg_history_t ctor that takes creation epoch+stamp

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27656 into master
Sage Weil [Wed, 24 Apr 2019 13:14:55 +0000 (08:14 -0500)]
Merge PR #27656 into master

* refs/pull/27656/head:
doc/dev/erasure-coded-pool: update
doc/rados/operations/erasure-code*: update default ec profile references
common/options: change default erasure-code-profile to k=2 m=2

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27683 from dillaman/wip-24668
Mykola Golub [Wed, 24 Apr 2019 12:40:43 +0000 (15:40 +0300)]
Merge pull request #27683 from dillaman/wip-24668

qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoimport-generated.sh: correct access to ceph-dencoder
Changcheng Liu [Sat, 13 Apr 2019 08:15:35 +0000 (16:15 +0800)]
import-generated.sh: correct access to ceph-dencoder

align with instructions in doc/dev/corpus.rst.
doc/dev/corpus.rst is under change by kchai@redhat.com
in PR 27552.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
6 years agoMerge pull request #27704 from tchaikov/wip-denc-dump
Kefu Chai [Wed, 24 Apr 2019 11:49:54 +0000 (19:49 +0800)]
Merge pull request #27704 from tchaikov/wip-denc-dump

denc: allow DencDumper to dump OOB buffer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: iSCSI: Limit members to 1 group (#27520)
Lenz Grimmer [Wed, 24 Apr 2019 11:41:03 +0000 (13:41 +0200)]
mgr/dashboard: iSCSI: Limit members to 1 group (#27520)

mgr/dashboard: iSCSI: Limit members to 1 group

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #27722 from rhcs-dashboard/39434-prometheus-replace-whitespaces
Lenz Grimmer [Wed, 24 Apr 2019 11:38:14 +0000 (13:38 +0200)]
Merge pull request #27722 from rhcs-dashboard/39434-prometheus-replace-whitespaces

mgr/prometheus: replace whitespaces in metrics' names

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #27678 from tspmelo/wip-iscsi-discovery-read
Lenz Grimmer [Wed, 24 Apr 2019 11:31:50 +0000 (13:31 +0200)]
Merge pull request #27678 from tspmelo/wip-iscsi-discovery-read

mgr/dashboard: Fix iSCSI Discovery user permissions

6 years agoMerge pull request #27609 from tspmelo/wip-iscsi-form-warn
Lenz Grimmer [Wed, 24 Apr 2019 11:29:46 +0000 (13:29 +0200)]
Merge pull request #27609 from tspmelo/wip-iscsi-form-warn

mgr/dashboard: Fix iSCSI target form warning

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agoMerge pull request #27720 from dillaman/wip-39407
Mykola Golub [Wed, 24 Apr 2019 10:00:52 +0000 (13:00 +0300)]
Merge pull request #27720 from dillaman/wip-39407

rbd-mirror: clear out bufferlist prior to listing mirror images

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agodenc: use scope_guard to close(fd)
Kefu Chai [Tue, 23 Apr 2019 11:23:05 +0000 (19:23 +0800)]
denc: use scope_guard to close(fd)

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrush,msg: silence -Wsign-compare warnings
Kefu Chai [Mon, 22 Apr 2019 14:08:15 +0000 (22:08 +0800)]
crush,msg: silence -Wsign-compare warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodenc: allow DencDumper to dump OOB buffer
Kefu Chai [Mon, 22 Apr 2019 13:27:16 +0000 (21:27 +0800)]
denc: allow DencDumper to dump OOB buffer

if encoder encodes buffers into the bufferlist directly, the
contiguous_appender's out_of_band_offset would be greater than zero, in
that case, we should dump the bufferlist passed to appender instead of
just looking at the [pre_pos, post_pos).

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodenc: rewrite DENC_DUMP_{PRE,POST} using template
Kefu Chai [Mon, 22 Apr 2019 13:13:03 +0000 (21:13 +0800)]
denc: rewrite DENC_DUMP_{PRE,POST} using template

* for better readability
* for better debugging experience
* so we can `friend` it in a class in a follow-up change

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27719 from xiexingguo/wip-crush-node-flags
Xie Xingguo [Wed, 24 Apr 2019 00:44:08 +0000 (08:44 +0800)]
Merge pull request #27719 from xiexingguo/wip-crush-node-flags

mon/OSDMonitor: crush node flags - two fixes; add tests

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26107 from alimaredia/wip-dpp-rgw-put-obj-processor
Casey Bodley [Tue, 23 Apr 2019 19:50:07 +0000 (15:50 -0400)]
Merge pull request #26107 from alimaredia/wip-dpp-rgw-put-obj-processor

rgw: log refactoring for putobj_processor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agorgw: log refactoring for AtomicObjProcessor complete
Ali Maredia [Mon, 4 Mar 2019 20:55:11 +0000 (15:55 -0500)]
rgw: log refactoring for AtomicObjProcessor complete

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agoMerge PR #27628 into master
Sage Weil [Tue, 23 Apr 2019 18:08:29 +0000 (13:08 -0500)]
Merge PR #27628 into master

* refs/pull/27628/head:
common: optimize check_utf8

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #27702 into master
Sage Weil [Tue, 23 Apr 2019 18:06:52 +0000 (13:06 -0500)]
Merge PR #27702 into master

* refs/pull/27702/head:
osd/PG: fix last_complete re-calculation on splitting

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27691 into master
Sage Weil [Tue, 23 Apr 2019 18:06:15 +0000 (13:06 -0500)]
Merge PR #27691 into master

* refs/pull/27691/head:
os/bluestore: fix out-of-bound access in bmap allocator.
os/bluestore: reproduce out-of-bound access for bmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27707 into master
Sage Weil [Tue, 23 Apr 2019 18:05:44 +0000 (13:05 -0500)]
Merge PR #27707 into master

* refs/pull/27707/head:
common/util: handle long lines in /proc/cpuinfo

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge PR #27537 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:53:52 +0000 (10:53 -0700)]
Merge PR #27537 into master

* refs/pull/27537/head:
mds: better output of 'ceph health detail'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27511 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:52:41 +0000 (10:52 -0700)]
Merge PR #27511 into master

* refs/pull/27511/head:
mds: fix SnapRealm::resolve_snapname for long name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #27077 into master
Patrick Donnelly [Tue, 23 Apr 2019 17:50:49 +0000 (10:50 -0700)]
Merge PR #27077 into master

* refs/pull/27077/head:
test: check listattr for snapshot btime entry
test: extend LibCephFS.Xattrs test
client: remove unused vxattr length helpers
client: fix _listxattr() vxattr buffer length calculation
test: add libcephfs snap.btime xattr test
client: add ceph.snap.btime vxattr
mds: carry snapshot creation time with InodeStat

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #27725 from theanalyst/perf-counter-names
Casey Bodley [Tue, 23 Apr 2019 16:48:55 +0000 (12:48 -0400)]
Merge pull request #27725 from theanalyst/perf-counter-names

rgw: sync counters: drop spaces from counter names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
6 years agorgw: sync counters: drop spaces from counter names
Abhishek Lekshmanan [Tue, 23 Apr 2019 15:27:08 +0000 (17:27 +0200)]
rgw: sync counters: drop spaces from counter names

Since this might break modules like prometheus and general json processing tools
aren't too happy with spaces.

Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
6 years agomgr/prometheus: replace whitespaces in metrics' names
alfonsomthd [Tue, 23 Apr 2019 14:20:47 +0000 (16:20 +0200)]
mgr/prometheus: replace whitespaces in metrics' names

Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
6 years agoMerge pull request #27703 from tchaikov/wip-rbd-replay-denc
Jason Dillaman [Tue, 23 Apr 2019 14:07:26 +0000 (10:07 -0400)]
Merge pull request #27703 from tchaikov/wip-rbd-replay-denc

rbd_replay: call the member decode() explicitly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agomgr/dashboard: Clean up TableComponent tests and code (#26784)
Lenz Grimmer [Tue, 23 Apr 2019 13:54:23 +0000 (15:54 +0200)]
mgr/dashboard: Clean up TableComponent tests and code (#26784)

mgr/dashboard: Clean up TableComponent tests and code

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
6 years agorbd_replay: call the member decode() explicitly
Kefu Chai [Mon, 22 Apr 2019 11:26:54 +0000 (19:26 +0800)]
rbd_replay: call the member decode() explicitly

otherwise, the one defined using WRITE_RAW_ENCODER is called instead.
so in this change, rename the the member function which happens to have
the same signature with

decode(type &v, ::ceph::bufferlist::const_iterator& p)

where `type` is `__u8`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agorbd-mirror: clear out bufferlist prior to listing mirror images
Jason Dillaman [Tue, 23 Apr 2019 12:24:52 +0000 (08:24 -0400)]
rbd-mirror: clear out bufferlist prior to listing mirror images

The second call to list mirrored images will fail deep within the
msgr code due to a "bad crc in data" error.

Fixes: http://tracker.ceph.com/issues/39407
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27521 from trociny/wip-rbd-remove-clone_v2-parent
Jason Dillaman [Tue, 23 Apr 2019 12:11:38 +0000 (08:11 -0400)]
Merge pull request #27521 from trociny/wip-rbd-remove-clone_v2-parent

librbd: optionally move parent image to trash on remove

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27484 from majianpeng/rbd-nbd
Jason Dillaman [Tue, 23 Apr 2019 12:11:07 +0000 (08:11 -0400)]
Merge pull request #27484 from majianpeng/rbd-nbd

rbd-nbd: sscanf return 0 mean not-match

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agodoc/rados/operations/devices: document device failure prediction
Sage Weil [Tue, 9 Apr 2019 21:45:47 +0000 (16:45 -0500)]
doc/rados/operations/devices: document device failure prediction

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27697 from cbodley/wip-rgw-bucket-list-unordered
Casey Bodley [Tue, 23 Apr 2019 12:07:13 +0000 (08:07 -0400)]
Merge pull request #27697 from cbodley/wip-rgw-bucket-list-unordered

 rgw: cls_bucket_list_unordered lists a single shard

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoosd: make use of pg history and past_intervals in pg_create2 messages
Sage Weil [Fri, 19 Apr 2019 21:41:04 +0000 (16:41 -0500)]
osd: make use of pg history and past_intervals in pg_create2 messages

If we get a mismatched epoch and past_intervals, error out early, or else
we'll end up asserting later in the PastIntervals code.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27713 from tchaikov/wip-24842
Kefu Chai [Tue, 23 Apr 2019 11:14:55 +0000 (19:14 +0800)]
Merge pull request #27713 from tchaikov/wip-24842

doc/rbd/rbd-cloudstack: update disk offering URL to new docs

Reviewed-by: Wido den Hollander <wido@42on.com>
6 years agoqa: add crush-node-flags test
xie xingguo [Tue, 23 Apr 2019 06:20:17 +0000 (14:20 +0800)]
qa: add crush-node-flags test

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agodoc/rbd/rbd-cloudstack: update disk offering URL to new docs
Kefu Chai [Tue, 23 Apr 2019 05:26:45 +0000 (13:26 +0800)]
doc/rbd/rbd-cloudstack: update disk offering URL to new docs

point hyperlinks to latest

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomon/OSDMonitor: remove crush node flags too on "crush rm"
xie xingguo [Tue, 23 Apr 2019 03:24:07 +0000 (11:24 +0800)]
mon/OSDMonitor: remove crush node flags too on "crush rm"

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agomon/OSDMonitor: make per-OSD no{out,down,in,out} flags prior to CRUSH nodes
xie xingguo [Tue, 23 Apr 2019 03:03:04 +0000 (11:03 +0800)]
mon/OSDMonitor: make per-OSD no{out,down,in,out} flags prior to CRUSH nodes

This way we 'll be more compatible with older versions, and can effectively
reduce the map size for large clusters.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoosd/PG: fix last_complete re-calculation on splitting
xie xingguo [Sat, 20 Apr 2019 08:34:12 +0000 (16:34 +0800)]
osd/PG: fix last_complete re-calculation on splitting

We add hard-limit for pg_logs now, which means we might keep trimming
old log entries irrespective of pg's current missing_set. This as a
result can cause the last_complete pointer moving far ahead of the real
on-disk version (the oldest need of missing_set, for instance) the
corresponding pg should have on splitting:

```
2019-04-19 06:41:52.559247 7efd4725c700 10 osd.2 271 Splitting pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 les/251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 unknown NOTIFY m=518 mbc={}] into 5.16
2019-04-19 06:41:52.561413 7efd4725c700 10 osd.2 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 c/f 251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 lcod 0'0 unknown NOTIFY m=261 mbc={}] release_backoffs [MIN,MAX)
```

For the above example, parent's last_complete cursor changed from **0'0** to
**238'300** directly due to the effort of trying to catch up the oldest
log entry changing when splitting was done. However, back into v12.2.9 primary
would still reference shard's last_complete field when trying to figure out all
possible locations of a currently missing object (see PG::MissingLoc::add_source_info):

```c++
  if (oinfo.last_complete < need) {
    if (omissing.is_missing(soid)) {
      ldout(pg->cct, 10) << "search_for_missing " << soid << " " << need
                         << " also missing on osd." << fromosd << dendl;
      continue;
    }
  }
```

Hence a wrongly calculated last_complete could then make primary mis-consider
that a specific shard might have the authoritative object it currently
looking for:

```
2019-04-19 06:41:52.904163 7fd4cfb5a700 10 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}] proc_replica_log for osd.2: 5.6( v 270'943 lc 238'30
0 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/c/f 251/224/0 250/271/229) log((249'563,270'943], crt=270'943) missing(261 may_include_del
etes = 1)
2019-04-19 06:41:52.904645 7fd4cfb5a700 20 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}]  after missing 5:624c3a7a:::benchmark_data_smithi190
_39968_object1382:head need 226'110 have 0'0
2019-04-19 06:41:53.567820 7fd4d035b700 10 osd.5 pg_epoch: 272 pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=271/272 n=471 ec=223/223 lis/c 250/223 les/c/f
251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 unknown m=16 u=13 mbc={255={(1+0)=220,(2+0)=28}}] search_for_missing 5:624c3a
7a:::benchmark_data_smithi190_39968_object1382:head 226'110 is on osd.2
```

note that ```5:624c3a7a:::benchmark_data_smithi190_39968_object1382:head 226'110```
was actually missing on both primary and shard osd.2 whereas primary insisted that
object should exist on shard osd.2!

https://github.com/ceph/ceph/pull/26175 posted an indirect fix
for the above problem by ignoring last_complete when checking the missing set,
but it should generally make more sense to fill in the last_complete field correctly
whenever possible.
Hence coming this additional fix.

Fixes: http://tracker.ceph.com/issues/26958
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #27708 into master
Sage Weil [Mon, 22 Apr 2019 22:07:27 +0000 (17:07 -0500)]
Merge PR #27708 into master

* refs/pull/27708/head:
doc/governance: add cbodey

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #27693 into master
Sage Weil [Mon, 22 Apr 2019 22:06:42 +0000 (17:06 -0500)]
Merge PR #27693 into master

* refs/pull/27693/head:
mgr/telemetry: default to reports every 24h; lower minimum
mgr/telemetry: exclude hostname field in crash reports

Reviewed-by: Dan Mick <dmick@redhat.com>
6 years agodoc/governance: add cbodey
Sage Weil [Mon, 22 Apr 2019 20:37:11 +0000 (15:37 -0500)]
doc/governance: add cbodey

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/util: handle long lines in /proc/cpuinfo
Sage Weil [Mon, 22 Apr 2019 17:51:41 +0000 (12:51 -0500)]
common/util: handle long lines in /proc/cpuinfo

Fixes: http://tracker.ceph.com/issues/38296
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/dev/erasure-coded-pool: update
Sage Weil [Mon, 22 Apr 2019 17:36:29 +0000 (12:36 -0500)]
doc/dev/erasure-coded-pool: update

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/workunits/rbd: use more recent qemu-iotests that support Bionic
Jason Dillaman [Thu, 18 Apr 2019 18:02:45 +0000 (14:02 -0400)]
qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Fixes: http://tracker.ceph.com/issues/24668
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agodoc/rados/operations/erasure-code*: update default ec profile references
Sage Weil [Mon, 22 Apr 2019 16:20:55 +0000 (11:20 -0500)]
doc/rados/operations/erasure-code*: update default ec profile references

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agorgw: thread DoutPrefixProvider into fetch_remote_obj
Ali Maredia [Mon, 4 Mar 2019 18:42:37 +0000 (13:42 -0500)]
rgw: thread DoutPrefixProvider into fetch_remote_obj

This is for the AtomicObjProcessor declared there

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agorgw: log refactoring for putobj_processor
Ali Maredia [Thu, 24 Jan 2019 04:53:53 +0000 (23:53 -0500)]
rgw: log refactoring for putobj_processor

Signed-off-by: Ali Maredia <amaredia@redhat.com>
6 years agorgw: cls_bucket_list_unordered lists a single shard
Casey Bodley [Fri, 19 Apr 2019 22:38:47 +0000 (18:38 -0400)]
rgw: cls_bucket_list_unordered lists a single shard

CLSRGWIssueBucketList sends the request to every shard, but this loop
intended to list only the current_shard

Fixes: http://tracker.ceph.com/issues/39393
Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agocls/rgw: expose cls_rgw_bucket_list_op for single shard
Casey Bodley [Fri, 19 Apr 2019 22:37:35 +0000 (18:37 -0400)]
cls/rgw: expose cls_rgw_bucket_list_op for single shard

Signed-off-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27694 from dillaman/wip-39386
Mykola Golub [Mon, 22 Apr 2019 15:13:02 +0000 (18:13 +0300)]
Merge pull request #27694 from dillaman/wip-39386

qa/suites/rbd: added writearound cache test permutations

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #27682 from dillaman/wip-39031
Mykola Golub [Mon, 22 Apr 2019 15:12:30 +0000 (18:12 +0300)]
Merge pull request #27682 from dillaman/wip-39031

librbd: async open/close should free ImageCtx before issuing callback

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #24744 from liewegas/wip-stale-prs
Sage Weil [Mon, 22 Apr 2019 14:58:24 +0000 (09:58 -0500)]
Merge pull request #24744 from liewegas/wip-stale-prs

.github/stale.yml: warn at 60, close at 90; adjust message

6 years agomgr/telemetry: default to reports every 24h; lower minimum
Sage Weil [Fri, 19 Apr 2019 17:24:48 +0000 (12:24 -0500)]
mgr/telemetry: default to reports every 24h; lower minimum

Allow more frequent telemetry reports.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/telemetry: exclude hostname field in crash reports
Sage Weil [Fri, 19 Apr 2019 17:09:36 +0000 (12:09 -0500)]
mgr/telemetry: exclude hostname field in crash reports

On some systems the hostname is a fully-qualified domain name and
(even when not a fqdn) may inadvertantly allow the cluster to be
identified.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/OSDMonitor: track history and past_intervals for creating pgs
Sage Weil [Fri, 19 Apr 2019 21:33:04 +0000 (16:33 -0500)]
mon/OSDMonitor: track history and past_intervals for creating pgs

PG create messages from mons are the last case where the OSD may have to
scan an unbounded number of old maps in order to construct a valid
pg_history_t and PastIntervals.  Try to avoid making that a difficult
case by maintaining those structures on the monitor.

It is still possible that the mon may send a pg create message to the OSD
and it sits in a message queue for a very long time, but this would be a
very difficult situation to get into, and is no different from inter-OSD
messages that include history and past_intervals.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd/osd_types: make PastIntervals pi_compact_rep print participants
Sage Weil [Fri, 19 Apr 2019 21:01:20 +0000 (16:01 -0500)]
osd/osd_types: make PastIntervals pi_compact_rep print participants

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd/osd_types: take bare const OSDMap * to check_new_interval
Sage Weil [Fri, 19 Apr 2019 19:52:14 +0000 (14:52 -0500)]
osd/osd_types: take bare const OSDMap * to check_new_interval

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoosd/osd_types: add pg_history_t ctor that takes creation epoch+stamp
Sage Weil [Fri, 19 Apr 2019 18:56:25 +0000 (13:56 -0500)]
osd/osd_types: add pg_history_t ctor that takes creation epoch+stamp

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27605 into master
Sage Weil [Mon, 22 Apr 2019 12:04:11 +0000 (07:04 -0500)]
Merge PR #27605 into master

* refs/pull/27605/head:
mon/OSDMonitor: osd add-no{up,down,in,out} - remove state checker

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27695 from dillaman/wip-39072
Mykola Golub [Mon, 22 Apr 2019 05:22:20 +0000 (08:22 +0300)]
Merge pull request #27695 from dillaman/wip-39072

test/librbd: drop 'ceph_test_librbd_api' target

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge PR #27288 into master
Sage Weil [Mon, 22 Apr 2019 02:28:02 +0000 (21:28 -0500)]
Merge PR #27288 into master

* refs/pull/27288/head:
osd/PG: discover missing objects when an OSD peers and PG is degraded

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #27571 into master
Sage Weil [Mon, 22 Apr 2019 02:26:04 +0000 (21:26 -0500)]
Merge PR #27571 into master

* refs/pull/27571/head:
mgr/telemetry: add report_timestamp to sent reports

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27572 into master
Sage Weil [Mon, 22 Apr 2019 02:20:37 +0000 (21:20 -0500)]
Merge PR #27572 into master

* refs/pull/27572/head:
denc: fix build error by calling global snprintf

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #27622 into master
Sage Weil [Mon, 22 Apr 2019 02:20:12 +0000 (21:20 -0500)]
Merge PR #27622 into master

* refs/pull/27622/head:
mon: fix INCOMPAT_OCTOPUS feature number
mon/Elector: format mon_release correctly

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27577 from dillaman/wip-rbd-suite-distros
Mykola Golub [Sun, 21 Apr 2019 17:19:15 +0000 (20:19 +0300)]
Merge pull request #27577 from dillaman/wip-rbd-suite-distros

qa/suites/rbd: add random distro selection to librbd tests

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #27690 from tchaikov/wip-crimson-one-safeguard-is-better-than-two
Kefu Chai [Sun, 21 Apr 2019 03:17:27 +0000 (11:17 +0800)]
Merge pull request #27690 from tchaikov/wip-crimson-one-safeguard-is-better-than-two

crimson: pass `Connection*` to Dispatch::ms_dispatch()

Reviewed-by: Samuel Just <sjust@redhat.com>
6 years agoMerge PR #27595 into master
Sage Weil [Sat, 20 Apr 2019 13:52:01 +0000 (08:52 -0500)]
Merge PR #27595 into master

* refs/pull/27595/head:
osd: add 'ceph osd stop <osd.nnn>' command

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #27619 into master
Sage Weil [Sat, 20 Apr 2019 13:48:30 +0000 (08:48 -0500)]
Merge PR #27619 into master

* refs/pull/27619/head:
mgr/BaseMgrModule: run MonCommandCompletion on the finisher
mgr/BaseMgrModule: fix leak

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
6 years agoMerge PR #27651 into master
Sage Weil [Sat, 20 Apr 2019 13:48:14 +0000 (08:48 -0500)]
Merge PR #27651 into master

* refs/pull/27651/head:
test/osd/safe-to-destroy.sh: fix typo

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #27663 into master
Sage Weil [Sat, 20 Apr 2019 13:47:48 +0000 (08:47 -0500)]
Merge PR #27663 into master

* refs/pull/27663/head:
osd/OSDMap: Replace get_out_osds with get_out_existing_osds

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27631 from dmick/wip-telemetry-crasharray
Kefu Chai [Sat, 20 Apr 2019 06:18:28 +0000 (14:18 +0800)]
Merge pull request #27631 from dmick/wip-telemetry-crasharray

mgr/telemetry: change crash dict to a list

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27599 from gregsfortytwo/wip-stale-prs
Kefu Chai [Sat, 20 Apr 2019 06:16:33 +0000 (14:16 +0800)]
Merge pull request #27599 from gregsfortytwo/wip-stale-prs

Miscellaneous lost fixes

Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge pull request #27615 from ifed01/wip-ifed-verbose-bluefs-replay
Kefu Chai [Sat, 20 Apr 2019 06:12:20 +0000 (14:12 +0800)]
Merge pull request #27615 from ifed01/wip-ifed-verbose-bluefs-replay

os/bluestore: be more verbose doing bluefs log replay.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27509 from Aran85/fix-pg-dout
Kefu Chai [Sat, 20 Apr 2019 06:10:39 +0000 (14:10 +0800)]
Merge pull request #27509 from Aran85/fix-pg-dout

osd/PG: the warning seems more serious than what it wanna transmit

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27592 from tchaikov/wip-ref
Kefu Chai [Sat, 20 Apr 2019 06:09:51 +0000 (14:09 +0800)]
Merge pull request #27592 from tchaikov/wip-ref

message,mgr: drop MessageFactory and friends and use ref_t<> in mgr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #27633 from xiexingguo/wip-39340
Kefu Chai [Sat, 20 Apr 2019 06:05:24 +0000 (14:05 +0800)]
Merge pull request #27633 from xiexingguo/wip-39340

osd/osd_types: pool_stat_t::dump - fix 'num_store_stats' field

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27658 from gregsfortytwo/wip-39367-crc-peer
Kefu Chai [Sat, 20 Apr 2019 06:04:33 +0000 (14:04 +0800)]
Merge pull request #27658 from gregsfortytwo/wip-39367-crc-peer

msg: output peer address when detecting bad CRCs

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: pass `Connection*` to Dispatch::ms_dispatch()
Kefu Chai [Fri, 19 Apr 2019 08:50:25 +0000 (16:50 +0800)]
crimson: pass `Connection*` to Dispatch::ms_dispatch()

currently, we use a `with_gate()` in `ProtocolV2::read_message()` for
ensuring that `this` (or `Connection` holding this protocol instance)
will outlive the continuation of `dispatcher.ms_dispatch()` which
references `this->dispatcher`. but we also pass a strong reference of
connection to dispatcher. in short, we have *two* safeguards for the
same purpose.

in this change, one of these safeguards is removed -- to pass the raw
pointer of `Connection` to `Dispatch::ms_dispatch()`. the reason why
the `with_gate()` is kept is that, if we have removed `with_gate()` in
Protocol, we need to

1. let `Dispatcher::ms_dispatch()` return `void`, as it should not block
any succeeding calls.
2. add a `with_gate()` in `Dispatcher::ms_dispatch()` to ensure that
`this` is alive during the lifecycle of the continuation(s) in
`Dispatcher::ms_dispatch()`.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27657 from anthonyeleven/patch-1
Xie Xingguo [Sat, 20 Apr 2019 00:50:29 +0000 (08:50 +0800)]
Merge pull request #27657 from anthonyeleven/patch-1

doc: operations: improve reweight-by-utilization

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agodoc: operations: reweight-by-utilization typo
Anthony D'Atri [Fri, 19 Apr 2019 22:15:12 +0000 (15:15 -0700)]
doc: operations: reweight-by-utilization typo

Add a missing backquote delimiter.

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
6 years agocommon/options: change default erasure-code-profile to k=2 m=2
Sage Weil [Wed, 17 Apr 2019 21:16:53 +0000 (16:16 -0500)]
common/options: change default erasure-code-profile to k=2 m=2

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agotest/librbd: drop 'ceph_test_librbd_api' target
Jason Dillaman [Fri, 19 Apr 2019 18:15:49 +0000 (14:15 -0400)]
test/librbd: drop 'ceph_test_librbd_api' target

This was used for API backwards compatibility testing, but now that
the C++ API will not remain stable, it serves no purpose.

Fixes: http://tracker.ceph.com/issues/39072
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoqa/suites/rbd: added writearound cache test permutations
Jason Dillaman [Fri, 19 Apr 2019 18:11:43 +0000 (14:11 -0400)]
qa/suites/rbd: added writearound cache test permutations

Also tweaked the existing writeback cache test permutations to
specify the use of the writeback cache policy.

Fixes: http://tracker.ceph.com/issues/39386
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agomgr/BaseMgrModule: run MonCommandCompletion on the finisher
Sage Weil [Thu, 18 Apr 2019 18:10:15 +0000 (13:10 -0500)]
mgr/BaseMgrModule: run MonCommandCompletion on the finisher

The completion takes the ActivePyModules::lock, but the caller may be
holding an internal lock for Objecter and/or MonClient.  Avoid deadlock
by putting the completion on our finisher.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/BaseMgrModule: fix leak
Sage Weil [Thu, 18 Apr 2019 18:28:05 +0000 (13:28 -0500)]
mgr/BaseMgrModule: fix leak

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoos/bluestore: fix out-of-bound access in bmap allocator.
Igor Fedotov [Fri, 19 Apr 2019 12:43:07 +0000 (15:43 +0300)]
os/bluestore: fix out-of-bound access in bmap allocator.

Fixes: https://tracker.ceph.com/issues/39334
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
6 years agoos/bluestore: reproduce out-of-bound access for bmap allocator
Igor Fedotov [Fri, 19 Apr 2019 12:04:48 +0000 (15:04 +0300)]
os/bluestore: reproduce out-of-bound access for bmap allocator

Reproduces: https://tracker.ceph.com/issues/39334

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
6 years agolibrbd: optionally move parent image to trash on remove
Mykola Golub [Mon, 15 Apr 2019 10:35:53 +0000 (11:35 +0100)]
librbd: optionally move parent image to trash on remove

and auto-delete when the last clone is detached

Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agolibrbd: reuse async trash remove state machine
Mykola Golub [Mon, 15 Apr 2019 10:34:07 +0000 (11:34 +0100)]
librbd: reuse async trash remove state machine

Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agolibrbd: async trash remove state machine
Mykola Golub [Mon, 15 Apr 2019 10:32:15 +0000 (11:32 +0100)]
librbd: async trash remove state machine

Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agorbd-nbd: sscanf return 0 mean not-match.
Jianpeng Ma [Fri, 19 Apr 2019 05:00:04 +0000 (13:00 +0800)]
rbd-nbd: sscanf return 0 mean not-match.

When exec:  rbd-nbd map rbd/image --device /dev/image
The error message is:
rbd-nbd: failed to open device: /dev/image.

In fact, it should print:
rbd-nbd: invalid device path: /dev/image (expected /dev/nbd{num})

Meantime change help which make more understand.

Fixes: http://tracker.ceph.com/issues/39269
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
6 years agoMerge PR #27685 into master
Patrick Donnelly [Thu, 18 Apr 2019 22:36:17 +0000 (15:36 -0700)]
Merge PR #27685 into master

* refs/pull/27685/head:
doc: fix boot transition in mds state diagram

Reviewed-by: Sage Weil <sage@redhat.com>