]>
git.apps.os.sepia.ceph.com Git - ceph.git/log
Niu Pengju [Sun, 8 Apr 2018 03:08:04 +0000 (11:08 +0800)]
rgw: Fix multisite Synchronization failed when read and write delete at the same time
This case is first write objA,then write and delete objA at the same
time,write early than delete.
When del objA, use information which stat of first write objA, so the
op should del the first write data.However when try to del objA, objA
header is second write, so osd "do_xattr_cmp_str" has found idtag change
and return -125(canceled),after rgw client receive the ret -125 , it
will still do "complete_del", then do cls_obj_complete_del to write
bilog。"complete_op" in cls_rgw module will write bilog with second
write mtime and second ".ver.epoch". Finally, del op append behind the
second write in bilog. And the slave rgw will merge write op and del op
as del op, and del data,but master rgw complete second write and cancel
del.
This logic is problematic, so bilog recording the del op should use
cancel op. And squash_map should skip the cancel op.
Fixes: http://tracker.ceph.com/issues/22804
Signed-off-by: Niu Pengju <pengju.niu@xtaotech.com>
(cherry picked from commit
a8fd508915a0acf7768fe6ba7f7414e71813f1d7 )
Yuri Weinstein [Thu, 12 Apr 2018 16:30:49 +0000 (09:30 -0700)]
Merge pull request #19883 from pdvian/wip-22635-luminous
luminous: rgw: add xml output header in RGWCopyObj_ObjStore_S3 response msg
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Wed, 11 Apr 2018 19:52:23 +0000 (12:52 -0700)]
Merge pull request #20829 from xiexingguo/backport-pr-20653
luminous: mon, osd: fix potential collided *Up Set* after PG remapping
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Wed, 11 Apr 2018 19:51:09 +0000 (12:51 -0700)]
Merge pull request #20840 from xiexingguo/backport-pr-20655
luminous: osd/OSDMap: skip out/crush-out osds
Reviewed-by: Sage Weil <sage@redhat.com>
Sage Weil [Wed, 11 Apr 2018 19:50:23 +0000 (14:50 -0500)]
Merge pull request #20674 from rzarzynski/wip-bs-drop-std_function-luminous
luminous: os/bluestore: avoid overhead of std::function in blob_t.
Yuri Weinstein [Wed, 11 Apr 2018 19:48:57 +0000 (12:48 -0700)]
Merge pull request #20907 from pdvian/wip-23315-luminous
luminous: mon/OSDMonitor.cc: fix expected_num_objects interpret error
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Wed, 11 Apr 2018 15:51:27 +0000 (08:51 -0700)]
Merge pull request #21231 from pdvian/wip-23544-luminous
luminous: os/bluestore: change the type of aio_t:res to long
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Wed, 11 Apr 2018 15:49:20 +0000 (08:49 -0700)]
Merge pull request #20973 from pdvian/wip-23409-luminous
luminous: mgr: fix MSG_MGR_MAP handling
Reviewed-by: John Spray <john.spray@redhat.com>
Yuri Weinstein [Wed, 11 Apr 2018 15:48:39 +0000 (08:48 -0700)]
Merge pull request #21229 from liewegas/wip-release-reserved-luminous
luminous: osd: do not release_reserved_pushes when requeuing
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
David Zafman [Wed, 11 Apr 2018 14:50:12 +0000 (07:50 -0700)]
Merge pull request #21103 from dzafman/wip-23267-luminous
luminous: scrub errors not cleared on replicas can cause inconsistent pg state when replica takes over primary
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Andrew Schoen [Wed, 11 Apr 2018 14:47:44 +0000 (16:47 +0200)]
Merge pull request #21335 from ceph/backport-rm23644
luminous: ceph-volume lvm.activate error if no bluestore OSDs are found
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
David Zafman [Wed, 28 Mar 2018 17:21:39 +0000 (10:21 -0700)]
osd: Fix stale scrub stats when a primary takes over
Fixes: http://tracker.ceph.com/issues/23267
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
5cfb8241f482ed53c63bb97262425b2acb733d7d )
David Zafman [Tue, 27 Mar 2018 00:27:19 +0000 (17:27 -0700)]
test: Eliminate use of bc (use awk) in get_timeout_delays()
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
ce9c029858d8deb249afe8effd5d80ae623f69a2 )
David Zafman [Fri, 23 Mar 2018 00:17:42 +0000 (17:17 -0700)]
osd: Improve object_stat_sum_t handling
Keep post split PGs as inconsistent from original
Keep num_scrub_errors as total of other 2
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
39adde2655ebb40ad5c010fa77509acc35b25b91 )
Conflicts:
src/osd/osd_types.h (trivial)
Yuri Weinstein [Tue, 10 Apr 2018 19:56:37 +0000 (12:56 -0700)]
Merge pull request #21124 from smithfarm/wip-23507-luminous
luminous: test_admin_socket.sh may fail on wait_for_clean
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Yuri Weinstein [Tue, 10 Apr 2018 19:55:56 +0000 (12:55 -0700)]
Merge pull request #21192 from pdvian/wip-23524-luminous
luminous: is_qemu_running in qemu_rebuild_object_map.sh and qemu_dynamic_features.sh may return false positive
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Yuri Weinstein [Tue, 10 Apr 2018 19:55:26 +0000 (12:55 -0700)]
Merge pull request #21230 from pdvian/wip-23542-luminous
luminous: rbd-nbd: fix ebusy when do map
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Alfredo Deza [Tue, 10 Apr 2018 00:00:27 +0000 (20:00 -0400)]
ceph-volume lvm.activate error if no bluestore OSDs are found
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
b1920d6e731f194c4829810d849dba6b087ce3a4 )
Yuri Weinstein [Mon, 9 Apr 2018 23:17:27 +0000 (16:17 -0700)]
Merge pull request #20431 from pdvian/wip-22967-luminous
luminous: osdc/Journaler: make sure flush() writes enough data
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Mon, 9 Apr 2018 23:16:54 +0000 (16:16 -0700)]
Merge pull request #21173 from smithfarm/wip-22383-luminous
luminous: qa: src/test/libcephfs/test.cc:376: Expected: (len) > (0), actual: -34 vs 0
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Yuri Weinstein [Mon, 9 Apr 2018 23:15:58 +0000 (16:15 -0700)]
Merge pull request #21176 from smithfarm/wip-23154-luminous
luminous: mds: FAILED assert (p != active_requests.end()) in MDRequestRef MDCache::request_get(metareqid_t)
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Patrick Donnelly [Mon, 9 Apr 2018 22:51:50 +0000 (15:51 -0700)]
Merge PR #21267 into luminous
* refs/pull/21267/head:
discard the mdsload clear after prep_rebalance in case we want to export it for debugging
make sure that MDBalancer uses heartbeat info from the same epoch
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Patrick Donnelly [Mon, 9 Apr 2018 22:51:44 +0000 (15:51 -0700)]
Merge PR #21266 into luminous
* refs/pull/21266/head:
make popular counter decay at proper rate
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Patrick Donnelly [Mon, 9 Apr 2018 22:51:38 +0000 (15:51 -0700)]
Merge PR #21256 into luminous
* refs/pull/21256/head:
mds: bump mds_log_max_segments for trim buffer
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Nathan Cutler [Sun, 8 Apr 2018 17:05:24 +0000 (19:05 +0200)]
Merge pull request #20885 from mslovy/fix-bug-21811
luminous: rgw: Admin API Support for bucket quota change
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Nathan Cutler [Sun, 8 Apr 2018 17:03:47 +0000 (19:03 +0200)]
Merge pull request #21048 from cbodley/wip-qa-swift-force-branch-luminous
luminous: tests: rgw: swift tests target ceph-luminous branch
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:28:34 +0000 (12:28 -0700)]
Merge pull request #20890 from pdvian/wip-23312-luminous
luminous: invalid JSON returned when querying pool parameters
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:28:09 +0000 (12:28 -0700)]
Merge pull request #20998 from pdvian/wip-23412-luminous
luminous: log: Fix AddressSanitizer: new-delete-type-mismatch
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:27:29 +0000 (12:27 -0700)]
Merge pull request #21053 from pdvian/wip-23224-luminous
luminous: mgr: quieten logging on missing OSD stats
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:26:59 +0000 (12:26 -0700)]
Merge pull request #21059 from pdvian/wip-23226-luminous
luminous: os/bluestore: trim cache every 50ms (instead of 200ms)
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:26:18 +0000 (12:26 -0700)]
Merge pull request #21090 from tchaikov/wip-23477-luminous
luminous: common: should not check for VERSION_ID
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:25:18 +0000 (12:25 -0700)]
Merge pull request #20675 from rzarzynski/wip-bs-no-blobrefing-in-do_read-luminous
luminous: os/bluestore: avoid unneeded BlobRefing in _do_read().
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:24:58 +0000 (12:24 -0700)]
Merge pull request #20676 from rzarzynski/wip-bs-dump-onode-by-ref-luminous
luminous: os/bluestore: _dump_onode() don't prolongate Onode anymore.
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:24:25 +0000 (12:24 -0700)]
Merge pull request #20712 from pdvian/wip-23175-luminous
luminous: mgr: die on bind() failure
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 19:24:01 +0000 (12:24 -0700)]
Merge pull request #20867 from pdvian/wip-23306-luminous
luminous: msg/async: size of EventCenter::file_events should be greater than fd.
Reviewed-by: Sage Weil <sage@redhat.com>
Patrick Donnelly [Fri, 6 Apr 2018 19:10:15 +0000 (12:10 -0700)]
Merge PR #21198 into luminous
* refs/pull/21198/head:
ceph_authtool: add mode option
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 15:59:07 +0000 (08:59 -0700)]
Merge pull request #20642 from b-ranto/wip-mgr-prom-luminous
[luminous] Backport recent prometheus exporter changes
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 15:57:28 +0000 (08:57 -0700)]
Merge pull request #20668 from pdvian/wip-23160-luminous
luminous: osd: Remove partially created pg known as DNE
Reviewed-by: David Zafman <dzafman@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 15:56:19 +0000 (08:56 -0700)]
Merge pull request #20710 from pdvian/wip-23174-luminous
luminous: common/dns_resolv.cc: Query for AAAA-record if ms_bind_ipv6 is True
Reviewed-by: Wido den Hollander <wido@widodh.nl>
Yuri Weinstein [Fri, 6 Apr 2018 15:55:03 +0000 (08:55 -0700)]
Merge pull request #20766 from t19903/wip-cache-tier-forward-hang
luminous: Objecter: add ignore overlay flag if got redirect reply
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Yuri Weinstein [Fri, 6 Apr 2018 15:52:59 +0000 (08:52 -0700)]
Merge pull request #21079 from tchaikov/wip-23471-luminous
luminous: add --add-bucket and --move options to crushtool
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Sage Weil [Wed, 30 Aug 2017 19:13:08 +0000 (15:13 -0400)]
qa/suites/rados/singleton/diverget_priors*: broaden whitelist
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
39e5efbad2c2834acf8e6463258b148321ea85a4 )
Jianyu Li [Fri, 17 Nov 2017 07:08:35 +0000 (15:08 +0800)]
discard the mdsload clear after prep_rebalance in case we want to export it for debugging
Signed-off-by: Jianyu Li <joannyli@tencent.com>
(cherry picked from commit
513f2e57666bd08242fc639ebc5c65db12742a01 )
Jianyu Li [Wed, 15 Nov 2017 07:43:44 +0000 (15:43 +0800)]
make sure that MDBalancer uses heartbeat info from the same epoch
Currently mds saves the heartbeat info from others in mds_load, once the mds_load.size( ) equals mds number, it considers that have received all heartbeats info and start the rebalance work. However, after prep_rebalance returns, it doesn't clear the mds_load immediately, but wait until receives the next round hearbeat from mds0. If there are mutiple mds(e.g. greater than 2), there is a chance for one mds receiving the first next round heartbeat other than mds0 due to the network delay.
Signed-off-by: Jianyu Li <joannyli@tencent.com>
(cherry picked from commit
591fd4970863ae51dad5e16be3cef24c1c6c92d6 )
Jianyu Li [Fri, 3 Nov 2017 08:48:10 +0000 (16:48 +0800)]
make popular counter decay at proper rate
Signed-off-by: Jianyu Li <joannyli@tencent.com>
(cherry picked from commit
6d4709e649beacb5f2d6197d1bea57c8590c5f71 )
Yuri Weinstein [Thu, 5 Apr 2018 20:21:53 +0000 (13:21 -0700)]
Merge pull request #20762 from pdvian/wip-23180-luminous
luminous: radosgw-admin data sync run crashes
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 20:20:36 +0000 (13:20 -0700)]
Merge pull request #20953 from pdvian/wip-23347-luminous
luminous: rgw: fix memory fragmentation problem reading data from client.
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 20:16:52 +0000 (13:16 -0700)]
Merge pull request #21054 from pdvian/wip-23225-luminous
luminous: cls/rgw: fix bi_log_iterate_entries return wrong truncated
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 20:09:32 +0000 (13:09 -0700)]
Merge pull request #20722 from theanalyst/wip-luminous-23221
luminous: fixes for openssl & libcurl
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 20:07:26 +0000 (13:07 -0700)]
Merge pull request #21214 from smithfarm/wip-23025-luminous
luminous: rgw: data sync of versioned objects, note updating bi marker
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Abhishek L [Thu, 5 Apr 2018 17:21:01 +0000 (19:21 +0200)]
Merge pull request #20896 from smithfarm/wip-23349-luminous
luminous: Couldn't init storage provider (RADOS)
Reviewed-By: Sage Weil <sage@redhat.com>
Abhishek L [Thu, 5 Apr 2018 17:20:20 +0000 (19:20 +0200)]
Merge pull request #20250 from smithfarm/wip-22856-luminous
luminous: build Debian installation packages failure
Reviewed-By: Sage Weil <sage@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 17:00:44 +0000 (10:00 -0700)]
Merge pull request #20737 from pdvian/wip-23178-luminous
luminous: build/ops: run-make-check.sh: fix SUSE support
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Yuri Weinstein [Thu, 5 Apr 2018 16:58:31 +0000 (09:58 -0700)]
Merge pull request #20748 from smithfarm/wip-jinja2-luminous
build/ops: deb: move python-jinja2 dependency to mgr
Reviewed-by: John Spray <john.spray@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 16:56:59 +0000 (09:56 -0700)]
Merge pull request #21016 from pdvian/wip-23077-luminous
luminous: mon: ops get stuck in "resend forwarded message to leader"
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 16:55:41 +0000 (09:55 -0700)]
Merge pull request #21111 from wido/luminous-21813
luminous: common/ipaddr: Do not select link-local IPv6 addresses
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 16:54:54 +0000 (09:54 -0700)]
Merge pull request #21118 from smithfarm/wip-23500-luminous
luminous: snapmapper inconsistency, crash on luminous
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Thu, 5 Apr 2018 16:54:00 +0000 (09:54 -0700)]
Merge pull request #21134 from smithfarm/wip-23522-luminous
luminous: tests: unittest_pglog timeout
Reviewed-by: Kefu Chai <kchai@redhat.com>
Patrick Donnelly [Wed, 4 Apr 2018 23:47:06 +0000 (16:47 -0700)]
mds: bump mds_log_max_segments for trim buffer
Under create heavy workloads, the MDS will sometimes get behind trimming but
catch up. This avoids unnecessary warnings.
Fixes: http://tracker.ceph.com/issues/23560
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
86e76d15f3b3c06e96d7b27bb01d5840d311cf5d )
Conflicts:
src/common/options.cc
Nathan Cutler [Sun, 25 Jun 2017 07:27:47 +0000 (09:27 +0200)]
tests: swift.py: clone the ceph-luminous branch
The master branch of ceph/swift.git contains tests that are incompatible with
Jewel and Luminous. The ceph-luminous branch omits these tests.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
a86ce728954a765797ce634025d43650d990e480 )
Conflicts: qa/tasks/swift.py (cherry-picked jewel version to luminous)
Jeegn Chen [Thu, 4 Jan 2018 07:59:00 +0000 (07:59 +0000)]
doc: Add usage doc for individual bucket quota Admin API
Add usage doc for a new Admin API for individual bucket quota management
Fixes: http://tracker.ceph.com/issues/21811
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>
(cherry picked from commit
0cbd26e88de3c9c71c1ca4532308502f3a8c7ff8 )
Jeegn Chen [Thu, 14 Dec 2017 08:08:49 +0000 (08:08 +0000)]
rgw: Add URL format support for bucket quota modification
Allow admin to change the quota of a individual bucket through Admin API
with params in URL
Fixes: http://tracker.ceph.com/issues/21811
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>
(cherry picked from commit
46b1186f144c0928394835301fc993d89598398b )
kungf [Fri, 30 Mar 2018 12:15:42 +0000 (20:15 +0800)]
os/bluestore: change the type of aio_t:res to long
io_event::res was assigned to io_t::rval, but io_event::res
is long, if rval is int, this will cause overflow.
Fixes: http://tracker.ceph.com/issues/23527
Signed-off-by: kungf <yang.wang@easystack.cn>
(cherry picked from commit
a9b9477a44d4556f6827e6dc4b94a5f1f61ce8e1 )
Sage Weil [Wed, 4 Apr 2018 02:24:07 +0000 (21:24 -0500)]
osd: do not release_reserved_pushes when requeuing
Back in
3cc48278bf0ee5c9535d04b60a661f988c50063b we refactoring the sharded
wq and incorrectly included code that would release_reserved_pushes for
items that were queued and deferred and then woken and put back in the
queue. The reserved_pushes are for recovery ops that are in flight in the
queue, which includes the priority queue *and* the waiting_for_pg; the code
we replaced would release these only when dequeueing an item (or items) for
processing (or discard).
In master, this code is fixed as part of the peering fast dispatch and
OSDShard refactor.
Signed-off-by: Sage Weil <sage@redhat.com>
Li Wang [Thu, 7 Dec 2017 14:03:45 +0000 (22:03 +0800)]
rbd-nbd: fix ebusy when do map
When doing rbd-nbd map, if the Ceph service is not available,
the codes will wait on rados.connect(), unless killing the process.
In that case, the close_nbd logic is skipped with NBD_CLEAR_SOCK ioctl
not called. On the CentOS 7 kernel, it leaves nbd->file not cleared, which
causes the subsequent map requests return EBUSY, this patch fixes it
by connecting Ceph first prior to calling NBD_SET_SOCK ioctl
Fixes: http://tracker.ceph.com/issues/23528
Signed-off-by: Li Wang <laurence.liwang@gmail.com>
(cherry picked from commit
ab77dcc0170c0d63795fe0d50427cda630bfd593 )
Jan Fajerski [Tue, 27 Feb 2018 09:19:39 +0000 (10:19 +0100)]
pybind/mgr/prometheus: extend daemon metadata with hostname and ceph version
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
d03c79676b1761b016ceaed82f7e3f03f99cda73 )
Jan Fajerski [Tue, 20 Feb 2018 10:06:19 +0000 (11:06 +0100)]
pybind/mgr/prometheus: add fs and mds metadata metrics
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
0095e6a1ba0fb9473cfb60b6977498f2c1159079 )
Jan Fajerski [Mon, 19 Feb 2018 11:25:02 +0000 (12:25 +0100)]
pybind/mgr/prometheus: new mon metrics
Remove mon_quorum_count and replace it with per-MON quorum status
(mon_quorum_status). Also add mon_metadata metrics.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
71a6408b1adfedab64208fe1a23231df43960924 )
Jan Fajerski [Sat, 17 Feb 2018 11:07:46 +0000 (12:07 +0100)]
pybind/mgr/prometheus: add Metrics class to manage Metric instances
The central change of this commit is that per-daemon metrics are now
managed by first appending the metric (using Metrics.append) to a
staging area. Then the metrics for specific paths (metric names) are
overwritten by the staged metrics (by calling Metrics.reset). This gets
rid of metrics from daemon that are no longer in the cluster. I.e. when
ceph no longer reports metrics for one OSD daemon (because it was
removed from the cluster) the prometheus module will no longer export
metrics for that daemon.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
f88877ab0bd895d8b67222225c22a12d479b7548 )
Christopher Blum [Fri, 23 Feb 2018 17:48:49 +0000 (18:48 +0100)]
pybind/mgr/prometheus: don't crash on OSDs without metadata
Fix issue where the ceph_exporter crashes after a Ceph upgrade with a broken OSD - that OSD was never online with Luminous and thus we have no metadata for it
Signed-off-by: Christopher Blum zeichenanonym@web.de
(cherry picked from commit
3beee663f6fd6a16cef5a65ea5f9937cbea7a537 )
Jan Fajerski [Thu, 30 Nov 2017 10:27:04 +0000 (11:27 +0100)]
pybing/mgr/prometheus: tidy up cherrypy engine start and stop
Calling cherrypy.engine.block() in the stanby module results in a failing
mgr failover.
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
ff471d485cd2068b6425b74a17875d8d6a823595 )
Jan Fajerski [Wed, 29 Nov 2017 11:23:55 +0000 (12:23 +0100)]
pybind/mgr/prometheus: return 503 if MON cluster is down
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
eda9f15dd939de3e7205ceb0dfcc9fcfce7d89bb )
Jan Fajerski [Wed, 29 Nov 2017 11:23:18 +0000 (12:23 +0100)]
pybind/mgr/prometheus: add StandbyModule; return empty answer
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit
18ffd7529b0975897121d8a5c37d0d20560a1fc7 )
Boris Ranto [Sat, 17 Feb 2018 03:45:01 +0000 (04:45 +0100)]
mgr/prometheus: Expose OSD Flags
Signed-off-by: Boris Ranto <branto@redhat.com>
Boris Ranto [Fri, 16 Feb 2018 17:45:58 +0000 (18:45 +0100)]
mgr/prometheus: Fix pg_* counts
Currently, the pg_* counts are not computed properly. We split the
current state by '+' sign but do not add the pg count to the already
found pg count. Instead, we overwrite any existing pg count with the new
count. This patch fixes it by adding all the pg counts together for all
the states.
It also introduces a new pg_total metric for pg_total that shows the
total count of PGs.
Signed-off-by: Boris Ranto <branto@redhat.com>
Boris Ranto [Fri, 16 Feb 2018 03:53:39 +0000 (04:53 +0100)]
mgr/prometheus: Skip bogus entries
The osd data can contain bogus '-' entries, skip these when populating
osd metadata and disk occupation.
Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit
965aaad5273eac311e4535559a0e0a9df8a39bee )
Yehuda Sadeh [Wed, 15 Feb 2017 00:18:53 +0000 (16:18 -0800)]
rgw: log unlink_instance mtime as object's mtime
This is needed so that when squashing entries during data
sync we get a consistent mtime for that object.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit
bef22609de95843008b5d02eb075252bb97fbe90 )
Sébastien Han [Fri, 23 Jun 2017 13:45:57 +0000 (15:45 +0200)]
ceph_authtool: add mode option
We now have the ability to set the keyring file mode with the help of
'--mode MODE'. The mode needs to be specified in octal using the format:
0600.
Closes: http://tracker.ceph.com/issues/23513
Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit
b179cd275526613a3838cb52e6f1c35950b6f3a8 )
Willem Jan Withagen [Fri, 30 Mar 2018 14:53:48 +0000 (16:53 +0200)]
common: FreeBSD wants the correct struct selection for ipv6
Lets see if this also works for Linux
Fixes: http://tracker.ceph.com/issues/21813
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit
3806d8ec7c145d0239c94fc5b601d460b36757a5 )
Kefu Chai [Tue, 3 Apr 2018 04:15:57 +0000 (12:15 +0800)]
Merge pull request #21177 from tchaikov/wip-luminous-pr-21159
luminous: src: s/--use-wheel//
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Mykola Golub [Thu, 29 Mar 2018 11:10:58 +0000 (14:10 +0300)]
qa/suites/rbd: set qemu task time_wait param
so workloads qemu_dynamic_features.sh and qemu_rebuild_object_map.sh,
which check if qemu is finished with periodicity 60 sec, have enough
time to detect this before the rbd image is removed.
Fixes: https://tracker.ceph.com/issues/23502
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit
6fe8d6470506a9947f685b05683e2e046457a9f5 )
Mykola Golub [Thu, 29 Mar 2018 11:06:13 +0000 (14:06 +0300)]
qa/tasks/qemu: add a parameter to wait for workloads detect qemu finished
In the case when a workload needs to detect qemu finished by running a
check with a periodicity of N sec it needs to set time_wait to 2 * N
in order to avoid races on finish.
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit
af15b2cef46c8d3ab1f9770601ae1abfe9c6b300 )
Kefu Chai [Sun, 1 Apr 2018 12:27:24 +0000 (20:27 +0800)]
src: s/--use-wheel//
--use-wheel was deprecated in favor of --only-binary in pip v7.0.0. and
--use-wheel was removed in a recent release of pip. but some packages
are source packages, so we cannot simply replace use-wheel with
only-binary. so a simpler approach is to drop --use-wheel option, as pip
respects --find-links, and will find the required package from the
wheelhouse.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
a80cce4ba6afee921a7c6adce802c91e30f8c0a9 )
Conflicts:
src/ceph-detect-init/CMakeLists.txt
src/ceph-disk/CMakeLists.txt: trivial resolution
src/pybind/mgr/dashboard/CMakeLists.txt: dashboard2 is not
in luminous, so drop this change.
Yan, Zheng [Fri, 23 Feb 2018 04:17:04 +0000 (12:17 +0800)]
mds: always handle SESSION_REQUEST_RENEWCAPS messages
Fixes: http://tracker.ceph.com/issues/23059
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
543547827c9cee2626952fa645ec8b05ab9c1366 )
Yan, Zheng [Fri, 23 Feb 2018 03:33:41 +0000 (11:33 +0800)]
mds: don't cleanup request that has pending remote authpin/wrlock/xlock
Fixes: http://tracker.ceph.com/issues/23059
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
a197a89d65c76c51cbc24890474ad27f72ebe382 )
Patrick Donnelly [Wed, 22 Nov 2017 05:34:32 +0000 (21:34 -0800)]
test: pass correct buffer length to ceph_getdents
This test fails if the randvar r is 0.
Fixes: http://tracker.ceph.com/issues/22221
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
ef5699441e056133967ffbf29076eb79594bf5e5 )
Nathan Cutler [Thu, 29 Mar 2018 09:40:09 +0000 (11:40 +0200)]
tests: remove TestPGLog ASSERT_DEATH test
Fixes: http://tracker.ceph.com/issues/23504
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
adcba97a41795c25de8b6dfe263a1014564a305f )
Kefu Chai [Tue, 27 Mar 2018 09:45:30 +0000 (17:45 +0800)]
mon: mark mgr reports and osd_failure as no_reply
see also: #20517
Fixes: http://tracker.ceph.com/issues/22114
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
0daccfb4951471ab3f70ae87870f23e14357d9c8 )
Yuri Weinstein [Thu, 29 Mar 2018 13:24:07 +0000 (06:24 -0700)]
Merge pull request #20254 from k0ste/luminous_backports
luminous: mgr: prometheus: set metadata metrics value to '1' (#22717).
Reviewed-by: John Spray <john.spray@redhat.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:23:31 +0000 (06:23 -0700)]
Merge pull request #20334 from tchaikov/wip-pr-20255-luminous
luminous: config: Change bluestore_cache_kv_max to type INT64
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:22:48 +0000 (06:22 -0700)]
Merge pull request #20358 from pdvian/wip-22940-luminous
luminous: Double free in rados_getxattrs_next
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:22:10 +0000 (06:22 -0700)]
Merge pull request #20359 from xiexingguo/wip-luminous-balancer-fixes
luminous: mgr/balancer: pool-specific optimization support and bug fixes
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:21:27 +0000 (06:21 -0700)]
Merge pull request #20399 from smithfarm/wip-22942-luminous
luminous: ceph osd force-create-pg cause all ceph-mon to crash and unable to come up again
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:20:49 +0000 (06:20 -0700)]
Merge pull request #20433 from pdvian/wip-22971-luminous
mon: allow removal of tier of ec overwritable pool
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:20:16 +0000 (06:20 -0700)]
Merge pull request #20495 from pdvian/wip-23024-luminous
luminous: osd: Fix assert when checking missing version
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:19:38 +0000 (06:19 -0700)]
Merge pull request #20554 from pdvian/wip-23074-luminous
luminous: bluestore: statfs available can go negative
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:18:54 +0000 (06:18 -0700)]
Merge pull request #20585 from pdvian/wip-23114-luminous
luminous: librados: make OPERATION_FULL_FORCE the default for rados_remove()
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:18:03 +0000 (06:18 -0700)]
Merge pull request #20609 from pdvian/wip-23075-luminous
luminous: osd: objecter sends out of sync with pg epochs for proxied ops
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Yuri Weinstein [Thu, 29 Mar 2018 13:17:24 +0000 (06:17 -0700)]
Merge pull request #20988 from pdvian/wip-23410-luminous
luminous: legal: remove doc license ambiguity
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Mykola Golub [Thu, 29 Mar 2018 09:29:40 +0000 (12:29 +0300)]
qa/workunits/rbd: remove sanity check in test_admin_socket.sh
Fixes: https://tracker.ceph.com/issues/23499
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit
d392393e1f92b5f35c76606a85e046211b7adddc )