]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoosd: combine recovery/scrub/snap sleep timer into one. 25507/head
Jianpeng Ma [Sat, 28 Apr 2018 02:46:55 +0000 (10:46 +0800)]
osd: combine recovery/scrub/snap sleep timer into one.

One SafeTime on thread. And those events if timer don't take long time.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 610d5196c0132739e1ae133db0c7d896063470ad)

6 years agoosd: Add support for osd_delete_sleep to throttle pg deletion
David Zafman [Thu, 25 Oct 2018 05:00:57 +0000 (22:00 -0700)]
osd: Add support for osd_delete_sleep to throttle pg deletion

Fixes: http://tracker.ceph.com/issues/36474
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 92f1f5859f142cb83cc1d1a367853ac91d6b01d4)

Conflicts:
    src/osd/OSD.cc
- use cct->_conf->get_val instead of cct->_conf.get_val

6 years agotest: Set any value for osd_delete_sleep to guarantee we are testing even SSD
David Zafman [Tue, 30 Oct 2018 18:42:05 +0000 (11:42 -0700)]
test: Set any value for osd_delete_sleep to guarantee we are testing even SSD

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3f621a1190f45a780b71b916b31c1a01a20ece96)

6 years agoMerge pull request #25332 from pdvian/wip-36503-mimic
Yuri Weinstein [Wed, 12 Dec 2018 16:01:14 +0000 (08:01 -0800)]
Merge pull request #25332 from pdvian/wip-36503-mimic

mimic: tests: use timeout for fs asok operations

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #25283 from jdurgin/wip-osd-memory-target-mimic
Josh Durgin [Mon, 10 Dec 2018 17:13:37 +0000 (12:13 -0500)]
Merge pull request #25283 from jdurgin/wip-osd-memory-target-mimic

mimic: core: os/bluestore: cache autotuning and memory limit

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25362 from noahdesu/mimic-docs
Noah Watkins [Fri, 7 Dec 2018 19:35:34 +0000 (11:35 -0800)]
Merge pull request #25362 from noahdesu/mimic-docs

mimic: docs: backport edit on github changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id
Alfredo Deza [Tue, 4 Dec 2018 16:50:42 +0000 (11:50 -0500)]
Merge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id

mimic ceph-volume add device_id to inventory listing

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agodoc/_templates/page.html: add link to report doc bugs 25362/head
Neha Ojha [Fri, 19 Oct 2018 20:26:53 +0000 (13:26 -0700)]
doc/_templates/page.html: add link to report doc bugs

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de30c4d3c5927bb58b49b44b391647e397e1fe4f)

6 years agoluminous: doc: show edit on github links and version warnings
Noah Watkins [Mon, 1 Oct 2018 23:54:19 +0000 (16:54 -0700)]
luminous: doc: show edit on github links and version warnings

backport of #24452 that adds edit on
github links to documentation and notification banners that display
warnings when old documentation is being viewed.

this is not a cherry-pick: it removes from the original patch the
dynamic generation of the releases schedule from a yaml database file.
backporting this portion requires modifying the patch to deal with a
different file / directory structure [in luminous] with no real added value.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
 Conflicts:
doc/conf.py

6 years agoceph-volume: add test for disk.udevadm_property 25349/head
Jan Fajerski [Wed, 28 Nov 2018 09:21:36 +0000 (10:21 +0100)]
ceph-volume: add test for disk.udevadm_property

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 58316e3f826f8bb91b375589aa3676b664ce3984)

6 years agoceph-volume: add device_id field to inventory output
Jan Fajerski [Wed, 21 Nov 2018 12:20:45 +0000 (13:20 +0100)]
ceph-volume: add device_id field to inventory output

This intends to mimic the C++ implementation in src/common/blkdev.cc.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 41da5e9a4334628db09e3166eaceff79b9b6c7e0)

6 years agoqa: use timeout for fs asok operations 25332/head
Patrick Donnelly [Sat, 6 Oct 2018 17:32:18 +0000 (10:32 -0700)]
qa: use timeout for fs asok operations

Otherwise a bug preventing an asok operation from completing will cause the
entire job to fail.

Fixes: http://tracker.ceph.com/issues/36335
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 296d89b0463ee3aad8dfb50b0e41a27e2c852ddb)

6 years agoMerge pull request #25294 from smithfarm/wip-37422-mimic
Nathan Cutler [Wed, 28 Nov 2018 21:13:02 +0000 (22:13 +0100)]
Merge pull request #25294 from smithfarm/wip-37422-mimic

mimic: build/ops: Can't compile Ceph on Fedora 29 as it doesn't recognize python*3*-tox as an install Tox

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #25040 from smithfarm/wip-36749-mimic
Yuri Weinstein [Wed, 28 Nov 2018 20:27:07 +0000 (12:27 -0800)]
Merge pull request #25040 from smithfarm/wip-36749-mimic

mimic: mgr: [restful] deep_scrub is not a valid OSD command

Reviewed-by: Boris Ranto <branto@redhat.com>
6 years agobuild/ops: rpm: stop install-deps.sh clobbering spec file Python build setting 25294/head
Nathan Cutler [Tue, 20 Nov 2018 10:43:13 +0000 (11:43 +0100)]
build/ops: rpm: stop install-deps.sh clobbering spec file Python build setting

Fedora 29 still ships a Python 2 binary, but some of Ceph's build
dependencies are only available in py3 versions there. In other
words, from F29 on, it is no longer possible to do a py2 Ceph build
on Fedora, even if a python2 binary exists on the system.

If that were not enough, the Python 2 that ships with Fedora 29 is
linked against a non-compatible version of OpenSSL.

Before this commit, install-deps.sh was overriding the spec file's
Python build setting based on the presence or absence of a python2
binary. As the bug cited below indicates, this was not a good idea.

It's better for the spec file to be explicit about which OS versions
are py2 and which are py3, and just stick to that.

Fixes: http://tracker.ceph.com/issues/37301
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7d4916618c419e4c85823cf259b96e49d9203035)

Conflicts:
ceph.spec.in
install-deps.sh
- no WITH_SEASTAR in mimic

6 years agobuild/ops: refrain from installing/using lsb_release in install-deps.sh
Nathan Cutler [Wed, 1 Aug 2018 10:52:45 +0000 (12:52 +0200)]
build/ops: refrain from installing/using lsb_release in install-deps.sh

Unfortunately the mapping between release number and codename (which is only
relevant for Debian and Ubuntu btw) is not available from /etc/os-release.
In that one respect, lsb_release was "better".

However, when I weigh the advantages of obtaining that mapping from an external
tool, with the (substantial) risk that the external dependency might cause
trouble on one or more supported distros (to say nothing of the non- or
semi-/pseudo-supported ones), against the work involved in maintaining a
hard-coded mapping (negligible), the needle on my scale immediately swings
toward eliminating the dependency.

Also, I see this commit as part of the longer-term effort to completely expunge
lsb_release from our codebase. See git log --grep lsb_release.

For another example of an external distro-detection tool (albeit one that was
included in Python 2) gone awry, see http://tracker.ceph.com/issues/18163.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 38508b41dc7f33a0b35e3633be3da1d2dae844dc)

6 years agoMerge pull request #25000 from MrStupnikov/mimic-docfix-1358
Nathan Cutler [Tue, 27 Nov 2018 21:05:34 +0000 (22:05 +0100)]
Merge pull request #25000 from MrStupnikov/mimic-docfix-1358

mimic: doc: Put command template into literal block

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoPendingReleaseNotes: add note about osd memory target and bluestore cache 25283/head
Josh Durgin [Tue, 27 Nov 2018 04:01:38 +0000 (23:01 -0500)]
PendingReleaseNotes: add note about osd memory target and bluestore cache

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
6 years agodoc/rados: Add bluestore memory autotuning docs.
Mark Nelson [Mon, 12 Nov 2018 23:17:42 +0000 (17:17 -0600)]
doc/rados: Add bluestore memory autotuning docs.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9e2d1a52abacbab2b4dab24bd209dc12d26fe2db)

Conflicts:
doc/rados/configuration/bluestore-config-ref.rst

Deployment docs updated in master, pulled in those here too

6 years agoos/bluestore: limit OSD memory usage by tuning the cache size.
Mark Nelson [Wed, 20 Jun 2018 02:59:26 +0000 (21:59 -0500)]
os/bluestore: limit OSD memory usage by tuning the cache size.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 1b8a87b39a52b10ac3b1bd3c70062d4dcd53ed5e)

Conflicts:
src/common/options.cc
src/os/bluestore/BlueStore.cc
trivial - no g_conf operator. or option TYPE_SIZE

6 years agokv/RocksDBStore: Handle nullptr if clock cache is chosen.
Mark Nelson [Thu, 9 Aug 2018 21:50:16 +0000 (16:50 -0500)]
kv/RocksDBStore: Handle nullptr if clock cache is chosen.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9202285b44ce144864a80cb063269f863bb7d6da)

Conflicts:
src/kv/RocksDBStore.cc

trivial - g_conf operator()

6 years agokv/rocksdb_cache/BinnedLRUCache: Don't promote data to the high pri pool.
Mark Nelson [Tue, 3 Jul 2018 22:30:23 +0000 (17:30 -0500)]
kv/rocksdb_cache/BinnedLRUCache: Don't promote data to the high pri pool.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 269713d0f80a611fc61f5946738a879dcd55597b)

6 years agosrc/kv: Initial import of a custom RocksDB cache.
Mark Nelson [Tue, 3 Jul 2018 15:42:14 +0000 (10:42 -0500)]
src/kv: Initial import of a custom RocksDB cache.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9dd6b2885e6d83b5ad468a7343aa2f1d476e818b)

Conflicts:
src/kv/RocksDBStore.cc
trivial - g_conf has no operator()

6 years agoos/bluestore: Fix potential uint64_t to int conversion issue.
Mark Nelson [Tue, 29 May 2018 19:23:41 +0000 (14:23 -0500)]
os/bluestore: Fix potential uint64_t to int conversion issue.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 1d51e764500f519018f572282ef0a269914edfd4)

6 years agocommon/PriorityCache: First pass at prototype.
Mark Nelson [Thu, 26 Apr 2018 14:57:00 +0000 (09:57 -0500)]
common/PriorityCache: First pass at prototype.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit cb465411921ab01ea47b4ac1d37a29662213355b)

Conflicts:
src/os/bluestore/BlueStore.cc (trivial)

6 years agoMerge pull request #24918 from smithfarm/wip-35071-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:56:40 +0000 (12:56 -0800)]
Merge pull request #24918 from smithfarm/wip-35071-mimic

mimic: core: FAILED assert(osdmap_manifest.pinned.empty()) in OSDMonitor::prune_init()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
6 years agoMerge pull request #24970 from smithfarm/wip-35843-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:55:47 +0000 (12:55 -0800)]
Merge pull request #24970 from smithfarm/wip-35843-mimic

mimic: core: objecter cannot resend split-dropped op when racing with con reset

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #24971 from smithfarm/wip-36432-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:54:25 +0000 (12:54 -0800)]
Merge pull request #24971 from smithfarm/wip-36432-mimic

mimic: core: Interactive mode CLI prints no output since Mimic

Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #25016 from smithfarm/wip-36433-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:53:55 +0000 (12:53 -0800)]
Merge pull request #25016 from smithfarm/wip-36433-mimic

mimic: core: monstore tool rebuild does not generate creating_pgs

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25017 from smithfarm/wip-36435-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:53:25 +0000 (12:53 -0800)]
Merge pull request #25017 from smithfarm/wip-36435-mimic

mimic: core: rados rm --force-full is blocked when cluster is in full status

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #25019 from smithfarm/wip-36505-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:52:52 +0000 (12:52 -0800)]
Merge pull request #25019 from smithfarm/wip-36505-mimic

mimic: core: mon osdmap cash too small during upgrade to mimic

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #25024 from smithfarm/wip-36557-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:52:20 +0000 (12:52 -0800)]
Merge pull request #25024 from smithfarm/wip-36557-mimic

mimic: osd: RBD client IOPS pool stats are incorrect (2x higher; includes IO hints as an op)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25026 from smithfarm/wip-36637-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:51:42 +0000 (12:51 -0800)]
Merge pull request #25026 from smithfarm/wip-36637-mimic

mimic: osd: race condition opening heartbeat connection

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #25027 from smithfarm/wip-36647-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:51:06 +0000 (12:51 -0800)]
Merge pull request #25027 from smithfarm/wip-36647-mimic

mimic: tests: librados api aio tests race condition

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #25029 from smithfarm/wip-36465-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:50:17 +0000 (12:50 -0800)]
Merge pull request #25029 from smithfarm/wip-36465-mimic

mimic: core: mgr crash on scrub of unconnected osd

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25045 from tchaikov/wip-36219-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:49:48 +0000 (12:49 -0800)]
Merge pull request #25045 from tchaikov/wip-36219-mimic

mimic: msg: ceph_abort() when there are enough accepter errors in msg server

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25062 from jonsger/wip-36754-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:49:01 +0000 (12:49 -0800)]
Merge pull request #25062 from jonsger/wip-36754-mimic

mimic: core: _aio_log_start inflight overlap of 0x10000~1000 with [65536~4096]

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #25075 from jonsger/wip-36658-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:48:14 +0000 (12:48 -0800)]
Merge pull request #25075 from jonsger/wip-36658-mimic

mimic: core: Objecter: add ignore cache flag if got redirect reply

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #25050 from gregsfortytwo/mimic-readable-failures
Nathan Cutler [Fri, 23 Nov 2018 18:15:45 +0000 (19:15 +0100)]
Merge pull request #25050 from gregsfortytwo/mimic-readable-failures

mimic: tests: make readable.sh fail if it doesn't run anything

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25228 from ceph/revert-25091-wip-36463-mimic
Yan, Zheng [Fri, 23 Nov 2018 03:31:51 +0000 (11:31 +0800)]
Merge pull request #25228 from ceph/revert-25091-wip-36463-mimic

Revert "mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen"

6 years agoRevert "mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen" 25228/head
Yan, Zheng [Fri, 23 Nov 2018 03:29:52 +0000 (11:29 +0800)]
Revert "mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen"

6 years agoMerge pull request #24913 from smithfarm/wip-36428-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:49:16 +0000 (08:49 -0800)]
Merge pull request #24913 from smithfarm/wip-36428-mimic

mimic: tests: rbd: move OpenStack devstack test to rocky release

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24915 from smithfarm/wip-36529-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:48:51 +0000 (08:48 -0800)]
Merge pull request #24915 from smithfarm/wip-36529-mimic

mimic: rbd: TokenBucketThrottle: use reference to m_blockers.front() and then update it.

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24916 from smithfarm/wip-36555-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:48:29 +0000 (08:48 -0800)]
Merge pull request #24916 from smithfarm/wip-36555-mimic

mimic: rbd: [rbd-mirror] periodic mirror status timer might fail to be scheduled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24946 from jonsger/wip-36689-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:47:55 +0000 (08:47 -0800)]
Merge pull request #24946 from jonsger/wip-36689-mimic

mimic: tests: unittest_rbd_mirror: TestMockImageMap.AddInstancePingPongImageTest: Value of: it != peer_ack_ctxs->end()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24952 from jonsger/wip-36693-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:47:32 +0000 (08:47 -0800)]
Merge pull request #24952 from jonsger/wip-36693-mimic

mimic: rbd: [rbd-mirror] forced promotion after killing remote cluster results in stuck state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24988 from iridescent-rsy/mimic-36713
Yuri Weinstein [Thu, 22 Nov 2018 16:47:11 +0000 (08:47 -0800)]
Merge pull request #24988 from iridescent-rsy/mimic-36713

mimic: rbd: rbd-nbd: do not ceph_abort() after print the usages

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25113 from swinds24/wip-36764-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:46:40 +0000 (08:46 -0800)]
Merge pull request #25113 from swinds24/wip-36764-mimic

mimic: mgr: hold lock while accessing the request list and submittin request

Reviewed-by: Boris Ranto <branto@redhat.com>
6 years agoMerge pull request #25115 from mcv21/mimic-36741
Yuri Weinstein [Thu, 22 Nov 2018 16:46:07 +0000 (08:46 -0800)]
Merge pull request #25115 from mcv21/mimic-36741

mimic: build/ops: debian: correct ceph-common relationship with older radosgw package

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25091 from jonsger/wip-36463-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:40:51 +0000 (08:40 -0800)]
Merge pull request #25091 from jonsger/wip-36463-mimic

mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #24987 from joscollin/wip-36690-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:40:10 +0000 (08:40 -0800)]
Merge pull request #24987 from joscollin/wip-36690-mimic

mimic: cephfs: client: request next osdmap for blacklisted client

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #24993 from jonsger/wip-36457-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:39:45 +0000 (08:39 -0800)]
Merge pull request #24993 from jonsger/wip-36457-mimic

mimic: cephfs: client: explicitly show blacklisted state via asok status command

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25036 from smithfarm/wip-36218-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:39:17 +0000 (08:39 -0800)]
Merge pull request #25036 from smithfarm/wip-36218-mimic

mimic: cephfs: some tool commands silently operate on only rank 0, even if multiple ranks exist

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25042 from smithfarm/wip-36461-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:38:48 +0000 (08:38 -0800)]
Merge pull request #25042 from smithfarm/wip-36461-mimic

mimic: mds: rctime not set on system inode (root) at startup

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25095 from jonsger/wip-37093-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:37:20 +0000 (08:37 -0800)]
Merge pull request #25095 from jonsger/wip-37093-mimic

mimic: mds: src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped) during max_mds thrashing

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge PR #25118 into mimic
Patrick Donnelly [Wed, 21 Nov 2018 18:11:16 +0000 (10:11 -0800)]
Merge PR #25118 into mimic

* refs/pull/25118/head:
mds: make timeout parameter optional for "cache drop"
test: add test for mds drop cache command
mds: command to trim mds cache and client caps
mds: implement journal flush as asynchronous context execution
mds: cleanup some asok commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25191 from badone/wip-mimic-ceph_ansible-specify-branch-stable-3.2
Brad Hubbard [Wed, 21 Nov 2018 07:05:43 +0000 (17:05 +1000)]
Merge pull request #25191 from badone/wip-mimic-ceph_ansible-specify-branch-stable-3.2

mimic: qa/ceph-ansible: Specify stable-3.2 branch

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #25193 from neha-ojha/wip-fix-thrash-old-clients-mimic
Neha Ojha [Wed, 21 Nov 2018 06:04:10 +0000 (22:04 -0800)]
Merge pull request #25193 from neha-ojha/wip-fix-thrash-old-clients-mimic

mimic: qa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoqa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel 25193/head
Neha Ojha [Tue, 20 Nov 2018 23:37:25 +0000 (23:37 +0000)]
qa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel

This fix goes directly into the mimic branch since the following packages
do not need to be installed on hammer and jewel.

- python34-rados
- python34-cephfs
- python3-rados
- python3-cephfs

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24899 from ivancich/rgw-renew-reshard-lock-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:12:33 +0000 (09:12 -0800)]
Merge pull request #24899 from ivancich/rgw-renew-reshard-lock-mimic

mimic: rgw: renew resharding locks to prevent expiration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24907 from smithfarm/wip-36223-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:12:07 +0000 (09:12 -0800)]
Merge pull request #24907 from smithfarm/wip-36223-mimic

mimic: rgw: default quota not set in radosgw for Openstack users

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24908 from smithfarm/wip-36415-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:11:39 +0000 (09:11 -0800)]
Merge pull request #24908 from smithfarm/wip-36415-mimic

mimic: librgw: crashes in multisite configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24910 from smithfarm/wip-36533-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:11:14 +0000 (09:11 -0800)]
Merge pull request #24910 from smithfarm/wip-36533-mimic

mimic: rgw: cls_user_remove_bucket does not write the modified cls_user_stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24911 from smithfarm/wip-36535-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:10:49 +0000 (09:10 -0800)]
Merge pull request #24911 from smithfarm/wip-36535-mimic

mimic: rgw: radosgw-admin user stats are incorrect when dynamic re-sharding is enabled

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24912 from smithfarm/wip-36539-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:10:15 +0000 (09:10 -0800)]
Merge pull request #24912 from smithfarm/wip-36539-mimic

mimic: use-after-free from RGWRadosGetOmapKeysCR::~RGWRadosGetOmapKeysCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25079 from jonsger/wip-36734-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:09:48 +0000 (09:09 -0800)]
Merge pull request #25079 from jonsger/wip-36734-mimic

mimic: rgw: beast frontend fails to parse ipv6 endpoints

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25085 from smithfarm/wip-37153-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:09:21 +0000 (09:09 -0800)]
Merge pull request #25085 from smithfarm/wip-37153-mimic

mimic: tests: ceph-admin-commands.sh workunit does not log what it's doing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #25087 from jonsger/wip-36756-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:08:41 +0000 (09:08 -0800)]
Merge pull request #25087 from jonsger/wip-36756-mimic

mimic: rgw-admin: reshard add can add a non existant bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoqa/ceph-ansible: Specify stable-3.2 branch 25191/head
Brad Hubbard [Sun, 18 Nov 2018 23:37:02 +0000 (09:37 +1000)]
qa/ceph-ansible: Specify stable-3.2 branch

Ceph-ansible no longer supports mimic post stable-3.2.

https://tracker.ceph.com/issues/37332

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
6 years agomimic: doc: put command template into literal block 25000/head
Alexey Stupnikov [Thu, 8 Nov 2018 19:44:33 +0000 (20:44 +0100)]
mimic: doc: put command template into literal block

``ceph-deploy osd create --data {device} {ceph-node}`` command
is not displayed properly because of missing double colons.

Signed-off-by: Alexey Stupnikov <aleksey.stupnikov@gmail.com>
(cherry picked from commit bff5a2122fd2a059c6576f36d7d1505c8d747776)

6 years agoMerge pull request #25133 from alfredodeza/mimic-c-v-dont-duplicated-rejected-reasons
Alfredo Deza [Fri, 16 Nov 2018 18:02:42 +0000 (13:02 -0500)]
Merge pull request #25133 from alfredodeza/mimic-c-v-dont-duplicated-rejected-reasons

mimic ceph-volume: rename Device property valid to available

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoc-v device test: rename valid to available 25133/head
Jan Fajerski [Fri, 16 Nov 2018 08:27:34 +0000 (09:27 +0100)]
c-v device test: rename valid to available

Update test to reflect 8a80990471108b0920d1d8aa1239733ae2b20e9c

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 047dc1c157e44642fa83873f02674b146e6d9028)

6 years agoinventory test: rename valid to available
Jan Fajerski [Fri, 16 Nov 2018 08:24:37 +0000 (09:24 +0100)]
inventory test: rename valid to available

Fixes test to reflect 8a80990471108b0920d1d8aa1239733ae2b20e9c

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit d895f97bb021c765f3c4d2d091b23bd0eef19f71)

6 years agoceph-volume: rename Device property valid to available
Jan Fajerski [Fri, 16 Nov 2018 08:22:06 +0000 (09:22 +0100)]
ceph-volume: rename Device property valid to available

This flag is used in the inventory reporting and available is deemed more
appropriate. Furthermore this fixes a bug where rejected_reasons
accumulated duplicate entries.

Fixes: http://tracker.ceph.com/issues/36701
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8a80990471108b0920d1d8aa1239733ae2b20e9c)

6 years agoMerge pull request #25130 from alfredodeza/mimic-c-v-inventory-doc
Alfredo Deza [Fri, 16 Nov 2018 15:22:24 +0000 (10:22 -0500)]
Merge pull request #25130 from alfredodeza/mimic-c-v-inventory-doc

mimic doc: add ceph-volume inventory sections

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agodoc: add ceph-volume inventory sections 25130/head
Jan Fajerski [Wed, 14 Nov 2018 13:09:37 +0000 (14:09 +0100)]
doc: add ceph-volume inventory sections

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit dfc0e1d9a69f6f9ffb1d3e0b0ec33a75827c36e0)

6 years agoMerge pull request #25116 from alfredodeza/mimic-rm26953
Andrew Schoen [Thu, 15 Nov 2018 17:23:54 +0000 (11:23 -0600)]
Merge pull request #25116 from alfredodeza/mimic-rm26953

mimic ceph-volume: allow to specify --cluster-fsid instead of reading from ceph.conf

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agomds: make timeout parameter optional for "cache drop" 25118/head
Venky Shankar [Thu, 11 Oct 2018 06:26:29 +0000 (02:26 -0400)]
mds: make timeout parameter optional for "cache drop"

Fixes: http://tracker.ceph.com/issues/36694
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 73459619747046d38df5f4ec6707db12dce77317)

 Conflicts:
        src/mds/MDSDaemon.cc

6 years agoceph-volume tests.lvm verify get_cluster_fsid can read from args or ceph.conf 25116/head
Alfredo Deza [Wed, 3 Oct 2018 19:21:03 +0000 (15:21 -0400)]
ceph-volume tests.lvm verify get_cluster_fsid can read from args or ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 02ea6dc0663cdaf764fdb1a1a5b31a736b854e6e)

6 years agoceph-volume lvm.prepare consume the value from --cluster-fsid if available
Alfredo Deza [Wed, 3 Oct 2018 17:32:05 +0000 (13:32 -0400)]
ceph-volume lvm.prepare consume the value from --cluster-fsid if available

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a92883e60f3f2206a11b43762b3afcb84b05f07e)

6 years agoceph-volume lvm.common add the --cluster-fsid flag
Alfredo Deza [Wed, 3 Oct 2018 17:31:43 +0000 (13:31 -0400)]
ceph-volume lvm.common add the --cluster-fsid flag

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 15546ae6991c244197537d81092875b12613e20b)

6 years agodebian: correct ceph-common relationship with older radosgw package 25115/head
Matthew Vernon [Thu, 8 Nov 2018 17:23:36 +0000 (17:23 +0000)]
debian: correct ceph-common relationship with older radosgw package

Fixes: https://tracker.ceph.com/issues/37273
9fd30b93f7281fad70b93512f0a25e3465f5b225 moved
/etc/bash_completion.d/radosgw-admin from radosgw to ceph-common. This
means that if you try and install a newer ceph-common over an older
radosgw, there's a conflict, and the install fails:

```
Unpacking ceph-common (12.2.8-1xenial) over (10.2.9-0ubuntu0.16.04.1) ...
dpkg: error processing archive ceph-common_12.2.8-1xenial_amd64.deb (--install):
 trying to overwrite '/etc/bash_completion.d/radosgw-admin', which is also in package radosgw 10.2.9-0ubuntu0.16.04.1
```

Per Debian policy (
https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-in-other-packages
) the correct way to handle a package taking over a file is for a
versioned Replaces and Breaks.

The change went into 12.0.3, so this commit adds Replaces and Breaks
against radosgw less than that version. It should be backported to
Luminous to avoid issues with upgrades from older versions (Jewel and
Kraken).

Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
(cherry picked from commit dd24ddcbf78104ec2370fc3e956984a57cde670b)

6 years agotest: add test for mds drop cache command
Venky Shankar [Wed, 22 Aug 2018 15:06:25 +0000 (11:06 -0400)]
test: add test for mds drop cache command

Fixes: https://tracker.ceph.com/issues/36282
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 43d1b8e174af333514d2b23d0a1be9ec3895fd25)

 Conflicts:
qa/tasks/cephfs/test_misc.py

6 years agomds: command to trim mds cache and client caps
Venky Shankar [Sun, 26 Aug 2018 13:48:56 +0000 (09:48 -0400)]
mds: command to trim mds cache and client caps

With this command, the MDS would request clients to release
caps followed by trimming its own cache and a journal flush.
The command accepts a timeout to wait for clients to respond
to session recall and flush messages.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit da70dde8aa23b0b24057089edaee4d6317f52bd4)

 Conflicts:
src/mds/MDSDaemon.cc
src/mds/MDSRank.cc
src/mds/Server.cc
src/mds/Server.h

6 years agomds: implement journal flush as asynchronous context execution
Venky Shankar [Fri, 14 Sep 2018 04:23:11 +0000 (00:23 -0400)]
mds: implement journal flush as asynchronous context execution

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 75f42340db0ee7263b0708675b8cdfda3bc78059)

 Conflicts:
src/mds/MDSRank.cc
src/mds/MDSRank.h

6 years agomds: cleanup some asok commands
Venky Shankar [Sun, 26 Aug 2018 12:55:05 +0000 (08:55 -0400)]
mds: cleanup some asok commands

Signed-off-by: Rishabh Dave <ridave@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 4c03cfd99eb9144e92740345d6874d1f07831b00)

6 years agoceph-mgr: hold lock while accessing the request list and submitting request 25113/head
Jerry Lee [Mon, 12 Nov 2018 06:27:53 +0000 (14:27 +0800)]
ceph-mgr: hold lock while accessing the request list and submitting request

The request creation can fire up the notify event early and it can cause
a race condition where the actual request was not yet added to the
self.requests list which makes the submit_request() function waits
forever without accepting new requests.

https://marc.info/?l=ceph-devel&m=154104291714160&w=2

Fixes: https://tracker.ceph.com/issues/36764
Signed-off-by: Jerry Lee <leisurelysw24@gmail.com>
(cherry picked from commit b09aefc2d6af3de8cc3df30258dfdc639c919a9f)

Conflicts:
src/pybind/mgr/restful/module.py

6 years agoMerge pull request #25103 from alfredodeza/mimic-rm-27062
Alfredo Deza [Wed, 14 Nov 2018 23:16:45 +0000 (18:16 -0500)]
Merge pull request #25103 from alfredodeza/mimic-rm-27062

mimic ceph-volume: reject devices that have existing GPT headers

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #25100 from alfredodeza/mimic-bz1644828
Andrew Schoen [Wed, 14 Nov 2018 20:17:35 +0000 (14:17 -0600)]
Merge pull request #25100 from alfredodeza/mimic-bz1644828

mimic ceph-volume: remove LVs when using zap --destroy

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: update tests after removing the LVPath arg validator 25103/head
Andrew Schoen [Wed, 14 Nov 2018 16:35:32 +0000 (10:35 -0600)]
ceph-volume: update tests after removing the LVPath arg validator

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit 0cb95ace88a731f7c30adf3f2286691fa3b2c49c)

6 years agoceph-volume: return lvs as vg/lv from ValidDevice if as_string is True
Andrew Schoen [Tue, 13 Nov 2018 19:02:39 +0000 (13:02 -0600)]
ceph-volume: return lvs as vg/lv from ValidDevice if as_string is True

All code in prepare and activate expect lvs to be in that format.
Eventually we can refactor all that to use the Device class as well.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit fdf7f3c575299b6be0ac40fb6eee5106cea010c9)

6 years agoceph-volume: adds lv_name property to Device class
Andrew Schoen [Tue, 13 Nov 2018 18:40:45 +0000 (12:40 -0600)]
ceph-volume: adds lv_name property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit cc0317efac5a3a25edcdb48af2a63a969c2ac43a)

6 years agoceph-volume: combine the LVPath arg validator with ValidDevice
Andrew Schoen [Tue, 13 Nov 2018 18:28:23 +0000 (12:28 -0600)]
ceph-volume: combine the LVPath arg validator with ValidDevice

This combines the two arg validators and makes both batch and
prepare/active both use ValidDevice.

This will also allow us to use lvs with a full path, like
/dev/vg/lv instead of always enforcing vg/lv.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit d6a5a7606c3a0c1f1624c5784abb02bb1a4e1620)

6 years agoceph-volume: adds has_gpt_headers property to Device class
Andrew Schoen [Tue, 13 Nov 2018 18:27:19 +0000 (12:27 -0600)]
ceph-volume: adds has_gpt_headers property to Device class

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit 568a0ce5be0161541772ceacb9b81652f51659f5)

6 years agoceph-volume: add PTTYPE to blkid parser
Andrew Schoen [Tue, 13 Nov 2018 15:35:21 +0000 (09:35 -0600)]
ceph-volume: add PTTYPE to blkid parser

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: rm#27062
(cherry picked from commit 3d86d2dda9fae87e143e5212d5f354a22f02909d)

6 years agoceph-volume tests update tests to handle new SystemExit exceptions in main 25100/head
Alfredo Deza [Tue, 13 Nov 2018 22:06:03 +0000 (17:06 -0500)]
ceph-volume tests update tests to handle new SystemExit exceptions in main

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 27d0a915d4a7f56aafc603a949b4e4c36772d236)

6 years agoceph-volume tests.api verify that Volume object can be removed
Alfredo Deza [Tue, 13 Nov 2018 22:05:31 +0000 (17:05 -0500)]
ceph-volume tests.api verify that Volume object can be removed

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9527cffbe335043f7c5a60162138dfe093ae51da)

6 years agoceph-volume api.lvm allow passing a Volume object to remove_lv
Alfredo Deza [Tue, 13 Nov 2018 21:56:02 +0000 (16:56 -0500)]
ceph-volume api.lvm allow passing a Volume object to remove_lv

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e7daa71604f4d08ace1125bb63dcfe909796d3ab)

6 years agoceph-volume tests.functional.lvm test full removal of LVs
Alfredo Deza [Tue, 13 Nov 2018 21:05:32 +0000 (16:05 -0500)]
ceph-volume tests.functional.lvm test full removal of LVs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4656bf0e333a50e0fca5fd0612cdda98ab5d70f9)

6 years agoceph-volume main raise sys.exit to prevent garbage in help
Alfredo Deza [Tue, 13 Nov 2018 13:32:16 +0000 (08:32 -0500)]
ceph-volume main raise sys.exit to prevent garbage in help

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f695d1716e9fc6252dd713eb8fb08c28b1284611)