]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoMerge pull request #25198 from pdvian/wip-37285-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:24:02 +0000 (08:24 -0800)]
Merge pull request #25198 from pdvian/wip-37285-mimic

mimic: rgw: radosgw-admin: translate reshard status codes (trivial)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25213 from badone/wip-mimic-fix-branch-3.2-placement
Yuri Weinstein [Thu, 3 Jan 2019 16:23:22 +0000 (08:23 -0800)]
Merge pull request #25213 from badone/wip-mimic-fix-branch-3.2-placement

mimic: qa/ceph-ansible: Move "branch" out of "vars" section

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #25218 from k0ste/mimic_backports
Yuri Weinstein [Thu, 3 Jan 2019 16:22:01 +0000 (08:22 -0800)]
Merge pull request #25218 from k0ste/mimic_backports

mimic: mgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric.

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #25227 from pdvian/wip-37288-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:21:16 +0000 (08:21 -0800)]
Merge pull request #25227 from pdvian/wip-37288-mimic

mimic: tests: qa/tasks/ceph: ignore failure when chown ceph:ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25242 from smithfarm/wip-37384-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:19:44 +0000 (08:19 -0800)]
Merge pull request #25242 from smithfarm/wip-37384-mimic

mimic: test: Start using GNU awk and fix archiving directory

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #25252 from pdvian/wip-37363-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:18:36 +0000 (08:18 -0800)]
Merge pull request #25252 from pdvian/wip-37363-mimic

mimic: librbd: fix missing unblock_writes if shrink is not allowed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25153 from jonsger/wip-24827-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:15:34 +0000 (08:15 -0800)]
Merge pull request #25153 from jonsger/wip-24827-mimic

mimic: build/ops: run-make-check.sh ccache tweaks

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25171 from pdvian/wip-36392-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:14:38 +0000 (08:14 -0800)]
Merge pull request #25171 from pdvian/wip-36392-mimic

mimic: rpm: Use hardened LDFLAGS

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25293 from smithfarm/wip-37419-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:13:35 +0000 (08:13 -0800)]
Merge pull request #25293 from smithfarm/wip-37419-mimic

mimic: mgr/balancer: add cmd to list all plans

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25295 from rzarzynski/wip-bug-36248-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:12:58 +0000 (08:12 -0800)]
Merge pull request #25295 from rzarzynski/wip-bug-36248-mimic

mimic: common: fix memory leaks in WeightedPriorityQueue.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25507 from smithfarm/wip-36729-mimic
Yuri Weinstein [Thu, 3 Jan 2019 16:11:42 +0000 (08:11 -0800)]
Merge pull request #25507 from smithfarm/wip-36729-mimic

mimic: osd: Add support for osd_delete_sleep configuration value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #25753 from alfredodeza/mimic-better-devid
Alfredo Deza [Thu, 3 Jan 2019 14:37:43 +0000 (09:37 -0500)]
Merge pull request #25753 from alfredodeza/mimic-better-devid

mimic common/blkdev, ceph-volume: improve get_device_id

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoceph-volume: Micron SSDs don't include vendor name in ID_SERIAL 25753/head
Sage Weil [Thu, 6 Dec 2018 16:02:35 +0000 (10:02 -0600)]
ceph-volume: Micron SSDs don't include vendor name in ID_SERIAL

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3e4a638c931a5198aca74dc4037d0b03d5e563ba)

6 years agoceph-volume: update get_device_id to match in-tree implementation
Sage Weil [Thu, 6 Dec 2018 15:43:36 +0000 (09:43 -0600)]
ceph-volume: update get_device_id to match in-tree implementation

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c967dbf81f85602d2cd044ee5a91ba38cd76bdb9)

6 years ago13.2.3 v13.2.3
Jenkins Build Slave User [Wed, 2 Jan 2019 18:00:57 +0000 (18:00 +0000)]
13.2.3

6 years agoosd: combine recovery/scrub/snap sleep timer into one. 25507/head
Jianpeng Ma [Sat, 28 Apr 2018 02:46:55 +0000 (10:46 +0800)]
osd: combine recovery/scrub/snap sleep timer into one.

One SafeTime on thread. And those events if timer don't take long time.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 610d5196c0132739e1ae133db0c7d896063470ad)

6 years agoMerge pull request #25419 from xiexingguo/wip-mimic-upmap-fixes
Xie Xingguo [Thu, 13 Dec 2018 05:54:59 +0000 (13:54 +0800)]
Merge pull request #25419 from xiexingguo/wip-mimic-upmap-fixes

mimic: osd: backport recent upmap fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25285 from smithfarm/wip-37396-mimic
Yuri Weinstein [Thu, 13 Dec 2018 00:46:16 +0000 (16:46 -0800)]
Merge pull request #25285 from smithfarm/wip-37396-mimic

mimic: tests: /usr/bin/ld: cannot find -lradospp in rados mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>n
6 years agoosd: Add support for osd_delete_sleep to throttle pg deletion
David Zafman [Thu, 25 Oct 2018 05:00:57 +0000 (22:00 -0700)]
osd: Add support for osd_delete_sleep to throttle pg deletion

Fixes: http://tracker.ceph.com/issues/36474
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 92f1f5859f142cb83cc1d1a367853ac91d6b01d4)

Conflicts:
    src/osd/OSD.cc
- use cct->_conf->get_val instead of cct->_conf.get_val

6 years agotest: Set any value for osd_delete_sleep to guarantee we are testing even SSD
David Zafman [Tue, 30 Oct 2018 18:42:05 +0000 (11:42 -0700)]
test: Set any value for osd_delete_sleep to guarantee we are testing even SSD

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3f621a1190f45a780b71b916b31c1a01a20ece96)

6 years agoMerge pull request #25332 from pdvian/wip-36503-mimic
Yuri Weinstein [Wed, 12 Dec 2018 16:01:14 +0000 (08:01 -0800)]
Merge pull request #25332 from pdvian/wip-36503-mimic

mimic: tests: use timeout for fs asok operations

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #25283 from jdurgin/wip-osd-memory-target-mimic
Josh Durgin [Mon, 10 Dec 2018 17:13:37 +0000 (12:13 -0500)]
Merge pull request #25283 from jdurgin/wip-osd-memory-target-mimic

mimic: core: os/bluestore: cache autotuning and memory limit

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25362 from noahdesu/mimic-docs
Noah Watkins [Fri, 7 Dec 2018 19:35:34 +0000 (11:35 -0800)]
Merge pull request #25362 from noahdesu/mimic-docs

mimic: docs: backport edit on github changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoosd/OSDMap: fix upmap mis-killing for erasure-coded PGs 25419/head
xie xingguo [Sat, 1 Dec 2018 09:42:01 +0000 (17:42 +0800)]
osd/OSDMap: fix upmap mis-killing for erasure-coded PGs

The up-set of erasure-coded PGs may include CRUSH_ITEM_NONE,
which as a result causes mis-killing of valid upmap items.

Fixes: https://tracker.ceph.com/issues/37493
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: ningtao <ningtao@sangfor.com.cn>
(cherry picked from commit f043dcc6d8663ed960a354b2441fd3f8dd66c62d)

Conflicts:
        - *maybe_remove_pg_upmaps* input changed, in master
          we now have a passed in *nextmap* parameter

6 years agomon: auto clean up stale upmap items
xie xingguo [Wed, 21 Nov 2018 06:25:45 +0000 (14:25 +0800)]
mon: auto clean up stale upmap items

See https://www.spinics.net/lists/ceph-users/msg49518.html

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4181aa86906886423af80e478e45554606fc7cf6)

Conflicts:
        - *nextmap* has been renamed to *tmpmap*
        - add the *have_pg_upmaps* wrapper which is necessary for unittest

6 years agoosd/OSDMap: add pg-existence sanity check
xie xingguo [Wed, 21 Nov 2018 01:36:21 +0000 (09:36 +0800)]
osd/OSDMap: add pg-existence sanity check

The reason why __get_pg_pool_size(pg)__ or __get_pg_pool_crush_rule(pg)__ fails is
that the pg does not exist anymore. So it generally makes sense to check __pg_exists(pg)__
before moving further.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 79ef3bea7550559b7ea4e2b68dc407c7eb25defa)

Conflicts:
- *nextmap* has been renamed to *tmpmap*

6 years agoosd/OSDMap.cc: move sanity checks to the proper place
xie xingguo [Wed, 20 Jun 2018 01:27:46 +0000 (09:27 +0800)]
osd/OSDMap.cc: move sanity checks to the proper place

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3dcd4eaf867369437720a21daf3c344cc4b9844e)

6 years agoosd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone
xie xingguo [Wed, 20 Jun 2018 01:04:19 +0000 (09:04 +0800)]
osd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone

If an osd is gone or moved out from the specific crush rule,
we should cancel any pg_upmap/pg_upmap_items still bound to
that osd too.

The original code does not work for the above case because
get_parent_of_type() will fail if that osd does not belong
to the crush_rule passed in and hence hits the assert below:

```
src/osd/OSDMap.cc: 4078: FAILED assert(target > 0)
```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 8a11ed479d7ea95f76170e74123ca2c48e3a3963)

6 years agoMerge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id
Alfredo Deza [Tue, 4 Dec 2018 16:50:42 +0000 (11:50 -0500)]
Merge pull request #25349 from alfredodeza/mimic-c-v-inventory-add-device_id

mimic ceph-volume add device_id to inventory listing

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agodoc/_templates/page.html: add link to report doc bugs 25362/head
Neha Ojha [Fri, 19 Oct 2018 20:26:53 +0000 (13:26 -0700)]
doc/_templates/page.html: add link to report doc bugs

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de30c4d3c5927bb58b49b44b391647e397e1fe4f)

6 years agoluminous: doc: show edit on github links and version warnings
Noah Watkins [Mon, 1 Oct 2018 23:54:19 +0000 (16:54 -0700)]
luminous: doc: show edit on github links and version warnings

backport of #24452 that adds edit on
github links to documentation and notification banners that display
warnings when old documentation is being viewed.

this is not a cherry-pick: it removes from the original patch the
dynamic generation of the releases schedule from a yaml database file.
backporting this portion requires modifying the patch to deal with a
different file / directory structure [in luminous] with no real added value.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
 Conflicts:
doc/conf.py

6 years agoceph-volume: add test for disk.udevadm_property 25349/head
Jan Fajerski [Wed, 28 Nov 2018 09:21:36 +0000 (10:21 +0100)]
ceph-volume: add test for disk.udevadm_property

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 58316e3f826f8bb91b375589aa3676b664ce3984)

6 years agoceph-volume: add device_id field to inventory output
Jan Fajerski [Wed, 21 Nov 2018 12:20:45 +0000 (13:20 +0100)]
ceph-volume: add device_id field to inventory output

This intends to mimic the C++ implementation in src/common/blkdev.cc.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 41da5e9a4334628db09e3166eaceff79b9b6c7e0)

6 years agoqa: use timeout for fs asok operations 25332/head
Patrick Donnelly [Sat, 6 Oct 2018 17:32:18 +0000 (10:32 -0700)]
qa: use timeout for fs asok operations

Otherwise a bug preventing an asok operation from completing will cause the
entire job to fail.

Fixes: http://tracker.ceph.com/issues/36335
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 296d89b0463ee3aad8dfb50b0e41a27e2c852ddb)

6 years agoMerge pull request #25294 from smithfarm/wip-37422-mimic
Nathan Cutler [Wed, 28 Nov 2018 21:13:02 +0000 (22:13 +0100)]
Merge pull request #25294 from smithfarm/wip-37422-mimic

mimic: build/ops: Can't compile Ceph on Fedora 29 as it doesn't recognize python*3*-tox as an install Tox

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #25040 from smithfarm/wip-36749-mimic
Yuri Weinstein [Wed, 28 Nov 2018 20:27:07 +0000 (12:27 -0800)]
Merge pull request #25040 from smithfarm/wip-36749-mimic

mimic: mgr: [restful] deep_scrub is not a valid OSD command

Reviewed-by: Boris Ranto <branto@redhat.com>
6 years agocommon: fix memory leaks in WeightedPriorityQueue. 25295/head
Radoslaw Zarzynski [Fri, 28 Sep 2018 12:57:10 +0000 (14:57 +0200)]
common: fix memory leaks in WeightedPriorityQueue.

Fixes: http://tracker.ceph.com/issues/36248
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 61395d44010f0ac1c183d030b244ae1ec0c8b99a)

6 years agobuild/ops: rpm: stop install-deps.sh clobbering spec file Python build setting 25294/head
Nathan Cutler [Tue, 20 Nov 2018 10:43:13 +0000 (11:43 +0100)]
build/ops: rpm: stop install-deps.sh clobbering spec file Python build setting

Fedora 29 still ships a Python 2 binary, but some of Ceph's build
dependencies are only available in py3 versions there. In other
words, from F29 on, it is no longer possible to do a py2 Ceph build
on Fedora, even if a python2 binary exists on the system.

If that were not enough, the Python 2 that ships with Fedora 29 is
linked against a non-compatible version of OpenSSL.

Before this commit, install-deps.sh was overriding the spec file's
Python build setting based on the presence or absence of a python2
binary. As the bug cited below indicates, this was not a good idea.

It's better for the spec file to be explicit about which OS versions
are py2 and which are py3, and just stick to that.

Fixes: http://tracker.ceph.com/issues/37301
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7d4916618c419e4c85823cf259b96e49d9203035)

Conflicts:
ceph.spec.in
install-deps.sh
- no WITH_SEASTAR in mimic

6 years agobuild/ops: refrain from installing/using lsb_release in install-deps.sh
Nathan Cutler [Wed, 1 Aug 2018 10:52:45 +0000 (12:52 +0200)]
build/ops: refrain from installing/using lsb_release in install-deps.sh

Unfortunately the mapping between release number and codename (which is only
relevant for Debian and Ubuntu btw) is not available from /etc/os-release.
In that one respect, lsb_release was "better".

However, when I weigh the advantages of obtaining that mapping from an external
tool, with the (substantial) risk that the external dependency might cause
trouble on one or more supported distros (to say nothing of the non- or
semi-/pseudo-supported ones), against the work involved in maintaining a
hard-coded mapping (negligible), the needle on my scale immediately swings
toward eliminating the dependency.

Also, I see this commit as part of the longer-term effort to completely expunge
lsb_release from our codebase. See git log --grep lsb_release.

For another example of an external distro-detection tool (albeit one that was
included in Python 2) gone awry, see http://tracker.ceph.com/issues/18163.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 38508b41dc7f33a0b35e3633be3da1d2dae844dc)

6 years agomgr/balancer: add cmd to list all plans 25293/head
Yang Honggang [Fri, 11 May 2018 02:30:09 +0000 (02:30 +0000)]
mgr/balancer: add cmd to list all plans

Signed-off-by: Yang Honggang <yanghonggang@umcloud.com>
(cherry picked from commit 99245c69829c53dab6790da5502420c7183c6bcd)

6 years agoMerge pull request #25000 from MrStupnikov/mimic-docfix-1358
Nathan Cutler [Tue, 27 Nov 2018 21:05:34 +0000 (22:05 +0100)]
Merge pull request #25000 from MrStupnikov/mimic-docfix-1358

mimic: doc: Put command template into literal block

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoqa/workunits/rados/test_librados_build.sh: grab files from explicit git branch 25285/head
Nathan Cutler [Mon, 26 Nov 2018 23:33:36 +0000 (00:33 +0100)]
qa/workunits/rados/test_librados_build.sh: grab files from explicit git branch

Fixes: https://tracker.ceph.com/issues/37392
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 6c6ae9e30d02a0844289f249269e7dcc0058ab88)

Conflicts:
    qa/workunits/rados/test_librados_build.sh
- use mimic as the branch in mimic
- similar change in the comment line, below

6 years agoPendingReleaseNotes: add note about osd memory target and bluestore cache 25283/head
Josh Durgin [Tue, 27 Nov 2018 04:01:38 +0000 (23:01 -0500)]
PendingReleaseNotes: add note about osd memory target and bluestore cache

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
6 years agodoc/rados: Add bluestore memory autotuning docs.
Mark Nelson [Mon, 12 Nov 2018 23:17:42 +0000 (17:17 -0600)]
doc/rados: Add bluestore memory autotuning docs.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9e2d1a52abacbab2b4dab24bd209dc12d26fe2db)

Conflicts:
doc/rados/configuration/bluestore-config-ref.rst

Deployment docs updated in master, pulled in those here too

6 years agoos/bluestore: limit OSD memory usage by tuning the cache size.
Mark Nelson [Wed, 20 Jun 2018 02:59:26 +0000 (21:59 -0500)]
os/bluestore: limit OSD memory usage by tuning the cache size.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 1b8a87b39a52b10ac3b1bd3c70062d4dcd53ed5e)

Conflicts:
src/common/options.cc
src/os/bluestore/BlueStore.cc
trivial - no g_conf operator. or option TYPE_SIZE

6 years agokv/RocksDBStore: Handle nullptr if clock cache is chosen.
Mark Nelson [Thu, 9 Aug 2018 21:50:16 +0000 (16:50 -0500)]
kv/RocksDBStore: Handle nullptr if clock cache is chosen.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9202285b44ce144864a80cb063269f863bb7d6da)

Conflicts:
src/kv/RocksDBStore.cc

trivial - g_conf operator()

6 years agokv/rocksdb_cache/BinnedLRUCache: Don't promote data to the high pri pool.
Mark Nelson [Tue, 3 Jul 2018 22:30:23 +0000 (17:30 -0500)]
kv/rocksdb_cache/BinnedLRUCache: Don't promote data to the high pri pool.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 269713d0f80a611fc61f5946738a879dcd55597b)

6 years agosrc/kv: Initial import of a custom RocksDB cache.
Mark Nelson [Tue, 3 Jul 2018 15:42:14 +0000 (10:42 -0500)]
src/kv: Initial import of a custom RocksDB cache.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9dd6b2885e6d83b5ad468a7343aa2f1d476e818b)

Conflicts:
src/kv/RocksDBStore.cc
trivial - g_conf has no operator()

6 years agoos/bluestore: Fix potential uint64_t to int conversion issue.
Mark Nelson [Tue, 29 May 2018 19:23:41 +0000 (14:23 -0500)]
os/bluestore: Fix potential uint64_t to int conversion issue.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 1d51e764500f519018f572282ef0a269914edfd4)

6 years agocommon/PriorityCache: First pass at prototype.
Mark Nelson [Thu, 26 Apr 2018 14:57:00 +0000 (09:57 -0500)]
common/PriorityCache: First pass at prototype.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit cb465411921ab01ea47b4ac1d37a29662213355b)

Conflicts:
src/os/bluestore/BlueStore.cc (trivial)

6 years agoMerge pull request #24918 from smithfarm/wip-35071-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:56:40 +0000 (12:56 -0800)]
Merge pull request #24918 from smithfarm/wip-35071-mimic

mimic: core: FAILED assert(osdmap_manifest.pinned.empty()) in OSDMonitor::prune_init()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
6 years agoMerge pull request #24970 from smithfarm/wip-35843-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:55:47 +0000 (12:55 -0800)]
Merge pull request #24970 from smithfarm/wip-35843-mimic

mimic: core: objecter cannot resend split-dropped op when racing with con reset

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #24971 from smithfarm/wip-36432-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:54:25 +0000 (12:54 -0800)]
Merge pull request #24971 from smithfarm/wip-36432-mimic

mimic: core: Interactive mode CLI prints no output since Mimic

Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #25016 from smithfarm/wip-36433-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:53:55 +0000 (12:53 -0800)]
Merge pull request #25016 from smithfarm/wip-36433-mimic

mimic: core: monstore tool rebuild does not generate creating_pgs

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25017 from smithfarm/wip-36435-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:53:25 +0000 (12:53 -0800)]
Merge pull request #25017 from smithfarm/wip-36435-mimic

mimic: core: rados rm --force-full is blocked when cluster is in full status

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #25019 from smithfarm/wip-36505-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:52:52 +0000 (12:52 -0800)]
Merge pull request #25019 from smithfarm/wip-36505-mimic

mimic: core: mon osdmap cash too small during upgrade to mimic

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #25024 from smithfarm/wip-36557-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:52:20 +0000 (12:52 -0800)]
Merge pull request #25024 from smithfarm/wip-36557-mimic

mimic: osd: RBD client IOPS pool stats are incorrect (2x higher; includes IO hints as an op)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25026 from smithfarm/wip-36637-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:51:42 +0000 (12:51 -0800)]
Merge pull request #25026 from smithfarm/wip-36637-mimic

mimic: osd: race condition opening heartbeat connection

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agoMerge pull request #25027 from smithfarm/wip-36647-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:51:06 +0000 (12:51 -0800)]
Merge pull request #25027 from smithfarm/wip-36647-mimic

mimic: tests: librados api aio tests race condition

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #25029 from smithfarm/wip-36465-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:50:17 +0000 (12:50 -0800)]
Merge pull request #25029 from smithfarm/wip-36465-mimic

mimic: core: mgr crash on scrub of unconnected osd

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25045 from tchaikov/wip-36219-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:49:48 +0000 (12:49 -0800)]
Merge pull request #25045 from tchaikov/wip-36219-mimic

mimic: msg: ceph_abort() when there are enough accepter errors in msg server

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25062 from jonsger/wip-36754-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:49:01 +0000 (12:49 -0800)]
Merge pull request #25062 from jonsger/wip-36754-mimic

mimic: core: _aio_log_start inflight overlap of 0x10000~1000 with [65536~4096]

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #25075 from jonsger/wip-36658-mimic
Yuri Weinstein [Mon, 26 Nov 2018 20:48:14 +0000 (12:48 -0800)]
Merge pull request #25075 from jonsger/wip-36658-mimic

mimic: core: Objecter: add ignore cache flag if got redirect reply

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
6 years agolibrbd: fix missing unblock_writes if shrink is not allowed 25252/head
runsisi [Mon, 12 Nov 2018 12:01:32 +0000 (20:01 +0800)]
librbd: fix missing unblock_writes if shrink is not allowed

Fixes: http://tracker.ceph.com/issues/36778
Signed-off-by: runsisi <luo.runbing@zte.com.cn>
(cherry picked from commit 3899bee9f5ea2c4b19fb1266a8b59f6e04e99926)

6 years agotest: Start using GNU awk and fix archiving directory 25242/head
Willem Jan Withagen [Thu, 6 Sep 2018 09:10:32 +0000 (11:10 +0200)]
test: Start using GNU awk and fix archiving directory

awk uses some tests that the native FreeBSD awk does not support:
    like: BEGIN{print 0 < 90}

And TESTDIR is not set when calling ceph-helpers from smoke.sh
    So fix with keeping the archive in /tmp

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit bfe7a2afaa5f41e785590d6315313553cadaf7dc)

6 years agoMerge pull request #25050 from gregsfortytwo/mimic-readable-failures
Nathan Cutler [Fri, 23 Nov 2018 18:15:45 +0000 (19:15 +0100)]
Merge pull request #25050 from gregsfortytwo/mimic-readable-failures

mimic: tests: make readable.sh fail if it doesn't run anything

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25228 from ceph/revert-25091-wip-36463-mimic
Yan, Zheng [Fri, 23 Nov 2018 03:31:51 +0000 (11:31 +0800)]
Merge pull request #25228 from ceph/revert-25091-wip-36463-mimic

Revert "mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen"

6 years agoRevert "mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen" 25228/head
Yan, Zheng [Fri, 23 Nov 2018 03:29:52 +0000 (11:29 +0800)]
Revert "mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen"

6 years agoqa/tasks/ceph: ignore failure when chown ceph:ceph 25227/head
Kefu Chai [Wed, 17 Jan 2018 14:20:39 +0000 (22:20 +0800)]
qa/tasks/ceph: ignore failure when chown ceph:ceph

we didn't have ceph:ceph user back in hammer. so ignore this error.
ceph-common will create this user in newer version. and `ceph` task will
take care of this, when upgrading to a newer version.

Fixes: http://tracker.ceph.com/issues/22597
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9f5238037a1afa9b1e5407c95466f20f7b990b41)

6 years agoMerge pull request #24913 from smithfarm/wip-36428-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:49:16 +0000 (08:49 -0800)]
Merge pull request #24913 from smithfarm/wip-36428-mimic

mimic: tests: rbd: move OpenStack devstack test to rocky release

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24915 from smithfarm/wip-36529-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:48:51 +0000 (08:48 -0800)]
Merge pull request #24915 from smithfarm/wip-36529-mimic

mimic: rbd: TokenBucketThrottle: use reference to m_blockers.front() and then update it.

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24916 from smithfarm/wip-36555-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:48:29 +0000 (08:48 -0800)]
Merge pull request #24916 from smithfarm/wip-36555-mimic

mimic: rbd: [rbd-mirror] periodic mirror status timer might fail to be scheduled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24946 from jonsger/wip-36689-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:47:55 +0000 (08:47 -0800)]
Merge pull request #24946 from jonsger/wip-36689-mimic

mimic: tests: unittest_rbd_mirror: TestMockImageMap.AddInstancePingPongImageTest: Value of: it != peer_ack_ctxs->end()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24952 from jonsger/wip-36693-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:47:32 +0000 (08:47 -0800)]
Merge pull request #24952 from jonsger/wip-36693-mimic

mimic: rbd: [rbd-mirror] forced promotion after killing remote cluster results in stuck state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24988 from iridescent-rsy/mimic-36713
Yuri Weinstein [Thu, 22 Nov 2018 16:47:11 +0000 (08:47 -0800)]
Merge pull request #24988 from iridescent-rsy/mimic-36713

mimic: rbd: rbd-nbd: do not ceph_abort() after print the usages

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25113 from swinds24/wip-36764-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:46:40 +0000 (08:46 -0800)]
Merge pull request #25113 from swinds24/wip-36764-mimic

mimic: mgr: hold lock while accessing the request list and submittin request

Reviewed-by: Boris Ranto <branto@redhat.com>
6 years agoMerge pull request #25115 from mcv21/mimic-36741
Yuri Weinstein [Thu, 22 Nov 2018 16:46:07 +0000 (08:46 -0800)]
Merge pull request #25115 from mcv21/mimic-36741

mimic: build/ops: debian: correct ceph-common relationship with older radosgw package

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25091 from jonsger/wip-36463-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:40:51 +0000 (08:40 -0800)]
Merge pull request #25091 from jonsger/wip-36463-mimic

mimic: cephfs: ceph-fuse client can't read or write due to backward cap_gen

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #24987 from joscollin/wip-36690-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:40:10 +0000 (08:40 -0800)]
Merge pull request #24987 from joscollin/wip-36690-mimic

mimic: cephfs: client: request next osdmap for blacklisted client

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #24993 from jonsger/wip-36457-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:39:45 +0000 (08:39 -0800)]
Merge pull request #24993 from jonsger/wip-36457-mimic

mimic: cephfs: client: explicitly show blacklisted state via asok status command

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25036 from smithfarm/wip-36218-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:39:17 +0000 (08:39 -0800)]
Merge pull request #25036 from smithfarm/wip-36218-mimic

mimic: cephfs: some tool commands silently operate on only rank 0, even if multiple ranks exist

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25042 from smithfarm/wip-36461-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:38:48 +0000 (08:38 -0800)]
Merge pull request #25042 from smithfarm/wip-36461-mimic

mimic: mds: rctime not set on system inode (root) at startup

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25095 from jonsger/wip-37093-mimic
Yuri Weinstein [Thu, 22 Nov 2018 16:37:20 +0000 (08:37 -0800)]
Merge pull request #25095 from jonsger/wip-37093-mimic

mimic: mds: src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped) during max_mds thrashing

Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agomgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric. 25218/head
Konstantin Shalygin [Tue, 30 Oct 2018 06:43:02 +0000 (13:43 +0700)]
mgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric.

Signed-off-by: Konstantin Shalygin <k0ste@k0ste.ru>
(cherry picked from commit 12fbf48e637ca807d4d489eebdfe64bb8cd1909d)

6 years agoqa/ceph-ansible: Move "branch" out of "vars" section 25213/head
Brad Hubbard [Thu, 22 Nov 2018 02:07:15 +0000 (12:07 +1000)]
qa/ceph-ansible: Move "branch" out of "vars" section

This was mistakenly added to the wrong section.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge PR #25118 into mimic
Patrick Donnelly [Wed, 21 Nov 2018 18:11:16 +0000 (10:11 -0800)]
Merge PR #25118 into mimic

* refs/pull/25118/head:
mds: make timeout parameter optional for "cache drop"
test: add test for mds drop cache command
mds: command to trim mds cache and client caps
mds: implement journal flush as asynchronous context execution
mds: cleanup some asok commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25191 from badone/wip-mimic-ceph_ansible-specify-branch-stable-3.2
Brad Hubbard [Wed, 21 Nov 2018 07:05:43 +0000 (17:05 +1000)]
Merge pull request #25191 from badone/wip-mimic-ceph_ansible-specify-branch-stable-3.2

mimic: qa/ceph-ansible: Specify stable-3.2 branch

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #25193 from neha-ojha/wip-fix-thrash-old-clients-mimic
Neha Ojha [Wed, 21 Nov 2018 06:04:10 +0000 (22:04 -0800)]
Merge pull request #25193 from neha-ojha/wip-fix-thrash-old-clients-mimic

mimic: qa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoradosgw-admin: translate reshard status codes (trivial) 25198/head
Matt Benjamin [Wed, 17 Oct 2018 14:43:01 +0000 (10:43 -0400)]
radosgw-admin: translate reshard status codes (trivial)

Fixes: http://tracker.ceph.com/issues/36486
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 731fca4f921e8227e907b204dec9f1016d66b8c3)

6 years agoqa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel 25193/head
Neha Ojha [Tue, 20 Nov 2018 23:37:25 +0000 (23:37 +0000)]
qa/suites/rados/thrash-old-clients: exclude packages for hammer, jewel

This fix goes directly into the mimic branch since the following packages
do not need to be installed on hammer and jewel.

- python34-rados
- python34-cephfs
- python3-rados
- python3-cephfs

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24899 from ivancich/rgw-renew-reshard-lock-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:12:33 +0000 (09:12 -0800)]
Merge pull request #24899 from ivancich/rgw-renew-reshard-lock-mimic

mimic: rgw: renew resharding locks to prevent expiration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24907 from smithfarm/wip-36223-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:12:07 +0000 (09:12 -0800)]
Merge pull request #24907 from smithfarm/wip-36223-mimic

mimic: rgw: default quota not set in radosgw for Openstack users

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24908 from smithfarm/wip-36415-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:11:39 +0000 (09:11 -0800)]
Merge pull request #24908 from smithfarm/wip-36415-mimic

mimic: librgw: crashes in multisite configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24910 from smithfarm/wip-36533-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:11:14 +0000 (09:11 -0800)]
Merge pull request #24910 from smithfarm/wip-36533-mimic

mimic: rgw: cls_user_remove_bucket does not write the modified cls_user_stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24911 from smithfarm/wip-36535-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:10:49 +0000 (09:10 -0800)]
Merge pull request #24911 from smithfarm/wip-36535-mimic

mimic: rgw: radosgw-admin user stats are incorrect when dynamic re-sharding is enabled

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24912 from smithfarm/wip-36539-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:10:15 +0000 (09:10 -0800)]
Merge pull request #24912 from smithfarm/wip-36539-mimic

mimic: use-after-free from RGWRadosGetOmapKeysCR::~RGWRadosGetOmapKeysCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25079 from jonsger/wip-36734-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:09:48 +0000 (09:09 -0800)]
Merge pull request #25079 from jonsger/wip-36734-mimic

mimic: rgw: beast frontend fails to parse ipv6 endpoints

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #25085 from smithfarm/wip-37153-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:09:21 +0000 (09:09 -0800)]
Merge pull request #25085 from smithfarm/wip-37153-mimic

mimic: tests: ceph-admin-commands.sh workunit does not log what it's doing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #25087 from jonsger/wip-36756-mimic
Yuri Weinstein [Tue, 20 Nov 2018 17:08:41 +0000 (09:08 -0800)]
Merge pull request #25087 from jonsger/wip-36756-mimic

mimic: rgw-admin: reshard add can add a non existant bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agorpm: Use hardened LDFLAGS 25171/head
Boris Ranto [Thu, 4 Oct 2018 11:05:14 +0000 (13:05 +0200)]
rpm: Use hardened LDFLAGS

Currently, we do pass the hardened CFLAGS and CPPFLAGS when building the
code. However, we do not pass the hardened flags to the linker. This
means that the binaries are linked without the options like -Wl,-z,now.
As a result, we do not fully harden the binaries that we build.

This commit fixes this by passing the RPM_LD_FLAGS to the linker so the
builds are linked with the properly hardened flags.

Fixes: http://tracker.ceph.com/issues/36316
Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit ea6d7d2c9878b08528d120aa35e2fd9e18b3ba0f)