]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agotest: Correct typo trying to call flush_pg_stats
David Zafman [Tue, 27 Nov 2018 00:46:41 +0000 (16:46 -0800)]
test: Correct typo trying to call flush_pg_stats

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 879d89aace47bff63ddd08bd470ded7f1904eeaf)

6 years agoMerge pull request #26129 from pdvian/wip-38039-mimic
Yuri Weinstein [Mon, 18 Feb 2019 17:37:42 +0000 (09:37 -0800)]
Merge pull request #26129 from pdvian/wip-38039-mimic

mimic: rgw: fails to start on Fedora 28 from default configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26266 from pdvian/wip-38147-mimic
Yuri Weinstein [Mon, 18 Feb 2019 17:37:01 +0000 (09:37 -0800)]
Merge pull request #26266 from pdvian/wip-38147-mimic

mimic: rgw: "radosgw-admin bucket rm ... --purge-objects" can hang

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #26277 from ceph/wip-wjw-boost_noncopyable-mimic
Yuri Weinstein [Mon, 18 Feb 2019 17:36:01 +0000 (09:36 -0800)]
Merge pull request #26277 from ceph/wip-wjw-boost_noncopyable-mimic

mimic: common: fix missing include boost/noncopyable.hpp

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26304 from dzafman/wip-38208
Yuri Weinstein [Fri, 15 Feb 2019 22:38:53 +0000 (14:38 -0800)]
Merge pull request #26304 from dzafman/wip-38208

mimic: mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_IN…

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #26342 from ifed01/wip-ifed-fix-compress-mimic2
Yuri Weinstein [Fri, 15 Feb 2019 22:37:46 +0000 (14:37 -0800)]
Merge pull request #26342 from ifed01/wip-ifed-fix-compress-mimic2

mimic: os/bluestore: do not assert on non-zero err codes from compress() call

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26354 from pdvian/wip-38239-mimic
Yuri Weinstein [Fri, 15 Feb 2019 22:37:09 +0000 (14:37 -0800)]
Merge pull request #26354 from pdvian/wip-38239-mimic

mimic: qa/tasks/radosbench: default to 64k writes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26344 from liewegas/wip-centos-notmcalloc-only-mimic
Sage Weil [Tue, 12 Feb 2019 19:01:26 +0000 (14:01 -0500)]
Merge pull request #26344 from liewegas/wip-centos-notmcalloc-only-mimic

mimic: qa: don't use notcmalloc flavor unless we need it

6 years agoMerge pull request #25944 from wido/mimic-zabbix-pg
Yuri Weinstein [Tue, 12 Feb 2019 17:45:38 +0000 (09:45 -0800)]
Merge pull request #25944 from wido/mimic-zabbix-pg

mimic: mgr/zabbix: Send more PG information to Zabbix

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26376 from tchaikov/mimic-38266
Yuri Weinstein [Tue, 12 Feb 2019 17:23:39 +0000 (09:23 -0800)]
Merge pull request #26376 from tchaikov/mimic-38266

mimic: tests: Allow multi instances of "make tests" on the same machine

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agotest: listen on random port in tests which start ceph-mon 26376/head
Kefu Chai [Sun, 27 Jan 2019 06:27:57 +0000 (14:27 +0800)]
test: listen on random port in tests which start ceph-mon

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 94a84b6f5adad75b3cbf956447ee5d9caf1018d2)

6 years agotest: write log file to current directory
Kefu Chai [Wed, 26 Dec 2018 04:58:08 +0000 (12:58 +0800)]
test: write log file to current directory

instead of writing log files to '/tmp', write them to current directory.

by default, ctest runs the tests in ${CMAKE_CURRENT_BINARY_DIR}, so, in
this case, the log files are located in build/src/test/. this ensure
that current user has enough previledge to remove the log files.

this also allows multiple instances of 'unittest_log' to run in parallel.

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a13a6f1d0b39a541896591cd04fc1980bb1acfee)

Conflicts:
src/log/test.cc: trivial resolution

6 years agoMerge pull request #26373 from pdvian/wip-38096-mimic
Nathan Cutler [Tue, 12 Feb 2019 15:10:42 +0000 (16:10 +0100)]
Merge pull request #26373 from pdvian/wip-38096-mimic

mimic: doc: rados/configuration: refresh osdmap section

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25721 from ashishkumsingh/wip-37689-mimic
Yuri Weinstein [Tue, 12 Feb 2019 00:03:52 +0000 (16:03 -0800)]
Merge pull request #25721 from ashishkumsingh/wip-37689-mimic

mimic: ceph-objectstore-tool: Dump hashinfo

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #26077 from ashishkumsingh/wip-37992-mimic
Yuri Weinstein [Tue, 12 Feb 2019 00:02:01 +0000 (16:02 -0800)]
Merge pull request #26077 from ashishkumsingh/wip-37992-mimic

mimic: osd: keep using cache even if op will invalid cache

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26235 from pdvian/wip-38109-mimic
Yuri Weinstein [Tue, 12 Feb 2019 00:01:25 +0000 (16:01 -0800)]
Merge pull request #26235 from pdvian/wip-38109-mimic

mimic: mgr will refuse connection from the monitor who starts behind it

Reviewed-by: Tim Serong <tserong@suse.com>
6 years agoMerge pull request #26239 from pdvian/wip-38106-mimic
Yuri Weinstein [Tue, 12 Feb 2019 00:00:49 +0000 (16:00 -0800)]
Merge pull request #26239 from pdvian/wip-38106-mimic

mimic: osd/PG.cc: account for missing set irrespective of last_complete

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26260 from pdvian/wip-38143-mimic
Yuri Weinstein [Tue, 12 Feb 2019 00:00:07 +0000 (16:00 -0800)]
Merge pull request #26260 from pdvian/wip-38143-mimic

mimic: os/bluestore: fixup access a destroy cond cause deadlock or undefine

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agodoc/rados/configuration: refresh osdmap section 26373/head
Ilya Dryomov [Thu, 24 Jan 2019 20:07:09 +0000 (21:07 +0100)]
doc/rados/configuration: refresh osdmap section

"osd map cache size" and "osd map message max" were reduced in commit
855955e58e63 ("osd: reduce size of osdmap cache, messages").

"osd map cache bl size" and "osd map cache bl inc size" were removed
six years ago.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ae88c63d0441b1dffcc1dc469f4693dc909650bf)

6 years agoMerge pull request #26288 from neha-ojha/wip-wl-slow-mimic
Yuri Weinstein [Mon, 11 Feb 2019 21:09:08 +0000 (13:09 -0800)]
Merge pull request #26288 from neha-ojha/wip-wl-slow-mimic

mimic: qa/tasks/thrashosds-health.yaml: whitelist "slow request"

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #26291 from pdvian/wip-38187-mimic
Yuri Weinstein [Mon, 11 Feb 2019 21:08:25 +0000 (13:08 -0800)]
Merge pull request #26291 from pdvian/wip-38187-mimic

mimic: deep fsck fails on inspecting very large onodes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #26324 from smithfarm/wip-38111-mimic
Yuri Weinstein [Mon, 11 Feb 2019 21:07:27 +0000 (13:07 -0800)]
Merge pull request #26324 from smithfarm/wip-38111-mimic

mimic: osd: force-backfill sets forced_recovery instead of forced_backfill in 13.2.1

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoqa/tasks/radosbench: default to 64k writes 26354/head
Sage Weil [Sat, 5 Jan 2019 02:56:56 +0000 (20:56 -0600)]
qa/tasks/radosbench: default to 64k writes

The default rados bench is 4MB writes, but this tends to fill up our
tiny test clusters.  Use 64KB instead.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 78070b96bf72c0724caad0afdb3d2ae0197cc95b)

6 years agoqa/tasks/keystone.py: no need for notcmalloc in example 26344/head
Sage Weil [Fri, 8 Feb 2019 15:12:01 +0000 (09:12 -0600)]
qa/tasks/keystone.py: no need for notcmalloc in example

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9c729a78be29b7ad017dc652039057a0861c48ba)

6 years agoqa/suites/rgw/tempest/tasks/rgw_tempest: no need for notcmalloc
Sage Weil [Fri, 8 Feb 2019 15:11:07 +0000 (09:11 -0600)]
qa/suites/rgw/tempest/tasks/rgw_tempest: no need for notcmalloc

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b8b398e1abfdfc9ad7b290378b5698c31675413b)

6 years agoos/bluestore: do not assert on non-zero err codes from compress() call 26342/head
Igor Fedotov [Thu, 10 Jan 2019 09:58:05 +0000 (12:58 +0300)]
os/bluestore: do not assert on non-zero err codes from  compress() call

Fixes: https://tracker.ceph.com/issues/37839
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 4083bc29b6b4191a5ef3c17976eeac091626f1b0)

6 years agoMerge pull request #26329 from dzafman/wip-37393
David Zafman [Fri, 8 Feb 2019 16:50:18 +0000 (08:50 -0800)]
Merge pull request #26329 from dzafman/wip-37393

mimic: osd-backfill-stats.sh fails in rados/standalone/osd.yaml

Fixes: http://tracker.ceph.com/issues/37393
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26333 from alfredodeza/mimic-rm38229
Andrew Schoen [Fri, 8 Feb 2019 15:58:50 +0000 (09:58 -0600)]
Merge pull request #26333 from alfredodeza/mimic-rm38229

mimic: doc: pin the version for "breathe" to 4.1.11

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agodoc: pin the version for `breathe` to 4.1.11 26333/head
Alfredo Deza [Fri, 8 Feb 2019 12:48:57 +0000 (07:48 -0500)]
doc: pin the version for `breathe` to 4.1.11

It has the compatibility needed for Sphinx version 1.6.3

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agotest: osd-backfill-stats.sh Fix check of multi backfill OSDs, skip remapped test 26329/head
David Zafman [Thu, 7 Feb 2019 23:48:48 +0000 (15:48 -0800)]
test: osd-backfill-stats.sh Fix check of multi backfill OSDs, skip remapped test

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 6d59fdd9710b79685325e8c445948745e7ec0002)

6 years agotest: Fix test to detect a test setup failure
David Zafman [Tue, 14 Aug 2018 22:59:29 +0000 (15:59 -0700)]
test: Fix test to detect a test setup failure

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit c1b2bd7f16fe4ffb63baa48e8a154f275bd20126)

6 years agotest: osd-backfill-stats.sh: Fix subtests to get primary which can change
David Zafman [Thu, 13 Sep 2018 20:15:09 +0000 (13:15 -0700)]
test: osd-backfill-stats.sh: Fix subtests to get primary which can change

Fixes: http://tracker.ceph.com/issues/35982
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ef6940fbb6c084b7f9abe893a18faef75d3a5680)

6 years agoqa/standalone/osd/osd-backfill-stats: fixes
Sage Weil [Fri, 31 Aug 2018 15:52:04 +0000 (10:52 -0500)]
qa/standalone/osd/osd-backfill-stats: fixes

Grep from the primary's log, not every osd's log.

For the backfill_remapped task in particular, after the pg_temp change it
just so happens that the primary changes across the pool size change and
thus two different primaries do (some) backfill.  Fix that test to pass
the correct primary.

Other tests are unaffected as they do not (happen to) trigger a primary
change and already satisfied the (removed) check that only one OSD does
backfill.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f47921f29397257f77c946eef7ba4346bf6481a4)

6 years agoMerge pull request #26294 from jan--f/25390-mimic
Andrew Schoen [Thu, 7 Feb 2019 21:30:31 +0000 (15:30 -0600)]
Merge pull request #26294 from jan--f/25390-mimic

[mimic] ceph-volume: revert partition as disk

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoosd/PG: fix misused FORCE_RECOVERY[BACKFILL] flags 26324/head
xie xingguo [Tue, 4 Sep 2018 07:34:24 +0000 (15:34 +0800)]
osd/PG: fix misused FORCE_RECOVERY[BACKFILL] flags

__set_force_backfill__ should set PG_STATE_FORCED_BACKFILL
instead of PG_STATE_FORCED_RECOVERY.

Fixes: http://tracker.ceph.com/issues/27985
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 60ef742a84d5201b2a6e6c1780741da0b31c49a8)

6 years agomon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_INCONSISTENT does 26304/head
David Zafman [Mon, 28 Jan 2019 22:30:29 +0000 (14:30 -0800)]
mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_INCONSISTENT does

Fixes: http://tracker.ceph.com/issues/38070
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a4b1574532ee4f789b30634f0eca03cc30ddcf8d)

6 years agoMerge pull request #25856 from ashishkumsingh/wip-37821-mimic
Yuri Weinstein [Wed, 6 Feb 2019 21:37:31 +0000 (13:37 -0800)]
Merge pull request #25856 from ashishkumsingh/wip-37821-mimic

mimic: qa/rados/upgrade: align thrashing with upgrade suite, don't import/export pgs

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26074 from wido/mimic-mgr-perf
Yuri Weinstein [Wed, 6 Feb 2019 21:36:50 +0000 (13:36 -0800)]
Merge pull request #26074 from wido/mimic-mgr-perf

mimic: mgr: add get_latest_counter() to C++ -> Python interface.

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #25957 from pdvian/wip-37900-mimic
Yuri Weinstein [Wed, 6 Feb 2019 17:28:50 +0000 (09:28 -0800)]
Merge pull request #25957 from pdvian/wip-37900-mimic

mimic: [journal] max journal order is incorrectly set at 64

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #25987 from ashishkumsingh/wip-37925-mimic
Yuri Weinstein [Wed, 6 Feb 2019 17:28:17 +0000 (09:28 -0800)]
Merge pull request #25987 from ashishkumsingh/wip-37925-mimic

mimic: common: fix for broken rbdmap parameter parsing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoceph-volume: add inventory call to functional tests 26294/head
Jan Fajerski [Fri, 7 Dec 2018 08:59:34 +0000 (09:59 +0100)]
ceph-volume: add inventory call to functional tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit a183a28c5e423c5658a37f7a1d3add9d64c6d683)

6 years agoceph-volume: add partition support to Device
Jan Fajerski [Tue, 4 Dec 2018 16:00:59 +0000 (17:00 +0100)]
ceph-volume: add partition support to Device

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 05ac680cb37fd15fdeadaea86780bf1b4f2d566a)

6 years agoRevert "ceph-volume tests.util ensure partitions are listed as top-level keys"
Jan Fajerski [Tue, 4 Dec 2018 07:33:53 +0000 (08:33 +0100)]
Revert "ceph-volume tests.util ensure partitions are listed as top-level keys"

This reverts commit 6dc01772245e557b73551111f7c48cc60aec0893.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 3c1c32cbc2736d72295f1bd7921c4a3cd2f4727e)

6 years agoRevert "ceph-volume util.disk include partitions as top level devices in get_devices"
Jan Fajerski [Tue, 4 Dec 2018 07:29:39 +0000 (08:29 +0100)]
Revert "ceph-volume util.disk include partitions as top level devices in get_devices"

This reverts commit 7f3c3598491e556ec1b93ea0a54f52144202fab6.
Fixes: http://tracker.ceph.com/issues/37506
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 50bc44f1c2327ad38504950fe62820a44ead9b14)

6 years agoMerge pull request #25385 from ifed01/wip-ifed-osd-mkfs-fix
Yuri Weinstein [Wed, 6 Feb 2019 04:09:48 +0000 (20:09 -0800)]
Merge pull request #25385 from ifed01/wip-ifed-osd-mkfs-fix

mimic: osd/OSD: OSD::mkfs asserts when reusing disk with existing superblock.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26201 from smithfarm/wip-37688-mimic
Yuri Weinstein [Wed, 6 Feb 2019 04:09:20 +0000 (20:09 -0800)]
Merge pull request #26201 from smithfarm/wip-37688-mimic

mimic: tests: Command failed on smithi191 with status 1: 'sudo yum -y install ceph-radosgw'

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26206 from neha-ojha/wip-37902-mimic
Yuri Weinstein [Wed, 6 Feb 2019 04:08:40 +0000 (20:08 -0800)]
Merge pull request #26206 from neha-ojha/wip-37902-mimic

mimic: osd/mon: fix upgrades for pg log hard limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agodoc: pglog_hardlimit flag recommendations 26206/head
Neha Ojha [Wed, 30 Jan 2019 17:50:21 +0000 (17:50 +0000)]
doc: pglog_hardlimit flag recommendations

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoos/bluestore: cap reads issued during deep fsck 26291/head
Igor Fedotov [Mon, 28 Jan 2019 15:17:52 +0000 (18:17 +0300)]
os/bluestore: cap reads issued during deep fsck

Fixes: https://tracker.ceph.com/issues/38065
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ced308000aef6050ca1fdbdfe3b4b63a887a1706)

6 years agoos/bluestore: do not buffer read data on deep fsck.
Igor Fedotov [Mon, 28 Jan 2019 14:49:32 +0000 (17:49 +0300)]
os/bluestore: do not buffer read data on deep fsck.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit a4e55dd352a28c48029a0f74cb253673e1aca109)

6 years agoMerge pull request #25348 from ifed01/wip-ifed-fix-expand-mimic
Yuri Weinstein [Tue, 5 Feb 2019 21:16:59 +0000 (13:16 -0800)]
Merge pull request #25348 from ifed01/wip-ifed-fix-expand-mimic

mimic: core: os/bluestore_tool: fix bluefs expand

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25525 from smithfarm/wip-36751-mimic
Yuri Weinstein [Tue, 5 Feb 2019 21:16:20 +0000 (13:16 -0800)]
Merge pull request #25525 from smithfarm/wip-36751-mimic

mimic: cmake: do not pass -B{symbolic,symbolic-functions} to linker on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoMerge pull request #25531 from smithfarm/wip-37642-mimic
Yuri Weinstein [Tue, 5 Feb 2019 21:15:16 +0000 (13:15 -0800)]
Merge pull request #25531 from smithfarm/wip-37642-mimic

mimic: ceph-create-keys: fix octal notation for Python 3 without losing compatibility with Python 2

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoqa/tasks/thrashosds-health.yaml: whitelist "slow request" 26288/head
Neha Ojha [Tue, 5 Feb 2019 19:28:58 +0000 (19:28 +0000)]
qa/tasks/thrashosds-health.yaml: whitelist "slow request"

https://github.com/ceph/ceph/pull/25824 brought back slow requests in luminous
OSD logs. This is resulting in the upgrade/luminous-x/stress-split* suites in
mimic to show these warnings.

Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25857 from ashishkumsingh/wip-37822-mimic
Yuri Weinstein [Tue, 5 Feb 2019 15:22:40 +0000 (07:22 -0800)]
Merge pull request #25857 from ashishkumsingh/wip-37822-mimic

mimic: mds: log evicted clients to clog/dbg

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25885 from pdvian/wip-37818-mimic
Yuri Weinstein [Tue, 5 Feb 2019 15:21:54 +0000 (07:21 -0800)]
Merge pull request #25885 from pdvian/wip-37818-mimic

mimic: mds: remove wrong assertion in Locker::snapflush_nudge

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #26010 from smithfarm/wip-37819-mimic
Yuri Weinstein [Tue, 5 Feb 2019 15:21:03 +0000 (07:21 -0800)]
Merge pull request #26010 from smithfarm/wip-37819-mimic

mimic: mds: create separate config for heartbeat timeout

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #26055 from smithfarm/wip-37898-mimic
Yuri Weinstein [Tue, 5 Feb 2019 15:20:29 +0000 (07:20 -0800)]
Merge pull request #26055 from smithfarm/wip-37898-mimic

mimic: mds: purge queue recovery hangs during boot if PQ journal is damaged

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #26214 from pdvian/wip-38101-mimic
Yuri Weinstein [Tue, 5 Feb 2019 15:20:02 +0000 (07:20 -0800)]
Merge pull request #26214 from pdvian/wip-38101-mimic

mimic: mds: trim cache after journal flush

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agocommon: fix missing include boost/noncopyable.hpp 26277/head
Willem Jan Withagen [Tue, 25 Sep 2018 21:36:30 +0000 (23:36 +0200)]
common: fix missing include boost/noncopyable.hpp

Clang complains:
```
In file included from /home/jenkins/workspace/ceph-master/src/osd/ReplicatedBackend.cc:15:
In file included from /home/jenkins/workspace/ceph-master/src/osd/ReplicatedBackend.h:18:
In file included from /home/jenkins/workspace/ceph-master/src/osd/PGBackend.h:27:
In file included from /home/jenkins/workspace/ceph-master/src/osd/PGTransaction.h:25:
/home/jenkins/workspace/ceph-master/src/common/inline_variant.h:88:64: error: expected class name
struct generic_visitor : boost::static_visitor<Result>, boost::noncopyable
                                                               ^
1 error generated.
gmake[2]: *** [src/osd/CMakeFiles/osd.dir/build.make:141: src/osd/CMakeFiles/osd.dir/ReplicatedBackend.cc.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit 0654099d4b2f9f9423d7e8cf00e9e3351a24ec67)

6 years agoMerge pull request #26009 from sebastian-philipp/dashboard-mirror_mode
Yuri Weinstein [Mon, 4 Feb 2019 20:49:18 +0000 (12:49 -0800)]
Merge pull request #26009 from sebastian-philipp/dashboard-mirror_mode

mimic: mgr/dashboard: Set mirror_mode to None

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by:  Volker Theile <vtheile@suse.com>

6 years agoMerge pull request #26208 from vumrao/wip-vumrao-38107
Yuri Weinstein [Mon, 4 Feb 2019 20:47:00 +0000 (12:47 -0800)]
Merge pull request #26208 from vumrao/wip-vumrao-38107

mimic: mgr/DaemonServer: log pgmap usage to cluster log

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agorgw: `radosgw-admin bucket rm ... --purge-objects` can hang... 26266/head
J. Eric Ivancich [Thu, 31 Jan 2019 19:21:07 +0000 (14:21 -0500)]
rgw: `radosgw-admin bucket rm ... --purge-objects` can hang...

This command can hang (i.e., enter an infinite loop) due to
problematic bucket index entries left as a result of bug
https://tracker.ceph.com/issues/38007 .

The fix is to ignore the false bucket index entries -- since they do
not represent actual objects -- and remove all actual objects in the
bucket, so that bucket itself can be removed.

This fixes the both code paths whether `--bypass-gc` is specified or
not.

Furthermore, to made these operations more efficient, the internal
listing of the bucket is done unordered. This would improve behavior
when removing buckets with a large number of objects.

Fixes: http://tracker.ceph.com/issues/38134
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 5b3dc38c628728651b26415ab3d2190cf06b9e39)

6 years agoos/bluestore: fixup access a destroy cond cause deadlock or undefine 26260/head
linbing [Fri, 21 Dec 2018 00:22:30 +0000 (08:22 +0800)]
os/bluestore: fixup access a destroy cond cause deadlock or undefine
              behavior.

Fixes: https://tracker.ceph.com/issues/37733
Signed-off-by: linbing <linbing@t2cloud.net>
(cherry picked from commit dbbde306484276251cbd7a7b7a6fde0093f98ba2)

Conflicts:
src/os/bluestore/BlockDevice.h : Resolved in try_aio_wake

6 years agoosd/PG.cc: account for missing set irrespective of last_complete 26239/head
Neha Ojha [Thu, 24 Jan 2019 20:46:49 +0000 (12:46 -0800)]
osd/PG.cc: account for missing set irrespective of last_complete

When adding source info from another OSD, check if an object that needs
recovery is present in its missing set. If yes, do not include the OSD
as a missing loc.

Fixes: http://tracker.ceph.com/issues/37919
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit e6b1637e00a3e98e588e4b83a13db5703fb96fce)

6 years agomgr: Apply sanity check for daemon metadata 26235/head
Xinying Song [Tue, 15 Jan 2019 11:18:20 +0000 (19:18 +0800)]
mgr: Apply sanity check for daemon metadata

Signed-off-by: Xinying Song <songxinying@sensetime.com>
(cherry picked from commit 3c333f3599642f29dc7a080570ed6b1954e49dce)

6 years agomgr: request mon metadata when receiving a report msg from an unknown monitor
cfanz [Sun, 30 Dec 2018 03:49:58 +0000 (11:49 +0800)]
mgr: request mon metadata when receiving a report msg from an unknown monitor

* Mgr needs to issue an metadata request when receiving a report message
  from a monitor who hasn't been in the mgr's daemon_state. Otherwise the
  connection between mgr and monitor cannot be established.

Fixes: http://tracker.ceph.com/issues/37753
Signed-off-by: Xinying Song <songxinying@sensetime.com>
(cherry picked from commit 430ba5e231d6929a004a9758d31b426ecbe43951)

Conflicts:
src/mgr/Mgr.cc : Resolved in finish

6 years agomds: allow boot on read-only 26055/head
Patrick Donnelly [Sun, 23 Dec 2018 22:22:49 +0000 (14:22 -0800)]
mds: allow boot on read-only

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c7ce967b778a0b86b335f6801301e484aaf6ebc3)

Conflicts:
src/mds/MDSRank.cc
- no ceph_assert in mimic

6 years agomds: setup readonly mode for PurgeQueue
Patrick Donnelly [Tue, 18 Dec 2018 23:11:02 +0000 (15:11 -0800)]
mds: setup readonly mode for PurgeQueue

If the PQ faces an error, it should go read-only along with the MDS rank.

Fixes: http://tracker.ceph.com/issues/37543
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4cccc4dffb0915ef9e7d3b446e9a32f277646562)

Conflicts:
src/mds/PurgeQueue.cc
- Mutex::Locker instead of std::lock_guard
- no ceph_assert in mimic

6 years agomds: add missing locks for PurgeQueue methods
Patrick Donnelly [Tue, 18 Dec 2018 23:08:11 +0000 (15:08 -0800)]
mds: add missing locks for PurgeQueue methods

These could race with the asynchronous workings of the PQ.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c7350ac23c73867b52cd9b7bb23b6c618eebe44d)

Conflicts:
src/mds/PurgeQueue.cc
- use Mutex::Locker instead of std::lock_guard
- no "ceph_assert" in mimic

6 years agomds: delete on_error context on des
Patrick Donnelly [Tue, 18 Dec 2018 22:00:29 +0000 (14:00 -0800)]
mds: delete on_error context on des

Otherwise it leaks.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 33279822eabb64380f5968cc645735a8f99a3ac1)

6 years agomds: trim cache after journal flush 26214/head
Patrick Donnelly [Tue, 22 Jan 2019 20:12:55 +0000 (12:12 -0800)]
mds: trim cache after journal flush

Dropping cache should trim the cache only after journal flush otherwise dirty
inodes will remain pinned in cache.

Fixes: http://tracker.ceph.com/issues/38010
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d7f6c1583b4162fade434e376176f3068472f53f)

6 years agomgr/DaemonServer: log pgmap usage to cluster log 26208/head
Neha Ojha [Thu, 24 Jan 2019 03:25:12 +0000 (19:25 -0800)]
mgr/DaemonServer: log pgmap usage to cluster log

- change default of mon_cluster_log_file_level to debug
- log to clog->debug()

Fixes: https://tracker.ceph.com/issues/37886
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f64a69e5054fb319e9f717bbcb7ddd8541e01c6f)

 Conflicts:
src/common/options.cc
        dropped master .add_service, .add_see_also

6 years agoMerge pull request #25306 from trociny/wip-37437-mimic
Yuri Weinstein [Wed, 30 Jan 2019 17:07:36 +0000 (09:07 -0800)]
Merge pull request #25306 from trociny/wip-37437-mimic

mimic: crushtool: add --reclassify operation to convert legacy crush maps to use device classes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #25313 from pdvian/wip-36639-mimic
Yuri Weinstein [Wed, 30 Jan 2019 17:06:04 +0000 (09:06 -0800)]
Merge pull request #25313 from pdvian/wip-36639-mimic

mimic: os/bluestore: rename does not old ref to replacement onode at old name

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
6 years agoMerge pull request #25526 from smithfarm/wip-36753-mimic
Yuri Weinstein [Wed, 30 Jan 2019 17:04:26 +0000 (09:04 -0800)]
Merge pull request #25526 from smithfarm/wip-36753-mimic

mimic: mon: log last command skips latest entry

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #25603 from smithfarm/wip-37686-mimic
Yuri Weinstein [Wed, 30 Jan 2019 17:03:47 +0000 (09:03 -0800)]
Merge pull request #25603 from smithfarm/wip-37686-mimic

mimic: core: list-inconsistent-obj output truncated, causing osd-scrub-repair.sh failure

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #25694 from joscollin/wip-37624-mimic
Yuri Weinstein [Wed, 30 Jan 2019 17:03:10 +0000 (09:03 -0800)]
Merge pull request #25694 from joscollin/wip-37624-mimic

mimic: pybind/mgr/status: fix ceph fs status in py3 environments.

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25708 from joscollin/wip-37755-mimic
Yuri Weinstein [Wed, 30 Jan 2019 17:02:01 +0000 (09:02 -0800)]
Merge pull request #25708 from joscollin/wip-37755-mimic

mimic: osd/PrimaryLogPG: fix the extent length error of the sync read

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #25822 from batrick/i36209
Yuri Weinstein [Wed, 30 Jan 2019 17:01:19 +0000 (09:01 -0800)]
Merge pull request #25822 from batrick/i36209

mimic: mds: runs out of file descriptors after several respawns

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25852 from ashishkumsingh/wip-37832-mimic
Yuri Weinstein [Wed, 30 Jan 2019 17:00:26 +0000 (09:00 -0800)]
Merge pull request #25852 from ashishkumsingh/wip-37832-mimic

mimic: objecter: avoid race when reset down osd's session

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #26128 from xiexingguo/wip-mimic-upmap-fixes
Yuri Weinstein [Wed, 30 Jan 2019 16:57:45 +0000 (08:57 -0800)]
Merge pull request #26128 from xiexingguo/wip-mimic-upmap-fixes

mimic: osd: backport recent upmap fixes

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #26147 from ashishkumsingh/wip-38045-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:56:34 +0000 (08:56 -0800)]
Merge pull request #26147 from ashishkumsingh/wip-38045-mimic

mimic: tests: qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #25413 from tombarron/mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:52:27 +0000 (08:52 -0800)]
Merge pull request #25413 from tombarron/mimic

mimic: ceph-volume-client: allow setting mode of CephFS volumes

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25515 from smithfarm/wip-36578-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:52:01 +0000 (08:52 -0800)]
Merge pull request #25515 from smithfarm/wip-36578-mimic

mimic: qa: teuthology may hang on diagnostic commands for fuse mount

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25517 from smithfarm/wip-37424-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:51:35 +0000 (08:51 -0800)]
Merge pull request #25517 from smithfarm/wip-37424-mimic

mimic: qa: wrong setting for msgr failures

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25519 from smithfarm/wip-37603-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:51:07 +0000 (08:51 -0800)]
Merge pull request #25519 from smithfarm/wip-37603-mimic

mimic: mds: severe internal fragment when decoding xattr_map from log event

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25521 from smithfarm/wip-37607-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:50:08 +0000 (08:50 -0800)]
Merge pull request #25521 from smithfarm/wip-37607-mimic

mimic: mds: directories pinned keep being replicated back and forth between exporting mds and importing mds

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25523 from smithfarm/wip-37605-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:49:38 +0000 (08:49 -0800)]
Merge pull request #25523 from smithfarm/wip-37605-mimic

mimic: mds: PurgeQueue write error handler does not handle EBLACKLISTED

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25557 from smithfarm/wip-37611-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:49:09 +0000 (08:49 -0800)]
Merge pull request #25557 from smithfarm/wip-37611-mimic

mimic: qa: pjd test appears to require more than 3h timeout for some configurations

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25559 from smithfarm/wip-37626-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:48:41 +0000 (08:48 -0800)]
Merge pull request #25559 from smithfarm/wip-37626-mimic

mimic: mds: fix incorrect l_pq_executing_ops statistics when meet an invalid item in purge queue

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25561 from smithfarm/wip-37628-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:47:23 +0000 (08:47 -0800)]
Merge pull request #25561 from smithfarm/wip-37628-mimic

mimic: mds: do not call Journaler::_trim twice

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25642 from vshankar/wip-37609
Yuri Weinstein [Wed, 30 Jan 2019 16:46:48 +0000 (08:46 -0800)]
Merge pull request #25642 from vshankar/wip-37609

mimic: mds: disallow dumping huge caches to formatter

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25676 from joscollin/wip-37699-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:46:18 +0000 (08:46 -0800)]
Merge pull request #25676 from joscollin/wip-37699-mimic

mimic: mds: fix bug filelock stuck at LOCK_XSYN leading client can't read data

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25678 from joscollin/wip-37695-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:45:35 +0000 (08:45 -0800)]
Merge pull request #25678 from joscollin/wip-37695-mimic

mimic: client: fix failure in quota size limitation when using samba

Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #25680 from joscollin/wip-37634-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:45:09 +0000 (08:45 -0800)]
Merge pull request #25680 from joscollin/wip-37634-mimic

mimic: mds: fix races of updating wanted caps

Reviewed-by:  Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25681 from joscollin/wip-37632-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:43:22 +0000 (08:43 -0800)]
Merge pull request #25681 from joscollin/wip-37632-mimic

mimic: mds: remove duplicated l_mdc_num_strays perfcounter set

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #25683 from joscollin/wip-37630-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:41:52 +0000 (08:41 -0800)]
Merge pull request #25683 from joscollin/wip-37630-mimic

mimic: client: do not move f->pos untill success write

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #25685 from joscollin/wip-37738-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:41:15 +0000 (08:41 -0800)]
Merge pull request #25685 from joscollin/wip-37738-mimic

mimic: MDSMonitor: allow beacons from stopping MDS that was laggy

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #25803 from ashishkumsingh/wip-37757-mimic
Yuri Weinstein [Wed, 30 Jan 2019 16:40:24 +0000 (08:40 -0800)]
Merge pull request #25803 from ashishkumsingh/wip-37757-mimic

mimic: mds: clean up log messages for standby-replay

Reviewed-by: Venky Shankar <vshankar@redhat.com>