]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agomgr/influx: Only split string on first occurence of dot (.) 21965/head
Wido den Hollander [Thu, 3 May 2018 10:11:58 +0000 (12:11 +0200)]
mgr/influx: Only split string on first occurence of dot (.)

Service names are not always osd.X or mon.X, they might be
rgw.radosgw.rgw1

This would lead to:

  Unhandled exception from module 'influx' while running on mgr.mon01: too many values to unpack

Only split on the first dot as the rest is the service name

Fixes: http://tracker.ceph.com/issues/23996
Signed-off-by: Wido den Hollander <wido@42on.com>
(cherry picked from commit bd7d0fc16a90b3fefadc2d6904180a7fd04ef5eb)

7 years agoMerge pull request #21936 from ceph/wip-yuriw-24021-luminous
Jason Dillaman [Fri, 11 May 2018 16:58:15 +0000 (12:58 -0400)]
Merge pull request #21936 from ceph/wip-yuriw-24021-luminous

qa/test: added client-upgrade-luminous suite

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21871 from tchaikov/wip-luminous-18649
Yuri Weinstein [Fri, 11 May 2018 16:28:42 +0000 (09:28 -0700)]
Merge pull request #21871 from tchaikov/wip-luminous-18649

luminous: mgr/restful: A couple of restful fixes

Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoMerge pull request #21911 from dzafman/backport_23850-zafman
Yuri Weinstein [Fri, 11 May 2018 16:28:02 +0000 (09:28 -0700)]
Merge pull request #21911 from dzafman/backport_23850-zafman

osd/ECBackend: only check required shards when finishing recovery reads

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
7 years agoqa/test: added client-upgrade-luminous suite 21936/head
Yuri Weinstein [Fri, 4 May 2018 20:37:55 +0000 (13:37 -0700)]
qa/test: added client-upgrade-luminous suite
(whitelisted CACHE_POOL_NO_HIT_SET)

Fixes https://tracker.ceph.com/issues/24021

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #21725 from smithfarm/wip-23631-luminous
Yuri Weinstein [Thu, 10 May 2018 15:47:11 +0000 (08:47 -0700)]
Merge pull request #21725 from smithfarm/wip-23631-luminous

luminous: rbd: python bindings fixes and improvements

Reviewed-by: Ricardo Dias <rdias@suse.com>
7 years agoMerge pull request #21726 from smithfarm/wip-23900-luminous
Yuri Weinstein [Thu, 10 May 2018 15:45:42 +0000 (08:45 -0700)]
Merge pull request #21726 from smithfarm/wip-23900-luminous

luminous: [rbd-mirror] asok hook for image replayer not re-registered after bootstrap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21870 from tchaikov/wip-luminous-24041
Yuri Weinstein [Thu, 10 May 2018 15:43:59 +0000 (08:43 -0700)]
Merge pull request #21870 from tchaikov/wip-luminous-24041

luminous: ceph-disk: write log to /var/log/ceph not to /var/run/ceph

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #21919 from smithfarm/wip-24070-luminous
Nathan Cutler [Thu, 10 May 2018 05:16:30 +0000 (07:16 +0200)]
Merge pull request #21919 from smithfarm/wip-24070-luminous

luminous: build-integration-branch script

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoqa/suites/rados: add coverage for osd_recovery_max_single_start > 1 21911/head
Josh Durgin [Fri, 6 Apr 2018 07:15:57 +0000 (03:15 -0400)]
qa/suites/rados: add coverage for osd_recovery_max_single_start > 1

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 234d652317baefb28fd498f476f1db573974db4c)

7 years agoosd/ECBackend: preserve requests for other objects when sending extra reads
Josh Durgin [Fri, 20 Apr 2018 22:42:56 +0000 (18:42 -0400)]
osd/ECBackend: preserve requests for other objects when sending extra reads

When multiple objects are in flight for the same ReadOp, swap() on the
map<hobject_t, read_request_t> would remove requests for all objects.

We just want to replace the requests for the single object we're
dealing with in send_all_remaining_reads().

This prevents crashing trying to look up rop.to_read[hoid] when another
object in the same ReadOp gets an EIO and tries to send more requests.

Test this by using osd-recovery-max-single-start to bundle multiple
reads into one ReadOp. Save and restore CEPH_ARGS so custom settings
are reset for each test.

Fixes: http://tracker.ceph.com/issues/23195 (the 2nd crash there)
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit d4808256d25851f85ce5e86704cc4e867e2fc905)

7 years agoosd/ECBackend: recover from EIO based on the minimum data necessary
Josh Durgin [Fri, 20 Apr 2018 22:39:40 +0000 (18:39 -0400)]
osd/ECBackend: recover from EIO based on the minimum data necessary

Discount shards that already returned EIO, and use minimum_to_decode()
to request just what is necessary to recover or read the originally
requested extents of the object.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit b162a5478d6a907cc0a9ddd5ae8442e81f8d8fb3)

Conflicts:
src/osd/ECBackend.cc (Adjust for Luminous not having subchunks)
src/osd/ECBackend.h (trivial)

7 years agoosd/ECBackend: only check required shards when finishing recovery reads
Josh Durgin [Fri, 6 Apr 2018 06:43:13 +0000 (02:43 -0400)]
osd/ECBackend: only check required shards when finishing recovery reads

1235810c2ad08ccb7ef5946686eb2b85798f5bca allowed recovery to use
multiple passes of reads to handle EIO, but the end condition for
checking whether we finished reading requires the full data to be
decodable (this is what get_want_to_read_shards returns).

This is just a loss of efficiency normally, since when there is only
one object the subsequent read works, and grabs all the data
necessary. The crash comes from having multiple objects in the same
ReadOp - in this case the sequence of events is:

- start recovery of two objects (osd_recovery_max_single_start > 1)
- read object a shard 3
- read object b shard 3
- fail minimum_to_decode because shard 3 can't reconstruct all of object a
- re-read all of object a, marking more reads in progress
- fail minimum_to_decode because shard 3 can't reconstruct all of object b
- skip re-reading object because there are now reads in progress
- finish reading k shards of object a
- still fail minimum_to_decode for object b, so no extra data was read
- send_all_remaining_reads tries to lookup object b in ReadOp object
- crash dereferencing to_read[object b], since this was cleared after handling the original object b read reply

This patch fixes the immediate inefficiency and crash by only checking
for the missing shards that were requested, rather than the entire
object, for recovery reads.

Fixes: http://tracker.ceph.com/issues/23195 (first crash)
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 468ad4b41010488c8d48ef65ccbebfdb4270690f)

Conflicts:
src/osd/ECBackend.cc (trivial)

7 years agoosd/ECBackend: store the original needed shards in a ReadOp
Josh Durgin [Tue, 10 Apr 2018 01:47:46 +0000 (21:47 -0400)]
osd/ECBackend: store the original needed shards in a ReadOp

This avoids extra logic recomputing them due to EIO or redundant
reads.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 0f9474d4e8e54cb8784cc2dbaa50318c2ff8e667)

Conflicts:
src/osd/ECBackend.cc (trivial)

7 years agoqa/standalone: s/delete_erasure_pool/delete_erasure_coded_pool/
Kefu Chai [Thu, 1 Mar 2018 11:09:15 +0000 (19:09 +0800)]
qa/standalone: s/delete_erasure_pool/delete_erasure_coded_pool/

it's a regression introduced by ac56a202

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fc43ae1724f6cdf3a00e1995281e1e4d414093da)

Brought into Luminous to make backport clean

7 years agoqa/standalone: extract delete_pool()
Kefu Chai [Wed, 28 Feb 2018 07:40:25 +0000 (15:40 +0800)]
qa/standalone: extract delete_pool()

some tests, like osd-backfill-stats.sh are using delete_pool(), but
they don't have this function defined. and this function is defined
in standalone tests separately, so would be simpler if we can
consolidate them in ceph-helper.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ac56a202fd57d6a18d3852d26a4e5cf4a214bf8a)

Brought into Luminous to make backport clean

7 years agoscript: build-integration-branch: avoid Unicode error 21919/head
Nathan Cutler [Thu, 3 May 2018 20:42:04 +0000 (22:42 +0200)]
script: build-integration-branch: avoid Unicode error

When run on PRs that have non-ASCII characters in their titles, the script
fails like this when run in a non UTF-8 environment:

UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2026' in
position 651: ordinal not in range(256)

Since failing like that is not useful, avoid the error by forcing ASCII
encoding and filtering out any non-ASCII characters.

Fixes: http://tracker.ceph.com/issues/24003
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 6dd31fdbec43e37bece24c0de886d2c4083901c8)

7 years agoscript/build-integration-branch: python3 compatible and pep8 clean
Kefu Chai [Fri, 29 Sep 2017 04:03:15 +0000 (12:03 +0800)]
script/build-integration-branch: python3 compatible and pep8 clean

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4e8e3116949d11f62ac027e300c09dc5f929cfc5)

7 years agoscript/build-integration-branch: check errors
Sage Weil [Thu, 7 Sep 2017 17:55:13 +0000 (13:55 -0400)]
script/build-integration-branch: check errors

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 686f2ae7fa8050d6113674ea5570ad956792e49a)

7 years agoscript/build-integration-branch: print pr url list with titles
Sage Weil [Fri, 1 Sep 2017 16:26:39 +0000 (12:26 -0400)]
script/build-integration-branch: print pr url list with titles

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5916683c52fe6c1d54e91415ba0c4af790d48485)

7 years agosrc/script/build-integration-branch
Sage Weil [Wed, 30 Aug 2017 20:42:19 +0000 (16:42 -0400)]
src/script/build-integration-branch

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b5c27826158f699960fcf887e901545c359ad68c)

7 years agoMerge pull request #21109 from pdvian/wip-23474-luminous
Yuri Weinstein [Wed, 9 May 2018 14:55:14 +0000 (07:55 -0700)]
Merge pull request #21109 from pdvian/wip-23474-luminous

luminous: client: add ceph_ll_sync_inode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
7 years agoMerge pull request #21346 from pdvian/wip-23632-luminous
Yuri Weinstein [Wed, 9 May 2018 14:54:30 +0000 (07:54 -0700)]
Merge pull request #21346 from pdvian/wip-23632-luminous

luminous: mds: mds shutdown fixes and optimization

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21354 from pdvian/wip-23635-luminous
Yuri Weinstein [Wed, 9 May 2018 14:53:56 +0000 (07:53 -0700)]
Merge pull request #21354 from pdvian/wip-23635-luminous

luminous: client: fix request send_to_auth was never really used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21638 from smithfarm/wip-23852-luminous
Yuri Weinstein [Tue, 8 May 2018 23:04:53 +0000 (16:04 -0700)]
Merge pull request #21638 from smithfarm/wip-23852-luminous

luminous: osd: do not crash on empty snapset

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21359 from xiexingguo/backport-pr-21315
Yuri Weinstein [Tue, 8 May 2018 23:04:09 +0000 (16:04 -0700)]
Merge pull request #21359 from xiexingguo/backport-pr-21315

luminous: crush/CrushWrapper: fix potential invalid use of iterator

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #21366 from pdvian/wip-23636-luminous
Yuri Weinstein [Tue, 8 May 2018 19:47:12 +0000 (12:47 -0700)]
Merge pull request #21366 from pdvian/wip-23636-luminous

luminous: mds: fix unhealth heartbeat during rejoin

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21372 from pdvian/wip-23637-luminous
Yuri Weinstein [Tue, 8 May 2018 19:46:47 +0000 (12:46 -0700)]
Merge pull request #21372 from pdvian/wip-23637-luminous

luminous: mds: mark damaged if sessions' preallocated inos don't match inotable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21441 from pdvian/wip-23702-luminous
Yuri Weinstein [Tue, 8 May 2018 19:46:24 +0000 (12:46 -0700)]
Merge pull request #21441 from pdvian/wip-23702-luminous

luminous: mds: properly dirty sessions opened by journal replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21447 from pdvian/wip-23703-luminous
Yuri Weinstein [Tue, 8 May 2018 19:45:58 +0000 (12:45 -0700)]
Merge pull request #21447 from pdvian/wip-23703-luminous

luminous: MDSMonitor: fix mds health printed in bad format

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21448 from pdvian/wip-23750-luminous
Yuri Weinstein [Tue, 8 May 2018 19:45:18 +0000 (12:45 -0700)]
Merge pull request #21448 from pdvian/wip-23750-luminous

luminous: mds: make rstat.rctime follow inodes' ctime

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21512 from pdvian/wip-23791-luminous
Yuri Weinstein [Tue, 8 May 2018 19:44:20 +0000 (12:44 -0700)]
Merge pull request #21512 from pdvian/wip-23791-luminous

luminous: MDSMonitor: initialize new Filesystem epoch from pending

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21542 from pdvian/wip-23802-luminous
Yuri Weinstein [Tue, 8 May 2018 19:43:54 +0000 (12:43 -0700)]
Merge pull request #21542 from pdvian/wip-23802-luminous

luminous:client: flush the mdlog in _fsync before waiting on unstable reqs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21596 from joscollin/wip-23771-luminous
Yuri Weinstein [Tue, 8 May 2018 19:43:26 +0000 (12:43 -0700)]
Merge pull request #21596 from joscollin/wip-23771-luminous

luminous: client: Fix the gid_count check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21732 from batrick/i23792
Yuri Weinstein [Tue, 8 May 2018 19:42:51 +0000 (12:42 -0700)]
Merge pull request #21732 from batrick/i23792

luminous: MDSMonitor: cleanup and protect fsmap access

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21796 from dalgaaf/luminous-issue-23873
Yuri Weinstein [Tue, 8 May 2018 19:42:07 +0000 (12:42 -0700)]
Merge pull request #21796 from dalgaaf/luminous-issue-23873

luminous: client: use common interp of st_nlink for dirs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21840 from pdvian/wip-23930-luminous
Yuri Weinstein [Tue, 8 May 2018 19:41:28 +0000 (12:41 -0700)]
Merge pull request #21840 from pdvian/wip-23930-luminous

luminous: mds: properly trim log segments after scrub repairs something

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #21844 from pdvian/wip-23934-luminous
Yuri Weinstein [Tue, 8 May 2018 19:41:03 +0000 (12:41 -0700)]
Merge pull request #21844 from pdvian/wip-23934-luminous

luminous: mds: properly check auth subtree count in MDCache::shutdown_pass()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoqa/restful: Test pg_num/pgp_num modifications 21871/head
Boris Ranto [Mon, 27 Nov 2017 13:35:26 +0000 (14:35 +0100)]
qa/restful: Test pg_num/pgp_num modifications

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 059588e758d4dad429012b8c452926781d6e5fea)

7 years agorestful: Support auid pool argument
Boris Ranto [Tue, 31 Oct 2017 11:35:01 +0000 (12:35 +0100)]
restful: Support auid pool argument

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 880c53c681e2b4fbf22160fcf34d7de527832d2a)

Conflicts:
src/pybind/mgr/restful/common.py: trivial resolution

7 years agorestful: Set the value of the argument
Boris Ranto [Tue, 31 Oct 2017 11:31:10 +0000 (12:31 +0100)]
restful: Set the value of the argument

Previously, we forgot to set the value of the argument for the second
group of pool variables.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 64ac3af77e0e6c81b842f72678829a15e1a60087)

7 years agorestful: Fix jsonification
Boris Ranto [Mon, 30 Oct 2017 14:43:38 +0000 (15:43 +0100)]
restful: Fix jsonification

The CommandsRequest.waiting is an array of arrays, not an array of
command results and so it cannot be represented that way. This commit
fixes that by outputting the array in json diretly.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 7005c6c0f68d82776c11cdfdd1a56e06f27d5e66)

7 years agoceph-disk: write log to /var/log/ceph not to /var/run/ceph 21870/head
Kefu Chai [Wed, 18 Oct 2017 10:03:31 +0000 (18:03 +0800)]
ceph-disk: write log to /var/log/ceph not to /var/run/ceph

Fixes: http://tracker.ceph.com/issues/24041
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e0856bd7e78828ea352290bd325aa4f5e06c3029)

7 years agomds: properly check auth subtree count in MDCache::shutdown_pass() 21844/head
Yan, Zheng [Mon, 23 Apr 2018 10:36:37 +0000 (18:36 +0800)]
mds: properly check auth subtree count in MDCache::shutdown_pass()

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/23813
(cherry picked from commit cb60472bd2803878c04dbb961fd7f7f54707f25b)

7 years agomds: properly trim log segments after scrub repairs something 21840/head
Yan, Zheng [Thu, 26 Apr 2018 07:50:39 +0000 (15:50 +0800)]
mds: properly trim log segments after scrub repairs something

Before calling MDLog::trim_all(), we need to make sure mdlog get flushed
Otherwise, there can be zero expiring log segments. This can cause
MDSGatherBuilder to malfunction. (MDSGatherBuilder::activate() does not
execute empty MDSGatherBuilder's finish context)

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
http://tracker.ceph.com/issues/23880
(cherry picked from commit 5be40f35c3365265b76a12a3bd0e17c3f8761bb9)

7 years agoMerge pull request #21740 from pdvian/wip-23881-luminous
Yuri Weinstein [Fri, 4 May 2018 17:17:05 +0000 (10:17 -0700)]
Merge pull request #21740 from pdvian/wip-23881-luminous

luminous: os/bluestore/bluefs_types: make block_mask 64-bit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21752 from pdvian/wip-23904-luminous
Yuri Weinstein [Fri, 4 May 2018 17:16:24 +0000 (10:16 -0700)]
Merge pull request #21752 from pdvian/wip-23904-luminous

luminous: osdc/Objecter: prevent double-invocation of linger op callback

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21778 from pdvian/wip-23924-luminous
Yuri Weinstein [Fri, 4 May 2018 17:15:41 +0000 (10:15 -0700)]
Merge pull request #21778 from pdvian/wip-23924-luminous

luminous: ceph_test_rados_api_aio: fix race with full pool and osdmap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21737 from pdvian/wip-23870-luminous
Yuri Weinstein [Fri, 4 May 2018 17:14:47 +0000 (10:14 -0700)]
Merge pull request #21737 from pdvian/wip-23870-luminous

luminous: osd: publish osdmap to OSDService before starting wq threads

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21518 from smithfarm/wip-23784-luminous
Yuri Weinstein [Fri, 4 May 2018 17:12:32 +0000 (10:12 -0700)]
Merge pull request #21518 from smithfarm/wip-23784-luminous

luminous: osd: Warn about objects with too many omap entries

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
7 years agoMerge pull request #21655 from liewegas/wip-23786
Yuri Weinstein [Fri, 4 May 2018 17:11:10 +0000 (10:11 -0700)]
Merge pull request #21655 from liewegas/wip-23786

luminous: qa: disable -Werror when compiling env_librados_test

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21760 from badone/wip-summarise-osds-when-issuing-slow-request...
Yuri Weinstein [Fri, 4 May 2018 16:54:20 +0000 (09:54 -0700)]
Merge pull request #21760 from badone/wip-summarise-osds-when-issuing-slow-request-warning-luminous

luminous: mon/PGMap: Summarise OSDs in blocked/stuck requests

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agomon/PGMap: Summarise OSDs in blocked/stuck requests 21760/head
Brad Hubbard [Tue, 3 Apr 2018 07:49:08 +0000 (17:49 +1000)]
mon/PGMap: Summarise OSDs in blocked/stuck requests

This fix is luminous specific since this code was changed in Mimic to
use the new SLOW_OPS errors.

Fixes: https://tracker.ceph.com/issues/23205
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
7 years agoclient: use common interp of st_nlink for dirs 21796/head
Patrick Donnelly [Wed, 25 Apr 2018 20:07:34 +0000 (13:07 -0700)]
client: use common interp of st_nlink for dirs

Apparently some applications use this (like mail servers) and since it's
trivial to support, let's do it. Idea is that st_nlinks for a directory is
either 0 (it is unlinked) or 2 + the number of sub-directories (which have ..
parent links).

Fixes: https://tracker.ceph.com/issues/23873
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 67c7e4619188cfd684e59e3cb538948ae8f2b74c)

Conflicts:
        src/client/Inode.cc

7 years agoqa/workunits/rados/test_large_omap_detection: Scrub pgs instead of OSDs 21518/head
Brad Hubbard [Fri, 13 Apr 2018 01:10:22 +0000 (11:10 +1000)]
qa/workunits/rados/test_large_omap_detection: Scrub pgs instead of OSDs

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit aba1e74e03c753366009996dedcb59ef83e155de)

7 years agoosd: Add a flag to ScrubMap to signal check needed
Brad Hubbard [Fri, 13 Oct 2017 05:46:47 +0000 (15:46 +1000)]
osd: Add a flag to ScrubMap to signal check needed

If we know none of the scrub maps have errors early in
be_large_omap_check() we can return without doing a lot of unnecessary
work.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 2094fda7655ddd82b56850b3ded961a8c5c66966)

7 years agoosd: Warn about objects with too many omap entries
Brad Hubbard [Tue, 27 Jun 2017 01:40:47 +0000 (11:40 +1000)]
osd: Warn about objects with too many omap entries

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 71bf04775bef90c9291bd825da626bc9de6f9ec1)

Conflicts:
src/osd/osd_types.h (trivial resolution)

7 years agoosd: Move creation of 'master_set' to scrub_compare_maps
Brad Hubbard [Mon, 9 Oct 2017 07:24:17 +0000 (17:24 +1000)]
osd: Move creation of 'master_set' to scrub_compare_maps

Moving it to scrub_compare_maps allows us to pass it to multiple
subsequently called functions.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 63ef97b3db81c0c725119943c8594a4a0d12154e)

7 years agoMerge pull request #21717 from croit/luminous-23915
Yuri Weinstein [Thu, 3 May 2018 13:50:51 +0000 (06:50 -0700)]
Merge pull request #21717 from croit/luminous-23915

luminous: mon: handle bad snapshot removal reqs gracefully

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21568 from cbodley/wip-22937
Yuri Weinstein [Thu, 3 May 2018 13:49:48 +0000 (06:49 -0700)]
Merge pull request #21568 from cbodley/wip-22937

luminous: rgw: beast frontend can listen on multiple endpoints

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
7 years agoMerge pull request #21083 from pdvian/wip-23227-luminous
Yuri Weinstein [Thu, 3 May 2018 13:43:14 +0000 (06:43 -0700)]
Merge pull request #21083 from pdvian/wip-23227-luminous

luminous: common: fix BoundedKeyCounter const_pointer_iterator

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoceph_test_rados_api_aio: fix race with full pool and osdmap 21778/head
Sage Weil [Fri, 27 Apr 2018 22:32:38 +0000 (17:32 -0500)]
ceph_test_rados_api_aio: fix race with full pool and osdmap

We send ops until we get a EDQUOT, and then assert our next op to
a different object also gets EDQUOT.  However, if the second osd
doesn't have as new a map it may succeed.  Make sure the client has
the latest (mon) map, and thus the one marking the pool full, before
we send the second op.  That ensures the second OSD also has that
newer map and also returns EDQUOT.

Fixes: http://tracker.ceph.com/issues/23917
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5e067b9dcf9bcccd9a1c0153d94818dccf6a0f71)

7 years agocmake: 'make check' builds radosgw and its cls dependencies 21568/head
Casey Bodley [Tue, 13 Feb 2018 15:32:11 +0000 (10:32 -0500)]
cmake: 'make check' builds radosgw and its cls dependencies

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a029d20b86c7d64626baf14bf0217c142e5e17cf)

7 years agorgw: beast frontend can listen on multiple endpoints
Casey Bodley [Tue, 23 Jan 2018 21:03:13 +0000 (16:03 -0500)]
rgw: beast frontend can listen on multiple endpoints

rgw_frontends options port= and endpoint= can be given multiple times,
and the beast frontend will listen on each endpoint

Fixes: http://tracker.ceph.com/issues/22779
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1bff073b0e3e19e341fc0a08918abb510f0711b4)

Conflicts: rgw_asio_frontend.cc (boost::asio::string_view not available,
                                 replaced with BOOST_ASIO_STRING_VIEW_PARAM
                                 which falls back to 'const std::string&')

7 years agorgw: civetweb accepts multiple port= entries
Casey Bodley [Tue, 23 Jan 2018 18:37:05 +0000 (13:37 -0500)]
rgw: civetweb accepts multiple port= entries

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 22df0fbffdfa8b1963092cbf67e1ca8e18f01595)

7 years agoosdc/Objecter: prevent double-invocation of linger op callback 21752/head
Jason Dillaman [Wed, 25 Apr 2018 17:23:00 +0000 (13:23 -0400)]
osdc/Objecter: prevent double-invocation of linger op callback

Fixes: http://tracker.ceph.com/issues/23872
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1bc6f31f8f5ce33176b8a346492b087728947a71)

7 years agoMDSMonitor: cleanup and protect fsmap access 21732/head
Patrick Donnelly [Tue, 17 Apr 2018 03:46:39 +0000 (20:46 -0700)]
MDSMonitor: cleanup and protect fsmap access

This commit is designed to permanently protect the current epoch FSMap from
accidental changes. Modifications to the pending_fsmap are protected by
checking if the monitor is the leader. Additionally, code must explicitly
request a writeable reference.

Fixes http://tracker.ceph.com/issues/23762

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 624efc64323f99b2e843f376879c1080276e036f)

Conflicts:
src/mon/MDSMonitor.cc
src/mon/MDSMonitor.h

7 years agoMerge pull request #21746 from alfredodeza/backport-rm23874
Andrew Schoen [Mon, 30 Apr 2018 19:52:50 +0000 (19:52 +0000)]
Merge pull request #21746 from alfredodeza/backport-rm23874

luminous  ceph-volume failed ceph-osd --mkfs command doesn't halt the OSD creation process

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume tests mkfs bluestore fails on non-zero exit status 21746/head
Alfredo Deza [Thu, 26 Apr 2018 20:35:45 +0000 (16:35 -0400)]
ceph-volume tests mkfs bluestore fails on non-zero exit status

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 978552a51f000952e0c820422369522063fd581a)

7 years agoceph-volume tests allow setting return values on stubbed process.call
Alfredo Deza [Thu, 26 Apr 2018 20:25:46 +0000 (16:25 -0400)]
ceph-volume tests allow setting return values on stubbed process.call

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b57e90105721d2817908b60ec39c2b4cbb1ca8f3)

7 years agoceph-volume util.prepare raise on non-zero exit statuso
Alfredo Deza [Thu, 26 Apr 2018 20:26:30 +0000 (16:26 -0400)]
ceph-volume util.prepare raise on non-zero exit statuso

Just for bluestore objectstore that uses process.call

Signed-off-by: Alfredo Deza <adeza@redhat.com>
7 years agoos/bluestore/bluefs_types: make block_mask 64-bit 21740/head
Sage Weil [Tue, 24 Apr 2018 22:11:15 +0000 (17:11 -0500)]
os/bluestore/bluefs_types: make block_mask 64-bit

Since block_size is 32 bits, we return 0x00000000fffff000 (or similar).
This breaks things if you get a file (e.g., bluefs metadata log file)
over 4GB.

Fixes: https://tracker.ceph.com/issues/23840
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 55dd28a9dee08a028c6a8db82a8be77b3af88e44)

7 years agoosd: publish osdmap to OSDService before starting wq threads 21737/head
Sage Weil [Tue, 24 Apr 2018 18:02:14 +0000 (13:02 -0500)]
osd: publish osdmap to OSDService before starting wq threads

The worker threads may request the map from the service (notably,
advance_pg() does this).

Fixes: http://tracker.ceph.com/issues/21977
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ef3e5b1069c81f4e5b64e6119b467aeb7193e0b7)

Conflicts:
src/osd/OSD.cc: Resolved in OSD::init

7 years agomon: handle bad snapshot removal reqs gracefully 21717/head
Paul Emmerich [Sun, 11 Mar 2018 00:26:45 +0000 (01:26 +0100)]
mon: handle bad snapshot removal reqs gracefully

Snapshot deletion requests on snap ids larger than the snap_seq of
the pool will leave the pool in a state with snap_seq being less
than max(removed_snaps).

This is bad because further deletion requests to a pool in this state
might crash the mon in some cases: the deletion also inserts the new
snap_seq into the removed_snaps set -- which might already exist
in this case and trigger an assert.

Such bad requests will be generated by rbd clients without a fix for
issue #21567.

The change in OSDMonitor prevents pools from getting into this state
and may prevent old broken clients from incorrectly deleting snaps.
The change in osd_types avoids a crash if a pool is already in this
state.

Fixes #18746

Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit f42a6ba4c37cb10869718ec296352ae55254b44a)

 Conflicts:
src/osd/osd_types.cc

7 years agorbd-mirror: rename asok hook to match image name when not replaying 21726/head
Jason Dillaman [Thu, 26 Apr 2018 16:24:27 +0000 (12:24 -0400)]
rbd-mirror: rename asok hook to match image name when not replaying

Fixes: http://tracker.ceph.com/issues/23888
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3ec26ff84d3b622a95142587a29b511e48c4bc85)

Conflicts: (trivial resolution)
src/test/rbd_mirror/test_mock_ImageReplayer.cc
src/tools/rbd_mirror/ImageReplayer.cc

7 years agopybind/rbd: always parse timestamps in UTC 21725/head
Ricardo Dias [Tue, 27 Mar 2018 14:05:01 +0000 (15:05 +0100)]
pybind/rbd: always parse timestamps in UTC

Fixes: http://tracker.ceph.com/issues/23609
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 08cee16d0a4b95a53aa31cbced2306e0d8c3a627)

7 years agopybind/rbd: add `data_pool_id()` image function
Ricardo Dias [Tue, 27 Mar 2018 12:41:26 +0000 (13:41 +0100)]
pybind/rbd: add `data_pool_id()` image function

Fixes: http://tracker.ceph.com/issues/23609
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit e127ec64b4fe900ec044521b130f7b83afd2161b)

7 years agopybind/rbd: fix python 3 str to bytes problems
Ricardo Dias [Fri, 23 Mar 2018 14:51:44 +0000 (14:51 +0000)]
pybind/rbd: fix python 3 str to bytes problems

Fixes: http://tracker.ceph.com/issues/23609
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 72dc17ce327dc6422a2d62348b2c42971d364b6b)

Conflicts:
src/pybind/rbd/rbd.pyx (deep_copy not in luminous)

7 years agorgw: frontend config uses multimap
Casey Bodley [Tue, 23 Jan 2018 18:21:11 +0000 (13:21 -0500)]
rgw: frontend config uses multimap

this allows us to configure multiple values for a given key without
resorting to string formatting to cram them into a single key/value pair

ex. port=80 port=443s instead of port=80+443s

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cf1b84195ebb8b0abf733889a3fa994a341be1b1)

Conflicts:
        rgw_frontend.cc: missing overload of get_str_vec()

7 years agoMerge pull request #21654 from alfredodeza/backport-rm23451
Andrew Schoen [Thu, 26 Apr 2018 15:13:29 +0000 (15:13 +0000)]
Merge pull request #21654 from alfredodeza/backport-rm23451

luminous ceph-volume include physical devices associated with an LV when listing

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoqa: disable -Werror when compiling env_librados_test 21655/head
Kefu Chai [Sun, 15 Apr 2018 14:38:08 +0000 (22:38 +0800)]
qa: disable -Werror when compiling env_librados_test

to silence warnings like

utilities/env_librados.cc:175:33: warning: unused parameter ‘offset’ [-Wunused-parameter]
   Status InvalidateCache(size_t offset, size_t length) {
                                 ^~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1a95710441baa1f14a57129cb6fe18559df63766)

7 years agodoc/ceph-volume include information for devices in listing 21654/head
Alfredo Deza [Wed, 25 Apr 2018 14:48:45 +0000 (10:48 -0400)]
doc/ceph-volume include information for devices in listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 009d49a4f5cf8c97a156b710342d4786b734747b)

7 years agoceph-volume tests verify associated pvs reporting on listing
Alfredo Deza [Wed, 25 Apr 2018 13:50:24 +0000 (09:50 -0400)]
ceph-volume tests verify associated pvs reporting on listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f57cc594c282cf06a75b2278ca1c2b2d38f87aa3)

7 years agoceph-volume tests update to include devices on listing
Alfredo Deza [Tue, 24 Apr 2018 21:35:08 +0000 (17:35 -0400)]
ceph-volume tests update to include devices on listing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 08dea1b7d1d92d893c2f0a5e6e750d838d9cdbe4)

7 years agoceph-volume lvm.listing include associated physical devices
Alfredo Deza [Tue, 24 Apr 2018 21:16:50 +0000 (17:16 -0400)]
ceph-volume lvm.listing include associated physical devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit be1e6c2ae9a6e6a77fcb5297badb1aa58548180c)

7 years agoceph-volume api.lvm include lv_uuid on pvs listings
Alfredo Deza [Tue, 24 Apr 2018 21:11:42 +0000 (17:11 -0400)]
ceph-volume api.lvm include lv_uuid on pvs listings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 12a7a48c53b16f52a76ad74773aaa67f8bfc7c52)

7 years agoosd: don't crash on empty snapset 21638/head
Mykola Golub [Tue, 27 Mar 2018 07:39:47 +0000 (10:39 +0300)]
osd: don't crash on empty snapset

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 3996c0ae9c344de12eac2226e260c586ef48b09f)

Conflicts:
src/osd/PrimaryLogPG.cc - retain legacy snapshot handling in luminous -
                 i.e., do not backport 89c3439191325c2ee39ab4490985e77e5865e28a

7 years ago12.2.5 v12.2.5
Jenkins Build Slave User [Mon, 23 Apr 2018 16:18:33 +0000 (16:18 +0000)]
12.2.5

7 years agoclient: Fix the gid_count check 21596/head
Jos Collin [Wed, 11 Apr 2018 05:43:51 +0000 (11:13 +0530)]
client: Fix the gid_count check

Make the right check for gid_count.

Fixes: http://tracker.ceph.com/issues/23652
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 956d5501facc6a65903b0bad0d4b87c88139c787)

7 years agoMerge pull request #21565 from alfredodeza/backport-wip-ceph-volume
Andrew Schoen [Fri, 20 Apr 2018 18:23:04 +0000 (18:23 +0000)]
Merge pull request #21565 from alfredodeza/backport-wip-ceph-volume

luminous: ceph-volume: Nits noticed while studying code

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoMerge pull request #21574 from alfredodeza/backport-cv-doc-updates
Andrew Schoen [Fri, 20 Apr 2018 18:07:51 +0000 (18:07 +0000)]
Merge pull request #21574 from alfredodeza/backport-cv-doc-updates

luminous: ceph-volume update man page for prepare/activate flags

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agodoc/man/ceph-volume update flags for activate 21574/head
Alfredo Deza [Fri, 20 Apr 2018 16:22:40 +0000 (12:22 -0400)]
doc/man/ceph-volume update flags for activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a454abc863817cd393e01e8c135174c1e1839be2)

7 years agodoc/man/ceph-volume update flags for prepare
Alfredo Deza [Fri, 20 Apr 2018 16:46:48 +0000 (12:46 -0400)]
doc/man/ceph-volume update flags for prepare

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ae96b9f0a283a65ddf6887461f3515a5a756572b)

7 years agoMerge pull request #21492 from b-ranto/wip-mgr-rgw-luminous
Yuri Weinstein [Fri, 20 Apr 2018 17:01:22 +0000 (10:01 -0700)]
Merge pull request #21492 from b-ranto/wip-mgr-rgw-luminous

[luminous] Backport recent prometheus rgw changes

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
7 years agoRemove unused and wrongly-described _lsblk_type 21565/head
Dan Mick [Tue, 17 Apr 2018 02:13:18 +0000 (19:13 -0700)]
Remove unused and wrongly-described _lsblk_type

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit a97419f5d2b23131242a3602f2fa5f91ac7bc74d)

7 years agoFix mixing of format() and %-interpolation
Dan Mick [Tue, 17 Apr 2018 01:37:30 +0000 (18:37 -0700)]
Fix mixing of format() and %-interpolation

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 9e9a07c9ae41cc72e587dfd6551fd83389cc35e6)

7 years agoSpelling/typos
Dan Mick [Tue, 17 Apr 2018 01:35:33 +0000 (18:35 -0700)]
Spelling/typos

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 15ef400bb35e70cbb27144d595759d31a075aa37)

7 years agoMerge pull request #21538 from alfredodeza/backport-rm23678
Andrew Schoen [Fri, 20 Apr 2018 15:13:28 +0000 (15:13 +0000)]
Merge pull request #21538 from alfredodeza/backport-rm23678

luminous: ceph-volume: allow skipping systemd interactions on activate/create

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agomgr/dashboard: performance counter browsers 21492/head
Rubab-Syed [Tue, 9 Jan 2018 23:32:42 +0000 (04:32 +0500)]
mgr/dashboard: performance counter browsers

Signed-off-by: Rubab-Syed <rubab.syed21@gmail.com>
(cherry picked from commit 25ecfbd9cfb620463de1545ca0be50001c805460)
Conflicts: Removed dashboard changes

7 years agomgr: expose have_mon_connection to python modules
John Spray [Thu, 9 Nov 2017 11:27:24 +0000 (06:27 -0500)]
mgr: expose have_mon_connection to python modules

So that they can warn their end users if the data
we have is probably stale.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5633328e28c0f7a5122229737c2ca132174a4349)