]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoMerge pull request #23024 from smithfarm/wip-24865-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:20:17 +0000 (07:20 -0700)]
Merge pull request #23024 from smithfarm/wip-24865-mimic

mimic: common: Abort in OSDMap::decode() during qa/standalone/erasure-code/test-erasure-eio.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #23065 from pdvian/wip-24887-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:17:28 +0000 (07:17 -0700)]
Merge pull request #23065 from pdvian/wip-24887-mimic

mimic: os/bluestore: fix races on SharedBlob::coll in ~SharedBlob.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
7 years agoMerge pull request #23070 from smithfarm/wip-24942-mimic
Yuri Weinstein [Wed, 18 Jul 2018 14:16:41 +0000 (07:16 -0700)]
Merge pull request #23070 from smithfarm/wip-24942-mimic

mimic: build/ops: ceph.git has two different versions of dpdk in the source tree

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22812 from smithfarm/wip-24705-mimic
Yuri Weinstein [Tue, 17 Jul 2018 20:13:01 +0000 (13:13 -0700)]
Merge pull request #22812 from smithfarm/wip-24705-mimic

mimic: cephfs: allow prohibiting user snapshots in CephFS

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22835 from batrick/i24758
Yuri Weinstein [Tue, 17 Jul 2018 20:11:17 +0000 (13:11 -0700)]
Merge pull request #22835 from batrick/i24758

mimic: test gets ENOSPC from bluestore block device

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #22869 from pdvian/wip-24583-mimic
Yuri Weinstein [Tue, 17 Jul 2018 20:10:36 +0000 (13:10 -0700)]
Merge pull request #22869 from pdvian/wip-24583-mimic

mimic: osdc: Fix the wrong BufferHead offset

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22956 from joscollin/wip-24829-mimic
Yuri Weinstein [Tue, 17 Jul 2018 20:09:27 +0000 (13:09 -0700)]
Merge pull request #22956 from joscollin/wip-24829-mimic

mimic: qa: fix ffsb cd argument

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22981 from joscollin/wip-24861-mimic
Yuri Weinstein [Tue, 17 Jul 2018 20:08:57 +0000 (13:08 -0700)]
Merge pull request #22981 from joscollin/wip-24861-mimic

mimic: cephfs-journal-tool: Fix purging when importing an zero-length journal.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22933 from smithfarm/wip-24813-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:37:16 +0000 (09:37 -0700)]
Merge pull request #22933 from smithfarm/wip-24813-mimic

mimic: rgw: REST admin metadata API paging failure bucket & bucket.instance: InvalidArgument

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22466 from pdvian/wip-24313-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:36:44 +0000 (09:36 -0700)]
Merge pull request #22466 from pdvian/wip-24313-mimic

mimic: test/rgw: fix for bucket checkpoints

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22523 from smithfarm/wip-24253-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:36:10 +0000 (09:36 -0700)]
Merge pull request #22523 from smithfarm/wip-24253-mimic

mimic: rgw: Admin OPS Api overwrites email when user is modified

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #22590 from pdvian/wip-24352-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:35:33 +0000 (09:35 -0700)]
Merge pull request #22590 from pdvian/wip-24352-mimic

mimic: rgw: add configurable AWS-compat invalid range get behavior

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
7 years agoMerge pull request #22643 from pdvian/wip-24385-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:34:15 +0000 (09:34 -0700)]
Merge pull request #22643 from pdvian/wip-24385-mimic

mimic: objects in cache never refresh after rgw_cache_expiry_interval

Reviewed-by: Adam Emerson <aemerson@redhat.com>
7 years agoMerge pull request #22815 from smithfarm/wip-24700-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:33:30 +0000 (09:33 -0700)]
Merge pull request #22815 from smithfarm/wip-24700-mimic

mimic: rgw-multisite: endless loop in RGWBucketShardIncrementalSyncCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22818 from smithfarm/wip-24701-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:33:07 +0000 (09:33 -0700)]
Merge pull request #22818 from smithfarm/wip-24701-mimic

mimic: rgw: index complete miss zones_trace set

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22821 from smithfarm/wip-24702-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:32:39 +0000 (09:32 -0700)]
Merge pull request #22821 from smithfarm/wip-24702-mimic

mimic: rgw: meta and data notify thread miss stop cr manager

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22880 from smithfarm/wip-24783-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:32:08 +0000 (09:32 -0700)]
Merge pull request #22880 from smithfarm/wip-24783-mimic

mimic: rgw: set cr state if aio_read err return in RGWCloneMetaLogCoroutine::state_send_rest_request

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22926 from smithfarm/wip-24619-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:29:06 +0000 (09:29 -0700)]
Merge pull request #22926 from smithfarm/wip-24619-mimic

mimic: multisite: RGWSyncTraceNode released twice and crashed in reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22927 from smithfarm/wip-24631-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:26:15 +0000 (09:26 -0700)]
Merge pull request #22927 from smithfarm/wip-24631-mimic

mimic: cls_bucket_list fails causes cascading osd crashes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22929 from smithfarm/wip-24633-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:25:49 +0000 (09:25 -0700)]
Merge pull request #22929 from smithfarm/wip-24633-mimic

mimic: rgw performance regression for luminous 12.2.4

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22935 from smithfarm/wip-24809-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:25:05 +0000 (09:25 -0700)]
Merge pull request #22935 from smithfarm/wip-24809-mimic

mimic: rgw: Invalid Access-Control-Request-Request may bypass validate_cors_rule_method

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #22941 from joscollin/wip-24807-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:24:32 +0000 (09:24 -0700)]
Merge pull request #22941 from joscollin/wip-24807-mimic

mimic: rgw: fix gc may cause a large number of read traffic

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #23001 from pdvian/wip-24831-mimic
Yuri Weinstein [Tue, 17 Jul 2018 16:00:57 +0000 (09:00 -0700)]
Merge pull request #23001 from pdvian/wip-24831-mimic

mimic: rgw: "radosgw-admin objects expire" always returns ok even if the pro…

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #23021 from pdvian/wip-24834-mimic
Yuri Weinstein [Tue, 17 Jul 2018 15:59:20 +0000 (08:59 -0700)]
Merge pull request #23021 from pdvian/wip-24834-mimic

mimic: rgw: radogw-admin reshard status command should print text for reshar…

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
7 years agoMerge pull request #22222 from pdvian/wip-24203-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:23:03 +0000 (08:23 -0700)]
Merge pull request #22222 from pdvian/wip-24203-mimic

mimic: rbd: Prevent the use of internal feature bits from outside cls/rbd

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22662 from pdvian/wip-24388-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:22:31 +0000 (08:22 -0700)]
Merge pull request #22662 from pdvian/wip-24388-mimic

mimic: librbd: utilize the journal disabled policy when removing images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22667 from pdvian/wip-24390-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:21:56 +0000 (08:21 -0700)]
Merge pull request #22667 from pdvian/wip-24390-mimic

mimic: rbd: rbd-mirror daemon failed to stop on active/passive test case

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22754 from pdvian/wip-24496-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:21:26 +0000 (08:21 -0700)]
Merge pull request #22754 from pdvian/wip-24496-mimic

mimic: librbd: force 'invalid object map' flag on-disk update

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22843 from smithfarm/wip-24734-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:20:55 +0000 (08:20 -0700)]
Merge pull request #22843 from smithfarm/wip-24734-mimic

mimic: build/ops: order rbdmap.service before remote-fs-pre.target

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoMerge pull request #22891 from smithfarm/wip-24476-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:19:59 +0000 (08:19 -0700)]
Merge pull request #22891 from smithfarm/wip-24476-mimic

mimic: rbd: rbd trash purge --threshold should support data pool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22892 from smithfarm/wip-24519-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:19:29 +0000 (08:19 -0700)]
Merge pull request #22892 from smithfarm/wip-24519-mimic

mimic: rbd: rbd-mirror simple image map policy doesn't always level-load instances

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22894 from smithfarm/wip-24596-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:18:58 +0000 (08:18 -0700)]
Merge pull request #22894 from smithfarm/wip-24596-mimic

mimic: rbd: yet another case when deep copying a clone may result in invalid object map

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22943 from pdvian/wip-24698-mimic
Yuri Weinstein [Mon, 16 Jul 2018 15:17:52 +0000 (08:17 -0700)]
Merge pull request #22943 from pdvian/wip-24698-mimic

mimic: librbd: fix crash when opening nonexistent snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agodpdk: drop dpdk submodule 23070/head
Kefu Chai [Mon, 7 May 2018 14:31:51 +0000 (22:31 +0800)]
dpdk: drop dpdk submodule

we are using spdk/dpdk for async msgr's dpdk backend since 01a9f178. so
no need to keep dpdk submodule anymore.

Fixes: http://tracker.ceph.com/issues/24032
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cb8087dfac31b8490fefdfca28d389b7b9901ef8)

7 years agoos/bluestore: fix races on SharedBlob::coll in ~SharedBlob. 23065/head
Radoslaw Zarzynski [Tue, 10 Jul 2018 23:14:54 +0000 (01:14 +0200)]
os/bluestore: fix races on SharedBlob::coll in ~SharedBlob.

Example scenario:
  ```
  A: BlueStore::Collection::split_cache(src, dest)
  A: std::lock(src->cache->lock, dest->cache->lock)

  B: SharedBlob::~SharedBlob
  B: waits coll->cache->lock // coll == src

  A: sb->coll := dest
  A: unlocks both src and dest's cache mutexes

  C: // any locked operation on dest's cache
  C: acquires dest->cache->lock
  C: begins the op

  B: // with the src's cache mutex acquired
  B: BufferSpace::_clear(coll->cache) // coll == dest
  B: // oops, B operates on cache already locked by C
  ```

Fixes: http://tracker.ceph.com/issues/24859
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit d1b16749c7d32eead6e326c492ef9edf0d03a2b8)

7 years agoMerge pull request #23034 from tchaikov/mimic-clang-4
Kefu Chai [Sat, 14 Jul 2018 12:50:49 +0000 (20:50 +0800)]
Merge pull request #23034 from tchaikov/mimic-clang-4

include: fix build with older clang

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
7 years agoqa: use 4 mds cluster for multifs tests 22835/head
Patrick Donnelly [Fri, 13 Jul 2018 04:35:46 +0000 (21:35 -0700)]
qa: use 4 mds cluster for multifs tests

Fixes: http://tracker.ceph.com/issues/24899
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3b7233a402ec68094ae86333c3f4f09706d173d1)

7 years agoqa: add one-off clusters to qa/cephfs/clusters
Patrick Donnelly [Fri, 29 Jun 2018 16:30:52 +0000 (09:30 -0700)]
qa: add one-off clusters to qa/cephfs/clusters

To consistently handle openstack configuration.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 654aaa90ed478b3e1e77fb7601dd5957f03dc760)

7 years agoinclude: fix build with older clang 23034/head
Kefu Chai [Fri, 13 Jul 2018 16:40:36 +0000 (00:40 +0800)]
include: fix build with older clang

to fix the FTBFS on clang shipped with xcode 9.1 (based on Clang 4.0).

Signed-off-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22997 from liewegas/wip-24597-mimic
Yuri Weinstein [Fri, 13 Jul 2018 15:18:12 +0000 (08:18 -0700)]
Merge pull request #22997 from liewegas/wip-24597-mimic

mimic: osd: do not blindly roll forward to log.head

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoglobal/global_init: fix stdout/stderr/stdin closing for daemonization 23024/head
Sage Weil [Mon, 9 Jul 2018 18:26:39 +0000 (13:26 -0500)]
global/global_init: fix stdout/stderr/stdin closing for daemonization

The global_init_postfork/prefork helpers close stdout/stdin/stderr on
fork and reopen /dev/null in their place.  This ensures that if later
code writes to those descriptors (e.g., a stray cout or cerr usage) the
output/input will go nowhere instead of interfering with some other open
fd.

However, with the use of preforker, there are other threads running when
these helpers are run, which means we can race with, say, filestore
opening an object file and end up sending log output there.

Fix by atomically replacing the fds with the dup2(2) syscall, which
will implicitly close and reopen the target fd in an atomic fashion.  This
behavior is present on both Linux and FreeBSD.

Fixes: http://tracker.ceph.com/issues/23492
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3c2d91ed11c19ee8769acef8268938d2cab59152)

7 years agorgw: radogw-admin reshard status command should print text for reshard status 23021/head
Orit Wasserman [Mon, 2 Jul 2018 13:03:12 +0000 (16:03 +0300)]
rgw: radogw-admin reshard status command should print text for reshard status

Fixes: http://tracker.ceph.com/issues/23257
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit 14d25dca343c73108dcd6a04d9695b82d4719a63)

7 years agoMerge pull request #22842 from pdvian/wip-24547-mimic
Yuri Weinstein [Thu, 12 Jul 2018 16:37:03 +0000 (09:37 -0700)]
Merge pull request #22842 from pdvian/wip-24547-mimic

mimic: rgw: have a configurable authentication order

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22888 from smithfarm/wip-24617-mimic
Yuri Weinstein [Thu, 12 Jul 2018 16:36:24 +0000 (09:36 -0700)]
Merge pull request #22888 from smithfarm/wip-24617-mimic

mimic: ValueError: too many values to unpack due to lack of subdir

Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #22986 from joscollin/wip-24843-mimic
Yuri Weinstein [Thu, 12 Jul 2018 16:35:53 +0000 (09:35 -0700)]
Merge pull request #22986 from joscollin/wip-24843-mimic

mimic: rgw: require --yes-i-really-mean-it to run radosgw-admin orphans find

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agorgw: "radosgw-admin objects expire" always returns ok even if the process 23001/head
Zhang Shaowen [Wed, 20 Jun 2018 03:45:45 +0000 (11:45 +0800)]
rgw: "radosgw-admin objects expire" always returns ok even if the process
fails.

Fixes: http://tracker.ceph.com/issues/24592
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit f6fe07e8f549f6c54e4bc81a89558f14e71d3d9f)

7 years agoMerge pull request #22291 from pdvian/wip-24204-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:06:41 +0000 (17:06 -0700)]
Merge pull request #22291 from pdvian/wip-24204-mimic

mimic: ceph_test_rados_api_misc: fix LibRadosMiscPool.PoolCreationRace

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22412 from tchaikov/mimic-cmake-boost-1.67
Yuri Weinstein [Thu, 12 Jul 2018 00:06:20 +0000 (17:06 -0700)]
Merge pull request #22412 from tchaikov/mimic-cmake-boost-1.67

mimic: cmake: bump up the required boost version to 1.67

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #22492 from tchaikov/mimic-24321
Yuri Weinstein [Thu, 12 Jul 2018 00:05:33 +0000 (17:05 -0700)]
Merge pull request #22492 from tchaikov/mimic-24321

mimic: qa: wait longer for osd to flush pg stats

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22777 from croit/mimic-22977
Yuri Weinstein [Thu, 12 Jul 2018 00:04:58 +0000 (17:04 -0700)]
Merge pull request #22777 from croit/mimic-22977

mimic:  common/hobject: improved hash calculation for hobject_t etc

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22805 from smithfarm/wip-24708-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:04:30 +0000 (17:04 -0700)]
Merge pull request #22805 from smithfarm/wip-24708-mimic

mimic: ceph osd safe-to-destroy crashes the mgr

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #22813 from smithfarm/wip-24747-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:03:47 +0000 (17:03 -0700)]
Merge pull request #22813 from smithfarm/wip-24747-mimic

mimic: change default filestore_merge_threshold to -10

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22887 from smithfarm/wip-24473-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:02:45 +0000 (17:02 -0700)]
Merge pull request #22887 from smithfarm/wip-24473-mimic

mimic: cosbench stuck at booting cosbench driver

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22889 from smithfarm/wip-24618-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:02:07 +0000 (17:02 -0700)]
Merge pull request #22889 from smithfarm/wip-24618-mimic

mimic: osd: choose_acting loop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22910 from ifed01/wip-ifed-fix-23540-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:00:33 +0000 (17:00 -0700)]
Merge pull request #22910 from ifed01/wip-ifed-fix-23540-mimic

mimic: os/bluestore: fix incomplete faulty range marking when doing compression

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22960 from pdvian/wip-24805-mimic
Yuri Weinstein [Thu, 12 Jul 2018 00:00:02 +0000 (17:00 -0700)]
Merge pull request #22960 from pdvian/wip-24805-mimic

mimic: osd/PrimaryLogPG: rebuild attrs from clients

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22550 from smithfarm/wip-24296-mimic
Yuri Weinstein [Wed, 11 Jul 2018 23:25:45 +0000 (16:25 -0700)]
Merge pull request #22550 from smithfarm/wip-24296-mimic

mimic: repeated eviction of idle client until some IO happens

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
7 years agoMerge pull request #22810 from smithfarm/wip-24703-mimic
Yuri Weinstein [Wed, 11 Jul 2018 23:24:54 +0000 (16:24 -0700)]
Merge pull request #22810 from smithfarm/wip-24703-mimic

mimic: PurgeQueue sometimes ignores Journaler errors

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22816 from pdvian/wip-24537-mimic
Yuri Weinstein [Wed, 11 Jul 2018 23:24:32 +0000 (16:24 -0700)]
Merge pull request #22816 from pdvian/wip-24537-mimic

mimic: common/DecayCounter: set last_decay to current time when decoding dec…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22884 from smithfarm/wip-24704-mimic
Yuri Weinstein [Wed, 11 Jul 2018 23:24:06 +0000 (16:24 -0700)]
Merge pull request #22884 from smithfarm/wip-24704-mimic

mimic: mds: low wrlock efficiency due to dirfrags traversal

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoqa/standalone/osd/ec-error-rollforward: reproduce bug 24597 22997/head
Sage Weil [Wed, 11 Jul 2018 21:14:38 +0000 (16:14 -0500)]
qa/standalone/osd/ec-error-rollforward: reproduce bug 24597

This reproduces http://tracker.ceph.com/issues/24597

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4108ebc0abae83777ebc69bda610167f39e22109)

7 years agoqa/standalone/osd/repro_long_log.sh: fix test
Sage Weil [Wed, 11 Jul 2018 12:10:28 +0000 (07:10 -0500)]
qa/standalone/osd/repro_long_log.sh: fix test

The log trimming case wasn't quite right.  Before HEAD^ we were
rolling forward too aggressively and miscalculating the can_rollforward_to,
which affected the trim_to calculation.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4f9fdd98e2dde3979ec88a3f761d76f7b297415c)

7 years agoosd/PG: do not blindly roll forward to log.head
Sage Weil [Wed, 11 Jul 2018 01:22:49 +0000 (20:22 -0500)]
osd/PG: do not blindly roll forward to log.head

If we are told we can roll forward by the primary, we should only roll
forward as far as the primary says we can.

This probably came out of the similar case in append_log(), but notably
that roll_forward() only happens if !transaction_applied (i.e., backfill
target), and that condition is not checked here.

Fixes: http://tracker.ceph.com/issues/24597
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 858ff647ff7bd1f6277823b5185ead5ffebc05b0)

7 years agoMerge pull request #22886 from smithfarm/wip-24707-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:19:14 +0000 (13:19 -0700)]
Merge pull request #22886 from smithfarm/wip-24707-mimic

mimic: mgr/dashboard: Float numbers incorrectly formatted

Reviewed-by: Neha Ojha <nojha@redhat.com>
7 years agoMerge pull request #22265 from liewegas/wip-24308-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:17:08 +0000 (13:17 -0700)]
Merge pull request #22265 from liewegas/wip-24308-mimic

mimic: mon/MDSMonitor: do not send redundant MDS health messages to cluster log

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22707 from pdvian/wip-24475-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:16:23 +0000 (13:16 -0700)]
Merge pull request #22707 from pdvian/wip-24475-mimic

mimic: librados: fix buffer overflow for aio_exec python binding

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #22826 from smithfarm/wip-24738-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:15:29 +0000 (13:15 -0700)]
Merge pull request #22826 from smithfarm/wip-24738-mimic

mimic: Bring back diff -y for non-FreeBSD

Reviewed-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #22851 from smithfarm/wip-24766-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:14:37 +0000 (13:14 -0700)]
Merge pull request #22851 from smithfarm/wip-24766-mimic

mimic: build/ops: Mimic build fails with -DWITH_RADOSGW=0

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Bassam Tabbara <bassam.tabbara@quantum.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
7 years agoMerge pull request #22859 from smithfarm/wip-24769-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:11:49 +0000 (13:11 -0700)]
Merge pull request #22859 from smithfarm/wip-24769-mimic

mimic: set correctly shard for existed Collection

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22861 from smithfarm/wip-24771-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:11:17 +0000 (13:11 -0700)]
Merge pull request #22861 from smithfarm/wip-24771-mimic

mimic: osd: may get empty info at recovery

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #22917 from smithfarm/wip-24803-mimic
Yuri Weinstein [Wed, 11 Jul 2018 20:09:29 +0000 (13:09 -0700)]
Merge pull request #22917 from smithfarm/wip-24803-mimic

mimic: Python bindings use iteritems method which is not Python 3 compatible

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agorgw: require --yes-i-really-mean-it to run radosgw-admin orphans find 22986/head
Matt Benjamin [Wed, 16 May 2018 17:04:55 +0000 (13:04 -0400)]
rgw: require --yes-i-really-mean-it to run radosgw-admin orphans find

Incorrect use of orphans find can lead to data loss. Warn users to be
extra cautious.

Fixes: http://tracker.ceph.com/issues/24146
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 3ff47c7f3eb5964464c8cd49144546ce532ed7f7)

7 years agocephfs-journal-tool: Fix purging when importing an zero-length journal. 22981/head
chenyupeng360 [Tue, 22 May 2018 04:34:51 +0000 (12:34 +0800)]
cephfs-journal-tool: Fix purging when importing an zero-length journal.

When importing a zero-length purge_queue journal exported previously, the last object and
the following one are now being purged for removing potential junks. In this case,
there will be no writing performed actually, so the purged last object get lost permanently.

This can be fixed by purging the object following the last object, and by zeroing the last object
starting from the offset determined by the write_pos.

Fixes: https://tracker.ceph.com/issues/24239
Signed-off-by: yupeng chen <chenyupeng-it@360.cn>
Signed-off-by: zhongyan gu <guzhongyan@360.cn>
(cherry picked from commit b6c68dd55272f9066d7b5cf2fa30ea44f7c36533)

7 years agolibrbd: fix gmock warnings introduced after 9e5b87c 22894/head
Mykola Golub [Tue, 19 Jun 2018 13:22:15 +0000 (16:22 +0300)]
librbd: fix gmock warnings introduced after 9e5b87c

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit f434c5c70bc3f51849d1439dcd8448022cb5e056)

7 years agolibrbd: deep_copy: update end_size only if zero interval caused truncate
Mykola Golub [Tue, 19 Jun 2018 12:05:27 +0000 (15:05 +0300)]
librbd: deep_copy: update end_size only if zero interval caused truncate

The problem shown up when stripping was used, and assembling a
nonexistent destination object from source objects, and one of these
objects existed and snap diff returned a zero interval. The non-zero
end_size in that case triggered invalid object map update.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d1122573b8c6a69857004d042cc75e9581deebda)

7 years agolibrbd: deep_copy: don't hide parent if zero overlap for snapshot
Mykola Golub [Sat, 16 Jun 2018 11:19:53 +0000 (14:19 +0300)]
librbd: deep_copy: don't hide parent if zero overlap for snapshot

Fixes: http://tracker.ceph.com/issues/24545
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d3d730ce0f9f7f287e4b60cface66e9d9fb6882e)

7 years agolibrbd: remove unused header
Mykola Golub [Sat, 9 Dec 2017 15:10:10 +0000 (17:10 +0200)]
librbd: remove unused header

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 84ed0c5726e6a0707189f7adec0dcfff0fe73c57)

7 years agoMerge pull request #22549 from smithfarm/wip-24391-mimic
Yuri Weinstein [Tue, 10 Jul 2018 16:01:44 +0000 (09:01 -0700)]
Merge pull request #22549 from smithfarm/wip-24391-mimic

mimic: [rbd-mirror] entries_behind_master will not be zero after mirror over

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #22768 from pdvian/wip-24499-mimic
Yuri Weinstein [Tue, 10 Jul 2018 16:00:53 +0000 (09:00 -0700)]
Merge pull request #22768 from pdvian/wip-24499-mimic

mimic: librbd: deep_copy: resize head object map if needed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #22770 from jcsp/wip-24689
Yuri Weinstein [Tue, 10 Jul 2018 15:59:35 +0000 (08:59 -0700)]
Merge pull request #22770 from jcsp/wip-24689

mimic: mon: exception for dashboard in config-key warning

Reviewed-by: Ricardo Dias <rdias@suse.com>
7 years agoMerge pull request #22268 from xiaoxichen/backport_22243_mimic
Kefu Chai [Tue, 10 Jul 2018 12:41:50 +0000 (20:41 +0800)]
Merge pull request #22268 from xiaoxichen/backport_22243_mimic

mimic: mon/OSDMonitor: Respect paxos_propose_interval

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoosd/PrimaryLogPG: rebuild attrs from clients 22960/head
Sage Weil [Wed, 4 Jul 2018 19:19:04 +0000 (14:19 -0500)]
osd/PrimaryLogPG: rebuild attrs from clients

Ensure that buffers coming in via client ops are rebuilt before populating
the PGTransaction.  This ensures that we don't pin the raw buffers for
the entire incoming message in memory.

In the past we've addressed this issue at the ObjectStore layer, but we
did not consider the attr_cache in ObjectContext.  Rebuilding the buffers
at this point will sanitize any incoming attribute before it reaches
either PGBackend or ObjectContext object_cache or the ObjectStore
implementation.

Fixes: http://tracker.ceph.com/issues/24768
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0a0f8d5e6fa8276f06a9b7a7b18ea6f15fa8cb86)

7 years agoqa: fix ffsb cd argument 22956/head
Patrick Donnelly [Wed, 27 Jun 2018 18:26:47 +0000 (11:26 -0700)]
qa: fix ffsb cd argument

Avoid potential failures relating to HEAD^.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 6c7e09568d0ebcc054267b6a33e075dbb6407982)

7 years agoqa/workunits/suites/ffsb.sh: fix strncpy bug in ffsb
Yan, Zheng [Tue, 26 Jun 2018 12:28:51 +0000 (20:28 +0800)]
qa/workunits/suites/ffsb.sh: fix strncpy bug in ffsb

Signed-off-by: Yan, Zheng <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/24579
(cherry picked from commit 843cddaaf83abdff0d900af85e8bd2382d834e0d)

7 years agoqa: fix cd argument during iogen unpack
Patrick Donnelly [Wed, 27 Jun 2018 18:24:09 +0000 (11:24 -0700)]
qa: fix cd argument during iogen unpack

Fixes: http://tracker.ceph.com/issues/24680
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit fe62ea36d4710a526c0d1d2379d595dc6a43b868)

7 years agoMerge pull request #22700 from yuriw/wip-yuriw-22532-backport
Yuri Weinstein [Mon, 9 Jul 2018 20:31:20 +0000 (13:31 -0700)]
Merge pull request #22700 from yuriw/wip-yuriw-22532-backport

mimic: qa/suites: Add supported-random-distro$ links.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Warren Usui <wusui@redhat.com>

7 years agoMerge pull request #22791 from pdvian/wip-24534-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:28:24 +0000 (13:28 -0700)]
Merge pull request #22791 from pdvian/wip-24534-mimic

mimic: client: fix bug #24491 _ll_drop_pins may access invalid iterator

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22819 from pdvian/wip-24539-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:27:59 +0000 (13:27 -0700)]
Merge pull request #22819 from pdvian/wip-24539-mimic

mimic: client:  update inode fields according to issued caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22841 from pdvian/wip-24541-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:27:26 +0000 (13:27 -0700)]
Merge pull request #22841 from pdvian/wip-24541-mimic

mimic: mds: add magic to header of open file table

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #22882 from smithfarm/wip-24310-mimic
Yuri Weinstein [Mon, 9 Jul 2018 20:25:59 +0000 (13:25 -0700)]
Merge pull request #22882 from smithfarm/wip-24310-mimic

mimic: pjd: cd: too many arguments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #22900 from pdvian/wip-24585-mimic
Alfredo Deza [Mon, 9 Jul 2018 19:36:42 +0000 (15:36 -0400)]
Merge pull request #22900 from pdvian/wip-24585-mimic

mimic: ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #22849 from smithfarm/wip-24764-mimic
Lenz Grimmer [Mon, 9 Jul 2018 18:52:50 +0000 (20:52 +0200)]
Merge pull request #22849 from smithfarm/wip-24764-mimic

mimic: mgr/dashboard: Missing breadcrumb on monitor performance counters page

Reviewed-by: Ricardo Marques <rimarques@suse.com>
7 years agoMerge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic
Sage Weil [Mon, 9 Jul 2018 13:29:13 +0000 (08:29 -0500)]
Merge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic

7 years agoMerge remote-tracking branch 'private/wip-cx-mimic' into mimic
Sage Weil [Mon, 9 Jul 2018 13:09:38 +0000 (08:09 -0500)]
Merge remote-tracking branch 'private/wip-cx-mimic' into mimic

7 years agolibrbd: fix crash when opening nonexistent snapshot 22943/head
Mykola Golub [Fri, 22 Jun 2018 17:50:51 +0000 (20:50 +0300)]
librbd: fix crash when opening nonexistent snapshot

Fixes: http://tracker.ceph.com/issues/24637
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 70936c3bb5c281eed1c0bb5e26337efdedfa1e06)

7 years agorgw: fix gc may cause a large number of read traffic 22941/head
liaoxin01 [Thu, 5 Jul 2018 06:23:34 +0000 (14:23 +0800)]
rgw: fix gc may cause a large number of read traffic

fix http://tracker.ceph.com/issues/24767

Signed-off-by: Xin Liao <liaoxin01@baidu.com>
(cherry picked from commit afc3b1ab1f6b24267c5ab846e2e7a5de5ec9a5d0)

7 years agorgw: reject invalid methods in validate_cors_rule_method 22935/head
Jeegn Chen [Tue, 22 May 2018 05:16:45 +0000 (13:16 +0800)]
rgw: reject invalid methods in validate_cors_rule_method

Fixes: http://tracker.ceph.com/issues/24223
Signed-off-by: Jeegn Chen <jeegnchen@tencent.com>
(cherry picked from commit 8e34c65e3f59adb0ed549ad7ea394872c6e4248f)

7 years agorgw: escape markers in RGWOp_Metadata_List::execute 22933/head
Matt Benjamin [Tue, 26 Jun 2018 21:40:48 +0000 (17:40 -0400)]
rgw: escape markers in RGWOp_Metadata_List::execute

Fixes: http://tracker.ceph.com/issues/23099
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 3055a67cc20f1e461f5f1a26cb1522b7e88c79f9)

7 years agorgw: remove expired entries from the cache 22929/head
Mark Kogan [Tue, 5 Jun 2018 08:18:30 +0000 (11:18 +0300)]
rgw: remove expired entries from the cache
Fixes: http://tracker.ceph.com/issues/23379
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 98f6dada873f32be8750bf3e101de4f33ea9b26f)

7 years agocls/rgw: don't assert in decode_list_index_key() 22927/head
Yehuda Sadeh [Wed, 6 Jun 2018 17:00:47 +0000 (10:00 -0700)]
cls/rgw: don't assert in decode_list_index_key()

Fixes: http://tracker.ceph.com/issues/24117
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit e71fba0c6f5b3b0572b5136840cf0ed3c9186569)