]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoMerge pull request #28717 from smithfarm/wip-40506-luminous
Yuri Weinstein [Mon, 22 Jul 2019 19:54:27 +0000 (12:54 -0700)]
Merge pull request #28717 from smithfarm/wip-40506-luminous

luminous: rgw: conditionally allow builtin users with non-unique email addresses

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28784 from chardan/wip-23223-luminous
Yuri Weinstein [Mon, 22 Jul 2019 19:54:01 +0000 (12:54 -0700)]
Merge pull request #28784 from chardan/wip-23223-luminous

luminous: rgw: gc use aio

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26381 from ypdai/patch-1
Gregory Farnum [Fri, 12 Jul 2019 19:02:22 +0000 (12:02 -0700)]
Merge pull request #26381 from ypdai/patch-1

Update layout.rst

6 years agoMerge pull request #28965 from liewegas/wip-set-aio-write-max-luminous
Neha Ojha [Thu, 11 Jul 2019 18:47:22 +0000 (11:47 -0700)]
Merge pull request #28965 from liewegas/wip-set-aio-write-max-luminous

luminous: os/bluestore: >2GB bluefs writes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agorgw: "radosgw-admin objects expire" always returns ok even if the process 28784/head
Zhang Shaowen [Wed, 20 Jun 2018 03:45:45 +0000 (11:45 +0800)]
rgw: "radosgw-admin objects expire" always returns ok even if the process
fails.

Fixes: http://tracker.ceph.com/issues/24592
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit f6fe07e8f549f6c54e4bc81a89558f14e71d3d9f)

6 years agorgw: use vector for remove_tags in gc aio
Yehuda Sadeh [Fri, 23 Feb 2018 16:41:19 +0000 (08:41 -0800)]
rgw: use vector for remove_tags in gc aio

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 9d37caccd907a7e58a836c5bcd9fb00a3dda8216)

6 years agorgw: gc aio, replace lists with other types
Yehuda Sadeh [Fri, 23 Feb 2018 16:43:16 +0000 (08:43 -0800)]
rgw: gc aio, replace lists with other types

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 278ca0352ac357426f435d1bb0343a6a815d1cdb)

6 years agorgw: make gc concurrenct io size configurable
Yehuda Sadeh [Thu, 22 Feb 2018 22:28:29 +0000 (14:28 -0800)]
rgw: make gc concurrenct io size configurable

and another tunable for log trim size

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 1cceef22cf26e1a4a0234fb2f977a5fefae82905)

6 years agorgw: trim gc index using aio
Yehuda Sadeh [Thu, 22 Feb 2018 22:21:37 +0000 (14:21 -0800)]
rgw: trim gc index using aio

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit f00ee63e981646d0dc19bcb3d1e4165ef0a8cfce)
Signed-off-by: Jesse Williamson <jesse.williamson@canonical.com>
Conflicts: rgw_gc.cc (uses a kind of aggregate initialization not available in C++11)

6 years agorgw: use a single gc io manager for all shards
Yehuda Sadeh [Thu, 22 Feb 2018 21:57:33 +0000 (13:57 -0800)]
rgw: use a single gc io manager for all shards

to allow cross shards concurrency

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit e60404cb18682dcc6dad03ee1806a601d5c8b2e0)

6 years agorgw: use aio for gc processing
Yehuda Sadeh [Thu, 22 Feb 2018 21:31:59 +0000 (13:31 -0800)]
rgw: use aio for gc processing

still need to deal with index cleanup asynchronously

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 1f6dc8c9b83fd0e74f7d524178fb37e90d06bbef)

6 years agorgw-admin: support for processing all gc objects including unexpired.
Yao Zongyou [Tue, 5 Sep 2017 09:33:04 +0000 (17:33 +0800)]
rgw-admin: support for processing all gc objects including unexpired.

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
(cherry picked from commit ceccb54f191182813cae70841e87b9969c3d4aeb)

6 years agodoc: modify the wrong word "defails" to "details". 26381/head
ypdai [Tue, 12 Feb 2019 09:50:07 +0000 (17:50 +0800)]
doc: modify the wrong word "defails" to "details".

Signed-off-by: ypdai <self19900924@gmail.com>
6 years agoceph_test_objectstore: add very_large_write test 28965/head
Sage Weil [Tue, 16 Apr 2019 13:53:56 +0000 (08:53 -0500)]
ceph_test_objectstore: add very_large_write test

Write 2GB to verify the blockdevice aio splitting.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1974f011137976edf277833d68bdcc4ff22779b0)
- fixed get_val/set_val calls
- add_block_device doesn't take third arg in luminous

6 years agoos/bluestore: fix aio pwritev lost data problem.
kungf [Tue, 16 Apr 2019 11:21:41 +0000 (19:21 +0800)]
os/bluestore: fix aio pwritev lost data problem.

On Linux, write() (and similar system calls) will transfer at most
0x7ffff000 (2,147,479,552) bytes, it will cap data if aio pwritev
more than 0x7ffff000, so we have the split the data to more aio submit.

Signed-off-by: kungf <yang.wang@easystack.cn>
(cherry picked from commit 4d33114a40d5ae0d541c36175977ca22789a3b88)

- conflict due to fd_direct instead of choose_fd()

6 years agobluestore: print aio in batch
Sage Weil [Wed, 10 Jul 2019 15:21:12 +0000 (10:21 -0500)]
bluestore: print aio in batch

KernelDevice::aio_{submit,write,read}() are critical paths. calling
cct->_conf->subsys.should_gather() multi-times is not optimal. the
downside of this issue is that if the aio is printed, the size of
buffer in PrebufferedStreambuf could be large if the number of iov is
large, that could be a heavy load to the memory subsystem.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 67c6f768add46d92d9403ca6b7709b892d8a686d)

# Conflicts:
# src/os/bluestore/aio.cc
 - adjacent to aio_queue_t::submit()

6 years agoMerge pull request #28926 from ceph/backport-luminous-28294
Alfredo Deza [Tue, 9 Jul 2019 18:37:29 +0000 (14:37 -0400)]
Merge pull request #28926 from ceph/backport-luminous-28294

luminous: tests: pass --ssh-config to pytest to resolve hosts when connecting

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #27365 from ifed01/wip-ifed-fix-alloc-overflow-luminous
Yuri Weinstein [Tue, 9 Jul 2019 16:32:57 +0000 (09:32 -0700)]
Merge pull request #27365 from ifed01/wip-ifed-fix-alloc-overflow-luminous

luminous: os/bluestore: fix length overflow.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jianpeng Ma <majianpeng@gmail.com>
6 years agoMerge pull request #28640 from tchaikov/luminous-38040
Yuri Weinstein [Tue, 9 Jul 2019 16:29:09 +0000 (09:29 -0700)]
Merge pull request #28640 from tchaikov/luminous-38040

luminous: mon,osd: limit MOSDMap messages by size as well as map count

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27459 from ifed01/wip-ifed-no-verify-fm-luminous
Yuri Weinstein [Tue, 9 Jul 2019 16:08:04 +0000 (09:08 -0700)]
Merge pull request #27459 from ifed01/wip-ifed-no-verify-fm-luminous

luminous: os/bluestore/BitmapFreelistManager: disable bluestore_debug_freelist

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27529 from ifed01/wip-ifed-fix-21312-luminous
Yuri Weinstein [Tue, 9 Jul 2019 16:07:24 +0000 (09:07 -0700)]
Merge pull request #27529 from ifed01/wip-ifed-fix-21312-luminous

luminous: os/bluestore: call fault_range properly prior to looking for blob to …

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #27583 from xiexingguo/wip-38242
Yuri Weinstein [Tue, 9 Jul 2019 16:06:55 +0000 (09:06 -0700)]
Merge pull request #27583 from xiexingguo/wip-38242

luminous: backport recent messenger fixes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27739 from ifed01/wip-ifed-fix-39444
Yuri Weinstein [Tue, 9 Jul 2019 16:06:13 +0000 (09:06 -0700)]
Merge pull request #27739 from ifed01/wip-ifed-fix-39444

luminous: os/bluestore: fix out-of-bound access in bmap allocator.

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27950 from smithfarm/wip-37612-luminous
Yuri Weinstein [Tue, 9 Jul 2019 16:05:39 +0000 (09:05 -0700)]
Merge pull request #27950 from smithfarm/wip-37612-luminous

luminous: build/ops: python-cephfs should depend on python-rados

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
6 years agoMerge pull request #28030 from ifed01/wip-ifed-dump-before-nospanid-luminous
Yuri Weinstein [Tue, 9 Jul 2019 16:05:07 +0000 (09:05 -0700)]
Merge pull request #28030 from ifed01/wip-ifed-dump-before-nospanid-luminous

luminous: os/bluestore: dump before "no spanning blob id" abort

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #28111 from smithfarm/wip-38750-luminous
Yuri Weinstein [Tue, 9 Jul 2019 16:04:39 +0000 (09:04 -0700)]
Merge pull request #28111 from smithfarm/wip-38750-luminous

luminous: core: should report EINVAL in ErasureCode::parse() if m<=0

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #28929 from ceph/backport-luminous-28866
Alfredo Deza [Mon, 8 Jul 2019 17:26:47 +0000 (13:26 -0400)]
Merge pull request #28929 from ceph/backport-luminous-28866

luminous: ceph-volume broken assertion errors after pytest changes

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #28927 from ceph/backport-luminous-28836
Alfredo Deza [Mon, 8 Jul 2019 17:26:20 +0000 (13:26 -0400)]
Merge pull request #28927 from ceph/backport-luminous-28836

luminous: ceph-volume tests add a sleep in tox for slow OSDs after booting

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume api.lvm catch IndexError when parsing dmmapper output 28929/head
Alfredo Deza [Wed, 3 Jul 2019 18:59:18 +0000 (14:59 -0400)]
ceph-volume api.lvm catch IndexError when parsing dmmapper output

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfff70d07a1ccf7fb11a96dd7c63335cfc9b5e26)

6 years agoceph-volume tests update to use error.value instead of str(error)
Alfredo Deza [Wed, 3 Jul 2019 18:58:31 +0000 (14:58 -0400)]
ceph-volume tests update to use error.value instead of str(error)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 484be7ffb1e0865ff840090a73036567a1f4f5d2)

6 years agoceph-volume tests add a sleep in tox for slow OSDs after booting 28927/head
Alfredo Deza [Tue, 2 Jul 2019 12:34:01 +0000 (08:34 -0400)]
ceph-volume tests add a sleep in tox for slow OSDs after booting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b398f99f810d97c94af6d943aed4c11a5fd69c19)

6 years agotests: pass --ssh-config to pytest to resolve hosts when connecting 28926/head
Alfredo Deza [Wed, 29 May 2019 12:56:37 +0000 (08:56 -0400)]
tests: pass --ssh-config to pytest to resolve hosts when connecting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ed708e520999c145c3cf8a0c51e4605f275a1771)

6 years agoMerge pull request #28536 from ukernel/luminous-40211
Yuri Weinstein [Mon, 1 Jul 2019 16:10:34 +0000 (09:10 -0700)]
Merge pull request #28536 from ukernel/luminous-40211

luminous: mds: fix corner case of replaying open sessions

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #28543 from batrick/i40041
Yuri Weinstein [Mon, 1 Jul 2019 16:10:06 +0000 (09:10 -0700)]
Merge pull request #28543 from batrick/i40041

luminous: mds: avoid trimming too many log segments after mds failover

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #28544 from ukernel/luminous-40221
Yuri Weinstein [Mon, 1 Jul 2019 16:09:36 +0000 (09:09 -0700)]
Merge pull request #28544 from ukernel/luminous-40221

luminous: mds: reset heartbeat inside big loop

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #28669 from smithfarm/wip-40347-luminous
Ali Maredia [Wed, 26 Jun 2019 17:46:31 +0000 (13:46 -0400)]
Merge pull request #28669 from smithfarm/wip-40347-luminous

luminous: tests: use curl in wait_for_radosgw() in util/rgw.py

6 years agoMerge pull request #27536 from pdvian/wip-38975-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:22:23 +0000 (08:22 -0700)]
Merge pull request #27536 from pdvian/wip-38975-luminous

luminous: rbd: krbd: return -ETIMEDOUT in polling

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27990 from smithfarm/wip-39589-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:21:58 +0000 (08:21 -0700)]
Merge pull request #27990 from smithfarm/wip-39589-luminous

luminous: qa/tasks/rbd_fio: fixed missing delimiter between 'cd' and 'configure'

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #28124 from dillaman/wip-39460-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:21:27 +0000 (08:21 -0700)]
Merge pull request #28124 from dillaman/wip-39460-luminous

luminous: rbd-mirror: clear out bufferlist prior to listing mirror images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28126 from dillaman/wip-39427-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:21:03 +0000 (08:21 -0700)]
Merge pull request #28126 from dillaman/wip-39427-luminous

luminous: librbd: async open/close should free ImageCtx before issuing callback

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28134 from dillaman/wip-38440-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:20:38 +0000 (08:20 -0700)]
Merge pull request #28134 from dillaman/wip-38440-luminous

luminous: librbd: ensure compare-and-write doesn't skip compare after copyup

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28137 from dillaman/wip-38674-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:20:11 +0000 (08:20 -0700)]
Merge pull request #28137 from dillaman/wip-38674-luminous

luminous: librbd: improve object map performance under high IOPS workloads

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28140 from dillaman/wip-38564-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:19:47 +0000 (08:19 -0700)]
Merge pull request #28140 from dillaman/wip-38564-luminous

luminous: librbd: race condition possible when validating RBD pool

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28745 from dillaman/wip-40551-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:19:22 +0000 (08:19 -0700)]
Merge pull request #28745 from dillaman/wip-40551-luminous

luminous: qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #26554 from pdvian/wip-38397-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:17:37 +0000 (08:17 -0700)]
Merge pull request #26554 from pdvian/wip-38397-luminous

luminous: rgw: when exclusive lock fails due existing lock, log add'l info

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #27110 from pdvian/wip-38748-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:16:43 +0000 (08:16 -0700)]
Merge pull request #27110 from pdvian/wip-38748-luminous

luminous: rgw_sync: drop ENOENT error logs from mdlog

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27123 from smithfarm/wip-38887-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:16:18 +0000 (08:16 -0700)]
Merge pull request #27123 from smithfarm/wip-38887-luminous

luminous: rgw: GetBucketCORS API returns Not Found error code when CORS configuration does not exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27330 from pdvian/wip-38908-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:15:51 +0000 (08:15 -0700)]
Merge pull request #27330 from pdvian/wip-38908-luminous

luminous: rgw: fix read not exists null version return wrong

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27424 from pdvian/wip-38925-luminous
Yuri Weinstein [Wed, 26 Jun 2019 15:15:24 +0000 (08:15 -0700)]
Merge pull request #27424 from pdvian/wip-38925-luminous

luminous: rgw: Adding tcp_nodelay option to Beast

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27040 from pdvian/wip-38686-luminous
Yuri Weinstein [Tue, 25 Jun 2019 16:31:00 +0000 (09:31 -0700)]
Merge pull request #27040 from pdvian/wip-38686-luminous

luminous: qa/cephfs: relax min_caps_per_client check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #27342 from vshankar/wip-38445
Yuri Weinstein [Tue, 25 Jun 2019 16:30:15 +0000 (09:30 -0700)]
Merge pull request #27342 from vshankar/wip-38445

luminous: qa: misc cache drop fixes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #27679 from rishabh-d-dave/wip-38877-luminous
Yuri Weinstein [Tue, 25 Jun 2019 16:29:45 +0000 (09:29 -0700)]
Merge pull request #27679 from rishabh-d-dave/wip-38877-luminous

luminous: mds: dont print subtrees if they are too big or too many

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #27737 from pdvian/wip-39191-luminous
Yuri Weinstein [Tue, 25 Jun 2019 16:29:16 +0000 (09:29 -0700)]
Merge pull request #27737 from pdvian/wip-39191-luminous

luminous: mds: drop reconnect message from non-existent session

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #27801 from pdvian/wip-39198-luminous
Yuri Weinstein [Tue, 25 Jun 2019 16:28:56 +0000 (09:28 -0700)]
Merge pull request #27801 from pdvian/wip-39198-luminous

luminous: mds/server: check directory split after rename.

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #27840 from pdvian/wip-39208-luminous
Yuri Weinstein [Tue, 25 Jun 2019 16:28:35 +0000 (09:28 -0700)]
Merge pull request #27840 from pdvian/wip-39208-luminous

luminous: mds: initialize cap_revoke_eviction_timeout with conf

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #27848 from ashishkumsingh/wip-39468-luminous
Yuri Weinstein [Tue, 25 Jun 2019 16:28:05 +0000 (09:28 -0700)]
Merge pull request #27848 from ashishkumsingh/wip-39468-luminous

luminous: mds: better output of 'ceph health detail'

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #28432 from ukernel/luminous-39221
Yuri Weinstein [Tue, 25 Jun 2019 16:27:44 +0000 (09:27 -0700)]
Merge pull request #28432 from ukernel/luminous-39221

luminous: mds: remove superfluous error in StrayManager::advance_delayed()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #28436 from batrick/i39231
Yuri Weinstein [Tue, 25 Jun 2019 16:27:09 +0000 (09:27 -0700)]
Merge pull request #28436 from batrick/i39231

luminous: cephfs: kclient: nofail option not supported

6 years agoMerge pull request #28437 from jtlayton/wip-40160-luminous
Yuri Weinstein [Tue, 25 Jun 2019 16:26:32 +0000 (09:26 -0700)]
Merge pull request #28437 from jtlayton/wip-40160-luminous

luminous: client: clean up error checking and return of _lookup_parent

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #28438 from batrick/i39213
Yuri Weinstein [Tue, 25 Jun 2019 16:25:58 +0000 (09:25 -0700)]
Merge pull request #28438 from batrick/i39213

luminous: mds: there is an assertion when calling Beacon::shutdown()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #28445 from jan--f/ceph_volume_client-use-fs-commands
Yuri Weinstein [Tue, 25 Jun 2019 16:24:58 +0000 (09:24 -0700)]
Merge pull request #28445 from jan--f/ceph_volume_client-use-fs-commands

luminous: ceph_volume_client: remove ceph mds calls in favor of ceph fs calls

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
6 years agoqa/workunits/rbd: use more recent qemu-iotests that support Bionic 28745/head
Jason Dillaman [Thu, 18 Apr 2019 18:02:45 +0000 (14:02 -0400)]
qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Fixes: http://tracker.ceph.com/issues/24668
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b184acd218b36b85f5a25678d773c35407a2b745)

6 years agorgw: conditionally allow non-unique email addresses 28717/head
Matt Benjamin [Thu, 30 May 2019 15:15:23 +0000 (11:15 -0400)]
rgw: conditionally allow non-unique email addresses

Conditionally allow non-unique email address values for builtin
RGW users.

Fixes: http://tracker.ceph.com/issues/40089
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 974791522007cca6d8fb30e83677f0ddd7c4e71d)

Conflicts:
    src/rgw/rgw_user.cc
- changed '_conf.get_val<bool>' to '_conf->get_val<bool>'

6 years agoqa: use curl in wait_for_radosgw() in util/rgw.py 28669/head
Ali Maredia [Wed, 12 Jun 2019 20:12:47 +0000 (16:12 -0400)]
qa: use curl in wait_for_radosgw() in util/rgw.py

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit 9c6afa3fb570629a57612f250a6a4890a65cbc6e)

Conflicts:
qa/tasks/rgw.py
- rgw task was refactored for mimic

6 years agoMerge pull request #27085 from pdvian/wip-38719-luminous
Yuri Weinstein [Thu, 20 Jun 2019 00:11:44 +0000 (17:11 -0700)]
Merge pull request #27085 from pdvian/wip-38719-luminous

luminous: crush/CrushWrapper: ensure crush_choose_arg_map.size == max_buckets

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27636 from smithfarm/wip-39343-luminous
Yuri Weinstein [Thu, 20 Jun 2019 00:09:21 +0000 (17:09 -0700)]
Merge pull request #27636 from smithfarm/wip-39343-luminous

luminous: ceph-objectstore-tool: rename dump-import to dump-export

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #27674 from smithfarm/wip-38873-luminous
Yuri Weinstein [Thu, 20 Jun 2019 00:08:55 +0000 (17:08 -0700)]
Merge pull request #27674 from smithfarm/wip-38873-luminous

luminous: pybind: Rados.get_fsid() returning bytes in python3

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #27715 from pdvian/wip-39042-luminous
Yuri Weinstein [Thu, 20 Jun 2019 00:08:17 +0000 (17:08 -0700)]
Merge pull request #27715 from pdvian/wip-39042-luminous

luminous: osd/PGLog: preserve original_crt to check rollbackability

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
6 years agoMerge pull request #27751 from ashishkumsingh/wip-39431-luminous
Yuri Weinstein [Thu, 20 Jun 2019 00:07:29 +0000 (17:07 -0700)]
Merge pull request #27751 from ashishkumsingh/wip-39431-luminous

luminous: osd/PG: discover missing objects when an OSD peers and PG is degraded

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27810 from pdvian/wip-39204-luminous
Yuri Weinstein [Thu, 20 Jun 2019 00:06:59 +0000 (17:06 -0700)]
Merge pull request #27810 from pdvian/wip-39204-luminous

luminous: osd: shutdown recovery_request_timer earlier

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27878 from pdvian/wip-39218-luminous
Yuri Weinstein [Thu, 20 Jun 2019 00:06:28 +0000 (17:06 -0700)]
Merge pull request #27878 from pdvian/wip-39218-luminous

luminous: osd: FAILED ceph_assert(attrs || !pg_log.get_missing().is_missing(soid) || (it_objects != pg_log.get_log().objects.end() && it_objects->second->op == pg_log_entry_t::LOST_REVERT)) in PrimaryLogPG::get_object_context()

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27908 from smithfarm/wip-39563-luminous
Yuri Weinstein [Thu, 20 Jun 2019 00:05:52 +0000 (17:05 -0700)]
Merge pull request #27908 from smithfarm/wip-39563-luminous

luminous: mon: Error message displayed when mon_osd_max_split_count would be exceeded is not as user-friendly as it could be

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoMerge pull request #28185 from dzafman/wip-39719
Yuri Weinstein [Thu, 20 Jun 2019 00:05:22 +0000 (17:05 -0700)]
Merge pull request #28185 from dzafman/wip-39719

luminous: osd: Include dups in copy_after() and copy_up_to()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agomon/OSDMonitor: limit MOSDMap message size by bytes 28640/head
Sage Weil [Fri, 8 Feb 2019 13:22:28 +0000 (07:22 -0600)]
mon/OSDMonitor: limit MOSDMap message size by bytes

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dcf1cd00cc6e2595c283b4a73a4de9df50e7330a)

Conflicts:
src/mon/OSDMonitor.cc:
1. replace g_conf() with g_conf, g_conf was replaced
with g_conf() to prepare for the crimson project. and that change was
not backported to luminous. so restore all `g_conf()` to `g_conf` in
this backport.
2. replace std::min() and std::max() with MIN() and
MAX() macros, as we modernize it in an unrelated change. and that
change was not backported to luminous. so let's be conservative, and
restore to the old style macros.

6 years agocommon/options: add osd_map_message_max_bytes
Sage Weil [Fri, 8 Feb 2019 13:22:20 +0000 (07:22 -0600)]
common/options: add osd_map_message_max_bytes

Limit MOSDMap message size by bytes as well as map count.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e15be00560c1492e0807d0779cfd0d6088396b3e)

Conflicts:
src/common/config_values.h: the changes in this file is applied
to src/common/config.h instead. because, src/common/config_values.h
is extracted in 4718b7cb2fac65b2ac7014f42ba1a10181350f0b. which
happened after luminous, and that commit was not backported.
  also, the OPT_SIZE is changed to OPT_U64, as we don't have OPT_SIZE by
then, the size_t option support was added after luminous, and that
change was not backported. so let's use the uint64_t as an alternative,
as other *bytes settings are also using OPT_U64. and OSDMonitor.cc
will be accessing that setting using the legacy way instead of using
conf.get_val<>() interface, so this change will have no impact on the
fix.

6 years agoMerge pull request #27982 from cbodley/wip-23237
Yuri Weinstein [Fri, 14 Jun 2019 14:29:19 +0000 (07:29 -0700)]
Merge pull request #27982 from cbodley/wip-23237

luminous: rgw: update civetweb submodule to match version in mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agomds: reset heartbeat inside big loop 28544/head
Yan, Zheng [Wed, 5 Jun 2019 10:05:15 +0000 (18:05 +0800)]
mds: reset heartbeat inside big loop

Fixes: https://tracker.ceph.com/issues/40171
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit d90403c8df90ed06564c84bb2dd38c16861cf34e)

 Conflicts:
src/mds/Locker.h
src/mds/MDCache.cc

6 years agomds: avoid sending too many osd requests at once after mds restarts 28543/head
simon gao [Tue, 28 May 2019 02:36:32 +0000 (22:36 -0400)]
mds: avoid sending too many osd requests at once after mds restarts
Fixes: http://tracker.ceph.com/issues/40028
Signed-off-by: simon gao <simon29rock@gmail.com>
(cherry picked from commit 78484352f553d6b67f7d5c82384a186455f05537)

6 years agoMerge pull request #27332 from pdvian/wip-38920-luminous
Yuri Weinstein [Thu, 13 Jun 2019 22:14:22 +0000 (15:14 -0700)]
Merge pull request #27332 from pdvian/wip-38920-luminous

luminous: rgw: don't crash on missing /etc/mime.types

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27935 from cbodley/wip-39572
Yuri Weinstein [Thu, 13 Jun 2019 22:10:47 +0000 (15:10 -0700)]
Merge pull request #27935 from cbodley/wip-39572

luminous: rgw:send x-amz-version-id header when upload files

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28332 from cbodley/wip-37497
Yuri Weinstein [Thu, 13 Jun 2019 22:08:36 +0000 (15:08 -0700)]
Merge pull request #28332 from cbodley/wip-37497

luminous: rgw: get or set realm zonegroup zone need check user's caps

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28069 from smithfarm/wip-39615-luminous
Yuri Weinstein [Thu, 13 Jun 2019 22:04:31 +0000 (15:04 -0700)]
Merge pull request #28069 from smithfarm/wip-39615-luminous

luminous: civetweb frontend: response is buffered in memory if content length is not explicitly specified

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28350 from smithfarm/wip-39409-luminous
Yuri Weinstein [Thu, 13 Jun 2019 22:03:40 +0000 (15:03 -0700)]
Merge pull request #28350 from smithfarm/wip-39409-luminous

luminous: rgw: inefficient unordered bucket listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agomds: ignore sessionmap version mismatch if mds_wipe_sessions is set 28536/head
Yan, Zheng [Sat, 8 Jun 2019 09:33:14 +0000 (17:33 +0800)]
mds: ignore sessionmap version mismatch if mds_wipe_sessions is set

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 2a20605c0393692bae4780ef3bd0cc684995d028)

 Conflicts:
src/mds/SessionMap.h
src/mds/journal.cc

6 years agomds: fix corner case of replaying open sessions
Yan, Zheng [Sat, 8 Jun 2019 05:08:21 +0000 (13:08 +0800)]
mds: fix corner case of replaying open sessions

Marking a session dirty may flush all existing dirty sessions. MDS
calls Server::finish_force_open_sessions() for log event that opens
multiple sessions. The function marks sessions dirty one by one. So
sessions opened by a log event may get flushed partially.

When replaying a log event that opens multiple sessions, mds need to
check if some of these sessions have already been flushed.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 277327565fae93fad8e03cc5f27b78c98354c0dc)

 Conflicts:
src/mds/SessionMap.cc
src/mds/SessionMap.h
src/mds/journal.cc

6 years agoMerge pull request #28519 from ceph/backport-luminous-28060
Andrew Schoen [Wed, 12 Jun 2019 19:44:04 +0000 (14:44 -0500)]
Merge pull request #28519 from ceph/backport-luminous-28060

luminous: ceph-volume: use the Device.rotational property instead of sys_api

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: skip missing interpreters when running tox tests 28519/head
Andrew Schoen [Fri, 10 May 2019 16:04:35 +0000 (11:04 -0500)]
ceph-volume: skip missing interpreters when running tox tests

This will skip tests for versions of python that don't exist on the node
that's running the tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f1db0a864814d9b81192d89b6eebb6c624ddec3b)

6 years agoceph-volume: use the Device.rotational property instead of sys_api
Andrew Schoen [Fri, 10 May 2019 14:02:08 +0000 (09:02 -0500)]
ceph-volume: use the Device.rotational property instead of sys_api

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1666822

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 8ebff475dfa136c88101ef7364820fe3c5cee547)

6 years agoMerge pull request #27453 from pdvian/wip-38958-luminous
Yuri Weinstein [Wed, 12 Jun 2019 15:41:30 +0000 (08:41 -0700)]
Merge pull request #27453 from pdvian/wip-38958-luminous

luminous: rgw-admin: fix data sync report for master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27668 from smithfarm/wip-39360-luminous
Yuri Weinstein [Wed, 12 Jun 2019 15:40:27 +0000 (08:40 -0700)]
Merge pull request #27668 from smithfarm/wip-39360-luminous

luminous: rgw: failed to pass test_bucket_create_naming_bad_punctuation in s3test

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27881 from pdvian/wip-39227-luminous
Yuri Weinstein [Wed, 12 Jun 2019 15:39:44 +0000 (08:39 -0700)]
Merge pull request #27881 from pdvian/wip-39227-luminous

luminous: rgw_file: save etag and acl info in setattr

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #28314 from smithfarm/wip-39180-luminous
Yuri Weinstein [Wed, 12 Jun 2019 15:39:03 +0000 (08:39 -0700)]
Merge pull request #28314 from smithfarm/wip-39180-luminous

luminous: rgw: orphans find perf improvments

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoqa: fix mdsmap lookup 28445/head
Patrick Donnelly [Tue, 24 Oct 2017 18:36:06 +0000 (11:36 -0700)]
qa: fix mdsmap lookup

We were returning the file system structure and not the mdsmap.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0e4027efdc4fd84ae3cf7eef0e2374b3429f111d)

6 years agoceph-volume-client: fix command argument order
Patrick Donnelly [Tue, 27 Feb 2018 11:35:00 +0000 (03:35 -0800)]
ceph-volume-client: fix command argument order

Fixes: http://tracker.ceph.com/issues/22038
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit cd30d28c6e691c974d6a10b4441d03febfec544f)

6 years agoqa: use fs rm_data_pool
Patrick Donnelly [Tue, 24 Oct 2017 18:51:02 +0000 (11:51 -0700)]
qa: use fs rm_data_pool

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b897ef9445e31004595df4d6243ac8333691c116)

 Conflicts:
doc/cephfs/administration.rst
qa/workunits/cephtool/test.sh
qa/workunits/fs/misc/mkpool_layout_vxattrs.sh
    Changes were dropped.

6 years agoceph_volume_client: fix usage of get_mds_map()
Ramana Raja [Thu, 21 Dec 2017 12:51:23 +0000 (18:21 +0530)]
ceph_volume_client: fix usage of get_mds_map()

get_mds_map() is an instance method, and not a global
function. Use it as such.

Fixes: http://tracker.ceph.com/issues/22524
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 181e3ea6ee8890a66e460f66a4e8c2aa74a9c002)

6 years agoceph_volume_client: remove use of obsolete mds cmd
Patrick Donnelly [Mon, 2 Oct 2017 00:08:23 +0000 (17:08 -0700)]
ceph_volume_client: remove use of obsolete mds cmd

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit cbbdd0da7d40e4e5def5cc0b9a9250348e71019f)

6 years agoMerge pull request #28493 from tchaikov/luminous-39164
Yuri Weinstein [Tue, 11 Jun 2019 23:08:51 +0000 (16:08 -0700)]
Merge pull request #28493 from tchaikov/luminous-39164

luminous: qa: install python36-{cephfs,rados} instead of python34-*

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoqa: install python3-{cephfs,rados} instead of python34-* 28493/head
Kefu Chai [Tue, 11 Jun 2019 15:17:46 +0000 (23:17 +0800)]
qa: install python3-{cephfs,rados} instead of python34-*

we install the latest python-rpm-macros on all builders since
https://github.com/ceph/ceph-build/pull/1283 . now that we started
building python36-* after that change, for testing the python3 packages on
CentOS/RHEL 7, we need to install python36-* instead of python34-*.

and after the change of 8ae1947, python36-* now "Provides" python3-*, we
can just install python3-* for fulfill the requirement for testing
python3 cephfs bindings.

Fixes: http://tracker.ceph.com/issues/39164
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: this change is not cherry-picked from master, because,
  in master, we don't install python3 packages after 7e5c85b604.
(cherry picked from commit 6790821afc749b14b1ddac68a0889059419eebb3)

6 years agoinstall-deps.sh: install '*rpm-macros'
Kefu Chai [Thu, 11 Apr 2019 16:11:42 +0000 (00:11 +0800)]
install-deps.sh: install '*rpm-macros'

so `yum-builddep` can have access to the latest macros for preparing the
build dependencies

Fixes: http://tracker.ceph.com/issues/39164
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 087ea813a061a1d6858aeae57950e90965f4ae15)

Conflicts:
install-deps.sh: trivial resolution