]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoqa/rgw: add credentials to zone 'remove' in test_zonegroup_remove() 28332/head
Casey Bodley [Wed, 28 Nov 2018 18:18:05 +0000 (13:18 -0500)]
qa/rgw: add credentials to zone 'remove' in test_zonegroup_remove()

zone credentials are required to 'period update --commit' from
--rgw-zone remove

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Conflicts:
src/test/rgw/rgw_multi/tests.py
(cherry picked from commit 56021eab9584f10fa43053621cce93b623c36265)

6 years agorgw: get or set realm zonegroup zone need check user's caps
yuliyang [Tue, 20 Nov 2018 09:19:38 +0000 (17:19 +0800)]
rgw: get or set realm zonegroup zone need check user's caps

fix: https://tracker.ceph.com/issues/37352

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
(cherry picked from commit 6ecaec926fb81810f6be43744cd5c48d6ccfaf5a)

Conflicts: RGWOp::name() returns std::string
src/rgw/rgw_rest_config.h
src/rgw/rgw_rest_realm.cc

6 years agoMerge pull request #28130 from dillaman/wip-39727-luminous
Yuri Weinstein [Thu, 30 May 2019 20:38:49 +0000 (13:38 -0700)]
Merge pull request #28130 from dillaman/wip-39727-luminous

luminous: qa/workunits/rbd: use https protocol for devstack git operations

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #28183 from yuriw/wip-yuriw-fix-distro-luminous
Yuri Weinstein [Tue, 28 May 2019 22:49:51 +0000 (15:49 -0700)]
Merge pull request #28183 from yuriw/wip-yuriw-fix-distro-luminous

qa/tests: cleaned up supported distro

6 years agoqa/tests: cleaned up supported distro 28183/head
Yuri Weinstein [Mon, 20 May 2019 20:11:37 +0000 (13:11 -0700)]
qa/tests: cleaned up supported distro

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #28117 from yuriw/wip-yuriw-clean-luminous
Yuri Weinstein [Tue, 28 May 2019 21:38:07 +0000 (14:38 -0700)]
Merge pull request #28117 from yuriw/wip-yuriw-clean-luminous

luminous: qa/test: clean up

6 years agoMerge pull request #28216 from smithfarm/wip-40004-luminous
Nathan Cutler [Tue, 28 May 2019 09:41:54 +0000 (11:41 +0200)]
Merge pull request #28216 from smithfarm/wip-40004-luminous

luminous: do_cmake.sh: source not found

Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agodo_cmake.sh: use bash 28216/head
Nathan Cutler [Mon, 20 May 2019 15:01:40 +0000 (17:01 +0200)]
do_cmake.sh: use bash

Fixes: http://tracker.ceph.com/issues/39981
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit ac8d7c5fd4045d79ab6caacc0b72d64866fcc5c8)

6 years agoqa/workunits/rbd: use https protocol for devstack git operations 28130/head
Jason Dillaman [Thu, 9 May 2019 19:48:30 +0000 (15:48 -0400)]
qa/workunits/rbd: use https protocol for devstack git operations

Fixes: http://tracker.ceph.com/issues/39656
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fb4f9a8a08edd18b1a23e1be4c1285d0ec4d1de6)

6 years agoqa/test: clean up 28117/head
Yuri Weinstein [Wed, 15 May 2019 16:34:59 +0000 (09:34 -0700)]
qa/test: clean up

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #28115 from yuriw/wip-yuriw-clients-upgrades-luminous-2-nautilus 28143/head
Yuri Weinstein [Wed, 15 May 2019 16:27:15 +0000 (09:27 -0700)]
Merge pull request #28115 from yuriw/wip-yuriw-clients-upgrades-luminous-2-nautilus

qa/tests: renamed suite

6 years agoqa/tests: renamed suite 28115/head
Yuri Weinstein [Wed, 15 May 2019 16:20:47 +0000 (09:20 -0700)]
qa/tests: renamed suite

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoqa/tests: renamed suite to clients-upgrades-luminous-nautilus to be consistent
Yuri Weinstein [Fri, 10 May 2019 16:19:52 +0000 (09:19 -0700)]
qa/tests: renamed suite to clients-upgrades-luminous-nautilus to be consistent

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #28066 from yuriw/wip-yuriw-clients-upgrades-luminous-2-nautilus
Yuri Weinstein [Fri, 10 May 2019 16:38:53 +0000 (09:38 -0700)]
Merge pull request #28066 from yuriw/wip-yuriw-clients-upgrades-luminous-2-nautilus

qa/tests: renamed suite to clients-upgrades-luminous-nautilus to be c…

6 years agoqa/tests: renamed suite to clients-upgrades-luminous-nautilus to be consistent 28066/head
Yuri Weinstein [Fri, 10 May 2019 16:19:52 +0000 (09:19 -0700)]
qa/tests: renamed suite to clients-upgrades-luminous-nautilus to be consistent

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #27934 from yuriw/wip-yuriw-clients-upgrades-luminous-2-nautilus
Josh Durgin [Thu, 9 May 2019 22:48:17 +0000 (15:48 -0700)]
Merge pull request #27934 from yuriw/wip-yuriw-clients-upgrades-luminous-2-nautilus

qa/tests: new suite to test `luminous` clients on `nautilus` cluster

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoqa/tests: new suite to test `luminous` clients on `nautilus` cluster 27934/head
Yuri Weinstein [Thu, 2 May 2019 20:17:13 +0000 (13:17 -0700)]
qa/tests: new suite to test `luminous` clients on `nautilus` cluster

This suite will run on `nautilus` relases only and we will keep running older suite `client-upgrade-luminous`
on previous releases

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #27910 from yuriw/wip-yuriw-fix-clients-upgrates-luminous 27768/head
Yuri Weinstein [Wed, 1 May 2019 23:38:58 +0000 (16:38 -0700)]
Merge pull request #27910 from yuriw/wip-yuriw-fix-clients-upgrates-luminous

qa/tests:  Excluded  "'python34-cephfs','python34-rados'" for runs on nautilus

6 years agoqa/tests: Excluded "'python34-cephfs','python34-rados'" for runs on nautilus 27910/head
Yuri Weinstein [Wed, 1 May 2019 20:08:38 +0000 (13:08 -0700)]
qa/tests:  Excluded  "'python34-cephfs','python34-rados'" for runs on nautilus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
6 years ago12.2.12 v12.2.12
Jenkins Build Slave User [Thu, 11 Apr 2019 12:33:50 +0000 (12:33 +0000)]
12.2.12

6 years agoMerge pull request #26989 from ceph/backport-luminous-26957
Yuri Weinstein [Mon, 8 Apr 2019 19:44:57 +0000 (12:44 -0700)]
Merge pull request #26989 from ceph/backport-luminous-26957

luminous: ceph-volume: look for rotational data in lsblk

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #27312 from ifed01/wip-ifed-fix-bmap-alloc-luminous 27364/head
Yuri Weinstein [Wed, 3 Apr 2019 20:22:15 +0000 (13:22 -0700)]
Merge pull request #27312 from ifed01/wip-ifed-fix-bmap-alloc-luminous

luminous: os/bluestore: bitmap allocator might fail to return contiguous chunk despite having enough space

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26475 from pdvian/wip-38338-luminous
Yuri Weinstein [Wed, 3 Apr 2019 20:18:40 +0000 (13:18 -0700)]
Merge pull request #26475 from pdvian/wip-38338-luminous

luminous: tests: run-standalone.sh set local library location so mgr can find li…

Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge pull request #27322 from liewegas/wip-valgrind-luminous
Sage Weil [Wed, 3 Apr 2019 14:57:12 +0000 (09:57 -0500)]
Merge pull request #27322 from liewegas/wip-valgrind-luminous

luminous: qa/valgrind.supp: backport valgrind suppressions

6 years agoMerge pull request #26191 from cbodley/wip-38081-luminous
Yuri Weinstein [Tue, 2 Apr 2019 20:04:41 +0000 (13:04 -0700)]
Merge pull request #26191 from cbodley/wip-38081-luminous

luminous: rgw: fetch_remote_obj filters out olh attrs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26549 from cbodley/wip-38412-luminous
Yuri Weinstein [Tue, 2 Apr 2019 20:03:40 +0000 (13:03 -0700)]
Merge pull request #26549 from cbodley/wip-38412-luminous

luminous: rgw: fix rgw_data_sync_info::json_decode()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27295 from theanalyst/luminous-sse-c-backport
Yuri Weinstein [Tue, 2 Apr 2019 20:02:38 +0000 (13:02 -0700)]
Merge pull request #27295 from theanalyst/luminous-sse-c-backport

luminous: sse-c-fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27299 from cbodley/wip-39073
Yuri Weinstein [Tue, 2 Apr 2019 20:01:44 +0000 (13:01 -0700)]
Merge pull request #27299 from cbodley/wip-39073

luminous: rgw multisite: data sync checks empty next_marker for datalog

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26064 from pdvian/wip-37987-luminous
Yuri Weinstein [Tue, 2 Apr 2019 15:59:28 +0000 (08:59 -0700)]
Merge pull request #26064 from pdvian/wip-37987-luminous

luminous: core: Objecter::calc_op_budget: Fix invalid access to extent union member

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoqa/valgrind.supp: backport valgrind suppressions 27322/head
Sage Weil [Tue, 2 Apr 2019 15:57:31 +0000 (10:57 -0500)]
qa/valgrind.supp: backport valgrind suppressions

in the correction location

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #27173 from pdvian/wip-38755-luminous
Yuri Weinstein [Tue, 2 Apr 2019 15:44:44 +0000 (08:44 -0700)]
Merge pull request #27173 from pdvian/wip-38755-luminous

luminous: rgw: ldap: fix LDAPAuthEngine::init() when uri !empty()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27180 from pdvian/wip-38771-luminous
Yuri Weinstein [Tue, 2 Apr 2019 15:44:05 +0000 (08:44 -0700)]
Merge pull request #27180 from pdvian/wip-38771-luminous

luminous : rgw: nfs: skip empty (non-POSIX) path segments

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoos/bluestore: unconditionally cap chunks returned by allocator to 2^31 27312/head
Igor Fedotov [Wed, 20 Mar 2019 18:10:04 +0000 (21:10 +0300)]
os/bluestore: unconditionally cap chunks returned by allocator to 2^31

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 1337443e37ffbfcaf494bdb2bb80db4cf41b97f9)

6 years agoos/bluestore: start using 64-bit intervals for bitmap allocator
Igor Fedotov [Wed, 13 Mar 2019 16:09:40 +0000 (19:09 +0300)]
os/bluestore: start using 64-bit intervals for bitmap allocator

This helps to avoid 4Gb+ values wrapping which resulted in
allocation failures.

Fixes: https://tracker.ceph.com/issues/38761
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit d1466963a60ed2345a6fc159071ec09ce422b656)

6 years agoos/bluestore: make bluestore interval base template.
Igor Fedotov [Wed, 13 Mar 2019 15:51:46 +0000 (18:51 +0300)]
os/bluestore: make bluestore interval base template.

We might need intervals at BlueStore with both 32-bit and 64-bit
lengths. Having base template eliminates the need for copy/paste

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit f775d4227fb9432840c795115669766565c5e718)

6 years agotests/fastbmap_alloc: UT to reproduce 4G allocation bug
Igor Fedotov [Wed, 13 Mar 2019 14:02:13 +0000 (17:02 +0300)]
tests/fastbmap_alloc: UT to reproduce 4G allocation bug

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ed8a93ba387458961444913a8695f538595efacf)

6 years agoos/bluestore: os/bluestore: implement dump for bitmap allocator
Igor Fedotov [Tue, 12 Mar 2019 11:04:17 +0000 (14:04 +0300)]
os/bluestore: os/bluestore: implement dump for bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 80310d98a7456afdb2002eaf2891853ea21eaf68)

6 years agorgw: fixup_range() clamps offsets to valid part range 27295/head
Casey Bodley [Wed, 27 Mar 2019 17:52:59 +0000 (13:52 -0400)]
rgw: fixup_range() clamps offsets to valid part range

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4b3bac20d550cae274c07a1d555751483dde463b)

6 years agotest_rgw_crypto: add invalid ranges beyond obj boundary
Abhishek Lekshmanan [Wed, 27 Mar 2019 12:45:27 +0000 (13:45 +0100)]
test_rgw_crypto: add invalid ranges beyond obj boundary

This is just to assert we're not doing an invalid memory access

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit b34a00de0f6ecea6d97fd80f65d2a2dca0876e6e)

6 years agoMerge pull request #26387 from smithfarm/wip-38188-luminous
Yuri Weinstein [Mon, 1 Apr 2019 21:47:10 +0000 (14:47 -0700)]
Merge pull request #26387 from smithfarm/wip-38188-luminous

luminous: osd/bluestore: deep fsck fails on inspecting very large onodes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #26535 from k0ste/luminous_backports3
Yuri Weinstein [Mon, 1 Apr 2019 21:46:42 +0000 (14:46 -0700)]
Merge pull request #26535 from k0ste/luminous_backports3

luminous: mgr/prometheus: add interface and objectstore to osd metadata

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
6 years agoMerge pull request #26616 from pdvian/wip-38442-luminous
Yuri Weinstein [Mon, 1 Apr 2019 21:21:57 +0000 (14:21 -0700)]
Merge pull request #26616 from pdvian/wip-38442-luminous

luminous: qa/standalone/osd/osd-markdown: disable CLI command dups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #26751 from noahdesu/cherrypy-ipv6-luminous
Yuri Weinstein [Mon, 1 Apr 2019 21:08:49 +0000 (14:08 -0700)]
Merge pull request #26751 from noahdesu/cherrypy-ipv6-luminous

luminous: mgr/dashboard: fix for using '::' on hosts without ipv6

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
6 years agoMerge pull request #26979 from ifed01/wip-ifed-bmap-alloc-luminous
Yuri Weinstein [Mon, 1 Apr 2019 20:40:55 +0000 (13:40 -0700)]
Merge pull request #26979 from ifed01/wip-ifed-bmap-alloc-luminous

luminous: os/bluestore: backport new bitmap allocator

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agorgw: data sync checks empty next_marker for datalog 27299/head
Casey Bodley [Fri, 29 Mar 2019 14:27:56 +0000 (10:27 -0400)]
rgw: data sync checks empty next_marker for datalog

RGWReadRemoteDataLogShardCR tracks the marker and next_marker
separately, because next_marker will be empty when it reaches the end

this allows RGWDataSyncShardCR to avoid clearing its sync_marker and
restarting datalog listing from the beginning

Fixes: http://tracker.ceph.com/issues/39033
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2ff63778342282ab5c676529f6e7d90f30e6da1a)

Conflicts:
src/rgw/rgw_data_sync.cc: no sync tracing

6 years agotests: rgw crypto tests with partlen vector
Abhishek Lekshmanan [Tue, 26 Mar 2019 18:12:15 +0000 (19:12 +0100)]
tests: rgw crypto tests with partlen vector

4 tests are added when either obj_size for an aligned object, and when an object
is not aligned to a boundary (1 B last part), or part_size is not aligned to a
boundary. Comments on boundary case explains the logic on choosing whether to
align to a 4095B boundary or not

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 1bf59f68db608084a1e7a05aca4cca8a8ab76062)
luminous changes: std::make_unique -> ceph::make_unique

6 years agotest_rgw_crypt: BlockCryptNone takes a block_size param
Abhishek Lekshmanan [Tue, 26 Mar 2019 18:11:28 +0000 (19:11 +0100)]
test_rgw_crypt: BlockCryptNone takes a block_size param

So that block sizes like 4096 can be tested

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 9e13a42daaba26751f56f2454f9ec314d7d0d460)

6 years agorgw_crypt: make decrypt's parts_len protected
Abhishek Lekshmanan [Tue, 26 Mar 2019 18:10:04 +0000 (19:10 +0100)]
rgw_crypt: make decrypt's parts_len protected

This is so that testing classes can set this and use that to test the branches
in fixup_range tests

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 78b37a5ccd67c464f3c6e61568e9e827c327ad6d)

6 years agorgw_crypt: take care of start & end boundaries when parsing range
Abhishek Lekshmanan [Mon, 25 Mar 2019 14:24:15 +0000 (15:24 +0100)]
rgw_crypt: take care of start & end boundaries when parsing range

As http range header is inclusive of start and end ranges, this otherwise breaks
at ranges close to multipart boundaries

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 9c92b56d77932283ccc0336645dc3b76bd0fffa2)

6 years agorgw: decrypt filter does not cross multipart boundaries
Casey Bodley [Fri, 22 Mar 2019 13:57:33 +0000 (09:57 -0400)]
rgw: decrypt filter does not cross multipart boundaries

multipart uploads with sse encrypts each part separately, using an
initialization vector based on the part offset

decryption must respect the same part boundaries, and start each part
with a fresh initialization vector. this means that the decrypt filter
must flush data up to part boundaries before starting the next

Fixes: http://tracker.ceph.com/issues/38700
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b782902954abd20e49c0ed1c14cebee4c8ab6f88)
luminous modifications: std::make_unique -> ceph::make_unique

6 years agoMerge pull request #26642 from pdvian/wip-38449-luminous
Yuri Weinstein [Mon, 1 Apr 2019 13:59:14 +0000 (06:59 -0700)]
Merge pull request #26642 from pdvian/wip-38449-luminous

luminous: MDSMonitor: do not assign standby-replay when degraded

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #26776 from pdvian/wip-38541-luminous
Yuri Weinstein [Mon, 1 Apr 2019 13:58:49 +0000 (06:58 -0700)]
Merge pull request #26776 from pdvian/wip-38541-luminous

luminous : qa: bump fsstress timeout to 6h

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #26805 from pdvian/wip-38543-luminous
Yuri Weinstein [Mon, 1 Apr 2019 13:58:18 +0000 (06:58 -0700)]
Merge pull request #26805 from pdvian/wip-38543-luminous

luminous: qa: unmount clients before deleting fs

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #26820 from smithfarm/wip-38545-luminous
Yuri Weinstein [Mon, 1 Apr 2019 13:57:38 +0000 (06:57 -0700)]
Merge pull request #26820 from smithfarm/wip-38545-luminous

luminous: qa: Loading libcephfs-jni: Failure!

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #26964 from ashishkumsingh/wip-38735-luminous
Yuri Weinstein [Mon, 1 Apr 2019 13:57:14 +0000 (06:57 -0700)]
Merge pull request #26964 from ashishkumsingh/wip-38735-luminous

luminous: qa: extend MDS heartbeat grace for valgrind

Reviewed-by: Venky Shankar <vshankar@redhat.com>
6 years agoMerge pull request #27024 from pdvian/wip-38669-luminous
Yuri Weinstein [Mon, 1 Apr 2019 13:56:42 +0000 (06:56 -0700)]
Merge pull request #27024 from pdvian/wip-38669-luminous

luminous: mds: wait for client to release shared cap when re-acquiring xlock

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
6 years agoMerge pull request #27207 from pdvian/wip-38854-luminous
Yuri Weinstein [Fri, 29 Mar 2019 19:45:31 +0000 (12:45 -0700)]
Merge pull request #27207 from pdvian/wip-38854-luminous

luminous: mon/MgrStatMonitor: ensure only one copy of initial service map

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #27224 from xiexingguo/wip-luminous-upmap-backports
Yuri Weinstein [Fri, 29 Mar 2019 19:44:55 +0000 (12:44 -0700)]
Merge pull request #27224 from xiexingguo/wip-luminous-upmap-backports

luminous: osd: backport recent upmap fixes

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #27226 from pdvian/wip-38857-luminous
Yuri Weinstein [Fri, 29 Mar 2019 19:44:25 +0000 (12:44 -0700)]
Merge pull request #27226 from pdvian/wip-38857-luminous

luminous: should set EPOLLET flag on del_event()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoqa/suites: fix bluestore links 26979/head
Sage Weil [Wed, 5 Dec 2018 16:27:53 +0000 (10:27 -0600)]
qa/suites: fix bluestore links

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 269910fc8b2bdf64b98534ce525318e108aa9409)

6 years agoMerge pull request #26358 from smithfarm/wip-37557-luminous
Yuri Weinstein [Thu, 28 Mar 2019 15:45:34 +0000 (08:45 -0700)]
Merge pull request #26358 from smithfarm/wip-37557-luminous

luminous: multisite: es sync null versioned object failed because of olh info

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26530 from pdvian/wip-38354-luminous
Yuri Weinstein [Thu, 28 Mar 2019 15:44:28 +0000 (08:44 -0700)]
Merge pull request #26530 from pdvian/wip-38354-luminous

luminous: rgw: check for non-existent bucket in RGWGetACLs

Reviewed-by: Adam Emerson <aemerson@redhat.com>
6 years agoMerge pull request #26619 from pdvian/wip-38446-luminous
Yuri Weinstein [Thu, 28 Mar 2019 15:43:58 +0000 (08:43 -0700)]
Merge pull request #26619 from pdvian/wip-38446-luminous

luminous: rgw: only update last_trim marker on ENODATA

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26761 from pdvian/wip-38529-luminous
Yuri Weinstein [Thu, 28 Mar 2019 15:43:36 +0000 (08:43 -0700)]
Merge pull request #26761 from pdvian/wip-38529-luminous

luminous: rgw: data sync drains lease stack on lease failure

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26946 from ashishkumsingh/wip-38727-luminous
Yuri Weinstein [Thu, 28 Mar 2019 15:43:09 +0000 (08:43 -0700)]
Merge pull request #26946 from ashishkumsingh/wip-38727-luminous

luminous: rgw: bucket limit check misbehaves for > max-entries buckets (usually 1000)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #27030 from pdvian/wip-38671-luminous
Yuri Weinstein [Thu, 28 Mar 2019 15:42:46 +0000 (08:42 -0700)]
Merge pull request #27030 from pdvian/wip-38671-luminous

luminous: rgw: sync module: avoid printing attrs of objects in log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26516 from yehudasa/wip-rgw-es-fixes-luminous
Yuri Weinstein [Thu, 28 Mar 2019 15:14:03 +0000 (08:14 -0700)]
Merge pull request #26516 from yehudasa/wip-rgw-es-fixes-luminous

 luminous: rgw: multiple es related fixes and improvements

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
6 years agoosd/OSDMap: add log for better debugging 27224/head
xie xingguo [Mon, 25 Mar 2019 10:24:16 +0000 (18:24 +0800)]
osd/OSDMap: add log for better debugging

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit a89281ffbb50a4dfc700398e864138b5faaf00f5)

6 years agoosd/OSDMap: calc_pg_upmaps - restrict optimization to origin pools only
xie xingguo [Sat, 23 Mar 2019 01:50:27 +0000 (09:50 +0800)]
osd/OSDMap: calc_pg_upmaps - restrict optimization to origin pools only

The current implementation will try to cancel any pg_upmaps that
would otherwise re-map a PG out from an underfull osd, which is wrong,
e.g., because it could reliably fire the following assert:

src/osd/OSDMap.cc: 4405: FAILED assert(osd_weight.count(i.first))

Also it would not match the expectation if automatic balancing
has been strictly restricted to some specific pools by admin.

Fix by excluding any wild PG that does not belong to the origin pools
passed in when trying to do upmap/unmap.

Fixes: http://tracker.ceph.com/issues/38897
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 01e8e9482ce7194d347e02ef41acfa6d8d14f614)

6 years agoosd/OSDMap: drop local pool filter in calc_pg_upmaps
xie xingguo [Sat, 23 Feb 2019 00:33:40 +0000 (08:33 +0800)]
osd/OSDMap: drop local pool filter in calc_pg_upmaps

The local pre-loaded pool filter is completely redundant since
the below check:

if (!only_pools.empty() && !only_pools.count(i.first))

could reliably catch both cases - either optimization should be
restricted to specific pools feeded, or all existing pools.

Let's clean it up.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3e6bad9821b5fb3e780d970666fbdfbf217d905e)

6 years agocrush: add root_bucket to identify underfull buckets
huangjun [Wed, 20 Mar 2019 08:44:02 +0000 (16:44 +0800)]
crush: add root_bucket to identify underfull buckets

All underfull buckets under root_buckets will be taken as target

For the crule rule:
    step take datacenter0
    step chooseleaf firstn 2 type host
    step emit
    step take datacenter1
    step chooseleaf firstn 2 type host
    step emit

If one host contains overfull osd but no underfull osd,
it will use other underfull buckets as target, which
maybe not in the same datacenter, that will
broke the rule.

Fixes: http://tracker.ceph.com/issues/38826
Signed-off-by: huangjun <huangjun@xsky.com>
(cherry picked from commit 3d5678d3561d90a10d9de3cb6e7e0405dbe8fdfe)

6 years agoMerge pull request #27104 from rzarzynski/wip-nss-importsymkey-in-fips-luminous
Neha Ojha [Thu, 28 Mar 2019 01:20:36 +0000 (18:20 -0700)]
Merge pull request #27104 from rzarzynski/wip-nss-importsymkey-in-fips-luminous

luminous: crypto: don't use PK11_ImportSymKey() in FIPS mode

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoEventEpoll: refactor del_event() a bit 27226/head
Roman Penyaev [Tue, 19 Mar 2019 10:55:52 +0000 (11:55 +0100)]
EventEpoll: refactor del_event() a bit

The main purpose of the patch is to set 'ee' members only on EPOLL_CTL_MOD
path, deletion path does not requie any of the members set.

This will make code a bit better.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
(cherry picked from commit 2292120ffd2730a20fd2750a605952982f031c83)

6 years agoEventEpoll: set EPOLLET flag on del_event()
Roman Penyaev [Tue, 5 Mar 2019 12:22:59 +0000 (13:22 +0100)]
EventEpoll: set EPOLLET flag on del_event()

When new fd is being added to epoll fd EPOLLET (edge triggered) flag
is set, i.e. we assume that in and out works in edge triggered mode.

But on del_event(), when eventually epoll_ctl(EPOLL_CTL_MOD) is
invoked, EPOLLET flag is cleared, i.e. level triggered mode is
enabled.

Do not lose EPOLLET.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
(cherry picked from commit c415b9fed4fbecd2fb0709fa0c0c783102f60e01)

6 years agoauth, rgw, common: switch to PK11_ImportSymKey_FIPS wrapper. 27104/head
Radoslaw Zarzynski [Tue, 12 Mar 2019 14:17:43 +0000 (15:17 +0100)]
auth, rgw, common: switch to PK11_ImportSymKey_FIPS wrapper.

PK11_ImportSymKey() is a part of NSS API that becomes unavailable
in the FIPS mode. Apparently NSS targets stricter restrictions
than those coming from Level 1 of FIPS 140-2. In the consequence,
loading a symmetric key from plain keyring or key db (which Ceph
needs to do due to architectural reasons) fails.

The same issue affected corosync and this patchset deals with it
the same way like already followed by corosync [1]:
a raw crypto key is in-memory wrapped with fresh, random wrapping
key just before being imported via PK11_UnwrapSymKey(). Of course,
this effectively lowers to FIPS level 1. Still, this would be no
different from what OpenSSL (to which we are currently migrating
in master) gives in the matter.

The patch can be *roughly* verified in following steps:
  1. mkdir ./nssdb
  2. certutil -N -d ./nssdb --empty-password
  3. modutil -dbdir ./nssdb -fips true
  4. ../src/vstart.sh -l -n -b -o "nss_db_path=/work/ceph-3/build/nssdb"

This fix is dedicated to Luminous. In master we're switching to OpenSSL.

[1] 5dadebd21862074deaeb9a337fc9e49f5e9f692a in corosync's public repo.

Fixes: http://tracker.ceph.com/issues/38843
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agocrypto: bring PK11_ImportSymKey_FIPS doing PK11_UnwrapSymKey.
Radoslaw Zarzynski [Tue, 12 Mar 2019 01:23:53 +0000 (02:23 +0100)]
crypto: bring PK11_ImportSymKey_FIPS doing PK11_UnwrapSymKey.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agomon/MgrStatMonitor: ensure only one copy of initial service map 27207/head
Sage Weil [Thu, 21 Mar 2019 16:58:29 +0000 (11:58 -0500)]
mon/MgrStatMonitor: ensure only one copy of initial service map

It's possible for create_pending to be called multiple times.

Fixes: http://tracker.ceph.com/issues/38839
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 204a8deacd2068989bd52eb130f4aac434b45f35)

Conflicts:
src/mon/MgrStatMonitor.cc : Resolved in create_initial

6 years agoMerge pull request #26758 from pdvian/wip-38510-luminous
Yuri Weinstein [Tue, 26 Mar 2019 16:56:26 +0000 (09:56 -0700)]
Merge pull request #26758 from pdvian/wip-38510-luminous

luminous: CLI: ability to change file ownership

6 years agoMerge pull request #26830 from pdvian/wip-38562-luminous
Yuri Weinstein [Tue, 26 Mar 2019 16:49:48 +0000 (09:49 -0700)]
Merge pull request #26830 from pdvian/wip-38562-luminous

luminous: mgr/BaseMgrModule: drop GIL for ceph_send_command

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26900 from pdvian/wip-38586-luminous
Yuri Weinstein [Tue, 26 Mar 2019 16:48:50 +0000 (09:48 -0700)]
Merge pull request #26900 from pdvian/wip-38586-luminous

luminous: OSD crashes in get_str_map while creating with ceph-volume

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26962 from pdvian/wip-38665-luminous
Yuri Weinstein [Tue, 26 Mar 2019 16:48:21 +0000 (09:48 -0700)]
Merge pull request #26962 from pdvian/wip-38665-luminous

luminous: qa: ignore slow metadata io wrn during osd thrash

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agorgw: nfs: skip empty (non-POSIX) path segments 27180/head
Matt Benjamin [Thu, 14 Mar 2019 17:58:17 +0000 (13:58 -0400)]
rgw: nfs: skip empty (non-POSIX) path segments

Such a path could be created through S3 upload,
e.g., s3://my_files//data/file.pdf.

Previously we asserted for this case, but it's harmless--such a
path segment should just be ignored.

Fixes: http://tracker.ceph.com/issues/38744
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit a53e6c1d5b785a2a46007292041d0a8cb552e3e6)

Conflicts:
src/rgw/rgw_file.h : Resolved in operator()

6 years agorgw: ldap: fix LDAPAuthEngine::init() when uri !empty() 27173/head
Matt Benjamin [Tue, 12 Mar 2019 12:58:53 +0000 (08:58 -0400)]
rgw: ldap: fix LDAPAuthEngine::init() when uri !empty()

Fixes: https://tracker.ceph.com/issues/38699
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 6ef98c6e0fcf4f9b6e431b3409975e0966c5c21a)

6 years agoMerge pull request #27033 from cbodley/wip-38796
Nathan Cutler [Fri, 22 Mar 2019 13:22:02 +0000 (14:22 +0100)]
Merge pull request #27033 from cbodley/wip-38796

luminous: doc/rgw: document placement target configuration

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #26557 from dzafman/wip-38244
David Zafman [Tue, 19 Mar 2019 23:59:39 +0000 (16:59 -0700)]
Merge pull request #26557 from dzafman/wip-38244

luminous: scrub warning check incorrectly uses mon scrub interval

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
6 years agoMerge pull request #26793 from dzafman/wip-38274
David Zafman [Tue, 19 Mar 2019 23:48:17 +0000 (16:48 -0700)]
Merge pull request #26793 from dzafman/wip-38274

luminous: Fix recovery and backfill priority handling

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26544 from ifed01/wip-ifed-fix-compress-luminous
Yuri Weinstein [Mon, 18 Mar 2019 18:59:14 +0000 (11:59 -0700)]
Merge pull request #26544 from ifed01/wip-ifed-fix-compress-luminous

luminous: os/bluestore: do not assert on non-zero err codes from compress() call

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agodoc/rgw: document placement target configuration 27033/head
Casey Bodley [Fri, 15 Mar 2019 18:35:13 +0000 (14:35 -0400)]
doc/rgw: document placement target configuration

Fixes: http://tracker.ceph.com/issues/24508
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c95585b1984e60708addee436a774d0c92ce57a6)

6 years agorgw: sync module: avoid printing attrs of objects in log 27030/head
Abhishek Lekshmanan [Thu, 13 Dec 2018 14:57:40 +0000 (15:57 +0100)]
rgw: sync module: avoid printing attrs of objects in log

Since attrs are verbose and not necessarily needed as a debug info (as they can
be obtained from the required object itself) avoid logging them. Also
additionally removed another place where such a log is done by the ES sync
module

Fixes: http://tracker.ceph.com/issues/37646
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 8f37b70630ceb62aff4019e9ad2508977590a873)

Conflicts:
src/rgw/rgw_sync_module.cc : Resolved in RGWCallStatRemoteObjCR::operate

6 years agomds: wait for client to release shared cap when re-acquiring xlock 27024/head
Yan, Zheng [Thu, 28 Feb 2019 13:36:28 +0000 (21:36 +0800)]
mds: wait for client to release shared cap when re-acquiring xlock

MDS may revoke shared cap from xlocker client during 'LOCK_XLOCKDONE ->
LOCK_LOCK_XLOCK' state transition.

Fixes: http://tracker.ceph.com/issues/38491
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 218e784f2eb1eec6bbb7910b5432f7f36263dbe7)

Conflicts:
src/mds/SimpleLock.h : Resovled in put_xlock

6 years agoqa/objectstore: bluestore -> bluestore-{bitmap,stupid}
Sage Weil [Fri, 30 Nov 2018 15:55:38 +0000 (09:55 -0600)]
qa/objectstore: bluestore -> bluestore-{bitmap,stupid}

The new default is bitmap, so we were testing bitmap twice.  Instead,
explicitly call out stupid and bitmap cases so a future default change
won't break coverage.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f390704b4616c837352e7d6a8df91526cb047f0d)

6 years agoos/bluestore: fix minor issues in bitmap logging output
Igor Fedotov [Mon, 25 Jun 2018 14:49:49 +0000 (17:49 +0300)]
os/bluestore: fix minor issues in bitmap logging output

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit dc4b5fc5afcea4be9e3b4c91dcc9dcce38d4e906)

6 years agoos/bluestore: fix overlappedd allocation returned by bitmap allocator
Igor Fedotov [Mon, 25 Jun 2018 14:42:41 +0000 (17:42 +0300)]
os/bluestore: fix overlappedd allocation returned by bitmap allocator

Fixes: https://tracker.ceph.com/issues/24598
Signed-off-by: Igor Fedotov <ifedotov@suse.som>
(cherry picked from commit 539af4f27e773cb21d187b75086c7c9498dd7749)

6 years agoos/bluestore: provide a rough estimate for bitmap allocator
Igor Fedotov [Fri, 15 Jun 2018 14:04:57 +0000 (17:04 +0300)]
os/bluestore: provide a rough estimate for bitmap allocator
fragmentation.
The approach is counting 'partial' and 'free' slots at L1 on the fly and
use partial / (partial+free) value as rough fragmentation estimate.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit dbbe12fe6f543fc6942f41a108effc7eb1632f67)

6 years agoos/bluestore: rename new bitmap allocator class to BitmapAllocator.
Igor Fedotov [Tue, 12 Jun 2018 11:50:30 +0000 (14:50 +0300)]
os/bluestore: rename new bitmap allocator class to BitmapAllocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit a8e8c192acb48ce93553b04d1953ce8bee1b16e4)

6 years agoos/bluestore: perform allocations aligned with min_length in new bitmap allocator
Igor Fedotov [Fri, 1 Jun 2018 18:59:15 +0000 (21:59 +0300)]
os/bluestore: perform allocations aligned with min_length  in new bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 51974c48e38e7080eb12c6028f95f4da666fbe76)

6 years agotest/objectstore/unitetest_fastbmap_allocator: replace ASSERT_TRUE with
Igor Fedotov [Fri, 1 Jun 2018 13:35:04 +0000 (16:35 +0300)]
test/objectstore/unitetest_fastbmap_allocator: replace ASSERT_TRUE with
ASSERT_EQ

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 69ccb4367dcdcc51fbcfdaaea308c761291621e1)

6 years agoos/bluestore: respect min_length as allocation granularity for new
Igor Fedotov [Thu, 31 May 2018 13:53:31 +0000 (16:53 +0300)]
os/bluestore: respect min_length as allocation granularity for new
bitmap allocator.
It was used a real minimum threshold before this fix which allowed e.g.
allocated extent length to be equal to min_length + 1.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 8a188b2aa3892b7f5a6c85042ffae8c2847ca869)

6 years agoos/bluestore: cosmetic new allocator internal method rename.
Igor Fedotov [Thu, 31 May 2018 11:28:28 +0000 (14:28 +0300)]
os/bluestore: cosmetic new allocator internal method rename.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit fa06ff234f11f266bb10d7fa9ea28847e5ce72fe)

6 years agoos/bluestore: properly respect min_length when analysing partially free
Igor Fedotov [Thu, 31 May 2018 10:53:37 +0000 (13:53 +0300)]
os/bluestore: properly respect min_length when analysing partially free
slots in bitmap allocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 288bbd42531bebd8f6c619c848e75bfd9b8f93fd)