Jenkins Build Slave User [Thu, 11 Apr 2019 12:33:50 +0000 (12:33 +0000)]
12.2.12
Yuri Weinstein [Mon, 8 Apr 2019 19:44:57 +0000 (12:44 -0700)]
Merge pull request #26989 from ceph/backport-luminous-26957
luminous: ceph-volume: look for rotational data in lsblk
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Yuri Weinstein [Wed, 3 Apr 2019 20:22:15 +0000 (13:22 -0700)]
Merge pull request #27312 from ifed01/wip-ifed-fix-bmap-alloc-luminous
luminous: os/bluestore: bitmap allocator might fail to return contiguous chunk despite having enough space
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Wed, 3 Apr 2019 20:18:40 +0000 (13:18 -0700)]
Merge pull request #26475 from pdvian/wip-38338-luminous
luminous: tests: run-standalone.sh set local library location so mgr can find li…
Reviewed-by: David Zafman <dzafman@redhat.com>
Sage Weil [Wed, 3 Apr 2019 14:57:12 +0000 (09:57 -0500)]
Merge pull request #27322 from liewegas/wip-valgrind-luminous
luminous: qa/valgrind.supp: backport valgrind suppressions
Yuri Weinstein [Tue, 2 Apr 2019 20:04:41 +0000 (13:04 -0700)]
Merge pull request #26191 from cbodley/wip-38081-luminous
luminous: rgw: fetch_remote_obj filters out olh attrs
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 2 Apr 2019 20:03:40 +0000 (13:03 -0700)]
Merge pull request #26549 from cbodley/wip-38412-luminous
luminous: rgw: fix rgw_data_sync_info::json_decode()
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 2 Apr 2019 20:02:38 +0000 (13:02 -0700)]
Merge pull request #27295 from theanalyst/luminous-sse-c-backport
luminous: sse-c-fixes
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 2 Apr 2019 20:01:44 +0000 (13:01 -0700)]
Merge pull request #27299 from cbodley/wip-39073
luminous: rgw multisite: data sync checks empty next_marker for datalog
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 2 Apr 2019 15:59:28 +0000 (08:59 -0700)]
Merge pull request #26064 from pdvian/wip-37987-luminous
luminous: core: Objecter::calc_op_budget: Fix invalid access to extent union member
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Sage Weil [Tue, 2 Apr 2019 15:57:31 +0000 (10:57 -0500)]
qa/valgrind.supp: backport valgrind suppressions
in the correction location
Signed-off-by: Sage Weil <sage@redhat.com>
Yuri Weinstein [Tue, 2 Apr 2019 15:44:44 +0000 (08:44 -0700)]
Merge pull request #27173 from pdvian/wip-38755-luminous
luminous: rgw: ldap: fix LDAPAuthEngine::init() when uri !empty()
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Tue, 2 Apr 2019 15:44:05 +0000 (08:44 -0700)]
Merge pull request #27180 from pdvian/wip-38771-luminous
luminous : rgw: nfs: skip empty (non-POSIX) path segments
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Igor Fedotov [Wed, 20 Mar 2019 18:10:04 +0000 (21:10 +0300)]
os/bluestore: unconditionally cap chunks returned by allocator to 2^31
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
1337443e37ffbfcaf494bdb2bb80db4cf41b97f9)
Igor Fedotov [Wed, 13 Mar 2019 16:09:40 +0000 (19:09 +0300)]
os/bluestore: start using 64-bit intervals for bitmap allocator
This helps to avoid 4Gb+ values wrapping which resulted in
allocation failures.
Fixes: https://tracker.ceph.com/issues/38761
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
d1466963a60ed2345a6fc159071ec09ce422b656)
Igor Fedotov [Wed, 13 Mar 2019 15:51:46 +0000 (18:51 +0300)]
os/bluestore: make bluestore interval base template.
We might need intervals at BlueStore with both 32-bit and 64-bit
lengths. Having base template eliminates the need for copy/paste
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
f775d4227fb9432840c795115669766565c5e718)
Igor Fedotov [Wed, 13 Mar 2019 14:02:13 +0000 (17:02 +0300)]
tests/fastbmap_alloc: UT to reproduce 4G allocation bug
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
ed8a93ba387458961444913a8695f538595efacf)
Igor Fedotov [Tue, 12 Mar 2019 11:04:17 +0000 (14:04 +0300)]
os/bluestore: os/bluestore: implement dump for bitmap allocator
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
80310d98a7456afdb2002eaf2891853ea21eaf68)
Casey Bodley [Wed, 27 Mar 2019 17:52:59 +0000 (13:52 -0400)]
rgw: fixup_range() clamps offsets to valid part range
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
4b3bac20d550cae274c07a1d555751483dde463b)
Abhishek Lekshmanan [Wed, 27 Mar 2019 12:45:27 +0000 (13:45 +0100)]
test_rgw_crypto: add invalid ranges beyond obj boundary
This is just to assert we're not doing an invalid memory access
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit
b34a00de0f6ecea6d97fd80f65d2a2dca0876e6e)
Yuri Weinstein [Mon, 1 Apr 2019 21:47:10 +0000 (14:47 -0700)]
Merge pull request #26387 from smithfarm/wip-38188-luminous
luminous: osd/bluestore: deep fsck fails on inspecting very large onodes
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Yuri Weinstein [Mon, 1 Apr 2019 21:46:42 +0000 (14:46 -0700)]
Merge pull request #26535 from k0ste/luminous_backports3
luminous: mgr/prometheus: add interface and objectstore to osd metadata
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Yuri Weinstein [Mon, 1 Apr 2019 21:21:57 +0000 (14:21 -0700)]
Merge pull request #26616 from pdvian/wip-38442-luminous
luminous: qa/standalone/osd/osd-markdown: disable CLI command dups
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Yuri Weinstein [Mon, 1 Apr 2019 21:08:49 +0000 (14:08 -0700)]
Merge pull request #26751 from noahdesu/cherrypy-ipv6-luminous
luminous: mgr/dashboard: fix for using '::' on hosts without ipv6
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Yuri Weinstein [Mon, 1 Apr 2019 20:40:55 +0000 (13:40 -0700)]
Merge pull request #26979 from ifed01/wip-ifed-bmap-alloc-luminous
luminous: os/bluestore: backport new bitmap allocator
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Casey Bodley [Fri, 29 Mar 2019 14:27:56 +0000 (10:27 -0400)]
rgw: data sync checks empty next_marker for datalog
RGWReadRemoteDataLogShardCR tracks the marker and next_marker
separately, because next_marker will be empty when it reaches the end
this allows RGWDataSyncShardCR to avoid clearing its sync_marker and
restarting datalog listing from the beginning
Fixes: http://tracker.ceph.com/issues/39033
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
2ff63778342282ab5c676529f6e7d90f30e6da1a)
Conflicts:
src/rgw/rgw_data_sync.cc: no sync tracing
Abhishek Lekshmanan [Tue, 26 Mar 2019 18:12:15 +0000 (19:12 +0100)]
tests: rgw crypto tests with partlen vector
4 tests are added when either obj_size for an aligned object, and when an object
is not aligned to a boundary (1 B last part), or part_size is not aligned to a
boundary. Comments on boundary case explains the logic on choosing whether to
align to a 4095B boundary or not
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit
1bf59f68db608084a1e7a05aca4cca8a8ab76062)
luminous changes: std::make_unique -> ceph::make_unique
Abhishek Lekshmanan [Tue, 26 Mar 2019 18:11:28 +0000 (19:11 +0100)]
test_rgw_crypt: BlockCryptNone takes a block_size param
So that block sizes like 4096 can be tested
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit
9e13a42daaba26751f56f2454f9ec314d7d0d460)
Abhishek Lekshmanan [Tue, 26 Mar 2019 18:10:04 +0000 (19:10 +0100)]
rgw_crypt: make decrypt's parts_len protected
This is so that testing classes can set this and use that to test the branches
in fixup_range tests
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit
78b37a5ccd67c464f3c6e61568e9e827c327ad6d)
Abhishek Lekshmanan [Mon, 25 Mar 2019 14:24:15 +0000 (15:24 +0100)]
rgw_crypt: take care of start & end boundaries when parsing range
As http range header is inclusive of start and end ranges, this otherwise breaks
at ranges close to multipart boundaries
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit
9c92b56d77932283ccc0336645dc3b76bd0fffa2)
Casey Bodley [Fri, 22 Mar 2019 13:57:33 +0000 (09:57 -0400)]
rgw: decrypt filter does not cross multipart boundaries
multipart uploads with sse encrypts each part separately, using an
initialization vector based on the part offset
decryption must respect the same part boundaries, and start each part
with a fresh initialization vector. this means that the decrypt filter
must flush data up to part boundaries before starting the next
Fixes: http://tracker.ceph.com/issues/38700
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
b782902954abd20e49c0ed1c14cebee4c8ab6f88)
luminous modifications: std::make_unique -> ceph::make_unique
Yuri Weinstein [Mon, 1 Apr 2019 13:59:14 +0000 (06:59 -0700)]
Merge pull request #26642 from pdvian/wip-38449-luminous
luminous: MDSMonitor: do not assign standby-replay when degraded
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 1 Apr 2019 13:58:49 +0000 (06:58 -0700)]
Merge pull request #26776 from pdvian/wip-38541-luminous
luminous : qa: bump fsstress timeout to 6h
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 1 Apr 2019 13:58:18 +0000 (06:58 -0700)]
Merge pull request #26805 from pdvian/wip-38543-luminous
luminous: qa: unmount clients before deleting fs
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 1 Apr 2019 13:57:38 +0000 (06:57 -0700)]
Merge pull request #26820 from smithfarm/wip-38545-luminous
luminous: qa: Loading libcephfs-jni: Failure!
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 1 Apr 2019 13:57:14 +0000 (06:57 -0700)]
Merge pull request #26964 from ashishkumsingh/wip-38735-luminous
luminous: qa: extend MDS heartbeat grace for valgrind
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Yuri Weinstein [Mon, 1 Apr 2019 13:56:42 +0000 (06:56 -0700)]
Merge pull request #27024 from pdvian/wip-38669-luminous
luminous: mds: wait for client to release shared cap when re-acquiring xlock
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Yuri Weinstein [Fri, 29 Mar 2019 19:45:31 +0000 (12:45 -0700)]
Merge pull request #27207 from pdvian/wip-38854-luminous
luminous: mon/MgrStatMonitor: ensure only one copy of initial service map
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Fri, 29 Mar 2019 19:44:55 +0000 (12:44 -0700)]
Merge pull request #27224 from xiexingguo/wip-luminous-upmap-backports
luminous: osd: backport recent upmap fixes
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Fri, 29 Mar 2019 19:44:25 +0000 (12:44 -0700)]
Merge pull request #27226 from pdvian/wip-38857-luminous
luminous: should set EPOLLET flag on del_event()
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Sage Weil [Wed, 5 Dec 2018 16:27:53 +0000 (10:27 -0600)]
qa/suites: fix bluestore links
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
269910fc8b2bdf64b98534ce525318e108aa9409)
Yuri Weinstein [Thu, 28 Mar 2019 15:45:34 +0000 (08:45 -0700)]
Merge pull request #26358 from smithfarm/wip-37557-luminous
luminous: multisite: es sync null versioned object failed because of olh info
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 28 Mar 2019 15:44:28 +0000 (08:44 -0700)]
Merge pull request #26530 from pdvian/wip-38354-luminous
luminous: rgw: check for non-existent bucket in RGWGetACLs
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Yuri Weinstein [Thu, 28 Mar 2019 15:43:58 +0000 (08:43 -0700)]
Merge pull request #26619 from pdvian/wip-38446-luminous
luminous: rgw: only update last_trim marker on ENODATA
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 28 Mar 2019 15:43:36 +0000 (08:43 -0700)]
Merge pull request #26761 from pdvian/wip-38529-luminous
luminous: rgw: data sync drains lease stack on lease failure
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 28 Mar 2019 15:43:09 +0000 (08:43 -0700)]
Merge pull request #26946 from ashishkumsingh/wip-38727-luminous
luminous: rgw: bucket limit check misbehaves for > max-entries buckets (usually 1000)
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 28 Mar 2019 15:42:46 +0000 (08:42 -0700)]
Merge pull request #27030 from pdvian/wip-38671-luminous
luminous: rgw: sync module: avoid printing attrs of objects in log
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Yuri Weinstein [Thu, 28 Mar 2019 15:14:03 +0000 (08:14 -0700)]
Merge pull request #26516 from yehudasa/wip-rgw-es-fixes-luminous
luminous: rgw: multiple es related fixes and improvements
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
xie xingguo [Mon, 25 Mar 2019 10:24:16 +0000 (18:24 +0800)]
osd/OSDMap: add log for better debugging
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit
a89281ffbb50a4dfc700398e864138b5faaf00f5)
xie xingguo [Sat, 23 Mar 2019 01:50:27 +0000 (09:50 +0800)]
osd/OSDMap: calc_pg_upmaps - restrict optimization to origin pools only
The current implementation will try to cancel any pg_upmaps that
would otherwise re-map a PG out from an underfull osd, which is wrong,
e.g., because it could reliably fire the following assert:
src/osd/OSDMap.cc: 4405: FAILED assert(osd_weight.count(i.first))
Also it would not match the expectation if automatic balancing
has been strictly restricted to some specific pools by admin.
Fix by excluding any wild PG that does not belong to the origin pools
passed in when trying to do upmap/unmap.
Fixes: http://tracker.ceph.com/issues/38897
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit
01e8e9482ce7194d347e02ef41acfa6d8d14f614)
xie xingguo [Sat, 23 Feb 2019 00:33:40 +0000 (08:33 +0800)]
osd/OSDMap: drop local pool filter in calc_pg_upmaps
The local pre-loaded pool filter is completely redundant since
the below check:
if (!only_pools.empty() && !only_pools.count(i.first))
could reliably catch both cases - either optimization should be
restricted to specific pools feeded, or all existing pools.
Let's clean it up.
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit
3e6bad9821b5fb3e780d970666fbdfbf217d905e)
huangjun [Wed, 20 Mar 2019 08:44:02 +0000 (16:44 +0800)]
crush: add root_bucket to identify underfull buckets
All underfull buckets under root_buckets will be taken as target
For the crule rule:
step take datacenter0
step chooseleaf firstn 2 type host
step emit
step take datacenter1
step chooseleaf firstn 2 type host
step emit
If one host contains overfull osd but no underfull osd,
it will use other underfull buckets as target, which
maybe not in the same datacenter, that will
broke the rule.
Fixes: http://tracker.ceph.com/issues/38826
Signed-off-by: huangjun <huangjun@xsky.com>
(cherry picked from commit
3d5678d3561d90a10d9de3cb6e7e0405dbe8fdfe)
Neha Ojha [Thu, 28 Mar 2019 01:20:36 +0000 (18:20 -0700)]
Merge pull request #27104 from rzarzynski/wip-nss-importsymkey-in-fips-luminous
luminous: crypto: don't use PK11_ImportSymKey() in FIPS mode
Reviewed-by: Neha Ojha <nojha@redhat.com>
Roman Penyaev [Tue, 19 Mar 2019 10:55:52 +0000 (11:55 +0100)]
EventEpoll: refactor del_event() a bit
The main purpose of the patch is to set 'ee' members only on EPOLL_CTL_MOD
path, deletion path does not requie any of the members set.
This will make code a bit better.
Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
(cherry picked from commit
2292120ffd2730a20fd2750a605952982f031c83)
Roman Penyaev [Tue, 5 Mar 2019 12:22:59 +0000 (13:22 +0100)]
EventEpoll: set EPOLLET flag on del_event()
When new fd is being added to epoll fd EPOLLET (edge triggered) flag
is set, i.e. we assume that in and out works in edge triggered mode.
But on del_event(), when eventually epoll_ctl(EPOLL_CTL_MOD) is
invoked, EPOLLET flag is cleared, i.e. level triggered mode is
enabled.
Do not lose EPOLLET.
Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
(cherry picked from commit
c415b9fed4fbecd2fb0709fa0c0c783102f60e01)
Radoslaw Zarzynski [Tue, 12 Mar 2019 14:17:43 +0000 (15:17 +0100)]
auth, rgw, common: switch to PK11_ImportSymKey_FIPS wrapper.
PK11_ImportSymKey() is a part of NSS API that becomes unavailable
in the FIPS mode. Apparently NSS targets stricter restrictions
than those coming from Level 1 of FIPS 140-2. In the consequence,
loading a symmetric key from plain keyring or key db (which Ceph
needs to do due to architectural reasons) fails.
The same issue affected corosync and this patchset deals with it
the same way like already followed by corosync [1]:
a raw crypto key is in-memory wrapped with fresh, random wrapping
key just before being imported via PK11_UnwrapSymKey(). Of course,
this effectively lowers to FIPS level 1. Still, this would be no
different from what OpenSSL (to which we are currently migrating
in master) gives in the matter.
The patch can be *roughly* verified in following steps:
1. mkdir ./nssdb
2. certutil -N -d ./nssdb --empty-password
3. modutil -dbdir ./nssdb -fips true
4. ../src/vstart.sh -l -n -b -o "nss_db_path=/work/ceph-3/build/nssdb"
This fix is dedicated to Luminous. In master we're switching to OpenSSL.
[1]
5dadebd21862074deaeb9a337fc9e49f5e9f692a in corosync's public repo.
Fixes: http://tracker.ceph.com/issues/38843
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Radoslaw Zarzynski [Tue, 12 Mar 2019 01:23:53 +0000 (02:23 +0100)]
crypto: bring PK11_ImportSymKey_FIPS doing PK11_UnwrapSymKey.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Sage Weil [Thu, 21 Mar 2019 16:58:29 +0000 (11:58 -0500)]
mon/MgrStatMonitor: ensure only one copy of initial service map
It's possible for create_pending to be called multiple times.
Fixes: http://tracker.ceph.com/issues/38839
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
204a8deacd2068989bd52eb130f4aac434b45f35)
Conflicts:
src/mon/MgrStatMonitor.cc : Resolved in create_initial
Yuri Weinstein [Tue, 26 Mar 2019 16:56:26 +0000 (09:56 -0700)]
Merge pull request #26758 from pdvian/wip-38510-luminous
luminous: CLI: ability to change file ownership
Yuri Weinstein [Tue, 26 Mar 2019 16:49:48 +0000 (09:49 -0700)]
Merge pull request #26830 from pdvian/wip-38562-luminous
luminous: mgr/BaseMgrModule: drop GIL for ceph_send_command
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Yuri Weinstein [Tue, 26 Mar 2019 16:48:50 +0000 (09:48 -0700)]
Merge pull request #26900 from pdvian/wip-38586-luminous
luminous: OSD crashes in get_str_map while creating with ceph-volume
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Tue, 26 Mar 2019 16:48:21 +0000 (09:48 -0700)]
Merge pull request #26962 from pdvian/wip-38665-luminous
luminous: qa: ignore slow metadata io wrn during osd thrash
Reviewed-by: Neha Ojha <nojha@redhat.com>
Matt Benjamin [Thu, 14 Mar 2019 17:58:17 +0000 (13:58 -0400)]
rgw: nfs: skip empty (non-POSIX) path segments
Such a path could be created through S3 upload,
e.g., s3://my_files//data/file.pdf.
Previously we asserted for this case, but it's harmless--such a
path segment should just be ignored.
Fixes: http://tracker.ceph.com/issues/38744
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
a53e6c1d5b785a2a46007292041d0a8cb552e3e6)
Conflicts:
src/rgw/rgw_file.h : Resolved in operator()
Matt Benjamin [Tue, 12 Mar 2019 12:58:53 +0000 (08:58 -0400)]
rgw: ldap: fix LDAPAuthEngine::init() when uri !empty()
Fixes: https://tracker.ceph.com/issues/38699
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit
6ef98c6e0fcf4f9b6e431b3409975e0966c5c21a)
Nathan Cutler [Fri, 22 Mar 2019 13:22:02 +0000 (14:22 +0100)]
Merge pull request #27033 from cbodley/wip-38796
luminous: doc/rgw: document placement target configuration
Reviewed-by: Nathan Cutler <ncutler@suse.com>
David Zafman [Tue, 19 Mar 2019 23:59:39 +0000 (16:59 -0700)]
Merge pull request #26557 from dzafman/wip-38244
luminous: scrub warning check incorrectly uses mon scrub interval
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
David Zafman [Tue, 19 Mar 2019 23:48:17 +0000 (16:48 -0700)]
Merge pull request #26793 from dzafman/wip-38274
luminous: Fix recovery and backfill priority handling
Reviewed-by: Neha Ojha <nojha@redhat.com>
Yuri Weinstein [Mon, 18 Mar 2019 18:59:14 +0000 (11:59 -0700)]
Merge pull request #26544 from ifed01/wip-ifed-fix-compress-luminous
luminous: os/bluestore: do not assert on non-zero err codes from compress() call
Reviewed-by: Sage Weil <sage@redhat.com>
Casey Bodley [Fri, 15 Mar 2019 18:35:13 +0000 (14:35 -0400)]
doc/rgw: document placement target configuration
Fixes: http://tracker.ceph.com/issues/24508
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit
c95585b1984e60708addee436a774d0c92ce57a6)
Abhishek Lekshmanan [Thu, 13 Dec 2018 14:57:40 +0000 (15:57 +0100)]
rgw: sync module: avoid printing attrs of objects in log
Since attrs are verbose and not necessarily needed as a debug info (as they can
be obtained from the required object itself) avoid logging them. Also
additionally removed another place where such a log is done by the ES sync
module
Fixes: http://tracker.ceph.com/issues/37646
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit
8f37b70630ceb62aff4019e9ad2508977590a873)
Conflicts:
src/rgw/rgw_sync_module.cc : Resolved in RGWCallStatRemoteObjCR::operate
Yan, Zheng [Thu, 28 Feb 2019 13:36:28 +0000 (21:36 +0800)]
mds: wait for client to release shared cap when re-acquiring xlock
MDS may revoke shared cap from xlocker client during 'LOCK_XLOCKDONE ->
LOCK_LOCK_XLOCK' state transition.
Fixes: http://tracker.ceph.com/issues/38491
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit
218e784f2eb1eec6bbb7910b5432f7f36263dbe7)
Conflicts:
src/mds/SimpleLock.h : Resovled in put_xlock
Sage Weil [Fri, 30 Nov 2018 15:55:38 +0000 (09:55 -0600)]
qa/objectstore: bluestore -> bluestore-{bitmap,stupid}
The new default is bitmap, so we were testing bitmap twice. Instead,
explicitly call out stupid and bitmap cases so a future default change
won't break coverage.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
f390704b4616c837352e7d6a8df91526cb047f0d)
Igor Fedotov [Mon, 25 Jun 2018 14:49:49 +0000 (17:49 +0300)]
os/bluestore: fix minor issues in bitmap logging output
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
dc4b5fc5afcea4be9e3b4c91dcc9dcce38d4e906)
Igor Fedotov [Mon, 25 Jun 2018 14:42:41 +0000 (17:42 +0300)]
os/bluestore: fix overlappedd allocation returned by bitmap allocator
Fixes: https://tracker.ceph.com/issues/24598
Signed-off-by: Igor Fedotov <ifedotov@suse.som>
(cherry picked from commit
539af4f27e773cb21d187b75086c7c9498dd7749)
Igor Fedotov [Fri, 15 Jun 2018 14:04:57 +0000 (17:04 +0300)]
os/bluestore: provide a rough estimate for bitmap allocator
fragmentation.
The approach is counting 'partial' and 'free' slots at L1 on the fly and
use partial / (partial+free) value as rough fragmentation estimate.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
dbbe12fe6f543fc6942f41a108effc7eb1632f67)
Igor Fedotov [Tue, 12 Jun 2018 11:50:30 +0000 (14:50 +0300)]
os/bluestore: rename new bitmap allocator class to BitmapAllocator.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
a8e8c192acb48ce93553b04d1953ce8bee1b16e4)
Igor Fedotov [Fri, 1 Jun 2018 18:59:15 +0000 (21:59 +0300)]
os/bluestore: perform allocations aligned with min_length in new bitmap allocator
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
51974c48e38e7080eb12c6028f95f4da666fbe76)
Igor Fedotov [Fri, 1 Jun 2018 13:35:04 +0000 (16:35 +0300)]
test/objectstore/unitetest_fastbmap_allocator: replace ASSERT_TRUE with
ASSERT_EQ
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
69ccb4367dcdcc51fbcfdaaea308c761291621e1)
Igor Fedotov [Thu, 31 May 2018 13:53:31 +0000 (16:53 +0300)]
os/bluestore: respect min_length as allocation granularity for new
bitmap allocator.
It was used a real minimum threshold before this fix which allowed e.g.
allocated extent length to be equal to min_length + 1.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
8a188b2aa3892b7f5a6c85042ffae8c2847ca869)
Igor Fedotov [Thu, 31 May 2018 11:28:28 +0000 (14:28 +0300)]
os/bluestore: cosmetic new allocator internal method rename.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
fa06ff234f11f266bb10d7fa9ea28847e5ce72fe)
Igor Fedotov [Thu, 31 May 2018 10:53:37 +0000 (13:53 +0300)]
os/bluestore: properly respect min_length when analysing partially free
slots in bitmap allocator.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
288bbd42531bebd8f6c619c848e75bfd9b8f93fd)
Igor Fedotov [Thu, 31 May 2018 10:34:21 +0000 (13:34 +0300)]
os/bluestore: cosmetic cleanup in new bitmap allocator.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
3db9d9a66cbd62c543cdd88ff4ab636fba2c2a2b)
Igor Fedotov [Wed, 30 May 2018 12:24:05 +0000 (15:24 +0300)]
os/bluestore: more verbose logging in new bitmap allocator
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
160c73032db2e122f5e643090c86eed85e679eb4)
Nathan Cutler [Fri, 15 Mar 2019 16:18:32 +0000 (17:18 +0100)]
Merge pull request #26597 from smithfarm/wip-release-notes-12_2_12-luminous
luminous: doc: move keyring caps RN to correct point release
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Nathan Cutler [Fri, 15 Mar 2019 15:43:41 +0000 (16:43 +0100)]
Merge pull request #26861 from ashishkumsingh/wip-38647-luminous
luminous: doc: Fix incorrect mention of 'osd_deep_mon_scrub_interval'
Reviewed-by: Neha Ojha <nojha@redhat.com>
Andrew Schoen [Thu, 14 Mar 2019 20:14:40 +0000 (15:14 -0500)]
ceph-volume: look for rotational data in lsblk
Also ask lsblk if a device is rotational if no information
is found in /sys/block, default to True if nothing is found.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=
1666822
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit
de748f9aef894b247622ee5dc49c1fbc31097ad0)
Nathan Cutler [Fri, 15 Mar 2019 15:19:41 +0000 (16:19 +0100)]
Merge pull request #26372 from pdvian/wip-38095-luminous
luminous: doc: rados/configuration: refresh osdmap section
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Igor Fedotov [Mon, 14 May 2018 13:09:22 +0000 (16:09 +0300)]
os/bluestore: remove original bitmap allocator
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
95bd72fc22aa313112de096ac3ef98e45c71bce5)
Igor Fedotov [Fri, 11 May 2018 21:26:49 +0000 (00:26 +0300)]
os/bluestore: align BitMap allocator's init_rm_free/init_add_free parameters with min_alloc_size
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
1b6a56e80066decf8c3727090a2cd51ba9ded9bc)
Igor Fedotov [Thu, 10 May 2018 15:54:36 +0000 (18:54 +0300)]
os/bluestore: fix improper access to a BitmapFastAllocator::last_pos
from multiple threads.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
0c69672fce55362a7e6eb8ed6955f163afe85482)
Igor Fedotov [Fri, 4 May 2018 17:03:37 +0000 (20:03 +0300)]
test/allocator: move bluestore allocator's benchmarks to a standalone UT
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
ac2ec80cddc51f1642849c7bf226e9bed116c0d6)
Igor Fedotov [Thu, 3 May 2018 17:22:15 +0000 (20:22 +0300)]
os/bluestore: add new bitmap allocator
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
631041a7b788eaaf010f1e39f58b5546ab5580fc)
Nathan Cutler [Fri, 15 Mar 2019 11:31:40 +0000 (12:31 +0100)]
Merge pull request #26969 from pdvian/wip-38667-luminous
luminous: doc/radosgw: Document mappings of S3 Operations to ACL grants
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Patrick Donnelly [Wed, 13 Mar 2019 16:13:02 +0000 (09:13 -0700)]
qa: extend MDS heartbeat grace for valgrind
Valgrind makes the MDS slowwwww. The newish mds_heartbeat_grace config allows
us to keep sending beacons to the mons even if the internal heartbeat is slow.
This avoids the laggy messages which are useful to grep for unrelated messaging
issues.
Fixes: http://tracker.ceph.com/issues/38723
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
7b520755ce62461060b6dc75fb7a39c464d2bdcb)
Adam C. Emerson [Thu, 7 Mar 2019 20:18:42 +0000 (15:18 -0500)]
doc/radosgw: Document mappings of S3 Operations to ACL grants
Since the ACL documentation is a bit sparse, let people know exactly
what operations they can expect to enable with a given ACL grant.
Fixes: https://tracker.ceph.com/issues/38523
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit
bd94c93fada952537c7015743db55bff7619afa8)
Patrick Donnelly [Fri, 8 Mar 2019 18:49:10 +0000 (10:49 -0800)]
qa: ignore slow metadata io wrn during osd thrash
Fixes: http://tracker.ceph.com/issues/38651
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit
5abcc32ff642b03c4b53e565e995ebb94802579a)
Yuri Weinstein [Thu, 14 Mar 2019 21:34:23 +0000 (14:34 -0700)]
Merge pull request #26798 from dzafman/wip-slow-luminous
luminous: qa: Add "slow request" where missing based on REQUEST_SLOW
Reviewed-by: Neha Ojha <nojha@redhat.com>
Igor Fedotov [Thu, 3 May 2018 16:25:01 +0000 (19:25 +0300)]
test/allocator: get rid off bitmap allocator specifics and introduce new
performance test cases
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
dcd30610fb1c87711c9f43f7d3d6636888792345)
Igor Fedotov [Mon, 7 May 2018 13:58:43 +0000 (16:58 +0300)]
os/bluestore: add release(PExtentVector) helper to Allocator class to
free PExtentVector explicitly.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
328ea72a84edef4a6c4e0558dff52d0e85f05122)
Igor Fedotov [Mon, 7 May 2018 13:26:03 +0000 (16:26 +0300)]
os/bluestore: get rid off allocator's reserve method
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit
88bea46242d4eb6bdfd8309ddf8843c9a14e5f25)