Yan, Zheng [Mon, 29 Jan 2018 02:59:00 +0000 (10:59 +0800)]
osdc/Journaler: make sure flush() writes enough data
If _do_flush() fails to write all data because of prezere, _do_flush()
should be called again when prezero finishes. Otherwise flush waiters
may sleep forever.
Orit Wasserman [Sun, 21 Jan 2018 10:11:34 +0000 (12:11 +0200)]
rgw: resharding needs to set back the bucket ACL after link
Jewel only fix. New implementation of resharding in Luminous. Fixes: http://tracker.ceph.com/issues/22703 Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Huan Zhang [Fri, 24 Jun 2016 03:27:53 +0000 (11:27 +0800)]
rbd discard return -EINVAL if len > MAX_INT32
rbd discard use 'int' to return discarded length, but the 'len' user
passed is 'uint64', in some case, the ret value will be truncated
and return a negative value which means discard failed. ret -EINVAL
if len > MAX_INT32 to indicate support len <= MAX_INT32 only.
Jason Dillaman [Wed, 15 Nov 2017 14:09:15 +0000 (09:09 -0500)]
librbd: prevent overflow of discard API result code
Prevent discard/writesame lengths larger than 2GB.
Fixes: http://tracker.ceph.com/issues/21966 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3effd324db181e625665be33b5c6529dca723cc5) Signed-off-by: Nathan Cutler <ncutler@suse.com>
Conflicts:
PendingReleaseNotes (adapted for jewel)
src/librbd/librbd.cc (no writesame in jewel)
Li Wang [Wed, 1 Nov 2017 09:21:29 +0000 (09:21 +0000)]
rbd-nbd: fix unused nbd device search bug in container
In some container scenarios, the host may choose to
map a specific nbd device, for example, /dev/nbd6 into the
container, in that case, the nbd device available in the
container is not numbered from 0. The current unused
nbd device search function will return no result.
This patch fixes it.
Fixes: http://tracker.ceph.com/issues/22012 Signed-off-by: Li Wang <laurence.liwang@gmail.com> Reviewed-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>
(cherry picked from commit be0f9581f9727187ca03232e0b368e7da7a60609)
Jason Dillaman [Fri, 27 Oct 2017 20:45:54 +0000 (16:45 -0400)]
cls/journal: ensure tags are properly expired
Previously, if only the local image was using the journal or if
a disconnected peer was attached, the tag entries could not be
expired even if unreferenced.
Fixes: http://tracker.ceph.com/issues/21960 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 19fa1c7f5b2809e9a223b7b196dfc031e97a5dcd)
Matt Benjamin [Tue, 3 Oct 2017 21:48:29 +0000 (17:48 -0400)]
rgw: release cls lock if taken in RGWCompleteMultipart
Follows Casey's proposal to conditionally release the lock in
::complete(), in order to avoid duplicated code in various early
return cases.
Fixes: http://tracker.ceph.com/issues/21596 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 704f793f08a02760d23eb5778b738bb07be0e7cf)
Casey Bodley [Thu, 5 Oct 2017 20:39:30 +0000 (16:39 -0400)]
rgw: RGWUser::init no longer overwrites user_id
if an admin op specifies a user_id and does not find a user with that
id, but does find a user based on a later field (email, access key,
etc), RGWUser::user_id will be overwritten with the existing user's id
when this happens on 'radosgw-admin user create', RGWUser::execute_add()
will modify that existing user, instead of trying to create a new user
with the given user_id (and failing due to the conflicting email,
access key, etc)
by preserving the original user_id (when specified), this uid conflict
is detected in RGWUser::check_op() and a "user id mismatch" error is
returned
Conflicts:
src/tools/ceph_objectstore_tool.cc (in jewel, ::encode() takes only two
arguments, while in luminous/master it takes a third which we omit
here)
Matt Benjamin [Fri, 19 Jan 2018 18:05:27 +0000 (13:05 -0500)]
rgw_file: alternate fix deadlock on lru eviction
This change is an alternate fix for two problems found and fixed
by Yao Zongyou <yaozongyou@vip.qq.com>.
The deadlock can be avoided just by not taking it in the recycle
case, which invariantly holds the lock.
The invalidation of the insert iterator by the recyle-path unlink
we'd like to handle as a condition in order to preserve the cached
insertion point optimization we get in the common case. (The
original behavior was, indeed, incorrect.)
Based on feedback from Yao, removed the RGWFileHandle dtor version
of the unlink check, which I think happened twice.
Yan, Zheng [Thu, 11 Jan 2018 09:50:22 +0000 (17:50 +0800)]
client: fix cap revoke race
If caps are been revoking by the auth MDS, don't consider them as
issued even they are still issued by non-auth MDS. The non-auth
MDS should also be revoking/exporting these caps, the client just
hasn't received the cap revoke/export message.
The race I encountered is: When caps are exporting to new MDS, the
client receives cap import message and cap revoke message from the
new MDS, then receives cap export message from the old MDS. When
the client receives cap revoke message from the new MDS, the revoking
caps are still issued by the old MDS, so the client does nothing.
Later when the cap export message is received, the client removes
the caps issued by the old MDS. (Another way to fix the race is
calling ceph_check_caps() in handle_cap_export())
Casey Bodley [Mon, 18 Dec 2017 16:42:21 +0000 (11:42 -0500)]
rgw: dont log EBUSY errors in 'sync error list'
these temporary errors get retried automatically, so no admin
intervention is required. logging them only serves to waste space in
omap and obscure the more serious sync errors
In cls_timeindex_list() though `to_index` has expired for a timespan, the marker is set for a subsequent index during the time boundary check.
This marker is further returned to RGWObjectExpirer::process_single_shard(), where this out_marker is trimmed from the respective shard,
resulting in a lost removal hint and a leaked object.
Jason Dillaman [Wed, 27 Sep 2017 13:40:08 +0000 (09:40 -0400)]
librbd: hold cache_lock while clearing cache nonexistence flags
When transitioning from a snapshot that had an associated parent
to a snapshot where the parent was flattened and removed, the cache
was being referenced without holding the required lock.
Fixes: http://tracker.ceph.com/issues/21558 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 16ef97830cde30efb96f7aee69834b3a5c2d5248)
rgw: data sync: set num_shards when building full maps
When radosgw-admin data sync init is called on a cluster, the next run
of rgw crashes as when it processes ListBucketIndexesCR, num_shards
isn't set which is later referenced in ListBucketIndexesCR. Setting the
n sync_info.num_shards correctly to handle this case
Adam C. Emerson [Wed, 20 Dec 2017 22:06:32 +0000 (17:06 -0500)]
rgw: Plumb refresh logic into object cache
Now when we force a refetch of bucket info it will actually go to the
OSD rather than simply using the objects in the object cache.
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit d997f657750faf920170843e62deacab70008d8b) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam C. Emerson [Tue, 19 Dec 2017 21:47:09 +0000 (16:47 -0500)]
rgw: Add expiration in the object cache
We had it in the chained caches, but it doesn't do much good if
they just fetch objects out of the object cache.
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 82a7e6ca31b416a7f0e41b5fda4c403d1d6be947) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam C. Emerson [Tue, 19 Dec 2017 17:53:05 +0000 (12:53 -0500)]
rgw: retry CORS put/delete operations on ECANCELLED
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit bff7e61ca5a66b301ec49c1cf9054d1b74535832) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam C. Emerson [Fri, 17 Nov 2017 22:15:26 +0000 (17:15 -0500)]
rgw: Expire entries in bucket info cache
To bound the degree to which an RGW instance can go out to lunch if
the watch/notify breaks down, force refresh of any cache entry over a
certain age.
Fifteen minutes by default, and expiration can be turned off entirely.
This is separate from the LRU. The LRU removes entries based on the
last time of access. This expiration patch forces refresh based on the
last time they were updated.
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 4489cb58a15647a31ac0546d70400af5668404cb) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam C. Emerson [Fri, 17 Nov 2017 21:05:06 +0000 (16:05 -0500)]
rgw: Handle stale bucket info in RGWDeleteBucketWebsite
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit f4d274248e43cb38ff2b27782c010b2c35b12b2b)
Adam C. Emerson [Fri, 17 Nov 2017 21:03:13 +0000 (16:03 -0500)]
rgw: Handle stale bucket info in RGWSetBucketWebsite
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit b2b7385f194def1025a8947bab876c9856b06400) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam C. Emerson [Fri, 17 Nov 2017 20:59:44 +0000 (15:59 -0500)]
rgw: Handle stale bucket info in RGWSetBucketVersioning
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit a0a1e7c2ef992b8758bcfb20d893730c1b202475) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam C. Emerson [Fri, 17 Nov 2017 20:53:05 +0000 (15:53 -0500)]
rgw: Handle stale bucket info in RGWPutMetadataBucket
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit ebb86301b20098e15824f469001f6153b27965f5) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Adam C. Emerson [Fri, 17 Nov 2017 20:51:42 +0000 (15:51 -0500)]
rgw: Add retry_raced_bucket_write
If the OSD informs us that our bucket info is out of date when we need
to write, we should have a way to update it.
This template function allows us to wrap relevant sections of code so
they'll be retried against new bucket info on -ECANCELED.
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 1a3fcc70c0747791aa423cd0aa7d2596eaf3d73c)
Adam C. Emerson [Thu, 16 Nov 2017 19:42:58 +0000 (14:42 -0500)]
rgw: Add try_refresh_bucket_info function
Sometimes operations fail with -ECANCELED. This means we got raced. If
this happens we should update our bucket info from cache and try again.
Some user reports suggest that our cache may be getting and staying
out of sync. This is a bug and should be fixed, but it would also be
nice if we were robust enough to notice the problem and refresh.
So in that case, we invalidate the cache and fetch direct from the
OSD, putting a warning in the log.
Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit 9114e5e50995f0c7d2be5c24aa4712d89cd89f48) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Marcus Watts [Tue, 2 Jan 2018 22:24:38 +0000 (17:24 -0500)]
radosgw: fix doubled underscore with s3/swift server-side copy
A name is almost an oid. Except that if a name starts
with a _, the corresponding oid starts with a double _.
The copy logic passes the oid straight in as a name,
which results in a triple _ on the oid, resulting in
an object name with a double __.
Fix: remove one _ when converting an oid back into
a name, so that the final oid only has a double _.
Fixes: http://tracker.ceph.com/issues/22529
This change is not needed on master or Luminous, due to
already-merged #18662.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Kefu Chai [Fri, 22 Dec 2017 14:42:16 +0000 (22:42 +0800)]
install-deps.sh: update g++ symlink also
we need to update g++ symlink also, if it points to the wrong version
http://tracker.ceph.com/issues/22220 Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: the libboost issue does not affect master. as master builds
boost from source. so, it's not cherry-picked from master.
Kefu Chai [Wed, 13 Dec 2017 05:36:54 +0000 (13:36 +0800)]
install-deps.sh: point gcc to the one shipped by distro
to define a struct in a method is legal in C++11, but it causes internal
compiler error due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82155
if we are using GCC-7. so we need to either workaround in our source
code by moving the struct definition out of the member method or revert
to a GCC without this bug. but if we go with the first route, the jewel
build still fails, because GCC-7 starts to use the new CXX11 ABI, which
is not compatible with the libboost we use in jewel. the libboost was
still built with the old ABI for backward compatibility. so let's just
fix the install-deps.sh to point gcc to the origin one.
See: http://tracker.ceph.com/issues/22220 Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: the libboost issue does not affect master. as master builds
boost from source. so, it's not cherry-picked from master.
This is needed for jewel-x point to point upgrade because earlier point
releases can't handle our ec profiles with ruleset-* (later ones can) and
the test races with the mon upgrades.
projects / ceph.git / log