Matt Benjamin [Mon, 13 Feb 2017 01:18:26 +0000 (20:18 -0500)]
rgw_file: fix hiwat behavior
Removed logic to skip reclaim processing conditionally on hiwat,
this probably meant to be related to a lowat value, which does
not exist.
Having exercised the hiwat reclaim behavior, noticed that the
path which moves unreachable objects to LRU, could and probably
should remove them altogether when q.size exceeds hiwat. Now
the max unreachable float is lane hiwat, for all lanes.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit b8791b2217e9ca87b2d17b51f283fa14bd68b581) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sun, 12 Feb 2017 23:20:43 +0000 (18:20 -0500)]
rgw_file: refcnt bugfixes
This change includes 3 related changes:
1. add required lock flags for FHCache updates--this is a crash
bug under concurrent update/lookup
2. omit to inc/dec refcnt on root filehandles in 2 places--the
root handle current is not on the lru list, so it's not
valid to do so
3. based on observation of LRU behavior during creates/deletes,
update (cohort) LRU unref to move objects to LRU when their
refcount falls to SENTINEL_REFCNT--this cheaply primes the
current reclaim() mechanism, so very significanty improves
space use (e.g., after deletes) in the absence of scans
(which is common due to nfs-ganesha caching)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit beaeff059375b44188160dbde8a81dd4f4f8c6eb) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sat, 11 Feb 2017 21:38:05 +0000 (16:38 -0500)]
rgw_file: add refcount dout traces at debuglevel 17
These are helpful for checking RGWFileHandle refcnt invariants.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 462034e17f919fb783ee33e2c9fa8089f93fd97d) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Fri, 10 Feb 2017 22:14:16 +0000 (17:14 -0500)]
rgw_file: add pretty-print for RGWFileHandle
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit ef330f385d3407af5f470b5093145f59cc4dcc79) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Javier M. Mellid [Tue, 26 Jul 2016 12:53:44 +0000 (14:53 +0200)]
rgw: multipart uploads copy part support
Add multipart uploads copy part feature.
Fixes: http://tracker.ceph.com/issues/12790 Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
(cherry picked from commit 949480c2e9760855ed6a0501d364d5e766c8207d)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
rgw: make sending Content-Length in 204 and 304 controllable
This commit introduces a new configurable "rgw print prohibited
content length" to let operator decide whether RadosGW complies
to RFC 7230 (a part of the HTTP specification) or violates it
but follows the Swift's behavior.
Casey Bodley [Thu, 9 Mar 2017 20:24:08 +0000 (15:24 -0500)]
rgw: clear old zone short ids on period update
the short ids of old, removed zones were being kept in the period to
guard against hash collisions with new zones
but for a hash collision to cause a wrong object to sync, that object
would have to be uploaded simultaneously to two different zones that had
the same short id
to avoid this, we just have to prevent the period from containing two
colliding zones at the same time - we don't have to remember old zone
short ids forever
Nathan Cutler [Thu, 23 Mar 2017 08:46:53 +0000 (09:46 +0100)]
Merge branch 'jewel-next' into jewel
Branch "jewel-next" was merged to jewel on Dec 22 (5b402f8a7b), and then PR
https://github.com/ceph/ceph/pull/13143 merged to jewel-next much later (Feb
1st), and then jewel-next was never merged to jewel again before we tagged
v10.2.6.
This merge brings in a single commit, i.e. the one from
https://github.com/ceph/ceph/pull/13143
Reported-by: Ken Dreyer <kdreyer@redhat.com> Signed-off-by: Nathan Cutler <ncutler@suse.com>
Matt Benjamin [Thu, 23 Feb 2017 15:21:38 +0000 (10:21 -0500)]
rgw_file: return of RGWFileHandle::FLAG_EXACT_MATCH
Allow callers of rgw_lookup() on objects attested in an
rgw_readdir() callback the ability to bypass exact match in
RGWLibFS::stat_leaf() case 2, but restore exact match enforcement
for general lookups.
This preserves required common_prefix namespace behavior, but
prevents clients from eerily permitting things like "cd sara0" via
partial name match on "sara01."
Fixes: http://tracker.ceph.com/issues/19059 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 70ef7d45e0abf2661bd4e23161d4e70cf5178079)
Matt Benjamin [Sun, 19 Feb 2017 22:43:17 +0000 (17:43 -0500)]
rgw_file: invalid use of RGWFileHandle::FLAG_EXACT_MATCH
The change which introduced this flag also caused it to be
given as the flags argument to RGWLibFS::stat_leaf() when called
from rgw_lookup().
This was incorrect: in particular, when a directory is known only
as a common prefix of other objects, the AWS namespace mapping
convention requires lookup("foo") to match a non-materialized
instance of "foo/" (case 2 in RGWLibFS::stat_leaf's stat loop).
Fixes: http://tracker.ceph.com/issues/18992 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit e31e9eb980f958640150e8d7f17de1b9e5478b1e)
Matt Benjamin [Wed, 22 Feb 2017 19:57:59 +0000 (14:57 -0500)]
rgw_file: rgw_lookup: don't ref for "/" or ".."
These refs won't be returned by nfs-ganesha, and are sufficiently
magical that other consumers should be persuaded to understand
their specialness.
Fixes: http://tracker.ceph.com/issues/19060 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit dea8d1ee373399a21851690a9753388b659b8ede)
Matt Benjamin [Wed, 22 Feb 2017 15:24:29 +0000 (10:24 -0500)]
rgw_file: avoid stranding invalid-name bucket handles in fhcache
To avoid a string copy in the common mkdir path, handles for
proposed buckets currently are staged in the handle table, before
being rejected. They need to be destaged, not just marked deleted
(because deleted objects are now assumed not to be linked, as of beaeff059375b44188160dbde8a81dd4f4f8c6eb).
This triggered an unhandled Boost assert when deleting staged
handles, as current safe_link mode requires first removing from
the FHCache.
Fixes: http://tracker.ceph.com/issues/19036 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 6cde812c92e5bba9f85fbf8486ebe69b55952370)
ff0e521 resolved the issue for the other daemons but not for rgw since
it calls setuid (via civetweb) after the new code sets PR_SET_DUMPABLE.
Add another prctl call before wait_shutdown.
Jing Wenjun [Wed, 8 Feb 2017 07:07:43 +0000 (15:07 +0800)]
rgw: fix swift cannot disable object versioning with empty X-Versions-Location
we should be able to disable object verioning by removing its X-Versions-Location
metadata header by sending an empty key value. this description can be found at
No.8 in http://docs.openstack.org/user-guide/cli-swift-set-object-versions.html.
This change does prioritize backfill of PGs which don't
have min_size active copies. Such PGs would cause IO stalls
for clients and would increase throttlers usage.
This change also fixes few subtlle out-of-bounds bugs.
Matt Benjamin [Fri, 3 Feb 2017 18:44:45 +0000 (13:44 -0500)]
rgw_file: fix RGWLibFS::setattr for directory objects
Fixes: http://tracker.ceph.com/issues/18808 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 4ad5a9226852d6d564baf2e63278ed6c4c185ecb)
Matt Benjamin [Thu, 23 Feb 2017 21:02:07 +0000 (16:02 -0500)]
rgw_file: ensure valid_s3_object_name for directories, too
The logic in RGWLibFS::mkdir() validated bucket names, but not
object names (though RGWLibFS::create() did so).
The negative side effect of this was not creating illegal objects
(we won't), but in a) failing with -EIO and b) more importantly,
not removing up the proposed object from FHCache, which produced a
boost assert when recycled.
Fixes: http://tracker.ceph.com/issues/19066 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit eb1cd3b30c0504385f05bf2d2dd5e2251b7efed7)
Jason Dillaman [Tue, 21 Feb 2017 18:09:39 +0000 (13:09 -0500)]
rbd-mirror: object copy should always reference valid snapshots
If a remote snapshot is deleted while an image sync is in-progress,
associate the read request against the most recent, valid remote
snapshot for a given snapshot object clone.
Matt Benjamin [Sat, 31 Dec 2016 04:30:16 +0000 (23:30 -0500)]
rgw_file: interned RGWFileHandle objects need parent refs
RGW NFS fhcache/RGWFileHandle operators assume existence of the
full chain of parents from any object to the its fs_root--this is
a consequence of the weakly-connected namespace design goal, and
not a defect.
This change ensures the invariant by taking a parent ref when
objects are interned (when a parent ref is guaranteed). Parent
refs are returned when objects are destroyed--essentially by the
invariant, such a ref must exist.
The extra ref is omitted when parent->is_root(), as that node is
not in the LRU cache.
Fixes: http://tracker.ceph.com/issues/18650 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 0e5299f3f43e633a5d8a9360893b4b11f6217d81)
Nathan Cutler [Mon, 20 Feb 2017 10:44:49 +0000 (11:44 +0100)]
tests: reduce stress-split-erasure-code-x86_64 dependency on stress-split
This patch breaks the symlinks for two files from stress-split that
are being changed by https://github.com/ceph/ceph/pull/13222 in a way
that would break this test otherwise.
Nathan Cutler [Mon, 20 Feb 2017 10:44:49 +0000 (11:44 +0100)]
tests: reduce stress-split-erasure-code dependency on stress-split
This patch breaks the symlinks for two files from stress-split that
are being changed by https://github.com/ceph/ceph/pull/13222 in a way
that would break this test otherwise.
projects / ceph.git / log