Matt Benjamin [Tue, 28 Feb 2017 20:49:06 +0000 (15:49 -0500)]
rgw_file: use fh_hook::is_linked() to check residence
Previously we assumed that !deleted handles were resident--there
is an observed case where a !deleted handle is !linked. Since
we currently use safe_link mode, an is_linked() check is
available, and exhaustive.
Fixes: http://tracker.ceph.com/issues/19111 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit c0aa515f8d8c57ec5ee09e3b60df3cac60453c40)
Matt Benjamin [Wed, 1 Mar 2017 01:24:12 +0000 (20:24 -0500)]
rgw_file: RGWFileHandle dtor must also cond-unlink from FHCache
Formerly masked in part by the reclaim() action, direct-delete now
substitutes for reclaim() iff its LRU lane is over its high-water
mark, and in particular, like reclaim() the destructor is certain
to see handles still interned on the FHcache when nfs-ganesha is
recycling objects from its own LRU.
Fixes: http://tracker.ceph.com/issues/19112 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit d51a3b1224ba62bb53c6c2c7751fcf7853c35a4b)
Matt Benjamin [Tue, 17 Jan 2017 16:23:45 +0000 (11:23 -0500)]
rgw_file: split last argv on ws, if provided
This is intended to allow an "extra" unparsed argument string
containing various cmdline options to be passed as the last argument
in the argv array of librgw_create(), which nfs-ganesha is
expecting to happen.
Matt Benjamin [Mon, 13 Feb 2017 01:18:26 +0000 (20:18 -0500)]
rgw_file: fix hiwat behavior
Removed logic to skip reclaim processing conditionally on hiwat,
this probably meant to be related to a lowat value, which does
not exist.
Having exercised the hiwat reclaim behavior, noticed that the
path which moves unreachable objects to LRU, could and probably
should remove them altogether when q.size exceeds hiwat. Now
the max unreachable float is lane hiwat, for all lanes.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit b8791b2217e9ca87b2d17b51f283fa14bd68b581) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sun, 12 Feb 2017 23:20:43 +0000 (18:20 -0500)]
rgw_file: refcnt bugfixes
This change includes 3 related changes:
1. add required lock flags for FHCache updates--this is a crash
bug under concurrent update/lookup
2. omit to inc/dec refcnt on root filehandles in 2 places--the
root handle current is not on the lru list, so it's not
valid to do so
3. based on observation of LRU behavior during creates/deletes,
update (cohort) LRU unref to move objects to LRU when their
refcount falls to SENTINEL_REFCNT--this cheaply primes the
current reclaim() mechanism, so very significanty improves
space use (e.g., after deletes) in the absence of scans
(which is common due to nfs-ganesha caching)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit beaeff059375b44188160dbde8a81dd4f4f8c6eb) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sat, 11 Feb 2017 21:38:05 +0000 (16:38 -0500)]
rgw_file: add refcount dout traces at debuglevel 17
These are helpful for checking RGWFileHandle refcnt invariants.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 462034e17f919fb783ee33e2c9fa8089f93fd97d) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Fri, 10 Feb 2017 22:14:16 +0000 (17:14 -0500)]
rgw_file: add pretty-print for RGWFileHandle
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit ef330f385d3407af5f470b5093145f59cc4dcc79) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Mon, 20 Feb 2017 20:05:18 +0000 (15:05 -0500)]
rgw_file: fix marker computation
Fixes: http://tracker.ceph.com/issues/19018 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 4454765e7dd08535c50d24205858e18dba4b454c)
Matt Benjamin [Mon, 20 Feb 2017 01:34:31 +0000 (20:34 -0500)]
rgw_file: rgw_readdir can't list multi-segment dirs
This issue has one root cause in librgw, namely that the marker
argument to these requests was incorrectly formatted (though the
marker cache was working as intended).
Secondarily, for nfs-ganesha users, there is a compounding issue
that the RGW fsal was required by "temporary" convention to
populate the entire dirent cache for a directory on a single
readdir() invocation--the cache_inode/mdcache implementations
invariantly pass (before future 2.5 changesets, currently in
progress) a null pointer for the start cookie offset, intended
to convey this.
Fixes: http://tracker.ceph.com/issues/18991 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 2cd60ee9712291b906123aca1704288b18a9742b)
Matt Benjamin [Sun, 19 Feb 2017 23:21:06 +0000 (18:21 -0500)]
rgw_file: allow setattr on placeholder directories
When a POSIX path <bucket>/foo/ is known only as an implicit path
segment from other objects (e.g., <bucket>/foo/bar.txt), a case
that would usually arise from S3 upload of such an object, an
RGWFileHandle object representing "<bucket>/foo/" will be constructed
as needed, with no backing in RGW.
This is by design, but subsequently, if a setattr is performed on
such a handle, we must be ready to create the object inline with
storing the attributes.
Fixes: http://tracker.ceph.com/issues/18989 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 55eec1c0a0e136736961423b7b6244d0f3693c1a)
Gui Hecheng [Thu, 2 Mar 2017 09:21:57 +0000 (17:21 +0800)]
rgw_file: posix style atime,ctime,mtime
As an ganesha FSAL backend, rgw_file should properly maintain
the atime,ctime,mtime properly against operations such as:
(read,write) for file
(create,unlink,mkdir,rmdir,rename) for dir
(setattr) for file and dir
Matt Benjamin [Mon, 20 Feb 2017 20:05:18 +0000 (15:05 -0500)]
rgw_file: fix marker computation
Fixes: http://tracker.ceph.com/issues/19018 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 4454765e7dd08535c50d24205858e18dba4b454c) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Mon, 20 Feb 2017 01:34:31 +0000 (20:34 -0500)]
rgw_file: rgw_readdir can't list multi-segment dirs
This issue has one root cause in librgw, namely that the marker
argument to these requests was incorrectly formatted (though the
marker cache was working as intended).
Secondarily, for nfs-ganesha users, there is a compounding issue
that the RGW fsal was required by "temporary" convention to
populate the entire dirent cache for a directory on a single
readdir() invocation--the cache_inode/mdcache implementations
invariantly pass (before future 2.5 changesets, currently in
progress) a null pointer for the start cookie offset, intended
to convey this.
Fixes: http://tracker.ceph.com/issues/18991 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 2cd60ee9712291b906123aca1704288b18a9742b) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sun, 19 Feb 2017 23:21:06 +0000 (18:21 -0500)]
rgw_file: allow setattr on placeholder directories
When a POSIX path <bucket>/foo/ is known only as an implicit path
segment from other objects (e.g., <bucket>/foo/bar.txt), a case
that would usually arise from S3 upload of such an object, an
RGWFileHandle object representing "<bucket>/foo/" will be constructed
as needed, with no backing in RGW.
This is by design, but subsequently, if a setattr is performed on
such a handle, we must be ready to create the object inline with
storing the attributes.
Fixes: http://tracker.ceph.com/issues/18989 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 55eec1c0a0e136736961423b7b6244d0f3693c1a) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Javier M. Mellid [Tue, 26 Jul 2016 12:53:44 +0000 (14:53 +0200)]
rgw: multipart uploads copy part support
Add multipart uploads copy part feature.
Fixes: http://tracker.ceph.com/issues/12790 Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
(cherry picked from commit 949480c2e9760855ed6a0501d364d5e766c8207d)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
rgw: make sending Content-Length in 204 and 304 controllable
This commit introduces a new configurable "rgw print prohibited
content length" to let operator decide whether RadosGW complies
to RFC 7230 (a part of the HTTP specification) or violates it
but follows the Swift's behavior.
Casey Bodley [Thu, 9 Mar 2017 20:24:08 +0000 (15:24 -0500)]
rgw: clear old zone short ids on period update
the short ids of old, removed zones were being kept in the period to
guard against hash collisions with new zones
but for a hash collision to cause a wrong object to sync, that object
would have to be uploaded simultaneously to two different zones that had
the same short id
to avoid this, we just have to prevent the period from containing two
colliding zones at the same time - we don't have to remember old zone
short ids forever
Nathan Cutler [Thu, 23 Mar 2017 08:46:53 +0000 (09:46 +0100)]
Merge branch 'jewel-next' into jewel
Branch "jewel-next" was merged to jewel on Dec 22 (5b402f8a7b), and then PR
https://github.com/ceph/ceph/pull/13143 merged to jewel-next much later (Feb
1st), and then jewel-next was never merged to jewel again before we tagged
v10.2.6.
This merge brings in a single commit, i.e. the one from
https://github.com/ceph/ceph/pull/13143
Reported-by: Ken Dreyer <kdreyer@redhat.com> Signed-off-by: Nathan Cutler <ncutler@suse.com>
Matt Benjamin [Thu, 23 Feb 2017 15:21:38 +0000 (10:21 -0500)]
rgw_file: return of RGWFileHandle::FLAG_EXACT_MATCH
Allow callers of rgw_lookup() on objects attested in an
rgw_readdir() callback the ability to bypass exact match in
RGWLibFS::stat_leaf() case 2, but restore exact match enforcement
for general lookups.
This preserves required common_prefix namespace behavior, but
prevents clients from eerily permitting things like "cd sara0" via
partial name match on "sara01."
Fixes: http://tracker.ceph.com/issues/19059 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 70ef7d45e0abf2661bd4e23161d4e70cf5178079)
Matt Benjamin [Sun, 19 Feb 2017 22:43:17 +0000 (17:43 -0500)]
rgw_file: invalid use of RGWFileHandle::FLAG_EXACT_MATCH
The change which introduced this flag also caused it to be
given as the flags argument to RGWLibFS::stat_leaf() when called
from rgw_lookup().
This was incorrect: in particular, when a directory is known only
as a common prefix of other objects, the AWS namespace mapping
convention requires lookup("foo") to match a non-materialized
instance of "foo/" (case 2 in RGWLibFS::stat_leaf's stat loop).
Fixes: http://tracker.ceph.com/issues/18992 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit e31e9eb980f958640150e8d7f17de1b9e5478b1e)
Matt Benjamin [Wed, 22 Feb 2017 19:57:59 +0000 (14:57 -0500)]
rgw_file: rgw_lookup: don't ref for "/" or ".."
These refs won't be returned by nfs-ganesha, and are sufficiently
magical that other consumers should be persuaded to understand
their specialness.
Fixes: http://tracker.ceph.com/issues/19060 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit dea8d1ee373399a21851690a9753388b659b8ede)
Matt Benjamin [Wed, 22 Feb 2017 15:24:29 +0000 (10:24 -0500)]
rgw_file: avoid stranding invalid-name bucket handles in fhcache
To avoid a string copy in the common mkdir path, handles for
proposed buckets currently are staged in the handle table, before
being rejected. They need to be destaged, not just marked deleted
(because deleted objects are now assumed not to be linked, as of beaeff059375b44188160dbde8a81dd4f4f8c6eb).
This triggered an unhandled Boost assert when deleting staged
handles, as current safe_link mode requires first removing from
the FHCache.
Fixes: http://tracker.ceph.com/issues/19036 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 6cde812c92e5bba9f85fbf8486ebe69b55952370)
ff0e521 resolved the issue for the other daemons but not for rgw since
it calls setuid (via civetweb) after the new code sets PR_SET_DUMPABLE.
Add another prctl call before wait_shutdown.
Jing Wenjun [Wed, 8 Feb 2017 07:07:43 +0000 (15:07 +0800)]
rgw: fix swift cannot disable object versioning with empty X-Versions-Location
we should be able to disable object verioning by removing its X-Versions-Location
metadata header by sending an empty key value. this description can be found at
No.8 in http://docs.openstack.org/user-guide/cli-swift-set-object-versions.html.
This change does prioritize backfill of PGs which don't
have min_size active copies. Such PGs would cause IO stalls
for clients and would increase throttlers usage.
This change also fixes few subtlle out-of-bounds bugs.
Matt Benjamin [Fri, 3 Feb 2017 18:44:45 +0000 (13:44 -0500)]
rgw_file: fix RGWLibFS::setattr for directory objects
Fixes: http://tracker.ceph.com/issues/18808 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 4ad5a9226852d6d564baf2e63278ed6c4c185ecb)
Matt Benjamin [Thu, 23 Feb 2017 21:02:07 +0000 (16:02 -0500)]
rgw_file: ensure valid_s3_object_name for directories, too
The logic in RGWLibFS::mkdir() validated bucket names, but not
object names (though RGWLibFS::create() did so).
The negative side effect of this was not creating illegal objects
(we won't), but in a) failing with -EIO and b) more importantly,
not removing up the proposed object from FHCache, which produced a
boost assert when recycled.
Fixes: http://tracker.ceph.com/issues/19066 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit eb1cd3b30c0504385f05bf2d2dd5e2251b7efed7)
projects / ceph.git / log