Matt Benjamin [Thu, 7 Jan 2016 01:57:36 +0000 (20:57 -0500)]
librgw: likely correct marker handling
New logic propagates is_truncated traveral-finished indicator up
from bucket and object list operations in all cases, and correctly
expands relative paths.
Unit test output looks promising.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Wed, 6 Jan 2016 00:03:49 +0000 (19:03 -0500)]
librgw: rework full_object_name() and friends
Flex RGWFileHandle::full_object_name() to build either a search
prefix (omits leading "/" and bucket name) or else the full path
to the root (e.g., for pretty-printing), depending on the value
of a boolean omit_bucket argument.
New convenience methods search_prefix() and relative_object_name()
select these, respectively.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Tue, 5 Jan 2016 17:26:36 +0000 (12:26 -0500)]
librgw: add continued readdir test
Verify that files created in a single, large directory are correctly
enumerated through a sequence of rgw_readdir() calls, at successive
offsets checking cookie/marker and bounds invariants.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Thu, 24 Dec 2015 17:45:16 +0000 (12:45 -0500)]
librgw: dir traversal state GC added
To minimize overhead, traversal events are modeled as a linear
sequence of fh_key, timestamp tuples. GC operates out-of-line in
LIFO order, and includes a ref cycle. If the matching directory
object can't be found by handle, it's already gone, and nothing
happens.
To avoid lock contention, scanning of the event sequence is done
in groups of 24 by default, but in groups of sequence/4 if the
event sequence grows larger than 500 events.
Push READDIR events on successful component readdir (bucket or
object listing partial result, as appropriate).
Revise locking, timestamp and update link count in all cases. Link
counts are approximate (metadata to represent it is not stored).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sat, 19 Dec 2015 21:01:44 +0000 (16:01 -0500)]
common: add (tweaked) sstring.h
Add sstring.h from Seastar, slightly adapted for c++2011 and
boost::string_ref.
An sstring resembles std::string, but the size of its small-string
optimization (static allocation) is controlled by a template
argument (cf. small_vector).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Fri, 11 Dec 2015 21:19:57 +0000 (16:19 -0500)]
librgw: fix last enumeration bug, move tracing to debug
The last issue with enumeration was caused by inserting objects
into handle cache with their key-ified full_object_name()--this
should be the base name.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Fri, 11 Dec 2015 02:33:01 +0000 (21:33 -0500)]
librgw: incremental listing fixes
* remove path in RGWStatLeafRequest--use bool is_dir instead
* remove unused uri in RGWListBucketRequest
* more dbg prints; problem w/final segment in obj_rec
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Thu, 10 Dec 2015 23:13:27 +0000 (18:13 -0500)]
librgw: incremental enumeration progress
As of this change, enumeration is -almost- correct, but the
rules for name, object_name(), and full_object_name() need to be
checked.
The rationale for storing just the last component name in dirents
means that name can't ever have the form "foo/bar" (make sure it
doesn't). But there may still be code which assumes name ==
object_name() OR assumes object_name() is hashable (it isn't, that
has to be the full object name).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Thu, 10 Dec 2015 21:12:37 +0000 (16:12 -0500)]
librgw: namespace bugfixes
1. stat_leaf() and stat_bucket() require RGWFileHandle::FLAG_CREATE
2. refactor RGWStatLeafRequest on the model of RGWListBucketRequest
3. fix uses of is_object() which meant is_file()
4. fix parent_fh passing
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Thu, 10 Dec 2015 01:25:22 +0000 (20:25 -0500)]
librgw: unify result cases in rgw_mkdir
In both paths, instantiate a file handle iff the remote op
succeeded.
If this operation lost a race to create an object with the same
name, we cannot correctly return a valid result and handle in all
cases, so don't attempt it.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Wed, 9 Dec 2015 22:24:04 +0000 (17:24 -0500)]
librgw: RGWListBucketRequest and related bugfixes
1. instantiate handles as required in RGWLibFS::stat_bucket
2. fix bucket_name() when bucket
3. fix full_object_name(), which should not return paths w/buckets
by default (allow doing so by passing non-default depth)
4. fix uri, prefix, and delimiter logic
5. add logic to parse and forward common_prefixes
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Wed, 9 Dec 2015 17:47:34 +0000 (12:47 -0500)]
librgw: annotate unlink's rgw_lookup
An rgw_lookup w/RGW_LOOKUP_FLAG_NONE will return an existing (hence
validated, modulo CREATE races) handle, but fail with ENOENT if none
exists (which is the desired behavior).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Tue, 8 Dec 2015 21:18:50 +0000 (16:18 -0500)]
librgw: reimplement RGWLibFS::stat_leaf
Now the code attempts to read an object header (which may be cached),
and if this fails, uses RGWStatLeafRequest to find a potential matching
file or directory leaf.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Mon, 7 Dec 2015 14:38:58 +0000 (09:38 -0500)]
librgw: variant handle subtype and marker cache
RGWFileHandle gets a type-specific variant part. For the directory
variant, define a flat_map of cookie to marker. Hash dirent names and
update the marker_cache in readdirs.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Thu, 12 Nov 2015 15:32:12 +0000 (10:32 -0500)]
librgw: forge Unix attrs in rgw_create
Possibly RGW vnodes should cache Unix attrs, it would reduce the
latency of (frequent) getattr calls, and eventually lines up with
more materialized Unix attributes.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
projects / ceph.git / log