Matt Benjamin [Sun, 4 Oct 2015 18:51:52 +0000 (14:51 -0400)]
librgw: fix 2 bugs in RGWListBuckets, remove scaffolding
1. the prior commit had authorize cognate with S3
1.1 which may be right
1.2 but maybe not for each request
1.3 and lacked priming/forged auth data
The forging of auth data is now easy, but the interesting part is
caching and efficiently attaching it so I paused, and
2. disabled auth, which was now overriding and anonymizing all
requests (good!)
3. fixed another instance of reinterpret_cast where (one of)
dynamic_cast or a saved pointer descendant request (avoids a runtime
check, so will add later) is needed
4. fixed bad offset argument to rgw_readdir
The list bucket facility now works, modulo non-auth.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Fri, 2 Oct 2015 20:30:21 +0000 (16:30 -0400)]
librgw: remove RGWLibRequestEnv
The REST cognates of this class largely deal with HTTP, and is 1-1
w/request, client-io, op, etc. At worst, its functions can be absorbed
into one of those (client-io, initially).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Fri, 2 Oct 2015 16:50:47 +0000 (12:50 -0400)]
librgw: wire up request initializers
There's some tricky overloading induced by multiple inheritance,
also this design makes repeatedly shared handles in current framework
very visible.
The most important non-boilerplate is in descendant RGWRequest
initializers (rgw_lib.h), which need to set up req_state appropriately
for their action/op.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Thu, 1 Oct 2015 22:26:37 +0000 (18:26 -0400)]
librgw: re-derive RGWLibRequest
Derive RGWLibRequest from RGWRequest, RGWHandler.
This strategy defines the dialect of ops following the derivation
pattern for RGWLib actions to be their handler.
On the negative side, RGWLibRequest has struct req_state* s from two
ancestors. Since we don't use virtal inheritance, the only overhead is
the extra copy, and need for disambiguation in the direct descendants
(this is only 1 method in 1 file, atm).
The motivation is as follows. RGWHandler currently has 3 responsibilities:
1) op-chasing; 2) to hold authorize() and read_permissions() methods
3) to initialize req_state
RGWLib actions have no use for the op-chasing behavior, and authorize()
and read_permissions() methods are arguably more naturally expressed as
methods on the actions. Meanwhile struct req_state appears to be a
candidate for refactoring into the RGWRequest class hierarchy, which would
mean that initalization probably should move from hanlder to request, as
well. Refactoring req_state is would be quite intrusive, so this change
defers that for later, but keeps the 1-1 mappings from request to
handler at 0 cost.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
The motivation for this change is to allow a (maybe) transitional
refactoring stage which allows to implement the RGWHandler
interface, without implementing get_handler() or get_op(), put_op(),
or op_*().
Prior refactoring (which could be reversed) moved authorize() and
read_permissions() into the precursor RGWHandler_ObjStore
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Wed, 30 Sep 2015 21:34:50 +0000 (17:34 -0400)]
librgw: incremental
* move RGWLibRequest decls into rgw_file.h
* revise include guard token in rados/rgw_file.h (avoid conflict)
* save CephContext* in RGWLibRequest
* initialize RGWLibRequest cct in ctor, req_state in process_request
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Thu, 10 Sep 2015 17:21:17 +0000 (13:21 -0400)]
refactor RGWClientIO
In order to support direct/library clients, we want to
more fully decouple RGWOp processing from client HTTP
processing, since in the direct case, there is no HTTP
client.
This change tackles essentially just RGWClientIO, by moving
stream operations to a new subclass RGWStreamIO, and updating
the stream backends (e.g., SWIFT, S3) and REST handlers
accordingly.
In addition, the RGWLib backend is incrementally updated, with
RGWLibIO still deriving from the base RGWClientIO. The RGWLib
path is incomplete as of this change, but is moving in the
direction of its own process_request call path (incomplete,
partly due to header conflicts that will be resolved in
subsequent commits).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Tue, 1 Sep 2015 18:20:02 +0000 (14:20 -0400)]
librgw: rgw shared library
Refactor rgw_a and radosgw_srcs, and add librgw_srcs, permitting
librgw to contain a smaller subset of objects, and in particular,
not link with fcgi or civitweb.
Fix aliasing of librgw_t rgw in test_rgw_file. Also remove global_init
there, since it can't do the right thing. Instead, assign g_ceph_context
(ref'd) in librgw_create, if not present.
(The current rgw codebase has many dependencies on g_ceph_context to
trivially remove--it would be easier to remove it entirely, and that seems
likely to happen eventually.)
We link librgw as SHARED, with PRIVATE dependencies. The test_librgw_file
link test demonstrates that librgw.a can be linked with no Ceph library
dependencies except librados (and its dependencies, platform dependencies,
etc).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sun, 30 Aug 2015 20:28:48 +0000 (16:28 -0400)]
librgw: compile and implement more of rgw_file
The rgw_file body wasn't being built, so had some impossible
constructs. More had been added by the commit which renamed
struct nfs_handle to struct rgw_file_handle.
Several key RGWLib methods hadn't been implemented (in progress).
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Orit Wasserman [Tue, 12 May 2015 10:12:03 +0000 (12:12 +0200)]
rgw: introduce libradosgw
This change exposes the radosgw service as a dynamic
library, and also introduces a file-oriented view of the
RGW corpus as a namespace (e.g., for interop with NFS
clients).
The change concatenates the following commits:
* rgw: add rgw_file.h
* rgw: add RGWLibRequest, RGWLibFrontend and RGWLibProcess
* rgw: Add user command interface for librgw
* user_commands
* Implement gen_request
* Install librgw.a
* add rgw_request.cc to Makefile.am
* Add empty API to librgw.h
* Add rgw_file.cc
* Make RGWLib a global object
* Implement lookup and readdir - wip
* Add rgw_rest_lib.cc
* Install rgw_file.h
* wip implement mount
* RGWLibIO
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Fri, 8 May 2015 18:07:01 +0000 (20:07 +0200)]
rgw: refactor rgw_main
This change is an extended version of work by Orit Wasserman
moving some request-processing class headers and definition to
their own files (RGWProcess, RGWProcessEnv, RGWProcessFrontend),
along with process_request.
I have taken the refactoring further, so as to remove all class
declarations and member definitions from rgw_main.cc, so that the
file retains just:
* the responsibility to parse arguments and supervise the daemon
* the responsibility to select from front-ends/configure the RGW service
* usage
I have made small adjustments:
* moving some processing out of headers, to remove circular deps
* removed unused headers from rgw_main.cc
* added editor boilerplate and tweaked include guards in some files
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Loic Dachary [Thu, 11 Feb 2016 11:47:55 +0000 (18:47 +0700)]
python: use pip instead of python setup.py
python setup.py develop may try to pull dependencies from the net and
has no way to collect them from the wheelhouse that was populated by
install-deps.sh. Use pip install -e instead
Kefu Chai [Thu, 11 Feb 2016 12:08:32 +0000 (20:08 +0800)]
tests: ceph-disk.sh: use "readlink -f" instead for fullpath
if $PATH has ".." in it, and the program happen to be located in
"..", `which program` will print `../program` instead of its fullpath,
so we should always use `readlink -f` for the fullpath.
Boris Ranto [Thu, 11 Feb 2016 11:03:06 +0000 (12:03 +0100)]
selinux: Allow log files to be located in /var/log/radosgw
We do suggest users to put their logs in /var/log/radosgw in the
documentation at times. We should also label that directory with
ceph_var_log_t so that ceph daemons can also write there.
The commit also updates the man page for this policy. This man page is
automatically generated by
* sepolicy manpage -p . -d ceph_t
and have not been reloaded in a while. Hence, it contains few more
changes than the new radosgw directory.
Jianjian Huo [Thu, 11 Feb 2016 01:44:33 +0000 (20:44 -0500)]
os/bluestore: fix a typo in SPDK path parsing
SPDK block path with bluestore is something like spdk:55cd2e404bd73932.
The size of the prefix to compare should be 5.
sizeof(SPDK_PREFIX)-1 returns 5, while sizeof(SPDK_PREFIX-1) returns 8.
Signed-off-by: Jianjian Huo <samuel.huo@gmail.com>
Jason Dillaman [Thu, 11 Feb 2016 00:06:02 +0000 (19:06 -0500)]
librbd: protect journal replay against overlapping writes
If fsx issues a back-to-back synchronous write, these will be
replayed as AIO writes. If object map is enabled, it's possible
for the two writes to commit to disk out-of-order if the first
write required an object map update.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Kefu Chai [Fri, 5 Feb 2016 07:56:55 +0000 (15:56 +0800)]
log: remove "boost/asio.hpp" from Log.h
graylog uses boost/asio.hpp which introduces the link-time dependency on
libboost_system to the compilation units which includes Log.h and
LogClient.h. so it appears that perfglue/heap_profiler.cc is referencing
libboost_system.so, and fails the cmake build of all tests which links
against tcmalloc.
in this change, we:
* remove unnecessary #includes from Graylog.h
* forward declare Graylog class, so that "Graylog.h" is not included in
any header files to avoid the link-time dependency pollution
projects / ceph.git / log