rgw: change configurable

rgw keystone accepted roles

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: fix build

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: use a different boost thread library

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: keystone tokens revocation thread

Use a thread to invalidate revoked tokens.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

json_spirit: thread safe

apparently the library is not thread safe by default.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: decode pki tokens

If we can decode pki tokens then we don't need to send them to
the Keystone server for verification.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: invalidate revoked tokens

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: keep pki tokens by hash

and also fix the token parsing (needed after the json
parser fix).

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: decode revocation message

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

crypto: add cms utility function

Taken from a libnss tool (cmsutil), stripped code so that we
only get the decode functionality that we need.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: parse keystone token expiration

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: cache keystone tokens

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: map keystone tenant id to rgw user

Instead of mapping the tenant name. Keep the name as the display_name.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: handle keystone roles

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: auto create rgw user for new keystone users

We now also read the user info when authenticating the keystone
user.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: handle keystone tokens

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: extend rgw json parser api

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: refactor curl functionality

Move curl stuff into its own class, use it in swift token validation.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: configurable swift auth url

This is relevant when using external swift v1 authentication. The
url was hard coded, now it's configurable.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: better error responses, logging for specific error cases

Also, adjusting return values

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: POST fixes, policy range, policy expiration

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: handle expiration and content length policy conditions.

Signed-off-by: caleb miles <caleb.miles@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: implement POST key can use ${filename}

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: return correct responses with POST

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: handle success_action_redirect

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: handle policy as case insensitive where required

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: handle missing content length in POST

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: handle x-ignore- in POST request

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: verify that request variables are in policy

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

json_spirit: update grammar

Support a more relaxed grammar, where last entity in an array
or an object can have a comma trailing.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: add policy decoding code

merge in changes from commit 57668c14 by caleb

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: basic policy functionality

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: add a test tool for json parser

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: adjustments for 100-continue

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: handle content_type, meta

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: helper functions to parse multipart form

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: Create JSON parser

Create JSON object and parser classes similar to the XMLObj and
RGWXMLParser based on JSON Spirit.

Signed-off-by: caleb miles <caleb.miles@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: Implement POST object to publicly writeable buckets.

The S3 API supports object upload using POST requests, for browser based
uploads, the specification is described at:

http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectPOST.html

In response to feature request 771:

http://tracker.newdream.net/issues/771

Signed-off-by: caleb miles <caleb.miles@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rbd: don't issue usage on errors

Change bare calls to usage() to an informative targeted error message
Remove all calls to usage() except when requested with -h/--help
Regularize all errors to start with rbd:
Remove a few commented cerrs, wrap cerr calls at 80 cols

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Added radosgw temp remove.

fixes: #3278

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Minor clean-up.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: minor clean-up.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

rgw: fix compilation

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge branch 'master' of https://github.com/ceph/ceph

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

Merge branch 'wip-rbd-openstack-doc'

mon: drop command replies on paxos reset

If paxos resets, do not send the reply for the commit we were waiting for;
let the command be reprocessed and re-proposed.

Among other things, this could lead to nondeterministic results for
'ceph osd create <uuid>'.

Signed-off-by: Sage Weil <sage@inktank.com>

doc: Editing and cleanup.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

rbd cli tests: copy.sh was looking for old version of rbd ls -l output

Signed-off-by: Dan Mick <dan.mick@inktank.com>

clean up of rbd functionality script

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

cleaned up the perl module

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

cleaned up the script

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>

radosgw-admin: don't try to parse date if wasn't specified

This fixes a regression following the merge of wip-admin-rest.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

ceph-debugpack: updates

- avoid copying data around; tar things directly into the tgz
- 'ceph report' instead of all the little bits
- unrotated logs only
- ensure target doesn't already exist

Signed-off-by: Sage Weil <sage@inktank.com>

doc: separate config from I/O in openstack diagram

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

doc: clarify rbd openstack docs

* give upstart example
* restart all relevant services
* explain OpenStack terminology more

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

gitignore: Update to match packaging changes and additions, rest-bench addition.

Signed-off-by: Tommi Virtanen <tv@inktank.com>

configure.ac: Typo in human-readable description.

Signed-off-by: Tommi Virtanen <tv@inktank.com>

mon: Monitor: resolve keyring option to a file before loading keyring

Otherwise our keyring default location, or any other similarly formatted
location, will be taken as the actual location for the keyring and fail.

Reported-by: tziOm (at) #ceph
Fixes: 3276
Backport: argonaut

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

doc: simplify wording for rbd-openstack config

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge remote-tracking branch 'gh/wip-gtest-libcephfs'

Reviewed-by: Sage Weil <sage@inktank.com>

doc: Add doxygen style documentation for libcephfs

As well as doxygen doc, we document a few parts of
the interface that are otherwise undocumented and
don't match the standard calls.

Signed-off-by: Sam Lang <sam.lang@inktank.com>

test: Refactored testceph.cc into gtest framework

Moved all the functionality tests for the libcephfs
API into the gtest framework.  Also adds tests for
directories to improve test coverage of the libcephfs
interfaces.

Signed-off-by: Sam Lang <sam.lang@inktank.com>

test: Allow readdir test to be repeated

Use test dir/files that include the pid to avoid
hitting errors when running the same test twice.

Signed-off-by: Sam Lang <sam.lang@inktank.com>

libcephfs: Check that buffer is non-null

Signed-off-by: Sam Lang <sam.lang@inktank.com>

Merge remote-tracking branch 'gh/wip-client-stale'

Reviewed-by: Sage Weil <sage@inktank.com>

mds: Fix removexattr ENODATA error

If a client sets an xattr and then attempts to remove in a short
time window, the xattr may not have been committed to the journal
and will only be set on the projected xattrs of the inode.  This
results in a removexattr returning ENODATA at present.  The fix
here checks the xattr name in the projected xattrs, and only
returns ENODATA in the case where the name doesn't exist in that
map either.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

osdmap: fix null deref in dump

CID 731938 (#1 of 1): Dereference after null check (FORWARD_NULL)
At (15): Passing null pointer "f" to function "ceph::Formatter::dump_int(char const *, int64_t)", which dereferences it. (The dereference happens because this is a virtual function call.)

Signed-off-by: Sage Weil <sage@inktank.com>

osdmap: fix osdmap dump null check

CID 731939 (#1 of 1): Dereference after null check (FORWARD_NULL)
At (5): Passing null pointer "out" to function "std::operator << <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> > &, char const *)", which dereferences it.

Broken by 13b841c74605d67a172d787079bcef007d1a8fa5.

Signed-off-by: Sage Weil <sage@inktank.com>

mds: note fall-thru to please coverity

CID 731940 (#1 of 1): Missing break in switch (MISSING_BREAK)
This case (value 23) is not terminated by a 'break' statement.
1719    case CEPH_MSG_CLIENT_RECONNECT:
1720      ALLOW_MESSAGES_FROM(CEPH_ENTITY_TYPE_CLIENT);
The above case falls through to this one.

Signed-off-by: Sage Weil <sage@inktank.com>

test_cfuse_cache_invalidate: check for errors

CID 731941 (#1 of 2): Argument cannot be negative (NEGATIVE_RETURNS)
At (6): "fd" is passed to a parameter that cannot be negative.

Signed-off-by: Sage Weil <sage@inktank.com>

mon/Elector: fix leak on monmap from peer

CID 731942 (#1 of 1): Resource leak (RESOURCE_LEAK)
At (12): Variable "peermap" going out of scope leaks the storage it points to.

Signed-off-by: Sage Weil <sage@inktank.com>

test_cfuse_cache_invalidate: zero buffer

CID 731943 (#1 of 1): Uninitialized scalar variable (UNINIT)
At (5): Using uninitialized element of array "buf" when calling "pwrite(int, void const *, size_t, __off64_t)".

Signed-off-by: Sage Weil <sage@inktank.com>

FileJournal: ignore osd journal size for block devices

Using part of a block device doesn't make much sense. This makes the
common case, where you partition a disk to act as several journals,
easier to configure.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

rgw: switch perm to uint32_t

Should fix a warning.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: flexible attr fields

Fixes: #3114
Instead of having a few hard coded attrs that are
special cased, make it more generic. This allows supporting
more header fields, such as cache-control, expires, etc.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: get bucket logging stub

Trivial implementation that just returns that
bucket logging is not enabled.

Fixes: #3225
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: check_disk_state also updates other object metadata

When we do a check_disk_state() on an object that is not complete,
we need to also update etag, content_type, owner, and display_name.
Otherwise, we'd end up missing that from the bucket index. In
practice this only affects swift, as only there we're actually
looking at that info (when listing a container).

Fixes: #3234
Backport: argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'wip-rgw-coverity2'

rgw: set null termination explicitly

Make sure that string is null terminated.
Fixes coverity issue.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

cls_rgw: return returned ret code

Wasn't setting the ret code. A coverity issue.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: add missing parantheses

.. around logic operations at process_intent_log().
Another coverity fix.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: check NULL before dereference

Another coverity fix.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw-admin: pass object by reference

Large object should be passed by reference (coverity fix).

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: fix leaks in certain error paths

Fixes a few coverity issues

Conflicts:
src/rgw/rgw_rados.cc

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: fix (im)possible leak

Fixes coverity issue

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: catch possible exception

Fix coverity issue 717168.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: initialize member variables at req_state

Fixes more coverity issues

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: initialize data members

Fixes a few coverity issues.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: RGWObjState() initialize memebers

Initialize members at constructor. Fixes coverity issues.

Conflicts:
src/rgw/rgw_rados.h

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: initialize members at RGWRados()

fixes coverity issues

Conflicts:
src/rgw/rgw_rados.h

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

cls_rgw: check pointer before dereferencing

Fixes coverity issue 719577

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: multiple coverity fixes of uninitialized fields

Fields were not initialized explicitly by constractor.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: trivial fix

fix coverity issue 728052

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: change size_t to ssize_t for string.rfind() result

fixes coverity bug 728053

string.rfind() may return negative value, switch to ssize_t instead
of size_t.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: fix resource leak

coverity bug 728055

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Revert "admin/build-doc: Use installed Sphinx and its dependencies, when possible."

This reverts commit c65ffe9f7f27bc0fe5afab36e9ef0de42bc3c74c.

oneiric, where the doc gitbuilder runs, doesn't have the
--system-site-packages option; let's get docs back and sort this out
offline.  (virtualenv 1.6.4-0ubuntu1 on oneiric, 1.7.1.2-1 on precise)

Merge branch 'wip-admin-rest'

Conflicts:
doc/radosgw/index.rst
src/rgw/rgw_admin.cc
src/rgw/rgw_main.cc
src/rgw/rgw_op.cc
src/rgw/rgw_op.h
src/rgw/rgw_rest.cc
src/rgw/rgw_rest.h
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_rest_s3.h
src/rgw/rgw_rest_swift.cc
src/rgw/rgw_rest_swift.h
src/rgw/rgw_swift_auth.cc
src/rgw/rgw_swift_auth.h

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: virtual bucket affects resource lookup

Removed special casing of virtual bucket handling
and just prepending it to the URI. So now virtual
bucket also affects admin and swift* protocols.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>