]> git-server-git.apps.pok.os.sepia.ceph.com Git - radosgw-agent.git/log
radosgw-agent.git
11 years agoMerge pull request #21 from ktdreyer/rpm-dist
Alfredo Deza [Thu, 22 Jan 2015 13:04:05 +0000 (08:04 -0500)]
Merge pull request #21 from ktdreyer/rpm-dist

packaging: use dist tag in RPM Release field

11 years agopackaging: use dist tag in RPM Release field 21/head
Ken Dreyer [Wed, 21 Jan 2015 21:47:55 +0000 (14:47 -0700)]
packaging: use dist tag in RPM Release field

This change adds ".el6", ".el7", etc to the end of the RPM filename.

More information about the dist tag can be found at
http://fedoraproject.org/wiki/Packaging:DistTag

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agoMerge pull request #19 from ceph/wip-10584
Josh Durgin [Tue, 20 Jan 2015 16:54:10 +0000 (08:54 -0800)]
Merge pull request #19 from ceph/wip-10584

fix exception attribute errors in sync.py

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
11 years agofix exception attribute errors in sync.py 19/head
Alfredo Deza [Tue, 20 Jan 2015 16:30:01 +0000 (11:30 -0500)]
fix exception attribute errors in sync.py

Signed-off-by: Alfredo Deza <adeza@redhat.com>
11 years agoMerge pull request #18 from ceph/wip-10569
Josh Durgin [Mon, 19 Jan 2015 17:39:05 +0000 (09:39 -0800)]
Merge pull request #18 from ceph/wip-10569

exceptions refactoring

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
11 years agofix the worker exception tests 18/head
Alfredo Deza [Mon, 19 Jan 2015 16:58:34 +0000 (11:58 -0500)]
fix the worker exception tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
11 years agoadd the worker exceptions into their own module
Alfredo Deza [Mon, 19 Jan 2015 16:58:15 +0000 (11:58 -0500)]
add the worker exceptions into their own module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
11 years agoget worker exceptions into the exceptions module
Alfredo Deza [Mon, 19 Jan 2015 16:57:58 +0000 (11:57 -0500)]
get worker exceptions into the exceptions module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
11 years agoupdate tests for the new module changes
Alfredo Deza [Mon, 19 Jan 2015 16:06:22 +0000 (11:06 -0500)]
update tests for the new module changes

Signed-off-by: Alfredo Deza <adeza@redhat.com>
11 years agocreate an exceptions module
Alfredo Deza [Mon, 19 Jan 2015 16:05:54 +0000 (11:05 -0500)]
create an exceptions module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
11 years agoremove exceptions and errors from client onto exceptions.py
Alfredo Deza [Mon, 19 Jan 2015 16:05:00 +0000 (11:05 -0500)]
remove exceptions and errors from client onto exceptions.py

Signed-off-by: Alfredo Deza <adeza@redhat.com>
11 years agoadd tests for listing objects in buckets with real http body from responses
Alfredo Deza [Thu, 15 Jan 2015 19:28:02 +0000 (14:28 -0500)]
add tests for listing objects in buckets with real http body from responses

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoMerge pull request #17 from ktdreyer/spec-cleanup
Josh Durgin [Mon, 12 Jan 2015 23:45:32 +0000 (15:45 -0800)]
Merge pull request #17 from ktdreyer/spec-cleanup

packaging: RPM .spec cleanup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
11 years agopackaging: use full PyPI URL for RPM Source tarball 17/head
Ken Dreyer [Mon, 12 Jan 2015 21:22:44 +0000 (14:22 -0700)]
packaging: use full PyPI URL for RPM Source tarball

Alfredo has published the radosgw-agent package to PyPI today. Add the
full URL.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agopackaging: add missing build deps to RPM
Ken Dreyer [Mon, 12 Jan 2015 20:37:56 +0000 (13:37 -0700)]
packaging: add missing build deps to RPM

In order to build the radosgw-agent package in mock, the python-devel
and python-setuptools packages must be present in the buildroot.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agopackaging: remove RPM defattr setting
Ken Dreyer [Mon, 12 Jan 2015 20:36:39 +0000 (13:36 -0700)]
packaging: remove RPM defattr setting

The %defattr setting in the %files list is only necessary for very old
versions of RPM.  For RHEL 6 and later, it's fine to leave it out.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agopackaging: remove RPM %clean section
Ken Dreyer [Mon, 12 Jan 2015 20:35:38 +0000 (13:35 -0700)]
packaging: remove RPM %clean section

The %clean section is only necessary for very old versions of RPM. For
RHEL 6 and later, it's fine to leave it out.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agopackaging: simplify RPM %setup section
Ken Dreyer [Mon, 12 Jan 2015 20:34:18 +0000 (13:34 -0700)]
packaging: simplify RPM %setup section

Remove duplicated -n settings to %setup. There were two that both did
the same thing, and neither one is needed, since %setup unpacks
%{name}-%{version} by default.

Make %setup run quietly (with -q). This fixes an rpmlint warning.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agopackaging: add RPM %description
Ken Dreyer [Mon, 12 Jan 2015 20:31:49 +0000 (13:31 -0700)]
packaging: add RPM %description

Add a simple description to the RPM packaging. This phrase was
more-or-less taken from the docs in ceph.git.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agopackaging: use upper-case URL tag in RPM
Ken Dreyer [Mon, 12 Jan 2015 20:25:04 +0000 (13:25 -0700)]
packaging: use upper-case URL tag in RPM

Either "Url:" or "URL:" works, but the Fedora project has standardized
on "URL:"

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agopackaging: rm unnecessary RPM tags
Ken Dreyer [Mon, 12 Jan 2015 19:26:34 +0000 (12:26 -0700)]
packaging: rm unnecessary RPM tags

The BuildRoot tag is only needed on RHEL 5. RHEL 6 and newer are fine
without this setting.

The %{_prefix} macro is already /usr by default.

The Vendor tag is something that should be left undefined in the
packaging .spec itself. Ordinarily this is added during the build by the
build system.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agopackaging: rm unnecessary RPM macro definitions
Ken Dreyer [Mon, 12 Jan 2015 19:23:56 +0000 (12:23 -0700)]
packaging: rm unnecessary RPM macro definitions

Remove the extraneous macro definitions, %{name}, %{version},
%{unmangled_version}, %{release}. RPM automatically sets these when we
use the Name, Version, and Release tags.

Discussed %{unmangled_version} in #ceph-devel today; not sure where that
one came from.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
11 years agocreate a changelog v1.2
Alfredo Deza [Mon, 12 Jan 2015 20:16:07 +0000 (15:16 -0500)]
create a changelog

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoupdate author email
Alfredo Deza [Mon, 12 Jan 2015 19:56:46 +0000 (14:56 -0500)]
update author email

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoMerge pull request #16 from ktdreyer/fixes-from-v1.2
Josh Durgin [Mon, 12 Jan 2015 19:22:08 +0000 (11:22 -0800)]
Merge pull request #16 from ktdreyer/fixes-from-v1.2

#10518: add fixes from v1.2

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
11 years agoUpdate Debian Changelog. 16/head
Sandon Van Ness [Wed, 2 Apr 2014 19:25:17 +0000 (12:25 -0700)]
Update Debian Changelog.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
(cherry picked from commit cc845481a60d6d0da92879e5ef411d8d89c572f3)

11 years agoBranch for 1.2 release.
Sandon Van Ness [Wed, 2 Apr 2014 18:16:55 +0000 (11:16 -0700)]
Branch for 1.2 release.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
(cherry picked from commit 17fa0f7dc85f658547dd79a5416f55fa709a4956)

11 years agoMerge pull request #15 from ceph/wip-socket-leak
Alfredo Deza [Mon, 5 Jan 2015 18:22:29 +0000 (13:22 -0500)]
Merge pull request #15 from ceph/wip-socket-leak

client: workaround socket leak

11 years agoclient: workaround socket leak wip-socket-leak 15/head
Josh Durgin [Tue, 23 Dec 2014 06:16:36 +0000 (01:16 -0500)]
client: workaround socket leak

Recreate the boto connection object after every 512 requests in an
attempt to workaround a socket leak when the remote end closes the
connection. Boto does not seem to be causing httplib to close() the
sockets appropriately in some cases. I'm not sure exactly where
the leak is occurring, but forcing boto to reinitialize its
connection pool like this avoids it.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoMerge pull request #14 from ceph/wip-full-sync
Alfredo Deza [Fri, 19 Dec 2014 19:13:30 +0000 (13:13 -0600)]
Merge pull request #14 from ceph/wip-full-sync

Remove dependency on requests and other full sync fixes

11 years agoworker: check op state for progress on any HTTP error 14/head
Josh Durgin [Thu, 18 Dec 2014 05:49:35 +0000 (21:49 -0800)]
worker: check op state for progress on any HTTP error

500 can be caused by an fcgi timeout when the operation still ends up
succeeding. It doesn't hurt to check for op state being in progress in
general in case other error codes happen but don't indicate actual
failure of the copy, since wait_for_object fails immediately if the
op state is not in progress.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agosync: use ' ' as the default shard so retries can be stored
Josh Durgin [Thu, 18 Dec 2014 05:28:38 +0000 (21:28 -0800)]
sync: use ' ' as the default shard so retries can be stored

The replica log api doesn't allow empty markers, but we may need to
store retries in the replica log even if there are no log entries. Use
' ' as a marker instead, since it is before all possible markers.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agofull data sync: treat 404 from bucket list as success
Josh Durgin [Thu, 18 Dec 2014 05:20:21 +0000 (21:20 -0800)]
full data sync: treat 404 from bucket list as success

Since results are paged, the exception may not occur immediately, so
add a new type to distinguish it from other client exceptions, and
catch it at any point during full sync of a bucket.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoworker: ignore 404 when removing op_state
Josh Durgin [Thu, 18 Dec 2014 00:16:14 +0000 (16:16 -0800)]
worker: ignore 404 when removing op_state

It's fine if the gateway deleted it first.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agore-purpose urlencode to craft the query string for us without quoting again
Alfredo Deza [Thu, 18 Dec 2014 21:18:42 +0000 (15:18 -0600)]
re-purpose urlencode to craft the query string for us without quoting again

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agocreate a test to check/prevent double quoting of urls
Alfredo Deza [Thu, 18 Dec 2014 21:16:58 +0000 (15:16 -0600)]
create a test to check/prevent double quoting of urls

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoadd get_bucket_list test
Alfredo Deza [Thu, 18 Dec 2014 19:24:12 +0000 (14:24 -0500)]
add get_bucket_list test

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agotests for get API calls
Alfredo Deza [Thu, 18 Dec 2014 19:07:37 +0000 (14:07 -0500)]
tests for get API calls

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agopep8/style changes to client.py
Alfredo Deza [Thu, 18 Dec 2014 18:58:03 +0000 (13:58 -0500)]
pep8/style changes to client.py

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agohttp response does not have .raw anymore
Alfredo Deza [Thu, 18 Dec 2014 00:40:05 +0000 (19:40 -0500)]
http response does not have .raw anymore

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoMerge pull request #13 from ceph/wip-full-sync
Alfredo Deza [Mon, 8 Dec 2014 13:34:22 +0000 (08:34 -0500)]
Merge pull request #13 from ceph/wip-full-sync

Bug fixes for full sync and a few other data sync cases

11 years agoclient: return object names instead of boto resultsets 13/head
Josh Durgin [Sat, 6 Dec 2014 01:35:02 +0000 (17:35 -0800)]
client: return object names instead of boto resultsets

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoAdd tests for boto_call decorator
Josh Durgin [Tue, 25 Nov 2014 23:25:34 +0000 (15:25 -0800)]
Add tests for boto_call decorator

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoMore tests for wait_for_object()
Josh Durgin [Tue, 25 Nov 2014 01:21:43 +0000 (17:21 -0800)]
More tests for wait_for_object()

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoImmediately fail when op state is not found
Josh Durgin [Tue, 25 Nov 2014 00:26:27 +0000 (16:26 -0800)]
Immediately fail when op state is not found

There's a very narrow window in which our connection sending the copy
request could fail before radosgw records the op state, and we query
the op state in that time. Retrying later at that point is fine, and
avoids a long timeout if there is a bug causing us to reach this point
without the op state ever being recorded. This race hasn't been
observed in practice.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoTreat HTTP 500 as any other HTTP error for a copy request
Josh Durgin [Mon, 24 Nov 2014 23:59:20 +0000 (15:59 -0800)]
Treat HTTP 500 as any other HTTP error for a copy request

We still want to retry later, since 500 could be the gateway timing
out, or it being upgraded to fix a bug, etc.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoclient: escape copy source header
Yehuda Sadeh [Mon, 10 Nov 2014 21:29:07 +0000 (13:29 -0800)]
client: escape copy source header

Need to esacpe the x-amz-copy-source header.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
11 years agoworker: process full sync for buckets with no log
Josh Durgin [Mon, 10 Nov 2014 23:39:22 +0000 (15:39 -0800)]
worker: process full sync for buckets with no log

This way we catch objects that were uploaded before zones or data logs
were set up. If there are actually no objects in the bucket, the
bucket listing will tell us that.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoclient: don't hide boto return values
Josh Durgin [Mon, 10 Nov 2014 22:14:11 +0000 (14:14 -0800)]
client: don't hide boto return values

The decorator for translating boto exceptions needs to return
explicitly, or it will lose the return value of the original function,
and cause full sync to fail for objects that aren't in the bucket
index log.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
11 years agoadd a test that ensures that wait_for_object is not called if object is not found wip-10102 11/head
Alfredo Deza [Wed, 19 Nov 2014 20:23:59 +0000 (15:23 -0500)]
add a test that ensures that wait_for_object is not called if object is not found

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoby default object is not found, changing only when sync_object_intra_region() works
Alfredo Deza [Wed, 19 Nov 2014 20:23:04 +0000 (15:23 -0500)]
by default object is not found, changing only when sync_object_intra_region() works

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoraise SyncFailed on transient errors
Alfredo Deza [Wed, 19 Nov 2014 20:09:50 +0000 (15:09 -0500)]
raise SyncFailed on transient errors

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoallow a stringified version of the http status code
Alfredo Deza [Wed, 19 Nov 2014 20:09:09 +0000 (15:09 -0500)]
allow a stringified version of the http status code

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agotests for sync_object in worker.py
Alfredo Deza [Wed, 19 Nov 2014 20:08:42 +0000 (15:08 -0500)]
tests for sync_object in worker.py

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoconfigure logging for testing
Alfredo Deza [Wed, 19 Nov 2014 20:08:24 +0000 (15:08 -0500)]
configure logging for testing

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agolog the exception as error
Alfredo Deza [Wed, 19 Nov 2014 14:03:16 +0000 (09:03 -0500)]
log the exception as error

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoMerge pull request #10 from ceph/wip-coverage
Josh Durgin [Mon, 17 Nov 2014 21:13:59 +0000 (22:13 +0100)]
Merge pull request #10 from ceph/wip-coverage

add parametrization for endpoint tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
11 years agoadd parametrization for endpoint tests 10/head
Alfredo Deza [Mon, 17 Nov 2014 17:20:54 +0000 (12:20 -0500)]
add parametrization for endpoint tests

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
11 years agoclient: fix unicode handling
Josh Durgin [Fri, 25 Apr 2014 23:52:16 +0000 (16:52 -0700)]
client: fix unicode handling

Use unicode format strings, and don't url-encode the object copy path
before passing it to requests, which will url-encode it for us.

Partially-fixes: #8202
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
11 years agologrotate: don't rotate empty logs
Josh Durgin [Thu, 27 Mar 2014 18:54:50 +0000 (11:54 -0700)]
logrotate: don't rotate empty logs

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
11 years agodebian: fix init script installation
Josh Durgin [Sat, 22 Mar 2014 03:44:11 +0000 (20:44 -0700)]
debian: fix init script installation

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
11 years agopackaging: fix up spec file for non-python files
Josh Durgin [Sat, 22 Mar 2014 12:10:28 +0000 (05:10 -0700)]
packaging: fix up spec file for non-python files

Correct the installation of the binary, logrotate conf, and init script.
Stop using INSTALLED_FILES. Instead explicitly list files and glob the
python package, as recommended by Fedora's python packaging guide.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
11 years agocli: add a default log file
Josh Durgin [Wed, 19 Mar 2014 11:31:29 +0000 (04:31 -0700)]
cli: add a default log file

Now that the packages include /var/log/ceph/radosgw-agent, default to
storing logs there, named after the configuration file or
radosgw-agent.log if no config file was used.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
11 years agodebian: depend on python 2.6 or higher
Josh Durgin [Wed, 19 Mar 2014 11:30:02 +0000 (04:30 -0700)]
debian: depend on python 2.6 or higher

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
11 years agopackaging: add logrotate config
Josh Durgin [Wed, 19 Mar 2014 10:32:07 +0000 (03:32 -0700)]
packaging: add logrotate config

There's no need to signal running agents since they use
a WatchedFileHandler that detects rotation itself.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
11 years agopackaging: add init scripts
Josh Durgin [Wed, 19 Mar 2014 10:11:50 +0000 (03:11 -0700)]
packaging: add init scripts

create log, run, and etc dirs as well

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
11 years agopackaging: add dependency on python-requests
Josh Durgin [Wed, 19 Mar 2014 09:27:32 +0000 (02:27 -0700)]
packaging: add dependency on python-requests

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>