]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agorgw_file: pre-assign times 11335/head
Matt Benjamin [Wed, 21 Sep 2016 21:18:20 +0000 (17:18 -0400)]
rgw_file:  pre-assign times

Set unix timestamps based on RGW values for creation, modification
time for buckets, objects in stat requests.  Regard any saved
value of these in unix attributes as an overlay.

Fixes: http://tracker.ceph.com/issues/17367
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 969750e96380859a350e25015bc8cbf9bdf88512)

Fixes: http://tracker.ceph.com/issues/17394
8 years agorgw_file: fix set_attrs operation
Matt Benjamin [Tue, 20 Sep 2016 21:32:03 +0000 (17:32 -0400)]
rgw_file: fix set_attrs operation

The effective part of this change is to always set a value
for RGW_ATTR_UNIX_KEY1 (because it is expected later).

Secondarily, do not pass the address of the to-set attributes
buffer::list as remove attrs--this is confusing.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 08afb40b2812270a6af3cd1965e8c03c10392ace)

Fixes: http://tracker.ceph.com/issues/17393
8 years agorgw_file: implement rgw_setattr
Matt Benjamin [Thu, 14 Apr 2016 23:18:37 +0000 (19:18 -0400)]
rgw_file: implement rgw_setattr

Introduce a new RGWSetattrs RGWOp descendant, to create or replace
sets of attrs on buckets or objects.

This version of the change uses the standard RGWRADOS::set_attrs op
(we want attribute changes to (e.g.) sync with other changes).

Previous versions of this changed incorrectly masked the values
of st->st_ino in RGWFileHandle::stat(), now fixed.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 4de1c3c260265f821ebee842d49cb35bf49d8e4e)

Fixes: http://tracker.ceph.com/issues/17311
Fixes: http://tracker.ceph.com/issues/17332
8 years agorgw: remove duplicated calls to getattr
zhangweibing [Fri, 26 Aug 2016 02:49:48 +0000 (10:49 +0800)]
rgw: remove duplicated calls to getattr

Signed-off-by: Weibing Zhang <zhangweibing@unitedstack.com>
(cherry picked from commit 87b550ed2fba472d4ef2e1f771bb5f9fd248804f)

Fixes: http://tracker.ceph.com/issues/17327
8 years agorgw: rgw file fix bug of rgw_lookup can not exact match file name
Min Chen [Fri, 19 Aug 2016 05:02:50 +0000 (01:02 -0400)]
rgw: rgw file fix bug of rgw_lookup can not exact match file name

bug reproduce steps:
in nfs-client (connect to nfs-ganesha server):
1. mv file file.rename
2. stat file // get file attrs, not return with -ENOENT

the reason is that:
RGWStatLeafRequest does not exact match the file name,
just take file name as a prefix filter

Signed-off-by: Min Chen <chenmin@xsky.com>
(cherry picked from commit 9d813bafc8e197507457c58ab4f365ccdb7f3589)

Fixes: http://tracker.ceph.com/issues/17326
8 years agorgw: fix the wrong return value
Yan Jun [Tue, 23 Aug 2016 04:51:10 +0000 (12:51 +0800)]
rgw: fix the wrong return value

Here `parent->readdir` will return negative numbers When there is an error.
so we should just return it.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
(cherry picked from commit c979be7810793534715c24c6a9f7c8b0ed31de8d)

Fixes: http://tracker.ceph.com/issues/17325
8 years agorgw_file: restore local definition of RGWLibFS gc interval
Matt Benjamin [Wed, 17 Aug 2016 14:28:48 +0000 (10:28 -0400)]
rgw_file: restore local definition of RGWLibFS gc interval

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 31936caa09b46e86700faad969adfa9d43176206)

Fixes: http://tracker.ceph.com/issues/17323
8 years agorgw_file: explain semantics of RGWFileHandle::commit
Matt Benjamin [Tue, 23 Aug 2016 20:50:42 +0000 (16:50 -0400)]
rgw_file:  explain semantics of RGWFileHandle::commit

Add a comment explaining why the method currently returns 0
unconditionally.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit b1da1354320cc5f69277b225a293a03cc2a5054f)

8 years agoceph_timer: prefer using directive to typedef
Matt Benjamin [Tue, 23 Aug 2016 19:57:06 +0000 (15:57 -0400)]
ceph_timer: prefer using directive to typedef

This change replaces the existing member-hook typedef as well as
the new set-type typedefs, so committed separately.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit f7d4d418658d2091652033045abdb4bfbeb637aa)

8 years agolibrgw: add API version defines for librgw and rgw_file
Matt Benjamin [Tue, 23 Aug 2016 18:18:43 +0000 (14:18 -0400)]
librgw: add API version defines for librgw and rgw_file

This change borrows the major, minor+extra format used by libcephfs.
The version numbering is starting at 1,1,0 on the theory that the
implicit version at Jewel is 1,0,0.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 355ccc5ded7f0b459bca24bc8b504b41807c583d)

8 years agorgw file: allow RGWLibFS::write_completion_interval_s to be set in conf
Matt Benjamin [Wed, 17 Aug 2016 15:24:01 +0000 (11:24 -0400)]
rgw file: allow RGWLibFS::write_completion_interval_s to be set in conf

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 70dad0fba561edaf0bb4e746e29b92a4e9b014b2)

8 years agorgw_file: log on write completion events at level 10
Matt Benjamin [Wed, 17 Aug 2016 15:00:22 +0000 (11:00 -0400)]
rgw_file:  log on write completion events at level 10

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit d1e512d8c0e20df5a96ccb5f775a7a2ca123efbe)

8 years agorgw_file: add RGW_OPEN_FLAG_STATELESS as a synonym for V3
Matt Benjamin [Wed, 17 Aug 2016 14:52:58 +0000 (10:52 -0400)]
rgw_file: add RGW_OPEN_FLAG_STATELESS as a synonym for V3

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit da596eb3af69c2cdf191893eb44a8947662634cf)

8 years agorgw_file: implement stateless write completion timer
Matt Benjamin [Tue, 16 Aug 2016 22:17:53 +0000 (18:17 -0400)]
rgw_file: implement stateless write completion timer

Implements a temporal mechanism to enforce write completion for setups
which lack open state tracking (e.g., NFS3).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 2c83ed4bd43fcb5d5497151a157b1dc08022fed9)

8 years agorgw_file: fix/update rgw_create flag arguments
Matt Benjamin [Fri, 12 Aug 2016 20:52:51 +0000 (16:52 -0400)]
rgw_file: fix/update rgw_create flag arguments

Allow passing POSIX open flags as well as api call flags.  Needed
for NFS3 support.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 2d0ea2b4649e76cbbb87f44c57754d7ea9c46dce)

8 years agorgw_file: fix/update rgw_open flag arguments
Matt Benjamin [Fri, 12 Aug 2016 14:39:21 +0000 (10:39 -0400)]
rgw_file: fix/update rgw_open flag arguments

Allow passing POSIX open flags as well as api call flags.  Needed
for NFS3 support.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit ada29f7f8323be2301588941bcd5c60190b32a4b)

8 years agorgw file: remove busy-wait in RGWLibFS::gc()
Matt Benjamin [Tue, 9 Aug 2016 20:49:41 +0000 (16:49 -0400)]
rgw file: remove busy-wait in RGWLibFS::gc()

This is a background thread.  However, CPU is wasted.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit ca33241286f52d849dbde8092507131b8b1108cc)

Fixes: http://tracker.ceph.com/issues/17321
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
8 years agorgw_file: unlock() must precede out label
Matt Benjamin [Tue, 9 Aug 2016 18:23:00 +0000 (14:23 -0400)]
rgw_file: unlock() must precede out label

In lookup_handle(...).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 9f64f34e481278a683e962b17c7aa811005783a8)

8 years agorgw nfs: fix write continuations
Matt Benjamin [Sun, 7 Aug 2016 22:39:33 +0000 (18:39 -0400)]
rgw nfs: fix write continuations

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit b4d079f2b5a4d49b2ba7576b272a2a3f3b0d66c8)

8 years agorgw nfs: don't leak fh->mtx in lookup_handle()
Matt Benjamin [Fri, 5 Aug 2016 14:03:33 +0000 (10:03 -0400)]
rgw nfs: don't leak fh->mtx in lookup_handle()

This change fixes a serious latent locking problem, noticed after
updating the ganesha/rgw driver invalidation after renames.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit d74d46170d7104a6553674f111bbdbe3a116cf54)

8 years agorgw file: refuse to rename directories
Matt Benjamin [Mon, 8 Aug 2016 14:18:35 +0000 (10:18 -0400)]
rgw file: refuse to rename directories

The representation of paths as an aggregate involving any
number of objects argues against ever permitting such
operations.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 297220fd2a96c0cf34701127b8cf724e24e6865f)

8 years agorgw_file: refuse ops on deleted targets
Matt Benjamin [Wed, 3 Aug 2016 17:53:15 +0000 (13:53 -0400)]
rgw_file: refuse ops on deleted targets

Detect these illegal cases, as they are indicate various incorrect
behaviors/bugs.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 34224ecd8360e03603a1a5dd1c1c9f0a31f1c1d5)

8 years agorgw_file: implement rgw_setattr
Matt Benjamin [Thu, 14 Apr 2016 23:18:37 +0000 (19:18 -0400)]
rgw_file: implement rgw_setattr

Introduce a new RGWSetattrs RGWOp descendant, to create or replace
sets of attrs on buckets or objects.

This version of the change uses the standard RGWRADOS::set_attrs op
(we want attribute changes to (e.g.) sync with other changes).

Previous versions of this changed incorrectly masked the values
of st->st_ino in RGWFileHandle::stat(), now fixed.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 4de1c3c260265f821ebee842d49cb35bf49d8e4e)

8 years agorgw_file: refuse partial, out-of-order writes
Matt Benjamin [Wed, 13 Jul 2016 14:16:59 +0000 (10:16 -0400)]
rgw_file: refuse partial, out-of-order writes

A single file object may be opened only once per gateway
instance, and writes to that object must be complete, and in-order.
Enforce this.

If an invalid write is seen, deletes the current write transaction.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 12aded803e24539266ce9698c678088e2158a82a)

8 years agorgw_file: fix rename cases and unify unlink
Matt Benjamin [Tue, 12 Jul 2016 21:43:43 +0000 (17:43 -0400)]
rgw_file: fix rename cases and unify unlink

Rather ordinary rename cases failed in the unlink step.  Fix this,
unifying the unlink path while at it.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 05af1fbb870a905ea8903594dfc607d346c9dd31)

8 years agoMerge pull request #10656 from dachary/wip-16564-jewel
Matt Benjamin [Fri, 30 Sep 2016 18:02:43 +0000 (14:02 -0400)]
Merge pull request #10656 from dachary/wip-16564-jewel

jewel: cors auto memleak

8 years agoMerge pull request #11252 from ceph/wip-offline-split-jewel
Josh Durgin [Thu, 29 Sep 2016 22:54:05 +0000 (15:54 -0700)]
Merge pull request #11252 from ceph/wip-offline-split-jewel

ceph-objectstore-tool: add a way to split filestore directories offline

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #10866 from dachary/wip-17122-jewel
Yehuda Sadeh [Thu, 29 Sep 2016 18:34:44 +0000 (11:34 -0700)]
Merge pull request #10866 from dachary/wip-17122-jewel

jewel: COPY broke multipart files uploaded under dumpling

8 years agoMerge pull request #10890 from dachary/wip-17140-jewel
Yehuda Sadeh [Thu, 29 Sep 2016 18:32:09 +0000 (11:32 -0700)]
Merge pull request #10890 from dachary/wip-17140-jewel

jewel: rgw: period commit loses zonegroup changes: region_map converted repeatedly

8 years agoMerge pull request #11021 from cbodley/wip-16794
Yehuda Sadeh [Thu, 29 Sep 2016 17:51:09 +0000 (10:51 -0700)]
Merge pull request #11021 from cbodley/wip-16794

jewel: rgw: back off bucket sync on failures, don't store marker

8 years agoMerge pull request #10657 from dachary/wip-16792-jewel
Yehuda Sadeh [Thu, 29 Sep 2016 17:49:28 +0000 (10:49 -0700)]
Merge pull request #10657 from dachary/wip-16792-jewel

jewel: metadata sync can skip markers for failed/incomplete entries

8 years agoceph-objectstore-tool: add a way to split filestore directories offline 11252/head
Josh Durgin [Fri, 5 Aug 2016 18:45:00 +0000 (11:45 -0700)]
ceph-objectstore-tool: add a way to split filestore directories offline

Use the usual split code, and split each dir that meets the
usual split criteria.

This can be run with lower than usual split settings, to avoid costly
online splits. To make sure the directories are not merged again, use
a load merge threshold (e.g. filestore merge threshold = 1), and
adjust the split multiplier accordingly.

Fixes: http://tracker.ceph.com/issues/17220
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit e7b0428e0e8d8f5459311dc698d94a3ac4f04684)

Conflicts:
src/os/filestore/FileStore.cc
src/tools/ceph_objectstore_tool.cc
(trivial resolution)

8 years agoMerge pull request #10888 from dachary/wip-17062-jewel
Sage Weil [Tue, 27 Sep 2016 20:37:17 +0000 (15:37 -0500)]
Merge pull request #10888 from dachary/wip-17062-jewel

jewel: tests: fix TestClsRbd.mirror_image failure in upgrade:jewel-x-master-distro-basic-vps

8 years ago10.2.3 v10.2.3
Jenkins Build Slave User [Tue, 20 Sep 2016 22:04:23 +0000 (22:04 +0000)]
10.2.3

8 years agorgw: fix collection of object sync errors 11021/head
Yehuda Sadeh [Wed, 20 Jul 2016 19:59:50 +0000 (12:59 -0700)]
rgw: fix collection of object sync errors

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit aacc31f72cef6f151459e4a0543850edeeaf1938)

8 years agorgw: fix marker tracker completion handling
Yehuda Sadeh [Wed, 20 Jul 2016 19:43:48 +0000 (12:43 -0700)]
rgw: fix marker tracker completion handling

Was not tracking high markers correctly. Could only work if there was a single
hole in the completion range. Just keep a map of all the complete entries.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit f5801c726efdc2f3067a071e6bb5ac83fd0cd147)

8 years agorgw: collect() stops if error encoutered
Yehuda Sadeh [Wed, 20 Jul 2016 19:42:52 +0000 (12:42 -0700)]
rgw: collect() stops if error encoutered

and returns true if needed to be called again

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 5528932e4c327a0fb0ce34a676eb59707a575325)

8 years agorgw: back off bucket sync on failures, don't store marker
Yehuda Sadeh [Tue, 19 Jul 2016 22:32:03 +0000 (15:32 -0700)]
rgw: back off bucket sync on failures, don't store marker

Fixes: http://tracker.ceph.com/issues/16742
If we fail on any single entry in bucket, skip updating the marker tracker
so that next time we'll go over that entry, and back off. This will trigger
a report to the data sync error repo and eventually a retry on the failing
object.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 1f3fec807043fd313ef7c66ff48f18b82c8bfa66)

8 years agoMerge pull request #10804 from dachary/wip-17089-jewel
Loic Dachary [Mon, 29 Aug 2016 11:43:15 +0000 (13:43 +0200)]
Merge pull request #10804 from dachary/wip-17089-jewel

jewel: OSD failed to subscribe skipped osdmaps after ceph osd pause

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #10760 from dachary/wip-16865-jewel
Loic Dachary [Mon, 29 Aug 2016 11:43:00 +0000 (13:43 +0200)]
Merge pull request #10760 from dachary/wip-16865-jewel

jewel: saw valgrind issues in ReplicatedPG::new_repop

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #10662 from dachary/wip-16901-jewel
Loic Dachary [Fri, 26 Aug 2016 20:12:39 +0000 (22:12 +0200)]
Merge pull request #10662 from dachary/wip-16901-jewel

jewel: segfault in RGWOp_MDLog_Notify

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
8 years agoMerge pull request #10763 from dachary/wip-17034-jewel
Orit Wasserman [Fri, 26 Aug 2016 08:31:06 +0000 (10:31 +0200)]
Merge pull request #10763 from dachary/wip-17034-jewel

jewel: rgw: object expirer's hints might be trimmed without processing in some circumstances
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #10658 from dachary/wip-16862-jewel
Orit Wasserman [Fri, 26 Aug 2016 08:28:10 +0000 (10:28 +0200)]
Merge pull request #10658 from dachary/wip-16862-jewel

jewel: default zone and zonegroup cannot be added to a realm
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #10659 from dachary/wip-16863-jewel
Orit Wasserman [Fri, 26 Aug 2016 08:26:53 +0000 (10:26 +0200)]
Merge pull request #10659 from dachary/wip-16863-jewel

jewel: use zone endpoints instead of zonegroup endpoints
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #10660 from dachary/wip-16864-jewel
Orit Wasserman [Fri, 26 Aug 2016 08:24:20 +0000 (10:24 +0200)]
Merge pull request #10660 from dachary/wip-16864-jewel

jewel: multisite segfault on ~RGWRealmWatcher if realm was deleted
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agorgw: delete region map after upgrade to zonegroup map 10890/head
Casey Bodley [Tue, 23 Aug 2016 19:10:44 +0000 (15:10 -0400)]
rgw: delete region map after upgrade to zonegroup map

convert_regionmap() reads the region map and uses it to initialize the
zonegroup map.  but it doesn't remove the region_map afterwards, so
radosgw (and some radosgw-admin commands) will keep doing this on
startup, overwriting any changes made to the period/zonegroup map

Fixes: http://tracker.ceph.com/issues/17051
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 215cd6f6ff7b8cb74df6dc3b94a5928f56ab9540)

8 years agoqa/workunits: support filtering cls_rbd unit test cases 10888/head
Jason Dillaman [Fri, 12 Aug 2016 18:44:09 +0000 (14:44 -0400)]
qa/workunits: support filtering cls_rbd unit test cases

Fixes: http://tracker.ceph.com/issues/16529
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0891ede57cd254aa75c66301fde4a2eb6957a3d6)

8 years agoMerge pull request #10847 from gregsfortytwo/jewel-fix-mds
Loic Dachary [Thu, 25 Aug 2016 22:46:17 +0000 (00:46 +0200)]
Merge pull request #10847 from gregsfortytwo/jewel-fix-mds

mds: fix double-unlock on shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agoMerge pull request #10663 from dachary/wip-16934-jewel
Loic Dachary [Thu, 25 Aug 2016 21:04:37 +0000 (23:04 +0200)]
Merge pull request #10663 from dachary/wip-16934-jewel

jewel: Add zone rename to radosgw_admin

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10664 from dachary/wip-16958-jewel
Loic Dachary [Thu, 25 Aug 2016 21:04:22 +0000 (23:04 +0200)]
Merge pull request #10664 from dachary/wip-16958-jewel

jewel: Bug when using port 443s in rgw.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10764 from dachary/wip-17032-jewel
Loic Dachary [Thu, 25 Aug 2016 21:04:07 +0000 (23:04 +0200)]
Merge pull request #10764 from dachary/wip-17032-jewel

jewel: multisite: RGWPeriodPuller tries to pull from itself

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10765 from dachary/wip-17006-jewel
Loic Dachary [Thu, 25 Aug 2016 21:03:40 +0000 (23:03 +0200)]
Merge pull request #10765 from dachary/wip-17006-jewel

jewel: Increase log level for messages occuring while running rgw admin command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10845 from cbodley/wip-16702
Loic Dachary [Thu, 25 Aug 2016 20:25:21 +0000 (22:25 +0200)]
Merge pull request #10845 from cbodley/wip-16702

jewel: rgw: add tenant support to multisite sync

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agorgw: adjust manifest head object 10866/head
Yehuda Sadeh [Fri, 8 Jul 2016 01:01:54 +0000 (18:01 -0700)]
rgw: adjust manifest head object

adjust the manifest head object:
 - when reading manifest, set the head object to the object
   we read the manifest from (and its size). Some manifests are
   broken and point at a different object
 - when copying multipart object, set the manifest head object to
   point at the new head object

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 419264586fc46337322f0d60b0ffcdcce3bb5c5a)

8 years agorgw: adjust objs when copying obj with explicit_objs set
Yehuda Sadeh [Thu, 7 Jul 2016 22:49:07 +0000 (15:49 -0700)]
rgw: adjust objs when copying obj with explicit_objs set

If the head object contains data, need to set it in the list of
objs (in addition to being pointed at by head_obj).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 1e012112b99004745952accc6fb11ada5e8e0045)

8 years agorgw: patch manifest to handle explicit objs copy issue
Yehuda Sadeh [Thu, 7 Jul 2016 22:36:33 +0000 (15:36 -0700)]
rgw: patch manifest to handle explicit objs copy issue

Fixes: http://tracker.ceph.com/issues/16435
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit b51476466e5bb03bbaa6e428bb6bb189a259b9fd)

8 years agoMerge pull request #9405 from SUSE/wip-16083-jewel
Loic Dachary [Thu, 25 Aug 2016 07:17:52 +0000 (09:17 +0200)]
Merge pull request #9405 from SUSE/wip-16083-jewel

jewel: mds: wrongly treat symlink inode as normal file/dir when symlink inode is stale on kcephfs

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10654 from dachary/wip-9577-jewel
Loic Dachary [Thu, 25 Aug 2016 07:02:17 +0000 (09:02 +0200)]
Merge pull request #10654 from dachary/wip-9577-jewel

jewel: mon: "mon metadata" fails when only one monitor exists

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agomds: fix double-unlock on shutdown 10847/head
Greg Farnum [Wed, 24 Aug 2016 20:34:43 +0000 (13:34 -0700)]
mds: fix double-unlock on shutdown

We did a bad backport or something and accidentally ended up with two Unlock()
calls on mds_lock. Don't.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
8 years agorgw: collect skips a specific coroutine stack 10845/head
Yehuda Sadeh [Wed, 13 Jul 2016 03:36:35 +0000 (20:36 -0700)]
rgw: collect skips a specific coroutine stack

Fixes: http://tracker.ceph.com/issues/16665
Instead of drain_all_but() that specifies number of stacks to leave behind,
added drain_all_but_stack() that has a specific stack specified. This is needed
so that we don't call wakeup() through lease_cr->go_down() on a cr stack that
was already collected.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 5a2e8f0526db92a290c711f82627fc5042c290ea)

8 years agoMerge pull request #10791 from dachary/wip-17004-jewel
Loic Dachary [Wed, 24 Aug 2016 20:08:58 +0000 (22:08 +0200)]
Merge pull request #10791 from dachary/wip-17004-jewel

jewel: rbd-mirror: FAILED assert(m_state == STATE_STOPPING)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10827 from dillaman/wip-16950-jewel
Loic Dachary [Wed, 24 Aug 2016 20:08:30 +0000 (22:08 +0200)]
Merge pull request #10827 from dillaman/wip-16950-jewel

jewel: librbd: delay acquiring lock if image watch has failed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agorgw: fix compilation
Josh Durgin [Tue, 12 Jul 2016 00:38:43 +0000 (17:38 -0700)]
rgw: fix compilation

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 874de58024e12d893ae050cd421ef67c9521c753)

8 years agorgw: remove datalog keys from error repo on ENOENT
Casey Bodley [Thu, 7 Jul 2016 18:42:00 +0000 (14:42 -0400)]
rgw: remove datalog keys from error repo on ENOENT

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 16976eedb70292e821193e39bb577a68df1bc95a)

8 years agotest/rgw: add optional --tenant flag to test_multi.py
Casey Bodley [Fri, 1 Jul 2016 16:36:04 +0000 (12:36 -0400)]
test/rgw: add optional --tenant flag to test_multi.py

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6f65d192cd3917cdcf7d9deb5396750d2c641842)

8 years agorgw: add tenant id to GetBucketInstanceInfo
Casey Bodley [Thu, 30 Jun 2016 19:50:00 +0000 (15:50 -0400)]
rgw: add tenant id to GetBucketInstanceInfo

use the rgw_bucket overload of get_bucket_instance_info() so it can
supply the tenant id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3cf715c6828d7d9732bf6a6fd9b46bbd8c08de5f)

8 years agorgw: carry tenant id with data sync
Casey Bodley [Wed, 29 Jun 2016 15:58:55 +0000 (11:58 -0400)]
rgw: carry tenant id with data sync

use rgw_bucket_shard to track buckets, which includes tenant id

Fixes: http://tracker.ceph.com/issues/16469
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d945e2099525e39588b572e357b115df98c8cdca)

8 years agorgw: add tenant to url for RGWRESTStreamWriteRequest
Casey Bodley [Thu, 30 Jun 2016 21:43:40 +0000 (17:43 -0400)]
rgw: add tenant to url for RGWRESTStreamWriteRequest

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2a1e1a776c58f62fbcb57877d56b92eb1e8933e6)

8 years agorgw: add tenant to rgw_bucket json format
Casey Bodley [Thu, 30 Jun 2016 21:42:54 +0000 (17:42 -0400)]
rgw: add tenant to rgw_bucket json format

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fc0df4802b48fa095587b93557c82da5446d9ca3)

8 years agorgw: add rgw_bucket_parse_bucket_key()
Casey Bodley [Thu, 30 Jun 2016 04:13:47 +0000 (00:13 -0400)]
rgw: add rgw_bucket_parse_bucket_key()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d5ac140040ea2f17e33d2846870cb033c7ae6ffa)

8 years agorgw: add tenant name to datalog entries
Casey Bodley [Tue, 28 Jun 2016 15:24:46 +0000 (11:24 -0400)]
rgw: add tenant name to datalog entries

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c8aa668fc3e34a7651ce6e2dda62902ced808541)

8 years agorgw: convert bucket instance listings back to metadata key format
Casey Bodley [Wed, 6 Jul 2016 15:50:47 +0000 (11:50 -0400)]
rgw: convert bucket instance listings back to metadata key format

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3eae201c8a1513bae23233700c754ec076360664)

8 years agorgw: use tenant/ for bucket instance metadata keys
Casey Bodley [Tue, 5 Jul 2016 13:00:19 +0000 (09:00 -0400)]
rgw: use tenant/ for bucket instance metadata keys

to work around the ambiguity of parsing tenant: and :shard in the same
bucket instance metadata key, use tenant/ instead

to preserve backward compatibility with existing objects, new helper
function rgw_bucket_instance_key_to_oid() converts this / back to a :
before being used as an object name

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a0befa1e7be3aeb37209bac2e2a7de0edf5d5a95)

8 years agorgw: add get_key() methods to format rgw_buckets
Casey Bodley [Thu, 30 Jun 2016 22:09:03 +0000 (18:09 -0400)]
rgw: add get_key() methods to format rgw_buckets

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 51ff8ef7c6bb7c9f55f3ca8ad059b029a4088cdf)

8 years agorgw: data sync debug logging
Yehuda Sadeh [Mon, 6 Jun 2016 23:16:33 +0000 (16:16 -0700)]
rgw: data sync debug logging

a high level debug logging of the data sync process.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 97ef16c0cc0ad8345b5a897108717d83ab0aa9ab)

8 years agorgw: modifying multi-site log messages.
Pritha Srivastava [Thu, 16 Jun 2016 09:15:57 +0000 (14:45 +0530)]
rgw: modifying multi-site log messages.

Increasing the log level and removing 'ERROR' from messages that appear during normal multi-site operation.
Fixes: http://tracker.ceph.com/issues/16121
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit e1737ddaa035f23757d021dd34b0c87d24ce372c)

8 years agoMerge pull request #10653 from dachary/wip-10495-jewel
Loic Dachary [Wed, 24 Aug 2016 16:57:23 +0000 (18:57 +0200)]
Merge pull request #10653 from dachary/wip-10495-jewel

jewel : 60-ceph-partuuid-workaround-rules still needed by debian jessie (udev 215-17)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #10357 from SUSE/wip-16748-jewel
Loic Dachary [Wed, 24 Aug 2016 16:51:54 +0000 (18:51 +0200)]
Merge pull request #10357 from SUSE/wip-16748-jewel

jewel: mount.ceph: move from ceph-base to ceph-common and add symlink in /sbin for SUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #10790 from dachary/wip-17005-jewel
Loic Dachary [Wed, 24 Aug 2016 15:48:00 +0000 (17:48 +0200)]
Merge pull request #10790 from dachary/wip-17005-jewel

jewel: ImageReplayer::is_replaying does not include flush state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10792 from dachary/wip-16978-jewel
Loic Dachary [Wed, 24 Aug 2016 15:47:45 +0000 (17:47 +0200)]
Merge pull request #10792 from dachary/wip-16978-jewel

jewel: rbd-mirror: FAILED assert(m_on_update_status_finish == nullptr)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10646 from dachary/wip-16576-jewel
Loic Dachary [Wed, 24 Aug 2016 15:47:36 +0000 (17:47 +0200)]
Merge pull request #10646 from dachary/wip-16576-jewel

jewel: rbd-mirror: FAILED assert(m_local_image_ctx->object_map != nullptr)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10647 from dachary/wip-16593-jewel
Loic Dachary [Wed, 24 Aug 2016 15:47:18 +0000 (17:47 +0200)]
Merge pull request #10647 from dachary/wip-16593-jewel

jewel: FAILED assert(object_no < m_object_map.size())

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10786 from dachary/wip-17061-jewel
Loic Dachary [Wed, 24 Aug 2016 15:47:08 +0000 (17:47 +0200)]
Merge pull request #10786 from dachary/wip-17061-jewel

jewel: bashism in src/rbdmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10421 from SUSE/wip-16659-jewel
Loic Dachary [Wed, 24 Aug 2016 12:12:34 +0000 (14:12 +0200)]
Merge pull request #10421 from SUSE/wip-16659-jewel

jewel: ReplicatedBackend doesn't increment stats on pull, only push

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #10501 from Abhishekvrshny/wip-16621-jewel
Loic Dachary [Tue, 23 Aug 2016 13:42:11 +0000 (15:42 +0200)]
Merge pull request #10501 from Abhishekvrshny/wip-16621-jewel

jewel: mds:  tell command blocks forever with async messenger (TestVolumeClient.test_evict_client failure)

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10816 from SUSE/wip-17092-jewel
Loic Dachary [Tue, 23 Aug 2016 13:23:34 +0000 (15:23 +0200)]
Merge pull request #10816 from SUSE/wip-17092-jewel

jewel: build/ops: need rocksdb commit 7ca731b12ce for ppc64le build

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agolibrbd: delay acquiring exclusive lock if watch has failed 10827/head
Jason Dillaman [Thu, 4 Aug 2016 17:24:54 +0000 (13:24 -0400)]
librbd: delay acquiring exclusive lock if watch has failed

Fixes: http://tracker.ceph.com/issues/16923
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit dfe9f3eac9cca3b83962e0e1b7eac38e6e76d7a5)

8 years agolibrbd: convert ImageWatcher class to template
Jason Dillaman [Thu, 4 Aug 2016 17:47:33 +0000 (13:47 -0400)]
librbd: convert ImageWatcher class to template

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 814c305ce8c35b5ce01d7e29a912d5ef3978754b)

 Conflicts:
src/librbd/ImageWatcher.cc: no shrink guard
src/librbd/Operations.cc: no shrink guard

8 years agoMerge pull request #10103 from xiaoxichen/wip-16037-jewel
Loic Dachary [Tue, 23 Aug 2016 06:48:19 +0000 (08:48 +0200)]
Merge pull request #10103 from xiaoxichen/wip-16037-jewel

jewel: MDSMonitor::check_subs() is very buggy

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10105 from xiaoxichen/wip-16515-jewel
Loic Dachary [Tue, 23 Aug 2016 06:48:01 +0000 (08:48 +0200)]
Merge pull request #10105 from xiaoxichen/wip-16515-jewel

jewel: Session::check_access() is buggy

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10106 from xiaoxichen/wip-16215-jewel
Loic Dachary [Tue, 23 Aug 2016 06:47:37 +0000 (08:47 +0200)]
Merge pull request #10106 from xiaoxichen/wip-16215-jewel

jewel: client: crash in unmount when fuse_use_invalidate_cb is enabled

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10108 from xiaoxichen/wip-16320-jewel
Loic Dachary [Tue, 23 Aug 2016 06:47:23 +0000 (08:47 +0200)]
Merge pull request #10108 from xiaoxichen/wip-16320-jewel

jewel: fuse mounted file systems fails SAMBA CTDB ping_pong rw test with v9.0.2

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10199 from SUSE/wip-16625-jewel
Loic Dachary [Tue, 23 Aug 2016 06:47:10 +0000 (08:47 +0200)]
Merge pull request #10199 from SUSE/wip-16625-jewel

jewel: Failing file operations on kernel based cephfs mount point leaves unaccessible file behind on hammer 0.94.7

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10499 from Abhishekvrshny/wip-16299-jewel
Loic Dachary [Tue, 23 Aug 2016 06:46:53 +0000 (08:46 +0200)]
Merge pull request #10499 from Abhishekvrshny/wip-16299-jewel

jewel: mds: fix SnapRealm::have_past_parents_open()

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10500 from Abhishekvrshny/wip-16620-jewel
Loic Dachary [Tue, 23 Aug 2016 06:46:38 +0000 (08:46 +0200)]
Merge pull request #10500 from Abhishekvrshny/wip-16620-jewel

jewel: Fix shutting down mds timed-out due to deadlock

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10502 from Abhishekvrshny/wip-16797-jewel
Loic Dachary [Tue, 23 Aug 2016 06:46:13 +0000 (08:46 +0200)]
Merge pull request #10502 from Abhishekvrshny/wip-16797-jewel

jewel: MDS Deadlock on shutdown active rank while busy with metadata IO

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10104 from xiaoxichen/wip-16560-jewel
Loic Dachary [Tue, 23 Aug 2016 06:45:41 +0000 (08:45 +0200)]
Merge pull request #10104 from xiaoxichen/wip-16560-jewel

jewel: mds: enforce a dirfrag limit on entries

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agoMerge pull request #10074 from ceph/jewel-16002
Loic Dachary [Mon, 22 Aug 2016 23:19:33 +0000 (01:19 +0200)]
Merge pull request #10074 from ceph/jewel-16002

jewel: ObjectCacher split BufferHead read fix

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agobuild/ops: bump rocksdb submodule 10816/head
Nathan Cutler [Mon, 22 Aug 2016 18:56:38 +0000 (20:56 +0200)]
build/ops: bump rocksdb submodule

Fixes a FTBFS on ppc64le

Fixes: http://tracker.ceph.com/issues/17092
Signed-off-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #10537 from theanalyst/wip-16778-jewel
Loic Dachary [Mon, 22 Aug 2016 08:37:14 +0000 (10:37 +0200)]
Merge pull request #10537 from theanalyst/wip-16778-jewel

jewel: rgw multisite: preserve zone's extra pool

Reviewed-by: Orit Wasserman <owasserm@redhat.com>