]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agocommon: only call crypto::init once per CephContext 11409/head
Casey Bodley [Fri, 2 Sep 2016 18:13:19 +0000 (14:13 -0400)]
common: only call crypto::init once per CephContext

Fixes: http://tracker.ceph.com/issues/17205
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9dfc1537564c4ce65e4d0230a83f0d8d47503b16)

8 years agoMerge pull request #10812 from SUSE/wip-17094-jewel
Loic Dachary [Mon, 10 Oct 2016 18:30:54 +0000 (20:30 +0200)]
Merge pull request #10812 from SUSE/wip-17094-jewel

jewel: build/ops: ceph-osd-prestart.sh fails confusingly when data directory does not exist

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #11335 from linuxbox2/jewel-nfs
Matt Benjamin [Mon, 10 Oct 2016 14:46:33 +0000 (10:46 -0400)]
Merge pull request #11335 from linuxbox2/jewel-nfs

jewel: rgw: nfs backports

8 years agoMerge pull request #11332 from linuxbox2/jewel-ldap
Matt Benjamin [Mon, 10 Oct 2016 14:45:07 +0000 (10:45 -0400)]
Merge pull request #11332 from linuxbox2/jewel-ldap

jewel: rgw: combined LDAP backports

8 years agoMerge pull request #10761 from dachary/wip-16377-jewel
Sage Weil [Mon, 10 Oct 2016 13:54:02 +0000 (08:54 -0500)]
Merge pull request #10761 from dachary/wip-16377-jewel

jewel: msgr/async: Messenger thread long time lock hold risk

Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #10278 from SUSE/wip-16667-jewel
Sage Weil [Mon, 10 Oct 2016 13:52:59 +0000 (08:52 -0500)]
Merge pull request #10278 from SUSE/wip-16667-jewel

jewel: incorrect value of CINIT_FLAG_DEFER_DROP_PRIVILEGES

Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #10794 from SUSE/wip-17082-jewel
Loic Dachary [Mon, 10 Oct 2016 10:26:02 +0000 (12:26 +0200)]
Merge pull request #10794 from SUSE/wip-17082-jewel

jewel: disable LTTng-UST in openSUSE builds

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #9739 from vumrao/wip-vumrao-16337
Loic Dachary [Mon, 10 Oct 2016 10:21:59 +0000 (12:21 +0200)]
Merge pull request #9739 from vumrao/wip-vumrao-16337

jewel: osd: add peer_addr in heartbeat_check log message

Reviewed-by: Samuel Just <sjust@redhat.com>
8 years agoMerge pull request #9388 from vumrao/wip-vumrao-16069
Loic Dachary [Mon, 10 Oct 2016 10:20:30 +0000 (12:20 +0200)]
Merge pull request #9388 from vumrao/wip-vumrao-16069

jewel: mon: Display full flag in ceph status if full flag is set

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #10496 from Abhishekvrshny/wip-16583-jewel
Loic Dachary [Mon, 10 Oct 2016 10:03:51 +0000 (12:03 +0200)]
Merge pull request #10496 from Abhishekvrshny/wip-16583-jewel

jewel: mon crash: crush/CrushWrapper.h: 940: FAILED assert(successful_detach)

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #11018 from ceph/wip-17223-jewel
Loic Dachary [Mon, 10 Oct 2016 07:31:29 +0000 (09:31 +0200)]
Merge pull request #11018 from ceph/wip-17223-jewel

jewel: krbd qa scripts and concurrent.sh test fix

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agorgw_file: pre-assign times 11335/head
Matt Benjamin [Wed, 21 Sep 2016 21:18:20 +0000 (17:18 -0400)]
rgw_file:  pre-assign times

Set unix timestamps based on RGW values for creation, modification
time for buckets, objects in stat requests.  Regard any saved
value of these in unix attributes as an overlay.

Fixes: http://tracker.ceph.com/issues/17367
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 969750e96380859a350e25015bc8cbf9bdf88512)

Fixes: http://tracker.ceph.com/issues/17394
8 years agorgw_file: fix set_attrs operation
Matt Benjamin [Tue, 20 Sep 2016 21:32:03 +0000 (17:32 -0400)]
rgw_file: fix set_attrs operation

The effective part of this change is to always set a value
for RGW_ATTR_UNIX_KEY1 (because it is expected later).

Secondarily, do not pass the address of the to-set attributes
buffer::list as remove attrs--this is confusing.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 08afb40b2812270a6af3cd1965e8c03c10392ace)

Fixes: http://tracker.ceph.com/issues/17393
8 years agorgw_file: implement rgw_setattr
Matt Benjamin [Thu, 14 Apr 2016 23:18:37 +0000 (19:18 -0400)]
rgw_file: implement rgw_setattr

Introduce a new RGWSetattrs RGWOp descendant, to create or replace
sets of attrs on buckets or objects.

This version of the change uses the standard RGWRADOS::set_attrs op
(we want attribute changes to (e.g.) sync with other changes).

Previous versions of this changed incorrectly masked the values
of st->st_ino in RGWFileHandle::stat(), now fixed.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 4de1c3c260265f821ebee842d49cb35bf49d8e4e)

Fixes: http://tracker.ceph.com/issues/17311
Fixes: http://tracker.ceph.com/issues/17332
8 years agorgw: remove duplicated calls to getattr
zhangweibing [Fri, 26 Aug 2016 02:49:48 +0000 (10:49 +0800)]
rgw: remove duplicated calls to getattr

Signed-off-by: Weibing Zhang <zhangweibing@unitedstack.com>
(cherry picked from commit 87b550ed2fba472d4ef2e1f771bb5f9fd248804f)

Fixes: http://tracker.ceph.com/issues/17327
8 years agorgw: rgw file fix bug of rgw_lookup can not exact match file name
Min Chen [Fri, 19 Aug 2016 05:02:50 +0000 (01:02 -0400)]
rgw: rgw file fix bug of rgw_lookup can not exact match file name

bug reproduce steps:
in nfs-client (connect to nfs-ganesha server):
1. mv file file.rename
2. stat file // get file attrs, not return with -ENOENT

the reason is that:
RGWStatLeafRequest does not exact match the file name,
just take file name as a prefix filter

Signed-off-by: Min Chen <chenmin@xsky.com>
(cherry picked from commit 9d813bafc8e197507457c58ab4f365ccdb7f3589)

Fixes: http://tracker.ceph.com/issues/17326
8 years agorgw: fix the wrong return value
Yan Jun [Tue, 23 Aug 2016 04:51:10 +0000 (12:51 +0800)]
rgw: fix the wrong return value

Here `parent->readdir` will return negative numbers When there is an error.
so we should just return it.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
(cherry picked from commit c979be7810793534715c24c6a9f7c8b0ed31de8d)

Fixes: http://tracker.ceph.com/issues/17325
8 years agorgw_file: restore local definition of RGWLibFS gc interval
Matt Benjamin [Wed, 17 Aug 2016 14:28:48 +0000 (10:28 -0400)]
rgw_file: restore local definition of RGWLibFS gc interval

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 31936caa09b46e86700faad969adfa9d43176206)

Fixes: http://tracker.ceph.com/issues/17323
8 years agorgw_file: explain semantics of RGWFileHandle::commit
Matt Benjamin [Tue, 23 Aug 2016 20:50:42 +0000 (16:50 -0400)]
rgw_file:  explain semantics of RGWFileHandle::commit

Add a comment explaining why the method currently returns 0
unconditionally.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit b1da1354320cc5f69277b225a293a03cc2a5054f)

8 years agoceph_timer: prefer using directive to typedef
Matt Benjamin [Tue, 23 Aug 2016 19:57:06 +0000 (15:57 -0400)]
ceph_timer: prefer using directive to typedef

This change replaces the existing member-hook typedef as well as
the new set-type typedefs, so committed separately.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit f7d4d418658d2091652033045abdb4bfbeb637aa)

8 years agolibrgw: add API version defines for librgw and rgw_file
Matt Benjamin [Tue, 23 Aug 2016 18:18:43 +0000 (14:18 -0400)]
librgw: add API version defines for librgw and rgw_file

This change borrows the major, minor+extra format used by libcephfs.
The version numbering is starting at 1,1,0 on the theory that the
implicit version at Jewel is 1,0,0.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 355ccc5ded7f0b459bca24bc8b504b41807c583d)

8 years agorgw file: allow RGWLibFS::write_completion_interval_s to be set in conf
Matt Benjamin [Wed, 17 Aug 2016 15:24:01 +0000 (11:24 -0400)]
rgw file: allow RGWLibFS::write_completion_interval_s to be set in conf

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 70dad0fba561edaf0bb4e746e29b92a4e9b014b2)

8 years agorgw_file: log on write completion events at level 10
Matt Benjamin [Wed, 17 Aug 2016 15:00:22 +0000 (11:00 -0400)]
rgw_file:  log on write completion events at level 10

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit d1e512d8c0e20df5a96ccb5f775a7a2ca123efbe)

8 years agorgw_file: add RGW_OPEN_FLAG_STATELESS as a synonym for V3
Matt Benjamin [Wed, 17 Aug 2016 14:52:58 +0000 (10:52 -0400)]
rgw_file: add RGW_OPEN_FLAG_STATELESS as a synonym for V3

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit da596eb3af69c2cdf191893eb44a8947662634cf)

8 years agorgw_file: implement stateless write completion timer
Matt Benjamin [Tue, 16 Aug 2016 22:17:53 +0000 (18:17 -0400)]
rgw_file: implement stateless write completion timer

Implements a temporal mechanism to enforce write completion for setups
which lack open state tracking (e.g., NFS3).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 2c83ed4bd43fcb5d5497151a157b1dc08022fed9)

8 years agorgw_file: fix/update rgw_create flag arguments
Matt Benjamin [Fri, 12 Aug 2016 20:52:51 +0000 (16:52 -0400)]
rgw_file: fix/update rgw_create flag arguments

Allow passing POSIX open flags as well as api call flags.  Needed
for NFS3 support.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 2d0ea2b4649e76cbbb87f44c57754d7ea9c46dce)

8 years agorgw_file: fix/update rgw_open flag arguments
Matt Benjamin [Fri, 12 Aug 2016 14:39:21 +0000 (10:39 -0400)]
rgw_file: fix/update rgw_open flag arguments

Allow passing POSIX open flags as well as api call flags.  Needed
for NFS3 support.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit ada29f7f8323be2301588941bcd5c60190b32a4b)

8 years agorgw file: remove busy-wait in RGWLibFS::gc()
Matt Benjamin [Tue, 9 Aug 2016 20:49:41 +0000 (16:49 -0400)]
rgw file: remove busy-wait in RGWLibFS::gc()

This is a background thread.  However, CPU is wasted.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit ca33241286f52d849dbde8092507131b8b1108cc)

Fixes: http://tracker.ceph.com/issues/17321
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
8 years agorgw_file: unlock() must precede out label
Matt Benjamin [Tue, 9 Aug 2016 18:23:00 +0000 (14:23 -0400)]
rgw_file: unlock() must precede out label

In lookup_handle(...).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 9f64f34e481278a683e962b17c7aa811005783a8)

8 years agorgw nfs: fix write continuations
Matt Benjamin [Sun, 7 Aug 2016 22:39:33 +0000 (18:39 -0400)]
rgw nfs: fix write continuations

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit b4d079f2b5a4d49b2ba7576b272a2a3f3b0d66c8)

8 years agorgw nfs: don't leak fh->mtx in lookup_handle()
Matt Benjamin [Fri, 5 Aug 2016 14:03:33 +0000 (10:03 -0400)]
rgw nfs: don't leak fh->mtx in lookup_handle()

This change fixes a serious latent locking problem, noticed after
updating the ganesha/rgw driver invalidation after renames.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit d74d46170d7104a6553674f111bbdbe3a116cf54)

8 years agorgw file: refuse to rename directories
Matt Benjamin [Mon, 8 Aug 2016 14:18:35 +0000 (10:18 -0400)]
rgw file: refuse to rename directories

The representation of paths as an aggregate involving any
number of objects argues against ever permitting such
operations.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 297220fd2a96c0cf34701127b8cf724e24e6865f)

8 years agorgw_file: refuse ops on deleted targets
Matt Benjamin [Wed, 3 Aug 2016 17:53:15 +0000 (13:53 -0400)]
rgw_file: refuse ops on deleted targets

Detect these illegal cases, as they are indicate various incorrect
behaviors/bugs.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 34224ecd8360e03603a1a5dd1c1c9f0a31f1c1d5)

8 years agorgw_file: implement rgw_setattr
Matt Benjamin [Thu, 14 Apr 2016 23:18:37 +0000 (19:18 -0400)]
rgw_file: implement rgw_setattr

Introduce a new RGWSetattrs RGWOp descendant, to create or replace
sets of attrs on buckets or objects.

This version of the change uses the standard RGWRADOS::set_attrs op
(we want attribute changes to (e.g.) sync with other changes).

Previous versions of this changed incorrectly masked the values
of st->st_ino in RGWFileHandle::stat(), now fixed.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 4de1c3c260265f821ebee842d49cb35bf49d8e4e)

8 years agorgw_file: refuse partial, out-of-order writes
Matt Benjamin [Wed, 13 Jul 2016 14:16:59 +0000 (10:16 -0400)]
rgw_file: refuse partial, out-of-order writes

A single file object may be opened only once per gateway
instance, and writes to that object must be complete, and in-order.
Enforce this.

If an invalid write is seen, deletes the current write transaction.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 12aded803e24539266ce9698c678088e2158a82a)

8 years agorgw_file: fix rename cases and unify unlink
Matt Benjamin [Tue, 12 Jul 2016 21:43:43 +0000 (17:43 -0400)]
rgw_file: fix rename cases and unify unlink

Rather ordinary rename cases failed in the unlink step.  Fix this,
unifying the unlink path while at it.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 05af1fbb870a905ea8903594dfc607d346c9dd31)

8 years agoadd ldap auth custom search filter feature patch - http://tracker.ceph.com/issues... 11332/head
Harald Klein [Wed, 31 Aug 2016 15:41:26 +0000 (17:41 +0200)]
add ldap auth custom search filter feature patch - http://tracker.ceph.com/issues/17185

Signed-off-by: Harald Klein <hari@vt100.at>
(cherry picked from commit c935885ae9d5fd413495448a0b0e5fce899c9b73)

Fixes: http://tracker.ceph.com/issues/17185
8 years agorgw ldap: protect rgw::from_base64 from non-base64 input
Matt Benjamin [Thu, 18 Aug 2016 14:54:16 +0000 (10:54 -0400)]
rgw ldap:  protect rgw::from_base64 from non-base64 input

Also adds unit tests for:
1. empty output from from_base64 (turns out to be harmless)
2. random and specific non-base64 and sort strings
(modified from upstream to avoid alteration of src/test/test_rgw_token.cc)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 0a4c91ec7652d02673a9b156cd16144d778a3844)

Fixes: http://tracker.ceph.com/issues/17324
8 years agorgw ldap: enforce simple_bind w/LDAPv3
Matt Benjamin [Fri, 5 Aug 2016 14:02:03 +0000 (10:02 -0400)]
rgw ldap: enforce simple_bind w/LDAPv3

Found by Harald Klein <hklein@redhat.com>.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 08d54291435e4d1cb5e02cda3951bc6e8510b0e2)

8 years agorgw: add reinit/rebind logic (ldap)
Matt Benjamin [Mon, 1 Aug 2016 21:36:17 +0000 (17:36 -0400)]
rgw: add reinit/rebind logic (ldap)

Gracefully handle stale LDAP connections by rebinding--verified
with MS AD.  Rebind is attempted once per request until rebound--
not worse than misconfiguring the environment.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 9b8fccf68789ba5c5986766c544b23aeada0e87e)

8 years agoMerge pull request #10656 from dachary/wip-16564-jewel
Matt Benjamin [Fri, 30 Sep 2016 18:02:43 +0000 (14:02 -0400)]
Merge pull request #10656 from dachary/wip-16564-jewel

jewel: cors auto memleak

8 years agoMerge pull request #11252 from ceph/wip-offline-split-jewel
Josh Durgin [Thu, 29 Sep 2016 22:54:05 +0000 (15:54 -0700)]
Merge pull request #11252 from ceph/wip-offline-split-jewel

ceph-objectstore-tool: add a way to split filestore directories offline

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #10866 from dachary/wip-17122-jewel
Yehuda Sadeh [Thu, 29 Sep 2016 18:34:44 +0000 (11:34 -0700)]
Merge pull request #10866 from dachary/wip-17122-jewel

jewel: COPY broke multipart files uploaded under dumpling

8 years agoMerge pull request #10890 from dachary/wip-17140-jewel
Yehuda Sadeh [Thu, 29 Sep 2016 18:32:09 +0000 (11:32 -0700)]
Merge pull request #10890 from dachary/wip-17140-jewel

jewel: rgw: period commit loses zonegroup changes: region_map converted repeatedly

8 years agoMerge pull request #11021 from cbodley/wip-16794
Yehuda Sadeh [Thu, 29 Sep 2016 17:51:09 +0000 (10:51 -0700)]
Merge pull request #11021 from cbodley/wip-16794

jewel: rgw: back off bucket sync on failures, don't store marker

8 years agoMerge pull request #10657 from dachary/wip-16792-jewel
Yehuda Sadeh [Thu, 29 Sep 2016 17:49:28 +0000 (10:49 -0700)]
Merge pull request #10657 from dachary/wip-16792-jewel

jewel: metadata sync can skip markers for failed/incomplete entries

8 years agoceph-objectstore-tool: add a way to split filestore directories offline 11252/head
Josh Durgin [Fri, 5 Aug 2016 18:45:00 +0000 (11:45 -0700)]
ceph-objectstore-tool: add a way to split filestore directories offline

Use the usual split code, and split each dir that meets the
usual split criteria.

This can be run with lower than usual split settings, to avoid costly
online splits. To make sure the directories are not merged again, use
a load merge threshold (e.g. filestore merge threshold = 1), and
adjust the split multiplier accordingly.

Fixes: http://tracker.ceph.com/issues/17220
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit e7b0428e0e8d8f5459311dc698d94a3ac4f04684)

Conflicts:
src/os/filestore/FileStore.cc
src/tools/ceph_objectstore_tool.cc
(trivial resolution)

8 years agoMerge pull request #10888 from dachary/wip-17062-jewel
Sage Weil [Tue, 27 Sep 2016 20:37:17 +0000 (15:37 -0500)]
Merge pull request #10888 from dachary/wip-17062-jewel

jewel: tests: fix TestClsRbd.mirror_image failure in upgrade:jewel-x-master-distro-basic-vps

8 years ago10.2.3 v10.2.3
Jenkins Build Slave User [Tue, 20 Sep 2016 22:04:23 +0000 (22:04 +0000)]
10.2.3

8 years agorgw: fix collection of object sync errors 11021/head
Yehuda Sadeh [Wed, 20 Jul 2016 19:59:50 +0000 (12:59 -0700)]
rgw: fix collection of object sync errors

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit aacc31f72cef6f151459e4a0543850edeeaf1938)

8 years agorgw: fix marker tracker completion handling
Yehuda Sadeh [Wed, 20 Jul 2016 19:43:48 +0000 (12:43 -0700)]
rgw: fix marker tracker completion handling

Was not tracking high markers correctly. Could only work if there was a single
hole in the completion range. Just keep a map of all the complete entries.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit f5801c726efdc2f3067a071e6bb5ac83fd0cd147)

8 years agorgw: collect() stops if error encoutered
Yehuda Sadeh [Wed, 20 Jul 2016 19:42:52 +0000 (12:42 -0700)]
rgw: collect() stops if error encoutered

and returns true if needed to be called again

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 5528932e4c327a0fb0ce34a676eb59707a575325)

8 years agorgw: back off bucket sync on failures, don't store marker
Yehuda Sadeh [Tue, 19 Jul 2016 22:32:03 +0000 (15:32 -0700)]
rgw: back off bucket sync on failures, don't store marker

Fixes: http://tracker.ceph.com/issues/16742
If we fail on any single entry in bucket, skip updating the marker tracker
so that next time we'll go over that entry, and back off. This will trigger
a report to the data sync error repo and eventually a retry on the failing
object.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 1f3fec807043fd313ef7c66ff48f18b82c8bfa66)

8 years agokrbd: don't segfault if images are unmapped concurrently 11018/head
Ilya Dryomov [Fri, 3 Jun 2016 15:24:48 +0000 (17:24 +0200)]
krbd: don't segfault if images are unmapped concurrently

"rbd map c" can die from a NULL dereference on any of this_pool,
this_image or this_snap in wait_for_udev_add():

    <image a is mapped>
    rbd map c
                                    rbd map b
                                    rbd unmap a
                                    rbd unmap b

However unlikely, this segfault is triggered by the rbd/concurrent.sh
workunit on a regular basis.

Similarly, "rbd showmapped" can die if an image to be listed is
unmapped.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 2bfecb1c0a0b2a314a5d137e6ca2cccd9ddc9b54)

8 years agoqa: rbd/simple_big.sh: drop extraneous sudo
Ilya Dryomov [Fri, 13 May 2016 15:57:12 +0000 (17:57 +0200)]
qa: rbd/simple_big.sh: drop extraneous sudo

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 6643f4d9723e5d9c5195680ac9abb9915ef42a56)

8 years agoqa: rbd/concurrent.sh: suppress rbd map output
Ilya Dryomov [Fri, 13 May 2016 13:05:53 +0000 (15:05 +0200)]
qa: rbd/concurrent.sh: suppress rbd map output

rbd_map_image() echoes the id of the image and is used in command
substitution:

    id=$(rbd_map_image "${image}")

Output from rbd map isn't consumed and clobbers the return.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit a7d4189587e343a581660f1a118f6667c26e5bd4)

8 years agoqa: rbd: don't modprobe, chown sysfs files or udevadm settle
Ilya Dryomov [Thu, 12 May 2016 13:07:12 +0000 (15:07 +0200)]
qa: rbd: don't modprobe, chown sysfs files or udevadm settle

This is a followup to 38a572011dbb ("qa: rbd/map-snapshot-io.sh: don't
chown sysfs files") for the rest of the workunits:

- /sys/bus/rbd/add_single_major is used if present, so chown'ing
  /sys/bus/rbd/add doesn't help.  Use sudo on rbd map instead.

- Don't modprobe or udevadm settle - rbd CLI tool takes care of that.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 6bddef8ecff97f732bf707d9ba6d2386bc147f5a)

8 years agoqa: rbd/map-snapshot-io.sh: don't chown sysfs files
Ilya Dryomov [Wed, 11 May 2016 15:26:39 +0000 (17:26 +0200)]
qa: rbd/map-snapshot-io.sh: don't chown sysfs files

/sys/bus/rbd/add_single_major is used if present, so chown'ing
/sys/bus/rbd/add doesn't help.  Use sudo on rbd map instead.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 38a572011dbbf8298cb7ba8ebf43909967eec689)

8 years agoMerge pull request #10804 from dachary/wip-17089-jewel
Loic Dachary [Mon, 29 Aug 2016 11:43:15 +0000 (13:43 +0200)]
Merge pull request #10804 from dachary/wip-17089-jewel

jewel: OSD failed to subscribe skipped osdmaps after ceph osd pause

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #10760 from dachary/wip-16865-jewel
Loic Dachary [Mon, 29 Aug 2016 11:43:00 +0000 (13:43 +0200)]
Merge pull request #10760 from dachary/wip-16865-jewel

jewel: saw valgrind issues in ReplicatedPG::new_repop

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #10662 from dachary/wip-16901-jewel
Loic Dachary [Fri, 26 Aug 2016 20:12:39 +0000 (22:12 +0200)]
Merge pull request #10662 from dachary/wip-16901-jewel

jewel: segfault in RGWOp_MDLog_Notify

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
8 years agoMerge pull request #10763 from dachary/wip-17034-jewel
Orit Wasserman [Fri, 26 Aug 2016 08:31:06 +0000 (10:31 +0200)]
Merge pull request #10763 from dachary/wip-17034-jewel

jewel: rgw: object expirer's hints might be trimmed without processing in some circumstances
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #10658 from dachary/wip-16862-jewel
Orit Wasserman [Fri, 26 Aug 2016 08:28:10 +0000 (10:28 +0200)]
Merge pull request #10658 from dachary/wip-16862-jewel

jewel: default zone and zonegroup cannot be added to a realm
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #10659 from dachary/wip-16863-jewel
Orit Wasserman [Fri, 26 Aug 2016 08:26:53 +0000 (10:26 +0200)]
Merge pull request #10659 from dachary/wip-16863-jewel

jewel: use zone endpoints instead of zonegroup endpoints
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #10660 from dachary/wip-16864-jewel
Orit Wasserman [Fri, 26 Aug 2016 08:24:20 +0000 (10:24 +0200)]
Merge pull request #10660 from dachary/wip-16864-jewel

jewel: multisite segfault on ~RGWRealmWatcher if realm was deleted
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agorgw: delete region map after upgrade to zonegroup map 10890/head
Casey Bodley [Tue, 23 Aug 2016 19:10:44 +0000 (15:10 -0400)]
rgw: delete region map after upgrade to zonegroup map

convert_regionmap() reads the region map and uses it to initialize the
zonegroup map.  but it doesn't remove the region_map afterwards, so
radosgw (and some radosgw-admin commands) will keep doing this on
startup, overwriting any changes made to the period/zonegroup map

Fixes: http://tracker.ceph.com/issues/17051
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 215cd6f6ff7b8cb74df6dc3b94a5928f56ab9540)

8 years agoqa/workunits: support filtering cls_rbd unit test cases 10888/head
Jason Dillaman [Fri, 12 Aug 2016 18:44:09 +0000 (14:44 -0400)]
qa/workunits: support filtering cls_rbd unit test cases

Fixes: http://tracker.ceph.com/issues/16529
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0891ede57cd254aa75c66301fde4a2eb6957a3d6)

8 years agoMerge pull request #10847 from gregsfortytwo/jewel-fix-mds
Loic Dachary [Thu, 25 Aug 2016 22:46:17 +0000 (00:46 +0200)]
Merge pull request #10847 from gregsfortytwo/jewel-fix-mds

mds: fix double-unlock on shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agoMerge pull request #10663 from dachary/wip-16934-jewel
Loic Dachary [Thu, 25 Aug 2016 21:04:37 +0000 (23:04 +0200)]
Merge pull request #10663 from dachary/wip-16934-jewel

jewel: Add zone rename to radosgw_admin

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10664 from dachary/wip-16958-jewel
Loic Dachary [Thu, 25 Aug 2016 21:04:22 +0000 (23:04 +0200)]
Merge pull request #10664 from dachary/wip-16958-jewel

jewel: Bug when using port 443s in rgw.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10764 from dachary/wip-17032-jewel
Loic Dachary [Thu, 25 Aug 2016 21:04:07 +0000 (23:04 +0200)]
Merge pull request #10764 from dachary/wip-17032-jewel

jewel: multisite: RGWPeriodPuller tries to pull from itself

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10765 from dachary/wip-17006-jewel
Loic Dachary [Thu, 25 Aug 2016 21:03:40 +0000 (23:03 +0200)]
Merge pull request #10765 from dachary/wip-17006-jewel

jewel: Increase log level for messages occuring while running rgw admin command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10845 from cbodley/wip-16702
Loic Dachary [Thu, 25 Aug 2016 20:25:21 +0000 (22:25 +0200)]
Merge pull request #10845 from cbodley/wip-16702

jewel: rgw: add tenant support to multisite sync

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agorgw: adjust manifest head object 10866/head
Yehuda Sadeh [Fri, 8 Jul 2016 01:01:54 +0000 (18:01 -0700)]
rgw: adjust manifest head object

adjust the manifest head object:
 - when reading manifest, set the head object to the object
   we read the manifest from (and its size). Some manifests are
   broken and point at a different object
 - when copying multipart object, set the manifest head object to
   point at the new head object

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 419264586fc46337322f0d60b0ffcdcce3bb5c5a)

8 years agorgw: adjust objs when copying obj with explicit_objs set
Yehuda Sadeh [Thu, 7 Jul 2016 22:49:07 +0000 (15:49 -0700)]
rgw: adjust objs when copying obj with explicit_objs set

If the head object contains data, need to set it in the list of
objs (in addition to being pointed at by head_obj).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 1e012112b99004745952accc6fb11ada5e8e0045)

8 years agorgw: patch manifest to handle explicit objs copy issue
Yehuda Sadeh [Thu, 7 Jul 2016 22:36:33 +0000 (15:36 -0700)]
rgw: patch manifest to handle explicit objs copy issue

Fixes: http://tracker.ceph.com/issues/16435
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit b51476466e5bb03bbaa6e428bb6bb189a259b9fd)

8 years agoMerge pull request #9405 from SUSE/wip-16083-jewel
Loic Dachary [Thu, 25 Aug 2016 07:17:52 +0000 (09:17 +0200)]
Merge pull request #9405 from SUSE/wip-16083-jewel

jewel: mds: wrongly treat symlink inode as normal file/dir when symlink inode is stale on kcephfs

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10654 from dachary/wip-9577-jewel
Loic Dachary [Thu, 25 Aug 2016 07:02:17 +0000 (09:02 +0200)]
Merge pull request #10654 from dachary/wip-9577-jewel

jewel: mon: "mon metadata" fails when only one monitor exists

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agomds: fix double-unlock on shutdown 10847/head
Greg Farnum [Wed, 24 Aug 2016 20:34:43 +0000 (13:34 -0700)]
mds: fix double-unlock on shutdown

We did a bad backport or something and accidentally ended up with two Unlock()
calls on mds_lock. Don't.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
8 years agorgw: collect skips a specific coroutine stack 10845/head
Yehuda Sadeh [Wed, 13 Jul 2016 03:36:35 +0000 (20:36 -0700)]
rgw: collect skips a specific coroutine stack

Fixes: http://tracker.ceph.com/issues/16665
Instead of drain_all_but() that specifies number of stacks to leave behind,
added drain_all_but_stack() that has a specific stack specified. This is needed
so that we don't call wakeup() through lease_cr->go_down() on a cr stack that
was already collected.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 5a2e8f0526db92a290c711f82627fc5042c290ea)

8 years agoMerge pull request #10791 from dachary/wip-17004-jewel
Loic Dachary [Wed, 24 Aug 2016 20:08:58 +0000 (22:08 +0200)]
Merge pull request #10791 from dachary/wip-17004-jewel

jewel: rbd-mirror: FAILED assert(m_state == STATE_STOPPING)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10827 from dillaman/wip-16950-jewel
Loic Dachary [Wed, 24 Aug 2016 20:08:30 +0000 (22:08 +0200)]
Merge pull request #10827 from dillaman/wip-16950-jewel

jewel: librbd: delay acquiring lock if image watch has failed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agorgw: fix compilation
Josh Durgin [Tue, 12 Jul 2016 00:38:43 +0000 (17:38 -0700)]
rgw: fix compilation

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 874de58024e12d893ae050cd421ef67c9521c753)

8 years agorgw: remove datalog keys from error repo on ENOENT
Casey Bodley [Thu, 7 Jul 2016 18:42:00 +0000 (14:42 -0400)]
rgw: remove datalog keys from error repo on ENOENT

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 16976eedb70292e821193e39bb577a68df1bc95a)

8 years agotest/rgw: add optional --tenant flag to test_multi.py
Casey Bodley [Fri, 1 Jul 2016 16:36:04 +0000 (12:36 -0400)]
test/rgw: add optional --tenant flag to test_multi.py

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6f65d192cd3917cdcf7d9deb5396750d2c641842)

8 years agorgw: add tenant id to GetBucketInstanceInfo
Casey Bodley [Thu, 30 Jun 2016 19:50:00 +0000 (15:50 -0400)]
rgw: add tenant id to GetBucketInstanceInfo

use the rgw_bucket overload of get_bucket_instance_info() so it can
supply the tenant id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3cf715c6828d7d9732bf6a6fd9b46bbd8c08de5f)

8 years agorgw: carry tenant id with data sync
Casey Bodley [Wed, 29 Jun 2016 15:58:55 +0000 (11:58 -0400)]
rgw: carry tenant id with data sync

use rgw_bucket_shard to track buckets, which includes tenant id

Fixes: http://tracker.ceph.com/issues/16469
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d945e2099525e39588b572e357b115df98c8cdca)

8 years agorgw: add tenant to url for RGWRESTStreamWriteRequest
Casey Bodley [Thu, 30 Jun 2016 21:43:40 +0000 (17:43 -0400)]
rgw: add tenant to url for RGWRESTStreamWriteRequest

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2a1e1a776c58f62fbcb57877d56b92eb1e8933e6)

8 years agorgw: add tenant to rgw_bucket json format
Casey Bodley [Thu, 30 Jun 2016 21:42:54 +0000 (17:42 -0400)]
rgw: add tenant to rgw_bucket json format

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit fc0df4802b48fa095587b93557c82da5446d9ca3)

8 years agorgw: add rgw_bucket_parse_bucket_key()
Casey Bodley [Thu, 30 Jun 2016 04:13:47 +0000 (00:13 -0400)]
rgw: add rgw_bucket_parse_bucket_key()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d5ac140040ea2f17e33d2846870cb033c7ae6ffa)

8 years agorgw: add tenant name to datalog entries
Casey Bodley [Tue, 28 Jun 2016 15:24:46 +0000 (11:24 -0400)]
rgw: add tenant name to datalog entries

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c8aa668fc3e34a7651ce6e2dda62902ced808541)

8 years agorgw: convert bucket instance listings back to metadata key format
Casey Bodley [Wed, 6 Jul 2016 15:50:47 +0000 (11:50 -0400)]
rgw: convert bucket instance listings back to metadata key format

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3eae201c8a1513bae23233700c754ec076360664)

8 years agorgw: use tenant/ for bucket instance metadata keys
Casey Bodley [Tue, 5 Jul 2016 13:00:19 +0000 (09:00 -0400)]
rgw: use tenant/ for bucket instance metadata keys

to work around the ambiguity of parsing tenant: and :shard in the same
bucket instance metadata key, use tenant/ instead

to preserve backward compatibility with existing objects, new helper
function rgw_bucket_instance_key_to_oid() converts this / back to a :
before being used as an object name

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a0befa1e7be3aeb37209bac2e2a7de0edf5d5a95)

8 years agorgw: add get_key() methods to format rgw_buckets
Casey Bodley [Thu, 30 Jun 2016 22:09:03 +0000 (18:09 -0400)]
rgw: add get_key() methods to format rgw_buckets

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 51ff8ef7c6bb7c9f55f3ca8ad059b029a4088cdf)

8 years agorgw: data sync debug logging
Yehuda Sadeh [Mon, 6 Jun 2016 23:16:33 +0000 (16:16 -0700)]
rgw: data sync debug logging

a high level debug logging of the data sync process.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 97ef16c0cc0ad8345b5a897108717d83ab0aa9ab)

8 years agorgw: modifying multi-site log messages.
Pritha Srivastava [Thu, 16 Jun 2016 09:15:57 +0000 (14:45 +0530)]
rgw: modifying multi-site log messages.

Increasing the log level and removing 'ERROR' from messages that appear during normal multi-site operation.
Fixes: http://tracker.ceph.com/issues/16121
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit e1737ddaa035f23757d021dd34b0c87d24ce372c)

8 years agoMerge pull request #10653 from dachary/wip-10495-jewel
Loic Dachary [Wed, 24 Aug 2016 16:57:23 +0000 (18:57 +0200)]
Merge pull request #10653 from dachary/wip-10495-jewel

jewel : 60-ceph-partuuid-workaround-rules still needed by debian jessie (udev 215-17)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #10357 from SUSE/wip-16748-jewel
Loic Dachary [Wed, 24 Aug 2016 16:51:54 +0000 (18:51 +0200)]
Merge pull request #10357 from SUSE/wip-16748-jewel

jewel: mount.ceph: move from ceph-base to ceph-common and add symlink in /sbin for SUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #10790 from dachary/wip-17005-jewel
Loic Dachary [Wed, 24 Aug 2016 15:48:00 +0000 (17:48 +0200)]
Merge pull request #10790 from dachary/wip-17005-jewel

jewel: ImageReplayer::is_replaying does not include flush state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10792 from dachary/wip-16978-jewel
Loic Dachary [Wed, 24 Aug 2016 15:47:45 +0000 (17:47 +0200)]
Merge pull request #10792 from dachary/wip-16978-jewel

jewel: rbd-mirror: FAILED assert(m_on_update_status_finish == nullptr)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>