]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agoExclusiveArch for suse_version 10614/head
Michel Normand [Fri, 5 Aug 2016 14:34:48 +0000 (16:34 +0200)]
ExclusiveArch for suse_version

for SLES supports only x86_64 and aarch64 targets
for openSUSE (Tumbleweed and Leap) add ppc64/ppc64le targets.

fixes: http://tracker.ceph.com/issues/16936
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
(cherry picked from commit a8db0f4d96d71223083bf6c2275acd892666a13e)

8 years agoMerge pull request #9952 from dillaman/wip-fix-use-after-free-jewel
Loic Dachary [Mon, 8 Aug 2016 06:47:34 +0000 (08:47 +0200)]
Merge pull request #9952 from dillaman/wip-fix-use-after-free-jewel

jewel: librbd: potential use after free on refresh error

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #9752 from dillaman/wip-fix-task-finisher-jewel
Loic Dachary [Mon, 8 Aug 2016 06:45:58 +0000 (08:45 +0200)]
Merge pull request #9752 from dillaman/wip-fix-task-finisher-jewel

jewel: librbd: cancel all tasks should wait until finisher is done

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #10561 from dzafman/wip-16672-jewel
David Zafman [Fri, 5 Aug 2016 19:46:10 +0000 (12:46 -0700)]
Merge pull request #10561 from dzafman/wip-16672-jewel

Reviewed-by: Samuel Just <sjust@redhat.com>
8 years agoMerge pull request #9562 from Abhishekvrshny/wip-16152-jewel
Loic Dachary [Fri, 5 Aug 2016 13:30:52 +0000 (15:30 +0200)]
Merge pull request #9562 from Abhishekvrshny/wip-16152-jewel

jewel: client: fstat cap release

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #9561 from Abhishekvrshny/wip-16136-jewel
Loic Dachary [Fri, 5 Aug 2016 13:30:41 +0000 (15:30 +0200)]
Merge pull request #9561 from Abhishekvrshny/wip-16136-jewel

jewel: MDSMonitor fixes

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #9557 from Abhishekvrshny/wip-15898-jewel
Loic Dachary [Fri, 5 Aug 2016 13:29:33 +0000 (15:29 +0200)]
Merge pull request #9557 from Abhishekvrshny/wip-15898-jewel

jewel: Confusing MDS log message when shut down with stalled journaler reads

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #9560 from Abhishekvrshny/wip-16135-jewel
Loic Dachary [Fri, 5 Aug 2016 13:29:23 +0000 (15:29 +0200)]
Merge pull request #9560 from Abhishekvrshny/wip-16135-jewel

jewel: MDS: fix getattr starve setattr

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #9559 from Abhishekvrshny/wip-16041-jewel
Loic Dachary [Fri, 5 Aug 2016 13:29:08 +0000 (15:29 +0200)]
Merge pull request #9559 from Abhishekvrshny/wip-16041-jewel

jewel: mds/StrayManager.cc: 520: FAILED assert(dnl->is_primary())

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #9996 from dachary/wip-16437-jewel
Loic Dachary [Fri, 5 Aug 2016 13:17:22 +0000 (15:17 +0200)]
Merge pull request #9996 from dachary/wip-16437-jewel

jewel: async messenger mon crash

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>
8 years agoMerge pull request #9997 from dachary/wip-16431-jewel
Loic Dachary [Fri, 5 Aug 2016 13:10:27 +0000 (15:10 +0200)]
Merge pull request #9997 from dachary/wip-16431-jewel

jewel: librados,osd: bad flags can crash the osd

Reviewed-by: Samuel Just <sjust@redhat.com>
8 years agoMerge pull request #9998 from dachary/wip-16429-jewel
Loic Dachary [Fri, 5 Aug 2016 13:10:08 +0000 (15:10 +0200)]
Merge pull request #9998 from dachary/wip-16429-jewel

jewel: OSDMonitor: drop pg temps from not the current primary

Reviewed-by: Samuel Just <sjust@redhat.com>
8 years agoMerge pull request #10001 from dachary/wip-16427-jewel
Loic Dachary [Fri, 5 Aug 2016 13:09:52 +0000 (15:09 +0200)]
Merge pull request #10001 from dachary/wip-16427-jewel

jewel: prepare_pgtemp needs to only update up_thru if newer than the existing one

Reviewed-by: Samuel Just <sjust@redhat.com>
8 years agoMerge pull request #9743 from vumrao/wip-vumrao-16339
Casey Bodley [Fri, 5 Aug 2016 13:09:23 +0000 (09:09 -0400)]
Merge pull request #9743 from vumrao/wip-vumrao-16339

jewel : rgw: support size suffixes for --max-size in radosgw-admin command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10007 from dachary/wip-15806-jewel
Loic Dachary [Thu, 4 Aug 2016 12:45:05 +0000 (14:45 +0200)]
Merge pull request #10007 from dachary/wip-15806-jewel

jewel: New pools have bogus stuck inactive/unclean HEALTH_ERR messages until they are first active and clean

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoMerge pull request #10006 from dachary/wip-16249-jewel
Loic Dachary [Thu, 4 Aug 2016 12:43:36 +0000 (14:43 +0200)]
Merge pull request #10006 from dachary/wip-16249-jewel

jewel: sparse_read on ec pool should return extends with correct offset

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #9740 from vumrao/wip-vumrao-16338
Loic Dachary [Thu, 4 Aug 2016 12:43:22 +0000 (14:43 +0200)]
Merge pull request #9740 from vumrao/wip-vumrao-16338

jewel : rados: Add cleanup message with time to rados bench output

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #10004 from dachary/wip-16374-jewel
Loic Dachary [Thu, 4 Aug 2016 12:42:53 +0000 (14:42 +0200)]
Merge pull request #10004 from dachary/wip-16374-jewel

jewel: AsyncConnection::lockmsg/async lockdep cycle: AsyncMessenger::lock, MDSDaemon::mds_lock, AsyncConnection::lock

Reviewed-by: Haomai Wang <haomai@xsky.com>
8 years agoMerge pull request #10003 from dachary/wip-16380-jewel
Loic Dachary [Thu, 4 Aug 2016 12:42:32 +0000 (14:42 +0200)]
Merge pull request #10003 from dachary/wip-16380-jewel

jewel: msg/async: connection race hang

Reviewed-by: Haomai Wang <haomai@xsky.com>
8 years agoMerge pull request #10036 from liewegas/wip-16297-jewel
Loic Dachary [Thu, 4 Aug 2016 10:35:14 +0000 (12:35 +0200)]
Merge pull request #10036 from liewegas/wip-16297-jewel

mon: Monitor: validate prefix on handle_command()

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #9547 from jcsp/wip-jewel-15705
Loic Dachary [Thu, 4 Aug 2016 07:53:10 +0000 (09:53 +0200)]
Merge pull request #9547 from jcsp/wip-jewel-15705

jewel backport: mds: fix mdsmap print_summary with standby replays

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoos: Fix HashIndex::recursive_remove() to remove everything but original path 10561/head
David Zafman [Wed, 3 Aug 2016 05:32:02 +0000 (22:32 -0700)]
os: Fix HashIndex::recursive_remove() to remove everything but original path

Fixes: http://tracker.ceph.com/issues/16672
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ba88a3aac0d3b620986e32ed718237513d6592f6)

8 years agofilestore: Clear objects before calling collection_list() again
David Zafman [Wed, 3 Aug 2016 06:26:41 +0000 (23:26 -0700)]
filestore: Clear objects before calling collection_list() again

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 892303cd02b64b622669183c2acffcdd7a8f3547)

8 years agofilestore: Improve logging
David Zafman [Wed, 3 Aug 2016 06:24:12 +0000 (23:24 -0700)]
filestore: Improve logging

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b78c457785553e3a6e4f901bfebb41fc9c4d9bd5)

8 years agoMerge pull request #10293 from theanalyst/wip-16589-jewel
Casey Bodley [Tue, 2 Aug 2016 14:28:00 +0000 (10:28 -0400)]
Merge pull request #10293 from theanalyst/wip-16589-jewel

jewel: multisite sync races with deletes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #10453 from ajarr/wip-manila-backports-jewel
John Spray [Tue, 2 Aug 2016 11:31:18 +0000 (12:31 +0100)]
Merge pull request #10453 from ajarr/wip-manila-backports-jewel

jewel: essential backports for OpenStack Manila

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoceph_volume_client: version on-disk metadata 10453/head
Ramana Raja [Thu, 7 Jul 2016 11:45:13 +0000 (17:15 +0530)]
ceph_volume_client: version on-disk metadata

Version on-disk metadata with two attributes,
'compat version', the minimum CephFSVolume Client
version that can decode the metadata, and
'version', the version that encoded the metadata.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 1c1d65a45f4574ca5f33cc9d949089a5c956e363)

8 years agoceph_volume_client: add versioning
Ramana Raja [Wed, 6 Jul 2016 10:03:06 +0000 (15:33 +0530)]
ceph_volume_client: add versioning

Add class attributes to CephFSVolumeClient to version
its capabilities.

'version' attribute stores the current version number
of CephFSVolumeClient.

'compat_version' attribute stores the earliest version
number of CephFSVolumeClient that the current version is
compatible with.

Fixes: http://tracker.ceph.com/issues/15406
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 46876fb2ceb22082c0a1703fe77ad1694b508ad8)

8 years agoceph_volume_client: disallow tenants to share auth IDs
Ramana Raja [Thu, 23 Jun 2016 17:52:12 +0000 (23:22 +0530)]
ceph_volume_client: disallow tenants to share auth IDs

Restrict an auth ID to a single OpenStack tenant to enforce
strong tenant isolation of shares.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 82445a20a258a4c8800f273dc5f2484aace0e413)

8 years agoceph_volume_client: cleanup auth meta files
Ramana Raja [Thu, 23 Jun 2016 11:41:33 +0000 (17:11 +0530)]
ceph_volume_client: cleanup auth meta files

Remove auth meta files on last rule for an auth ID deletion

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit ec2e6e37d01c961d269d83661d0b95ada6a8449e)

8 years agoceph_volume_client: fix log messages
Ramana Raja [Thu, 23 Jun 2016 11:25:30 +0000 (16:55 +0530)]
ceph_volume_client: fix log messages

Log the path of the volume during creation and deletion of volumes.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 7731287761f91e1fdee0e6306d9ecf9b04ad363c)

8 years agoceph_volume_client: create/delete VMeta for create/delete volume
Ramana Raja [Thu, 23 Jun 2016 11:01:23 +0000 (16:31 +0530)]
ceph_volume_client: create/delete VMeta for create/delete volume

Create and delete volume meta files during creation and deletion of
volumes.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 37fbfc7aa8f10d37f5202603a906425507271ff5)

8 years agoceph_volume_client: modify locking of meta files
Ramana Raja [Thu, 23 Jun 2016 10:36:53 +0000 (16:06 +0530)]
ceph_volume_client: modify locking of meta files

File locks are applied on meta files before updating the meta
file contents. These meta files would need to be cleaned up
sometime, which could lead to locks being held on unlinked meta
files. Prevent this by checking whether the file had been deleted
after lock was acquired on it.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit f7c037229bcf3f5a3d06897ec7fe0c5419dd7143)

8 years agocephfs.pyx: implement python bindings for fstat
Ramana Raja [Thu, 23 Jun 2016 12:09:32 +0000 (17:39 +0530)]
cephfs.pyx: implement python bindings for fstat

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit f58403f3d19e22edeb8f91b6f87a0b7947b0ff21)

8 years agoceph_volume_client: restrict volume group names
Ramana Raja [Wed, 8 Jun 2016 11:27:01 +0000 (16:57 +0530)]
ceph_volume_client: restrict volume group names

Prevent craftily-named volume groups from colliding with meta files.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 7f7d2a76ae9b556c1de418f0eab8461c538f91d9)

8 years agoceph_volume_client: use fsync instead of syncfs
Ramana Raja [Wed, 8 Jun 2016 11:04:56 +0000 (16:34 +0530)]
ceph_volume_client: use fsync instead of syncfs

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 27eb51baab4cda6b385aef53fc7c3962a9debae5)

8 years agopybind: expose fsync in cephfs binding.
Xiaoxi Chen [Fri, 29 Apr 2016 19:47:42 +0000 (14:47 -0500)]
pybind: expose fsync in cephfs binding.

So we don't necessary to syncfs when want to persistent
some file.

Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>
(cherry picked from commit 1c952fbaf0fd393ef2dcb83a3db721a077b4274e)

8 years agoceph_volume_client: recover from dirty auth and auth meta updates
Ramana Raja [Tue, 7 Jun 2016 19:12:18 +0000 (00:42 +0530)]
ceph_volume_client: recover from dirty auth and auth meta updates

Check dirty flag after locking something and call recover() if we are
opening something dirty (racing with another instance of the driver
restarting after failure) -- only required if someone running multiple
manila-share instances with Ceph loaded.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 647a2447f0c4354dc21d1083043591d2b6f6f94f)

8 years agoceph_volume_client: modify data layout in meta files
Ramana Raja [Tue, 21 Jun 2016 06:44:56 +0000 (12:14 +0530)]
ceph_volume_client: modify data layout in meta files

Notable changes to data layout in auth meta and volume meta files:

In the auth meta files, add a 'dirty' flag to track the status of auth
updates to a single volume.

In the volume meta file, make the 'dirty' flag track the status of
auth updates for a single ID.

Optimize the recovery of partial auth update changes to auth meta,
volume meta, and the Ceph backend, facilitated by changes in the
data layout in the meta files.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 7c8a28a7e817d030a4d419b0cc627c30c1064270)

8 years agopybind: ceph_volume_client authentication metadata
John Spray [Wed, 2 Mar 2016 12:30:45 +0000 (12:30 +0000)]
pybind: ceph_volume_client authentication metadata

Store a two-way mapping between auth IDs and volumes.

Enables us to record some metadata on auth ids (which
openstack tenant created it) so that we can avoid exposing
keys to other tenants who try to use the same ceph
auth id.

Enables us to expose the list of which auth ids have access
to a volume, so that Manila's update_access() can be
implemented efficiently.

DNM: see TODOs inline.

Fixes: http://tracker.ceph.com/issues/15615
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit d2e9eb55ca6ed5daa094cf323faf143615b9380b)

8 years agopybind: enable integer flags to libcephfs open
John Spray [Mon, 7 Mar 2016 13:06:41 +0000 (13:06 +0000)]
pybind: enable integer flags to libcephfs open

The 'rw+' style flags are handy and convenient, but
they don't capture all possibilities.  Change to
optionally accept an integer here for advance users
who want to specify arbitrary combinations of
flags.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5678584f4176d07301acd7f57acc4efd7fb20e43)

8 years agoceph_volume_client: allow read-only authorization for volumes
Ramana Raja [Wed, 27 Apr 2016 17:56:44 +0000 (23:26 +0530)]
ceph_volume_client: allow read-only authorization for volumes

Allow clients to be restricted to read-only mount of the volume
by restricting their ceph auth ID's MDS and OSD caps to read-only.

Fixes: http://tracker.ceph.com/issues/15614
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 011ea5e7fb35ee07848e0c3abac24702a778ad63)

9 years agoMerge pull request #10373 from ceph/jewel-mds-snap-failover
John Spray [Thu, 21 Jul 2016 12:58:16 +0000 (13:58 +0100)]
Merge pull request #10373 from ceph/jewel-mds-snap-failover

Jewel mds snap failover

Reviewed-by: John Spray <john.spray@redhat.com>
9 years agoclient: fix MetaRequest::set_other_inode() 10373/head
Yan, Zheng [Mon, 11 Jul 2016 07:32:41 +0000 (15:32 +0800)]
client: fix MetaRequest::set_other_inode()

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 265f96bda7e106883063f074133450ce39ee262c)

9 years agoclient: close directory's snapdir when deleting directory
Yan, Zheng [Mon, 11 Jul 2016 03:25:37 +0000 (11:25 +0800)]
client: close directory's snapdir when deleting directory

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 3099cabd11a16d22c6707631861bef0aa176ed02)

9 years agoclient: invalidate snap inodes after removing snapshot
Yan, Zheng [Mon, 11 Jul 2016 03:07:01 +0000 (11:07 +0800)]
client: invalidate snap inodes after removing snapshot

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit f180ad149ab510626fcd6cbd8221f550f9858126)

9 years agomds: fix incorrect "unconnected snaprealm xxx" warning
Yan, Zheng [Mon, 11 Jul 2016 02:51:13 +0000 (10:51 +0800)]
mds: fix incorrect "unconnected snaprealm xxx" warning

If a snaprealm has no child/parent snaprelam, and the snaprealm inode
is not in the cache while client reconnects. The snaprealm does not
get properly removed from MDCache::reconnected_snaplrealm. This causes
incorrect "unconnected snaprealm xxx" warning

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 85e687de876ca7e0174734ba81130949c4ab6a40)

9 years agoqa/workunits/fs: fix expect_failure function in test scripts
Yan, Zheng [Thu, 30 Jun 2016 08:05:57 +0000 (16:05 +0800)]
qa/workunits/fs: fix expect_failure function in test scripts

The origin expect_failure function return 0 regardness of command's
return value.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 16f7d7c71e65cd68cecde3e5b89d189ab5950548)

9 years agoclient: make sure snapflush is sent before normal cap message
Yan, Zheng [Wed, 29 Jun 2016 12:49:40 +0000 (20:49 +0800)]
client: make sure snapflush is sent before normal cap message

MDS does null snapflush when it receives normal cap message. So client
must send snapflush first.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit d3916717e2edc8000400f678fa7134ca1406a074)

9 years agoclient: unify cap flush and snapcap flush
Yan, Zheng [Tue, 28 Jun 2016 12:39:08 +0000 (20:39 +0800)]
client: unify cap flush and snapcap flush

This patch includes following changes
- assign flush tid to snapcap flush
- remove session's flushing_capsnaps list. add inode with snapcap
  flushes to session's flushing_caps list instead.
- when reconnecting to MDS, re-send one inode's snapcap flushes and
  cap flushes at the same time.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit a05e996b2a2c36496abd8538829ac4897392f6eb)

9 years agomds: handle partly purged directory
Yan, Zheng [Wed, 29 Jun 2016 09:15:01 +0000 (17:15 +0800)]
mds: handle partly purged directory

For a snapshoted direcotry whose snaprealm parents are being opened,
MDS does not know if the directory is purgeable. So MDS can't skip
committing dirfrags of the directory. But if the direcotry is purgeale,
some dirfrags could have already been deleted during MDS failover.
Committing them could return -ENOENT.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit bc50e0309280c08c3ca79dfa5514ac3a15f81a23)

9 years agomds: do files recovery after processing cap flushes
Yan, Zheng [Wed, 29 Jun 2016 03:42:42 +0000 (11:42 +0800)]
mds: do files recovery after processing cap flushes

File recovery may update inode and trigger inode COW. MDS relies on
client caps to setup CInode::client_need_snapflush. But for a given
client, the reconnected caps may not include the flushing caps.
(Before MDS failover, client released and flushed some caps at the
same time. When MDS recovers, client re-send the cap flush and send
cap reconnect to the MDS.) This may cause later snapflush to get
dropped.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit dd98448d3d0bbb7d338f03f7ad1e40f217eebe0d)

9 years agomds: combine MDCache::{reconnected_caps,cap_imports_dirty}
Yan, Zheng [Wed, 29 Jun 2016 03:25:12 +0000 (11:25 +0800)]
mds: combine MDCache::{reconnected_caps,cap_imports_dirty}

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 57067e032e84bbdb69c499aa7ea93ca68082569b)

9 years agomds: remove CEPH_LOCK_IFLOCKL from cinode_lock_info
Yan, Zheng [Fri, 24 Jun 2016 09:09:34 +0000 (17:09 +0800)]
mds: remove CEPH_LOCK_IFLOCKL from cinode_lock_info

Currently we don't support dirty CEPH_CAP_FLOCK_EXCL

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit cfc3ec17b6f245e6d8a0be4fdf6cfa64d2fb725f)

9 years agomds: rebuild the internal states that tracking pending snapflush
Yan, Zheng [Thu, 23 Jun 2016 08:59:46 +0000 (16:59 +0800)]
mds: rebuild the internal states that tracking pending snapflush

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 1b7d198f633a8608b704f82f01a3f4a4a3a4892b)

9 years agomds: using cap_reconnect_t to track cap recconect
Yan, Zheng [Wed, 22 Jun 2016 12:34:41 +0000 (20:34 +0800)]
mds: using cap_reconnect_t to track cap recconect

Previous commit extended cap_reconnect_t to include snapflush
related information. This information are needed in various
places

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 25b6011dcf07c1fc663cbe29ffd119b66545a0ac)

9 years agomds: add 'follows' of first pending snapflush to cap reconnect message
Yan, Zheng [Wed, 22 Jun 2016 07:56:29 +0000 (15:56 +0800)]
mds: add 'follows' of first pending snapflush to cap reconnect message

This helps the recovering MDS to reconstruct the internal states that
tracking pending snapflush.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit dd3963a878d4bb2fb5992278ccbc9f7633fb8786)

9 years agomds: journal snap inodes that need flush when expiring log segment
Yan, Zheng [Tue, 21 Jun 2016 09:17:56 +0000 (17:17 +0800)]
mds: journal snap inodes that need flush when expiring log segment

Treat snap inodes that need flush in the same way as open files.
When MDS recovers, this make sure that journal replay bring snap
inodes that need flush into the cache

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 306153b3d012832bdfa20402077fa60a9a5d626c)

9 years agomds: properly update client_snap_caps when splitting snap inode
Yan, Zheng [Tue, 21 Jun 2016 08:20:58 +0000 (16:20 +0800)]
mds: properly update client_snap_caps when splitting snap inode

update the new snap inode's client_snap_caps according to the old
snap inode.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 4883779935219817c7e391940a025be1679daeb5)

9 years agoinstall-deps: Get the latest virtualenv in a temporary folder
Oleh Prypin [Wed, 13 Jul 2016 22:06:51 +0000 (01:06 +0300)]
install-deps: Get the latest virtualenv in a temporary folder
to work around a bug in old virtualenv

Signed-off-by: Oleh Prypin <oleh@pryp.in>
(cherry picked from commit 2699b6d89b8c1c193fd86b5233d1ea86458753a0)

9 years agoMerge remote-tracking branch 'upstream/pull/10298/head' into jewel
Patrick Donnelly [Thu, 14 Jul 2016 16:10:11 +0000 (12:10 -0400)]
Merge remote-tracking branch 'upstream/pull/10298/head' into jewel

* upstream/pull/10298/head:
  doc: fix standby replay config

9 years agodoc: fix standby replay config 10298/head
Patrick Donnelly [Tue, 12 Jul 2016 19:43:23 +0000 (15:43 -0400)]
doc: fix standby replay config

I tried using these settings in tests without success. The correct config names
are prefixed with "mds".

Fixes: http://tracker.ceph.com/issues/16664
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
9 years agorgw: add pg_ver to tombstone_cache 10293/head
Casey Bodley [Tue, 21 Jun 2016 19:09:53 +0000 (15:09 -0400)]
rgw: add pg_ver to tombstone_cache

a tombstone cache was added to remember the mtime of deleted objects for
use with the HTTP_IF_MODIFIED_SINCE header, but the comparison was still
failing because of a missing pg_ver. added pg_ver to the tombstone cache
so it can be passed with HTTP_DEST_PG_VER

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit adb529f2fb983df671a1db58a0b17862a29762f0)

9 years agorgw: add obj tombstone cache
Yehuda Sadeh [Fri, 10 Jun 2016 21:35:01 +0000 (14:35 -0700)]
rgw: add obj tombstone cache

The obj tombstone cache is used in multi-zone environmet to keep
track of removed objects' mtime. This is then used to fetch remote
object only if its newer than the object that was removed, otherwise
we're just fetching ghost of the past.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit eb10214920c23b24edd94ca53d0f36c85404644d)

9 years agorgw: rgw_obj comparison also accounts for bucket id
Yehuda Sadeh [Fri, 10 Jun 2016 21:34:41 +0000 (14:34 -0700)]
rgw: rgw_obj comparison also accounts for bucket id

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit a7f32c4b1bd951b1a18b7d13013dc8e2822d6ffc)

9 years agocls/rgw: reduce tag_timeout significantly
Yehuda Sadeh [Thu, 9 Jun 2016 21:30:42 +0000 (14:30 -0700)]
cls/rgw: reduce tag_timeout significantly

The tag timeout is used for applying dir_suggest changes. Shorten it
so that changes will be reported early and can be used in data sync.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 99efdc673b7aed439eebdaa92ff117ba9646dd7c)

9 years agocls_rgw: dir_suggest entries bucket index logging
Yehuda Sadeh [Thu, 9 Jun 2016 21:22:07 +0000 (14:22 -0700)]
cls_rgw: dir_suggest entries bucket index logging

Fixes: http://tracker.ceph.com/issues/16222
Log entries that were set through the dir_suggest mechanism in
the bucket index log.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit ae00c5529219d81987425160dbb2a4e937661a6c)

9 years agocls/rgw: fix timespan comparison
Yehuda Sadeh [Thu, 9 Jun 2016 19:59:35 +0000 (12:59 -0700)]
cls/rgw: fix timespan comparison

Fixes: http://tracker.ceph.com/issues/16220
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit e0a07f70f7740a66ffa2646e0a57e8bdc0285373)

9 years agorgw: data sync squash, prioritize object removal
Yehuda Sadeh [Sat, 4 Jun 2016 12:47:50 +0000 (05:47 -0700)]
rgw: data sync squash, prioritize object removal

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 2fcd8b1d49aae2fd03b986dd10bb3f98d3b8f32e)

9 years agorgw: squash bi complete bi log entries
Yehuda Sadeh [Sat, 4 Jun 2016 10:29:12 +0000 (03:29 -0700)]
rgw: squash bi complete bi log entries

only process the newest complete operation in a bilog listing.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit f69db8e455f07c5594363c5beac329cb964be1ff)

9 years agorgw: stop bucket lease only after draining object sync operations
Yehuda Sadeh [Fri, 3 Jun 2016 09:03:13 +0000 (02:03 -0700)]
rgw: stop bucket lease only after draining object sync operations

Fixes: http://tracker.ceph.com/issues/16143
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit be6ad9a76c3008ea25a737c0de718faab8fca611)

9 years agorgw: don't leak RGWRados::cr_registry on error
Casey Bodley [Tue, 24 May 2016 16:57:19 +0000 (12:57 -0400)]
rgw: don't leak RGWRados::cr_registry on error

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 53a7f1a9951fa21cb46b5fb3914db3547b568aa5)

9 years agorgw: dont need atomic RGWRados::next_rados_handle
Casey Bodley [Tue, 24 May 2016 14:40:25 +0000 (10:40 -0400)]
rgw: dont need atomic RGWRados::next_rados_handle

next_rados_handle is only accessed under an exclusive handle_lock

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1adff94b720d01240040fdffebdbf53efdd528a5)

9 years agorgw: remove unneeded RGWRados::num_rados_handles
Casey Bodley [Tue, 24 May 2016 14:23:26 +0000 (10:23 -0400)]
rgw: remove unneeded RGWRados::num_rados_handles

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 72d5a485e41ac5824c30556b6cfe659094cd303c)

9 years agorgw: use vector for librados handles
Casey Bodley [Tue, 3 May 2016 14:35:41 +0000 (10:35 -0400)]
rgw: use vector for librados handles

using a vector instead of an array of pointers cleans up our
initialization/shutdown logic

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 02fb39d7f5835ada4d6304f318203444dc7eedc9)

9 years agoMerge pull request #9268 from theanalyst/wip-15992-jewel
Casey Bodley [Tue, 12 Jul 2016 14:09:34 +0000 (10:09 -0400)]
Merge pull request #9268 from theanalyst/wip-15992-jewel

jewel: radosgw-admin: EEXIST messages for create operations

Reviewed-by: Casey Bodley <cbodley@redhat.com>
9 years agoMerge pull request #8497 from sabaini/jewel
Loic Dachary [Sun, 10 Jul 2016 21:03:16 +0000 (23:03 +0200)]
Merge pull request #8497 from sabaini/jewel

ceph-disk: Accept bcache devices as data disks

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9545 from Abhishekvrshny/wip-16117-jewel
Matt Benjamin [Wed, 6 Jul 2016 18:18:44 +0000 (14:18 -0400)]
Merge pull request #9545 from Abhishekvrshny/wip-16117-jewel

jewel: rgw: aws4 parsing issue

9 years agoMerge pull request #9568 from dreamhost/wip-16182
Loic Dachary [Wed, 6 Jul 2016 09:55:13 +0000 (11:55 +0200)]
Merge pull request #9568 from dreamhost/wip-16182

jewel: rgw: backport merge of static sites fixes

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
9 years agoMerge pull request #9099 from Abhishekvrshny/wip-15841-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:36:20 +0000 (10:36 +0200)]
Merge pull request #9099 from Abhishekvrshny/wip-15841-jewel

jewel: s3website: x-amz-website-redirect-location header returns malformed HTTP response
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9265 from Abhishekvrshny/wip-15965-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:35:23 +0000 (10:35 +0200)]
Merge pull request #9265 from Abhishekvrshny/wip-15965-jewel

jewel: No Last-Modified, Content-Size and X-Object-Manifest headers if no segments in DLO manifest
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9267 from Abhishekvrshny/wip-15960-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:34:42 +0000 (10:34 +0200)]
Merge pull request #9267 from Abhishekvrshny/wip-15960-jewel

jewel: rgw: custom metadata aren't camelcased in Swift's responses
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9316 from Abhishekvrshny/wip-15954-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:28:45 +0000 (10:28 +0200)]
Merge pull request #9316 from Abhishekvrshny/wip-15954-jewel

jewel: rgw: initial slashes are not properly handled in Swift's BulkDelete
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9390 from vumrao/wip-vumrao-16071
Orit Wasserman [Thu, 30 Jun 2016 08:26:09 +0000 (10:26 +0200)]
Merge pull request #9390 from vumrao/wip-vumrao-16071

jewel: rgw : cleanup radosgw-admin temp command as it was deprecated
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9542 from Abhishekvrshny/wip-15967-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:25:51 +0000 (10:25 +0200)]
Merge pull request #9542 from Abhishekvrshny/wip-15967-jewel

jewel: rgw: account/container metadata not actually present in a request are deleted during POST through Swift API
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9543 from Abhishekvrshny/wip-16040-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:25:30 +0000 (10:25 +0200)]
Merge pull request #9543 from Abhishekvrshny/wip-16040-jewel

jewel: rgw: updating CORS/ACLs might not work in some circumstances
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9631 from dillaman/wip-16232
Loic Dachary [Thu, 30 Jun 2016 07:41:20 +0000 (09:41 +0200)]
Merge pull request #9631 from dillaman/wip-16232

jewel: test: improve rbd-mirror test case coverage

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agomon: Monitor: validate prefix on handle_command() 10036/head
youji [Tue, 14 Jun 2016 18:12:16 +0000 (11:12 -0700)]
mon: Monitor: validate prefix on handle_command()

Fixes: http://tracker.ceph.com/issues/16297
Signed-off-by: You Ji <youji@ebay.com>
(cherry picked from commit 7cb3434fed03a5497abfd00bcec7276b70df0654)

9 years agoMerge pull request #9294 from theanalyst/wip-16009-jewel
Loic Dachary [Wed, 29 Jun 2016 14:47:15 +0000 (16:47 +0200)]
Merge pull request #9294 from theanalyst/wip-16009-jewel

jewel: radosgw-admin: failure for user create after upgrade from hammer to jewel

Reviewed-by: Casey Bodley <cbodley@redhat.com>
9 years agoosd/PG: set last_* PG fields on a creating PG 10007/head
Sage Weil [Mon, 2 May 2016 18:28:55 +0000 (14:28 -0400)]
osd/PG: set last_* PG fields on a creating PG

Use the value from pg_history_t that ultimately came from the
mon if last_epoch_started == 0.  This establishes a sane lower
bound on these timestamps, and prevents a mon health error from
coming up about how a PG is stuck in whatever state while it is
being created.

(We addressed half of this problem in commit 6ca6aeac, but the
warning comes back as soon as the OSD reports with a creating
state.)

Fixes: http://tracker.ceph.com/issues/14952
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3be3bc60c12448a36f607c8d4fbf3300c7bbdbee)

9 years agoosd: set pg history last_clean_scrub_stamp on create
Sage Weil [Mon, 2 May 2016 18:27:16 +0000 (14:27 -0400)]
osd: set pg history last_clean_scrub_stamp on create

We were setting the other two; set this one as well.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 47f540d5b7ecc4ae193057df429db24ca6b3de8d)

9 years agoosd: sparse_read offset may not be zero for ecpool 10006/head
kofiliu [Fri, 27 May 2016 07:45:06 +0000 (03:45 -0400)]
osd: sparse_read offset may not be zero for ecpool

Signed-off-by: kofiliu <liu.xuan@h3c.com>
(cherry picked from commit 65e8738611cde0090619b3566a2e25f83b4c8468)

9 years agoMerge pull request #9721 from trociny/wip-16289-jewel
Mykola Golub [Wed, 29 Jun 2016 08:42:43 +0000 (11:42 +0300)]
Merge pull request #9721 from trociny/wip-16289-jewel

jewel: qa/workunits/rbd: respect RBD_CREATE_ARGS environment variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agomsg/async/AsyncConnection: make verify auth callback without connection lock 10004/head
Haomai Wang [Sat, 11 Jun 2016 05:39:23 +0000 (13:39 +0800)]
msg/async/AsyncConnection: make verify auth callback without connection lock

Fixes: http://tracker.ceph.com/issues/16237
Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 2b7776545c3f87d7f54a53190e65ec48378eaa05)

9 years agoAsyncConnection: create writable event for in progress connection 10003/head
Haomai Wang [Thu, 12 May 2016 04:01:35 +0000 (12:01 +0800)]
AsyncConnection: create writable event for in progress connection

Previously we use a tricky with ceph msgr protocol, if initiator side got
in progress connection state, it will wait until read event. Because if
tcp session built successfully server side will send the banner firstly
and initiator side will get read event. Otherwise, if connection failed,
read event also be called.

But actually man(2)[http://man7.org/linux/man-pages/man2/connect.2.html]
specify if we want to get notification whether connection built, we need
to listen writable event. It means when connection built, send buffer
is ready to be written.

This patch follow the strict nonblock connect process. Not fully sure fix
http://tracker.ceph.com/issues/15849

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit a74ce419133881ff8618733a0501c4a47e1368e3)

9 years agoOSDMonitor::prepare_pgtemp: only update up_thru if newer 10001/head
Samuel Just [Wed, 8 Jun 2016 00:15:05 +0000 (17:15 -0700)]
OSDMonitor::prepare_pgtemp: only update up_thru if newer

Fixes: http://tracker.ceph.com/issues/16185
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 5f2bd7b6b28aad96d68444b22c04b8b24564616b)

9 years agoOSDMonitor: drop pg temp from sources other than the current primary 9998/head
Samuel Just [Thu, 2 Jun 2016 17:43:17 +0000 (10:43 -0700)]
OSDMonitor: drop pg temp from sources other than the current primary

Fixes: http://tracker.ceph.com/issues/16127
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1a07123c38e3fecb3fb2e43bbbae962d8411d287)

9 years agoosd: reject PARALLELEXEC ops with EINVAL 9997/head
Sage Weil [Tue, 24 May 2016 18:02:32 +0000 (14:02 -0400)]
osd: reject PARALLELEXEC ops with EINVAL

Errors are better than crashing.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 407495197dd878b62f1356f6d939e33ab50d78c6)

9 years agoceph_test_rados_api_misc: test rados op with bad flas
Sage Weil [Tue, 24 May 2016 18:02:14 +0000 (14:02 -0400)]
ceph_test_rados_api_misc: test rados op with bad flas

Pass the bad PARALLELEXEC flag to remove(), which takes a flags arg.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1aa807f4f29b4b016cc737ffa443e8f30c3b7693)

9 years agomsg/async: close STATE_WAIT connection in short period 9996/head
Haomai Wang [Sun, 19 Jun 2016 15:42:36 +0000 (23:42 +0800)]
msg/async: close STATE_WAIT connection in short period

1. in practice, STATE_WAIT connection caused by racing connect should be
resolved in milliseconds level. we don't need to keep this connection
forever.
2. it will avoid unexpected osd peering hang because of outside network
problem.

Fixes: http://tracker.ceph.com/issues/16378
Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 4de5407ac96686748497253e4daf51177f809a95)