]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agoceph_volume_client: modify locking of meta files
Ramana Raja [Thu, 23 Jun 2016 10:36:53 +0000 (16:06 +0530)]
ceph_volume_client: modify locking of meta files

File locks are applied on meta files before updating the meta
file contents. These meta files would need to be cleaned up
sometime, which could lead to locks being held on unlinked meta
files. Prevent this by checking whether the file had been deleted
after lock was acquired on it.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit f7c037229bcf3f5a3d06897ec7fe0c5419dd7143)

8 years agocephfs.pyx: implement python bindings for fstat
Ramana Raja [Thu, 23 Jun 2016 12:09:32 +0000 (17:39 +0530)]
cephfs.pyx: implement python bindings for fstat

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit f58403f3d19e22edeb8f91b6f87a0b7947b0ff21)

8 years agoceph_volume_client: restrict volume group names
Ramana Raja [Wed, 8 Jun 2016 11:27:01 +0000 (16:57 +0530)]
ceph_volume_client: restrict volume group names

Prevent craftily-named volume groups from colliding with meta files.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 7f7d2a76ae9b556c1de418f0eab8461c538f91d9)

8 years agoceph_volume_client: use fsync instead of syncfs
Ramana Raja [Wed, 8 Jun 2016 11:04:56 +0000 (16:34 +0530)]
ceph_volume_client: use fsync instead of syncfs

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 27eb51baab4cda6b385aef53fc7c3962a9debae5)

8 years agopybind: expose fsync in cephfs binding.
Xiaoxi Chen [Fri, 29 Apr 2016 19:47:42 +0000 (14:47 -0500)]
pybind: expose fsync in cephfs binding.

So we don't necessary to syncfs when want to persistent
some file.

Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>
(cherry picked from commit 1c952fbaf0fd393ef2dcb83a3db721a077b4274e)

8 years agoceph_volume_client: recover from dirty auth and auth meta updates
Ramana Raja [Tue, 7 Jun 2016 19:12:18 +0000 (00:42 +0530)]
ceph_volume_client: recover from dirty auth and auth meta updates

Check dirty flag after locking something and call recover() if we are
opening something dirty (racing with another instance of the driver
restarting after failure) -- only required if someone running multiple
manila-share instances with Ceph loaded.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 647a2447f0c4354dc21d1083043591d2b6f6f94f)

8 years agoceph_volume_client: modify data layout in meta files
Ramana Raja [Tue, 21 Jun 2016 06:44:56 +0000 (12:14 +0530)]
ceph_volume_client: modify data layout in meta files

Notable changes to data layout in auth meta and volume meta files:

In the auth meta files, add a 'dirty' flag to track the status of auth
updates to a single volume.

In the volume meta file, make the 'dirty' flag track the status of
auth updates for a single ID.

Optimize the recovery of partial auth update changes to auth meta,
volume meta, and the Ceph backend, facilitated by changes in the
data layout in the meta files.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 7c8a28a7e817d030a4d419b0cc627c30c1064270)

8 years agopybind: ceph_volume_client authentication metadata
John Spray [Wed, 2 Mar 2016 12:30:45 +0000 (12:30 +0000)]
pybind: ceph_volume_client authentication metadata

Store a two-way mapping between auth IDs and volumes.

Enables us to record some metadata on auth ids (which
openstack tenant created it) so that we can avoid exposing
keys to other tenants who try to use the same ceph
auth id.

Enables us to expose the list of which auth ids have access
to a volume, so that Manila's update_access() can be
implemented efficiently.

DNM: see TODOs inline.

Fixes: http://tracker.ceph.com/issues/15615
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit d2e9eb55ca6ed5daa094cf323faf143615b9380b)

8 years agopybind: enable integer flags to libcephfs open
John Spray [Mon, 7 Mar 2016 13:06:41 +0000 (13:06 +0000)]
pybind: enable integer flags to libcephfs open

The 'rw+' style flags are handy and convenient, but
they don't capture all possibilities.  Change to
optionally accept an integer here for advance users
who want to specify arbitrary combinations of
flags.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5678584f4176d07301acd7f57acc4efd7fb20e43)

9 years agoceph_volume_client: allow read-only authorization for volumes
Ramana Raja [Wed, 27 Apr 2016 17:56:44 +0000 (23:26 +0530)]
ceph_volume_client: allow read-only authorization for volumes

Allow clients to be restricted to read-only mount of the volume
by restricting their ceph auth ID's MDS and OSD caps to read-only.

Fixes: http://tracker.ceph.com/issues/15614
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 011ea5e7fb35ee07848e0c3abac24702a778ad63)

9 years agoMerge pull request #10373 from ceph/jewel-mds-snap-failover
John Spray [Thu, 21 Jul 2016 12:58:16 +0000 (13:58 +0100)]
Merge pull request #10373 from ceph/jewel-mds-snap-failover

Jewel mds snap failover

Reviewed-by: John Spray <john.spray@redhat.com>
9 years agoclient: fix MetaRequest::set_other_inode() 10373/head
Yan, Zheng [Mon, 11 Jul 2016 07:32:41 +0000 (15:32 +0800)]
client: fix MetaRequest::set_other_inode()

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 265f96bda7e106883063f074133450ce39ee262c)

9 years agoclient: close directory's snapdir when deleting directory
Yan, Zheng [Mon, 11 Jul 2016 03:25:37 +0000 (11:25 +0800)]
client: close directory's snapdir when deleting directory

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 3099cabd11a16d22c6707631861bef0aa176ed02)

9 years agoclient: invalidate snap inodes after removing snapshot
Yan, Zheng [Mon, 11 Jul 2016 03:07:01 +0000 (11:07 +0800)]
client: invalidate snap inodes after removing snapshot

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit f180ad149ab510626fcd6cbd8221f550f9858126)

9 years agomds: fix incorrect "unconnected snaprealm xxx" warning
Yan, Zheng [Mon, 11 Jul 2016 02:51:13 +0000 (10:51 +0800)]
mds: fix incorrect "unconnected snaprealm xxx" warning

If a snaprealm has no child/parent snaprelam, and the snaprealm inode
is not in the cache while client reconnects. The snaprealm does not
get properly removed from MDCache::reconnected_snaplrealm. This causes
incorrect "unconnected snaprealm xxx" warning

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 85e687de876ca7e0174734ba81130949c4ab6a40)

9 years agoqa/workunits/fs: fix expect_failure function in test scripts
Yan, Zheng [Thu, 30 Jun 2016 08:05:57 +0000 (16:05 +0800)]
qa/workunits/fs: fix expect_failure function in test scripts

The origin expect_failure function return 0 regardness of command's
return value.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 16f7d7c71e65cd68cecde3e5b89d189ab5950548)

9 years agoclient: make sure snapflush is sent before normal cap message
Yan, Zheng [Wed, 29 Jun 2016 12:49:40 +0000 (20:49 +0800)]
client: make sure snapflush is sent before normal cap message

MDS does null snapflush when it receives normal cap message. So client
must send snapflush first.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit d3916717e2edc8000400f678fa7134ca1406a074)

9 years agoclient: unify cap flush and snapcap flush
Yan, Zheng [Tue, 28 Jun 2016 12:39:08 +0000 (20:39 +0800)]
client: unify cap flush and snapcap flush

This patch includes following changes
- assign flush tid to snapcap flush
- remove session's flushing_capsnaps list. add inode with snapcap
  flushes to session's flushing_caps list instead.
- when reconnecting to MDS, re-send one inode's snapcap flushes and
  cap flushes at the same time.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit a05e996b2a2c36496abd8538829ac4897392f6eb)

9 years agomds: handle partly purged directory
Yan, Zheng [Wed, 29 Jun 2016 09:15:01 +0000 (17:15 +0800)]
mds: handle partly purged directory

For a snapshoted direcotry whose snaprealm parents are being opened,
MDS does not know if the directory is purgeable. So MDS can't skip
committing dirfrags of the directory. But if the direcotry is purgeale,
some dirfrags could have already been deleted during MDS failover.
Committing them could return -ENOENT.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit bc50e0309280c08c3ca79dfa5514ac3a15f81a23)

9 years agomds: do files recovery after processing cap flushes
Yan, Zheng [Wed, 29 Jun 2016 03:42:42 +0000 (11:42 +0800)]
mds: do files recovery after processing cap flushes

File recovery may update inode and trigger inode COW. MDS relies on
client caps to setup CInode::client_need_snapflush. But for a given
client, the reconnected caps may not include the flushing caps.
(Before MDS failover, client released and flushed some caps at the
same time. When MDS recovers, client re-send the cap flush and send
cap reconnect to the MDS.) This may cause later snapflush to get
dropped.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit dd98448d3d0bbb7d338f03f7ad1e40f217eebe0d)

9 years agomds: combine MDCache::{reconnected_caps,cap_imports_dirty}
Yan, Zheng [Wed, 29 Jun 2016 03:25:12 +0000 (11:25 +0800)]
mds: combine MDCache::{reconnected_caps,cap_imports_dirty}

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 57067e032e84bbdb69c499aa7ea93ca68082569b)

9 years agomds: remove CEPH_LOCK_IFLOCKL from cinode_lock_info
Yan, Zheng [Fri, 24 Jun 2016 09:09:34 +0000 (17:09 +0800)]
mds: remove CEPH_LOCK_IFLOCKL from cinode_lock_info

Currently we don't support dirty CEPH_CAP_FLOCK_EXCL

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit cfc3ec17b6f245e6d8a0be4fdf6cfa64d2fb725f)

9 years agomds: rebuild the internal states that tracking pending snapflush
Yan, Zheng [Thu, 23 Jun 2016 08:59:46 +0000 (16:59 +0800)]
mds: rebuild the internal states that tracking pending snapflush

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 1b7d198f633a8608b704f82f01a3f4a4a3a4892b)

9 years agomds: using cap_reconnect_t to track cap recconect
Yan, Zheng [Wed, 22 Jun 2016 12:34:41 +0000 (20:34 +0800)]
mds: using cap_reconnect_t to track cap recconect

Previous commit extended cap_reconnect_t to include snapflush
related information. This information are needed in various
places

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 25b6011dcf07c1fc663cbe29ffd119b66545a0ac)

9 years agomds: add 'follows' of first pending snapflush to cap reconnect message
Yan, Zheng [Wed, 22 Jun 2016 07:56:29 +0000 (15:56 +0800)]
mds: add 'follows' of first pending snapflush to cap reconnect message

This helps the recovering MDS to reconstruct the internal states that
tracking pending snapflush.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit dd3963a878d4bb2fb5992278ccbc9f7633fb8786)

9 years agomds: journal snap inodes that need flush when expiring log segment
Yan, Zheng [Tue, 21 Jun 2016 09:17:56 +0000 (17:17 +0800)]
mds: journal snap inodes that need flush when expiring log segment

Treat snap inodes that need flush in the same way as open files.
When MDS recovers, this make sure that journal replay bring snap
inodes that need flush into the cache

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 306153b3d012832bdfa20402077fa60a9a5d626c)

9 years agomds: properly update client_snap_caps when splitting snap inode
Yan, Zheng [Tue, 21 Jun 2016 08:20:58 +0000 (16:20 +0800)]
mds: properly update client_snap_caps when splitting snap inode

update the new snap inode's client_snap_caps according to the old
snap inode.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 4883779935219817c7e391940a025be1679daeb5)

9 years agoinstall-deps: Get the latest virtualenv in a temporary folder
Oleh Prypin [Wed, 13 Jul 2016 22:06:51 +0000 (01:06 +0300)]
install-deps: Get the latest virtualenv in a temporary folder
to work around a bug in old virtualenv

Signed-off-by: Oleh Prypin <oleh@pryp.in>
(cherry picked from commit 2699b6d89b8c1c193fd86b5233d1ea86458753a0)

9 years agoMerge remote-tracking branch 'upstream/pull/10298/head' into jewel
Patrick Donnelly [Thu, 14 Jul 2016 16:10:11 +0000 (12:10 -0400)]
Merge remote-tracking branch 'upstream/pull/10298/head' into jewel

* upstream/pull/10298/head:
  doc: fix standby replay config

9 years agodoc: fix standby replay config 10298/head
Patrick Donnelly [Tue, 12 Jul 2016 19:43:23 +0000 (15:43 -0400)]
doc: fix standby replay config

I tried using these settings in tests without success. The correct config names
are prefixed with "mds".

Fixes: http://tracker.ceph.com/issues/16664
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
9 years agoMerge pull request #9268 from theanalyst/wip-15992-jewel
Casey Bodley [Tue, 12 Jul 2016 14:09:34 +0000 (10:09 -0400)]
Merge pull request #9268 from theanalyst/wip-15992-jewel

jewel: radosgw-admin: EEXIST messages for create operations

Reviewed-by: Casey Bodley <cbodley@redhat.com>
9 years agoMerge pull request #8497 from sabaini/jewel
Loic Dachary [Sun, 10 Jul 2016 21:03:16 +0000 (23:03 +0200)]
Merge pull request #8497 from sabaini/jewel

ceph-disk: Accept bcache devices as data disks

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9545 from Abhishekvrshny/wip-16117-jewel
Matt Benjamin [Wed, 6 Jul 2016 18:18:44 +0000 (14:18 -0400)]
Merge pull request #9545 from Abhishekvrshny/wip-16117-jewel

jewel: rgw: aws4 parsing issue

9 years agoMerge pull request #9568 from dreamhost/wip-16182
Loic Dachary [Wed, 6 Jul 2016 09:55:13 +0000 (11:55 +0200)]
Merge pull request #9568 from dreamhost/wip-16182

jewel: rgw: backport merge of static sites fixes

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
9 years agoMerge pull request #9099 from Abhishekvrshny/wip-15841-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:36:20 +0000 (10:36 +0200)]
Merge pull request #9099 from Abhishekvrshny/wip-15841-jewel

jewel: s3website: x-amz-website-redirect-location header returns malformed HTTP response
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9265 from Abhishekvrshny/wip-15965-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:35:23 +0000 (10:35 +0200)]
Merge pull request #9265 from Abhishekvrshny/wip-15965-jewel

jewel: No Last-Modified, Content-Size and X-Object-Manifest headers if no segments in DLO manifest
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9267 from Abhishekvrshny/wip-15960-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:34:42 +0000 (10:34 +0200)]
Merge pull request #9267 from Abhishekvrshny/wip-15960-jewel

jewel: rgw: custom metadata aren't camelcased in Swift's responses
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9316 from Abhishekvrshny/wip-15954-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:28:45 +0000 (10:28 +0200)]
Merge pull request #9316 from Abhishekvrshny/wip-15954-jewel

jewel: rgw: initial slashes are not properly handled in Swift's BulkDelete
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9390 from vumrao/wip-vumrao-16071
Orit Wasserman [Thu, 30 Jun 2016 08:26:09 +0000 (10:26 +0200)]
Merge pull request #9390 from vumrao/wip-vumrao-16071

jewel: rgw : cleanup radosgw-admin temp command as it was deprecated
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9542 from Abhishekvrshny/wip-15967-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:25:51 +0000 (10:25 +0200)]
Merge pull request #9542 from Abhishekvrshny/wip-15967-jewel

jewel: rgw: account/container metadata not actually present in a request are deleted during POST through Swift API
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9543 from Abhishekvrshny/wip-16040-jewel
Orit Wasserman [Thu, 30 Jun 2016 08:25:30 +0000 (10:25 +0200)]
Merge pull request #9543 from Abhishekvrshny/wip-16040-jewel

jewel: rgw: updating CORS/ACLs might not work in some circumstances
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
9 years agoMerge pull request #9631 from dillaman/wip-16232
Loic Dachary [Thu, 30 Jun 2016 07:41:20 +0000 (09:41 +0200)]
Merge pull request #9631 from dillaman/wip-16232

jewel: test: improve rbd-mirror test case coverage

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9294 from theanalyst/wip-16009-jewel
Loic Dachary [Wed, 29 Jun 2016 14:47:15 +0000 (16:47 +0200)]
Merge pull request #9294 from theanalyst/wip-16009-jewel

jewel: radosgw-admin: failure for user create after upgrade from hammer to jewel

Reviewed-by: Casey Bodley <cbodley@redhat.com>
9 years agoMerge pull request #9721 from trociny/wip-16289-jewel
Mykola Golub [Wed, 29 Jun 2016 08:42:43 +0000 (11:42 +0300)]
Merge pull request #9721 from trociny/wip-16289-jewel

jewel: qa/workunits/rbd: respect RBD_CREATE_ARGS environment variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9425 from cbodley/wip-rgw-sync-retry-jewel
Loic Dachary [Wed, 29 Jun 2016 07:56:58 +0000 (09:56 +0200)]
Merge pull request #9425 from cbodley/wip-rgw-sync-retry-jewel

jewel: rgw: retry on bucket sync errors

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9327 from cbodley/wip-rgw-zone-modify-master-jewel
Loic Dachary [Wed, 29 Jun 2016 07:55:26 +0000 (09:55 +0200)]
Merge pull request #9327 from cbodley/wip-rgw-zone-modify-master-jewel

jewel: rgw: add_zone only clears master_zone if --master=false

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9242 from yehudasa/wip-15911-jewel
Loic Dachary [Wed, 29 Jun 2016 07:50:04 +0000 (09:50 +0200)]
Merge pull request #9242 from yehudasa/wip-15911-jewel

jewel: rgw: don't unregister request if request is not connected to manager

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9883 from dillaman/wip-16422
Jason Dillaman [Tue, 28 Jun 2016 19:53:03 +0000 (15:53 -0400)]
Merge pull request #9883 from dillaman/wip-16422

jewel: librbd: flag image as updated after proxying maintenance op

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9105 from Abhishekvrshny/wip-15768-jewel
Samuel Just [Mon, 27 Jun 2016 19:43:11 +0000 (12:43 -0700)]
Merge pull request #9105 from Abhishekvrshny/wip-15768-jewel

jewel: FileStore: umount hang because sync thread doesn't exit

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #9633 from liewegas/wip-async-jewel
Loic Dachary [Mon, 27 Jun 2016 15:50:39 +0000 (17:50 +0200)]
Merge pull request #9633 from liewegas/wip-async-jewel

msg/async: backports of all asyncmsgr fixes to jewel

Reviewed-by: Haomai Wang <haomai@xsky.com>
9 years agoMerge pull request #9576 from Abhishekvrshny/wip-16148-jewel
Sage Weil [Mon, 27 Jun 2016 13:12:14 +0000 (09:12 -0400)]
Merge pull request #9576 from Abhishekvrshny/wip-16148-jewel

jewel: Scrub error: 0/1 pinned

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #9575 from Abhishekvrshny/wip-16150-jewel
Sage Weil [Mon, 27 Jun 2016 13:11:43 +0000 (09:11 -0400)]
Merge pull request #9575 from Abhishekvrshny/wip-16150-jewel

jewel: crash adding snap to purged_snaps in ReplicatedPG::WaitingOnReplicas

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #8754 from kmroz/wip-backport-fix-run-dir-chown
Sage Weil [Mon, 27 Jun 2016 13:10:36 +0000 (09:10 -0400)]
Merge pull request #8754 from kmroz/wip-backport-fix-run-dir-chown

jewel: global-init: fixup chown of the run directory along with log and asok files

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #9578 from Abhishekvrshny/wip-16080-jewel
Loic Dachary [Mon, 27 Jun 2016 08:12:35 +0000 (10:12 +0200)]
Merge pull request #9578 from Abhishekvrshny/wip-16080-jewel

jewel: osd:sched_time not actually randomized

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9574 from Abhishekvrshny/wip-16153-jewel
Loic Dachary [Mon, 27 Jun 2016 08:08:25 +0000 (10:08 +0200)]
Merge pull request #9574 from Abhishekvrshny/wip-16153-jewel

jewel: Missing export for rados_aio_get_version in src/include/rados/librados.h

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9565 from dzafman/wip-15767-jewel
Loic Dachary [Mon, 27 Jun 2016 08:06:54 +0000 (10:06 +0200)]
Merge pull request #9565 from dzafman/wip-15767-jewel

jewel: Fixes for list-inconsistent-*

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #8904 from SUSE/wip-15700-jewel
Loic Dachary [Mon, 27 Jun 2016 07:42:28 +0000 (09:42 +0200)]
Merge pull request #8904 from SUSE/wip-15700-jewel

jewel: rados/test.sh workunit timesout on OpenStack

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agolibrbd: flag image as updated after proxying maintenance op 9883/head
Jason Dillaman [Tue, 21 Jun 2016 18:56:43 +0000 (14:56 -0400)]
librbd: flag image as updated after proxying maintenance op

Fixes: http://tracker.ceph.com/issues/16404
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 478dd5f173591914b41d87b8c6d035c32cac8d3e)

9 years agoMerge branch 'wip-dmick-install-deps-jewel' into jewel
Josh Durgin [Tue, 21 Jun 2016 23:43:20 +0000 (16:43 -0700)]
Merge branch 'wip-dmick-install-deps-jewel' into jewel

install-deps.sh: use mk-build-deps instead of processing control

Reviewed-by: Erwan Velu <erwan@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
9 years agoinstall-deps.sh: use mk-build-deps instead of processing control
Dan Mick [Thu, 24 Mar 2016 00:43:53 +0000 (17:43 -0700)]
install-deps.sh: use mk-build-deps instead of processing control

mk-build-deps is designed to handle installing build dependencies;
use that instead, so '|' indications are handled properly.

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 99bca09b8d09c41077a9d2141ff556b74328e57a)

9 years agoxio: add MNop.h to dist tarball
Roi Dayan [Wed, 20 Apr 2016 14:56:16 +0000 (17:56 +0300)]
xio: add MNop.h to dist tarball

This file is needed when we want to build a package with xio messenger
but was not added to dist tarball.

Signed-off-by: Roi Dayan <roid@mellanox.com>
(cherry picked from commit ac50842d81d271569d4149737834ac53b2f51db2)

9 years agoqa/workunits/rbd: respect RBD_CREATE_ARGS environment variable 9721/head
Mykola Golub [Tue, 14 Jun 2016 12:28:06 +0000 (15:28 +0300)]
qa/workunits/rbd: respect RBD_CREATE_ARGS environment variable

Fixes: http://tracker.ceph.com/issues/16289
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 3048d3c7ce800a6174c54946b838e4e4079ec609)

9 years agorados: Improve list-inconsistent json format 9565/head
David Zafman [Fri, 6 May 2016 06:20:58 +0000 (23:20 -0700)]
rados: Improve list-inconsistent json format

Use array "errors" of strings for the error bits
Change snapshot number to integer instead of hex

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 07dc844d6804db93055573ec83e6753773a4c049)

9 years agotest: Fix test to not use jq -S which isn't avail in all distributions
David Zafman [Sat, 7 May 2016 02:54:11 +0000 (19:54 -0700)]
test: Fix test to not use jq -S which isn't avail in all distributions

Fixes: http://tracker.ceph.com/issues/15766
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a2147ad3cb256c34541ff7dc285594a22a24dfc6)

9 years agotest: Add testing of new scrub commands in rados
David Zafman [Tue, 3 May 2016 19:10:38 +0000 (12:10 -0700)]
test: Add testing of new scrub commands in rados

Test list-inconsistent-pg
Test list-inconsistent-obj
Test inconsistent-snapset

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit db517ba9176f1ee829453b016b1bd43c6054a555)

9 years agorados: Don't bother showing list-inconsistent-* errors that aren't set
David Zafman [Fri, 29 Apr 2016 18:25:18 +0000 (11:25 -0700)]
rados: Don't bother showing list-inconsistent-* errors that aren't set

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit a551b6bd5c4eba11c170afe63994258e7598c3d2)

9 years agoosd, rados: Fixes for list-inconsistent-snapset
David Zafman [Sat, 30 Apr 2016 00:09:13 +0000 (17:09 -0700)]
osd, rados: Fixes for list-inconsistent-snapset

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fdca113fc24adbb4f60bfdc55fbbd5044b711b5d)

9 years agoinclude, rados: Fixes for list-inconsistent-obj and librados
David Zafman [Fri, 29 Apr 2016 18:26:39 +0000 (11:26 -0700)]
include, rados: Fixes for list-inconsistent-obj and librados

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 4fc4326b36aa00092043514746aa39aedad06b0f)

9 years agorados: Balance format sections in same do_get_inconsistent_cmd()
David Zafman [Wed, 27 Apr 2016 01:16:22 +0000 (18:16 -0700)]
rados: Balance format sections in same do_get_inconsistent_cmd()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit de4681b80a5d24e72bbc7ac6dfee4775987fd834)

9 years agorados: Include epoch in the list-inconsistent-* command output
David Zafman [Tue, 22 Mar 2016 03:41:03 +0000 (20:41 -0700)]
rados: Include epoch in the list-inconsistent-* command output

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit df4bc16c4d49489f7398eb1ecc7b9aef93258414)

9 years agorados: Improve error messages for list-inconsistent commands
David Zafman [Tue, 22 Mar 2016 03:39:53 +0000 (20:39 -0700)]
rados: Improve error messages for list-inconsistent commands

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit cea7cf56ee3b9593c935d0c74ce6e4b28b14c2da)

9 years ago10.2.1
Jenkins Build Slave User [Fri, 13 May 2016 17:50:34 +0000 (17:50 +0000)]
10.2.1

9 years agoqa/workunits/rbd: specify source path 9631/head
Mykola Golub [Sat, 11 Jun 2016 17:01:31 +0000 (20:01 +0300)]
qa/workunits/rbd: specify source path

otherwise it looks in $PATH for the script

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 698242e7435c9c4d1db7cb6d5aab3faf57eda840)

9 years agoqa/workunits/rbd: additional rbd-mirror stress tests
Jason Dillaman [Thu, 9 Jun 2016 20:23:52 +0000 (16:23 -0400)]
qa/workunits/rbd: additional rbd-mirror stress tests

Fixes: http://tracker.ceph.com/issues/16197
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit db3e583a5fe4a7985b1e7f1740114da414835af5)

9 years agovstart: add --nolockdep option
Jason Dillaman [Thu, 9 Jun 2016 21:14:36 +0000 (17:14 -0400)]
vstart: add --nolockdep option

rbd-mirror stress test cases need a way to disable lockdep when
initiated in a vstart environment.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d5eec7b2f4222dde0cfdaa9e0e4b43be015c3692)

9 years ago10.2.2 v10.2.2
Jenkins Build Slave User [Tue, 14 Jun 2016 11:43:24 +0000 (11:43 +0000)]
10.2.2

9 years agoMerge pull request #9692 from dillaman/wip-16292
Jason Dillaman [Tue, 14 Jun 2016 11:32:44 +0000 (07:32 -0400)]
Merge pull request #9692 from dillaman/wip-16292

jewel: librbd: remove should ignore mirror errors from older OSDs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agolibrbd: remove should ignore mirror errors from older OSDs 9692/head
Jason Dillaman [Mon, 13 Jun 2016 19:29:43 +0000 (15:29 -0400)]
librbd: remove should ignore mirror errors from older OSDs

Fixes: http://tracker.ceph.com/issues/16268
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5a97a9b7ef238c4ceb16225a0b937f9a0cdbb2b6)

9 years agoMerge pull request #9691 from dillaman/wip-16260-jewel
Jason Dillaman [Tue, 14 Jun 2016 11:24:28 +0000 (07:24 -0400)]
Merge pull request #9691 from dillaman/wip-16260-jewel

jewel: librbd: do not shut down exclusive lock while acquiring'

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9690 from dillaman/wip-16290
Jason Dillaman [Tue, 14 Jun 2016 10:57:38 +0000 (06:57 -0400)]
Merge pull request #9690 from dillaman/wip-16290

jewel: rbd-mirror: FAILED assert(!m_status_watcher)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agolibrbd: track in-flight flush events for journaling 9691/head
Jason Dillaman [Tue, 14 Jun 2016 00:20:33 +0000 (20:20 -0400)]
librbd: track in-flight flush events for journaling

The tracking for flush events was accidentally dropped. This
can result in the journal being closed too early since it
might miss in-flight flush ops.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a1ea1edce9332fa1730625629e3b4e16f02caaf4)

9 years agolibrbd: do not shut down exclusive lock while acquiring
Jason Dillaman [Mon, 13 Jun 2016 16:00:28 +0000 (12:00 -0400)]
librbd: do not shut down exclusive lock while acquiring

Fixes: http://tracker.ceph.com/issues/16260
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c5694fc6766fb8e213c4b65d2cd7b9d066b07ff7)

9 years agorbd-mirror: FAILED assert(!m_status_watcher) 9690/head
Mykola Golub [Sun, 12 Jun 2016 17:32:45 +0000 (20:32 +0300)]
rbd-mirror: FAILED assert(!m_status_watcher)

Fixes: http://tracker.ceph.com/issues/16245
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit b8bfc0f2efb44d3eff7a5fd8bca24516c8815d62)

9 years agoMerge pull request #9654 from dillaman/wip-16246
Jason Dillaman [Tue, 14 Jun 2016 10:54:51 +0000 (06:54 -0400)]
Merge pull request #9654 from dillaman/wip-16246

jewel: librbd: recursive lock possible when disabling journaling

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9673 from dillaman/wip-16265
Jason Dillaman [Tue, 14 Jun 2016 10:54:31 +0000 (06:54 -0400)]
Merge pull request #9673 from dillaman/wip-16265

jewel: rbd-mirror: do not re-use image id from mirror directory if creating image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9674 from dillaman/wip-16267
Jason Dillaman [Tue, 14 Jun 2016 03:37:06 +0000 (23:37 -0400)]
Merge pull request #9674 from dillaman/wip-16267

jewel: qa/workunits: ensure replay has started before checking position

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoqa/workunits: ensure replay has started before checking position 9674/head
Jason Dillaman [Mon, 13 Jun 2016 04:19:37 +0000 (00:19 -0400)]
qa/workunits: ensure replay has started before checking position

Fixes: http://tracker.ceph.com/issues/16248
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6a68b68e0d2741e6694fd28d31766808ba881815)

9 years agorbd-mirror: reset local image id when creating new image 9673/head
Jason Dillaman [Mon, 13 Jun 2016 06:09:00 +0000 (02:09 -0400)]
rbd-mirror: reset local image id when creating new image

Fixes: http://tracker.ceph.com/issues/16253
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1948047acbc129de2982d53e130da1f577af224b)

9 years agorbd-mirror: handle an attempt to delete a non-mirrored image
Jason Dillaman [Mon, 13 Jun 2016 06:07:54 +0000 (02:07 -0400)]
rbd-mirror: handle an attempt to delete a non-mirrored image

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 45dda67b3ef3d5dc5abb327fc1f242140f004d90)

9 years agocls_rbd: disallow setting an empty mirror image entry to disabling
Jason Dillaman [Mon, 13 Jun 2016 06:03:29 +0000 (02:03 -0400)]
cls_rbd: disallow setting an empty mirror image entry to disabling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7b2f99a97ff64451f3ae72081186dff41c507271)

9 years agolibrbd: avoid holding owner_lock waiting for exclusive lock 9654/head
Jason Dillaman [Mon, 13 Jun 2016 01:42:59 +0000 (21:42 -0400)]
librbd: avoid holding owner_lock waiting for exclusive lock

Fixes: http://tracker.ceph.com/issues/16235
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b96ab805eed49d4bdb494c8919d2e1ee7ad0a7e6)

9 years agoMerge pull request #9656 from gregsfortytwo/wip-jewel-16024
Gregory Farnum [Mon, 13 Jun 2016 08:26:53 +0000 (01:26 -0700)]
Merge pull request #9656 from gregsfortytwo/wip-jewel-16024

Jewel client: fix bugs accidentally disabling readahead

9 years agoMerge pull request #9655 from gregsfortytwo/wip-jewel-15508
Gregory Farnum [Mon, 13 Jun 2016 08:22:22 +0000 (01:22 -0700)]
Merge pull request #9655 from gregsfortytwo/wip-jewel-15508

Jewel mds: order directories by hash and fix simultaneous readdir races

9 years agoclient: fix simultaneous readdirs race 9655/head
Yan, Zheng [Mon, 9 May 2016 14:12:48 +0000 (22:12 +0800)]
client: fix simultaneous readdirs race

Current readdir code uses list to track the order of the dentries
in readdir replies.  When handling a readdir reply, it pushes the
resulting dentries to the back of directory's dentry_list. After
readdir finishes, the dentry_list reflects how MDS sorts dentries.

This method is racy when there are simultaneous readdirs. The fix
is use vector instead of list to trace how dentries are sorted in
its parent directory. As long as shared_gen doesn't change, each
dentry is at fixed position of the vector. So cocurrent readdirs
do not affect each other.

Fixes: http://tracker.ceph.com/issues/15508
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 9d297c5e98f814b282dadc379ab70dfa678db73e)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: move dir_{release,ordered}_count into class Inode
Yan, Zheng [Mon, 25 Apr 2016 15:24:46 +0000 (23:24 +0800)]
client: move dir_{release,ordered}_count into class Inode

We close Inode::dir when it's empty. Once closing the dir, we lose
track of {release,ordered}_count. This causes direcotry to be wrongly
marked as complete. (dir is trimmed to empty in the middle of readdir)

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 235fcf68f5d96a4d9d6cc260d12da912fa7ea4a8)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoceph_test_libcephfs: check order of entries in readdir result
Yan, Zheng [Mon, 25 Apr 2016 09:59:35 +0000 (17:59 +0800)]
ceph_test_libcephfs: check order of entries in readdir result

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 9b17d145c15a469caa29853312b8dcb0b19f55f0)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agomds: don't reset readdir offset if client supports hash order dentry
Yan, Zheng [Mon, 25 Apr 2016 07:52:32 +0000 (15:52 +0800)]
mds: don't reset readdir offset if client supports hash order dentry

Now the ordering of dentries is stable across directory fragmentation.
There is no need to reset readdir offset if directory get fragmented
in the middle of readdir.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 98a01af2cd8e01f14ab7be32c31a43571ef8bb87)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: using hash value to compose dentry offset
Yan, Zheng [Mon, 25 Apr 2016 07:31:27 +0000 (15:31 +0800)]
client: using hash value to compose dentry offset

If MDS sorts dentries in dirfrag in hash order, we use hash value to
compose dentry offset. dentry offset is:

  (0xff << 52) | ((24 bits hash) << 28) |
  (the nth entry hash hash collision)

This offset is stable across directory fragmentation.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 680766ec131b95271e320f54dfe6d69ea8d4fbb3)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: record 'offset' for each entry of dir_result_t::buffer
Yan, Zheng [Mon, 25 Apr 2016 02:50:17 +0000 (10:50 +0800)]
client: record 'offset' for each entry of dir_result_t::buffer

This is preparation for using hash value as dentry 'offset'

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit bd6546e5e8bfb4ef8927807492d63d5285534618)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: fix cached readdir after seekdir
Yan, Zheng [Mon, 9 May 2016 03:51:36 +0000 (11:51 +0800)]
client: fix cached readdir after seekdir

Client::seekdir doesn't reset dirp->at_cache_name for a forward seek
within same frag. So the dentry with name == at_cache_name may not be
the one prior to the readdir postion.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 0e32115bae2f1ac2c59b57852976b0de5587abac)

Signed-off-by: Greg Farnum <gfarnum@redhat.com