]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agorgw: swift: disable revocation thread if sleep == 0 || cache_size == 0 14789/head
Marcus Watts [Thu, 13 Apr 2017 09:33:55 +0000 (05:33 -0400)]
rgw: swift: disable revocation thread if sleep == 0 || cache_size == 0

Keystone tokens can be revoked.  This causes them to fail
validation.  However, in ceph, we cache them.  As long as
they're in the cache we trust them.  To find revoked tokens
there's a call OSI-PKI/revoked but that's only useful for
pki tokens.  Installations using fernet/uuid may not even
have the proper credentials to support the call, in which
case the call blows up in various ways filling up logs
with complaints.

This code makes the revocation thread optional; by disabling it,
the complaints go away.  A further fix is in the works
to use other more modern calls available in modern keystone
installations to properly deal with non-PKI/PKIZ tokens.

(NB: jewel has this logic in src/rgw/rgw_swift.cc not in src/rgw/rgw_keystone.h)

To disable the revocation thread, use at least one of these:
        rgw_keystone_token_cache_size = 0
using this will cause tokens to be validated on every call.
You may instead want to set
        rgw_keystone_revocation_interval = 0
using just this will disable the revocation thread,
but leaves the cache in use.  That avoids the extra
validation overhead, but means token revocation won't
work very well.

Fixes: http://tracker.ceph.com/issues/9493
Fixes: http://tracker.ceph.com/issues/19499
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit 003291a8cbca455c0e8731f66759395a0bb1f555)

8 years agoMerge pull request #14686 from smithfarm/wip-19686-jewel
Nathan Cutler [Tue, 25 Apr 2017 19:06:26 +0000 (21:06 +0200)]
Merge pull request #14686 from smithfarm/wip-19686-jewel

jewel: osd: Give requested scrubs a higher priority

Reviewed-by: David Zafman <dzafman@redhat.com>
8 years agoMerge pull request #14605 from asheplyakov/19476-jewel
Nathan Cutler [Tue, 25 Apr 2017 15:38:21 +0000 (17:38 +0200)]
Merge pull request #14605 from asheplyakov/19476-jewel

jewel: rgw: don't return skew time in pre-signed url

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #14660 from smithfarm/wip-19478-jewel
Nathan Cutler [Tue, 25 Apr 2017 15:37:16 +0000 (17:37 +0200)]
Merge pull request #14660 from smithfarm/wip-19478-jewel

jewel: rgw: zonegroupmap set does not work

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #14607 from asheplyakov/19607-jewel
Nathan Cutler [Tue, 25 Apr 2017 15:36:23 +0000 (17:36 +0200)]
Merge pull request #14607 from asheplyakov/19607-jewel

jewel: rgw: multisite: fetch_remote_obj() gets wrong version when copying from remote

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #14587 from asheplyakov/19617-jewel
Nathan Cutler [Tue, 25 Apr 2017 13:50:26 +0000 (15:50 +0200)]
Merge pull request #14587 from asheplyakov/19617-jewel

jewel: mon/MonClient: make get_mon_log_message() atomic

Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #14665 from smithfarm/wip-19610-jewel
Nathan Cutler [Tue, 25 Apr 2017 13:42:43 +0000 (15:42 +0200)]
Merge pull request #14665 from smithfarm/wip-19610-jewel

jewel: [librados_test_stub] cls_cxx_map_get_XYZ methods don't return correct value

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14653 from smithfarm/wip-19662-jewel
Nathan Cutler [Tue, 25 Apr 2017 07:20:19 +0000 (09:20 +0200)]
Merge pull request #14653 from smithfarm/wip-19662-jewel

jewel: rgw_file: fix event expire check, don't expire directories being read

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
8 years agoMerge pull request #14635 from smithfarm/wip-19690-jewel
Loic Dachary [Tue, 25 Apr 2017 07:17:33 +0000 (09:17 +0200)]
Merge pull request #14635 from smithfarm/wip-19690-jewel

jewel: doc: Improvements to crushtool manpage

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #13608 from smithfarm/wip-19063-jewel
Nathan Cutler [Tue, 25 Apr 2017 05:20:41 +0000 (07:20 +0200)]
Merge pull request #13608 from smithfarm/wip-19063-jewel

jewel: tests: eliminate race condition in Thrasher constructor

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
8 years agoMerge pull request #14195 from cbodley/wip-19353
Nathan Cutler [Mon, 24 Apr 2017 20:27:33 +0000 (22:27 +0200)]
Merge pull request #14195 from cbodley/wip-19353

jewel: rgw: use separate http_manager for read_sync_status

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #14066 from asheplyakov/19321-bp-jewel
Nathan Cutler [Mon, 24 Apr 2017 20:26:50 +0000 (22:26 +0200)]
Merge pull request #14066 from asheplyakov/19321-bp-jewel

jewel: rgw: fix break inside of yield in RGWFetchAllMetaCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #14064 from asheplyakov/19211-bp-jewel
Nathan Cutler [Mon, 24 Apr 2017 20:26:14 +0000 (22:26 +0200)]
Merge pull request #14064 from asheplyakov/19211-bp-jewel

jewel: rgw: "cluster [WRN] bad locator @X on object @X...." in cluster log

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #13842 from smithfarm/wip-19145-jewel
Nathan Cutler [Mon, 24 Apr 2017 20:25:21 +0000 (22:25 +0200)]
Merge pull request #13842 from smithfarm/wip-19145-jewel

jewel: rgw: a few cases where rgw_obj is incorrectly initialized

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #13837 from smithfarm/wip-19048-jewel
Nathan Cutler [Mon, 24 Apr 2017 20:24:29 +0000 (22:24 +0200)]
Merge pull request #13837 from smithfarm/wip-19048-jewel

jewel: rgw: multisite: some yields in RGWMetaSyncShardCR::full_sync() resume in incremental_sync()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #13724 from asheplyakov/18626-bp-jewel
Nathan Cutler [Mon, 24 Apr 2017 20:23:24 +0000 (22:23 +0200)]
Merge pull request #13724 from asheplyakov/18626-bp-jewel

jewel: rgw: Use decoded URI when verifying TempURL

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #14643 from smithfarm/wip-revert-14427
Nathan Cutler [Fri, 21 Apr 2017 20:51:59 +0000 (22:51 +0200)]
Merge pull request #14643 from smithfarm/wip-revert-14427

Wip revert 14427

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14602 from asheplyakov/19646-jewel
Nathan Cutler [Thu, 20 Apr 2017 20:05:57 +0000 (22:05 +0200)]
Merge pull request #14602 from asheplyakov/19646-jewel

jewel: ceph-disk: enable directory backed OSD at boot time

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #14449 from smithfarm/wip-test-doc-oversight
Nathan Cutler [Thu, 20 Apr 2017 20:04:41 +0000 (22:04 +0200)]
Merge pull request #14449 from smithfarm/wip-test-doc-oversight

tests: fix oversight in yaml comment

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #13606 from smithfarm/wip-19062-jewel
Nathan Cutler [Thu, 20 Apr 2017 17:26:52 +0000 (19:26 +0200)]
Merge pull request #13606 from smithfarm/wip-19062-jewel

jewel: build/ops: enable build of ceph-resource-agents package on rpm-based os

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
8 years agoMerge pull request #14654 from smithfarm/wip-19461-jewel
Nathan Cutler [Thu, 20 Apr 2017 16:18:05 +0000 (18:18 +0200)]
Merge pull request #14654 from smithfarm/wip-19461-jewel

jewel: admin ops: fix the quota section

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoosd: Give requested scrub work a higher priority 14686/head
David Zafman [Mon, 17 Apr 2017 21:58:02 +0000 (14:58 -0700)]
osd: Give requested scrub work a higher priority

Once started we now queue scrub work at higher priority than
scheduled scrubs.

Fixes: http://tracker.ceph.com/issues/15789
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ebab8b1f4f67fbdec1e147c580329c1e2b5cf7cd)

Conflicts:
        src/osd/OSD.h - in jewel, the PGScrub() call is enclosed within
            op_wq.queue(make_pair(...)) instead of enqueue_back()

8 years agoMerge pull request #14680 from smithfarm/wip-19711-jewel
Nathan Cutler [Thu, 20 Apr 2017 12:53:30 +0000 (14:53 +0200)]
Merge pull request #14680 from smithfarm/wip-19711-jewel

jewel: [test] test_notify.py: rbd.InvalidArgument: error updating features for image test_notify_clone2

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agotest: rbd master/slave notify test should test active features 14680/head
Jason Dillaman [Wed, 19 Apr 2017 13:26:31 +0000 (09:26 -0400)]
test: rbd master/slave notify test should test active features

Fixes: http://tracker.ceph.com/issues/19692
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0dcba41cba96566d0b8da54cf0316d523b88ded2)

8 years agoMerge pull request #14402 from shinobu-x/wip-17331-jewel
Kefu Chai [Thu, 20 Apr 2017 11:10:33 +0000 (19:10 +0800)]
Merge pull request #14402 from shinobu-x/wip-17331-jewel

 jewel: ceph-disk: ceph-disk list reports mount error for OSD having mount options with SELinux context

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #13865 from smithfarm/wip-19158-jewel
Nathan Cutler [Thu, 20 Apr 2017 11:06:33 +0000 (13:06 +0200)]
Merge pull request #13865 from smithfarm/wip-19158-jewel

jewel: rgw: health check errors out incorrectly

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agotest/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value 14665/head
Jason Dillaman [Wed, 12 Apr 2017 14:47:28 +0000 (10:47 -0400)]
test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value

Fixes: http://tracker.ceph.com/issues/19597
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9ffd464dac102f684d6dfa78e58d2cb45e165ed6)

8 years agoradosgw-admin: use zone id when creating a zone 14660/head
Orit Wasserman [Wed, 5 Apr 2017 10:31:08 +0000 (13:31 +0300)]
radosgw-admin: use zone id when creating a zone

Fixes: http://tracker.ceph.com/issues/19498
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit 3fea36d635fcba8ca584a1c0ec9f07840009402c)

8 years agoqa: rgw task uses period instead of region-map
Casey Bodley [Tue, 14 Mar 2017 19:43:13 +0000 (15:43 -0400)]
qa: rgw task uses period instead of region-map

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e3e3a71d1f1fb43bb4172ce2dfac9a28ca89df0f)

8 years agorgw-admin: remove deprecated regionmap commands
Casey Bodley [Tue, 14 Mar 2017 18:18:15 +0000 (14:18 -0400)]
rgw-admin: remove deprecated regionmap commands

Fixes: http://tracker.ceph.com/issues/18725
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5830c1849a0c0110d17c37784808e456e6dcb7b3)

Conflicts:
src/rgw/rgw_admin.cc (trivial resolution)

8 years agodoc: rgw: correct the quota section 14654/head
hrchu [Wed, 29 Mar 2017 02:17:04 +0000 (02:17 +0000)]
doc: rgw: correct the quota section

Add the missing option and fix typo.

Fixes: http://tracker.ceph.com/issues/19397
Signed-off-by: Chu, Hua-Rong <hrchu@cht.com.tw>
(cherry picked from commit 51a88267f0d7f51aeb62092949b66b9f6c062e15)

8 years agorgw_file: remove unused rgw_key variable 14653/head
Matt Benjamin [Wed, 15 Mar 2017 20:35:16 +0000 (16:35 -0400)]
rgw_file:  remove unused rgw_key variable

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 1100a1c26e76485569cfebcf863b18cf908f6161)

8 years agorgw_file: fix readdir after dirent-change
Matt Benjamin [Fri, 14 Apr 2017 19:56:37 +0000 (15:56 -0400)]
rgw_file: fix readdir after dirent-change

Also, fixes link count computation off-by-one, update of state.nlink
after computation, link computation reset at start, and a time print
in debug log.

Fixes: http://tracker.ceph.com/issues/19634
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
link count

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit e0f80266ecd424bf9466579b3edc03911a7c5719)

8 years agorgw_file: don't expire directories being read
Matt Benjamin [Tue, 11 Apr 2017 10:42:07 +0000 (06:42 -0400)]
rgw_file: don't expire directories being read

If a readdir expire event turns out to be older than last_readdir,
just reschedule it (but actually, we should just discard it, as
another expire event must be in queue.

Fixes: http://tracker.ceph.com/issues/19625
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 007b7451c26716c51207c161dc347e9a00da53f1)

8 years agorgw_file: rgw_readdir: return dot-dirs only when *offset is 0
Matt Benjamin [Wed, 15 Mar 2017 20:40:35 +0000 (16:40 -0400)]
rgw_file:  rgw_readdir:  return dot-dirs only when *offset is 0

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 61482c2b85a07519f2256b1a3f2b6d8aa99d5f06)

8 years agorgw_file: chunked readdir
Matt Benjamin [Tue, 11 Apr 2017 09:56:13 +0000 (05:56 -0400)]
rgw_file:  chunked readdir

Adjust readdir callback path for new nfs-ganesha chunked readdir,
including changes to respect the result of callback to not
continue.

Pending introduction of offset name hint, our caller will just be
completely enumerating, so it is possible to remove the offset map
and just keep a last offset.

Fixes: http://tracker.ceph.com/issues/19624
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit e0191d74e3aef06bf300df045a53a3952a71f651)

8 years agorgw_file: fix missing unlock in unlink
Gui Hecheng [Fri, 31 Mar 2017 02:42:40 +0000 (10:42 +0800)]
rgw_file: fix missing unlock in unlink

Fixes: http://tracker.ceph.com/issues/19435
Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>
(cherry picked from commit cb6808a6366a70f54d0cc16437d16aa1b7819c84)

8 years agorgw_file: implement reliable has-children check (unlink dir)
Matt Benjamin [Tue, 14 Mar 2017 01:52:08 +0000 (21:52 -0400)]
rgw_file: implement reliable has-children check (unlink dir)

Bug report and discussion provided by
Gui Hecheng <guihecheng@cmss.chinamobile.com> in nfs-ganesha upstream
github.  Briefly, while a reliable check is potentially costly,
it is necessary.

Fixes: http://tracker.ceph.com/issues/19270
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit b05f1c6d61aa4501a971e87de6dcaf3e58c3d9b4)

8 years agorgw_file: introduce rgw_lookup type hints
Matt Benjamin [Wed, 5 Apr 2017 00:16:13 +0000 (20:16 -0400)]
rgw_file: introduce rgw_lookup type hints

The new type hints optimize object type deduction, when the
rgw_lookup is called from an rgw_readdir callback.

Fixes: http://tracker.ceph.com/issues/19623
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 2e66c7a7cc763c5c0d6f5db04855f60f2b2ceed3)

8 years agoMerge pull request #14383 from smithfarm/wip-19547-jewel
Nathan Cutler [Thu, 20 Apr 2017 09:08:45 +0000 (11:08 +0200)]
Merge pull request #14383 from smithfarm/wip-19547-jewel

jewel: build/ops: rbdmap.service not included in debian packaging (jewel-only)

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
8 years agoMerge pull request #14143 from smithfarm/wip-19355-jewel
Nathan Cutler [Thu, 20 Apr 2017 09:02:39 +0000 (11:02 +0200)]
Merge pull request #14143 from smithfarm/wip-19355-jewel

jewel: rgw: when converting region_map we need to use rgw_zone_root_pool

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #14136 from smithfarm/wip-19330-jewel
Nathan Cutler [Thu, 20 Apr 2017 09:01:49 +0000 (11:01 +0200)]
Merge pull request #14136 from smithfarm/wip-19330-jewel

jewel: rgw: upgrade to multisite v2 fails if there is a zone without zone info

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #13872 from smithfarm/wip-19163-jewel
Nathan Cutler [Thu, 20 Apr 2017 09:00:35 +0000 (11:00 +0200)]
Merge pull request #13872 from smithfarm/wip-19163-jewel

jewel: doc: radosgw-admin: add the 'object stat' command to usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #13863 from smithfarm/wip-19155-jewel
Nathan Cutler [Thu, 20 Apr 2017 08:59:33 +0000 (10:59 +0200)]
Merge pull request #13863 from smithfarm/wip-19155-jewel

jewel: rgw: typo in rgw_admin.cc

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #13779 from smithfarm/wip-18866-jewel
Nathan Cutler [Thu, 20 Apr 2017 08:56:35 +0000 (10:56 +0200)]
Merge pull request #13779 from smithfarm/wip-18866-jewel

jewel: rgw: 'radosgw-admin sync status' on master zone of non-master zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoRevert "osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag." 14643/head
Nathan Cutler [Wed, 19 Apr 2017 14:53:57 +0000 (16:53 +0200)]
Revert "osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag."

This reverts commit 2d68822c784eb4d62d3b0198ed4ec04404dbffb3.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
8 years agoRevert "osdc/Objecter: resend RWORDERED ops on full"
Nathan Cutler [Wed, 19 Apr 2017 14:53:49 +0000 (16:53 +0200)]
Revert "osdc/Objecter: resend RWORDERED ops on full"

This reverts commit f2474042ecd6560323673170c13f2cb964406e70.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
8 years agodoc: mention --show-mappings in crushtool manpage 14635/head
Nathan Cutler [Tue, 18 Apr 2017 06:06:01 +0000 (08:06 +0200)]
doc: mention --show-mappings in crushtool manpage

Fixes: http://tracker.ceph.com/issues/19649
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit b48b6f4ed8b5f5b5852cbbfd5b3d5b650efb7f1b)

8 years agoMerge pull request #14596 from gregsfortytwo/wip-17916-divergent
Gregory Farnum [Wed, 19 Apr 2017 06:47:27 +0000 (02:47 -0400)]
Merge pull request #14596 from gregsfortytwo/wip-17916-divergent

Wip 17916 divergent

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agorgw: fix for null version_id in fetch_remote_obj() 14607/head
Casey Bodley [Wed, 5 Apr 2017 20:19:57 +0000 (16:19 -0400)]
rgw: fix for null version_id in fetch_remote_obj()

commit 8b43c9781206c22d9aedb4beb8d669bf1e23169f fixed the wrong use of
the dest_obj's version, but removed the check for "null" version

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 915370776df5b964c2ee8d9f9329562919eef8d5)

8 years agorgw: version id doesn't work in fetch_remote_obj
Zhang Shaowen [Fri, 17 Mar 2017 08:26:56 +0000 (16:26 +0800)]
rgw: version id doesn't work in fetch_remote_obj

Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit 8b43c9781206c22d9aedb4beb8d669bf1e23169f)

Conflicts:
src/rgw/rgw_rados.cc: trivial: dest_obj.key.instance in master
  versus dest_obj.get_instance() in Jewel

8 years agorgw: don't return skew time in pre-signed url 14605/head
liuchang0812 [Fri, 10 Feb 2017 10:02:03 +0000 (18:02 +0800)]
rgw: don't return skew time in pre-signed url

Fixes: http://tracker.ceph.com/issues/18828
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
(cherry picked from commit dd8b348f4aad0124e8a4457117bf3f5f76af7bdb)

8 years agoceph-disk: enable directory backed OSD at boot time 14602/head
Loic Dachary [Thu, 13 Apr 2017 21:49:50 +0000 (23:49 +0200)]
ceph-disk: enable directory backed OSD at boot time

https://github.com/ceph/ceph/commit/539385b143feee3905dceaf7a8faaced42f2d3c6
introduced a regression preventing directory backed OSD from starting at
boot time.

For device backed OSD the boot sequence starts with ceph-disk@.service
and proceeds to

    systemctl enable --runtime ceph-osd@.service

where the --runtime ensure ceph-osd@12 is removed when the machine
reboots so that it does not compete with the ceph-disk@/dev/sdb1 unit at
boot time.

However directory backed OSD solely rely on the ceph-osd@.service unit
to start at boot time and will therefore fail to boot.

The --runtime flag is selectively set for device backed OSD only.

Fixes: http://tracker.ceph.com/issues/19628
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit f425a127b7487d2093c8c943f0bcdec3d673d601)

Conflicts:
src/ceph-disk/ceph_disk/main.py: trivial: Jewel does not support
   OpenRC and other inits, hence no corresponding 'elif'

8 years agopglog: require users set a config option before ignoring divergent_priors 14596/head
Greg Farnum [Tue, 18 Apr 2017 01:09:55 +0000 (18:09 -0700)]
pglog: require users set a config option before ignoring divergent_priors

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
8 years agoosd: pglog: clean up divergent_priors off disk when running; don't assert on startup
Greg Farnum [Fri, 7 Apr 2017 21:33:20 +0000 (14:33 -0700)]
osd: pglog: clean up divergent_priors off disk when running; don't assert on startup

Fixes: http://tracker.ceph.com/issues/17916
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #14492 from gregsfortytwo/wip-jewel-snaptrim
Gregory Farnum [Mon, 17 Apr 2017 21:41:19 +0000 (17:41 -0400)]
Merge pull request #14492 from gregsfortytwo/wip-jewel-snaptrim

Backport snap trimming improvements to Jewel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge branch 'jewel' into wip-jewel-snaptrim 14492/head
Gregory Farnum [Mon, 17 Apr 2017 21:39:59 +0000 (17:39 -0400)]
Merge branch 'jewel' into wip-jewel-snaptrim

8 years agoPendingReleaseNotes: discuss snap trim improvements
Greg Farnum [Mon, 17 Apr 2017 21:32:38 +0000 (14:32 -0700)]
PendingReleaseNotes: discuss snap trim improvements

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
8 years agoPrimaryLogPG: reimplement osd_snap_trim_sleep within the state machine
Greg Farnum [Wed, 12 Apr 2017 23:30:55 +0000 (16:30 -0700)]
PrimaryLogPG: reimplement osd_snap_trim_sleep within the state machine

Rather than blocking the main op queue, just pause for that amount of
time between state machine cycles.

Also, add osd_snap_trim_sleep to a few of the thrasher yamls.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 2ed7759cfeb03e71f0fbd98fe7c2db2bb741861c)

Conflicts:
src/osd/PrimaryLogPG.cc

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 67336454a4cee66522bc0ca01b2c58b8960f75ec)

Conflicts:
qa/suites/rados/thrash/thrashers/default.yaml
qa/suites/rados/thrash/thrashers/pggrow.yaml
src/osd/OSD.h
src/osd/ReplicatedPG.cc
src/osd/ReplicatedPG.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
8 years agorados: check that pool is done trimming before removing it
Samuel Just [Thu, 26 Jan 2017 23:41:21 +0000 (15:41 -0800)]
rados: check that pool is done trimming before removing it

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 4aebf59d906fa3e03d21bdac182f89fe3cd4c802)
(cherry picked from commit 34398c29b3c57f00d932cf96570f882dce64a82b)

8 years agoosd/ReplicatedPG: limit the number of concurrently trimming pgs
Greg Farnum [Tue, 11 Apr 2017 21:04:19 +0000 (14:04 -0700)]
osd/ReplicatedPG: limit the number of concurrently trimming pgs

This patch introduces an AsyncReserver for snap trimming to limit the
number of pgs on any single OSD which can be trimming, as with backfill.
Unlike backfill, we don't take remote reservations on the assumption
that the set of pgs with trimming work to do is already well
distributed, so it doesn't seem worth the implementation overhead to get
reservations from the peers as well.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 21cc515adfb225ba70f1d80b1b76f0345c214c22)

Conflicts:
src/osd/PrimaryLogPG.cc
src/osd/PrimaryLogPG.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 68ea24396ca6450d4d8042a7c5f51306b7d199fa)
(cherry picked from commit c7176b869898c870b56b1762958652d801af4c4c)

Conflicts: Many. As evidenced by involving two distinct patches
in this one commit, it wasn't a clean backport.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
8 years agomon/MonClient: make get_mon_log_message() atomic 14587/head
Kefu Chai [Mon, 10 Apr 2017 06:53:46 +0000 (14:53 +0800)]
mon/MonClient: make get_mon_log_message() atomic

* LogClient: move reset_session() into get_mon_log_message() and add a
  "flush" param to the latter. so it can get_mon_log_message()
  atomically. otherwise another call changing the log queue could sneak
  in between reset_session() and get_mon_log_message().
* MonClient: add a "flush" param to do_send() so we can reset the
  LogClient session once we are connected to a monitor.

Fixes: http://tracker.ceph.com/issues/19427
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5215e291da2b527d85e129eda86043490843178e)

Conflicts:
src/mon/MonClient.cc: handle_auth: replaced 'log_client->reset_session();
send_log();' sequence with newly introduced 'send_log(true);' like
the original patch does

8 years agoMerge pull request #13834 from smithfarm/wip-18969-jewel
Nathan Cutler [Fri, 14 Apr 2017 20:28:51 +0000 (22:28 +0200)]
Merge pull request #13834 from smithfarm/wip-18969-jewel

jewel: rgw: Change loglevel to 20 for 'System already converted' message

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #13833 from smithfarm/wip-18908-jewel
Nathan Cutler [Fri, 14 Apr 2017 20:26:51 +0000 (22:26 +0200)]
Merge pull request #13833 from smithfarm/wip-18908-jewel

jewel: rgw: the swift container acl does not support field .ref

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agoMerge pull request #13214 from ovh/bp-osd-updateable-throttles-jewel
Nathan Cutler [Thu, 13 Apr 2017 09:09:27 +0000 (11:09 +0200)]
Merge pull request #13214 from ovh/bp-osd-updateable-throttles-jewel

jewel: osd: allow client throttler to be adjusted on-fly, without restart

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14326 from shinobu-x/wip-15025-jewel
Nathan Cutler [Thu, 13 Apr 2017 08:44:23 +0000 (10:44 +0200)]
Merge pull request #14326 from shinobu-x/wip-15025-jewel

jewel: osd: new added OSD always down when full flag is set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #13874 from smithfarm/wip-19171-jewel
Nathan Cutler [Wed, 12 Apr 2017 09:38:48 +0000 (11:38 +0200)]
Merge pull request #13874 from smithfarm/wip-19171-jewel

jewel: doc: rgw S3 create bucket should not do response in json

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
8 years agoMerge pull request #13492 from shinobu-x/wip-18516-jewel
Nathan Cutler [Wed, 12 Apr 2017 09:06:40 +0000 (11:06 +0200)]
Merge pull request #13492 from shinobu-x/wip-18516-jewel

jewel: build/ops: "osd marked itself down" will not recognised if host runs mon + osd on shutdown/reboot

Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #13254 from shinobu-x/wip-14609-jewel
Nathan Cutler [Wed, 12 Apr 2017 09:01:23 +0000 (11:01 +0200)]
Merge pull request #13254 from shinobu-x/wip-14609-jewel

jewel: common: radosstriper: protect aio_write API from calls with 0 bytes

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #13489 from shinobu-x/wip-18955-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:52:10 +0000 (10:52 +0200)]
Merge pull request #13489 from shinobu-x/wip-18955-jewel

jewel: ceph-disk: bluestore --setgroup incorrectly set with user

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #14070 from smithfarm/wip-19339-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:51:42 +0000 (10:51 +0200)]
Merge pull request #14070 from smithfarm/wip-19339-jewel

jewel: tests: dummy suite fails in OpenStack

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #14329 from smithfarm/wip-19493-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:51:12 +0000 (10:51 +0200)]
Merge pull request #14329 from smithfarm/wip-19493-jewel

jewel: ceph-disk: Racing between partition creation & device node creation

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
8 years agoMerge pull request #14427 from smithfarm/wip-19140-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:48:29 +0000 (10:48 +0200)]
Merge pull request #14427 from smithfarm/wip-19140-jewel

jewel: osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14324 from shinobu-x/wip-19371-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:48:08 +0000 (10:48 +0200)]
Merge pull request #14324 from shinobu-x/wip-19371-jewel

jewel: common: monitor creation with IPv6 public network segfaults

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14112 from shinobu-x/wip-19192-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:45:35 +0000 (10:45 +0200)]
Merge pull request #14112 from shinobu-x/wip-19192-jewel

jewel: tools: brag fails to count "in" mds

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14150 from smithfarm/wip-18823-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:45:11 +0000 (10:45 +0200)]
Merge pull request #14150 from smithfarm/wip-18823-jewel

jewel: tests: run-rbd-unit-tests.sh assert in lockdep_will_lock, TestLibRBD.ObjectMapConsistentSnap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14152 from smithfarm/wip-18893-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:44:47 +0000 (10:44 +0200)]
Merge pull request #14152 from smithfarm/wip-18893-jewel

jewel: librbd: Incomplete declaration for ContextWQ in librbd/Journal.h

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14154 from smithfarm/wip-18948-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:44:25 +0000 (10:44 +0200)]
Merge pull request #14154 from smithfarm/wip-18948-jewel

jewel: tests: additional rbd-mirror test stability improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14148 from smithfarm/wip-18778-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:44:04 +0000 (10:44 +0200)]
Merge pull request #14148 from smithfarm/wip-18778-jewel

jewel: rbd: rbd --pool=x rename y z does not work

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14083 from smithfarm/wip-19357-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:43:41 +0000 (10:43 +0200)]
Merge pull request #14083 from smithfarm/wip-19357-jewel

jewel: rbd: systemctl stop rbdmap unmaps all rbds and not just the ones in /etc/ceph/rbdmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #13154 from smithfarm/wip-18496-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:43:21 +0000 (10:43 +0200)]
Merge pull request #13154 from smithfarm/wip-18496-jewel

jewel: librbd: Possible deadlock performing a synchronous API action while refresh in-progress

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #13244 from smithfarm/wip-18775-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:42:54 +0000 (10:42 +0200)]
Merge pull request #13244 from smithfarm/wip-18775-jewel

jewel: rbd: qemu crash triggered by network issues

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #13809 from asheplyakov/18321-bp-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:42:23 +0000 (10:42 +0200)]
Merge pull request #13809 from asheplyakov/18321-bp-jewel

jewel: librbd: remove image header lock assertions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #13107 from smithfarm/wip-18669-jewel
Nathan Cutler [Wed, 12 Apr 2017 08:41:56 +0000 (10:41 +0200)]
Merge pull request #13107 from smithfarm/wip-18669-jewel

jewel: tests: [  FAILED  ] TestLibRBD.ImagePollIO in upgrade:client-upgrade-kraken-distro-basic-smithi

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #13585 from asheplyakov/jewel-bp-16585
Nathan Cutler [Wed, 12 Apr 2017 08:39:26 +0000 (10:39 +0200)]
Merge pull request #13585 from asheplyakov/jewel-bp-16585

jewel: msg: set close on exec flag

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14371 from tchaikov/wip-19429-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:56:22 +0000 (03:56 +0200)]
Merge pull request #14371 from tchaikov/wip-19429-jewel

jewel: tests: clone workunit using the branch specified by task

Nathan Cutler <ncutler@suse.com>

8 years agoMerge pull request #14325 from shinobu-x/wip-18619-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:54:11 +0000 (03:54 +0200)]
Merge pull request #14325 from shinobu-x/wip-18619-jewel

jewel: osd: degraded and misplaced status output inaccurate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14236 from smithfarm/wip-19392-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:53:25 +0000 (03:53 +0200)]
Merge pull request #14236 from smithfarm/wip-19392-jewel

jewel: mon: remove bad rocksdb option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14181 from mslovy/wip-19394-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:52:41 +0000 (03:52 +0200)]
Merge pull request #14181 from mslovy/wip-19394-jewel

jewel: osd: bypass readonly ops when osd full

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14113 from shinobu-x/wip-19319-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:49:49 +0000 (03:49 +0200)]
Merge pull request #14113 from shinobu-x/wip-19319-jewel

 jewel: cli: RadosImport::import should return an error if Rados::connect fails

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
8 years agoMerge pull request #14047 from asheplyakov/reindex-on-pg-split
Nathan Cutler [Wed, 12 Apr 2017 01:45:33 +0000 (03:45 +0200)]
Merge pull request #14047 from asheplyakov/reindex-on-pg-split

jewel: osd: reindex properly on pg log split

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14044 from mslovy/wip-19311-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:45:15 +0000 (03:45 +0200)]
Merge pull request #14044 from mslovy/wip-19311-jewel

jewel: core: os/filestore: fix clang static check warn use-after-free

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #13932 from asheplyakov/18911-bp-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:44:25 +0000 (03:44 +0200)]
Merge pull request #13932 from asheplyakov/18911-bp-jewel

jewel: rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #13831 from jan--f/wip-19206-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:42:13 +0000 (03:42 +0200)]
Merge pull request #13831 from jan--f/wip-19206-jewel

jewel: fs: Invalid error code returned by MDS is causing a kernel client WARNING

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #13827 from tchaikov/wip-19185-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:41:36 +0000 (03:41 +0200)]
Merge pull request #13827 from tchaikov/wip-19185-jewel

jewel: osd: ReplicatedPG: try with pool's use-gmt setting if hitset archive not found

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #13788 from shinobu-x/wip-18235-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:40:09 +0000 (03:40 +0200)]
Merge pull request #13788 from shinobu-x/wip-18235-jewel

jewel: core: os/filestore/HashIndex: be loud about splits

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #13786 from shinobu-x/wip-19129-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:39:27 +0000 (03:39 +0200)]
Merge pull request #13786 from shinobu-x/wip-19129-jewel

jewel: build/ops: ceph-base package missing dependency for psmisc

Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #13732 from liewegas/wip-19119-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:38:34 +0000 (03:38 +0200)]
Merge pull request #13732 from liewegas/wip-19119-jewel

jewel: doc: PendingReleaseNotes: note about 19119

Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #13541 from shinobu-x/wip-18929-jewel
Nathan Cutler [Wed, 12 Apr 2017 01:36:21 +0000 (03:36 +0200)]
Merge pull request #13541 from shinobu-x/wip-18929-jewel

 jewel: osd: restrict want_acting to up+acting on recovery completion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #13477 from asheplyakov/jewel-bp-18951
Nathan Cutler [Wed, 12 Apr 2017 01:34:54 +0000 (03:34 +0200)]
Merge pull request #13477 from asheplyakov/jewel-bp-18951

jewel: osd: --flush-journal: sporadic segfaults on exit

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>