]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
9 years agoosd: fix send_failures() locking 6755/head
Sage Weil [Fri, 18 Sep 2015 01:42:53 +0000 (21:42 -0400)]
osd: fix send_failures() locking

It is unsafe to check failure_queue.empty() without the lock.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b3ca828ae8ebc9068073494c46faf3e8e1443ada)

9 years agoMerge pull request #6402 from SUSE/wip-13621-hammer
Loic Dachary [Tue, 17 Nov 2015 09:24:45 +0000 (10:24 +0100)]
Merge pull request #6402 from SUSE/wip-13621-hammer

CephFS restriction on removing cache tiers is overly strict

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #6354 from theanalyst/wip-13387-hammer
Loic Dachary [Mon, 16 Nov 2015 21:06:51 +0000 (22:06 +0100)]
Merge pull request #6354 from theanalyst/wip-13387-hammer

librbd: reads larger than cache size hang

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #6289 from dillaman/wip-13461-hammer
Loic Dachary [Mon, 16 Nov 2015 18:48:07 +0000 (19:48 +0100)]
Merge pull request #6289 from dillaman/wip-13461-hammer

librbd: invalidate object map on error even w/o holding lock

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #6153 from dachary/wip-13205-hammer
Loic Dachary [Mon, 16 Nov 2015 17:06:25 +0000 (18:06 +0100)]
Merge pull request #6153 from dachary/wip-13205-hammer

ReplicatedBackend: populate recovery_info.size for clone (bug symptom is size mismatch on replicated backend on a clone in scrub)

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #6158 from dachary/wip-13336-hammer
Loic Dachary [Mon, 16 Nov 2015 17:06:12 +0000 (18:06 +0100)]
Merge pull request #6158 from dachary/wip-13336-hammer

osd: we do not ignore notify from down osds

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #6335 from Abhishekvrshny/wip-13488-hammer
Loic Dachary [Mon, 16 Nov 2015 17:06:01 +0000 (18:06 +0100)]
Merge pull request #6335 from Abhishekvrshny/wip-13488-hammer

object_info_t::decode() has wrong version

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #6401 from SUSE/wip-13620-hammer
Loic Dachary [Mon, 16 Nov 2015 17:05:39 +0000 (18:05 +0100)]
Merge pull request #6401 from SUSE/wip-13620-hammer

osd: pg stuck in replay

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #6213 from SUSE/wip-13425-hammer
Loic Dachary [Wed, 11 Nov 2015 10:59:36 +0000 (11:59 +0100)]
Merge pull request #6213 from SUSE/wip-13425-hammer

wrong conditional for boolean function KeyServer::get_auth()

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #6336 from Abhishekvrshny/wip-13535-hammer
Loic Dachary [Wed, 11 Nov 2015 05:59:55 +0000 (06:59 +0100)]
Merge pull request #6336 from Abhishekvrshny/wip-13535-hammer

LibRadosWatchNotify.WatchNotify2Timeout

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6391 from SUSE/wip-13590-hammer
Loic Dachary [Wed, 11 Nov 2015 05:59:42 +0000 (06:59 +0100)]
Merge pull request #6391 from SUSE/wip-13590-hammer

mon: should not set isvalid = true when cephx_verify_authorizer return false

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6398 from SUSE/wip-13588-hammer
Loic Dachary [Wed, 11 Nov 2015 05:58:55 +0000 (06:58 +0100)]
Merge pull request #6398 from SUSE/wip-13588-hammer

OSD::build_past_intervals_parallel() shall reset primary and up_primary when begin a new past_interval.

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6199 from liewegas/wip-randomize-scrub-hammer
Loic Dachary [Tue, 10 Nov 2015 19:24:35 +0000 (20:24 +0100)]
Merge pull request #6199 from liewegas/wip-randomize-scrub-hammer

osd: randomize scrub times

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5825 from tchaikov/wip-12848-hammer
Loic Dachary [Tue, 10 Nov 2015 19:23:23 +0000 (20:23 +0100)]
Merge pull request #5825 from tchaikov/wip-12848-hammer

ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoReplicatedBackend::prepare_pull: set recover_info.size for clones 6153/head
Samuel Just [Fri, 28 Aug 2015 19:46:57 +0000 (12:46 -0700)]
ReplicatedBackend::prepare_pull: set recover_info.size for clones

Fixes: #12828
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 1d3e46bedb9a8c0d2200e39f62f4e2c3337619f3)

9 years agoMerge pull request #6157 from dachary/wip-13040-hammer
Loic Dachary [Tue, 10 Nov 2015 09:34:01 +0000 (10:34 +0100)]
Merge pull request #6157 from dachary/wip-13040-hammer

common/Thread:pthread_attr_destroy(thread_attr) when done with it

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6155 from dachary/wip-13171-hammer
Loic Dachary [Tue, 10 Nov 2015 09:33:41 +0000 (10:33 +0100)]
Merge pull request #6155 from dachary/wip-13171-hammer

objecter: cancellation bugs

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6152 from dachary/wip-13233-hammer
Loic Dachary [Tue, 10 Nov 2015 09:33:19 +0000 (10:33 +0100)]
Merge pull request #6152 from dachary/wip-13233-hammer

mon: include min_last_epoch_clean as part of PGMap::print_summary and PGMap::dump

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6146 from dachary/wip-13337-hammer
Loic Dachary [Tue, 10 Nov 2015 09:33:01 +0000 (10:33 +0100)]
Merge pull request #6146 from dachary/wip-13337-hammer

segfault in agent_work

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6145 from dachary/wip-13338-hammer
Loic Dachary [Tue, 10 Nov 2015 09:32:53 +0000 (10:32 +0100)]
Merge pull request #6145 from dachary/wip-13338-hammer

filestore: fix peek_queue for OpSequencer

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6144 from dachary/wip-13339-hammer
Loic Dachary [Tue, 10 Nov 2015 09:32:42 +0000 (10:32 +0100)]
Merge pull request #6144 from dachary/wip-13339-hammer

mon: check for store writeablility before participating in election

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #5891 from Abhishekvrshny/wip-13035-hammer
Loic Dachary [Tue, 10 Nov 2015 09:30:54 +0000 (10:30 +0100)]
Merge pull request #5891 from Abhishekvrshny/wip-13035-hammer

requeue_scrub when kick_object_context_blocked

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #5890 from Abhishekvrshny/wip-13037-hammer
Loic Dachary [Tue, 10 Nov 2015 09:30:38 +0000 (10:30 +0100)]
Merge pull request #5890 from Abhishekvrshny/wip-13037-hammer

hit set clear repops fired in same epoch as map change -- segfault since they fall into the new interval even though the repops are cleared

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6322 from sponce/WIP-13210-hammer
Loic Dachary [Tue, 10 Nov 2015 09:21:07 +0000 (10:21 +0100)]
Merge pull request #6322 from sponce/WIP-13210-hammer

tests : BACKPORT #13210 Fixed broken Makefiles after integration of ttng into rados

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #5810 from Abhishekvrshny/wip-12948-hammer
Loic Dachary [Mon, 9 Nov 2015 07:38:31 +0000 (08:38 +0100)]
Merge pull request #5810 from Abhishekvrshny/wip-12948-hammer

Heavy memory shuffling in rados bench

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #5530 from SUSE/wip-12587-hammer
Loic Dachary [Thu, 5 Nov 2015 12:12:11 +0000 (13:12 +0100)]
Merge pull request #5530 from SUSE/wip-12587-hammer

FileStore calls syncfs(2) even it is not supported

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #5889 from Abhishekvrshny/wip-13042-hammer
Loic Dachary [Wed, 4 Nov 2015 18:38:53 +0000 (19:38 +0100)]
Merge pull request #5889 from Abhishekvrshny/wip-13042-hammer

ThreadPool add/remove work queue methods not thread safe

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
9 years agoMerge pull request #6151 from dachary/wip-13245-hammer
Loic Dachary [Wed, 4 Nov 2015 18:14:28 +0000 (19:14 +0100)]
Merge pull request #6151 from dachary/wip-13245-hammer

client nonce collision due to unshared pid namespaces

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
9 years agoMerge pull request #6159 from dachary/wip-13341-hammer
Loic Dachary [Wed, 4 Nov 2015 10:36:39 +0000 (11:36 +0100)]
Merge pull request #6159 from dachary/wip-13341-hammer

ceph upstart script rbdmap.conf incorrectly processes parameters

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #6154 from dachary/wip-13195-hammer
Loic Dachary [Wed, 4 Nov 2015 10:34:07 +0000 (11:34 +0100)]
Merge pull request #6154 from dachary/wip-13195-hammer

should recalc the min_last_epoch_clean when decode PGMap

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6132 from SUSE/wip-13307-hammer
Loic Dachary [Wed, 4 Nov 2015 10:28:03 +0000 (11:28 +0100)]
Merge pull request #6132 from SUSE/wip-13307-hammer

dumpling incrementals do not work properly on hammer and newer

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #6097 from dillaman/wip-13045-hammer
Loic Dachary [Wed, 4 Nov 2015 10:23:14 +0000 (11:23 +0100)]
Merge pull request #6097 from dillaman/wip-13045-hammer

librbd: diff_iterate needs to handle holes in parent images

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #6060 from XinzeChi/wip-hammer-outdata-set
Loic Dachary [Wed, 4 Nov 2015 10:22:25 +0000 (11:22 +0100)]
Merge pull request #6060 from XinzeChi/wip-hammer-outdata-set

bug fix: osd: avoid multi set osd_op.outdata in tier pool

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5897 from tchaikov/wip-12940-hammer
Loic Dachary [Wed, 4 Nov 2015 10:20:50 +0000 (11:20 +0100)]
Merge pull request #5897 from tchaikov/wip-12940-hammer

IO error on kvm/rbd with an erasure coded pool tier

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5787 from SUSE/wip-12923-hammer
Loic Dachary [Wed, 4 Nov 2015 10:11:00 +0000 (11:11 +0100)]
Merge pull request #5787 from SUSE/wip-12923-hammer

logrotate reload error on Ubuntu 14.04

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5766 from dachary/wip-12590-hammer
Loic Dachary [Wed, 4 Nov 2015 10:10:26 +0000 (11:10 +0100)]
Merge pull request #5766 from dachary/wip-12590-hammer

ceph mds add_data_pool check for EC pool is wrong

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #5361 from ceph/wip-11786.hammer
Loic Dachary [Wed, 4 Nov 2015 10:06:18 +0000 (11:06 +0100)]
Merge pull request #5361 from ceph/wip-11786.hammer

mon: MonitorDBStore: get_next_key() only if prefix matches

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoqa: update cephtool test for CephFS tier cases 6402/head
John Spray [Mon, 1 Jun 2015 12:55:22 +0000 (13:55 +0100)]
qa: update cephtool test for CephFS tier cases

1. Creating a filesystem using a
   readonly tier on an EC pool (should be forbidden)
2. Removing a tier from a replicated base pool (should
   be permitted)

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit f9b11b0abe7c2b0dcdfc5802336be8a0426ccb3a)

9 years agomon: forbid readonly cache tiers for CephFS
John Spray [Mon, 1 Jun 2015 12:54:25 +0000 (13:54 +0100)]
mon: forbid readonly cache tiers for CephFS

For EC pools, we need a tier that is not just forwarding
writes, in order to provide the ops required
by CephFS.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 11e5faf3c1d4fa28a8b41e3a92f0f684a844e5f5)

9 years agomon: refine check_remove_tier checks
John Spray [Thu, 7 May 2015 13:23:37 +0000 (14:23 +0100)]
mon: refine check_remove_tier checks

Fixes: #11504
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit a50c8f1f2ad8845c7f77110868f9376f2d1ff883)

9 years agoosd: fix requeue of replay requests during activating 6401/head
Sage Weil [Tue, 22 Sep 2015 17:57:37 +0000 (13:57 -0400)]
osd: fix requeue of replay requests during activating

If the replay period expires while we are still in the activating
state, we can simply insert our list of requests at the front of
the waiting_for_active list.

Fixes: #13116
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d18cf51d9419819cdda3782b188b010969288911)

9 years agoOSD:shall reset primary and up_primary fields when beginning a new past_interval. 6398/head
xiexingguo [Tue, 13 Oct 2015 06:04:20 +0000 (14:04 +0800)]
OSD:shall reset primary and up_primary fields when beginning a new past_interval.

Shall reset primary and up_primary fields when we start over a new past_interval in OSD::build_past_intervals_parallel().
Fixes: #13471
Signed-off-by: xie.xingguo@zte.com.cn
(cherry picked from commit 65064ca05bc7f8b6ef424806d1fd14b87add62a4)

9 years agomon: should not set isvalid = true when cephx_verify_authorizer return false 6391/head
yangruifeng [Mon, 19 Oct 2015 12:08:12 +0000 (08:08 -0400)]
mon: should not set isvalid = true when cephx_verify_authorizer return false

Fixes: #13525
Signed-off-by: Ruifeng Yang <yangruifeng.09209@h3c.com>
(cherry picked from commit c7f75b8f7c0a773148ec16141941efd00ee76626)

9 years ago0.94.5 v0.94.5
Jenkins [Mon, 26 Oct 2015 14:05:32 +0000 (07:05 -0700)]
0.94.5

9 years agoMerge pull request #6348 from dillaman/wip-13567-hammer
Loic Dachary [Fri, 23 Oct 2015 18:41:33 +0000 (20:41 +0200)]
Merge pull request #6348 from dillaman/wip-13567-hammer

librbd: potential assertion failure during cache read

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6362 from liewegas/wip-shut-up-osd-hammer
Loic Dachary [Fri, 23 Oct 2015 13:20:32 +0000 (15:20 +0200)]
Merge pull request #6362 from liewegas/wip-shut-up-osd-hammer

osd/ReplicatedPG: remove stray debug line

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoosd/ReplicatedPG: remove stray debug line 6362/head
Sage Weil [Sat, 12 Sep 2015 12:33:44 +0000 (08:33 -0400)]
osd/ReplicatedPG: remove stray debug line

This snuck in

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ff9600a6cef613d40e875597b6392778df1bb04c)

9 years agolibrbd: potential assertion failure during cache read 6348/head
Jason Dillaman [Wed, 21 Oct 2015 17:12:48 +0000 (13:12 -0400)]
librbd: potential assertion failure during cache read

It's possible for a cache read from a clone to trigger a writeback if a
previous read op determined the object doesn't exist in the clone,
followed by a cached write to the non-existent clone object, followed
by another read request to the same object.  This causes the cache to
flush the pending writeback ops while not holding the owner lock.

Fixes: #13559
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4692c330bd992a06b97b5b8975ab71952b22477a)

9 years agotests: reproduce crash during read-induced CoW
Jason Dillaman [Wed, 21 Oct 2015 17:09:54 +0000 (13:09 -0400)]
tests: reproduce crash during read-induced CoW

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 382219b6bba09156f8cf3d420c6348738e7ed4c6)

9 years agoMerge pull request #6330 from dachary/wip-13550-hammer
Jason Dillaman [Thu, 22 Oct 2015 16:14:42 +0000 (12:14 -0400)]
Merge pull request #6330 from dachary/wip-13550-hammer

qemu workunit refers to apt-mirror.front.sepia.ceph.com

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agolibrbd:reads larger than cache size hang. 6354/head
Lu Shi [Thu, 24 Sep 2015 08:02:41 +0000 (16:02 +0800)]
librbd:reads larger than cache size hang.

Fixes:#13164

Signed-off-by: Lu Shi <shi.lu@h3c.com>
(cherry picked from commit 9c8200bb5d1ac9359803a182df03298b565b8479)

9 years agoradosstriper : Fixed broken Makefiles after integration of lttng 6322/head
Sebastien Ponce [Tue, 20 Oct 2015 07:22:16 +0000 (09:22 +0200)]
radosstriper : Fixed broken Makefiles after integration of lttng

Fixes: #13210, backport of #6322 into hammer
Signed-off-by: Sebastien Ponce <sebastien.ponce@cern.ch>
9 years agoosdc/Objecter: distinguish between multiple notify completions 6336/head
Sage Weil [Thu, 1 Oct 2015 18:50:34 +0000 (14:50 -0400)]
osdc/Objecter: distinguish between multiple notify completions

We may send a notify to the cluster multiple times due to OSDMap
changes.  In some cases, earlier notify attempts may complete with
an error, while later attempts succeed.  We need to only pay
attention to the most-recently send notify's completion.

Do this by making note of the notify_id in the initial ACK (only
present when talking to newer OSDs).  When we get a notify
completion, match it against our expected notify_id (if we have
one) or else discard it.

This is important because in some cases an early notify completion
may be an error while a later one succeeds.

Note that if we are talking to an old cluster we will simply not record a
notify_id and our behavior will be the same as before (we will trust any
notify completion we get).

Fixes: #13114
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e86d033854c76f344c678e92016c4e5c5e0385e2)

Conflicts:
src/osdc/Objecter.cc
In Objecter::handle_watch_notify, a conflict was there due to a modified comment by commit 47277c51db7bb2725ea117e4e8834869ae93e006, which was not backported

9 years agoosd: reply to notify request with our unique notify_id
Sage Weil [Thu, 1 Oct 2015 18:50:00 +0000 (14:50 -0400)]
osd: reply to notify request with our unique notify_id

The OSD assigns a unique ID to each notify it queues for
processing.  Include this in the reply to the notifier so that
they can match it up with the eventual completions they receive.

This is necessary to distinguish between multiple completions
they may receive if there is PG peering and the notify is resent.
In particular, an earlier notify may return an error when a later
attempt succeeds.

This is forwards and backwards compatible: new clients will make use of
this reply payload but older clients ignore it.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 049ea702b9bf4a8a62ae2770d9ba55c0730b3eef)

9 years agoosd: Correct the object_info_t::decode() version 6335/head
David Zafman [Mon, 12 Oct 2015 20:24:31 +0000 (13:24 -0700)]
osd: Correct the object_info_t::decode() version

Caused by: 13b9dc70

Fixes: #13462
Backport: Hammer, Firefly

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2b7ddde685213f3908f2dd9545b35f1d97f35a79)

9 years agoqa: Use public qemu repo 6330/head
Yuan Zhou [Thu, 18 Jun 2015 16:02:20 +0000 (00:02 +0800)]
qa: Use public qemu repo

This would allow some external tests outside of sepia lab

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
(cherry picked from commit 4731c1e35539c9506ff3fe3141553fad099d0eee)

9 years ago0.94.4 v0.94.4
Jenkins [Mon, 19 Oct 2015 14:43:50 +0000 (07:43 -0700)]
0.94.4

9 years agolibrbd: invalidate object map on error even w/o holding lock 6289/head
Jason Dillaman [Tue, 6 Oct 2015 16:31:59 +0000 (12:31 -0400)]
librbd: invalidate object map on error even w/o holding lock

If there is a deep bug that prevents the object map from owning
the lock before objects are updated, flag the object map as invalid
regardless.

Fixes: #13372
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e675400df7f8ba796d60e9bac4234857f0cb1392)

 Conflicts:
src/librbd/ObjectMap.h/cc: hammer branch doesn't support rebuild

9 years agoMerge pull request #6287 from jcsp/wip-test-fixes
Sage Weil [Thu, 15 Oct 2015 20:35:10 +0000 (16:35 -0400)]
Merge pull request #6287 from jcsp/wip-test-fixes

git path backports for hammer

9 years agouse git://git.ceph.com 6287/head
Sage Weil [Thu, 16 Apr 2015 15:29:41 +0000 (08:29 -0700)]
use git://git.ceph.com

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 34c467ba06345eacd2fd0e79c162dfd0b22f0486)

9 years agoqa: http://ceph.com/qa -> http://download.ceph.com/qa
Sage Weil [Thu, 24 Sep 2015 22:10:46 +0000 (18:10 -0400)]
qa: http://ceph.com/qa -> http://download.ceph.com/qa

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d4d65fbd6ad15320339751c604613ac63511e645)

9 years agoMerge pull request #6161 from dachary/wip-13227-hammer
Loic Dachary [Mon, 12 Oct 2015 22:28:52 +0000 (00:28 +0200)]
Merge pull request #6161 from dachary/wip-13227-hammer

With root as default user, unable to have multiple RGW instances running

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
9 years agoinit-radosgw.sysv: remove 6161/head
Sage Weil [Thu, 7 May 2015 22:40:05 +0000 (15:40 -0700)]
init-radosgw.sysv: remove

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9d8c115b0ab05cfc1db6670d1d367737e61fe0ea)

9 years agoMerge pull request #6166 from dachary/wip-13046-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:16:17 +0000 (08:16 -0700)]
Merge pull request #6166 from dachary/wip-13046-hammer

RGW : setting max number of buckets for user via ceph.conf option

9 years agoMerge pull request #6039 from ceph/wip-13015-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:15:21 +0000 (08:15 -0700)]
Merge pull request #6039 from ceph/wip-13015-hammer

rgw: don't preserve acls when copying object

9 years agoMerge pull request #6042 from dreamhost/wip-hammer-crossftp
Yehuda Sadeh [Mon, 12 Oct 2015 15:14:47 +0000 (08:14 -0700)]
Merge pull request #6042 from dreamhost/wip-hammer-crossftp

rgw: remove trailing :port from host for purposes of subdomain matching

9 years agoMerge pull request #6160 from dachary/wip-13228-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:14:09 +0000 (08:14 -0700)]
Merge pull request #6160 from dachary/wip-13228-hammer

rgw: segments are read during HEAD on Swift DLO

9 years agoMerge pull request #6162 from dachary/wip-13226-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:13:25 +0000 (08:13 -0700)]
Merge pull request #6162 from dachary/wip-13226-hammer

Keystone Fernet tokens break auth

9 years agoMerge pull request #6163 from dachary/wip-13054-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:12:26 +0000 (08:12 -0700)]
Merge pull request #6163 from dachary/wip-13054-hammer

rgw: region data still exist in region-map after region-map update

9 years agoMerge pull request #6164 from dachary/wip-13053-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:12:07 +0000 (08:12 -0700)]
Merge pull request #6164 from dachary/wip-13053-hammer

rgw: GWWatcher::handle_error -> common/Mutex.cc: 95: FAILED assert(r == 0)

9 years agoMerge pull request #5718 from dachary/wip-12859-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:11:16 +0000 (08:11 -0700)]
Merge pull request #5718 from dachary/wip-12859-hammer

testGetContentType and testHead failed

9 years agoMerge pull request #5860 from lebauce/wip-12960-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:10:55 +0000 (08:10 -0700)]
Merge pull request #5860 from lebauce/wip-12960-hammer

rgw: add delimiter to prefix only when path is specified

9 years agoMerge pull request #6165 from dachary/wip-13052-hammer
Yehuda Sadeh [Mon, 12 Oct 2015 15:08:59 +0000 (08:08 -0700)]
Merge pull request #6165 from dachary/wip-13052-hammer

rgw: init_rados failed leads to repeated delete

9 years agoReplicatedPG::hit_set_setup: fix hit_set_remove_all call 5825/head
Samuel Just [Mon, 21 Sep 2015 18:16:49 +0000 (11:16 -0700)]
ReplicatedPG::hit_set_setup: fix hit_set_remove_all call

We only want to do it if the pool config changed AND we are
primary && active.

fd38902dd4693b9b72f7532833e78f5df2c9efa5 partially fixed a related
bug.  This should be backported along with it.

Fixes: 13192
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 36e4a80c39f7daab2a35efc04650ae8b7b11c4e0)

9 years agoosd/: assert in HitSet constructor if type is TYPE_NONE as well
Samuel Just [Tue, 22 Sep 2015 18:13:44 +0000 (11:13 -0700)]
osd/: assert in HitSet constructor if type is TYPE_NONE as well

Otherwise impl is unpopulated and we risk segfaults in several
methods.  It also seems like it would always indicate a bug.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 8e5a801b43babf751b619ed9606d7be7e5b12837)

9 years agoosd: only remove all the hit set objects on primary osd
Zhiqiang Wang [Thu, 23 Jul 2015 01:26:44 +0000 (09:26 +0800)]
osd: only remove all the hit set objects on primary osd

Only the primary PG is allowed to remove all the hit set objects. And
the PG should be in the active or peered states.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
(cherry picked from commit fd38902dd4693b9b72f7532833e78f5df2c9efa5)

9 years agoosd: implement hit_set_remove_all
Zhiqiang Wang [Thu, 18 Jun 2015 01:05:28 +0000 (09:05 +0800)]
osd: implement hit_set_remove_all

When hit set is not configured on startup or on a change, remove all
previous hitsets.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
(cherry picked from commit be28319bf3dc54b4b6f400d83405a29facfe3bd4)
Conflicts:
        src/osd/ReplicatedPG.cc
          pass the new argument of `p->using_gmt` to get_hit_set_archive_object()

9 years agomon: disable gmt_hitset if not supported
Kefu Chai [Tue, 8 Sep 2015 08:52:32 +0000 (16:52 +0800)]
mon: disable gmt_hitset if not supported

the gmt_hitset is enabled by default in the ctor of pg_pool_t, this
is intentional. because we want to remove this setting and make
gmt_hitset=true as a default in future. but this forces us to
disable it explicitly when preparing a new pool if any OSD does
not support gmt hitset.

Fixes: #12968
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e48cec3dc93b3988dcd8924933deb1b3a43e1d0f)

Conflicts:
src/mon/OSDMonitor.cc
remove unrelated hunk

9 years agoosd: do not let OSD_HITSET_GMT reuse the feature bit
Kefu Chai [Thu, 27 Aug 2015 16:32:38 +0000 (00:32 +0800)]
osd: do not let OSD_HITSET_GMT reuse the feature bit

* to ease the backport to hammer

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5a4f6a866bb675195ad83c38fd23238d078c78ed)

Conflicts:
src/include/ceph_features.h
minor changes to resolve conflicts

9 years agoosd: Decode use_gmt_hitset with a unique version
David Zafman [Thu, 27 Aug 2015 18:24:25 +0000 (11:24 -0700)]
osd: Decode use_gmt_hitset with a unique version

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2bc5a48f4c5d3667213be3a7b5a0e0f5ef9daf4f)

Conflicts:
src/osd/osd_types.cc
minor changes to resolve conflicts

9 years agomon: print use_gmt_hitset in "ceph osd pool get"
Kefu Chai [Tue, 18 Aug 2015 08:53:50 +0000 (16:53 +0800)]
mon: print use_gmt_hitset in "ceph osd pool get"

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cc2bcf760f2d2f20fc4a2fa78ba52475a23e55bf)

Conflicts:
src/mon/OSDMonitor.cc
use the old way to dump pool info

9 years agomon: add "ceph osd pool set $pool use_gmt_hitset true" cmd
Kefu Chai [Mon, 17 Aug 2015 16:04:23 +0000 (00:04 +0800)]
mon: add "ceph osd pool set $pool use_gmt_hitset true" cmd

allow "ceph osd pool set $pool use_gmt_hitset <true|1>" as long as
the cluster supports gmt hitset.

Fixes: #9732
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 03a1a3cf023a9aeb2fa26820e49e5efe3f3b3789)

9 years agoosd: use GMT time for the object name of hitsets
Kefu Chai [Fri, 5 Jun 2015 13:06:48 +0000 (21:06 +0800)]
osd: use GMT time for the object name of hitsets

* bump the encoding version of pg_hit_set_info_t to 2, so we can
  tell if the corresponding hit_set is named using localtime or
  GMT
* bump the encoding version of pg_pool_t to 20, so we can know
  if a pool is using GMT to name the hit_set archive or not. and
  we can tell if current cluster allows OSDs not support GMT
  mode or not.
* add an option named `osd_pool_use_gmt_hitset`. if enabled,
  the cluster will try to use GMT mode when creating a new pool
  if all the the up OSDs support GMT mode. if any of the
  pools in the cluster is using GMT mode, then only OSDs
  supporting GMT mode are allowed to join the cluster.

Fixes: #9732
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42f8c5daad16aa849a0b99871d50161673c0c370)

 Conflicts:
src/include/ceph_features.h
src/osd/ReplicatedPG.cc
src/osd/osd_types.cc
src/osd/osd_types.h
fill pg_pool_t with default settings in master branch.

9 years agoCephxServiceHandler.cc: fix get_auth conditional 6213/head
Nathan Cutler [Sun, 27 Sep 2015 19:55:00 +0000 (21:55 +0200)]
CephxServiceHandler.cc: fix get_auth conditional

The KeyServer class has a public method get_auth() that returns a boolean
value. This value is being checked here - fix the conditional so it triggers
when get_auth() returns false.

http://tracker.ceph.com/issues/9756 References: #9756

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 7250db62cb7ac1e9c0b1f5956d60b9d874496702)

9 years agoMerge pull request #6201 from dachary/wip-13410-hammer
Loic Dachary [Fri, 9 Oct 2015 07:17:19 +0000 (09:17 +0200)]
Merge pull request #6201 from dachary/wip-13410-hammer

TEST_crush_rule_create_erasure consistently fails on i386 builder

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #5885 from Abhishekvrshny/wip-13034-hammer
Sage Weil [Thu, 8 Oct 2015 16:18:49 +0000 (12:18 -0400)]
Merge pull request #5885 from Abhishekvrshny/wip-13034-hammer

osd: copy-from doesn't preserve truncate_{seq,size}

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agotests: robust test for the pool create crushmap test 6201/head
Loic Dachary [Wed, 22 Jul 2015 08:42:32 +0000 (10:42 +0200)]
tests: robust test for the pool create crushmap test

The test that goes with f1e86be589803596e86acc964ac5c5c03b4038d8 to
verify that a bugous crush ruleset will prevent the creation of a pool
trying to use it was fragile. I depends on the implementation of the
erasure code lrc plugin and turns out to not work on i386.

The test is modified to use a fake crushtool that always returns false
and validate that it prevents the creation of a pool, which demonstrate
it is used for crushmap validation prior to the pool creation.

http://tracker.ceph.com/issues/12419 Fixes: #12419

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit e47903727b5d53f795014bf29b417d357f2bd65a)

9 years agoosd: use another name for randomize scrub option 6199/head
Kefu Chai [Fri, 15 May 2015 16:07:27 +0000 (00:07 +0800)]
osd: use another name for randomize scrub option

s/osd_scrub_interval_limit/osd_scrub_interval_randomize_ratio/

Fixes: #10973
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoosd: randomize scrub times to avoid scrub wave
Kefu Chai [Mon, 9 Mar 2015 08:42:34 +0000 (16:42 +0800)]
osd: randomize scrub times to avoid scrub wave

- to avoid the scrub wave when the osd_scrub_max_interval reaches in a
  high-load OSD, the scrub time is randomized.
- extract scrub_load_below_threshold() out of scrub_should_schedule()
- schedule an automatic scrub job at a time which is uniformly distributed
  over [now+osd_scrub_min_interval,
        now+osd_scrub_min_interval*(1+osd_scrub_time_limit]. before
  this change this sort of scrubs will be performed once the hard interval
  is end or system load is below the threshold, but with this change, the
  jobs will be performed as long as the load is low or the interval of
  the scheduled scrubs is longer than conf.osd_scrub_max_interval. all
  automatic jobs should be performed in the configured time period, otherwise
  they are postponed.
- the requested scrub job will be scheduled right away, before this change
  it is queued with the timestamp of `now` and postponed after
  osd_scrub_min_interval.

Fixes: #10973
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5e44040e8528bff06cc0a5a3f3293ab146e0e4e1)

Conflicts:
src/osd/OSD.cc

9 years agoosd: use __func__ in log messages
Kefu Chai [Mon, 9 Mar 2015 08:47:30 +0000 (16:47 +0800)]
osd: use __func__ in log messages

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0f7f35670f03f3f58329c00d323963710e7e495d)

9 years agoosd: simplify OSD::scrub_load_below_threshold() a little bit
Kefu Chai [Wed, 15 Apr 2015 04:04:05 +0000 (12:04 +0800)]
osd: simplify OSD::scrub_load_below_threshold() a little bit

avoid unnecessary comparison

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2ab0e606dfd2847e1918be9f21f572d91befbf7b)

9 years agolibrbd: invoking RBD::open twice will leak memory 6097/head
Jason Dillaman [Fri, 1 May 2015 18:21:35 +0000 (14:21 -0400)]
librbd: invoking RBD::open twice will leak memory

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a5e88fcf11ebb16d4a1405b21cf6f895b37f185e)

9 years agoMerge pull request #6192 from dachary/wip-13401-hammer
Loic Dachary [Wed, 7 Oct 2015 22:09:24 +0000 (00:09 +0200)]
Merge pull request #6192 from dachary/wip-13401-hammer

mon: fix crush testing for new pools

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agocrush/CrushTester: test fewer inputs when running crushtool 6192/head
Sage Weil [Fri, 21 Aug 2015 20:40:34 +0000 (16:40 -0400)]
crush/CrushTester: test fewer inputs when running crushtool

If there are a lot of crush rules (say, 100) then the test can
take a long time.  100 values per rule should be enough to catch
most issues.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0f82f461b33d93d868e185912a2c7e4074d06900)

Conflicts:
src/crush/CrushTester.cc
  in hammer the crushtool validation is via a shell
       and not via an internal subprocess utility

9 years agotests: update to match crushmap validation message
Loic Dachary [Sun, 20 Sep 2015 21:42:45 +0000 (23:42 +0200)]
tests: update to match crushmap validation message

http://tracker.ceph.com/issues/13182 Fixes: #13182

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 21a1e75d8a7bad89a48cd9d36902c5d609be5015)

Conflicts:
qa/workunits/cephtool/test.sh
  the test structure in hammer is different, adapt
  the relevant test.

9 years agomon/OSDMonitor: fix crush injection error message
Sage Weil [Fri, 18 Sep 2015 13:42:47 +0000 (09:42 -0400)]
mon/OSDMonitor: fix crush injection error message

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1b3090d50e5bd5ca3e6e396b23d2d9826896c718)

Conflicts:
src/mon/OSDMonitor.cc
  hammer has a separate error message for EINTR
  which is in the context of the fix for error message

9 years agomon/OSDMonitor: only test crush ruleset for the newly created pool
Sage Weil [Fri, 18 Sep 2015 13:41:25 +0000 (09:41 -0400)]
mon/OSDMonitor: only test crush ruleset for the newly created pool

Otherwise, we test *all* crush rules.. which might be a lot, and
which is a big waste of time and effort.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 524b0bdcc45c2f4b95f2239c988e93250f337f3d)

Conflicts:
src/mon/OSDMonitor.cc : trivial resolution

9 years agocrush/CrushTester: allow testing by ruleset
Sage Weil [Fri, 18 Sep 2015 13:40:13 +0000 (09:40 -0400)]
crush/CrushTester: allow testing by ruleset

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit aa238e5ed50f44a94caf84567267e4f6be8732a2)

Conflicts:
src/crush/CrushTester.cc
          in hammer the crushtool validation is via a shell
  and not via an internal subprocess utility
src/tools/crushtool.cc
  ceph_argparse_withint is preferred to ceph_argparse_witharg

9 years agoMerge pull request #5887 from Abhishekvrshny/wip-13044-hammer
Sage Weil [Tue, 6 Oct 2015 15:11:14 +0000 (11:11 -0400)]
Merge pull request #5887 from Abhishekvrshny/wip-13044-hammer

LibCephFS.GetPoolId failure

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #6172 from dachary/wip-13354-hammer
Loic Dachary [Sun, 4 Oct 2015 13:33:49 +0000 (15:33 +0200)]
Merge pull request #6172 from dachary/wip-13354-hammer

qa/workunits/cephtool/test.sh: don't assume crash_replay_interval=45

Reviewed-by: Loic Dachary <ldachary@redhat.com>