]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
11 years agomds: fix scattered wrlock rejoin
Yan, Zheng [Sun, 19 Jan 2014 04:13:06 +0000 (12:13 +0800)]
mds: fix scattered wrlock rejoin

If unstable scatter lock is encountered when handling weak cache
rejoin, don't remove the recovering MDS from the scatter lock's
gather list. The reason is the recovering MDS may hold rejoined
wrlock on the scatter lock. (Rejoined wrlocks were created when
handling strong cache rejoins from survivor MDS)

When composing cache rejoin ack, if the recovering MDS is in lock's
gather list, set lock state of the recovering MDS to a compatible
unstable stable.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: fixes for thrash fragment
Yan, Zheng [Sun, 19 Jan 2014 02:36:45 +0000 (10:36 +0800)]
mds: fixes for thrash fragment

This patch contains 3 changes:
- limit the number of in progress fragmenting processes.
- reduce the probability of splitting small dirfrag.
- process the merge_queue when thrash_fragments is enabled.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: force fragment subtree bounds when replaying ESubtreeMap
Yan, Zheng [Sun, 19 Jan 2014 01:38:58 +0000 (09:38 +0800)]
mds: force fragment subtree bounds when replaying ESubtreeMap

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: fix 'force dirfrags' during journal replay
Yan, Zheng [Sat, 18 Jan 2014 13:34:47 +0000 (21:34 +0800)]
mds: fix 'force dirfrags' during journal replay

For rename operation, null dentry is first replayed, it detaches
the inode from the FS hierarchy. Then primary dentry is replayed,
it updates the inode and re-attaches the inode to the FS hierarchy.
We may call CInode::force_dirfrag() when updating the inode. But
CInode::force_dirfrag() doesn't work well when inode is detached
from the FS hierarchy because adjusting fragments may also adjust
subtree map. The fix is don't detach the inode when replaying the
null dentry.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: journal dirfragtree change
Yan, Zheng [Sat, 18 Jan 2014 12:31:36 +0000 (20:31 +0800)]
mds: journal dirfragtree change

Introduce new flag DIRTYDFT to CDir and EMetaBlob::dirlump, the new
flag indicates the dirfrag is newly fragmented and the corresponding
dirfragtree change hasn't been propagate to the directory inode.

After fragmenting subtree dirfrags, make sure DIRTYDFT flag is set
on EMetaBlob::dirlump that correspond to the resulting dirfrags.
Journal replay code uses DIRTYDFT frag to decide if dirfragtree is
scattered dirty.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: allow fragmenting subtree dirfrags
Yan, Zheng [Sat, 18 Jan 2014 00:27:02 +0000 (08:27 +0800)]
mds: allow fragmenting subtree dirfrags

We can't wait until object becomes auth pinnable after freezing a
dirfrag/subtree, because it can cause deadlock. Current fragmenting
dirfrag code checks if the directory inode is auth pinnable, then
calls Locker::acquire_locks(). It avoids deadlock, but also forbids
fragmenting subtree dirfrags. We can get rid of the limitation by
using 'nonlocking auth pin' mode of Locker::acquire_locks().

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: preserve dir_auth when spliting/merging dirfrags
Yan, Zheng [Thu, 16 Jan 2014 11:54:07 +0000 (19:54 +0800)]
mds: preserve dir_auth when spliting/merging dirfrags

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: minor cleanup for EFragment and MMDSFragmentNotify
Yan, Zheng [Thu, 16 Jan 2014 03:24:08 +0000 (11:24 +0800)]
mds: minor cleanup for EFragment and MMDSFragmentNotify

pass dirfrag_t to their constructors, instead of passing inode_t
and frag_t separately.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: freeze dir deadlock detection
Yan, Zheng [Thu, 16 Jan 2014 00:15:08 +0000 (08:15 +0800)]
mds: freeze dir deadlock detection

freezing dir and freezing tree have the same deadlock cases.
This patch adds freeze dir deadlock detection, which imitates
commit ab93aa59 (mds: freeze tree deadlock detection)

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: improve freeze tree deadlock detection
Yan, Zheng [Wed, 15 Jan 2014 08:49:23 +0000 (16:49 +0800)]
mds: improve freeze tree deadlock detection

Current code uses the start time of freezing tree to detect deadlock.
It is better to check how long the auth pin count of freezing tree
stays unchanged to decide if there is potential deadlock.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: handle frag mismatch for cache expire
Yan, Zheng [Wed, 15 Jan 2014 07:31:25 +0000 (15:31 +0800)]
mds: handle frag mismatch for cache expire

When sending MDSFragmentNotify to peers, also replicate the new
dirfrags. This guarantees peers get new replica nonces for the
new dirfrags. So it's safe to ignore mismatched/old dirfrags in
the cache expire message.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: handle frag mismatch for cache rejoin weak
Yan, Zheng [Wed, 15 Jan 2014 05:24:54 +0000 (13:24 +0800)]
mds: handle frag mismatch for cache rejoin weak

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: fix open undef dirfrags
Yan, Zheng [Wed, 15 Jan 2014 04:13:52 +0000 (12:13 +0800)]
mds: fix open undef dirfrags

Undef inode may contain a undef dirfrag (*). When undef inode is
opened, we should force fragment the undef dirfrag (*). because
we may open other dirfrags later, the undef dirfrag (*) will
overlap with them.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: properly set COMPLETE flag when merging dirfrags
Yan, Zheng [Wed, 15 Jan 2014 03:12:43 +0000 (11:12 +0800)]
mds: properly set COMPLETE flag when merging dirfrags

don't keep the COMPLETE flag when merging dirfrags during journal
replay, because it's inconvenience to check if the all dirfrags
under the 'basefrag' are in the cache and complete. One special case
is that newly created dirfrag get fragmented, then the fragment
operation get rolled back. The COMPLETE flag should be preserved in
this case because the dirfrag still doesn't exist on object store.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: fix CInode::get_dirfrags_under()
Yan, Zheng [Wed, 15 Jan 2014 01:23:52 +0000 (09:23 +0800)]
mds: fix CInode::get_dirfrags_under()

make the function work when opened dirfrags don't match the
dirfragtree.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: fix MDCache::adjust_subtree_after_rename()
Yan, Zheng [Tue, 14 Jan 2014 02:08:40 +0000 (10:08 +0800)]
mds: fix MDCache::adjust_subtree_after_rename()

process subtree dirfrags first, then process nested dirfrags. because
the code that processes nested dirfrags treats unprocessed subtree
dirfrags as child directories' dirfrags.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: fix MDCache::get_force_dirfrag_bound_set()
Yan, Zheng [Tue, 14 Jan 2014 01:07:16 +0000 (09:07 +0800)]
mds: fix MDCache::get_force_dirfrag_bound_set()

don't force dir fragments according to the subtree bounds in resolve
message. The resolve message was not sent by the auth MDS of these
subtree bounds dirfrags.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: handle frag mismatch for discover
Yan, Zheng [Mon, 13 Jan 2014 06:52:20 +0000 (14:52 +0800)]
mds: handle frag mismatch for discover

When handle discover dirfrag message, choose an approximate frag if
the requested dirfrag doesn't exist. When handling discover dirfrag
replay, wake up appropriate waiters if the reply is different from
the the requested dirfrag.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: use discover_path to open remote inode
Yan, Zheng [Mon, 13 Jan 2014 03:32:38 +0000 (11:32 +0800)]
mds: use discover_path to open remote inode

MDCache::discover_ino() doesn't work well for directories that are
fragmented to several dirfrags. Because MDCache::handle_discover()
doesn't know which dirfrags the inode lives in when the sender has
outdatad frag information.

This patch replaces all use of MDCache::discover_ino() with
MDCache::discover_path().

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: introduce fine-grained discover dirfrag wait queue
Yan, Zheng [Mon, 13 Jan 2014 02:26:08 +0000 (10:26 +0800)]
mds: introduce fine-grained discover dirfrag wait queue

Current discover dirfrag code only allows discover one dirfrag at
a time. This can cause deadlock if there are directories that are
fragmented to several dirfrags. For example:

mds.0                        mds.1
-----------------------------------------------------------------
                             freeze subtree (1.*) with bound (2.1*)
discover (2.0*) ->
                             handle discover (2.0*), frozen tree, wait
                          <- export subtree (1.*) to with bound (2.1*)
discover (2.1*), wait

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: revert commit 15a5d37a
Yan, Zheng [Sun, 12 Jan 2014 12:36:25 +0000 (20:36 +0800)]
mds: revert commit 15a5d37a

commit 15a5d37a (mds: fix race between scatter gather and dirfrag export)
is incomplete, it doesn't handles the race that no fragstat/neststat is
gathered. Previous commit prevents scatter gather during exporting dir,
which eliminates races of this type.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: acquire scatter locks when exporting dir
Yan, Zheng [Sun, 12 Jan 2014 10:51:18 +0000 (18:51 +0800)]
mds: acquire scatter locks when exporting dir

If auth MDS of the subtree root inode is neither the exporter MDS
nor the importer MDS and it gathers subtree root's fragstat/neststat
while the subtree is exporting. It's possible that the exporter MDS
and the importer MDS both are auth MDS of the subtree root or both
are not auth MDS of the subtree root at the time they receive the
lock messages. So the auth MDS of the subtree root inode may get no
or duplicated fragstat/neststat for the subtree root dirfrag.

The fix is, during exporting a subtree, both the exporter MDS and
the importer MDS hold locks on scatter locks of the subtree root
inode. For the importer MDS, it tries acquiring locks on the scatter
locks when handling the MExportDirPrep message. If fails to acquire
all locks, it sends a NACK to the exporter MDS. The exporter MDS
cancels the exporting when receiving the NACK.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: acquire locks required by exporting dir
Yan, Zheng [Sun, 12 Jan 2014 09:31:57 +0000 (17:31 +0800)]
mds: acquire locks required by exporting dir

Start internal MDS request to acquire locks required by exporting dir.
It's more reliable than using Locker::rdlock_take_set(), It also allows
acquiring locks besides rdlock.

Only use Locker::acquire_locks() to acquire locks in the first stage of
exporting dir (before freeze the subtree). After the subtree is frozen,
to minimize the time of frozen tree, still use 'try lock' to re-acquire
the locks.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: introduce nonlocking auth pin
Yan, Zheng [Sun, 12 Jan 2014 07:13:42 +0000 (15:13 +0800)]
mds: introduce nonlocking auth pin

Add a parameter to Locker::acquire_locks() to enabled nonblocking
auth pin. If nonblocking mode is enabled and an object that can't
be auth pinned is encountered, Locker::acquire_locks() aborts the
MDRequest instead of waiting.

The nonlocking mode is acquired by the cases that we want to acquire
locks after auth pinning a directory or freezing a dirfrag/subtree.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: allow acquiring wrlock and remote wrlock at the same time
Yan, Zheng [Fri, 10 Jan 2014 02:58:41 +0000 (10:58 +0800)]
mds: allow acquiring wrlock and remote wrlock at the same time

Rename may move file from one dirfrag to another dirfrag of the same
directory inode. If two dirfrags belong to different auth MDS, both
MDS should hold wrlocks on filelock/nestlock of the directory inode.
If a lock is in both wrlocks list and remote_wrlocks list, current
Locker::acquire_locks() only acquires the local wrlock. The auth MDS
of the source dirfrag doesn't have the wrlock, so slave request of
the operation may modify the dirfrag after fragstat/neststat of the
dirfrag have already been gathered. It corrupts the dirstat/neststat
accounting.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agoMerge pull request #1251 from ceph/wip-7371
Sage Weil [Sun, 16 Feb 2014 16:26:33 +0000 (08:26 -0800)]
Merge pull request #1251 from ceph/wip-7371

ReplicatedPG: return no data if read size is trimmed to zero

Backport: emperor, dumpling
Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoReplicatedPG: return no data if read size is trimmed to zero 1251/head
Yan, Zheng [Sun, 16 Feb 2014 14:14:50 +0000 (22:14 +0800)]
ReplicatedPG: return no data if read size is trimmed to zero

OSD should return no data if the read size is trimmed to zero by the
truncate_seq/truncate_size check. We can't rely on ObjectStore::read()
to do that because it reads the entire object when the 'len' parameter
is zero.

Fixes: #7371
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agoMerge pull request #1223 from ceph/wip-7395
Sage Weil [Sun, 16 Feb 2014 06:36:23 +0000 (22:36 -0800)]
Merge pull request #1223 from ceph/wip-7395

Improve OSD subscription handling

Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1234 from dachary/wip-format
Sage Weil [Sun, 16 Feb 2014 06:21:57 +0000 (22:21 -0800)]
Merge pull request #1234 from dachary/wip-format

mon: remove format argument from osd crush dump

Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1176 from ceph/wip-primary-affinity
Sage Weil [Sun, 16 Feb 2014 00:59:35 +0000 (16:59 -0800)]
Merge pull request #1176 from ceph/wip-primary-affinity

osd: primary affinity

Added primary-affinity thrashing to thrashosd.py.

Reviewed-by: Loic Dachary <loic@dachary.org>
11 years agoMerge pull request #1249 from dachary/wip-qa-erasure-test
Sage Weil [Sun, 16 Feb 2014 00:48:36 +0000 (16:48 -0800)]
Merge pull request #1249 from dachary/wip-qa-erasure-test

qa: do not create erasure pools yet

11 years agoqa: do not create erasure pools yet 1249/head
Loic Dachary [Sat, 15 Feb 2014 23:53:13 +0000 (00:53 +0100)]
qa: do not create erasure pools yet

comment out erasure pool related tests when an OSD is involved because
it does not work yet. See http://tracker.ceph.com/issues/7360.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agoosd/OSDMap: include primary affinity in OSDMap::print 1176/head
Sage Weil [Fri, 14 Feb 2014 23:18:11 +0000 (15:18 -0800)]
osd/OSDMap: include primary affinity in OSDMap::print

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoosd/OSDMap: remove bad assert
Sage Weil [Tue, 11 Feb 2014 23:40:22 +0000 (15:40 -0800)]
osd/OSDMap: remove bad assert

You can have an erasure poool with all CRUSH_ITEM_NONE and primary == -1.
acting is not empty.

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agomon/OSDMonitor: add 'mon osd allow primary affinity' bool option
Sage Weil [Mon, 3 Feb 2014 01:34:25 +0000 (17:34 -0800)]
mon/OSDMonitor: add 'mon osd allow primary affinity' bool option

By default, disallow adjustment of primary affinity unless the user has
opted in by adjusting their monitor config.  This will avoid some user
pain because inadvertantly setting the affinity will prevent older clients
from connecting to and using the cluster.

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoceph_psim: some futzing to test primary_affinity
Sage Weil [Thu, 30 Jan 2014 16:22:58 +0000 (08:22 -0800)]
ceph_psim: some futzing to test primary_affinity

- map to acting
- count first position, primary

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoosd/OSDMap: add primary_affinity feature bit
Sage Weil [Thu, 30 Jan 2014 03:47:21 +0000 (19:47 -0800)]
osd/OSDMap: add primary_affinity feature bit

Indicate that we support it.  Indicate when an OSDMap requires it.

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoosd/OSDMap: apply primary_affinity to mapping
Sage Weil [Tue, 11 Feb 2014 17:25:04 +0000 (09:25 -0800)]
osd/OSDMap: apply primary_affinity to mapping

The behavior is a bit different for replicated and indep/erasure mode.
In the first case, we are rearranging the result.  In the second case,
we can just set the primary argument to the right value.

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1245 from ceph/wip-brag
Sage Weil [Sat, 15 Feb 2014 18:37:26 +0000 (10:37 -0800)]
Merge pull request #1245 from ceph/wip-brag

ceph-brag

Sebastien Han confirms that this is under the default (LGPL2) license, thus:

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoceph.spec: add ceph-brag 1245/head
Sage Weil [Sat, 15 Feb 2014 17:18:18 +0000 (09:18 -0800)]
ceph.spec: add ceph-brag

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agodebian: add ceph-brag
Sage Weil [Sat, 15 Feb 2014 17:18:12 +0000 (09:18 -0800)]
debian: add ceph-brag

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoceph-brag: add Makefile
Sage Weil [Sat, 15 Feb 2014 17:17:33 +0000 (09:17 -0800)]
ceph-brag: add Makefile

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1181 from dachary/wip-7277
Sage Weil [Sat, 15 Feb 2014 18:08:45 +0000 (10:08 -0800)]
Merge pull request #1181 from dachary/wip-7277

DNM: mon: s/ENOSYS/ENOTSUP/

Reviewed-by: Christophe Courtaut <christophe.courtaut@gmail.com>
Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoMerge branch 'master' of https://github.com/enovance/ceph-brag into wip-brag
Sage Weil [Sat, 15 Feb 2014 17:17:22 +0000 (09:17 -0800)]
Merge branch 'master' of https://github.com/enovance/ceph-brag into wip-brag

11 years agoOSD: use the osdmap_subscribe helper 1223/head
Greg Farnum [Tue, 11 Feb 2014 20:51:19 +0000 (12:51 -0800)]
OSD: use the osdmap_subscribe helper

Signed-off-by: Greg Farnum <greg@inktank.com>
11 years agoOSD: create a helper for handling OSDMap subscriptions, and clean them up
Greg Farnum [Tue, 11 Feb 2014 21:34:39 +0000 (13:34 -0800)]
OSD: create a helper for handling OSDMap subscriptions, and clean them up

We've had some trouble with not clearing out subscription requests and
overloading the monitors (though only because of other bugs). Write a
helper for handling subscription requests that we can use to centralize
safety logic. Clear out the subscription whenever we get a map that covers
it; if there are more maps available than we received, we will issue another
subscription request based on "m->newest_map" at the end of handle_osd_map().

Notice that the helper will no longer request old maps which we already have,
and that unless forced it will not dispatch multiple subscribe requests
to a single monitor.
Skipping old maps is safe:
1) we only trim old maps when the monitor tells us to,
2) we do not send messages to our peers until we have updated our maps
from the monitor.
That means only old and broken OSDs will send us messages based on maps
in our past, and we can (and should) ignore any directives from them anyway.

Signed-off-by: Greg Farnum <greg@inktank.com>
11 years agomonc: new fsub_want_increment( function to make handling subscriptions easier
Greg Farnum [Tue, 11 Feb 2014 21:31:26 +0000 (13:31 -0800)]
monc: new fsub_want_increment( function to make handling subscriptions easier

Provide a subscription-modifying function which will not decrement
the start version.

Signed-off-by: Greg Farnum <greg@inktank.com>
11 years agodoc/release-notes: v0.67.6
Sage Weil [Fri, 14 Feb 2014 22:20:43 +0000 (14:20 -0800)]
doc/release-notes: v0.67.6

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1237 from dachary/wip-hashpspool
Sage Weil [Fri, 14 Feb 2014 17:27:33 +0000 (09:27 -0800)]
Merge pull request #1237 from dachary/wip-hashpspool

mon: ceph hashpspool false clears the flag

Reviewed-by: Christophe Courtaut <christophe.courtaut@gmail.com>
Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1235 from ceph/wip-osdmaptool-pool-fix
Loic Dachary [Fri, 14 Feb 2014 12:33:26 +0000 (13:33 +0100)]
Merge pull request #1235 from ceph/wip-osdmaptool-pool-fix

wip-osdmaptool-pool-fix

Reviewed-by: Loic Dachary <loic@dachary.org>
11 years agoosdmaptool: add tests for --pool option 1235/head
Ilya Dryomov [Fri, 14 Feb 2014 10:23:25 +0000 (12:23 +0200)]
osdmaptool: add tests for --pool option

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
11 years agoosdmaptool: add --pool option for --test-map-pgs mode to usage()
Ilya Dryomov [Thu, 13 Feb 2014 17:38:11 +0000 (19:38 +0200)]
osdmaptool: add --pool option for --test-map-pgs mode to usage()

--test-map-pgs mode allows to map all pgs from either all pools or just
one pool.  Mention it in usage output.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
11 years agoosdmaptool: fix --pool option for --test-map-object mode
Ilya Dryomov [Thu, 13 Feb 2014 17:38:10 +0000 (19:38 +0200)]
osdmaptool: fix --pool option for --test-map-object mode

Commit 7f1b12f2ef97 ("osdmaptool: add --test-map-pgs mode") broke
--pool for --test-map-object mode.  Fix it, and improve --pool option
handling for both modes while at it (report strict_strtol() errors,
check if specified pool exists).

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
11 years agotest: fix signed/unsigned warnings in TestCrushWrapper.cc
Greg Farnum [Fri, 14 Feb 2014 01:28:42 +0000 (17:28 -0800)]
test: fix signed/unsigned warnings in TestCrushWrapper.cc

Irritatingly, using 0 binds to int and generates warnings
if the thing we're checking is unsigned, so we have to be
explicit.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sam Just <sam.just@inktank.com>
11 years agomon: ceph hashpspool false clears the flag 1237/head
Loic Dachary [Thu, 13 Feb 2014 17:18:43 +0000 (18:18 +0100)]
mon: ceph hashpspool false clears the flag

instead of toggling it.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: remove format argument from osd crush dump 1234/head
Loic Dachary [Thu, 13 Feb 2014 15:20:51 +0000 (16:20 +0100)]
mon: remove format argument from osd crush dump

The --format argument of the ceph cli is used to send the desired format
argument. The format argument is always part of the command sent to the
server. Adding it to the command description in MonCommand is not
necessary.

partially revert cec1893310005eca29186145de66391789211e70
revert fce4d6840448053b685ca3407b2a2680109d6df2

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agoMerge pull request #1231 from dachary/wip-mon-create-simple
Sage Weil [Thu, 13 Feb 2014 15:00:18 +0000 (07:00 -0800)]
Merge pull request #1231 from dachary/wip-mon-create-simple

mon: do not goto reply if a ruleset exists in pending

Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1216 from ceph/wip-null-xattr
Sage Weil [Thu, 13 Feb 2014 14:59:15 +0000 (06:59 -0800)]
Merge pull request #1216 from ceph/wip-null-xattr

mds: remove xattr when null value is given to setxattr()

Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoMerge branch 'wip-libcephfs-firefly-rb' of https://github.com/linuxbox2/linuxbox...
Sage Weil [Thu, 13 Feb 2014 14:51:37 +0000 (06:51 -0800)]
Merge branch 'wip-libcephfs-firefly-rb' of https://github.com/linuxbox2/linuxbox-ceph

Reviewed-by: Sage Weil <sage@inktank.com>
This went through the fs suite and passed:

http://pulpito.ceph.com/sage-2014-02-12_13:38:53-fs-wip-libcephfs-testing-basic-plana

11 years agomon: do not goto reply if a ruleset exists in pending 1231/head
Loic Dachary [Thu, 13 Feb 2014 14:40:30 +0000 (15:40 +0100)]
mon: do not goto reply if a ruleset exists in pending

If the crush ruleset is found in pending, do not goto reply because it
does not exist yet. Wait for the pending proposal (and the ruleset) to
be accepted and then only return that it exists.

revert 4b687ba673574bcd3b77b0f79a07d5c74cdcedc2

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agoMerge pull request #1202 from dachary/wip-mon-create-simple
Loic Dachary [Thu, 13 Feb 2014 14:06:27 +0000 (15:06 +0100)]
Merge pull request #1202 from dachary/wip-mon-create-simple

mon: create simple should goto reply when it exists in pending

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
11 years agomon: osd crush rule create-simple functional tests 1202/head
Loic Dachary [Thu, 13 Feb 2014 12:54:49 +0000 (13:54 +0100)]
mon: osd crush rule create-simple functional tests

Basic tests and a test that create the conditions where an OSDMap
is pending with a ruleset that is not yet in the OSDMap. An attempt to
create a rule by the same name will return success and not create it again.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: osd crush rule functional tests
Loic Dachary [Sun, 9 Feb 2014 21:44:25 +0000 (22:44 +0100)]
mon: osd crush rule functional tests

* A set of test for the simplest operations
* A test covering all cases of osd crush rule

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agoMerge pull request #1230 from ceph/wip-open-truncate
Sage Weil [Thu, 13 Feb 2014 13:00:17 +0000 (05:00 -0800)]
Merge pull request #1230 from ceph/wip-open-truncate

mds: properly replay dentry trace for open truncate.

Reviewed-by: Sage Weil <sage@inktank.com>
11 years agomon: add ruleset name to crush rule dump
Loic Dachary [Tue, 11 Feb 2014 17:25:51 +0000 (18:25 +0100)]
mon: add ruleset name to crush rule dump

The crush rule name is made an optional first argument. If not provided, it
defaults to dumping all rules.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: add missing format argument to crush dump
Loic Dachary [Sun, 9 Feb 2014 19:31:44 +0000 (20:31 +0100)]
mon: add missing format argument to crush dump

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: add missing format argument to crush rule dump
Loic Dachary [Sun, 9 Feb 2014 19:30:33 +0000 (20:30 +0100)]
mon: add missing format argument to crush rule dump

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: osd create pool test refactor
Loic Dachary [Thu, 13 Feb 2014 09:23:28 +0000 (10:23 +0100)]
mon: osd create pool test refactor

* Use test-helpers.sh instead of duplicating it
* Isolate each test in a separate function for clarity

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: minor improvement to test-helpers.sh
Loic Dachary [Sun, 9 Feb 2014 17:26:57 +0000 (18:26 +0100)]
mon: minor improvement to test-helpers.sh

* kill -9 daemons to better cope with stuck daemons
* reduce paxos propose interval to speedup tests
* look for erasure code plugins from the sources directory

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: if crush rule create-simple exists return
Loic Dachary [Sun, 9 Feb 2014 11:31:30 +0000 (12:31 +0100)]
mon: if crush rule create-simple exists return

And it is consistent with the above. Also use a slightly different
string to allow the caller to differentiate between the two cases.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agocrush: display item name in CrushWrapper::dump_rule()
Loic Dachary [Sun, 9 Feb 2014 20:40:08 +0000 (21:40 +0100)]
crush: display item name in CrushWrapper::dump_rule()

When an item name is available used it instead of displaying the number.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agocrush: cosmetic changes to CrushWrapper::dump_rule()
Loic Dachary [Sun, 9 Feb 2014 20:21:23 +0000 (21:21 +0100)]
crush: cosmetic changes to CrushWrapper::dump_rule()

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agocrush: add the dump_rule(int ruleset) method
Loic Dachary [Sun, 9 Feb 2014 20:18:45 +0000 (21:18 +0100)]
crush: add the dump_rule(int ruleset) method

And implement dump_rules() using dump_rule(). The indentiation and
variable names are intentionaly left as is to not confuse code being
moved around and the code changes.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agocrush: add CrushWrapper::dump_rules() unit test
Loic Dachary [Sun, 9 Feb 2014 20:17:31 +0000 (21:17 +0100)]
crush: add CrushWrapper::dump_rules() unit test

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agocrush: fix leaks in TestCrushWrapper.cc
Loic Dachary [Sun, 9 Feb 2014 20:11:09 +0000 (21:11 +0100)]
crush: fix leaks in TestCrushWrapper.cc

To get a meaningfull valgrind output.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agoMerge pull request #1222 from dachary/wip-pending-osdmap
Loic Dachary [Thu, 13 Feb 2014 09:16:40 +0000 (10:16 +0100)]
Merge pull request #1222 from dachary/wip-pending-osdmap

mon: control pending OSDMap for testing

Reviewed-by: Joao Eduardo Luis <joao@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
11 years agomon: remove racy tests from osd-pool-create.sh 1222/head
Loic Dachary [Wed, 12 Feb 2014 15:58:23 +0000 (16:58 +0100)]
mon: remove racy tests from osd-pool-create.sh

Use the admin socket to create the conditions by which a pool creation
is made to wait for the next paxos proposal because the required crush
ruleset is pending.

It replaces a fragile time sensitive workaround that could fail because
of race conditions. It also has the benefit of increase the speed of the
test because there is no need to wait for a long time just to accomodate
the slowest machines.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: add osdmonitor_prepare_command to the admin socket
Loic Dachary [Wed, 12 Feb 2014 15:49:32 +0000 (16:49 +0100)]
mon: add osdmonitor_prepare_command to the admin socket

It provides a developer path allowing functional tests to modify the
pending OSDMap without triggering a PaxosProposal.

It can be used as follows:

   echo '{"prefix":"osdmonitor_prepare_command","prepare":"osd crush tunables","profile":"bobtail"}' | nc -U out/mon.a.asok

It will transform the command into:

  {"prefix":"osd crush tunables","profile":"bobtail"}

and feed it to OSDMonitor::prepare_command_impl(). The pending OSDMap won't
be proposed because it short circuit PaxosService::dispatch. It will,
however, be proposed next time PaxosService::dispatch() gets a chance.

It cannot be used via the ceph command line.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: add --mon-advanced-debug-mode
Loic Dachary [Wed, 12 Feb 2014 17:49:50 +0000 (18:49 +0100)]
mon: add --mon-advanced-debug-mode

When this flag is true, the mon is expected to provide functionalities
that are for developer oriented debug purposes only. It is meant to
be used by the developer and not the system administrator, because it
would allow a non-developer to break things in ways that would be very
difficult to diagnose. It should probably not be documented.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: Monitor::send_reply gracefully handles no connection
Loic Dachary [Wed, 12 Feb 2014 15:48:04 +0000 (16:48 +0100)]
mon: Monitor::send_reply gracefully handles no connection

If req->get_connection() returns a null pointer, drop the reply instead
of crashing the monitor.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomon: split prepare_command in two
Loic Dachary [Wed, 12 Feb 2014 15:45:40 +0000 (16:45 +0100)]
mon: split prepare_command in two

So that it is possible to call prepare_command without a session
established and a cmdmap that has already been parsed.

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agomds: remove xattr when null value is given to setxattr() 1216/head
Yan, Zheng [Tue, 11 Feb 2014 05:32:23 +0000 (13:32 +0800)]
mds: remove xattr when null value is given to setxattr()

setxattr() should remove xattr if parameter 'value' is NULL. To
distinguish it from the zero-length value case, introduce a new
flag CEPH_XATTR_REMOVE for the setxattr request.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agomds: properly replay dentry trace for open truncate. 1230/head
Yan, Zheng [Thu, 13 Feb 2014 02:28:49 +0000 (10:28 +0800)]
mds: properly replay dentry trace for open truncate.

Open request created by ceph_atomic_open() wants dentry trace. But
the corresponding code is missing in Server::do_open_truncate().

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
11 years agodoc/release-notes: do not downgrade from v0.67.6
Sage Weil [Thu, 13 Feb 2014 02:02:58 +0000 (18:02 -0800)]
doc/release-notes: do not downgrade from v0.67.6

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agodoc/release-notes: note about dumpling xattr backport
Sage Weil [Thu, 13 Feb 2014 01:55:50 +0000 (17:55 -0800)]
doc/release-notes: note about dumpling xattr backport

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoPendingReleaseNotes: note about cephfs backtrace updates
Sage Weil [Thu, 13 Feb 2014 01:42:16 +0000 (17:42 -0800)]
PendingReleaseNotes: note about cephfs backtrace updates

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1219 from ceph/wip-update-backtrace
Sage Weil [Thu, 13 Feb 2014 01:37:07 +0000 (17:37 -0800)]
Merge pull request #1219 from ceph/wip-update-backtrace

mds: force update backtraces for previously created FS

Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoosd/OSDMap: pass pps value out from _pg_to_osds
Sage Weil [Tue, 11 Feb 2014 17:31:20 +0000 (09:31 -0800)]
osd/OSDMap: pass pps value out from _pg_to_osds

The primary affinity code needs a seed to determine it's pseudo-random
behavior.  Pass pps (optionally) out of _pg_to_osds() for this purpose.

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoosd/OSDMap: fix _raw_to_up_osds for EC pools
Sage Weil [Tue, 11 Feb 2014 17:24:29 +0000 (09:24 -0800)]
osd/OSDMap: fix _raw_to_up_osds for EC pools

When filtering down OSDs out of the result set (i.e., raw -> up), we can
shift positions for replicated pools but need to preserve position for
EC pools.

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agomon/OSDMonitor: add 'osd primary-affinity ...' command
Sage Weil [Mon, 23 Dec 2013 19:30:13 +0000 (11:30 -0800)]
mon/OSDMonitor: add 'osd primary-affinity ...' command

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoosd/OSDMap: add osd_primary_affinity fields, accessors, encoding
Sage Weil [Tue, 11 Feb 2014 17:27:32 +0000 (09:27 -0800)]
osd/OSDMap: add osd_primary_affinity fields, accessors, encoding

Signed-off-by: Sage Weil <sage@inktank.com>
11 years agomon/OSDMonitor: fix legacy tunables warning
Sage Weil [Wed, 12 Feb 2014 21:18:04 +0000 (13:18 -0800)]
mon/OSDMonitor: fix legacy tunables warning

Warn on legacy tunables, not on non-optimal tunables.  Optimal is a moving
target, but it is really the legacy defaults that we want to push people
off of.

Fixes: #7399
Signed-off-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1224 from kdreyer-inktank/packaging-libdir
Sage Weil [Wed, 12 Feb 2014 18:35:07 +0000 (10:35 -0800)]
Merge pull request #1224 from kdreyer-inktank/packaging-libdir

packaging: do not package libdir/ceph recursively

11 years agopackaging: do not package libdir/ceph recursively 1224/head
Alexandre Oliva [Wed, 12 Feb 2014 17:46:50 +0000 (15:46 -0200)]
packaging: do not package libdir/ceph recursively

Package libdir/ceph non-recursively, to avoid duplicates, and
package libdir/ceph/ceph_common.sh explicitly.

Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
11 years agoMerge pull request #1215 from ceph/wip-7385
Sage Weil [Wed, 12 Feb 2014 17:54:30 +0000 (09:54 -0800)]
Merge pull request #1215 from ceph/wip-7385

Remove the max cached objects restriction for librbd

Reviewed-by: Sage Weil <sage@inktank.com>
11 years agoMerge pull request #1199 from ceph/wip-doc-librados-intro
Loic Dachary [Wed, 12 Feb 2014 16:25:42 +0000 (17:25 +0100)]
Merge pull request #1199 from ceph/wip-doc-librados-intro

Wip doc librados intro

Reviewed-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
11 years agoMerge pull request #1221 from dachary/wip-filestore
Loic Dachary [Wed, 12 Feb 2014 11:52:02 +0000 (12:52 +0100)]
Merge pull request #1221 from dachary/wip-filestore

tests: fix packaging for s/filestore/objectstore/

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
11 years agotests: fix packaging for s/filestore/objectstore/ 1221/head
Loic Dachary [Wed, 12 Feb 2014 11:44:21 +0000 (12:44 +0100)]
tests: fix packaging for s/filestore/objectstore/

The binaries file name have changed and need to be updated in the
packaging files for deb and rpm. Fix a few leftovers as well.

Fixing 1a588f18ba0e57df64f8a48c1393a4bc65019571

Signed-off-by: Loic Dachary <loic@dachary.org>
11 years agoMerge pull request #1220 from dachary/wip-filestore
Kri5 [Wed, 12 Feb 2014 11:26:28 +0000 (12:26 +0100)]
Merge pull request #1220 from dachary/wip-filestore

tests: fix objectstore tests

11 years agotests: fix objectstore tests 1220/head
Loic Dachary [Wed, 12 Feb 2014 10:52:37 +0000 (11:52 +0100)]
tests: fix objectstore tests

The objectstore test from 1a588f18ba0e57df64f8a48c1393a4bc65019571 was
missing a few changes.

Signed-off-by: Loic Dachary <loic@dachary.org>