]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
10 years agoRevert "Enforce cache size on read requests" 2797/head
Sage Weil [Fri, 24 Oct 2014 18:06:16 +0000 (11:06 -0700)]
Revert "Enforce cache size on read requests"

This reverts commit 4fc9fffc494abedac0a9b1ce44706343f18466f1.

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoRevert "rbd: ObjectCacher reads can hang when reading sparse files"
Sage Weil [Fri, 24 Oct 2014 18:06:08 +0000 (11:06 -0700)]
Revert "rbd: ObjectCacher reads can hang when reading sparse files"

This reverts commit cdb7675a21c9107e3596c90c2b1598def3c6899f.

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoRevert "Fix read performance regression in ObjectCacher"
Sage Weil [Fri, 24 Oct 2014 18:05:53 +0000 (11:05 -0700)]
Revert "Fix read performance regression in ObjectCacher"

This reverts commit 65be257e9295619b960b49f6aa80ecdf8ea4d16a.

Too late for giant.

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2795 from ceph/wip-9873
David Zafman [Fri, 24 Oct 2014 17:49:32 +0000 (10:49 -0700)]
Merge pull request #2795 from ceph/wip-9873

objecter: fix tick_event shutdown race (9873)

Reviewed-by: David Zafman <dzafman@redhat.com>
10 years agoosdc/Objecter: fix tick_event handling in shutdown vs tick race 2795/head
Sage Weil [Fri, 24 Oct 2014 16:32:20 +0000 (09:32 -0700)]
osdc/Objecter: fix tick_event handling in shutdown vs tick race

If we fail to cancel the tick_event, we rely on tick() itself to clear
tick_event.  I'm not quite sure how we got this wrong in the previous
commit, but this boils down to two cases:

1) shutdown() successfully cancels the event and clears tick_event.  tick()
   never runs.  tick_event == NULL when we finish.
2) shutdown() fails to cancel the event because it has already started.  In
   this case tick itself is blocking (or about to block) waiting on the
   rlock.  When it does run it will clear tick_event itself, then see
   initiazed == 0 and exit without rescheduling.

Fixes: #9873
Signed-off-by: Sage Weil <sage@redhat.com>
10 years agocommon/Timer: recheck stopping before sleep if we dropped the lock
Sage Weil [Fri, 24 Oct 2014 16:20:41 +0000 (09:20 -0700)]
common/Timer: recheck stopping before sleep if we dropped the lock

If we have safe_callbacks==false, the stopping flag may have changed while
we were doing our callback. Recheck it and exit to avoid a deadlock on
shutdown.

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2787 from ceph/fix-fstat-mode
Sage Weil [Fri, 24 Oct 2014 00:57:02 +0000 (17:57 -0700)]
Merge pull request #2787 from ceph/fix-fstat-mode

java: fill in stat structure correctly

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agojava: fill in stat structure correctly 2787/head
Noah Watkins [Thu, 23 Oct 2014 20:22:52 +0000 (13:22 -0700)]
java: fill in stat structure correctly

Added stat filling helper function but only stat and lstat were updated.
This patch makes fstat use it. Crucially the fstat wasn't updating the
mode flags.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
10 years agoMerge pull request #2785 from athanatos/wip-9821
Sage Weil [Thu, 23 Oct 2014 20:45:26 +0000 (13:45 -0700)]
Merge pull request #2785 from athanatos/wip-9821

PG:: reset_interval_flush and in set_last_peering_reset

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2766 from dachary/wip-9408-buffer-alignment-giant
Samuel Just [Thu, 23 Oct 2014 16:52:00 +0000 (09:52 -0700)]
Merge pull request #2766 from dachary/wip-9408-buffer-alignment-giant

erasure-code: buffer alignment (giant)

Reviewed-by: Samuel Just <sam.just@inktank.com>
10 years agoqa: use sudo when rsyncing /usr so we can read everything
Greg Farnum [Tue, 21 Oct 2014 17:55:06 +0000 (10:55 -0700)]
qa: use sudo when rsyncing /usr so we can read everything

Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit fa07c04231db2d130de54647957ffab4a7a53733)

10 years agoMerge pull request #2777 from ceph/wip-9859
Sage Weil [Wed, 22 Oct 2014 18:36:07 +0000 (11:36 -0700)]
Merge pull request #2777 from ceph/wip-9859

mon: Monitor: MMonGetMap doesn't require caps

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agomon: Monitor: MMonGetMap doesn't require caps 2777/head
Joao Eduardo Luis [Wed, 22 Oct 2014 18:30:08 +0000 (19:30 +0100)]
mon: Monitor: MMonGetMap doesn't require caps

We are dropping the requirement for MON_CAP_R for MMonGetMap.

Reason is simple enough: clients may need to contact the monitors and
obtain the latest monmap before authenticating.  This happens, for
instance, when a client calls MonClient::get_monmap_privately().  The
osd uses this function during mkfs, prior to initializing a keyring or
even so much as existing.

Fixes: #9859
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agoPG:: reset_interval_flush and in set_last_peering_reset 2785/head
Samuel Just [Mon, 20 Oct 2014 21:10:58 +0000 (14:10 -0700)]
PG:: reset_interval_flush and in set_last_peering_reset

If we have a change in the prior set, but not in the up/acting set, we go back
through Reset in order to reset peering state.  Previously, we would reset
last_peering_reset in the Reset constructor.  This did not, however, reset the
flush_interval, which caused the eventual flush event to be ignored and the
peering messages to not be sent.

Instead, we will always reset_interval_flush if we are actually changing the
last_peering_reset value.

Fixes: #9821
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
10 years agoobjecter: Unlock in shutdown before waiting for timer thread
David Zafman [Tue, 21 Oct 2014 07:52:37 +0000 (00:52 -0700)]
objecter: Unlock in shutdown before waiting for timer thread

Fixes: #9845
Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>
10 years agoerasure-code: use ErasureCode::SIMD_ALIGN in ceph_erasure_code_benchmark 2766/head
Loic Dachary [Mon, 13 Oct 2014 14:43:20 +0000 (16:43 +0200)]
erasure-code: use ErasureCode::SIMD_ALIGN in ceph_erasure_code_benchmark

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
10 years agoerasure-code: add ErasureCode::encode unit test
Loic Dachary [Mon, 13 Oct 2014 12:48:27 +0000 (14:48 +0200)]
erasure-code: add ErasureCode::encode unit test

Re-create and describe the situation that is fixed by
91a7e18f60bbc9acab3045baaa1b6505474ec4a9 which reworks the buffer
preparation function provided by ErasureCode::encode.

http://tracker.ceph.com/issues/9408 Refs: #9408

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
10 years agoerasure-code: expose ErasureCode::SIMD_ALIGN as a const
Loic Dachary [Mon, 13 Oct 2014 12:46:22 +0000 (14:46 +0200)]
erasure-code: expose ErasureCode::SIMD_ALIGN as a const

For test purposes and it will also be useful for plugins that must
ensure the chunk size is a multiple of SIMD_ALIGN.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
10 years agoceph_erasure_code_benchmark: use 32-byte aligned input
Janne Grunau [Mon, 29 Sep 2014 12:34:32 +0000 (14:34 +0200)]
ceph_erasure_code_benchmark: use 32-byte aligned input

The benchmark is supposed to measure the encoding/decoding speed and
not the overhead of buffer realignments.

Signed-off-by: Janne Grunau <j@jannau.net>
10 years agoerasure code: use 32-byte aligned buffers
Janne Grunau [Mon, 29 Sep 2014 12:34:31 +0000 (14:34 +0200)]
erasure code: use 32-byte aligned buffers

Requiring page aligned buffers and realigning the input if necessary
creates measurable oberhead. ceph_erasure_code_benchmark is between
10-20% faster depending on the workload.

Also prevents a misaligned buffer when bufferlist::c_str(bufferlist)
has to allocate a new buffer to provide continuous one. See bug #9408

Signed-off-by: Janne Grunau <j@jannau.net>
10 years agoerasure code: use a function for the chunk mapping index
Janne Grunau [Mon, 29 Sep 2014 12:34:30 +0000 (14:34 +0200)]
erasure code: use a function for the chunk mapping index

10 years agocommon: add an aligned buffer with less alignment than a page
Loic Dachary [Mon, 13 Oct 2014 14:32:18 +0000 (16:32 +0200)]
common: add an aligned buffer with less alignment than a page

SIMD optimized erasure code computation needs aligned memory. Buffers
aligned to a page boundary are not needed though. The buffers used
for the erasure code computation are typical smaller than a page.

The typical alignment requirements SIMD operations are 16 bytes for
SSE2 and NEON and 32 bytes for AVX/AVX2.

Add new prototypes with an align argument, similar to the one enforcing
page alignment. The implementation is exactly the same, except for the
align parameter. The page alignment method are then implemented as calls
to the more generic methods.

The align parameter is an unsigned (same type as CEPH_PAGE_SIZE). The
CEPH_PAGE_MASK value ( ~(CEPH_PAGE_SIZE - 1) ) was only used as
~CEPH_PAGE_MASK, i.e. equivalent of (CEPH_PAGE_SIZE - 1) once the double
~~ is reduced. These occurrence are replaced with (align - 1). The type
of CEPH_PAGE_MASK is an unsigned long which probably because it was
~(CEPH_PAGE_SIZE). When using (align - 1) as a mask for both
CEPH_PAGE_SIZE and SIMD alignment there is no need to use an unsigned
long because there is no risk of overflowing the unsigned value.

The CYGWIN specific code is also modified but not tested.

Unit tests are added for the new methods.

Signed-off-by: Janne Grunau <j@jannau.net>
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
10 years agocommon: remove dead code in buffer.cc
Loic Dachary [Mon, 13 Oct 2014 14:29:10 +0000 (16:29 +0200)]
common: remove dead code in buffer.cc

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
10 years agoFix read performance regression in ObjectCacher
Adam Crume [Wed, 8 Oct 2014 00:45:53 +0000 (17:45 -0700)]
Fix read performance regression in ObjectCacher

The regression was introduced in commit
4fc9fffc494abedac0a9b1ce44706343f18466f1.  The problem is that the cache
thinks it's full (when it's not), so it defers the read.  This change
frees up cache space if necessary and only defers the read if enough
space cannot be freed.

Fixes: 9513
Signed-off-by: Adam Crume <adamcrume@gmail.com>
(cherry picked from commit 82175ec94acc89dc75da0154f86187fb2e4dbf5e)

10 years agoMerge pull request #2758 from ceph/wip-9820
Sage Weil [Mon, 20 Oct 2014 17:46:48 +0000 (10:46 -0700)]
Merge pull request #2758 from ceph/wip-9820

qa/workunits: cephtool: don't remove self's key on auth tests

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agoqa/workunits: cephtool: don't remove self's key on auth tests 2758/head
Joao Eduardo Luis [Mon, 20 Oct 2014 17:00:15 +0000 (18:00 +0100)]
qa/workunits: cephtool: don't remove self's key on auth tests

Suites run with CEPH_TEST_CLI_DUP_COMMAND=1, which will send a duplicate
command for every command issued with the 'ceph' tool.  Behavior is to
get a reply from the command and then send a duplicate, looking for the
same outcome (guaranteeing idempotency of the operations).  However, it
so happens that if you remove the entity's own key from the keyring and
you happen to be unlucky enough so that the client's connection gets
failed (we also run tests with connection failure injections), the
'ceph' tool won't be able to reconnect to the cluster to send the
duplicate command (as it's entity no longer exists in the cluster's
keyring).

We rewrite the test instead of resorting to ugly hacks to work around
this behavior, simply having a new 'role-definer' added by the existing
'role-definer' (which we weren't testing anyway, so bonus points for
that) and then have one removing the other (to test the procedure) and
finally using 'client.admin' to remove the last 'role-definer'.

Fixes: #9820
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agoMerge pull request #2708 from ceph/wip-9718
Samuel Just [Fri, 17 Oct 2014 17:38:43 +0000 (10:38 -0700)]
Merge pull request #2708 from ceph/wip-9718

osd/osd_types: consider CRUSH_ITEM_NONE in check_new_interval() min_size

Reviewed-by: Samuel Just <sam.just@inktank.com>
10 years agoMerge pull request #2711 from guangyy/wip-9614-followup
Samuel Just [Fri, 17 Oct 2014 17:38:30 +0000 (10:38 -0700)]
Merge pull request #2711 from guangyy/wip-9614-followup

Follow-up fix for 9614

Reviewed-by: Samuel Just <sam.just@inktank.com>
10 years agoMerge pull request #2740 from ceph/giant-unknown-locktype
Sage Weil [Fri, 17 Oct 2014 15:21:16 +0000 (08:21 -0700)]
Merge pull request #2740 from ceph/giant-unknown-locktype

mds: reply -EOPNOTSUPP for unknown lock type

10 years agomds: reply -EOPNOTSUPP for unknown lock type 2740/head
Yan, Zheng [Tue, 14 Oct 2014 14:02:41 +0000 (22:02 +0800)]
mds: reply -EOPNOTSUPP for unknown lock type

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 675392335c53ff7879031fb9184e4f35bcc90fe2)

10 years agoqa/workunits/rbd/import_export.sh: be case insensitive
Sage Weil [Wed, 15 Oct 2014 19:26:00 +0000 (12:26 -0700)]
qa/workunits/rbd/import_export.sh: be case insensitive

Stop tripping over this change (from dumpling).

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2702 from ceph/wip-9706
Yehuda Sadeh [Wed, 15 Oct 2014 15:58:16 +0000 (08:58 -0700)]
Merge pull request #2702 from ceph/wip-9706

objecter: fix session locking, use after frees (#9706)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
10 years agoMerge pull request #2719 from ceph/wip-inotable-init
Gregory Farnum [Tue, 14 Oct 2014 20:38:50 +0000 (13:38 -0700)]
Merge pull request #2719 from ceph/wip-inotable-init

mds: fix inotable initialization/reset

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
10 years agomds: fix inotable initialization/reset 2719/head
Henry C Chang [Tue, 14 Oct 2014 02:06:04 +0000 (10:06 +0800)]
mds: fix inotable initialization/reset

interval_set::insert takes arguments start and len, not end.

Signed-off-by: Henry C Chang <henry.cy.chang@gmail.com>
(cherry picked from commit c95bb5943450be95e4302e35b3e2df68a6fc34bd)

10 years agoMerge pull request #2707 from ceph/wip-9731
Sage Weil [Mon, 13 Oct 2014 18:17:17 +0000 (11:17 -0700)]
Merge pull request #2707 from ceph/wip-9731

PGLog::IndexedLog::trim(): rollback_info_trimmed_to_riter may be log.ren...

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agorpm: 95-ceph-osd-alt.rules is not needed for centos7 / rhel7
Loic Dachary [Sat, 11 Oct 2014 16:20:36 +0000 (18:20 +0200)]
rpm: 95-ceph-osd-alt.rules is not needed for centos7 / rhel7

The || instead of && had it always installed. That was fixed in EPEL
already.

http://tracker.ceph.com/issues/9747 Fixes: #9747

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 5ff4a850a0d809b3f25988c6cceb82c35095ef84)

10 years agoThe fix for issue 9614 was not completed, as a result, for those erasure coded PGs... 2711/head
Guang Yang [Mon, 13 Oct 2014 04:18:45 +0000 (04:18 +0000)]
The fix for issue 9614 was not completed, as a result, for those erasure coded PGs with one OSD down, the state was wrongly marked as active+clean+degraded. This patch makes sure the clean flag is not set for such PG.
Signed-off-by: Guang Yang <yguang@yahoo-inc.com>
10 years agoosd/osd_types: consider CRUSH_ITEM_NONE in check_new_interval() min_size check 2708/head
Sage Weil [Sun, 12 Oct 2014 17:05:51 +0000 (10:05 -0700)]
osd/osd_types: consider CRUSH_ITEM_NONE in check_new_interval() min_size check

Fixes: #9718
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoosdc/Objecter: fix use-after-frees in close_session, shutdown 2702/head
Sage Weil [Fri, 10 Oct 2014 23:48:14 +0000 (16:48 -0700)]
osdc/Objecter: fix use-after-frees in close_session, shutdown

For,  linger ops, _session_linger_op_remove invalidates our iterator; add
it to the list first.  Same goes for the others.

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoosdc/Objecter: fix tick() session locking
Sage Weil [Fri, 10 Oct 2014 23:36:40 +0000 (16:36 -0700)]
osdc/Objecter: fix tick() session locking

We need to take the session read lock before traversing the ops lists.

Fixes: #9706
Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoPGLog::IndexedLog::trim(): rollback_info_trimmed_to_riter may be log.rend() 2707/head
Samuel Just [Fri, 10 Oct 2014 20:53:29 +0000 (13:53 -0700)]
PGLog::IndexedLog::trim(): rollback_info_trimmed_to_riter may be log.rend()

Fixes: #9731
Backport: giant, firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
10 years agoMerge pull request #2684 from ceph/wip-9696
Samuel Just [Fri, 10 Oct 2014 17:32:15 +0000 (10:32 -0700)]
Merge pull request #2684 from ceph/wip-9696

PG::choose_acting: in mixed cluster case, acting may include backfill

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2693 from ceph/giant-unused-variable
Gregory Farnum [Fri, 10 Oct 2014 13:59:22 +0000 (06:59 -0700)]
Merge pull request #2693 from ceph/giant-unused-variable

Giant unused variable

10 years agomds: Locker: remove unused variable 2693/head
Yan, Zheng [Fri, 10 Oct 2014 13:36:39 +0000 (21:36 +0800)]
mds: Locker: remove unused variable

Signed-off-by: Yan, Zheng <zyan@redhat.com>
10 years agoMerge pull request #2679 from ceph/giant-locker-null
Yan, Zheng [Fri, 10 Oct 2014 01:37:28 +0000 (09:37 +0800)]
Merge pull request #2679 from ceph/giant-locker-null

mds: Locker: fix a NULL deref in _update_cap_fields

10 years agoPG::choose_acting: in mixed cluster case, acting may include backfill 2684/head
Samuel Just [Thu, 9 Oct 2014 23:21:18 +0000 (16:21 -0700)]
PG::choose_acting: in mixed cluster case, acting may include backfill

Fixes: 9696
Backport: firefly, giant
Introduced: 92cfd370395385ca5537b5bc72220934c9f09026
Signed-off-by: Samuel Just <sam.just@inktank.com>
10 years agomds: Locker: fix a NULL deref in _update_cap_fields 2679/head
Greg Farnum [Thu, 9 Oct 2014 22:12:19 +0000 (15:12 -0700)]
mds: Locker: fix a NULL deref in _update_cap_fields

The MClientCaps* is allowed to be NULL, so we can't deref it unless
the dirty param is non-zero. So don't do the ahead-of-time lookup;
just call it explicitly in the if block.

Signed-off-by: Greg Farnum <greg@inktank.com>
10 years agoMerge pull request #2616 from guangyy/wip-giant-9614
Samuel Just [Wed, 8 Oct 2014 18:08:35 +0000 (11:08 -0700)]
Merge pull request #2616 from guangyy/wip-giant-9614

PG::actingset should be used when checking the number of acting OSDs for a given PG.

Backport: firefly
Reviewed-by: Samuel Just <sam.just@inktank.com>
10 years agoMerge pull request #2663 from ceph/wip-9496
Samuel Just [Wed, 8 Oct 2014 18:07:06 +0000 (11:07 -0700)]
Merge pull request #2663 from ceph/wip-9496

mon: PGMonitor: populate scrub timestamps with 'now' on pg creation

Reviewed-by: Samuel Just <sam.just@inktank.com>
10 years agoMerge pull request #2650 from ceph/wip-9128
Samuel Just [Wed, 8 Oct 2014 18:04:42 +0000 (11:04 -0700)]
Merge pull request #2650 from ceph/wip-9128

Add reset_tp_timeout in long loop in add_source_info for suicide timeout

Reviewed-by: Samuel Just <sam.just@inktank.com>
10 years agoMerge pull request #2543 from ceph/wip-9419
Samuel Just [Wed, 8 Oct 2014 18:01:26 +0000 (11:01 -0700)]
Merge pull request #2543 from ceph/wip-9419

Wip 9419

Reviewed-by: Samuel Just <sam.just@inktank.com>
10 years agoMerge pull request #2661 from dachary/wip-9677-ioprio-class-giant
Loic Dachary [Wed, 8 Oct 2014 06:44:08 +0000 (08:44 +0200)]
Merge pull request #2661 from dachary/wip-9677-ioprio-class-giant

common: ceph_ioprio_string_to_class always returns -EINVAL

10 years agomon: PGMonitor: populate scrub timestamps with 'now' on pg creation 2663/head
Joao Eduardo Luis [Tue, 7 Oct 2014 23:13:49 +0000 (00:13 +0100)]
mon: PGMonitor: populate scrub timestamps with 'now' on pg creation

Fixes: #9496
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agomon: PGMonitor: prettify access to pg_stats_t in register_pg
Joao Eduardo Luis [Tue, 7 Oct 2014 23:12:29 +0000 (00:12 +0100)]
mon: PGMonitor: prettify access to pg_stats_t in register_pg

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agoMerge pull request #2660 from athanatos/wip-9203
Sage Weil [Tue, 7 Oct 2014 21:08:39 +0000 (14:08 -0700)]
Merge pull request #2660 from athanatos/wip-9203

test/osd/Object: don't generate length of 0

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2659 from athanatos/wip-9113
Sage Weil [Tue, 7 Oct 2014 21:06:38 +0000 (14:06 -0700)]
Merge pull request #2659 from athanatos/wip-9113

Wip 9113

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2658 from athanatos/wip-9625
Sage Weil [Tue, 7 Oct 2014 21:03:55 +0000 (14:03 -0700)]
Merge pull request #2658 from athanatos/wip-9625

PG: release backfill reservations if a backfill peer rejects

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agoosd: Return EOPNOTSUPP if a set-alloc-hint occurs with OSDs that don't support 2543/head
David Zafman [Wed, 24 Sep 2014 23:02:21 +0000 (16:02 -0700)]
osd: Return EOPNOTSUPP if a set-alloc-hint occurs with OSDs that don't support

Add CEPH_FEATURE_OSD_SET_ALLOC_HINT feature bit
Collect the intersection of all peer feature bits during peering
When handling CEPH_OSD_OP_SETALLOCHINT check that all OSDs support it
by checking for CEPH_FEATURE_OSD_SET_ALLOC_HINT feature bit.

Fixes: #9419
Backport: firefly

Signed-off-by: David Zafman <dzafman@redhat.com>
10 years agoosd: Remove unused PG functions queue_notify(), queue_info(), queue_log()
David Zafman [Fri, 19 Sep 2014 22:12:55 +0000 (15:12 -0700)]
osd: Remove unused PG functions queue_notify(), queue_info(), queue_log()

Signed-off-by: David Zafman <dzafman@redhat.com>
10 years agoMerge tag 'v0.86' into giant
Sage Weil [Tue, 7 Oct 2014 19:34:10 +0000 (12:34 -0700)]
Merge tag 'v0.86' into giant

v0.86

10 years ago0.86
Jenkins [Tue, 7 Oct 2014 13:20:21 +0000 (06:20 -0700)]
0.86

10 years agoMerge remote-tracking branch 'guangyy/wip-9008' into giant
Samuel Just [Tue, 7 Oct 2014 18:42:14 +0000 (11:42 -0700)]
Merge remote-tracking branch 'guangyy/wip-9008' into giant

10 years agoMerge pull request #2631 from ceph/wip-rwtimer
Sage Weil [Tue, 7 Oct 2014 18:41:11 +0000 (11:41 -0700)]
Merge pull request #2631 from ceph/wip-rwtimer

DNM: osdc/Objecter: use SafeTimer; make callbacks race-tolerant

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
10 years agocommon: ceph_ioprio_string_to_class always returns -EINVAL 2661/head
Loic Dachary [Tue, 7 Oct 2014 12:06:38 +0000 (14:06 +0200)]
common: ceph_ioprio_string_to_class always returns -EINVAL

The l string is always empty because std::transform needs a
pre-allocated string. Replace with the in-place version. Add unit tests.

http://tracker.ceph.com/issues/9677 Fixes: #9677

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 3535b7aba3df8b54fa5117b8a9c2f52b8f0f118b)

10 years agoosd: log error if set_ioprio fails to parse class
Loic Dachary [Tue, 7 Oct 2014 12:05:08 +0000 (14:05 +0200)]
osd: log error if set_ioprio fails to parse class

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 5088e0d49332d579ba7e33c2c9baee3d5f701a3e)

10 years agocommon: set_ioprio debug message including pid
Loic Dachary [Tue, 7 Oct 2014 12:03:39 +0000 (14:03 +0200)]
common: set_ioprio debug message including pid

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 33339c7754875eb7e513345ee6b26a9b2b4d2707)

10 years agocommon: do not set ioprio if pid is not set
Loic Dachary [Tue, 7 Oct 2014 12:02:09 +0000 (14:02 +0200)]
common: do not set ioprio if pid is not set

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit c7e4c0bfe70bf29d3b8fe4df4e4b934853e33d26)

10 years agodoc: missing osd prefix to ioprio config options
Loic Dachary [Tue, 7 Oct 2014 08:09:46 +0000 (10:09 +0200)]
doc: missing osd prefix to ioprio config options

http://tracker.ceph.com/issues/9676 Fixes: #9676

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 62235688c417ed19a472bc8a94bba005e03afbc6)

10 years agoMerge pull request #2655 from ceph/wip-9635
João Eduardo Luís [Tue, 7 Oct 2014 15:54:50 +0000 (15:54 +0000)]
Merge pull request #2655 from ceph/wip-9635

mon: fix async write completion waits

Fixes: #9635
Reviewed-by: Joao Eduardo Luis <joao@redhat.com>
10 years agoMerge pull request #2560 from ceph/wip-9418
Sage Weil [Tue, 7 Oct 2014 13:32:53 +0000 (06:32 -0700)]
Merge pull request #2560 from ceph/wip-9418

mon: add new profiles & audit cap checks

Reviewed-by: Sage Weil <sage@redhat.com>
10 years ago0.86 v0.86
Jenkins [Tue, 7 Oct 2014 13:20:21 +0000 (06:20 -0700)]
0.86

10 years agoRun configure without liblttng on squeeze as well.
Sandon Van Ness [Tue, 7 Oct 2014 13:03:36 +0000 (06:03 -0700)]
Run configure without liblttng on squeeze as well.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
10 years agoAdd reset_tp_timeout in long loop in add_source_info for suicide timeout 2650/head
luowei [Thu, 11 Sep 2014 07:43:13 +0000 (07:43 +0000)]
Add reset_tp_timeout in long loop in add_source_info for suicide timeout

Fixes: #9128
Signed-off-by: luowei@yahoo-inc.com
10 years agoMerge branch 'wip-9658' into giant
Sage Weil [Mon, 6 Oct 2014 22:52:24 +0000 (15:52 -0700)]
Merge branch 'wip-9658' into giant

10 years agotest/osd/Object: don't generate length of 0 2660/head
Samuel Just [Mon, 6 Oct 2014 22:22:33 +0000 (15:22 -0700)]
test/osd/Object: don't generate length of 0

Fixes: #9203
Signed-off-by: Samuel Just <sam.just@inktank.com>
10 years agoMerge pull request #2645 from ceph/wip-9663
Sage Weil [Mon, 6 Oct 2014 16:37:32 +0000 (09:37 -0700)]
Merge pull request #2645 from ceph/wip-9663

client: clean-up objecter on failed client init

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agoceph.spec: make ceph -> python-ceph dependency specify version
Sage Weil [Mon, 6 Oct 2014 16:15:29 +0000 (09:15 -0700)]
ceph.spec: make ceph -> python-ceph dependency specify version

It does this indirectly via ceph -> ceph-common -> python-ceph, but do so
here too for clarity/consistency.

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2642 from johnugeorge/wip-9492-crush-giant
Loic Dachary [Mon, 6 Oct 2014 07:51:05 +0000 (09:51 +0200)]
Merge pull request #2642 from johnugeorge/wip-9492-crush-giant

Crush: Backporting fixes for #9492 to giant

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
10 years agoclient: clean-up objecter on failed client init 2645/head
Noah Watkins [Sun, 5 Oct 2014 20:15:13 +0000 (13:15 -0700)]
client: clean-up objecter on failed client init

During mount() the objecter isn't shutdown if the mon client fails to
initialize. Objecter asserts in destructor expect it to have been
shutdown but this skipped.

hadoop@plana85:~$ ./hadoop/bin/hadoop fs -ls /
14/10/05 12:35:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
osdc/Objecter.cc: In function 'virtual Objecter::~Objecter()' thread 7ff422705700 time 2014-10-05 12:35:51.090776
osdc/Objecter.cc: 3927: FAILED assert(!m_request_state_hook)
 ceph version 0.85-981-g25bcc39 (25bcc39bb809e2d13beea1529e4ab92d1b61fa5b)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7f) [0x7ff3f5c28f7f]
 2: (Objecter::~Objecter()+0x397) [0x7ff3f5bca707]
 3: (Objecter::~Objecter()+0x9) [0x7ff3f5bca8b9]
 4: (Client::~Client()+0x7d) [0x7ff3f5b6770d]
 5: (Client::~Client()+0x9) [0x7ff3f5b680a9]
 6: (ceph_mount_info::mount(std::string const&)+0x149) [0x7ff3f5b1fa49]
 7: (ceph_mount()+0x4e) [0x7ff3f5b1dcbe]
 8: (Java_com_ceph_fs_CephMount_native_1ceph_1mount()+0xb7) [0x7ff4158b1c97]
 9: [0x7ff41839dd68]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
terminate called after throwing an instance of 'ceph::FailedAssertion'
Aborted

fixes: #9663

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
10 years agoCrush: Ensuring at most num-rep osds are selected 2642/head
Johnu George [Mon, 29 Sep 2014 17:07:44 +0000 (10:07 -0700)]
Crush: Ensuring at most num-rep osds are selected

Crush temporary buffers are allocated as per replica size configured
by the user.When there are more final osds (to be selected as per
rule) than the replicas, buffer overlaps and it causes crash.Now, it
ensures that at most num-rep osds are selected even if more number of
osds are allowed by indep rule. The fix for firstn rules is already
merged as part of bug #9492. Required test files are added.

Fixes: #9492
Signed-off-by: Johnu George johnugeo@cisco.com
(cherry picked from commit 234b066ba04976783d15ff2abc3e81b6cc06fb10)

10 years agoCrush: Ensuring at most num-rep osds are selected
Johnu George [Wed, 24 Sep 2014 16:32:50 +0000 (09:32 -0700)]
Crush: Ensuring at most num-rep osds are selected

Crush temporary buffers are allocated as per replica size configured
by the user.When there are more final osds (to be selected as per
rule) than the replicas, buffer overlaps and it causes crash.Now, it
ensures that at most num-rep osds are selected even if more number of
osds are allowed by the rule.

Fixes: #9492
Signed-off-by: Johnu George <johnugeo@cisco.com>
(cherry picked from commit 6b4d1aa99718e3b367496326c1e64551330fabc0)

10 years agoMerge pull request #2640 from ceph/wip-9657
Sage Weil [Sat, 4 Oct 2014 00:02:56 +0000 (17:02 -0700)]
Merge pull request #2640 from ceph/wip-9657

messages: provide an explicit COMPAT_VERSION in MMDSBeacon

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agodebian/control: fix python-ceph -> ceph file move to allow upgrades
Sage Weil [Fri, 3 Oct 2014 23:22:57 +0000 (16:22 -0700)]
debian/control: fix python-ceph -> ceph file move to allow upgrades

See fe3434f41cd09433975d7d0f9dbb2fae662e4a1b.

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agomessages: provide an explicit COMPAT_VERSION in MMDSBeacon 2640/head
Greg Farnum [Fri, 3 Oct 2014 23:06:53 +0000 (16:06 -0700)]
messages: provide an explicit COMPAT_VERSION in MMDSBeacon

We weren't setting an explicit compat version and so the Message
was setting it to the head version, breaking communication with
older daemons.

Fixes: #9657
Signed-off-by: Greg Farnum <greg@inktank.com>
10 years agodebian/control: BuildDepend on lsb-release
Sage Weil [Fri, 3 Oct 2014 20:45:48 +0000 (13:45 -0700)]
debian/control: BuildDepend on lsb-release

debian/rules uses it to make lttng build conditional.

Signed-off-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #2635 from dachary/wip-9653-ceph-disk-bootstrap-osd-giant
Sage Weil [Fri, 3 Oct 2014 18:58:54 +0000 (11:58 -0700)]
Merge pull request #2635 from dachary/wip-9653-ceph-disk-bootstrap-osd-giant

ceph-disk: bootstrap-osd keyring ignores --statedir (giant)

10 years agoMerge pull request #2637 from dachary/wip-9245-test-mon-pending-giant
Sage Weil [Fri, 3 Oct 2014 18:24:44 +0000 (11:24 -0700)]
Merge pull request #2637 from dachary/wip-9245-test-mon-pending-giant

mon: obsolete osdmonitor_prepare_command (giant)

10 years agotests: remove tests requiring osdmonitor_prepare_command 2637/head
Loic Dachary [Fri, 3 Oct 2014 14:12:12 +0000 (16:12 +0200)]
tests: remove tests requiring osdmonitor_prepare_command

Although they have been useful, their maintenance and fragility caused
more trouble than their benefits.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 038de0031bc0892359201012dafb82ac1b1a4d0f)

10 years agomon: obsolete osdmonitor_prepare_command
Loic Dachary [Fri, 3 Oct 2014 14:10:30 +0000 (16:10 +0200)]
mon: obsolete osdmonitor_prepare_command

And the mon_advanced_debug_mode option that is only used in the context
of osdmonitor_prepare_command.

http://tracker.ceph.com/issues/9245 Fixes: #9245

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 2abc8f400f7db256c2a30f1d0886a593e6c194eb)

10 years agotest: minor case fix
Loic Dachary [Sat, 6 Sep 2014 14:06:12 +0000 (16:06 +0200)]
test: minor case fix

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
(cherry picked from commit 6f698374562de74806203c7b41072572b59bd256)

10 years agomon: Monitor: let 'handle_command()' deal with caps validation 2560/head
Joao Eduardo Luis [Tue, 30 Sep 2014 18:15:18 +0000 (19:15 +0100)]
mon: Monitor: let 'handle_command()' deal with caps validation

If a given client doesn't have the required caps when running a command,
it must receive an EACCES or EPERM reply.  This is already handled by
Monitor::handle_command(), which does an exceptionally good job at it.

Therefore, and unlike other messages that do not expect return values,
we can't simply drop the message if the client doesn't have the
appropriate capabilities, or things can get very weird very fast from
the user's perspective.  Dropping the message for a command without a
reply has roughly the same effect as loss of quorum (timeout, pipes
failing) and confusion may ensue from it.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agoqa/workunits: mon: auth_caps: account for mon blank caps
Joao Eduardo Luis [Tue, 30 Sep 2014 18:13:52 +0000 (19:13 +0100)]
qa/workunits: mon: auth_caps: account for mon blank caps

test creating and entity with blank caps with and without '--force'
being specified.  without '--force' they must fail with EINVAL as the
monitor will not be able to parse them.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agomon: MonCommands: adjust indentation for 'auth add'
Joao Eduardo Luis [Tue, 30 Sep 2014 18:11:01 +0000 (19:11 +0100)]
mon: MonCommands: adjust indentation for 'auth add'

Eye-candy.  We changed indentation of a few other entries and this one
was just too darn obvious, itching all over, night terrors, the whole
nine yards.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agoqa/workunits: mon: auth_caps: variables must be local
Joao Eduardo Luis [Tue, 30 Sep 2014 17:49:19 +0000 (18:49 +0100)]
qa/workunits: mon: auth_caps: variables must be local

We have variables with the same name that are being shared!  We don't
hit any issues with it currently because the code just kind of works
even though that happens.  Add a bit of new logic that relies on an
immutable return code (for instance) and we're in the woods.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agoqa/workunits: mon: auth_caps: cleanup on successful exit
Joao Eduardo Luis [Tue, 30 Sep 2014 17:46:49 +0000 (18:46 +0100)]
qa/workunits: mon: auth_caps: cleanup on successful exit

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agotest/vstart_wrapper.sh: use cephx
Joao Eduardo Luis [Wed, 24 Sep 2014 15:06:09 +0000 (16:06 +0100)]
test/vstart_wrapper.sh: use cephx

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
10 years agomon: Monitor: remove redudant call to osdmon's dispatch
Joao Eduardo Luis [Tue, 16 Sep 2014 16:32:41 +0000 (17:32 +0100)]
mon: Monitor: remove redudant call to osdmon's dispatch

Backport: firefly

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
10 years agomon: Monitor: check caps and source before dispatching messages
Joao Eduardo Luis [Tue, 16 Sep 2014 15:32:37 +0000 (16:32 +0100)]
mon: Monitor: check caps and source before dispatching messages

Only dispatch messages that a client may send if said client has at
least MON_CAP_R, and only dispatch internal monitor messages if peer is
a monitor.

Backport: firefly

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
10 years agomon: Monitor: create logical divisions on dispatch() based on msg nature
Joao Eduardo Luis [Fri, 12 Sep 2014 16:21:45 +0000 (17:21 +0100)]
mon: Monitor: create logical divisions on dispatch() based on msg nature

Instead of a single switch(), have multiple switch() and order them by
increasing necessity of privileges.

This patch thus divides the big switch into:

- messages not requiring auth/caps checks at all
- messages which caps shall be checked somewhere else
- messages the Monitor class needs to deal with but only require a
  client to have enough caps for the monitor to consider handling them
- messages that only a monitor is allowed to send.

Backport: firefly

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
10 years agotest: mon: test-mon-msg: Check which msgs the mons handle and reply to
Joao Eduardo Luis [Thu, 11 Sep 2014 16:40:04 +0000 (17:40 +0100)]
test: mon: test-mon-msg: Check which msgs the mons handle and reply to

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>