Yuri Weinstein [Thu, 12 Oct 2017 20:36:20 +0000 (13:36 -0700)]
tests - Removed `distors` and added `supported` to run on all supported OSs
Added `openstack` fragment to run on vps
Forced/hard-coded `machine_type=vps`
Added `print`s
Added only centos and ubuntu latest to support systemd (in distros dir)
Added `ceph osd set-require-min-compat-client luminous`
Yuri Weinstein [Tue, 10 Oct 2017 20:48:33 +0000 (13:48 -0700)]
tests - Added yaml fragmet to cover testing...
... for http://tracker.ceph.com/issues/21660#note-17
for jewel-x (see similar https://github.com/ceph/ceph/pull/18200)
Kefu Chai [Tue, 10 Oct 2017 12:32:08 +0000 (20:32 +0800)]
mon/PGMap: let pg_string_state() return boost::optional<>
better to be specific, so we don't run into the same problem even we
are using the MSB of uint64_t for a pg state bit in future. we could,
however use uint64_t(-1) to indicate the pg_string_state()'s failure to
parse the state string, because pg_string_state() only translate a
single state a time. but it's always better to be explicit than
implicit.
Josh Durgin [Tue, 10 Oct 2017 23:54:31 +0000 (19:54 -0400)]
common/options: enable multiple rocksdb compaction threads for filestore
One of the major benefits of rocksdb over leveldb is multithreaded
compaction. The default of 1 thread does not provide much benefit, and
is insufficient for heavy rgw workloads.
For high-write and delete omap workloads I've seen up to 8 compaction
threads be used. There's little overhead to having a higher max than
are needed, so set the default to 8.
Kefu Chai [Sat, 7 Oct 2017 14:15:11 +0000 (22:15 +0800)]
ceph-disk: retry on OSError
we are likely to
1) create partition, for instance, sdc1
2) partprobe sdc
3) udevadm settle
4) check the device by its path: /dev/sdc1
but there is chance that the uevent sent from kernel fails to reach udev
before we call "udevadm", hence "/dev/sdc1" does not exist even after
"udevadm settle" returns. so we retry in case of OSError here.
Ken Dreyer [Thu, 7 Sep 2017 17:07:59 +0000 (11:07 -0600)]
.gitignore: allow debian .patch files
The Ubuntu packaging layout with git-buildpackage assumes a
"debian/patches/" directory with several .patch files in it.
When upstream's .gitignore tells Git to ignore .patch files, we have to
edit that line out downstream. When we forget to do that downstream, it
can lead to missing patches and broken downstream builds.
Allow patches in the /debian/patches directory so it's easier to
maintain an Ubuntu package based on upstream's Git repo.
Sage Weil [Thu, 5 Oct 2017 20:26:16 +0000 (15:26 -0500)]
src/messages/MOSDMap: reencode OSDMap for older clients
We explicitly select which missing bits trigger a reencode. We
already had jewel and earlier covered, but kraken includes all of
the previously mentioned bits but not SERVER_LUMINOUS. This
prevents kraken clients from decoding luminous maps.
Kefu Chai [Thu, 31 Aug 2017 10:15:28 +0000 (18:15 +0800)]
cmake: disable VTA on options.cc
to silence following warning and to avoid compiling this file twice:
ceph/src/common/options.cc: In function ‘std::vector<Option> get_global_options()’:
ceph/src/common/options.cc:151:21: note: variable tracking
size limit exceeded with -fvar-tracking-assignments, retrying without
std::vector<Option> get_global_options() {
^~~~~~~~~~~~~~~~~~
Sage Weil [Wed, 4 Oct 2017 20:28:26 +0000 (15:28 -0500)]
osd/PG: separate event for RemoteReservationCanceled
Right now we transparently map a RemoteReservationRejected into a
*Canceled event because this what peers send over the wire. Even
once new peers start sending and explicit CANCEL, old peers will
still do so, so we'll maintain this mapping for a while.
Sage Weil [Sun, 1 Oct 2017 20:04:34 +0000 (15:04 -0500)]
osd/PG: handle RecoveryReservationRejected in RepWaitRecoveryReserved
This state is analogous to RepWaitBackfillReserved; just like we do there
we want to handle the REJECT from the primary by canceling our local
remote_reservation.
Sage Weil [Sun, 1 Oct 2017 20:03:22 +0000 (15:03 -0500)]
osd/PG: ignore RemoteReservationRejected if we are RepNotRecoverying
The primary may send us a REJECT (meaning cancel) if recovery/backfill is
preempted there. That can happen even if the recovery isn't reserved or
requested here (e.g., because the primary is still waiting for the local
reservation). Just ignore it and remain in RepNotRecovering.
Sage Weil [Sun, 1 Oct 2017 20:01:05 +0000 (15:01 -0500)]
osd/PG: cancel local reservation in RemoteReservationRejected handler
We can get a RemoteReservationRejected event either because *we* decide
to reject, or because we get a REJECT from the primary that means "cancel"
(e.g., because recovery/backfill was preempted there). In both cases we
want to cancel our remote_reservation.
Sage Weil [Sun, 1 Oct 2017 19:59:31 +0000 (14:59 -0500)]
osd/PG: move reject_reservation out of RemoteReservationRejected reaction
The RemoteReservationRejected event is also submitted when we are a
replica or backfill target and get a MBackfillReserve REJECT message
because the primary canceled or was preempted. In that case, we don't
want to send a REJECT back to the primary; we only need to send it in the
cases where *we*, locally, decide to reject. Move the call to those call
sites.
osd: make the PG's SORTBITWISE assert a more generous shutdown
We want to stop working if we get activated while sortbitwise is not set
on the cluster, but we might have old maps where it wasn't if the flag
was changed recently. And doing it in the PG code was a bit silly anyway.
Instead check SORTBITWISE in the main OSDMap handling code prior to
prepublishing it. Let it go through if we aren't active at the time.
Add _interfaces option to constrain the choice of IPs in the network
list to those on interfaces matching the provided list of interface names.
The _interfaces options only work in concert with the _network options,
so you must also specify a list of networks if you want to use a specific
interface, e.g., by specifying a broad network like "::" or "0.0.0.0/0".
Greg Farnum [Tue, 3 Oct 2017 22:54:06 +0000 (15:54 -0700)]
msgr: add a mechanism for Solaris to avoid dying on SIGPIPE
This is fairly clean: we define an RAII object in the Messenger.h on
Solaris, and "declare" it with a macro in the implementations. There's
no code duplication and on Linux it's just entirely compiled out.
Sage Weil [Tue, 3 Oct 2017 21:48:37 +0000 (16:48 -0500)]
os/bluestore: use normal Context for async deferred_try_submit
I'm not quite sure why the FunctionContext did not ever execute on the
finisher thread (perhaps the [&] captured some state on the stack that it
shouldn't have?). In any case, using a traditional Context here appears
to resolve the problem (of the async deferred_try_submit() never executing,
leading to a bluestore stall/deadlock).
Sage Weil [Fri, 29 Sep 2017 18:47:19 +0000 (13:47 -0500)]
os/bluestore: wake kv thread when blocking on deferred_bytes
We need to wake the kv thread whenever setting deferred_aggressive to
ensure that txns with deferred io that have committed but haven't submitted
their deferred writes get submitted. This aligns us with the other
users of deferred_aggressive (e.g., _osr_drain_all).
Sage Weil [Wed, 4 Oct 2017 13:25:38 +0000 (08:25 -0500)]
mgr/localpool: fix rule selection
The 'osd pool create' arg parsing is broken; the rule name for
'ceph osd pool create $name $numpgs replicated $rulename' is passed
via the erasure_code_profile param. Too many req=false options
without a way to disambiguate them.
Work around it by passing both 'rule' and 'erasure_code_profile'
keys, so that if/when the hack in OSDMonitor.cc is removed it will
still work. Blech.
rgw: dont reuse stale RGWObjectCtx for get_bucket_info()
if the earlier call to store->get_bucket_entrypoint_info() failed with
ENOENT, the obj_ctx will cache exists=false.
put_bucket_entrypoint_info() doesn't invalidate that, so this call to
get_bucket_info() was reading from a stale cache and failing with ENOENT
Sage Weil [Wed, 20 Sep 2017 16:38:46 +0000 (12:38 -0400)]
os/bluestore: ignore 0x2000~2000 extent oddity from luminous upgrade
Luminous does a block_size granularity freelist, and assumes that
0~ROUND_UP_TO(SUPER_RESERVED,block_size) is used. Current master uses
min_alloc_size granularity and changes that assumption to
0~ROUND_UP_TO(SUPER_RESERVED,min_alloc_size). That means if master
fsck's a luminous-created bluestore, it will think 0x2000~2000 is used
(current baked-in min_alloc_size-based assumption) but the old freelist
says it is free (old mkfs assumption). The disparity is harmless since
the extent is below min_alloc_size, so ignore it.
projects / ceph.git / log