ReplicatedPG,OSD: move dequeueing into on_shutdown

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: clarify PG::deleting comment

Signed-off-by: Samuel Just <sam.just@inktank.com>

FileJournal: quieter debugging on journal scanning

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

doc: Fixed some typos.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #128 from dalgaaf/wip-da-pylint

Fix some python issues

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #132 from ceph/wip-libcephfs-fix

libcephfs: fix ceph_get_osd_crush_location

Reviewed-by: Sage Weil <sage@inktank.com>

libcephfs: fix ceph_get_osd_crush_location

(a) Allow NULL buffer when length is zero to get the target buffer
length. (b) fix edge case where buffer length is the exact size
required.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #127 from alram/master

[doc] undisplayed notes, tips, and important sections

Reviewed-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #120 from ceph/wip-msgr-acks

msg/Pipe: fix seq handshake on reconnect

Reviewed-by: Greg Farnum <greg@inktank.com>

rados.py: remove unnecessary semicolon

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.py: fix bad indention

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.py: remove lenght modifier 'l' on format string

Remove length modifier 'l' since it's ignored for python.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.py: fix 'Rados' has no member 'name'

Lets store rados_id from __init__ and use this instead of 'name'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.py: add missing space after ','

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-create-keys: rename log to LOG since it's a constants

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge branch 'next'

doc/release-notes: v0.59

Signed-off-by: Sage Weil <sage@inktank.com>

msg/Pipe: fix seq handshake on reconnect

We go to the trouble to exchange our seq numbers during the handshake, but
the bit that then avoids resending old messages was broken because we
already requeue_sent() before we get to this point.  Fix it by discarding
queued items (in the high prio slot) that we don't need to resend, and
adjust out_seq as needed.

Drop the optional arg to requeue_sent() now that it is unused.

Signed-off-by: Sage Weil <sage@inktank.com>

Fix tips in documentation

Use of .. tip: instead of .. tip:: resulting in content not
being displayed.

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>

Fix important in documentation

Use of .. important: instead of .. important:: resulting in content
not being displayed.

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>

Fix notes in documentation

Use of .. note: instead of .. note:: resulting in content not
being displayed

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>

Merge pull request #126 from alram/master

Update Chef deployment documentation

Reviewed-by: Sage Weil <sage@inktank.com>

Update Chef deployment documentation

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>

Merge pull request #124 from ceph/wip-4509

mon: DataHealthService: shutdown mon if failed to obtain disk stats

Reviewed-by: Sage Weil <sage@inktank.com>

mon: DataHealthService: log to derr instead if we're about to shutdown

Otherwise the message would -- or could -- be lost.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: DataHealthService: shutdown mon if failed to obtain disk stats

Being unable to run a ::statfs() may be a symptom of something bigger.

We want to cleanly shutdown the monitor ASAP if such thing happens.

Fixes: #4509
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #123 from dalgaaf/wip-da-sca-misc-1

Some smaller misc fixes

Reviewed-by: Sage Weil <sage@inktank.com>

client/Client.cc: handle error if _lookup() fails

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

qa/workunits/direct_io/test_sync_io.c: add proper error handling

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_short_dio_read.c: add proper error handling

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Locker.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mount/mount.ceph.c: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #121 from ceph/wip-4448

mon: Monitor: clearer output on error during attempt to convert store

Reviewed-by: Sage Weil <sage@inktank.com>

mon: Monitor: clearer output on error during attempt to convert store

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

v0.59

Merge pull request #106 from ceph/wip-crush

crush: update weights properly for DAG (not tree) maps

Reviewed-by: caleb miles <caleb.miles@inktank.com>

Merge pull request #118 from dalgaaf/wip-da-enum

QuorumService.h: use enum instead of static const int

Reviewed-by: Sage Weil <sage@inktank.com>

QuorumService.h: use enum instead of static const int

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Missed adding rados_types.hpp to package

Caused by 3bd48cbbadb7908dd833ccde75359f085828fc5c
feature 4207 implementation

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>
(cherry picked from commit e1e2d5d2176cc9debd436ba944e6ca65b3253c8a)

PendingReleaseNotes: fix typo

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

librbd: optionally wait for a flush before enabling writeback

Older guests may not send flushes properly (i.e. never), so if this is
enabled, rbd_cache=true is safe for them transparently.

Disable by default, since it will unnecessarily slow down newer guest
boot, and prevent writeback caching for things that don't need to send
flushes, like the command line tool.

Refs: #3817
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

Makefile: missing header

Signed-off-by: Sage Weil <sage@inktank.com>

mon: use enum instead of static const int

This way it compiles.

Signed-off-by: Sage Weil <sage@inktank.com>

mon/Paxos: set state to RECOVERING during restart

This ensures that the paxos state is not active when the PaxosService
restart() methods run right afterwards, and that EAGAIN waiters will get
requeued appropriately.

Signed-off-by: Sage Weil <sage@inktank.com>

Makefile.am: fix misspelt header name

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon/PaxosService: handle non-zero return values

If 7aec13f749035b9bef5e398c1ac3d56ceec8eb81 we started passing non-zero
return values to these completions; now we have to deal with them
accordingly.

RetryMessage behaves just like the Monitor variant.

Propose and Committed update state but otherwise ignore non-zero
return values.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'next'

ceph-disk-prepare: 'mkfs -t' instead of 'mkfs --type='

Older mkfs (el6) doesn't like --type=.

Fixes: #4495
Reported-by: Alexandre Maragone <alexandre.maragone@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>

mds: Handle ENODATA returned from getxattr

The osds might return ENODATA if we request an
xattr that doesn't exist.  In this case, we're
requesting the 'parent' xattr so that we can
remove all the forwarding pointers, but the xattr
may not have been written (which only happens on
log segment trim), so we don't assert here.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

mon/PaxosService: fix proposal waiter handling

- Cancel the propsal waiters with EAGAIN on election, etc.
- Drop the wakeup helper and open-code the one caller.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

Makefile: fix header name

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #115 from ceph/wip-4199

Resolves #4199

Reviewed-by: Sage Weil <sage@inktank.com>

mon: Monitor: take advantage of the new HealthMonitor class.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: HealthMonitor: Keep track of monitor cluster's health

The HealthMonitor builds upon the QuorumService interface, and should be
used to keep track of all and any relevant information about the monitor
cluster (maybe even about all the cluster if need be).

This patch also introduces the HealthService interface, used to define
a HealthMonitor service, responsible for dispatching 'MMonHealth' messages
(the QuorumService interface dispatches generic 'Message').

Based on the HealthService interface, we introduce the DataHealthService
class, a service that will track disk space consumption by the monitors,
warn when a given threshold is crossed, and gracefully shutdown the monitor
if disk space usage hits critical levels that might affect the correct
monitor behavior.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: QuorumService: Allow for services quorum-bound to be easily created

As the monitor grows in features, we have been dumping them in the Monitor
class as they don't really fit anywhere else.

Most of those latest features have been, and some of the future changes
will also be, quorum-bounded.  By that we mean that these features tend
to require a quorum to be present in order to work.

Although we already have the PaxosService interface, it really isn't
adequate for this kind of features, as they don't really require Paxos,
nor do they access the store.  Furthermore, they don't really need to
tick at the same rate as the monitor, and can be fairly independent.

Therefore we now introduce the concept of a QuorumService, a class to be
built upon, managing the tick and dispatch for any kind of service
basically requiring a quorum to function.

Among the already existing monitor features that could take advantage of
this new class we can find the Timecheck infrastructure, as it is by
nature quorum bounded.  The monitor store sync could also take advantage
of this service, although it doesn't really require a quorum to work,
and even the PaxosService-related classes could use this.

This patch also introduces the MMonQuorumService base class, to be used
by any message that should want to.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

qa: add 16MB direct-io write test

See bug #2657.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #113 from dalgaaf/wip-da-performance-3

Iterator related fixes

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #112 from dalgaaf/wip-da-performance-2-v2

prefer prefix ++/--operator for e.g. iterators for performance reasons

Reviewed-by: Sage Weil <sage@inktank.com>

signal_handler: add func to queue an async signal

For example, queue a SIGINT without actually sending the signal via the
OS.

Signed-off-by: Sage Weil <sage@inktank.com>

client:  Remove unecessary set_inode() in _rmdir()

With the recent changes in fc80c1dc6ee315ae5e039986602ffadba46cb43b,
we only allow setting the inode once on a MetaRequest.  This triggered
a bug in _rmdir(), where the parent dir inode passed in and being set
on the MetaRequest, and then also setting the dir inode on the MetaRequest.
Removing the set_inode() using the parent dir inode resolves this issue.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

mon/Session.h: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/filestore/workload_generator.cc: prefer prefix ++operator for iterators

Prefer prefix ++operator for iterator. Move the iterator declaration into
the for-loop header since the iterator isn't used outside the loop.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/filestore/test_idempotent.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/filestore/test_idempotent.cc: fix ~TestFileStoreState()

Fix interator handling in ~TestFileStoreState(). After std::map::erase()
the used iterator is invalid. Use a while-loop and erase the object with
post-incremented iterator instead.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/test_snap_mapper.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/osd/RadosModel.h: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/filestore/TestFileStoreState.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/xattr_bench.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/osd/Object.cc: prefer prefix --/++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/encoding/ceph_dencoder.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test/bench/bencher.h: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ObjectMap/test_object_map.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

tools/rest_bench.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

tools/common.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-filestore-dump.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_user.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rest_swift.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSDMap.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDCache.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/CInode.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

client/Client.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

auth/AuthMethodList.cc: fix remove_supported_auth()

Make sure the iterator isn't incremented twice if a element gets erased.
Move increment out of the for-loop header and add it to a new else block.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/mdstypes.h: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.cc: make sure range_iter != ranges.end()

Make sure range_iter is valid, set range_iter = next_iter instead of
++range_iter, since next_iter is already checked against ranges.end() and
is the same as ++range_iter.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.cc: prefer prefix ++operator for iterators

Prefer prefix ++operator for iterators, move some iterator declaration into
the for-loop header, since the iterator is only used within the loop.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_rados.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_admin.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_acl_s3.cc: prefer prefix ++operator for iterators

Prefer prefix ++operator for iterators, move some iterator declaration into
the for-loop header, since the iterator is only used within the loops.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdmaptool.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/Striper.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/Objecter.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/ObjectCacher.cc: prefer prefix ++/--operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/ObjectCacher.h: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/Objecter.h: prefer prefix ++operator for iterators

Prefer prefix ++operator for iterators, move one iterator declaration into
the for-loop header, since the iterator is only used within the loop.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/Filer.h: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osdc/Filer.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/osd_types.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/SnapMapper.cc: use empty() instead of size() == 0

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PG.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>