ceph-disk: fix naming of a local variable in find_cluster_by_uuid

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: rename some constants to upper case variable names

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: add some more docstrings

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: print subprocess.CalledProcessError on error

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: fix indention

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge branch 'next'

client: don't set other if lookup fails in rename

On rename, only set the other inode if the
lookup for the destination succeeds, otherwise we hit
a segv in set_other_inode().

Fixes #4517.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
Tested-by: Noah Watkins <jayhawk@cs.ucsc.edu>

Merge branch 'next'

test/libcephfs: Test rename error cases

Make sure that rename fails with the ENOENT
if the source path doesn't exist.

Signed-off-by: Sam Lang <sam.lang@inktank.com>

client: Fix rename returning ENOENT for dest

Introduced by fc80c1dc6ee315ae5e039986602ffadba46cb43b,
the client should _not_ fail if the lookup for the
destination path on rename returns ENOENT.

The previous code also did not check that the lookup
returned ENOENT or success.  We add the check and fail
if we get any other errors.

Fixes #4517.
Signed-off-by: Sam Lang <sam.lang@inktank.com>

preserve /var/lib/ceph on deb/rpm purge

We should clobber configuration and log data, but *not* user data.  Leave
/var/lib/ceph alone.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>

Merge remote-tracking branch 'upstream/wip_4435'

Fixes: #4435
Reviewed-by: David Zafman <david.zafman@inktank.com>

PG::GetMissing: need to check need_up_thru in MLogRec handler

Backport: bobtail
Fixes: #4534
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

PG,osd_types: improve check_new_interval debugging

Backport: bobtail
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

common/MemoryModel: remove logging to /tmp/memlog

This was a hack for dev purposes ages ago; remove it.  The predictable
filename is a security issue.

CVE-2013-1882

Reported-by: Michael Scherer <misc@zarb.org>
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

init-ceph: clean up temp ceph.conf filename on exit

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

init-ceph: push temp conf file to a unique location on remote host

The predictable file name is a security problem.

CVE-2013-1882

Reported-by: Michael Scherer <misc@zarb.org>
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

mkcephfs: make remote temp directory name unique

The predictable file name is a security problem.

CVE-2013-1882

Reported-by: Michael Scherer <misc@zarb.org>
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

Merge pull request #130 from ceph/wip-fs-rename

test: add ceph_rename test

Reviewed-by: Sage Weil <sage@inktank.com>

doc: Added {id} argument to OSD lost.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

ceph-disk: re-add python 2.7 dependency comment

FIXME!

Signed-off-by: Sage Weil <sage@inktank.com>

Merge pull request #117 from ceph/wip-ceph-disk

ceph-disk-* refactor

Merge branch 'next'

osd: reenable 'journal aio = true'

Now that #4079 is resolved.  Reverts 1cfc3ae0.

Signed-off-by: Sage Weil <sage@inktank.com>

os/FileJournal: fix aio self-throttling deadlock

This block of code tries to limit the number of aios in flight by waiting
for the amount of data to be written to grow relative to a function of the
number of aios.  Strictly speaking, the condition we are waiting for is a
function of both aio_num and the write queue, but we are only woken by
changes in aio_num, and were (in rare cases) waiting when aio_num == 0 and
there was no possibility of being woken.

Fix this by verifying that aio_num > 0, and restructuring the loop to
recheck that condition on each wakeup.

Fixes: #4079
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

Merge pull request #137 from dalgaaf/wip-da-cleanup-includes

Cleanup some twice included header

Reviewed-by: Sage Weil <sage@inktank.com>

test/test_snap_mapper.cc: remove twice included <tr1/memory>

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MDSMonitor.cc: remove twice included MonitorDBStore.h

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/LogMonitor.cc: remove twice included <sstream>

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/AuthMonitor.cc: remove twice included <sstream>

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/Formatter.h: remove twice included <list>

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ReplicatedPG: add debug flag to skip full check at reservation

This will make it easier to test the check in do_scan.

Signed-off-by: Samuel Just <sam.just@inktank.com>

ReplicatedPG: replica should post BackfillTooFull in do_scan if full

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: halt backfill on RemoteReservationRejected in Backilling

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: add helper for adding a timer event to retry backfill

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: add BackfillTooFull event for RepRecovering

Replica will use this to notify Primary to stop backfilling.

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: add helper for rejecting backfill reservation

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: use OSDService::too_full_for_backfill in RepWaitBackfillReserved

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSDService: add too_full_for_backfill

Signed-off-by: Samuel Just <sam.just@inktank.com>

Merge remote-tracking branch 'upstream/wip_osd_shutdown_notification'

Fixes: #1857
Fixes: #4267
Reviewed-by: Sage Weil <sage@inktank.com>

Makefile: add MOSDMarkMeDown

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD: notify mon prior to shutdown

Signed-off-by: Samuel Just <sam.just@inktank.com>

Monitor: add MOSDMarkMeDown support

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSDMonitor: factor out check_source helper

Signed-off-by: Samuel Just <sam.just@inktank.com>

messages: add MOSDMarkMeDown

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD: clear OpHistory on shutdown

Signed-off-by: Samuel Just <sam.just@inktank.com>

OpRequest: use OpRequestRef for OpHistory

Signed-off-by: Samuel Just <sam.just@inktank.com>

FileStore::stat: valgrind: don't read *st on error

Signed-off-by: Samuel Just <sam.just@inktank.com>

shared_cache: clear lru in destructor

Otherwise, the live references will attempt to extricate
themselves from a disolving SharedLRU instance as the
member destructors run.

Signed-off-by: Samuel Just <sam.just@inktank.com>

ceph_osd: clear client_throttler prior to putting g_ceph_context

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD: reorder OSD::shutdown

Reorder teardown:
- pgs
- queues/threadpools
- persist superblock
- filestore
- timers
- messengers

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD: check for is_stopping after locking osd_lock or heartbeat_lock

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD: lookup_lock_raw_pg is dead

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD: rename timer to tick_timer

Only used for scheduling ticks - we should keep it
that way.

Signed-off-by: Samuel Just <sam.just@inktank.com>

OSD: move backfill_request_timer cleanup to OSDService::shutdown

Signed-off-by: Samuel Just <sam.just@inktank.com>

ReplicatedPG,OSD: move dequeueing into on_shutdown

Signed-off-by: Samuel Just <sam.just@inktank.com>

PG: clarify PG::deleting comment

Signed-off-by: Samuel Just <sam.just@inktank.com>

Merge pull request #135 from ceph/wip-4519

mon: AuthMonitor: delete auth_handler while increasing max_global_id

Reviewed-by: Sage Weil <sage@inktank.com>

mon: AuthMonitor: delete auth_handler while increasing max_global_id

By not deleting and setting NULL the session's auth_handler, we could
hit a scenario in which we'd end up dispatching a previously-wait-listed
auth message and we wouldn't start its auth session.

This only happened when increasing max_global_id via Paxos (in which case
we would wait-list the message) and would only be noticeable when running
with cephx disabled.

Fixes: #4519
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

FileJournal: quieter debugging on journal scanning

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>
(cherry picked from commit 6740d512ac12263f7bee370bc14b1179f83af5be)

FileJournal: quieter debugging on journal scanning

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>

doc: Fixed some typos.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #128 from dalgaaf/wip-da-pylint

Fix some python issues

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge pull request #132 from ceph/wip-libcephfs-fix

libcephfs: fix ceph_get_osd_crush_location

Reviewed-by: Sage Weil <sage@inktank.com>

libcephfs: fix ceph_get_osd_crush_location

(a) Allow NULL buffer when length is zero to get the target buffer
length. (b) fix edge case where buffer length is the exact size
required.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

test: add ceph_rename test

Triggers bug in ticket #4517

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge pull request #127 from alram/master

[doc] undisplayed notes, tips, and important sections

Reviewed-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #120 from ceph/wip-msgr-acks

msg/Pipe: fix seq handshake on reconnect

Reviewed-by: Greg Farnum <greg@inktank.com>

rados.py: remove unnecessary semicolon

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.py: fix bad indention

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.py: remove lenght modifier 'l' on format string

Remove length modifier 'l' since it's ignored for python.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.py: fix 'Rados' has no member 'name'

Lets store rados_id from __init__ and use this instead of 'name'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rados.py: add missing space after ','

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-create-keys: rename log to LOG since it's a constants

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge branch 'next'

doc/release-notes: v0.59

Signed-off-by: Sage Weil <sage@inktank.com>

msg/Pipe: fix seq handshake on reconnect

We go to the trouble to exchange our seq numbers during the handshake, but
the bit that then avoids resending old messages was broken because we
already requeue_sent() before we get to this point.  Fix it by discarding
queued items (in the high prio slot) that we don't need to resend, and
adjust out_seq as needed.

Drop the optional arg to requeue_sent() now that it is unused.

Signed-off-by: Sage Weil <sage@inktank.com>

Fix tips in documentation

Use of .. tip: instead of .. tip:: resulting in content not
being displayed.

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>

Fix important in documentation

Use of .. important: instead of .. important:: resulting in content
not being displayed.

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>

Fix notes in documentation

Use of .. note: instead of .. note:: resulting in content not
being displayed

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>

Merge pull request #126 from alram/master

Update Chef deployment documentation

Reviewed-by: Sage Weil <sage@inktank.com>

Update Chef deployment documentation

Signed-off-by: Alexandre Marangone <alexandre.marangone@inktank.com>

Merge pull request #124 from ceph/wip-4509

mon: DataHealthService: shutdown mon if failed to obtain disk stats

Reviewed-by: Sage Weil <sage@inktank.com>

mon: DataHealthService: log to derr instead if we're about to shutdown

Otherwise the message would -- or could -- be lost.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: DataHealthService: shutdown mon if failed to obtain disk stats

Being unable to run a ::statfs() may be a symptom of something bigger.

We want to cleanly shutdown the monitor ASAP if such thing happens.

Fixes: #4509
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

Merge pull request #123 from dalgaaf/wip-da-sca-misc-1

Some smaller misc fixes

Reviewed-by: Sage Weil <sage@inktank.com>

client/Client.cc: handle error if _lookup() fails

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

qa/workunits/direct_io/test_sync_io.c: add proper error handling

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_short_dio_read.c: add proper error handling

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Locker.cc: prefer prefix ++operator for iterators

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mount/mount.ceph.c: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #122 from dalgaaf/wip-da-ceph-disk-1

More fixes for ceph-disk

Reviewed-by: Sage Weil <sage@inktank.com>

ceph-disk: cast output of subprocess.Popen() to str()

Cast output of subprocess.Popen() to str() to be able to use
str.split() and str.splitlines() without warnings from pylint.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #121 from ceph/wip-4448

mon: Monitor: clearer output on error during attempt to convert store

Reviewed-by: Sage Weil <sage@inktank.com>

mon: Monitor: clearer output on error during attempt to convert store

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

ceph-disk: fix adjust_symlink() replace 'journal' with 'target'

Replace 'journal' variable with 'target' since journal doesn't
exist in this function.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: fix adjust_symlink() replace 'canonical' with 'path'

Replace 'canonical' variable with 'path' since canonical doesn't
exist in this function.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: there is no os.path.lstat use os.lstat

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: remove unused variable key from prepare_journal_dev()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph-disk: fix except to catch OSError

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>