Merge branch 'wip-4207'

Feature: #4207: osd/librados: add ops to list snaps for an object

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reported-by: Sam Just <sam.just@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

Merge branch 'next'

Add list_snaps() calls to snapshots test cases

Signed-off-by: David Zafman <david.zafman@inktank.com>

Add rados listsnaps command

Signed-off-by: David Zafman <david.zafman@inktank.com>

osd/librados: add op to list clones/snaps for an object

Returning snap_set_t with clone info
and snapshots in ascending order
Add clones with snapshots to obj_list_snap_response_t
New rados_types.hpp with snap_set_t/clone_info_t
Move snap_t to rados_types.hpp
Add generate_test_instances() and TYPE() to encoding/types.h

Feature: #4207

Signed-off-by: David Zafman <david.zafman@inktank.com>

Remove unused SnapContext in librados.hpp

Signed-off-by: David Zafman <david.zafman@inktank.com>

osd: Improve snapshot test

In SelfManagedSnapRollbackPP add some overlapping writes
New SelfManagedSnapOverlap creates overlapping writes

Signed-off-by: David Zafman <david.zafman@inktank.com>

osd: correct comment

SnapSet snaps are in descending order

Signed-off-by: David Zafman <david.zafman@inktank.com>

debian: require >= python2.6 for ceph as well

ceph-disk-prepare and ceph-disk-activate use the with statement,
str.format, and possibly other new features from python 2.6.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

v0.58

Add X-Python-Version >=2.6 to debian control file.

python-ceph complains when installed to debian squeeze about the 'with'
statement. Apparently installation tries to install the python-ceph package for
python 2.5, which does not support the 'with' statement natively.

Signed-off-by: Filippos Giannakos <philipgian@grnet.gr>

PG,ReplicatedPG: use pg_has_reset_since to discard old async events

Local async events are obsolete if the pg is deleting or if the
epoch at which they were created is prior to last_peering_reset.

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 62999ea2d362b32dfc86a15dcb03b06fb5cb0572)

PG::build_scrub_map: detect race with peering via last_peering_reset

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 67225339dc3d62d7fe5a32eec65d51e53e8d35bb)

ReplicatedPG::C_OSD_CommittedPushedObject: use intrusive_ptr for pg

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 04ee8f478bbd587a711d0668c471cfc5c1cab06c)

ReplicatedPG::C_OSD_CommittedPushedObject take epoch submitted

What we really care about is that the epoch in which the Context
was submitted is at complete() time >= last_peering_reset.

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit a01dea6af9aacf0614570ebb5fa161d9dde9b6b6)

init-ceph: fix --restart option

Reported-by: Bryan K. Wright <bkw1a@ayesha.phys.virginia.edu>
Signed-off-by: Sage Weil <sage@inktank.com>

doc: Updated year in footer.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Added instructions for ensuring ceph config file and keyring prior to mounting ceph fuse.

fixes: #3796

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #89 from Elbandi/master

remove unused variable from makefile

jounal: disable aio

There is a deadlock issue in the aio code, see #4079.  Disable for the time
being.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote-tracking branch 'gh/wip-3484'

Reviewed-by: Sage Weil <sage@inktank.com>

Build: no longer need to use LEVELDB_INCLUDE

Signed-off-by: Andras Elso <elso.andras@gmail.com>

tools: ceph: add 'ceph df' usage

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: Monitor: 'ceph df'

Fixes: #3484
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

README:  Add libsnappy-dev to build requirements.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

doc: missing blank line in radosgw/config-ref

Merge pull request #85 from dalgaaf/wip-da-spec-1

Fix package requirements to get sgdisk and merge some changes back from SUSE spec file.

Reviewed-by: Gary Lowell <gary.lowell@inktank.com>

Merge pull request #88 from nmtadam/master

librados.hpp typo

librados.hpp: replace paris with pairs

Signed-off-by: Adam Manzanares <nmtadam@gmail.com>
Working on a lua-rados library I noticed that paris was used
in the place of pairs in some comments.

doc: Instruction should have been noout, not nodown.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Fixed OSD pool config settings. Should be in [global], not [osd].

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

ceph.spec.in: Require gptfdisk for Suse, gdisk for otherwise

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

Merge pull request #87 from dalgaaf/wip-da-sca-style-2

Fix more issues found via cppcheck

Reviewed-by: Sage Weil <sage@inktank.com>

osd/Watch.cc: init NotifyTimeoutCB::canceled in constructor with false

Initialize NotifyTimeoutCB::canceled in the constructor with
false as already done in onter cases.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ReplicatedPG.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ReplicatedPG.cc: remove consecutive return/break statement

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PG.cc: reduce scope of listed_collection in read_log_old()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PG.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSD.cc: remove unused variable 'map<int64_t, int> pool_resize'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSD.cc: reduce scope of "PG *pg" in OSD::handle_pg_query()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSD.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/IndexManager.cc: reduce scope of 'int r' in IndexManager::build_index()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/LevelDBStore.h: pass const string by reference to past_prefix()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: fix warning about comparing int with bool

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: remove identical else branch

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SimpleMessenger.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

msg/Pipe.cc: reduce scope of some variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/Paxos.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/PGMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/PGMap.cc: remove useless assigment to 'first', it's never used

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/OSDMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MonmapMonitor.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MonmapMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/Monitor.cc: reduce scope of reuse_caps in Monitor::_ms_dispatch()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/Monitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MonClient.cc: use static_cast instead of C-Style cast

Use static_cast<T*>(m) instead of C-Style cast, add missing
include needed for static_cast.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MDSMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/LogMonitor.cc: move unused variable into related code block

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/LogMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/Elector.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/AuthMonitor.cc: remove unused 'std::stringstream ss'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/flock.cc: remove unused variable 'crossed_waiting_locks'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Server.cc: reduce scope of some variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Server.cc: : use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Migrator.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDS.cc: remove unused variable 'oldresolve'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDS.cc: : use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDCache.cc: remove unused variables recover_q and check_q

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDCache.cc: reduce scope of 'in' in MDCache::open_root_inode()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDCache.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDBalancer.h: init some double variables in constructor

Initialize double variables my_load and target_load with 0.0 in
the constructor.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDBalancer.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ESession.h: initialze 'open' in the constructor with false

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/LogEvent.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Locker.cc: reduce scope of 'pool' in Locker::issue_client_lease()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph.spec.in: merge fdupes support from SUSE packages

Use fdupes to avoid duplicated files as done in the SUSE
packages. Since it doesn't work as expected on Fedora/RHEL
do this only on SUSE based distros.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph.spec.in: add on non-RHEL sharutils to 'BuildRequires:'

Add sharutils to 'BuildRequires:', needed for configure.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph.spec.in: fix 'Requires:' to get sgdisk into the system

Fix package Requirements to get sgdisk. On RHEL/CentOS/Fedora the
package is gdisk. On SUSE/openSUSE < 12.2 it's scsirastools on
openSUSE >= 12.2 it's in gptfdisk.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/Locker.cc: use static_cast instead of C-Style cast

Use static_cast<T>() instead of C-Style cast.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/CDir.h: use static_cast instead of C-Style cast

Use static_cast<T>() instead of C-Style cast.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ceph.spec.in: gdisk for for rpm based systems instead of gptfdisk.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

Merge branch 'next'

msgr: drop messages on cons with CLOSED Pipes

Back in commit 6339c5d43974f4b495f15d199e01a141e74235f5, we tried to make
this deal with a race between a faulting pipe and new messages being
queued.  The sequence is

- fault starts on pipe
- fault drops pipe_lock to unregister the pipe
- user (objecter) queues new message on the con
- submit_message reopens a Pipe (due to this bug)
- the message managed to make it out over the wire
- fault finishes faulting, calls ms_reset
- user (objecter) closes the con
- user (objecter) resends everything

It appears as though the previous patch *meant* to drop *m on the floor in
this case, which is what this patch does.  And that fixes the crash I am
hitting; see #4271.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

osd: add 'osd debug op order' to track and assert order of client ops

Keep a big fat slow map in memory to ensure that client ops are applied to
objects in the order the clien sent them.  This will make it easier to
catch ordering bugs due to requeuing as soon as they happen, when the logs
are easier to wade through, and not later when the client notices something
is awry.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

osd: include osdmap epoch in osd_op message operator<<

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

osd: useful debugging for when we drop requests

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>

crush-map.rst: fix chooseleaf syntax in ssd example

Fixes: 4304
Backport: bobtail
Reviewed-by: Greg Farnum <greg@inktank.com>
Signed-off-by: Samuel Just <sam.just@inktank.com>

Merge pull request #84 from dalgaaf/wip-da-sca-style

Some more smaller fixes

Reviewed-by: Sage Weil <sage@inktank.com>

init-ceph: kick off ceph-create-keys when starting mons

We don't have a particularly tidy way to kick off ceph-create-keys from
sysvinit.  So, do something simple, ugly, and functional: if we are
starting a mon, and it looks like a 'new' type that is in /var/lib/ceph,
then start ceph-create-keys too.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

init-ceph: fix default admin socket path used for status

Signed-off-by: Sage Weil <sage@inktank.com>

osd: show retry attempt in MOSDOp operator<<

Signed-off-by: Sage Weil <sage@inktank.com>

msg/Pipe: allow tuning of TCP receive buffer size

Performance tests on high-end machines have indicated the Linux autotuning
of the receive buffer sizes can cause throughput collapse.  See bug
#2100, and this email discussion:

   http://marc.info/?l=ceph-devel&m=133009796706284&w=2

Initially default to 0, which leaves us with the default.  We may adjust
the default in the future.

Tested-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

msg/Pipe: move setting of socket options into a common method

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

FileJournal::wrap_read_bl: adjust pos before returning

Otherwise, we may feed an offset past the end of the journal to
check_header in read_entry and incorrectly determine that the entry is
corrupt.

Fixes: 4296
Backport: bobtail
Backport: argonaut
Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Samuel Just <sam.just@inktank.com>

librbd/internal.cc: remove unused variable total_write

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/internal.cc: reduce scope of 'int r'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>