Merge pull request #92 from jaharkes/content_length

Properly format Content-Length: header on 32-bit systems.

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

Properly format Content-Length: header on 32-bit systems.

- Promote len argument in dump_content_length to uint64_t.
- Make sure there is sufficient scratch space to format string.
- Use PRIu64 macro for formatting.

Signed-off-by: Jan Harkes <jaharkes@cs.cmu.edu>

Merge pull request #93 from jaharkes/range_requests

Avoid sending a success response when an error occurs.

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'wip-2733'

Add filestore_replica_fadvise config option default true

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Sam Just <sam.just@inktank.com>

osd: fadvise replica data don't use

Add transaction flag indicating if op is at a replica
After write has been sync'ed use posix_fadvise() to clear kernel cache

Feature: #2733

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Sam Just <sam.just@inktank.com>

Avoid sending a success response when an error occurs.

Functions called from RGWGetObj_ObjStore_S3::send_response_data may
change the value of the non-local variable 'ret'. But the response
relies on a local 'req_state' which copies ret at the start of the
function.

Right now none of the called functions actually changes ret so the
problem doesn't trigger, but to avoid future breakage it is safer
to not rely on the (early) copy of the ret variable.

Signed-off-by: Jan Harkes <jaharkes@cs.cmu.edu>

Fix FileStore to notice if filestore_flusher config is changed

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>

Merge branch 'wip-traceless'

Reviewed-by: Greg Farnum <greg@inktank.com>

client: move annoying traceless handling into verify_reply_trace()

Signed-off-by: Sage Weil <sage@inktank.com>

client: debug async cache invalidation

Signed-off-by: Sage Weil <sage@inktank.com>

mds: pass created_ino back to client on replayed requests

After an MDS restart, the client will resend uncommitted requests.  Use the
information we now have in the session_info_t to pass the created ino
back via the extra_bl payload in the reply.

Fixes: #4034
Signed-off-by: Sage Weil <sage@inktank.com>

mds: track created inos with completed request ids in session info

Along with each session completed request (tid), also track the created
ino (if any).  This will be used to pass back to the client when they
replay requests in the next patch.

Do not bother making this a backward compatible encoding.  The only
benefit is to allow ceph-mds code to run and then old mds code to run
after it.  Given all of the other incompat changes we *just* made, this
is highly unlikely and not worth the code clutter.  If we had spanned
more releases or a stable release the story would be different.

While we are here, inline the second add_completed_request() method variant
since there is only a single caller and having it overloaded somewhat
obscures what is going on.  This also avoids a duplicate lookup in the
session map in the have_session() check and then in the (old) helper.

Signed-off-by: Sage Weil <sage@inktank.com>

client: debug created ino

Signed-off-by: Sage Weil <sage@inktank.com>

client: unlink old_dentry on traceless rename reply

Signed-off-by: Sage Weil <sage@inktank.com>

client: force lookup on traceless reply

Call _do_lookup directly so that we avoid hitting the cache

Signed-off-by: Sage Weil <sage@inktank.com>

client: make _do_lookup take a const string

Signed-off-by: Sage Weil <sage@inktank.com>

mds: do not do traceless reply for open O_TRUNC requests

Even though these are "may_write" and may be a mutation, the MDS should not
fake out a traceless reply.  In a real failure, it looks like:

 - mds process request (mutation)
 - mds fails
 - client resends request
 - mds notes that it is O_TRUNC, and already committed, and proceeds with
   a regular open

And that is all fine.

But, that means that if we are trying to simulate this behavior without a
failure, then we should never do a traceless reply on an O_TRUNC request,
because the client will never see such a reply--it'll get a full open
request response instead.

Signed-off-by: Sage Weil <sage@inktank.com>

client: handle fh cleanup in ll_create() failure path

Release the fh if we fail the permissions check.

Signed-off-by: Sage Weil <sage@inktank.com>

client: root can write to any file

Signed-off-by: Sage Weil <sage@inktank.com>

client: clear I_COMPLETE on traceless reply for dentry request

If a request is against a dentry, and we get a traceless reply, clear
the directory I_COMPLETE flag on the parent directory because we can no
longer trust that our cache is complete.

It is possible we could do something a bit more intelligent here, but it
is not trivial because of racing requests, and traceless replies are
rare, so it's not worth the effort.

Signed-off-by: Sage Weil <sage@inktank.com>

client: pass ptarget from link, create, mkdir, symlink, setattr

Use the new make_request functionality to ensure that we properly handle
the re-lookup/getattr when we get a traceless reply back from the MDS.

Note that we have to do this on setattr() because ll_getattr() expects to
return the updated inode metadata to the caller.

Signed-off-by: Sage Weil <sage@inktank.com>

client: handle traceless replies from make_request()

Modify the generic make_request wrapper to retry lookups or getattrs on
requests when the ptarget pointer is specified but there is no trace in
the reply.

Refactor the _create() method to use this, effectively moving all of the
extra_bl code processing into make_request where it is generically
useful.

Signed-off-by: Sage Weil <sage@inktank.com>

vstart.sh: osd debug op order = true

Signed-off-by: Sage Weil <sage@inktank.com>

Missed adding rados_types.hpp to package

Caused by 3bd48cbbadb7908dd833ccde75359f085828fc5c
feature 4207 implementation

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>

osd: add ctor for clone_info

Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote-tracking branch 'gh/wip-expose-topo'

Reviewed-by: Sage Weil <sage@inktank.com>

Merge pull request #90 from grosskur/fix-debian-libsnappy

debian: require libsnappy-dev for ceph

debian: require libsnappy-dev for ceph

Debian builds are currently broken without this requirement.

libcephfs: return osd location from crush map

Adds ceph_get_osd_crush_location that returns a list of (type,name)
pairs for devices that exist between a given osd and the root of the
bucket hierarchy.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

release-process.rst:  Fix typos

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

client: expose extent/osd mapping

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

Merge branch 'wip-prepare'

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Alexandre Marangone <alexandre.marangone@inktank.com>
Tested-by: Tamil Muthamizhan <tamil.muthamizhan@inktank.com>

ceph-disk-prepare: move in-use checks to the top, before zap

Move the in-use checks to the very top, before we (say) zap!

Signed-off-by: Sage Weil <sage@inktank.com>

doc/release-notes: v0.58

Signed-off-by: Sage Weil <sage@inktank.com>

Update doc/radosgw/s3/python.rst

Note about creating conn object for non-ssl implementations.

Merge branch 'wip-4207'

Feature: #4207: osd/librados: add ops to list snaps for an object

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reported-by: Sam Just <sam.just@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

Merge branch 'next'

Add list_snaps() calls to snapshots test cases

Signed-off-by: David Zafman <david.zafman@inktank.com>

Add rados listsnaps command

Signed-off-by: David Zafman <david.zafman@inktank.com>

osd/librados: add op to list clones/snaps for an object

Returning snap_set_t with clone info
and snapshots in ascending order
Add clones with snapshots to obj_list_snap_response_t
New rados_types.hpp with snap_set_t/clone_info_t
Move snap_t to rados_types.hpp
Add generate_test_instances() and TYPE() to encoding/types.h

Feature: #4207

Signed-off-by: David Zafman <david.zafman@inktank.com>

Remove unused SnapContext in librados.hpp

Signed-off-by: David Zafman <david.zafman@inktank.com>

osd: Improve snapshot test

In SelfManagedSnapRollbackPP add some overlapping writes
New SelfManagedSnapOverlap creates overlapping writes

Signed-off-by: David Zafman <david.zafman@inktank.com>

osd: correct comment

SnapSet snaps are in descending order

Signed-off-by: David Zafman <david.zafman@inktank.com>

ceph-disk-prepare: verify device is not in use by device-mapper

Be nice and tell the user which devices/mappings are consuming the device,
too.

Signed-off-by: Sage Weil <sage@inktank.com>

ceph-disk-prepare: verify device is not mounted before using

Make sure the data and/or journal device(s) are not in use (mounted)
before using them.  Make room for additional "in-use" checks in the future.

Closes: #3256
Signed-off-by: Sage Weil <sage@inktank.com>

debian: require >= python2.6 for ceph as well

ceph-disk-prepare and ceph-disk-activate use the with statement,
str.format, and possibly other new features from python 2.6.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

ceph-disk-prepare: clean up stupid check for a digit

Signed-off-by: Sage Weil <sage@inktank.com>

ceph-disk-prepare: use os.path.realpath()

My janky symlink resolution is broken in various ways.

Signed-off-by: Sage Weil <sage@inktank.com>

v0.58

Add X-Python-Version >=2.6 to debian control file.

python-ceph complains when installed to debian squeeze about the 'with'
statement. Apparently installation tries to install the python-ceph package for
python 2.5, which does not support the 'with' statement natively.

Signed-off-by: Filippos Giannakos <philipgian@grnet.gr>

PG,ReplicatedPG: use pg_has_reset_since to discard old async events

Local async events are obsolete if the pg is deleting or if the
epoch at which they were created is prior to last_peering_reset.

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 62999ea2d362b32dfc86a15dcb03b06fb5cb0572)

PG::build_scrub_map: detect race with peering via last_peering_reset

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 67225339dc3d62d7fe5a32eec65d51e53e8d35bb)

ReplicatedPG::C_OSD_CommittedPushedObject: use intrusive_ptr for pg

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 04ee8f478bbd587a711d0668c471cfc5c1cab06c)

ReplicatedPG::C_OSD_CommittedPushedObject take epoch submitted

What we really care about is that the epoch in which the Context
was submitted is at complete() time >= last_peering_reset.

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit a01dea6af9aacf0614570ebb5fa161d9dde9b6b6)

init-ceph: fix --restart option

Reported-by: Bryan K. Wright <bkw1a@ayesha.phys.virginia.edu>
Signed-off-by: Sage Weil <sage@inktank.com>

doc: Updated year in footer.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge branch 'master' of https://github.com/ceph/ceph

doc: Added instructions for ensuring ceph config file and keyring prior to mounting ceph fuse.

fixes: #3796

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge pull request #89 from Elbandi/master

remove unused variable from makefile

jounal: disable aio

There is a deadlock issue in the aio code, see #4079.  Disable for the time
being.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote-tracking branch 'gh/wip-3484'

Reviewed-by: Sage Weil <sage@inktank.com>

Build: no longer need to use LEVELDB_INCLUDE

Signed-off-by: Andras Elso <elso.andras@gmail.com>

tools: ceph: add 'ceph df' usage

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

mon: Monitor: 'ceph df'

Fixes: #3484
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

README:  Add libsnappy-dev to build requirements.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

doc: missing blank line in radosgw/config-ref

Merge pull request #85 from dalgaaf/wip-da-spec-1

Fix package requirements to get sgdisk and merge some changes back from SUSE spec file.

Reviewed-by: Gary Lowell <gary.lowell@inktank.com>

Merge pull request #88 from nmtadam/master

librados.hpp typo

librados.hpp: replace paris with pairs

Signed-off-by: Adam Manzanares <nmtadam@gmail.com>
Working on a lua-rados library I noticed that paris was used
in the place of pairs in some comments.

doc: Instruction should have been noout, not nodown.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Fixed OSD pool config settings. Should be in [global], not [osd].

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

ceph.spec.in: Require gptfdisk for Suse, gdisk for otherwise

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>

Merge pull request #87 from dalgaaf/wip-da-sca-style-2

Fix more issues found via cppcheck

Reviewed-by: Sage Weil <sage@inktank.com>

osd/Watch.cc: init NotifyTimeoutCB::canceled in constructor with false

Initialize NotifyTimeoutCB::canceled in the constructor with
false as already done in onter cases.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ReplicatedPG.cc: remove unused variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ReplicatedPG.cc: remove consecutive return/break statement

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PG.cc: reduce scope of listed_collection in read_log_old()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/PG.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSD.cc: remove unused variable 'map<int64_t, int> pool_resize'

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSD.cc: reduce scope of "PG *pg" in OSD::handle_pg_query()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/OSD.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/IndexManager.cc: reduce scope of 'int r' in IndexManager::build_index()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/LevelDBStore.h: pass const string by reference to past_prefix()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: fix warning about comparing int with bool

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: remove identical else branch

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

os/FileStore.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

SimpleMessenger.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

msg/Pipe.cc: reduce scope of some variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/Paxos.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/PGMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/PGMap.cc: remove useless assigment to 'first', it's never used

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/OSDMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MonmapMonitor.cc: remove unused variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MonmapMonitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/Monitor.cc: reduce scope of reuse_caps in Monitor::_ms_dispatch()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/Monitor.cc: use static_cast instead of C-Style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mon/MonClient.cc: use static_cast instead of C-Style cast

Use static_cast<T*>(m) instead of C-Style cast, add missing
include needed for static_cast.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>