run_cmd: ret empty string on success;err otherwise

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/atomic cleanup

* Don't allow copying of class atomic_t.
* Remove common/Spinlock.h because it's unecessary
* SimpleMessenger: use atomic var for qlen

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/atomic.h: whitespace cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

bufferlist: get rid of derr usage

bufferlist doesn't need to use dout/derr. Only one function used it, and
that was only to return an error string.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/Completion: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ObjectStore: clean up includes a little bit

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/Context.h: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/MemoryModel: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/Finisher: use ldout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

lockdep: code cleanup and de-globalization

common/Mutex.cc: this is a duplicate of the code in common/lockdep.cc.
Delete the duplicate code.

common/lockdep.cc: require the programmer to register a CephContext to
use with lockdep. If we don't have one, we don't print anything out.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Thread.cc: clean up

Don't implement everything in the header file. It slows down compilation
and usually runtime.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Create de-globalized versions of the dout macro

The de-globalized versions are available in dout.h
Versions relying on globals continue to be available in debug.h

Split md_config_obs_t off into common/config_obs.h. This will make it
possible for DoutStreambuf.h to include just that structure, without
pulling in all of config.h.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Remove cdout

cdout just duplicates the functionality of dout.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

lockdep: add lockdep_register_ceph_context,cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

signal: const cleanup, don't use derr

In signal.cc: const cleanup, don't use derr, add a comment.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw: put_obj_data uses aio

This is just a trivial fix, making the etag calculation done while the io
is pending. Still some room for improvement.

rgw: put_obj_data doesn't need mtime

filestore: fix fiemap

rbd: fix read_iterate for sparse read

qa: tiobench workunit: remove timeout

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge remote branch 'origin/stable'

librados: return value on non void function

common: fix descriptinos for -i, -n too

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

update clitests with new usage

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: fix generic usage for -D vs -d

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rgw: suspend/enable buckets through pool async api

librados: add async pool op operations

qa: organize workunits

Signed-off-by: Sage Weil <sage@newdream.net>

secret.c: use safe_read when appropriate

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: fix use-after-free on ESession replay

Reported by: Sam Lang <samlang@gmail.com>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: make snap-rm-diff workunit clean up

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: wget -q

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rgw: remove stray semicolon

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: cast RGW_SUSPENDED_USER_AUID

Signed-off-by: Sage Weil <sage@newdream.net>

mds: CDir::log_mark_dirty needs to flush before waiting

We need to flush before waiting or else our completion (and mark_diryt())
will be out of order with respect to other dir updates.

Signed-off-by: Sage Weil <sage@newdream.net>

mount.ceph: specify full path for /sbin/modprobe

Fixes: #1188
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

radosgw_admin: fix cli test

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'stable'

osd: fix find_object_context debug output

Erroneously said get_object_context, confusing developers everywhere.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix snap_trimmer obc ref

As of fc4cc39 new_repop claims the obc reference (from the clonerange
branch) but this caller didn't get fixed.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: when prodding head inode on snapped behalf, avoid XSYN

If we try to rdlock a snapped inode in snap->sync state, we need to prod
the head inode into SYNC to get the data flushed.  Make sure we avoid
XSYN by passing as_anon=true to rdlock_start(head).

Also fix the waiting code.  If it succeeds, continue around the loop.  If
not, the rdlock_start will add the waiter for us--don't do it again!

Fixes workunit snaps/snaptest-snap-rm-cmp.sh.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: fix transitions away from XSYN state

If we are in XSYN, we need to transition first to EXCL before moving on.
Simply going to the other unstable state isn't enough, because if gather
is zero at the end of the method we end up jumping directly to the wrong
state without passing EXCL.   Instead, call file_excl(), and continue only
if we got there immediately.

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: user suspension

Merge branch 'rgw-multipart'

Merge remote-tracking branch 'origin/master' into rgw-multipart

Conflicts:
src/os/FileStore.cc
src/rgw/rgw_admin.cc
src/rgw/rgw_fs.h
src/rgw/rgw_log.cc
src/rgw/rgw_rados.cc
src/rgw/rgw_user.cc
src/rgw/rgw_user.h

qa: pjd must run as root

rgw: get multipart list marker to work

v0.29.1

Signed-off-by: Sage Weil <sage@newdream.net>

Merge remote branch 'origin/mds_xsyn'

rgw: some multipart cleanups, fixes

Merge remote branch 'origin/stable'

librbd: fix block_completion race condition

If block completions finished before all of them were scheduled, the
pending_count could reach 0, and the callback could be called more
than once. This caused the reference counting to be off, so the next
call to the callback passed an invalid pointer, resulting in crashes
like this (or more mysterious ones):

common/Mutex.h: In function 'void Mutex::Lock(bool)', in thread '0x7f5f37e62700'
common/Mutex.h: 118: FAILED assert(r == 0)
 1: (librbd::RBD::AioCompletion::get_return_value()+0x19c) [0x7f5f408835bc]
 2: /usr/bin/kvm() [0x4629fd]
 3: (librbd::AioCompletion::complete_block(librbd::AioBlockCompletion*, long)+0x13c) [0x7f5f4088ab5c]
 4: (librbd::rados_aio_sparse_read_cb(void*, void*)+0x8d) [0x7f5f4088b40d]
 5: (librados::RadosClient::C_aio_sparse_read_Ack::finish(int)+0x12e) [0x7f5f4050beee]
 6: (Objecter::handle_osd_op_reply(MOSDOpReply*)+0x783) [0x7f5f40517863]
 7: (librados::RadosClient::_dispatch(Message*)+0x3c) [0x7f5f404ffe8c]
 8: (librados::RadosClient::ms_dispatch(Message*)+0x33) [0x7f5f404fff53]
 9: (SimpleMessenger::dispatch_entry()+0x69d) [0x7f5f405bf5bd]
 10: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x7f5f4050052c]
 11: (()+0x6d8c) [0x7f5f40c9dd8c]
 12: (clone()+0x6d) [0x7f5f3e0b904d]

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rgw: implement list multiparts

doesn't work completely, getting there

librbd: add AioCompletion debugging

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

uclient: path_walk should notice when it gets absolute paths.

Previously it assumed everything was relative, but we need to handle
absolute paths on occasion, and already have the code to do so!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

vstart: put .journal outside of osd data dir

This prevents it from getting cleaned up during mkfs.  Then we can make it
a symlink, or at the very least avoid deleting + reallocating it.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: Make snaptest-git-ceph.sh clean up after itself.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

rgw: fix abort multipart upload

auth: remove CephCryptoManager

It doesn't do anything.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librbd: fix AioCompletion race condition

If block completions finished before all of them where scheduled, the
pending_count would reach 0, and the AioCompletion would be destroyed
since the refcount would also reach 0. To fix this, hold a reference
to the AioCompletion while scheduling more block completions.

This caused a crash like this when running qemu:

1: ceph::__ceph_assert_fail (assertion=<value optimized out>, file=<value optimized out>, line=<value optimized out>, func=<value optimized out>) at common/assert.cc:86
2: Lock (ictx=0x7f83d0001f00, off=5368705024, len=0, buf=0x2fac000 "", c=0x2cebe00) at common/Mutex.h:118
3: add_block_completion (ictx=0x7f83d0001f00, off=5368705024, len=0, buf=0x2fac000 "", c=0x2cebe00) at librbd.cc:159
4: librbd::aio_read (ictx=0x7f83d0001f00, off=5368705024, len=0, buf=0x2fac000 "", c=0x2cebe00) at librbd.cc:1368
5: rbd_aio_rw_vector (bs=<value optimized out>, sector_num=10485752, qiov=<value optimized out>, nb_sectors=<value optimized out>, cb=<value optimized out>, opaque=<value optimized out>, write=0) at block/rbd.c:626
6: qemu_rbd_aio_readv (bs=<value optimized out>, sector_num=<value optimized out>, qiov=<value optimized out>, nb_sectors=<value optimized out>, cb=<value optimized out>, opaque=<value optimized out>) at block/rbd.c:649
7: bdrv_aio_readv (bs=0x2ce92e0, sector_num=10485752, qiov=0x2de08f0, nb_sectors=8, cb=<value optimized out>, opaque=<value optimized out>)

Fixes: #998
Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

monmaptool: implement --help

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/bufferlist: don't alloc large array on stack

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

.gitignore man/Makefile

blech

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: do not use automake for workunit makefiles

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa/workunits/direct_io: fit .gitignore

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

radosgw_admin: fix clitest help

crbdnamer: man page

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

crbdnamer: +x

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: radosgw_admin: fix clitest

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: direct_io: fix warnings

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

configure: check for boost statechart header

Partially fixes: #1124
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

radosgw_admin: log show can dump info in xml, json

radosgw_admin: fix log show (with new poolid param)

radosgw_admin: pool create command

Merge branch 'wip-workunits'

librados: use empty(), not size()

Signed-off-by: Sage Weil <sage@newdream.net>

qa: test_sync_io: zero buffers prior to read

Be extra paranoid to ensure we read back the correct data and didn't
just find it in undefined memory.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: test_sync_io improvements

Exit (without removing foo) on error.

Allow read- or write-only operation.

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: log pool id, and store/retrieve pool id info

librados: expose pool id

Conflicts:

src/librados.cc

qa: fix test_shorT_dio_read

No args, hardcode filename.

Signed-off-by: Sage Weil <sage@newdream.net>

PG: clear scrub_received_maps in scrub_clear_state

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

qa: fix loopall, runallonce scripts

Run anything executable.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: move compiled workunits into organized directory

No need for a separate qa/src/ dir; just put them where we want them.  We
need to ignore non-executable files when doing the bulk runs anyway.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: add short directio read test

This tests for the bug fixed by linux commit:c3cd628.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: add henry's direct io tests

Not sure what the second one is testing, actually, but including it anyway.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: include $basedir/src/ (qa/src/) in path

This lets the workunits find stuff build in qa/src/, in case you are
running using these (old) scripts.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: add direct and sync io workunits

Just run the test programs in ../src

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: expand test_sync_io

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: organize workunits a bit

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: move compiled items into qa/src/

osd: fix head_exists fix

Needs to default to false, or else deletions don't work.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: don't update snapset head_exists unless full transaction succeeds

Make the update at the end of make_writeable, along with the other snapset
updates.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix make_writeable

Look at the ObjectState to determine if the head exists, not the snapset
(which doesn't reflect the prior object state).

Signed-off-by: Sage Weil <sage@newdream.net>

monc: protect msg source check with monc_lock

Push the locking up into ms_dispatch, so that we protect the source check
and don't race with session reopens.  Also simplifies things a bit.

Fixes: #1151
Signed-off-by: Sage Weil <sage@newdream.net>

Merge branch 'mds_issue_caps'

Merge remote branch 'origin/stable'

Conflicts:
src/dumpjournal.cc

Merge remote branch 'origin/deb-pybind'

debian: Move python-ceph into section python.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

debian: Strip and do -dbg magic for librgw too.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>