Move stat/*.log to the end of logrotate.conf

Logrotate ignores entries after a rule that doesn't match any files.

Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br>
Signed-off-by: Sage Weil <sage@newdream.net>

mds: fix file_excl assert

If we are in XSYN state and want to move to anything else, we must go via
EXCL, but we may not be loner anymore.  Weaken the file_excl() assert so we
don't crash.

Reported-by: Fyodor Ustinov <ufm@ufm.su>
Signed-off-by: Sage Weil <sage@newdream.net>

rgw: fix of users are created suspended

obsync: Depend on python-lxml on Debian derived platforms

Signed-off-by: Wido den Hollander <wido@widodh.nl>
Signed-off-by: Sage Weil <sage@newdream.net>

v0.30

debian: drop python-support version req

PG: assemble_backlog - fix comparison error, tail is non-inclusive

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

debian: obsync depends on python-xattr

Signed-off-by: Sage Weil <sage@newdream.net>

Merge remote branch 'origin/stable' into next

Conflicts:
src/librbd.cc

librados: fix end_block calculation for aio_read

Conflicts:

src/librbd.cc

librbd: track block_ofs instead of ofs in aio_sparse_read callback

Add ceph_argparse unit test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ceph_argparse_witharg: fix dashes in args

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: fix end_block calculation for aio_read

proflogger: only register one event for all loggers

If we already did logger_start(), don't schedule a new event.

Signed-off-by: Sage Weil <sage@newdream.net>

client: always use get_snap_context() accessor

There were a few places where we were using the member directly and not the
accessor, which rebuilds the internal value when needed.  This caused
inconsistent behavior based on whether debugging was enabled or not, since
we used the accessor to print the regenerated value.

Passes all snaps workunits with debugging off.

Signed-off-by: Sage Weil <sage@newdream.net>

librbd: track block_ofs instead of ofs in aio_sparse_read callback

Revert "debian: ceph-client-tools depends on librados2, librbd"

This reverts commit 96ef8a67bcf4a4a43f0a5c38224314abdd88a12c.

librbd: fix read_iterate for the case where block ofs is not 0

Conflicts:

src/librbd.cc

librbd: fix read_iterate for the case where block ofs is not 0

Merge remote branch 'origin/stable' into next

Conflicts:
src/librbd.cc

osd: fix merge

Signed-off-by: Sage Weil <sage@newdream.net>

librbd: sparse read cleanup, fixes

Conflicts:

src/librbd.cc

librbd: fix and cleanup a bit read_iterate

Merge branch 'stable' into next

Conflicts:
src/osd/OSD.cc

osd: do misdirected request checks before pg active test

We can process+discard a misdirected or invalid request without the pg
being active.  Reorder a few other checks too.

Signed-off-by: Sage Weil <sage@newdream.net>

mon: fix log state trimming

We need to stash last_consumed in order for trimming to work.

Signed-off-by: Sage Weil <sage@newdream.net>

filestore: fix fiemap

rbd: fix read_iterate for sparse read

osd: warn instead of error if cluster addr specified but not public addr

Signed-off-by: Sage Weil <sage@newdream.net>

mds: avoid EMetaBlob::fullbit copying with shared_ptr

The fullbit subclass has a pointer, making it unsafe to copy around.  Fix
by adding a undefined copy constructor and operator, and using a shared_ptr
to reference it in the containing STL list<>.

Tested-by: Sam Lang <samlang@gmail.com>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

encoding: add list<shared_ptr<T>> macros

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

debian: ceph-client-tools depends on librados2, librbd

It contains rados and rbd command line tools.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge remote branch 'origin/stable'

librados: return value on non void function

common: fix descriptinos for -i, -n too

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

update clitests with new usage

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: fix generic usage for -D vs -d

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rgw: suspend/enable buckets through pool async api

librados: add async pool op operations

qa: organize workunits

Signed-off-by: Sage Weil <sage@newdream.net>

secret.c: use safe_read when appropriate

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: fix use-after-free on ESession replay

Reported by: Sam Lang <samlang@gmail.com>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: make snap-rm-diff workunit clean up

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: wget -q

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rgw: remove stray semicolon

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: cast RGW_SUSPENDED_USER_AUID

Signed-off-by: Sage Weil <sage@newdream.net>

mds: CDir::log_mark_dirty needs to flush before waiting

We need to flush before waiting or else our completion (and mark_diryt())
will be out of order with respect to other dir updates.

Signed-off-by: Sage Weil <sage@newdream.net>

mount.ceph: specify full path for /sbin/modprobe

Fixes: #1188
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

radosgw_admin: fix cli test

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'stable'

osd: fix find_object_context debug output

Erroneously said get_object_context, confusing developers everywhere.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix snap_trimmer obc ref

As of fc4cc39 new_repop claims the obc reference (from the clonerange
branch) but this caller didn't get fixed.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: when prodding head inode on snapped behalf, avoid XSYN

If we try to rdlock a snapped inode in snap->sync state, we need to prod
the head inode into SYNC to get the data flushed.  Make sure we avoid
XSYN by passing as_anon=true to rdlock_start(head).

Also fix the waiting code.  If it succeeds, continue around the loop.  If
not, the rdlock_start will add the waiter for us--don't do it again!

Fixes workunit snaps/snaptest-snap-rm-cmp.sh.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: fix transitions away from XSYN state

If we are in XSYN, we need to transition first to EXCL before moving on.
Simply going to the other unstable state isn't enough, because if gather
is zero at the end of the method we end up jumping directly to the wrong
state without passing EXCL.   Instead, call file_excl(), and continue only
if we got there immediately.

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: user suspension

Merge branch 'rgw-multipart'

Merge remote-tracking branch 'origin/master' into rgw-multipart

Conflicts:
src/os/FileStore.cc
src/rgw/rgw_admin.cc
src/rgw/rgw_fs.h
src/rgw/rgw_log.cc
src/rgw/rgw_rados.cc
src/rgw/rgw_user.cc
src/rgw/rgw_user.h

qa: pjd must run as root

rgw: get multipart list marker to work

v0.29.1

Signed-off-by: Sage Weil <sage@newdream.net>

Merge remote branch 'origin/mds_xsyn'

rgw: some multipart cleanups, fixes

Merge remote branch 'origin/stable'

librbd: fix block_completion race condition

If block completions finished before all of them were scheduled, the
pending_count could reach 0, and the callback could be called more
than once. This caused the reference counting to be off, so the next
call to the callback passed an invalid pointer, resulting in crashes
like this (or more mysterious ones):

common/Mutex.h: In function 'void Mutex::Lock(bool)', in thread '0x7f5f37e62700'
common/Mutex.h: 118: FAILED assert(r == 0)
 1: (librbd::RBD::AioCompletion::get_return_value()+0x19c) [0x7f5f408835bc]
 2: /usr/bin/kvm() [0x4629fd]
 3: (librbd::AioCompletion::complete_block(librbd::AioBlockCompletion*, long)+0x13c) [0x7f5f4088ab5c]
 4: (librbd::rados_aio_sparse_read_cb(void*, void*)+0x8d) [0x7f5f4088b40d]
 5: (librados::RadosClient::C_aio_sparse_read_Ack::finish(int)+0x12e) [0x7f5f4050beee]
 6: (Objecter::handle_osd_op_reply(MOSDOpReply*)+0x783) [0x7f5f40517863]
 7: (librados::RadosClient::_dispatch(Message*)+0x3c) [0x7f5f404ffe8c]
 8: (librados::RadosClient::ms_dispatch(Message*)+0x33) [0x7f5f404fff53]
 9: (SimpleMessenger::dispatch_entry()+0x69d) [0x7f5f405bf5bd]
 10: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x7f5f4050052c]
 11: (()+0x6d8c) [0x7f5f40c9dd8c]
 12: (clone()+0x6d) [0x7f5f3e0b904d]

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rgw: implement list multiparts

doesn't work completely, getting there

librbd: add AioCompletion debugging

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

uclient: path_walk should notice when it gets absolute paths.

Previously it assumed everything was relative, but we need to handle
absolute paths on occasion, and already have the code to do so!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

vstart: put .journal outside of osd data dir

This prevents it from getting cleaned up during mkfs.  Then we can make it
a symlink, or at the very least avoid deleting + reallocating it.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: Make snaptest-git-ceph.sh clean up after itself.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

rgw: fix abort multipart upload

auth: remove CephCryptoManager

It doesn't do anything.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librbd: fix AioCompletion race condition

If block completions finished before all of them where scheduled, the
pending_count would reach 0, and the AioCompletion would be destroyed
since the refcount would also reach 0. To fix this, hold a reference
to the AioCompletion while scheduling more block completions.

This caused a crash like this when running qemu:

1: ceph::__ceph_assert_fail (assertion=<value optimized out>, file=<value optimized out>, line=<value optimized out>, func=<value optimized out>) at common/assert.cc:86
2: Lock (ictx=0x7f83d0001f00, off=5368705024, len=0, buf=0x2fac000 "", c=0x2cebe00) at common/Mutex.h:118
3: add_block_completion (ictx=0x7f83d0001f00, off=5368705024, len=0, buf=0x2fac000 "", c=0x2cebe00) at librbd.cc:159
4: librbd::aio_read (ictx=0x7f83d0001f00, off=5368705024, len=0, buf=0x2fac000 "", c=0x2cebe00) at librbd.cc:1368
5: rbd_aio_rw_vector (bs=<value optimized out>, sector_num=10485752, qiov=<value optimized out>, nb_sectors=<value optimized out>, cb=<value optimized out>, opaque=<value optimized out>, write=0) at block/rbd.c:626
6: qemu_rbd_aio_readv (bs=<value optimized out>, sector_num=<value optimized out>, qiov=<value optimized out>, nb_sectors=<value optimized out>, cb=<value optimized out>, opaque=<value optimized out>) at block/rbd.c:649
7: bdrv_aio_readv (bs=0x2ce92e0, sector_num=10485752, qiov=0x2de08f0, nb_sectors=8, cb=<value optimized out>, opaque=<value optimized out>)

Fixes: #998
Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

monmaptool: implement --help

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/bufferlist: don't alloc large array on stack

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

.gitignore man/Makefile

blech

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: do not use automake for workunit makefiles

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa/workunits/direct_io: fit .gitignore

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

radosgw_admin: fix clitest help

crbdnamer: man page

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

crbdnamer: +x

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: radosgw_admin: fix clitest

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: direct_io: fix warnings

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

configure: check for boost statechart header

Partially fixes: #1124
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

radosgw_admin: log show can dump info in xml, json

radosgw_admin: fix log show (with new poolid param)

radosgw_admin: pool create command

Merge branch 'wip-workunits'

librados: use empty(), not size()

Signed-off-by: Sage Weil <sage@newdream.net>

qa: test_sync_io: zero buffers prior to read

Be extra paranoid to ensure we read back the correct data and didn't
just find it in undefined memory.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: test_sync_io improvements

Exit (without removing foo) on error.

Allow read- or write-only operation.

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: log pool id, and store/retrieve pool id info

librados: expose pool id

Conflicts:

src/librados.cc

qa: fix test_shorT_dio_read

No args, hardcode filename.

Signed-off-by: Sage Weil <sage@newdream.net>

PG: clear scrub_received_maps in scrub_clear_state

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

qa: fix loopall, runallonce scripts

Run anything executable.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: move compiled workunits into organized directory

No need for a separate qa/src/ dir; just put them where we want them.  We
need to ignore non-executable files when doing the bulk runs anyway.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: add short directio read test

This tests for the bug fixed by linux commit:c3cd628.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: add henry's direct io tests

Not sure what the second one is testing, actually, but including it anyway.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: include $basedir/src/ (qa/src/) in path

This lets the workunits find stuff build in qa/src/, in case you are
running using these (old) scripts.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>