mds: fix discover tid assignment

Hmm!

Signed-off-by: Sage Weil <sage@newdream.net>

vstart.sh: remove cruft

Signed-off-by: Sage Weil <sage@newdream.net>

mon: fix standby-replay assignment (again)

Only assign a random node to standby-replay if they are marked as
STANDBY_ANY.

Signed-off-by: Sage Weil <sage@newdream.net>

vstart.sh: set up pairs for each rank when -s is on

Signed-off-by: Sage Weil <sage@newdream.net>

mon: rework assignment of standby-replay, expansion nodes

Signed-off-by: Sage Weil <sage@newdream.net>

mon: fix standby-replay assignment logic

Assign a standby-replay at any time based on rank, name, or no preference.
Previously this could only happen when the MDS first started, and we would
fail if the target MDS wasn't followable at that point in time.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge branch 'osd_trans'

Merge remote branch 'origin/stable'

osd: move watch/notify effects out of do_osd_ops

Apply watch/notify side effects in do_osd_op_effects() only if the
transaction will succeed.

Signed-off-by: Sage Weil <sage@newdream.net>

obsync: implement RadosStore

Implement RadosStore, a storage backend which accesses librados
directly, without going through RGW (Rados GateWay).

This version is still very preliminary because ACLs aren't supported.
We need ACLs even to do things like properly create buckets.
Instead, this version has ACL_HACK, which is just for testing purposes.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: mention invalid snapc in log

Signed-off-by: Sage Weil <sage@newdream.net>

osd: include (some) osd op flags in MOSDOp print method

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: add RWORDERED osd op flag

Order this op wrt reads the same way a read-modify-write would be.
(Otherwise we may get a fast/stale read result on a not-yet-complete
write.)

This fixes a problem where the Filer was marking a probe stat as a write
to get this same effect, but the OSD would EINVAL if it was a snapped
object (which happens in certain cases where the MDS is recovering the
file size of a snapped file).

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

radostool: fix getxattr / setxattr return code

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rbd: make showmapped output a bit prettier

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rbd: showmapped

Show mapped rbd devices.

Fixes: #1024
Signed-off-by: Sage Weil <sage@newdream.net>

pybind-rados: fix Ioctx::close

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: only include head dentries in check_rstats() rstat check

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: move ObjectState side effects out of do_osd_ops

We want to be able to handle a failure mid-way through an OSDOp
transaction and bail out with no side effects.  This patch

 * puts an ObjectState new_obs in the OoContext that modifications go in
 * only applies if it the transaction is a success
 * only does make_writeable (at the end!) if the transaction is a success

There are still side effects with the watch/notify stuff, though.

Signed-off-by: Sage Weil <sage@newdream.net>

objectstore: implement Transaction::swap()

Signed-off-by: Sage Weil <sage@newdream.net>

objectstore: transaction::append()

Combine two transactions into one.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

buffer: use std::swap

Signed-off-by: Sage Weil <sage@newdream.net>

test-obsync.py: add tests with --no-preserve-acls

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: remove obsolete noop cruft

The noop branching is all dead code.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: move snapset_context into ObjectContext from ObjectState

ObjectState is now just static info (object_info_t and bool exists).

Signed-off-by: Sage Weil <sage@newdream.net>

objectstore: drop decode support for <= v0.19 encoded transactions

This only affects online upgrade or journal replay of v0.19 generated
transactions.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mon: fix pg stat summary

Had the pg state counts in there twice.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: undump-journal

Signed-off-by: Sage Weil <sage@newdream.net>

buffer: break out read_fd method

Read N bytes from the provided fd into the bufferlist.

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: check if bucket is empty before removing it

obsync: another fix for --no-preserve-acls

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

obsync: fix bug in --no-preserve-acls

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: Don't twiddle lock states in the middle of an import.

This should have gone in a028c8954ca240ec9a12682678aaee02eb507ae3.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Merge branch 'stable'

Conflicts:
src/mds/MDLog.cc
src/osdc/Journaler.cc
src/osdc/Journaler.h

rgw: send content length on put operation

mds: only move the journaler expire_pos forward

We were seeing weird trim errors because expire_pos was getting moved
backwards after a standby-replay -> replay transition.  Make sure the two
places that update the expire_pos only move it forward--never backward.

Fixes: #1023
Signed-off-by: Sage Weil <sage@newdream.net>

mds: always trim standby segments after rereading the head

When we re-read the head we may get an expire_pos that has moved forward in
time.  That is the appropriate time to trim segments during standby-replay.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: only write head once after expiring logsegments

Signed-off-by: Sage Weil <sage@newdream.net>

mds: small journaler cleanups

Signed-off-by: Sage Weil <sage@newdream.net>

journaler: separate out trimmed_pos setter

Signed-off-by: Sage Weil <sage@newdream.net>

mds: wait for blacklisting osdmap on standby-replay -> replay final pass

Signed-off-by: Sage Weil <sage@newdream.net>

librados python binding: always use 64-bit offsets

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

pybind-rados: fix read API

Read should have the arguments in the same order as pread.
Add a test for reading from a nonzero offset.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

hypertable: update bindings to use new libceph API

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

hadoop: re-add ceph_set_default_preferred_pg

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libceph: re-add set_default_preferred_pg

Hadoop needs it.

Signed-off-by: Sage Weil <sage@newdream.net>

hadoop: get hadoop bindings to build again

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Client.cc: fix mode/flags confusion

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libceph: Remove *_precise

In Linux, struct stat includes nanosecond resolution already. No need to
create a separate stat structure that duplicates the fields of struct stat.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

client, libceph: clean up layout methods

Signed-off-by: Sage Weil <sage@newdream.net>

CephFSInterface: don't include Timer.h

We don't use it.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

hadoop: convert to new libceph interface

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

do_autogen: add -H for --with-hadoop

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

do_autogen.sh: don't clear CFLAGS, CXXFLAGS

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

testceph: add more tests

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libceph: fix some bugs

Fix some bugs in libceph and testceph.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libceph: avoid DIR*, combine connect and mount

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

hadoop: whitespace fixes

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Client: unmount should not be able to fail

If unmount fails, what is the API users supposed to do?
The client needs to be able to clean up after itself.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libceph API change

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mkcephfs: Fail if there are no mons defined.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

mkcephfs: Fail if a mon has no address set.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

mkcephfs: Remove temp dir, not just its contents.

Also fixes the case where no temp files were created in the dir.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

journaler: fix flush completion when nothing to flush

Signed-off-by: Sage Weil <sage@newdream.net>

journaler: default to readonly; fix asserts

Previously were were never in a readonly state, which made all the existing
asserts meaningless.

Signed-off-by: Sage Weil <sage@newdream.net>

vstart.sh: fix -s

Signed-off-by: Sage Weil <sage@newdream.net>

mon: consisder osd recovery state in health check

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mon: include unfound count in pg recovery summary

Fixes: #1003
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mon: fix up osd health report

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mon: fix up pg health report

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mon: generalize health check a bit

Any PaxosService can return a status (or not).

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'master' of ssh://ceph.newdream.net/home/sage/ceph.newdream.net/git/ceph

Makefile.am: add -fno-strict-aliasing

Until we audit every place we have a typecast to make sure it's safe,
this should be off.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Makefile.am: Wpointer-arith, Wstrict-null-sentinel

Add Wpointer-arith, Wstrict-null-sentinel, Winit-self

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

v0.27

Signed-off-by: Sage Weil <sage@newdream.net>

clitests: fix osdmap unit test

Broken by removal of casdata in 8cd3f1cd2546a61bb5402fa374de7f37147dc3a0.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Makefile: add some new warnings to CXXFLAGS

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: don't daemonize when doing journal reset.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

rbd: rename kernel add/remove to map/unmap

Unmap takes a device now, and uses the major number
to find the right sequence number in sysfs.

Also updated the man page.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

Merge remote branch 'origin/next'

rbd: add commands for manipulating the kernel sysfs interface

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

mount.ceph: use common implementations of secret handling and address resolving

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

addr_parsing: use strtok_r to be safe

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

addr_parsing: rename mount_resolve_dest to resolve_addrs and don't modify its argument

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

addr_parsing: move implementation out of header

Also include missing headers and make it linkable with C,
since it's duplicated from mount.ceph.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd: use rados' init_with_config because we already called common_init

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

common, mount.ceph: move functions for working with secrets into secret.h

Use sprintf instead of safe_cat, since we're just writing a string once.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

Merge remote branch 'origin/next'

testlibrbd: fix signed/unsigned comparisons

testlibrbd.c: In function 'write_test_data':
testlibrbd.c:191: warning: comparison between signed and unsigned integer expressions
testlibrbd.c: In function 'aio_read_test_data':
testlibrbd.c:207: warning: comparison between signed and unsigned integer expressions
testlibrbd.c: In function 'read_test_data':
testlibrbd.c:222: warning: comparison between signed and unsigned integer expressions

Signed-off-by: Sage Weil <sage@newdream.net>

osd: drop 'casdata' bucket by default

We don't use it for anything.  It just means we create a bunch of useless
PGs.

Signed-off-by: Sage Weil <sage@newdream.net>

cmds: make journal-check mode run in the foreground.

daemonize does a bunch of stuff we don't want to do.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: massage handling of oneshot-replay.

We accidentally broke it in our last round of changes to support
standby-replay; patch it up again.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

OSDMon: make the defaults in reweight-by-utilization work.

an oload <= 100 fails out; set the new default to 120.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

testlibrbd: fix signed/unsigned comparisons

testlibrbd.c: In function 'write_test_data':
testlibrbd.c:191: warning: comparison between signed and unsigned integer expressions
testlibrbd.c: In function 'aio_read_test_data':
testlibrbd.c:207: warning: comparison between signed and unsigned integer expressions
testlibrbd.c: In function 'read_test_data':
testlibrbd.c:222: warning: comparison between signed and unsigned integer expressions

Signed-off-by: Sage Weil <sage@newdream.net>

osd: drop 'casdata' bucket by default

We don't use it for anything.  It just means we create a bunch of useless
PGs.

Signed-off-by: Sage Weil <sage@newdream.net>

auth: const cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

obysnc: use python2.5 compatible exception syntax

Apparently using "as" in "except" blocks is a new (and not
backwards-compatible) thing in Python 2.6.x. Who knew?

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

PGMonitor: unregister the config obs in ~PGMonitor

Using the new unregister call, unregister the config observer in
~PGMonitor.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

config: add remove_observer

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

PGMon: use the config observer framework

We now register observers on mon_osd_full_ratio and nearfull_ratio
to update the OSD full/nearfull lists.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>