dout: properly output ceph version on opening dout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

monmaptool: set_foreground_logging

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

simplemessenger: Fix num_threads bug printout.

Also add documentation to get_num_threads since its contract
changed significantly.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

msgr: complain if there are > 1 threads, not 1

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

init-ceph: use do_cmd for pid_file dir creation

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix population on unconnected_watchers on obc load

Fixes: #807
Signed-off-by: Sage Weil <sage@newdream.net>

common: thread: get number of threads from /proc

The kernel knows how many threads we have; just ask it. One less atomic
variable to carry around.

We will eventually have to avoid doing this check for non-daemon code,
but that's a separate issue.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Remove ExportControl (we have better auth now)

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

init-ceph: status: use daemon_is_running

daemon_is_running does some nice things like check /proc/$pid/cmdline.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

init-ceph: fix status for multi-node clusters

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/osd: Fix indentation on RadosModel.h and TestSnaps.cc

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

testsnaps: add snapshot test

Uses RadosModel.h to check the results of a randomized sequence of
writes, reads, snapshots, snapshot removals, and rollbacks for errors.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

Merge branch 'dout2'

Journaler: add some checks for expire_pos.

I don't think these are necessary checks, but the expire_pos >= trim_pos
invariant got broken somehow by johnl, and these checks won't hurt!

Signed-off-by: Greg Farnum <gregf@hq.newdream.net>

Journaler: call set_layout after init_headers.

set_layout modifies last_committed, but then init_headers
uses operator= and overwrites those changes. In this case
it doesn't matter as they're both writing the same changes,
but make the ordering explicit for the future.

Signed-off-by: Greg Farnum <gregf@hq.newdream.net>

OSD: ignore osd_max_write_size if it's set to 0.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

common/lockdep.cc: don't use dout unlocked

Lockdep should use the regular dout() interfaces, rather than going
around them. In particular, we shouldn't output to dout without taking
dout_lock.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

assert: allow assertions inside calls to dout()

We should handle the situation where we assert() while already holding
the dout() lock. At the same time, we want to get the dout lock if we
can, because it makes the logs look nicer. pthread_mutex_trylock solves
the dilemma.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

dout: Convert _dout_lock to plain pthread_mutex_t

Convert _dout_lock to plain pthread_mutex_t. This way, we don't have to
depend on the order of global constructor initialization. It should also
be slightly more efficient. The dout_lock was never subject to lockdep
anyway, so that's not an issue.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

assert.cc: some cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/debug.h: use std::string rather than string

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: Remove common/tls.cc

Using ELF TLS via the __thread keyword is much faster than using
pthread_getspecific and pthread_setspecific. It's also much nicer
looking syntactically. Finally, the __thread keyword is going to be
standardized in C++0x. So there's no reason to have an infrastructure
dependent on pthread_getspecific.

There were no users so this shouldn't affect anything negatively.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

dout: use DoutLocker rather than Mutex::Locker

Use DoutLocker rather than Mutex::Locker, in preparation for making the
dout_lock a plain old pthread_mutex_t.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: use derr/dendl for dout locking

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: use ceph_abort rather than abort

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

lockdep: balance dout and dendl, fix whitespace

Make lockdep use dendl the same way as the other code. This is in
preparation for making lockdep use normal dout() rather than an unlocked
version.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Journaler: fix bad assert.

We can call reread_head during normal replay under
certain circumstances. So add the REREAD_HEAD state
as allowed.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

testlibrbd: fix printf args

Stupid me!

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

common: create dout_emergency interface and use it

Create the dout_emergency interface, which is safe to call from a signal
handler or from inside dout itself.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cfuse: use safe_read and check return value

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

testlibrbd: check return values

Stupid printf!

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

debian: add python, python-dev build-deps

Might be overkill?  The error I see from pbuilder is

checking for a Python interpreter with version >= 2.4... none
error: configure: in `/tmp/buildd/ceph-0.24.3-676-gcde53e9':
error: configure: Failed to find Python 2.4 or newer

...but I'm guessing python-dev is needed too?

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: clean up Pipe::queue_received locking

Ensure we maintain the invariant that a pipe has a non-empty queue IFF
the pipe is queued.

Prompted by #798.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Disable lockdep for ExportControl, ConfFile locks

Currently, we haven't read the configuration at the time we initialize
these locks. So we can't know whether lockdep has been enabled, or what
verbosity it is supposed to have. So just disable it on these locks.

Potentially ExportControl's initialization could be moved to after
g_conf.lockdep and g_conf.debug_lockdep have been read from the
configuration, if lockdep is needed for this component.

ConfFile probably doesn't need a lock at all, but that's another story.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

MDCache: switch CDir::_commit so that it can limit max write size.

This should fix #777.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

MDCache: add max_dir_commit_size.

Configured by setting mds_dir_max_commit_size in conf, or else
by looking at osd_max_write_size. This should lead to sane
max commits even if the user doesn't specify anything.

This will be used in the next commit or to by CDir.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

MDS: Don't always _commit_full just because we have a complete dir.

Instead, commit if a certain percentage of the dentries are dirty.
Configurable via mds_dir_commit_ratio!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

.gitignore: py-compile

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'pybind'

librados: include headers in dist tarball

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

ceph.spec.in: add librbd

Make git ignore librbd tests.

ceph: add --concise flag

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'stable'

Conflicts:
configure.ac

Merge remote branch 'origin/librbd'

objecter: set linger op target pg when a linger is resent

send_linger always creates a new Op, but op_submit does not fill in
the target pg if an existing session is passed in, so when a linger
was resent, it had the wrong pg set.

This caused a crash in cosd with debugging turned on when running
testlibrbd twice. This occurred because the object context for the
linger in the wrong pg had no object name set.

monclient: fix weirdness when we explicitly start with non-master mon

Fix

$ ./ceph -m 10.0.1.252:6790 health
2011-02-11 12:30:38.132728 mon <- [health]
2011-02-11 12:30:38.133243 mon <- [health]
2011-02-11 12:30:38.133697 mon0 -> 'Access denied' (-13)

when the IP we specify is, e.g., for mon.b.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

config: Remove g_conf.num_osd, num_mds, num_mon

These configuration options don't serve any use and were confusing.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Make git ignore qa/workunits/direct_io_test.

MonitorStore: check for ENOENT, not EEXIST

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: bufferlist::read_file: use safe_read

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth/Crypto.cc: use safe_read_exact

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: remove unused variable

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

v0.24.3

make:add messages/MOSDRepScrub.h to NOINST_HEADERS

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cephtool: get_indata should use safe_read_exact

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon/MonitorStore.cc: use safe_read

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rados.cc: use fwrite to output data to stdout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/common_init: use safe_read

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: fix error handling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

s3.sh: simple bug fix

rados: add create op

common/DoutStreambuf: use safe_io.h

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

tools/common: use fwrite to write stdout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: write_meta/read_meta: fix error handling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

msgr: better error checking for write_pid_file

Use safe_write. Check error code of close.

Fix what looks like a "lack of braces around multi-statement block"
error where we always complain that we're unable to write the pid file,
even if we succeed.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore.cc: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/buffer.cc: kill deadcode

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/ConfUtils.cc: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

MonitorStore::put_int: handle I/O errors

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/safe_io.h: warn about unchecked returns

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cfuse.cc: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds/Dumper: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

btrfs_ioc_test.c: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test_disk_bw.cc: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: safe_io: allow C files to include safe_io

Don't use extern C unless we're compiling with C++.
This allows C files to include safe_io.h

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

client/fuse_ll: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: convert some read/write uses to safe variants

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common:safe_io: create exact and non-exact variant

Create safe_read_exact, which fails with -EDOM unless it reads exactly
the requested number of bytes. More often, what we want is the
semantics of safe_read, which returns the number of bytes it read, or
negative on error. Likewise for pread.

Write doesn't need variants because failing to write all the bytes is
always an error.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: safe_io: handle EOF on read

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rbd: fix signed/unsigned comp on i386

We -D_FILE_OFFSET_BITS=64, so off_t is off64_t.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Removing files that look completely unused.

osd: avoid message leak in error path

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

config: fix -f option settings.

We don't want to force foreground logging just because we run in the
foreground -- use no-daemon for that!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

filestore: fix fiemap

testlibrbd: change strcmp to memcmp, other stuff

librbd: use watch/notify internally to keep ImageCtx updated

librbd: add C++ interface tests

librbd: fully specify std::string in list function

dout: don't print time to syslog

Syslog itself prepends a time to messages, so don't duplicate that.
Use a faster snprintf-based method to generate the time string rather
than streams.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

pybind/rados: expand on error handling a bit

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add python bindings for rados

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'fix32'

On i386, pthread ids are 8 hexdigits. Adjust cli tests.

msg/SimpleMessenger.c: fix signed/unsigned compare

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: fix signed/unsigned comparison warn

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileJournal: compare like types in MAX()

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

tools/gui.cc: fix signed/unsigned comparison

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge remote branch 'origin/rep_scrub_wq' into stable

rbd: fix watch tool