dout: use DoutLocker rather than Mutex::Locker

Use DoutLocker rather than Mutex::Locker, in preparation for making the
dout_lock a plain old pthread_mutex_t.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: use derr/dendl for dout locking

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: use ceph_abort rather than abort

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

lockdep: balance dout and dendl, fix whitespace

Make lockdep use dendl the same way as the other code. This is in
preparation for making lockdep use normal dout() rather than an unlocked
version.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: create dout_emergency interface and use it

Create the dout_emergency interface, which is safe to call from a signal
handler or from inside dout itself.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

testlibrbd: fix printf args

Stupid me!

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

cfuse: use safe_read and check return value

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

testlibrbd: check return values

Stupid printf!

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

debian: add python, python-dev build-deps

Might be overkill?  The error I see from pbuilder is

checking for a Python interpreter with version >= 2.4... none
error: configure: in `/tmp/buildd/ceph-0.24.3-676-gcde53e9':
error: configure: Failed to find Python 2.4 or newer

...but I'm guessing python-dev is needed too?

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: clean up Pipe::queue_received locking

Ensure we maintain the invariant that a pipe has a non-empty queue IFF
the pipe is queued.

Prompted by #798.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Disable lockdep for ExportControl, ConfFile locks

Currently, we haven't read the configuration at the time we initialize
these locks. So we can't know whether lockdep has been enabled, or what
verbosity it is supposed to have. So just disable it on these locks.

Potentially ExportControl's initialization could be moved to after
g_conf.lockdep and g_conf.debug_lockdep have been read from the
configuration, if lockdep is needed for this component.

ConfFile probably doesn't need a lock at all, but that's another story.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

MDCache: switch CDir::_commit so that it can limit max write size.

This should fix #777.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

MDCache: add max_dir_commit_size.

Configured by setting mds_dir_max_commit_size in conf, or else
by looking at osd_max_write_size. This should lead to sane
max commits even if the user doesn't specify anything.

This will be used in the next commit or to by CDir.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

MDS: Don't always _commit_full just because we have a complete dir.

Instead, commit if a certain percentage of the dentries are dirty.
Configurable via mds_dir_commit_ratio!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

.gitignore: py-compile

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'pybind'

librados: include headers in dist tarball

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

ceph.spec.in: add librbd

Make git ignore librbd tests.

ceph: add --concise flag

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'stable'

Conflicts:
configure.ac

Merge remote branch 'origin/librbd'

objecter: set linger op target pg when a linger is resent

send_linger always creates a new Op, but op_submit does not fill in
the target pg if an existing session is passed in, so when a linger
was resent, it had the wrong pg set.

This caused a crash in cosd with debugging turned on when running
testlibrbd twice. This occurred because the object context for the
linger in the wrong pg had no object name set.

monclient: fix weirdness when we explicitly start with non-master mon

Fix

$ ./ceph -m 10.0.1.252:6790 health
2011-02-11 12:30:38.132728 mon <- [health]
2011-02-11 12:30:38.133243 mon <- [health]
2011-02-11 12:30:38.133697 mon0 -> 'Access denied' (-13)

when the IP we specify is, e.g., for mon.b.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

config: Remove g_conf.num_osd, num_mds, num_mon

These configuration options don't serve any use and were confusing.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Make git ignore qa/workunits/direct_io_test.

MonitorStore: check for ENOENT, not EEXIST

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: bufferlist::read_file: use safe_read

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

auth/Crypto.cc: use safe_read_exact

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: remove unused variable

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

v0.24.3

make:add messages/MOSDRepScrub.h to NOINST_HEADERS

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cephtool: get_indata should use safe_read_exact

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon/MonitorStore.cc: use safe_read

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rados.cc: use fwrite to output data to stdout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/common_init: use safe_read

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: fix error handling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

s3.sh: simple bug fix

rados: add create op

common/DoutStreambuf: use safe_io.h

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

tools/common: use fwrite to write stdout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: write_meta/read_meta: fix error handling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

msgr: better error checking for write_pid_file

Use safe_write. Check error code of close.

Fix what looks like a "lack of braces around multi-statement block"
error where we always complain that we're unable to write the pid file,
even if we succeed.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore.cc: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/buffer.cc: kill deadcode

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/ConfUtils.cc: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

MonitorStore::put_int: handle I/O errors

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/safe_io.h: warn about unchecked returns

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cfuse.cc: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds/Dumper: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

btrfs_ioc_test.c: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test_disk_bw.cc: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: safe_io: allow C files to include safe_io

Don't use extern C unless we're compiling with C++.
This allows C files to include safe_io.h

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

client/fuse_ll: use safe_write

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: convert some read/write uses to safe variants

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common:safe_io: create exact and non-exact variant

Create safe_read_exact, which fails with -EDOM unless it reads exactly
the requested number of bytes. More often, what we want is the
semantics of safe_read, which returns the number of bytes it read, or
negative on error. Likewise for pread.

Write doesn't need variants because failing to write all the bytes is
always an error.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: safe_io: handle EOF on read

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rbd: fix signed/unsigned comp on i386

We -D_FILE_OFFSET_BITS=64, so off_t is off64_t.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Removing files that look completely unused.

osd: avoid message leak in error path

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

config: fix -f option settings.

We don't want to force foreground logging just because we run in the
foreground -- use no-daemon for that!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

filestore: fix fiemap

testlibrbd: change strcmp to memcmp, other stuff

librbd: use watch/notify internally to keep ImageCtx updated

librbd: add C++ interface tests

librbd: fully specify std::string in list function

dout: don't print time to syslog

Syslog itself prepends a time to messages, so don't duplicate that.
Use a faster snprintf-based method to generate the time string rather
than streams.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

pybind/rados: expand on error handling a bit

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add python bindings for rados

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'fix32'

On i386, pthread ids are 8 hexdigits. Adjust cli tests.

msg/SimpleMessenger.c: fix signed/unsigned compare

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileStore: fix signed/unsigned comparison warn

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os/FileJournal: compare like types in MAX()

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

tools/gui.cc: fix signed/unsigned comparison

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge remote branch 'origin/rep_scrub_wq' into stable

rbd: fix watch tool

rbd: modify watch

librbd: fix completion cb mixup

make:add messages/MOSDRepScrub.h to NOINST_HEADERS

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'stable'

Conflicts:
src/osd/OSD.cc

osd: discard scrub reply if pg changed

build_scrub_map will bail out if the pg changed.  Discard the result in
that case since the primary will ignore it anyway.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: avoid map_lock for scrub_map reply

Using osd->osdmap->epoch without map_lock is dangerous.  We can avoid it
entirely by replying on the same connection as the request.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: never rewrite log after {advance,activate}_map

pg->dirty_log is never true, so this is dead code.  And nothing in either
of those two methods updates the pg log.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: always write backlog after creation

dirty_log is never set to true, so we would set the log.backlog flag but
not write it to disk.  If we restarted the OSD, we would think we had the
backlog in the log but in reality we would not.  clean_up_local() could
then erase almost every object in the PG.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: fix no missing inferance

Add missing continue in last_update==last_complete (no missing) case.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

common_init: don't force profiling_logger = true

common_init: don't force profiling_logger = true for daemons. Users can
always set "profiling logger = true" when they want to activate it.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: Rename Logger to ProfilingLogger

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: combine set_fg_logging into common_init

Combine set_fg_logging into common_init. This eliminates the problem
where we briefly try to apply the logging settings from the config file
even in a program that needs to set foreground logging.

TestTimers, TestSignalHandlers, TestDoutStreambuf: don't initialize ceph
authentication. These applications don't need it.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

gitignore: ignore massif.out.*

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

config: kill more unused options

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

cfuse: remove high level fuse interface

Untested, unused, and inferior to the lowlevel implementation.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

PG: remove sub_op_scrub

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

PG: switch _request_scrub_map to send MOSDRepScrub

Also switches sub_op_scrub_reply to sub_op_scrub_map to handle the
OSD_OP_SCRUB_MAP response.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

OSD: Adds handler for MOSDRepScrub

handle_rep_scrub enqueues the message in rep_scrub_wq.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

PG: added replica_scrub

Adds handler in PG for MOSDRepScrub messages.  replica_scrub will
replace sub_op_scrub.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

OSD: Add rep_scrub_wq

Previously, replica scrubs would be handled in sub_op_scrub in the op
queue.  Replica scrubs will now be processed by rep_scrub_wq using the
disk tp.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

rados: Adds CEPH_OSD_OP_SCRUB_MAP sub op

Previously, maps were requested with a sub_op and sent with a
sub_op_reply.  As maps will now be requested using a different message,
replicas will transmit scrub maps requested via MOSDRepScrub messages by
sending a sub_op of type CEPH_OSD_OP_SCRUB_MAP.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

MOSDRepScrub: Adds a message for initiating a replica scrub

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

librbd: change tests to use new rbd_list and rbd_list_snaps interface