mount.ceph: put static at the beginning of decl

To squash a gcc warning.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

tools/gui.cc: remove empty else statement

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon/OSDMonitor: don't check if unsigned < 0

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd/OSD: remove always-true assert

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

const-ify some unchanging static structs

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw/rgw_rest.cc: fix uninitialized static struct

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osdc/Filer: assert(0) on stat error

Remove broken error handling code and replace with an assert(0) and a
TODO.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

client/fuse_ll: fully initialize callback struct

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds/locks: fully initialize lock states

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osdc/ObjectCacher.h: don't assert unsigned >= 0

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Client::readdir_r_cb: fix "always returns true"cmp

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

direct_io_test: fix signed/unsigned, printf flags

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

src/config.cc: fully initialize g_default_file_layout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

crush/builder.c: fix signed/unsigned confusion

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

include/buffer.h:unsigned variables are always >=0

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'master' of ssh://ceph.newdream.net/git/ceph

RGW: RGW::GetObj - signed/unsigned bug

get_obj returns an int, len is unsigned

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

RGW: fix ret in RGWCopyObj::init_common

int ret shadows RGWCopyObj::ret.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

RGW: fix shadowed variable error

int ret and bool ret shadow this->ret, setting ret before the goto
therefore had no effect in these cases.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

RGW: get_service_secret returns a bool, not an int

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

RBD.cc: pread returns a signed value

The check for len < 0 here always returned true since len is uint64_t.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

Merge branch 'objecter_balance_reads'

Fix base64-decoding when input ends in newline.

It used to return -EINVAL because it thought the end was not aligned
to 4 bytes.

Clean up superfluous src < end test in if, the while itself guarantees
that.

Fix inner loop index variable. i and j are too close together ;)

Before this fix, j would keep increasing well beyond in_len,
and lead to segfaults.

osd: Fix compile-time warning.

store is properly initialized inside a try block, but the
compiler doesn't notice that and so thinks it may be used
uninitialized. So initialize it to be NULL.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: Remove unused variable.

Fixes compile complaint. Was made useless in commit
6d29fc66326ff40443ac465c84d6f7bddf3cee4f.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

rados: Make rados tool "put" objects in chunks.

Defaults to 4MB, but can be configured.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: Remove redundant cap suppression code.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Speling fixes

Trivial spelling fixes...

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Improve mount.ceph.8

Seeing as I was wondering what "rbytes" might be, I guess others might
too...

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

monitorstore: check return values

And use open(2) instead of fopen.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: check return values in dumper

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

confutils: check return values

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

os: use scoped variables for fds

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test: add test_csyn

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

csyn: implement localize_reads cmdline option

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: change unfound handling when using localize

Change how cosd handles unfound objects when doing operations with
localize_reads. Specifically, don't wait for unfound objects unless we
are the primary for the relevant PG.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Hadoop: set localize_reads

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cfuse: add --localize-reads argument

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libceph: iface for CEPH_OSD_FLAG_LOCALIZE_READS

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

objecter: implement FLAG_LOCALIZE_READS

Read from a local replica, if any.

Signed-off-by: Sage Weil <sage@newdream.net>

objecter: send read to random replica if BALANCE_READS flag is set

The flag is just an existing one; we may want to rename.

This assumes the OSD will discard any request sent prior to any change in
the PG acting set.  We need to make sure the OSD behavior matches.

Signed-off-by: Sage Weil <sage@newdream.net>

FileStore: fix double close

curr_fd is already closed if cp == cur_seq.  This second close
occasionally ended up closing another thread's fd.  The next open would
tend to grab that fd in op_fd or current_fd which would then get closed
by the other thread leaving op_fd or current_fd pointing to some random
file (or a closed descriptor).

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

common: config.cc: use "admin" as the default id

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: move init_g_conf into md_config_t ctor

Make sure that g_conf is initialized with default values before anything
else happens.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: config.cc: whitespace cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: config.cc: de-globalize g_fake_kill_after

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: config.cc: de-globalize show_config

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: clean up g_conf.id initialization a bit

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: remove ceph_set_default_id

ceph_set_default_id was only ever used to set the default ID to "admin",
which it already was.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

FileStore: fix error handling for mkfs, umount

In FileStore::umount: check if FDs are valid before closing them. Make
them invalid after closing them. Shut down FileStore::timer.

In FileStore::mkfs: always properly shutdown and free the filestore if
an error is encountered during mkfs. Check all functions that can fail.
Print out error messages on failures.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: make --dump-journal preserve offset

Suggest user use tar -S to preserve sparseness.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

gitignore: ignore eclipse metadata

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

remove ancient active/ stuff

osd: don't leak fd on error

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

crypto: don't clobber errno

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge remote branch 'origin/ostimeo'

Merge branch 'mds_reset'

Fixes: #602

Merge remote branch 'origin/stable'

Conflicts:
src/osd/OSD.cc

OSD: update_osd_stat take heartbeat_lock

Previously update_osd_stat had a race with code modifying heartbeat_from
causing the iterator increment to occasionally segfault.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

mds: skip a few more inodes during journal reset

To be safe...

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: implement journal reset

This basically works.  Remaining issues:
 - mydir and root inodes are recreated from scratch but need to be
   reconciled with what's committed (outside the old journal)
 - ?

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: open mydir (along w/ root) inode from boot_start()

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Locker: Drop loner correctly!

Our previous check for if we want to drop the loner was incorrect.
Now, it's fixed. Resolves a serious bug with inode write access.

Reported-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Locker: Drop loner correctly!

Our previous check for if we want to drop the loner was incorrect.
Now, it's fixed. Resolves a serious bug with inode write access.

Reported-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

librados: fix C interface const, too.

See 561224e95d6c66661d1bd6dce0e3d9da6f4a7e13

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: defer sending resolves until mdsmap.failed.empty()

There is no point sending resolves while there are still failed nodes,
since we can't complete.  We also trigger an assert if we try to send to
a failed node.  Instead just wait until failed.empty() and then start.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: defer sending resolves until mdsmap.failed.empty()

There is no point sending resolves while there are still failed nodes,
since we can't complete.  We also trigger an assert if we try to send to
a failed node.  Instead just wait until failed.empty() and then start.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: standardize option parsing

- Use the standard macros.
- Simply --hot-standby and --journal-check options (always specify rank).
- Update usage().

Signed-off-by: Sage Weil <sage@newdream.net>

common: _dout_lock: initialize _dout_lock first

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

config: remove dead stringtable cruft

Signed-off-by: Sage Weil <sage@newdream.net>

os: FileStore: Add commit timeout

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rbd: Rados::init: clean up after failure

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>

os: FileStore: ctor should init all class vars

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: FileStore: remove default param

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os:FileStore:use std::string rather than huge bufs

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: fix mutual exclusion for _dispatch

We want only one thread dispatching messages (either new or requeued), so
that we can preserve ordering.  Previously we weren't doing so for all
callers of do_waiters (tick() and the first in ms_dispatch()).

This fixes osd_sub_op(_reply) ordering problems that trigger the
now-famous repop queue assert.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge remote branch 'origin/health2' into unstable

units: add signals unit test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: cluster_fail instead of reset_cluster

Mark all cluster members as failed, and blacklist.  Do not force up/failed
ranks to stopped, as that requires the admin to do other trickery.  This
keeps the cluster fail orthogonal to any journal discard/reset.

Signed-off-by: Sage Weil <sage@newdream.net>

mon: add mdsmap DOWN flag to prevent mdsmap updates

This is intended to be set while doing critical cluster manipulation to
avoid cmds instances from starting up and getting added to the map.

Require it be set for cluster_reset.

Signed-off-by: Sage Weil <sage@newdream.net>

mdsmap: add flags

Convert unused client_epoch field to flags to avoid a protocol change. It
is always 0 on current clusters.  Lucky us!

Signed-off-by: Sage Weil <sage@newdream.net>

mon: add 'mds reset_cluster' command

Reset an MDS cluster back to a single node.  The idea is:

 - wipe out mds journals
 - maybe set recovery flag
 - mds reset_cluster (this)

Then mds0 only recover from an (empty) journal.  Other MDS nodes would only
rejoin the cluster later.

See: #602
Signed-off-by: Sage Weil <sage@newdream.net>

.gitignore: vstart generated files

Signed-off-by: Sage Weil <sage@newdream.net>

vstart: put tmp files in /tmp

Signed-off-by: Sage Weil <sage@newdream.net>

osd: preserve ordering when ops are requeued

Requeue ops under osd_lock to preserve ordering wrt incoming messages.
Also drain the waiter queue when ms_dispatch takes the lock before calling
_dispatch(m).

Fixes: #743
Signed-off-by: Sage Weil <sage@newdream.net>

osd: restart if the osdmap client, heartbeat, OR cluster addrs don't match

If we somehow get ourselves into a situation where the OSDMap addresses do
not match our actual addresses, restart and try again.  This is still
possible if multiple MOSDBoot messages end up in flight in the monitor,
say due to a monitor disconnect/reconnect, and we race with something that
marks us down in the map.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: avoid extraneous send_boot() calls

Only send_boot() on osdmap update if we are restarting.  Otherwise we can
end up with too many MOSDBoot messages in flight and the monitor may
apply an old one instead of a new one.  For example:

- cosd starts
- send_boot with address set A
- get an osdmap update
- send_boot again with address set A
- get an osdmap update.  now we're up.
- get osdmap update, now we're marked down,
- bind to address set B
- send_boot with address set B

and the monitor may apply the second MOSDBoot (with adddress set A).

This results in an online OSD using a cluster address that differs from
that in the OSDMap.  Which causes problems with peering, among other
things.

Signed-off-by: Sage Weil <sage@newdream.net>

test_unfound.sh: kill cosds rather than mark out

For this test, we need to kill cosds rather than mark them as out.
Otherwise, we cannot force objects to become unfound.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

disable scrubs during test_unfound

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: Remove rados_pool_t& usage, and pointless consts.

For some reason when I wrote this I passed rados_pool_t by reference
in some functions instead of by value. It's just a void*, so this is
silly.
Also silly, some of the passed-by-value rados_pool_ts were declared
to be const. WTF?

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mon: implement PGMonitor::get_health

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon: OSDMonitor::get_health: const cleanup

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon: MonitorStore::mkfs: use run_cmd

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: FileStore: use run_cmd instead of system

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>

common: Add run_cmd

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>

ReplicatedPG: _rollback_to fix the just cloned condition

_rollback_to in the case that head was just cloned and that clone
includes snapid does not need to do anything.  Previously, snapid would
have to match the snap on the clone, but the condition should be that
snapid is contained within the clone's snaps set.

This bug was introduced in e189222f06ee287eeb6fd7f46cff7a6727806dea

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

mon: remove PGMap::pg_set

We don't need an additional data structure to hold the keys to pg_stat.
We can just look at the keys of pg_stat.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon: PGMap::apply_incremental must maintain pg_set

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

os: readdir_r: read into PATH_MAX-sized buf

Fix the readdir_r uses in FileStore.cc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>