librados: remove useless reference holding

librados: get reference to the io context for the pending async ops

heap_profiler: if log_dir is empty, don't try and log to root dir!

If log_dir was empty then the prefix would be of the form /mon.a
That isn't good, so fill it in so it writes to the current dir.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

coverage.sh: use .lcov instead of .info to avoid confusion

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

Merge remote branch 'origin/wip_coverage'

Conflicts:
src/mon/Monitor.cc

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rgw: remove required indexes when modifying user info

Conflicts:

src/rgw/rgw_admin.cc

mon: Follow the Message rules when forwarding tells

My initial commit was naughty, fix that up here by
creating a new message for each mon and just copying
the command.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Thread: remove globals. Thread create must succeed

Remove the references to global variables from Thread.h. They are really
unecessary. In every case, the printout is followed by an assert which
will deliver the exact same information.

Assert that thread creation succeeds. Nobody was checking the return
value of Thread::create() previously. Added a new function,
Thread::try_create(), which programmers can use if they do want to check
the value of Thread::create() and handle it appropriately.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

LogClient: pass in CephContext explicitly

Again, the Messenger passed to LogClient may be NULL, so pass in the
CephContext explicitly.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Monitor: pass in CephContext explicitly

Sometimes we create a Monitor without a Messenger. So we can't pull the
CephContext out of the Messenger, because it may be NULL. Just specify
it explicitly in the Monitor constructor.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds, mon, osd: with coverage enabled, exit(0) when SIGTERM is received

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

Revert "mon: add all_exit and exit commands"

This reverts commit 69092735c4b25c88ded58b17b1c198fafcf5931b.

Revert "mds: allow mds to 'exit immediately'"

This reverts commit c01961ff4b712a12e25e437adceae264273d7428.

Revert "osd: add command to exit cleanly"

This reverts commit 1cc78b6a864d36cbe1eda45890d7f361adb8201d.

mds: fix up MDCache::path_is_mine to remove a totally bogus assert.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mon: add new "tell" infrastructure.

This lets you issue commands
like "ceph mon tell 0 heap start_profiler". Hurray!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

CephContext: initialize module_type in ctor

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

CephContext: remove silly debug code

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Dispatcher should take a CephContext argument

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: fix tcmalloc check.

We can run commands to do things like start the profiler even if
it's not already running!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: xlock_finish should only do_issue in certain cases.

We accidentally (we think) initialized this variable to true when
we want it to be false: we should only do_issue if there aren't
any remaining locks, not in all cases.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

MDiscoverReply: encode/decode wanted_ino.

This wasn't encoded previously so it was always zero on the
receiver's end, which led to nasty hangs. :(

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mdcache: correct error message.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: replay: save the right dir!

We were using the wrong one and breaking replay.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

debian: move gceph into a separate package

Signed-off-by: Sage Weil <sage@newdream.net>

tools/common: fix C_ObserverRefresh constructor

Fixes #1159

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

gtest: add -lpthread to AM_LDFLAGS

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Makefile.am: Add -lpthread to UNITTEST_LDADD

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ceph tools: de-globalize CephToolContext

De-globalize CephToolContext. It's important to do this now because the
constructor for CephToolContext references the configuration (via
CephContext.)

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: drop workaround for embedded null decoding

We removed the embedded null check recently, 9b37f4f, so this workaround
isn't needed.

Signed-off-by: Sage Weil <sage@newdream.net>

rgw_admin: user info dumps auid

rgw_admin: can reset auid

Merge remote branch 'origin/mds_rename'

mds: rename: remove illicit assert.

We actually do want witnesses who aren't auth for anything
to do journaling in some cases, so kill the assert.
That also negates the need for the not_journaling check.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: try_trim_non_auth_subtree if we rename a dir away from a non-auth subtree

It's possible we have non-auth metadata only because we have a subtree
nested beneath. If we rename a directory out of a non-auth subtree, we
should try to trim any non-auth content from that subtree that may now
be possible due to the child subtrees being linked elsewhere.

Fixes: #1146
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: use CDIR_AUTH_UNDEF where possible

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: remove unlinked metadata from cache on replay

If we replay a metablob that unlinks something, throw it out immediately.
Recursively.  This comes up when:
 - we rename a file from one mds to another, and we replay the event on
   the source mds.  the inode gets thrown out.
 - we rename a directory from one mds to another, and when journaled, the
   source mds had no nested metadata.  same thing: we throw it out.  we
   may have something in our cache nested beneath that, though, that was
   since committed and such, but the fact that we didn't journal it being
   reattached elsewhere implies that it was clean and gone when our event
   was journaled, and we can throw it all out.  recursively.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Fix build errors in unit tests

libcommon uses symbols from the crypto libraries, so they must appear on
the link line whenever libcommon appears.  Later, we may want to revisit
this dependency; however, right now, having unit tests that build
consistently is pretty important.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

monitorstore: return error codes for consistency

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

qa: test witness subtree adjustment

This exercises the mds at least; we don't have a way to verify mds
behavior currently.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: test_sync_io

Currently only tests the read path for O_DIRECT and sync (if the ioctl is
in place).  Also currently crashes the kclient with

 read_direct buf_align 0 offset 4190720 len 4096

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: add dir rename tests

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: open renamed import child frags during journal replay

Open up any child frags of the imported renamed inode that are noted in
the journal event.  (Note we blindly open up that list here; it's up to the
journaler to only populate it when appropriate.)  If the listed frags are
not already open, open them up and set the dir_auth to unknown; presumably
they belong to the rename source/exporter.  If we already had them open,
then the adjust_subtree_after_rename call above will have caught them and
already done the necessary subtree adjustment.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: journal open srci frags on srci import (master)

If we are importing the renamed inode, and it is a directory, journal a
list of all open dirfrags (currently, this is actually all frags) so that
we can open them up during journal replay.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: journal renames on witnesses if we have nested subtrees

If a rename witness has any subtrees that are nested beneath the renamed
directory, we need to journal the rename event so that our cache is
properly updated on journal replay.

Further, if we are exporting srci, we also need to journal the dest
(even if we aren't auth for destdn) if we have any open dirfrags because
those will turn into nested subtrees shortly.

We still need to ensure that the cache is properly trimmed during replay.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: CDir::contains() should use projected hierarchy

The only current callers are in resolve stage, when nothing is projected.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

re-fix ignore return value change

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: fix check for no-op rename of two links to the same inode

touch a
ln a b
mv a b

is a no-op, according to posix.  Move the check earlier.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

cfuse: really ignore write() return value

Hopefully this will make gcc stop complaining. The void cast no longer
seems to silence unchecked return code errors, on gitbuilder at least.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge remote branch 'origin/wip_gcov'

qa: add rename dir checks

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

coverage: add ceph-coverage to run something with the right GCOV_PREFIX_STRIP

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

Makefile.am: add check-coverage to get coverage of 'make check'

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

Makefile.am: install gcno files

This is needed so that coverage data can be generated from the
gitbuilder tarballs.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

Makefile.am: clean gcno and gcda files in "make clean"

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

qa: rename tests

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: rename: add missing pop_and_dirty_projected_inode calls

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: rename: use temp *in in _rename_apply

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: rename: move unlink out of every possible branch

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

test/bufferlist: add copy_all test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

coverage: add helper script to get coverage for a local test

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

mon: add all_exit and exit commands

all_exit makes each daemon exit(0), for gcov data collection.
exit causes cmon to do this.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

mds: allow mds to 'exit immediately'

This is temporary until shutting down cleans up well.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

mon: ceph tell mds * is a valid command

Previously this fell through and returned -EINVAL to the user.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

osd: add command to exit cleanly

This is required for gcov to work on daemons since the coverage data
is written atexit, and the function that writes the data is not
exported.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

configure: add option for building with gcov coverage support

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

test/bufferlist.cc: retab for consistency

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cfuse.cc: fix warning

We genuinely don't care about the return code we get back from this call
to write(2). So cast the result to void to tell gcc that we don't care.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

bufferlist::iterator: add copy_all, use in KeyRing

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: rename: fix oldin pop

- make the pop match position with the project in prepare
- don't pop on linkmerge, since we don't project in that case

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: rename: some cleanup

- add oldin temp
- make if conditions match between _prepare and _apply

no functional changes.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

client: fix rename request target selection

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

client: fix mds selection for unlink

It was broken when a remote link target was on another mds.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

client: make debug prints for choose_target_mds more helpful

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

client: allow caps to be dropped on another inode for requests

We need the ability to drop caps on another inode that isn't req->inode
or req->old_inode in the request struct.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: fix unlink new snaprealm check

The bool was flipped.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

client: fix rename debug print

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

g_conf: change to pointer

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

simple_spin: fix compiler warnings

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

init: set cct->module_type in common_preinit

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: rename: fix handling of target that is remote dentry

If the target is a remote dentry, we need to consider that the destdn
and desti may have different auths.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

client: fix mds routing for link requests

We want to send link requests to the auth for the new name, not the
target inode.  We also want to drop FILE_SHARED caps on new name's
directory.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: rename: only add target to stray if destdn is a primary

Only add target into the stray dir if we are renaming over a primary
dentry.  (Otherwise we aren't moving the target.)

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: fix/clean up xlock import/export

- create xlock import/export helpers
- fix/simplify checks: we want to export/import only xlocks on the inode
that is being migrated, unless they are locallock.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

gitbuilder: quiet

radosgw_admin: link bucket to user

mds: adjust subtree roots on rename

If we replay a dir rename operation, we need to adjust the subtree map
accordingly.

This covers the case where the metablob contains both the src and dest
dentries.  Remaining cases will follow.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

common: add module_type to CephContext; use to initialize rotating keys

This fixes rotating keys.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge remote branch 'origin/wip_librbd_locking'

monclient: use default mon port if not specified in conf

Signed-off-by: Sage Weil <sage@newdream.net>

monmaptool: use default mon port if not specified

Signed-off-by: Sage Weil <sage@newdream.net>

dumpjnl: call msgr->register_entity before start

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

simple_spin: use file-scope global not function

function-scoped globals are protected by a mutex, and taking a mutex
inside a spin lock implementation kind of defeats the point...

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

simple_spin: fix linker error

For some stupid reason the linker is pulling this in twice, resulting in

/bin/sh ../libtool --tag=CXX   --mode=link g++  -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -Wtype-limits -Wignored-qualifiers -Winit-self -Wpointer-arith -fno-strict-aliasing -Wnon-virtual-dtor -Wno-invalid-offsetof -Wstrict-null-sentinel  -g -Wextra -Wno-missing-field-initializers -Wno-missing-declarations -Wno-unused-parameter -Wl,--as-needed -latomic_ops  -o radosgw_admin radosgw_admin-rgw_fs.o radosgw_admin-rgw_rados.o radosgw_admin-rgw_user.o radosgw_admin-rgw_access.o radosgw_admin-rgw_op.o radosgw_admin-rgw_rest.o radosgw_admin-rgw_rest_os.o radosgw_admin-rgw_rest_s3.o radosgw_admin-rgw_common.o radosgw_admin-rgw_cache.o radosgw_admin-rgw_os.o radosgw_admin-rgw_os_auth.o radosgw_admin-rgw_formats.o radosgw_admin-rgw_log.o rgw_escape.o radosgw_admin-rgw_admin.o librgw.a librados.a -lfcgi -lcurl -lexpat -lpthread -lm -lcrypto++
libcommon.a(libcommon_a-simple_spin.o):(.data+0x8): multiple definition of `foo'
test_trans.o:/home/sage/ceph/src/./common/debug.h:51: first defined here

...which points to some larger linking issue.  However, this fixes it for
now.

Signed-off-by: Sage Weil <sage@newdream.net>

fix the MonClient problems for --with-debug programs.
Still doesn't compile, though.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Refactor MonClient, KeyRing

MonClient should contain a KeyRing and a RotatingKeyRing. All the
MonClient users, except possibly csyn, don't want to manage those
objects themselves.

Don't chdir until after we have opened the KeyRing. If the KeyRing is at
a relative path, a chdir may make it inaccessible. Separate the chdir
function from the daemonize function.

Refactor the cmds argument parsing a little bit. Separate the special
actions from the normal operations of the daemon.

This should allow librados and libceph to support CephX finally! yay!

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

remove g_keyring

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

dout:remove stream from dout_emerg_streams earlier

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

DoutStreambuf: de-globalize dout lock

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

DoutStreambuf: de-globalize emergency logging

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>