auth: Make error message on bad AES key lenght say the bad length too.

Helps debugging.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

auth: Use AES IV constant directly, not via local static pointer.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

auth: Drop "using namespace", it's almost always used explicitly anyway.

This helps us be agnostic about what crypto library is in use.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

auth: Add unit tests to check basic crypto sanity.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

auth: Removed unused function generate_random_string.

It doesn't sound like a good idea in the first place, with
embedded zero bytes terminating the string prematurely.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

Add test/osd/RadosModel.h to noinst_HEADERS.

Caught by "make distcheck".

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

autoconf: Complain if libatomic-ops is not found.

Use --without-libatomic-ops if you don't want it.

Fixes part of #826.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

autoconf: Complain if tcmalloc is not found.

Use --without-tcmalloc if you don't want it.

Fixes part of #826.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

autoconf: Complain if FUSE is not found.

Use --without-fuse if you don't want it.

Fixes part of #826.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

osd: wait for handle_osd_map transaction ondisk without doing a full sync

Doing a full sync (forcing a btrfs transaction etc) was just wrong here.
All we (might) care about is whether our Objectstore::Transaction is
stable (in journal or fs) or not.

We are still waiting for those operations to flush to the fs (to be
readable).  That may not be necessary either, but shouldn't have a big
performance impact.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: avoid setting up_thru on new PGs

This trades off the possibility of peering blockage if the OSDs in the
first interval (after pg creation) go down and stay down with avoiding
two osdmap updates for any pg creation.  I think this is reasonable given
that:

 - If the pg did go active, then it did go RW and assuming as much changed
   nothing.
 - If the pg did not go active, then it is empty, and there is no data
   lost.

To do this:

- When peering during the first ever interval, don't bother setting
  up_thru.
- Always mark that interval maybe_went_rw, even though up_thru isn't set
  as such.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

testlibrbdpp: include assert.h

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: librados::WatchCtx needs virtual dtor

This one was definitely biting because librados itself calls delete on
these objects, deleting them through a pointer-to-base.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: CapMap: add virtual dtor to virtual class

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

crushtool: fix warning seen on i386

Fix implicit narrowing warning seen on i386

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

tests: add bufferlist unit test

pushing_back an empty bufferptr to an empty bufferlist should be a noop.
In particular, the bl.begin() should be equal to bl.end().

Signed-off-by: Samuel Just <rexludorum@gmail.com>

clitest: Fix tests after osdmaptool --clobber bugfix.

Commit 5c8146b55dbd60bdfa47b53b93f2769f7d0524dc fixed clobber,
adjust clitests to match. Reordered test logic to have "fsid
does not change" check cover a run without --clobber.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

rados: Add "stat" option, and fix "put" to work on larger block sizes.

We didn't have a stat option, now we do.
Previously, "put" allocated its read space on the stack. That meant
the max block size was a little under 8MB, or you got a segfault! Now
it's on the stack, and you can set it as you like.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mkcephfs: modularize

The goal is to support the old "ssh to everything" mode and also a
piecewise mode that lets the administrator do each step and handle
data copying and remote execution themselves.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: fix keyring loading

Load keyring if NOT mkfs.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osdmaptool: don't require clobber to create new osdmap

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Don't use -i to set id for non-daemons

Due to conflicts, don't use -i to set id except for daemons.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common: replace -I with -i, rework --name

Remove -I. All programs will take -i to set ID now.

Support setting the type and id simultaneously with --name or -n.

Revise common usage.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cauthtool: remove short form of --create-keyring

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'globalfoo'

librbd.cc: trim includes

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cram: test --conf=foo cconf syntax

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common_init: Split common_init into multiple parts

Split common_init into several different parts which can be called
individually from libraries. Eliminate startup_flags.

Create ceph_argparse_pull and ceph_argparse_next.

Split off keyring_init from common_init.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

config: trim deadcode

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ConfUtils::_parse: fix uninitialized var

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

config: ConfFile::open: return error code

Return a integer result code specifying why we failed, if we fail.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

build:add common/entity_name.h to noinst_HEADERS

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

makefile:Suppress hash warning in include/encoding

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Suppress gnu_cxx::hash warning a little bit better

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

config.cc: retab

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common_init: call to ProfilerFlush is unneeded

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Replace g_conf.name and g_conf.id with entityname

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

makefile: Add -Wno-deprecated

Until we can replace gnu_cxx::hash_map with std::tr1::unordered_set,
build with -Wno-deprecated.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'globalfoo'

rgw: cast size_t to unsigned long to fix a 32-bit warning.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

librbd: cast offset values to uint64_t for unsigned comparison warning.

It seems that size_t, off_t, and le64 have different signed/unsigned
properties on 32 and 64-bit Linux platforms, so just cast
them both (since offsets obviously can't be negative).

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Merge branch 'stable'

uclient: Clear the CEPH_CAP_FILE_BUFFER ref on _flush, if safe.

Previously we just returned if safe, but leaving the CEPH_CAP_FILE_BUFFER
ref around breaks _fsync horribly. The root cause of this is
update_inode_file_bits calling objectcacher->truncate_set without
clearing the BUFFER ref, but the mechanics of clearing it there are
complicated, and I don't believe there are any issues with keeping
around the extra reference, as long as it's cleared when necessary.

This should fix #862.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

common_init: no signal handlers for library code

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Create common/entity_name.h

Create common/entity_name.h. Refactor EntityName a bit.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add missing includes

Don't rely on header files being included in a certain order or
combination.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: use projected subtree in rename anchor check

We want to (try to) reanchor the directory on rename when our _projected_
subtree is not a leaf.  If we use the normal get_subtree_root() call,
we get NULL if we are unlinked, which makes is_leaf_subtree() crash.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

librados: add append to object

osd: document last_epoch_clean caveat

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: include all stray peers in might_have_unfound

We should always consider any OSD that has a copy of the PG as a possible
location for missing objects.  There are cases where might_have_unfound is
not completed.  For example,

 - objects on [1,2]
 - 2 marked down/out
 - objects on [1,3]
 - recovery completes, last_epoch_clean is set.
 - 2 comes back online

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

buffer.h: push_back,push_front of an empty buffer pointer should be noop

Also adds an assert to copy to ensure that bufferlist iterator copy
completes.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

libceph: when calling common_init,set library flag

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Bump debian/changelog timestamp so it's greater than previous one.

This shuts up lintian, and just makes sense.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

Make git ignore stamp files from debian build.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

Typos.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

do_autogen: add -O2 when compiling for profiling

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'mds'

Merge branch 'stable'

tests/cli/osdmaptool: default pg_num changed

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

testlibrbd: signed return types

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

filestore: fix signedness so we can handle errors from safe_pread

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

version: fix const warning

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

safe_io: fix signed/unsigned comparisons

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

tests/cli/osdmaptool: default pg_num changed

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

crush: remove misleading comment

This is a stale comment from an earlier iteration of the implementation.

Reported-by: Alexander Lyakas <alex.bolshoy@gmail.com>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

crush: fix memory leak

Reported-by: Alexander Lyakas <alex.bolshoy@gmail.com>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'next'

Conflicts:
configure.ac
debian/changelog
debian/control
src/Makefile.am
src/cmds.cc
src/cosd.cc
src/include/rados/librados.hpp
src/librados.cc
src/mds/Server.cc
src/osd/OSD.cc
src/testradospp.cc

v0.25

Bump librados soname to 2.

Update Debian packaging. For human-only mentions of librados,
use just "librados"

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
Conflicts:

debian/control
src/Makefile.am

config: back to 6 pg bits for now

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Merge branch 'osd_recovery' into next

rgw: put object request returns etag

test_missing_unfound: asdf

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: requeue pg for recovery if we may have found someting

If we get a peer log/missing and call search_for_missing, requeue the pg
for recovery so we can pull anything we may have just found.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: include all up peers in might_have_unfound when desperate

If our might_have_unfound calculation was off (it currently can be, see
#865) we could prematurely give up.  Try any up OSD at this stage just to
be sure.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: recover_primary if recover_replicas starts no ops

recover_replicas may fail to start anything if we see an unexpected error.
In that case, try recover_primary immediately instead of waiting for the
PG to (hopefully) get requeued for recovery later.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: discover more missing if unfound and do_recovery can't start anything

If we couldn't start any recovery ops and things are still
unfound, see if we can discover more missing object locations.
It may be that our initial locations were bad and we errored
out while trying to pull.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

do_autogen.sh: add -P option

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: debug the hell out of heartbeat thread

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Fix test/signals.cc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: cosmetic header changes

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

librados, librbd: use separate IoCtxs for data and metadata

Adds deep copy method IoCtx::dup, so that the data and metadata
contexts can have different snap_seqs and snap contexts.

Also avoid calling Rados::shutdown explicitly, since the destructor
will do this, and it must run after the Image destructor.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librbd: fix error message and unnamed constant

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librbd: change RBD::open to take a reference to an Image instead of a pointer

This makes the API more consistent with the librados API.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librados: remove unused member of IoCtx

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librados: IoCtx destructor should put reference only if initalized

librados: can set up object locator

librados:rados_ioctx_stat -> rados_ioctx_pool_stat

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

object_locator: fix clear()

librados: use atomic_t for reference count

Use an use atomic_t for the reference count in IoCtxImpl.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: make IoCtxImpl a pointer in WatchContext

Adds get and put to IoCtxImpl for refcounting,
and uses them in WatchContext, which shouldn't
be creating a copy of the IoCtxImpl.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librados: decrement refcount of old io_ctx_impl in assignment operator

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librados: fix IoCtx::from_rados_ioctx_t

IoCtx::from_rados_ioctx_t creates an IoCtx out of a rados_ioctx_t.
However, this IoCtx must share ownership of the IoCtxImpl pointer with
the C API user who first called rados_ioctx_create. This must be done
via a reference count inside the IoCtxImpl.

Also add a copy constructor and assignment operator to class IoCtx,
since it's now cheap to have them.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Rename radios_ioctx_{open,close} to create/destroy

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: remove IoCtx::close()

We decided we don't want IoCtx::close(), since IoCtx::~IoCtx() exists.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Make git ignore core files.

Make git ignore python generated files.

librados: Crashed on shutdown if connect was never called.

Add a trivial unit test to trigger this.

libradoshpp: put ceph stuff in namespace librados

Try a little bit harder to avoid polluting the user's global namespace
with our stuff.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: don't create unused SnapContext objs

There were some unused temporary variables hanging around.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>