test_stress_watch: exercise watch/unwatch/close from second client

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

osd: ack pending notify when unwatching

If an unwatch crosses paths with a notify, we currently force the notifier
to keep waiting.  Instead, implicitly ack any notify when we unwatch.

Fixes: #2491
Signed-off-by: Sage Weil <sage@inktank.com>

Merge remote branch 'gh/wip-admin'

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

librbd: Simplify timing init

Remove possibility of set_start_time before set_ictx error

Signed-off-by: Dan Mick <dan.mick@inktank.com>

librbd: Add latency (elapsed-time) stats for rbd operations

Fixes: #2408
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

rgw: put_bucket_info does not override attrs

This fixes #2487. When writing bucket info we just
wrote the object content, and were overriding any
attrs that object contained (that is -- corrupted
the ACLs).

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

rgw: return default policy if policy attr is broken

In case the policy attr on object/bucket is broken
return a default policy, in which the owner is set
as the bucket owner.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

log: add missing .cc file

Signed-off-by: Sage Weil <sage@newdream.net>

log: uninline SubsystemMap mutators

This lets you do something like

 p g_ceph_context->_conf->subsys.set_log_level(ceph_subsys_mon, 20)

from gdb.

Signed-off-by: Sage Weil <sage@newdream.net>

filestore: fix my broken ifdef flow

Signed-off-by: Sage Weil <sage@newdream.net>

doc: ceph.conf cleanup for cephx authentication.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: fix broken link.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Added -a to service ceph stop.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Consolidated mkcephfs deployment files.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

osd: fix null dereference

The pg arg is optional!

Signed-off-by: Sage Weil <sage@inktank.com>

doc: Added pools and authentication.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

mon: fix reweight_by_utilization waiting

Propose if we changed something, but not if we don't.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix rewewight_by_utilization

Update the incremental, not the in-memory OSDMap!

Fixes: #2454
Signed-off-by: Sage Weil <sage@newdream.net>

filestore: log about syncfs(2) etc support

Fixes: #2479
Signed-off-by: Sage Weil <sage@newdream.net>

Merge remote-tracking branch 'gh/stable'

doc: Updated authentication with header canonicalization.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge remote-tracking branch 'gh/wip-2349'

Reviewed-by: Sage Weil <sage@newdream.net>

workloadgen: Fix the statistics output and a bug triggered by coll destruction

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

filestore: initialize dummy xattr buffer

Shut up valgrind.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix misdirected op check

The sense of this check was wrong from the get-go, back in a5d99add.  Fix
it.  This may be silencing a lot of misdirected op noise!

Signed-off-by: Sage Weil <sage@inktank.com>

test_filestore_workloadgen: less noise

Signed-off-by: Sage Weil <sage@newdream.net>

test_filestore_workloadgen: write zeros, not random bytes

The random byte generation is sufficiently slow that it skews the results.
Zeros are good enough!

Signed-off-by: Sage Weil <sage@newdream.net>

admin_socket: fix unittest

Signed-off-by: Sage Weil <sage@inktank.com>

log: add log flush, dump_recent, reopen admin socket commands

Signed-off-by: Sage Weil <sage@inktank.com>

config: add 'set_config <var> <val>' admin socket command

Signed-off-by: Sage Weil <sage@inktank.com>

rados.cc: added listomap and getomap commands

Signed-off-by: Eleanor Cawthon <eleanor.cawthon@inktank.com>

config: add 'show_config' admin socket command

Signed-off-by: Sage Weil <sage@inktank.com>

admin_socket: initialize explicitly on startup; disallow changes

There is an annoying dependency between the config lock and the admin
socket lock due to the fact that we initialize (or reinitialize) the socket
via a config observer.

Instead, explicitly initialize on startup.  Do not allow the admin socket
location to be changed at runtime.

Signed-off-by: Sage Weil <sage@inktank.com>

librados.h: update rados_object_list_next documentation for key/locator

*entry and *key don't need to be freed, and are only valid until the
next call to rados_objects_list_*.

Signed-off-by: Samuel Just <sam.just@inktank.com>

cephcontext: generalize admin socket hook

We'll process non-perfcounter commands too.

Signed-off-by: Sage Weil <sage@inktank.com>

doc: needed carriage returns between headings, code tag and syntax.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: removed redundant install.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Filled in descriptions for object operations.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

ceph.spec.in: buildrequires libuuid-devel

Reported-by: Guido Winkelmann <guido-ceph@thisisnotatest.de>
Signed-off-by: Sage Weil <sage@inktank.com>

ceph.spec.in: drop unnecessary rmdirs

This hunk was inadvertantly included in a330c64.

Reported-by: Guido Winkelmann <guido-ceph@thisisnotatest.de>
Signed-off-by: Sage Weil <sage@inktank.com>

doc: Restored install for dev/unstable packages.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

filestore: make SNAP_DESTROY ioctl check more informative

Tell user about user_subvol_rm_allowed mount option.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: fix btrfs subvol ioctl tests

Create a temp subvol that *we own* to test snaps against.  Otherwise, we
won't be able to remove them if we weren't the owner of the source subvol
we were snapshotting when we are not root.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: verify that op_seq is nonzero

If we get zero from commit_op_seq something is very wrong, and we should
not continue.  Broken mkfs or something deleted commit_op_seq file.

Signed-off-by: Sage Weil <sage@inktank.com>

vstart.sh: wipe out btrfs subvols in osd dirs

For testing on btrfs.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: fix initial btrfs snap creation (on mkfs)

Create the snap_1 snap of current *after* we write the op_seq.  Otherwise
we will mistakingly believe the snapshot is of op seq 0.

Signed-off-by: Sage Weil <sage@inktank.com>

rgw: replace dump_format() with dump_int()

Cleanup, fixes issue #2469.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'stable'

librados, ObjectCacher: include limits.h for INT_MAX

This was required to build on CentOS 6.2. I'm not sure why it worked
without it on Debian.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

rbd.py: Fix seemingly-random segfaults

Avoid resetting Image.closed until image is successfully opened.

Fixes: #2475
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

librados: Remove errant sha1

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

rbd.py doc: Fix markup on :class: references

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge branch 'osd-mkfs'

Reviewed-by: Sam Just <sam.just@inktank.com>

Merge remote-tracking branch 'gh/wip-mon-auth'

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'stable'

filestore: make read_fsid() out arg be a pointer

As per style guide.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: more informative fsid mkfs debug output

Signed-off-by: Sage Weil <sage@inktank.com>

osd: verify fsid whoami to match on repeat mkfs

Signed-off-by: Sage Weil <sage@inktank.com>

osd: require cluster fsid for mkfs

Signed-off-by: Sage Weil <sage@inktank.com>

osd: do not clobber existing key with --mkkey

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: do not clobber exisiting journal if it has matching fsid

If the journal is for *this* fs, do not clobber.  If it appears invalid in
any way, initialize it.

Signed-off-by: Sage Weil <sage@inktank.com>

filejournal: add check() method

Check if a journal appears to be valid, where valid means the header is
intact and has a matching fsid.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: drop useless mkjournal error check

We don't care!

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: less noise about leveldb creation

and make it clear that we didn't necessarily create it; it may have already
been there.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: only create snap_0 if we created current_op_seq

We only need to create snap_0 if we just created current_op_seq and this is
a brand-new dir.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: fix misplaced #endif

Signed-off-by: Sage Weil <sage@inktank.com>

doc/api/s3/bucketops.rst Corrected heading.
doc/install/chef.rst  Added Sam's edits to the Chef install.

Signed-off-by: John <john@admin-host.(none)>

rgw: replace all bad use of formatter->dump_format()

This fixes #2465. We should never ever use the dumped value
as the format. Never.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

doc/api/s3 <-chipping away at objectops and clean up. Added Copy Object.
doc/source <-fixed a typo, updates for installing on precise, rm $ from cmds
doc/start  <-replaced clone link with tarball link. Link to Ceph Source Code.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

mon: use temp access_{r,all,cmd} variables to simplify the checks

Avoid duplicating the compound check all over the place.

Signed-off-by: Sage Weil <sage@inktank.com>

mon: drop redundant get_allow_all() cap check

check_privileges() already does this.

Signed-off-by: Sage Weil <sage@inktank.com>

monmap: return ENOENT from build_from_host_list when no mons

Make sure we return an error when no monitors are parsed.

Signed-off-by: Sage Weil <sage@inktank.com>

monmap: ignore conf mon entries when -m is specified

Use -m <addrs> instead of conf mons (instead of combining them into a
single large search pool).

Signed-off-by: Sage Weil <sage@inktank.com>

v0.47.2

Makefile: do not install librgw.h

Signed-off-by: Sage Weil <sage@newdream.net>

Update ceph.spec for ceph-0.47

Add BuildRequires: libxml2-devel.
Move BuildRequires: libcurl-devel to a more proper place.

[libs3 and librgw.h bits of patch dropped -sw]

Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br>
Signed-off-by: Sage Weil <sage@newdream.net>

libs3: remove make install target

Makefile.am: only append libs3 to SUBDIRS when needed

We appended libs3 to SUBDIRS unconditionally, we only need that
when we actually want to compile it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

monclient: sanity check on monmap size

Signed-off-by: Sage Weil <sage@newdream.net>

monmap: avoid dup addrs in generated initial monmap

We were getting a dup addr from -m <foo> and ceph.conf.  Prefer the
ceph.conf name in this case.

Also check for dup names here just to be thorough (even though I don't
think that can come up currently).

Fixes: #2464
Signed-off-by: Sage Weil <sage@newdream.net>

monmap: aseert addrs are unique on add(), calc_ranks()

Signed-off-by: Sage Weil <sage@newdream.net>

doc: librados: fix getxattr return code

Fixes: #2391
Signed-off-by: Sage Weil <sage@inktank.com>

mon: restrict mon 'join' message/command

Signed-off-by: Sage Weil <sage@inktank.com>

mon: apply 'r' and 'w' caps to mon and pg commands

Signed-off-by: Sage Weil <sage@inktank.com>

mon: apply 'r' and 'w' caps to mdsmap

Signed-off-by: Sage Weil <sage@inktank.com>

mon: apply 'r' and 'w' caps to osdmap

Signed-off-by: Sage Weil <sage@inktank.com>

mon: allow health, status, log, etc. messages with 'r' cap

Signed-off-by: Sage Weil <sage@inktank.com>

mon: fix error messages

Signed-off-by: Sage Weil <sage@inktank.com>

mon: push down 'auth ...' command auth checks

Signed-off-by: Sage Weil <sage@inktank.com>

mon: push down command auth checks to each command

Everything except 'fsid', which I figure is truly public information?

Signed-off-by: Sage Weil <sage@inktank.com>

mon: require admin privs to issue any monitor commands

This is overkill, but a first step before pushing caps enforcement down
into each subfunction that processes monitor commands.

Fixes: #2443
Signed-off-by: Sage Weil <sage@inktank.com>

vstart.sh: set HOSTNAME when -l is used

Signed-off-by: Sage Weil <sage@inktank.com>

mon: fix 'get-or-create-key'

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: leave existing current/

If current exists, leave it be, but:

 * make sure it's a dir
 * check if it's a btrfs snapshot, and set btrfs_stable_commits if so

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: create but do not clobber current/commit_op_seq

If it exists, preserve its value.  Otherwise, create and set it to 1.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: only (re)set fsid if not already set

Only set fsid if it is not set.

Also, error out if it exists and does not match.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: pass target into read_fsid()

Don't assume it should be read into this->fsid.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: rename buf -> fsid_fn

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: always open/verify basedir; clean up error messages

Verify basedir exists immediately.
Clean up error output.
Clean up label names.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: do not wipe on mkfs

Do not wipe out old content in the data dir.

Signed-off-by: Sage Weil <sage@inktank.com>