doc: Consolidated mkcephfs deployment files.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

osd: fix null dereference

The pg arg is optional!

Signed-off-by: Sage Weil <sage@inktank.com>

doc: Added pools and authentication.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

mon: fix reweight_by_utilization waiting

Propose if we changed something, but not if we don't.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix rewewight_by_utilization

Update the incremental, not the in-memory OSDMap!

Fixes: #2454
Signed-off-by: Sage Weil <sage@newdream.net>

filestore: log about syncfs(2) etc support

Fixes: #2479
Signed-off-by: Sage Weil <sage@newdream.net>

Merge remote-tracking branch 'gh/stable'

doc: Updated authentication with header canonicalization.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

Merge remote-tracking branch 'gh/wip-2349'

Reviewed-by: Sage Weil <sage@newdream.net>

workloadgen: Fix the statistics output and a bug triggered by coll destruction

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>

filestore: initialize dummy xattr buffer

Shut up valgrind.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix misdirected op check

The sense of this check was wrong from the get-go, back in a5d99add.  Fix
it.  This may be silencing a lot of misdirected op noise!

Signed-off-by: Sage Weil <sage@inktank.com>

test_filestore_workloadgen: less noise

Signed-off-by: Sage Weil <sage@newdream.net>

test_filestore_workloadgen: write zeros, not random bytes

The random byte generation is sufficiently slow that it skews the results.
Zeros are good enough!

Signed-off-by: Sage Weil <sage@newdream.net>

rados.cc: added listomap and getomap commands

Signed-off-by: Eleanor Cawthon <eleanor.cawthon@inktank.com>

librados.h: update rados_object_list_next documentation for key/locator

*entry and *key don't need to be freed, and are only valid until the
next call to rados_objects_list_*.

Signed-off-by: Samuel Just <sam.just@inktank.com>

doc: needed carriage returns between headings, code tag and syntax.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: removed redundant install.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

doc: Filled in descriptions for object operations.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

ceph.spec.in: buildrequires libuuid-devel

Reported-by: Guido Winkelmann <guido-ceph@thisisnotatest.de>
Signed-off-by: Sage Weil <sage@inktank.com>

ceph.spec.in: drop unnecessary rmdirs

This hunk was inadvertantly included in a330c64.

Reported-by: Guido Winkelmann <guido-ceph@thisisnotatest.de>
Signed-off-by: Sage Weil <sage@inktank.com>

doc: Restored install for dev/unstable packages.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

filestore: make SNAP_DESTROY ioctl check more informative

Tell user about user_subvol_rm_allowed mount option.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: fix btrfs subvol ioctl tests

Create a temp subvol that *we own* to test snaps against.  Otherwise, we
won't be able to remove them if we weren't the owner of the source subvol
we were snapshotting when we are not root.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: verify that op_seq is nonzero

If we get zero from commit_op_seq something is very wrong, and we should
not continue.  Broken mkfs or something deleted commit_op_seq file.

Signed-off-by: Sage Weil <sage@inktank.com>

vstart.sh: wipe out btrfs subvols in osd dirs

For testing on btrfs.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: fix initial btrfs snap creation (on mkfs)

Create the snap_1 snap of current *after* we write the op_seq.  Otherwise
we will mistakingly believe the snapshot is of op seq 0.

Signed-off-by: Sage Weil <sage@inktank.com>

rgw: replace dump_format() with dump_int()

Cleanup, fixes issue #2469.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'stable'

librados, ObjectCacher: include limits.h for INT_MAX

This was required to build on CentOS 6.2. I'm not sure why it worked
without it on Debian.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

rbd.py: Fix seemingly-random segfaults

Avoid resetting Image.closed until image is successfully opened.

Fixes: #2475
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

librados: Remove errant sha1

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

rbd.py doc: Fix markup on :class: references

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Merge branch 'osd-mkfs'

Reviewed-by: Sam Just <sam.just@inktank.com>

Merge remote-tracking branch 'gh/wip-mon-auth'

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'stable'

filestore: make read_fsid() out arg be a pointer

As per style guide.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: more informative fsid mkfs debug output

Signed-off-by: Sage Weil <sage@inktank.com>

osd: verify fsid whoami to match on repeat mkfs

Signed-off-by: Sage Weil <sage@inktank.com>

osd: require cluster fsid for mkfs

Signed-off-by: Sage Weil <sage@inktank.com>

osd: do not clobber existing key with --mkkey

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: do not clobber exisiting journal if it has matching fsid

If the journal is for *this* fs, do not clobber.  If it appears invalid in
any way, initialize it.

Signed-off-by: Sage Weil <sage@inktank.com>

filejournal: add check() method

Check if a journal appears to be valid, where valid means the header is
intact and has a matching fsid.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: drop useless mkjournal error check

We don't care!

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: less noise about leveldb creation

and make it clear that we didn't necessarily create it; it may have already
been there.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: only create snap_0 if we created current_op_seq

We only need to create snap_0 if we just created current_op_seq and this is
a brand-new dir.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: fix misplaced #endif

Signed-off-by: Sage Weil <sage@inktank.com>

doc/api/s3/bucketops.rst Corrected heading.
doc/install/chef.rst  Added Sam's edits to the Chef install.

Signed-off-by: John <john@admin-host.(none)>

rgw: replace all bad use of formatter->dump_format()

This fixes #2465. We should never ever use the dumped value
as the format. Never.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

doc/api/s3 <-chipping away at objectops and clean up. Added Copy Object.
doc/source <-fixed a typo, updates for installing on precise, rm $ from cmds
doc/start  <-replaced clone link with tarball link. Link to Ceph Source Code.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>

mon: use temp access_{r,all,cmd} variables to simplify the checks

Avoid duplicating the compound check all over the place.

Signed-off-by: Sage Weil <sage@inktank.com>

mon: drop redundant get_allow_all() cap check

check_privileges() already does this.

Signed-off-by: Sage Weil <sage@inktank.com>

monmap: return ENOENT from build_from_host_list when no mons

Make sure we return an error when no monitors are parsed.

Signed-off-by: Sage Weil <sage@inktank.com>

monmap: ignore conf mon entries when -m is specified

Use -m <addrs> instead of conf mons (instead of combining them into a
single large search pool).

Signed-off-by: Sage Weil <sage@inktank.com>

v0.47.2

Makefile: do not install librgw.h

Signed-off-by: Sage Weil <sage@newdream.net>

Update ceph.spec for ceph-0.47

Add BuildRequires: libxml2-devel.
Move BuildRequires: libcurl-devel to a more proper place.

[libs3 and librgw.h bits of patch dropped -sw]

Signed-off-by: Alexandre Oliva <oliva@lsd.ic.unicamp.br>
Signed-off-by: Sage Weil <sage@newdream.net>

libs3: remove make install target

Makefile.am: only append libs3 to SUBDIRS when needed

We appended libs3 to SUBDIRS unconditionally, we only need that
when we actually want to compile it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

monclient: sanity check on monmap size

Signed-off-by: Sage Weil <sage@newdream.net>

monmap: avoid dup addrs in generated initial monmap

We were getting a dup addr from -m <foo> and ceph.conf.  Prefer the
ceph.conf name in this case.

Also check for dup names here just to be thorough (even though I don't
think that can come up currently).

Fixes: #2464
Signed-off-by: Sage Weil <sage@newdream.net>

monmap: aseert addrs are unique on add(), calc_ranks()

Signed-off-by: Sage Weil <sage@newdream.net>

doc: librados: fix getxattr return code

Fixes: #2391
Signed-off-by: Sage Weil <sage@inktank.com>

mon: restrict mon 'join' message/command

Signed-off-by: Sage Weil <sage@inktank.com>

mon: apply 'r' and 'w' caps to mon and pg commands

Signed-off-by: Sage Weil <sage@inktank.com>

mon: apply 'r' and 'w' caps to mdsmap

Signed-off-by: Sage Weil <sage@inktank.com>

mon: apply 'r' and 'w' caps to osdmap

Signed-off-by: Sage Weil <sage@inktank.com>

mon: allow health, status, log, etc. messages with 'r' cap

Signed-off-by: Sage Weil <sage@inktank.com>

mon: fix error messages

Signed-off-by: Sage Weil <sage@inktank.com>

mon: push down 'auth ...' command auth checks

Signed-off-by: Sage Weil <sage@inktank.com>

mon: push down command auth checks to each command

Everything except 'fsid', which I figure is truly public information?

Signed-off-by: Sage Weil <sage@inktank.com>

mon: require admin privs to issue any monitor commands

This is overkill, but a first step before pushing caps enforcement down
into each subfunction that processes monitor commands.

Fixes: #2443
Signed-off-by: Sage Weil <sage@inktank.com>

vstart.sh: set HOSTNAME when -l is used

Signed-off-by: Sage Weil <sage@inktank.com>

mon: fix 'get-or-create-key'

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: leave existing current/

If current exists, leave it be, but:

 * make sure it's a dir
 * check if it's a btrfs snapshot, and set btrfs_stable_commits if so

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: create but do not clobber current/commit_op_seq

If it exists, preserve its value.  Otherwise, create and set it to 1.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: only (re)set fsid if not already set

Only set fsid if it is not set.

Also, error out if it exists and does not match.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: pass target into read_fsid()

Don't assume it should be read into this->fsid.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: mkfs: rename buf -> fsid_fn

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: always open/verify basedir; clean up error messages

Verify basedir exists immediately.
Clean up error output.
Clean up label names.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: do not wipe on mkfs

Do not wipe out old content in the data dir.

Signed-off-by: Sage Weil <sage@inktank.com>

filestore: remove unused 'filestore dev' option

Eons ago we would call mount/unmount on this, but it's been dead for a
while now.

Signed-off-by: Sage Weil <sage@inktank.com>

osd: only (re)write_meta files if old content is incorrect

Do nothing if the file already exists with the correct content.

Signed-off-by: Sage Weil <sage@inktank.com>

osd: only create superblock if not present

If it exists, leave it be.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'wip-quorum'

doc/config-cluster/chef.rst <-fixed an error in upload command.
doc/config-cluster/deploying-ceph-conf.rst <-added sudo. required.
doc/config-cluster/deploying-ceph-with-mkcephfs.rst <-cd to /etc/ceph so keyring goes there.
doc/install/chef.rst <-added update and install opscode-keyring, and upgrade.
doc/install/debian.rst <-added ceph-common to the install

Signed-off-by: John Wilkins <john.wilkins@dreamhost.com>

config: add --show-args option

The --show-args dumps the arguments passed on the command line.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Merge branch 'stable'

objectcacher: fix infinite loop in flusher_entry

The addition of accounting for simultaneous writers in
f3043fee3e22600cb4349072287842db129588eb could lead to an infinite
loop. This could happen because starting a flush doesn't decrease
dirty_waiters, only the number of dirty bytes (they change to tx). If
dirty_waiters is more than target_dirty, the loop would continue
forever while holding the object cacher lock, since the object cacher
lock is required by the write callbacks.

The extra while (!flusher_stop) loop is unnecessary, so remove it.
This means the flusher thread always releases the lock after starting
flushing, so progress can be made.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>

Merge branch 'master' of github.com:ceph/ceph

doc/config-cluster/ceph-conf.rst <-Fixed some formatting issues.
doc/config-cluster/deploying-ceph-conf.rst <-minor edit.
doc/start/get-involved-in-the-ceph-community.rst <-removed $ from command line instruction.
doc/start/quick-start.rst <-added additional steps.

Signed-off-by: John Wilkins <john.wilkins@dreamhost.com>

monmap: calc_ranks() on rename()

This is simpler than ensuring we track all the right invariants.

Signed-off-by: Sage Weil <sage@inktank.com>

Merge branch 'wip-defensive-alloc'

crush: check reweight return value

Signed-off-by: Sage Weil <sage@inktank.com>

doc: fix broken link in FAQ

Signed-off-by: Ross Turk <ross@inktank.com>

doc: added links from the chef install doc to the chef config doc and back

Signed-off-by: Ross Turk <ross@inktank.com>

doc: Added a FAQ page.

Added a FAQ page, which we will need to put more content in.

Signed-off-by: Ross Turk <ross@inktank.com>

Added the Swift API as is. Still needs a lot of work.

Signed-off-by: John Wilkins <john.wilkins@dreamhost.com>

obj_bencher: add --show-time option for dumping time

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

doc: document range of methods for cluster creation, expansion

This isn't quite complete, and should be integrated with the info in the
ops section on mon cluster expansion, but the raw content is here at least.

Signed-off-by: Sage Weil <sage@inktank.com>