config: print warning on injectargs parse fail

injectargs should print a warning message when its parser fails.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

injectargs test: test foo=true

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon: allow wildcards for mon cap command matching

We need wildcards, actually.  * is any token, ... is 1 or more tokens.

Signed-off-by: Sage Weil <sage@newdream.net>

common/config: fix get_val for booleans

md_config_t::get_val should return true/false for booleans, not 0/1.
This is for consistency with the setter.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

qa: add unit tests for mon cap 'allow command ...' stanzas

Signed-off-by: Sage Weil <sage@newdream.net>

mon: whitelist individual commands in mon cap string

This lets you whitelist a command prefix.  For example,

 'allow command osd foo'

will let you do any command that begins with 'osd foo', including
'osd foo' and 'osd foo bar baz bam'.

No deny, and no pattern matching.

Signed-off-by: Sage Weil <sage@newdream.net>

mon: adjust max_devices when adding to crush map

Signed-off-by: Sage Weil <sage@newdream.net>

crush: refuse to add item >= max_devices

Signed-off-by: Sage Weil <sage@newdream.net>

mon: 'osd gstcrushmap' or 'osd crush set' both work

Signed-off-by: Sage Weil <sage@newdream.net>

mon: fix arg count checks

Signed-off-by: Sage Weil <sage@newdream.net>

mon: 'osd crush reweight <name> <weight>'

Reweight a node in the crush map.

Signed-off-by: Sage Weil <sage@newdream.net>

mon: 'osd crush rm <name>'

This is similar to crushtool --remove-item.

Signed-off-by: Sage Weil <sage@newdream.net>

mon: 'osd crush add <id> <name> <weight> [<loc1=foo> [<loc2=bar> ...]]'

This is identical to the crushtool --add-item functionality, but does not
require a racy and tedious

 - get crush map
 - update crush map
 - set new crush map

Signed-off-by: Sage Weil <sage@newdream.net>

mon: return error message string when no other output string specified

Signed-off-by: Sage Weil <sage@newdream.net>

common/config: honor --

Argument parsing should stop after --

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mon: remove dead 'setmap' code

Signed-off-by: Sage Weil <sage@newdream.net>

mon: make auto-marking of booting osds 'in' optional

By default we mark a booting OSD 'in' so that it immediately gets data.
Make this optional.

Eventually we may want to make this default to off...

Signed-off-by: Sage Weil <sage@newdream.net>

client: kill dentry_of() macro

Signed-off-by: Sage Weil <sage@newdream.net>

Merge branch 'next'

osd_types: fix category encoding in object_info_t

This was mis-merged in f6ff0f14e9a5b70d5ce0277975d321f10e3b39c7.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: define _GNU_SOURCE for qa helpers; add btrfs dir makefile

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

librados: whitespace

Don't do

some::class::
method(int foo)
{

}

...because then I can't '^S::method'. :)

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

librbd: reimplement copy using read_iterate

Ah, so much simpler!

Signed-off-by: Sage Weil <sage@newdream.net>

rbd: fix check_io for snap

Need to look at image size for snapshot, not the head version.

Signed-off-by: Sage Weil <sage@newdream.net>

librbd: fix copy

This was croaking on sparse images.

Use an image ctx for the dest.

We should probably use read_iterate instead.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: add rbd copy workunit

Test copying an image and snapshot.

Signed-off-by: Sage Weil <sage@newdream.net>

mdcache: documentation for path_traverse

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mdcache: Reduce the gratuitous references to onfail

We've been setting flags at the beginning of path_traverse
for a while; use those instead of ongoing comparisons.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: remove the unused MDS_TRAVERSE_FAIL flag.

This isn't used, and judging by the comments was a horrible lie.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mdcache: handle unreadable null dentries

Previously, we were asserting (farther down) if the dentry was
null but blocked by a lock. Instead, we should here be waiting
until it's readable instead of just continuing on.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

librbd: specify copy src image as image handle

By specifying via an image handle we can set the snapshot on the src
image.  This also makes the API cleaner.

Fixes: #1416
Signed-off-by: Sage Weil <sage@newdream.net>

Define _GNU_SOURCE in src/Makefile.am

We end up needing _GNU_SOURCE in a bunch of places-- to get direct i/o,
pipe2, and some other Linux-specific interfaces.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

pipe.c: #define GNU_SOURCE

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'next'

mds: open+pin stray dirfrags on startup

This ensures that the stray dirfrags are always open, which in turn ensures
that whenever we add straydn items the rstats/fragstats will get updated
properly.  This is a better solution than d3d767a.

Now we can assert the stray dirfrag is open in
get_or_create_stray_dentry() instead of calling get_or_open_dirfrag().

Signed-off-by: Sage Weil <sage@newdream.net>

Revert "mds: do not complain/assert about stray inode rstat/fragstat consistency"

This reverts commit d3d767a2a36caefdeb8a94abba1673cf8304c793.

A better solution is to keep the stray frags loaded+pinned.

common/pipe: add function to do pipe2(O_CLOEXEC)

Support old systems that don't have O_CLOEXEC.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

PG: Move reset_last_warm_restart to Initial::exit

Previously, reset_last_warm_restart was only invoked when handle_create
was used.  This misses cases where the pg is initialized via a Notify,
Log, or Info message.  reset_last_warm_restart will now be called from
the Initial state exit handler in order to handle the other cases.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

common/pipe: add function to do pipe2(O_CLOEXEC)

Support old systems that don't have O_CLOEXEC.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

PG: Move reset_last_warm_restart to Initial::exit

Previously, reset_last_warm_restart was only invoked when handle_create
was used.  This misses cases where the pg is initialized via a Notify,
Log, or Info message.  reset_last_warm_restart will now be called from
the Initial state exit handler in order to handle the other cases.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

osd: flush writes prior to PGLS

Make sure previous writes (which will include both object creation and
PG creation) are flushed before listing objects.

Fixes: #1427
Signed-off-by: Sage Weil <sage@newdream.net>

osd: remove dead code, whitespace

Signed-off-by: Sage Weil <sage@newdream.net>

librados: prefix api test pools

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

librados: statically link gtest into librados unit tests

This way they can run via teuthology.

We should do this for anything that we expect to run outside of the
immediate build environment.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Makefile: allow gtest to be linked in statically

Use UNITTEST_STATIC_LDADD instead of UNITTEST_LDADD.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

.gitignore: scratchtool

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

Makefile: rule to upload + submit coverity job

You need to

 make project.tgz
 make submit-coverity

and have .coverity.build.pass with the right password in CWD
(ceph.git/src).

The submit-coverity rule doesn't do the project.tgz for you because it
always cleans+rebuilds every time.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

addr_parsing: fix warning

warning: include/addr_parsing.c:45: ‘saveptr’ may be used uninitialized in this function

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

escape: fix snprintf args

Include space for null termination.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

.gitignore: scratchtool

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

librados: fix shadow

Coverity cid 23

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: fix shadow

Coverity cid 24

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: fix shadow

Coverity cid 25

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: fix shadow

Coverity cid 26

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: fix shadow

Coverity cid 27

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: fix shadow

Coverity cid 28

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

moncaps: fix shadow

Coverity cid 30

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

monclient: fix shadow

Coverity cid 31

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

monclient: fix shadow

Coverity cid 32

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msg: fix shadow

Coverity cid 33

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: fix shadow

Coverity cid 34

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osdcaps: fix shadow

Coverity cid 35

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

escape: use snprintf

Coverity cid 41, 40

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

secret: use strncpy, snprintf

Coverity cid 42

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

addr_parsing: use strncpy

Coverity cid 43

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

addr_parsing: avoid strcpy

Make coverity shut up.

Coverity cid 44

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mount: fix path leak

canonicalize_path() does not free/touch it's argument; no need to strdup()
here.

Coverity cid 39

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mount: avoid unguarded strcpy

Use strdup here, mostly to make coverity shut up.

Coverity cid 45

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

test_ioctls: reduce stack use

Coverity cid 51

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

formatter: less big buffer for dump format string

Coverity cid 53

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mount: avoid big stack item

Coverity cid 54

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mount: avoid big stack item

Coverity cid 55

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

testlibrbd: add missing va_end()

Coverity cid 57, 58

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rgw: remove redundant line

mds: do not complain/assert about stray inode rstat/fragstat consistency

We instantiate the stray dirfrags without reading the fragstat off of disk
because it's faster, we know the dentry is unique, and we don't care about
the stats.  It can lead to inconsistency between the dirfrag frag/rstat
and the inodes, though.  Silently clean it up when we hit it; that's
simpler than not maintaining it at all for those directories.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

paxos: fix store_state

Stop if we empty out the map completely.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

monclient: be less chatty about stray messages

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

debian: librbd headers require librados headers

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

scratchtool: not a test suite

Revert "Remove old rados tests"

This reverts commit 665fd8214d4f8fa54a675c2998b22613424a6cb3.

Conflicts:

src/testradospp.cc

osd: fix compilation

osd: call incorrect osdmap addrs errors

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rgw: fail operation when trying to clone to the same shadow

not necessary to recreate the same shadow, and also provides mutual
exclusion

mds: xlocker_caps are supplemental to caps

Like loner_caps, xlocker_caps are additional caps to the any caps.

In practical terms, this only affects (currently) the LOCK_XLOCKDONE state
for the filelock, where it's less work than making sure what is in the any
column is also |'d onto the xlocker column.  Easier to read :)

Signed-off-by: Sage Weil <sage@newdream.net>

mds: only client hold xlock in xlockdone can change lock state

If we are in xlockdone, only the client holding that xlock can adjus the
lock state (e.g., relock).  Other clients have to wait until the xlock
cycle unwinds completely.

Fixes: #1417
Signed-off-by: Sage Weil <sage@newdream.net>

rgw: fix prepare_atomic_for_write

mds: update truncate_pending on O_TRUNC; pass old_size on O_TRUNC open

We were missing truncate_pending++ in O_TRUNC path.  Use a common helper.

Pass old_size on O_TRUNC open.  (This was the real bug we worked around in
79db9fc509933429f4a8f4aca6d3da5517c43d1f.)

Only do an actual truncate if old_size > 0.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rados-api: fix #include order

Fixes conflict with linux types on fatty.

Signed-off-by: Sage Weil <sage@newdream.net>

cconf: fix clitest

Signed-off-by: Sage Weil <sage@newdream.net>

cconf: --filter-key and --filter-key-value

Filter out section list to sections that define a given key or that
set a key to a given value.

This is useful to, for example, filter out sections mapped to the current
host.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: fix LOCK_LOCK_XLOCK lock state name/string

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: add missing cache locking

this was overlooked when switch to the multithreaded configuration

rgw: was missing return value

rgw: some more race fixes

test/rados-api/misc.cc: update tests after API change

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

monitorstore: drop unused sync value

This is inherently unsafe, because we write to a final filename without
ensuring the data is on disk.

Signed-off-by: Sage Weil <sage@newdream.net>

paxos: use MonitorStore::put_bl_sn_map() to commit batches of values

This allows us to (safely) do fsync vs sync optimizations.  The old code
would write values to the final names and then sync(2), but a crash in
between could leave 0-length (non-temporary) files and crash.

Fixes: #1414
Signed-off-by: Sage Weil <sage@newdream.net>

monitorstore: implement put_bl_sn_map

This will commit a big batch of values safely.  We push it down into
MonitorStore so we can avoid an fsync(2) per file.

Signed-off-by: Sage Weil <sage@newdream.net>

mon: whitespace

Signed-off-by: Sage Weil <sage@newdream.net>

atomic_t: update spinlock implementation.

Somebody added a 'set' function without adding the non-atomic_ops
version!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>