cconf: fix clitest

Signed-off-by: Sage Weil <sage@newdream.net>

cconf: --filter-key and --filter-key-value

Filter out section list to sections that define a given key or that
set a key to a given value.

This is useful to, for example, filter out sections mapped to the current
host.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: fix LOCK_LOCK_XLOCK lock state name/string

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: add missing cache locking

this was overlooked when switch to the multithreaded configuration

rgw: was missing return value

rgw: some more race fixes

test/rados-api/misc.cc: update tests after API change

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

monitorstore: drop unused sync value

This is inherently unsafe, because we write to a final filename without
ensuring the data is on disk.

Signed-off-by: Sage Weil <sage@newdream.net>

paxos: use MonitorStore::put_bl_sn_map() to commit batches of values

This allows us to (safely) do fsync vs sync optimizations.  The old code
would write values to the final names and then sync(2), but a crash in
between could leave 0-length (non-temporary) files and crash.

Fixes: #1414
Signed-off-by: Sage Weil <sage@newdream.net>

monitorstore: implement put_bl_sn_map

This will commit a big batch of values safely.  We push it down into
MonitorStore so we can avoid an fsync(2) per file.

Signed-off-by: Sage Weil <sage@newdream.net>

mon: whitespace

Signed-off-by: Sage Weil <sage@newdream.net>

atomic_t: update spinlock implementation.

Somebody added a 'set' function without adding the non-atomic_ops
version!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

rgw: reread obj meta on racing write operation

ReplicatedPG: Don't crash when a clone is missing

Print an error rather than crashing when _scrub finds a missing
clone.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

file_layout.sh: add test for setting a new data pool

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Merge branch 'wip-librados-tests'

Conflicts:
src/include/rados/librados.hpp
src/librados.cc
src/rgw/rgw_rados.cc
src/testradospp.cc

Remove old rados tests

These tests have been supplanted by the rados-api gtests.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw: fix locator for shadow objects

mds: share max_size on any client_range adjustment

We were missing this on open + O_CREAT (now triggered by cfuse), on open +
O_TRUNC, and on mknod.

Signed-off-by: Sage Weil <sage@newdream.net>

add WatchNotifyTimeoutTestPP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw: fix a double free in GetObj

test/rados-api/misc.cc: test CloneRange

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/misc.cc: test CloneRangePP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/misc.cc: test AioOperatePP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: fix _activate_committed() crash

Do not dereference acting[0] unless we know it is still valid.

Take a reference when scheduling the transaction, and drop it in the
completion, to ensure that the PG isn't removed out from underneath us.

Fixes: #1379
Signed-off-by: Sage Weil <sage@newdream.net>

test/rados-api/misc.cc: test Operate2PP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Operate1PP: test CEPH_OSD_CMPXATTR_OP_EQ

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

compatset: dump json

Signed-off-by: Sage Weil <sage@newdream.net>

librados: remove unused operate param

The last parameter of operate, aio_operate was unused and unchecked.
Remove it.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

uninline a bunch of buffer.h stuff

uninline osd types

mds: add debug_mds_locker

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

librbd: rbd_list: return buffer size, not element count

Fixes: #1397
Signed-off-by: Sage Weil <sage@newdream.net>

mon: add/remove data_pg_pools from mdsmap

Signed-off-by: Sage Weil <sage@newdream.net>

PG: last_warm_restart must be set to the current epoch

Previously, last_warm_restart did not get set until the
first time the pg was restarted.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

mds: explain weird encoding

Signed-off-by: Sage Weil <sage@newdream.net>

mds: whitespace

Signed-off-by: Sage Weil <sage@newdream.net>

mds: drop unused projected dirfrag_t arg for EMetaBlob::add_primary_dentry

Not used.  If it is later, we need to project it properly.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: drop unused snapbl arg

Signed-off-by: Sage Weil <sage@newdream.net>

mds: avoid explicit passing of projected_xattrs

No need to pass this in explicitly; we can look in the projected inode for
it.  This actually fixes a race where a journaled inode following a
setxattr will not journal the projected xattrs.

Signed-off-by: Sage Weil <sage@newdream.net>

mds: avoid passing snapbl around explicitly

We now properly track projected snaprealm/sr_t's.  Use that for journaling
instead of manually passing around a snapbl.

Signed-off-by: Sage Weil <sage@newdream.net>

test/rados-api/misc.cc: test Operate1PP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: use snaprealm projection for MDCache::snaprealm_create()

Signed-off-by: Sage Weil <sage@newdream.net>

mds: simplify sr_t journaling

Drop encode/decode in SnapRealm and encode the sr_t's directly.  This way
the caller says what they mean (current or projected).

Signed-off-by: Sage Weil <sage@newdream.net>

mds: journal inode on mksnap

We were cowing but not journaling a simply-dirty inode.  Der.

Fixes: #1389
Signed-off-by: Sage Weil <sage@newdream.net>

mds: handle O_TRUNC when size is already 0

We always want to go through the truncation path.  Two reasons:

 - even if the size is already 0, we still need to update ctime/mtime
 - we may not have the correct size (client could have dirty data), in
   which case we still want to bump truncate_seq etc.

Basically, we can't trust pi->size here because we are holding a wrlock,
not an xlock.  And that's fine, since we need to update the inode
unconditionally anyway.

This broke cfuse pjd open/00.t tests when we added the fuse option
atomic_o_trunc.  libceph has always been broken in this regard.

Fixes: #1393
Signed-off-by: Sage Weil <sage@newdream.net>

client: send O_TRUNC open requests to MDS

Signed-off-by: Sage Weil <sage@newdream.net>

client: move Inode operator<<

Signed-off-by: Sage Weil <sage@newdream.net>

Test rados_exec, C and C++ version

* Test rados_exec, C and C++ version

* add test_exec action

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/misc.cc: test tmap_update

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rbd: remove dup 'rm' command from man page

Fixes: #1395
Signed-off-by: Sage Weil <sage@newdream.net>

cfuse: take out accidental bad channel add

This was a brainstorm that snuck into last commit, grr.

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: fix virtual bucket name handling

qa/workunits/rados/test.sh:explicitly invoke tests

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add qa/workunits/rados/test.sh

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

cfuse: make use of fuse invalidate callback optional (and off by default)

This is hitting a deadlock in fuse itself (or so it appears).  Disable it
until that's sorted out.

Signed-off-by: Sage Weil <sage@newdream.net>

formatter: remove bad asserts

rgw: list objects shows time in GMT, thread safety fix

src/perfglue/heap_profiler.cc: fix snprintf

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

client: compile with older libfuse2

Merge branch 'wip-librados-tests'

osd_types: fix merge

Signed-off-by: Sage Weil <sage@newdream.net>

Merge commit 'v0.33'

v0.33

osd: pg_stat_t: throw exception on unknown encoding

This is better then decoding nonsense, eating ram, etc.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: test rename into snapped dir

Tests fix for #1399.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: mark newly created snaprealms as open

Fixes: #1399
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

test/rados-api/misc.cc: test Rados::version

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/list.cc: C++ version

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/snapshots.cc: C++ version

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/watch_notify.cc: implement C++ ver

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/stat.cc: implement C++ tests

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rados/librados.h: use rados_snap_t consistently

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: test RoundTripWriteFull

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: implement FlushPP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: test ReturnValuePP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: test IsSafePP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: add IsCompletePP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: Add RoundTripAppendPP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: add RoundTripPP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Fix bug in turning off logging. Remove log_dir.

Fix a bug that made it impossible to turn off logging to a file.

Remove the old "log_dir" setting. It has been deprecated for a long
time, and it adds a lot of complexity to the code.

heap_profiler: use the directory that log_file is in to construct a
profiler output file name. Don't use log_dir.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'next'

osd: fix heartbeats after bad markdown

The heartbeat start message comes from hbin messenger, which has no port
and a nonce of the pid (at startup).  When we mark ourselves down/up, and
then resend a start, the peer will send a RESETSESSION and the stat
message will get lost, and then we'll miss heartbeats.

Mark down all connections, so that when we reconnect, our start message
is not lost.

Signed-off-by: Sage Weil <sage@newdream.net>

monmaptool: don't inc bool

Coverity cid 22

Signed-off-by: Sage Weil <sage@newdream.net>

crushtool: don't inc bool

Coverity cid 21

Signed-off-by: Sage Weil <sage@newdream.net>

mds: break #include recursion

CInode.h -> SessionMap.h -> CInode.h

Coverity cid 14

Signed-off-by: Sage Weil <sage@newdream.net>

rgw: some encoding fixes

rgw: handle zero and invalid max_keys

rgw: create CommonPrefixes section for each prefix

rgw: adjust returned error when got invalid auth request

obsync: add SRC_SECURE, DST_SECURE

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

LibRadosAio: WaitForSafePP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: SimpleWritePP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rados_setxattr: return 0 on success

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rados_write_full: remove offset parameter

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'next'

librbd: output copy progress

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

mds: avoid issue_caps on snapped inodes

Only head inodes have caps.  Don't set need_issue if it's not a head inode.

This is cleaner than bailing out of issue_caps(); we shouldn't get to that
point. The places that set need_issue = true usually set gather++ too,
which also shouldn't happen on a snapped inode.

Fixes: #1390
Reported-by: Damien Churchill <damoxc@gmail.com>
Signed-off-by: Sage Weil <sage@newdream.net>

test/rados-api/snapshots.cc:RollbackPP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/snapshots.cc:SnapGetNamePP

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>