test/rados-api/test.cc: add create_one_pool_pp,etc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: add IoCtx::close

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add test/rados-api/misc.cc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/snapshots: add SelfManagedSnapTest

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados:fix name of rados_ioctx_snap_set_read/wr

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/snapshots.cc: test SnapGetName

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/snapshots.cc: test Rollback

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/snapshots.cc: test SnapRemove

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/snapshots.cc: test SnapList

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add test/rados-api/watch_notify.cc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api: move TestAlarm into test.h

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/stat.cc: test ClusterStat,PoolStat

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add test/rados-api/stat.cc: test rados_stat

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/pool.cc:test PoolCreateWithCrushRule

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/pool.cc: test rados_ioctx_get_id

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/pool.cc: add AuidTest1, AuidTest2

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/pool.cc: test PoolLookup,PoolDelete

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

.gitignore: add test_rados_api_*

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

create_one_pool: improve error reporting

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add test/rados-api/pool.cc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add test_rados_api_list

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rados/librados.h: remove rados_aio_get_obj_ver

Remove the unused and unimplemented rados_aio_get_obj_ver from the
header file.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: add RoundTripWriteFull

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: fix shadowed class variable

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: test Flush

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: add ReturnValue

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: test IsSafe

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: test IsComplete

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: use rados_aio_release

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio.cc: add RoundTripAppend

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/aio: add WaitForSafe, RoundTrip

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add test/rados-api/aio: SimpleWrite

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/io.cc: add RmXattr, XattrIter

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/io: add XattrsRoundTrip

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/io: add RemoveTest

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/io: add TruncTest

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/io: add AppendRoundTrip

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/io: add WriteFullRoundTrip

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/rados-api/io: add roundtrip, overlappingwrite

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Makefile.am: build gtest in debug builds

Formerly, we only built gtest when doing 'make check'. However, the
librados-api tests, which are built in debug mode for the 'all' target,
also use gtest. So build it when needed.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add test-rados directory, I/O test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'wip-config'

osd: missing ! in ifs on bad map after osd boot

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mds: fix Migrator::audit() for IMPORT_ABORTING

Make audit match reality.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

objecter: halt compound operation only if result < 0

Remove unused dyn_sprintf

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

config.cc: clean up includes

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: chain rename subtree projections

We can have two renames for the same file in flight to the journal.  Stack
them up in a list.  The old project_subtree_rename() should have asserted
that the item wasn't already in the map before inserting it to catch this
at the front end.  Now it doesn't matter; it's a list.

Signed-off-by: Sage Weil <sage@newdream.net>

client: whitespace cleanup

Signed-off-by: Sage Weil <sage@newdream.net>

client: send renames to the correct node

AFAICS we got this wrong in b794aeb39, which is annoying given I remember
thinking reasonably carefully about it.

The current reasoning: the request operates on the dentry target
directory from the MDS's perspective; that's where the request needs to go.

Signed-off-by: Sage Weil <sage@newdream.net>

config: more thread-safety stuff

* Don't allow parse_argv, parse_env, or parse_config_files to be used
after threads have been started.

* Don't allow set_val to be used to change unsafe variables after
threads have been started.

* Test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

injectargs: forbid unsafe string modifications

Don't allow string-valued configuration items to be changed using
injectargs unless they have observers. Otherwise, we could have
crashes, since one thread could be reading the std::string's internal
buffer after another thread frees that buffer during assignment.

Write a unit test to validate this behavior.

Also test that we can turn on and off the log_file using injectargs.
This is something that injectargs often gets used for in practice.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/daemon_config: add injectargs test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add daemon_config, libceph_config, etc.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

add config unit test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados.h: fix out-of-date comment

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

config: more cleanup

* any place where we know a set_val can't fail should be set_val_or_die

* in private md_config_t functions that need the mutex, assert that the
mutex is locked

* make md_config_t::expand_meta private, since nobody uses it outside of
the class.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libceph.h: fix C compilation

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

config: add proper locking, fix comments

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Merge branch 'rgw-formatter'

In RGW, use the common/Formatter

Implement a new subclass of Formatter, RGWFormatter_Plain, using code
from the old RGWFormatter and RGWFormatter_Plain classes.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

filestore: remove unused var

Signed-off-by: Sage Weil <sage@newdream.net>

Formatter: add open_object_section_in_ns, etc.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

test/formatter: test stream clearing

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Formatter: must use str("") to clear ostreams

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

More changes towards using common/Formatter in rgw

* rename RgwFormatter::init to RgwForamtter::reset.
Roll old reset() into base_reset().

* add write_raw_data to Formatter, to support DTDs.
Remove other DTD support code.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

mds: add hooks to test multimds link/unlink failures

Signed-off-by: Sage Weil <sage@newdream.net>

qa: move fsx.sh

Revert "qa: Remove fsx workunit."

This reverts commit a66da4af2571f27f55e8ee819f22e70355fb42b0.  Still useful
to have this around.

os/FileStore: print out BTRFS_IOC_SNAP_CREATE errs

Print out the errno for BTRFS_IOC_SNAP_CREATE errors if they happen.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw_rest: don't leak formatters

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

XmlFormatter: add DTD option

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Formatter.cc: add get_len method

Add get_len method because rgw needs it.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

RGWFormatter: get rid of one flush variant

This flush variant won't exist in common/Formatter.cc.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw_formats: introduce dump_unsigned, dump_int,etc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw: rename dump_value_str to dump_format

rename dump_value_str to dump_format for compatibility with Formatter.cc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rados tool: fix cmdline parsing for --block-size

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Formatter: add XMLFormatter::reset

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Formatter.cc: whitespace fix

whitespace fix to conform to the coding standard

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw:rename open_obj_section to open_object_section

For compatibility with common/Formatter.cc

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Add XMLFormatter and unit test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Formatter: remove protected data, methods

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd, objecter, librados: add SRC_CMPXATTR operation

and test.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: change src_oid encoding -- FLAG DAY

The old encoding was mutually exclusive with putting any data payload on
the operation.  That was stupid.. we can't, for example, do xattr ops then
on a src_oid.

Fix this by just including the oid in the data payload inline whenever the
bit is set in the op code.  This changes the client protocol in an
incompatible way, which means users of the CLONERANGE operation need to be
upgrade/downgraded in unison.

Signed-off-by: Sage Weil <sage@newdream.net>

testradospp: test cmpxattr (eq, str)

No coverage for the other tests yet.

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix cmpxattr string construction

The bufferlist may not contain a terminating '\0'.

Signed-off-by: Sage Weil <sage@newdream.net>

objecter, osd: fix up cmpxattr

Objecter stubs, useful debug output.

Signed-off-by: Sage Weil <sage@newdream.net>

systest: add watch/notify tests with pool and object deletion

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

systest: st_rados_create_pool: optionally wait before creating the pool

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

systest: add object deletion to mirror object creation

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

systest: add st_rados_notify building block for watch/notify tests

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

systest: add st_rados_watch building block for watch/notify tests

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

systest: make pool configurable

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

systest: move pool deletion into its own file

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librados: fix notify deadlock

The success of the notify call needs to checked before waiting to
receive a notification. If we try notifying on an object that does not
exist, for example, it should fail with -ENOENT, and not hang.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

osd: document remove_watchers race avoidance

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

pg: remove do_complete_notify

This method has no dependence on the pg.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

osd: put_object_context: tolerate pgs being deleted

PGs that are queued for deletion won't be in the osdmap,
and may not be in the pg_map, but if they are, it's safe to
put object_context. Otherwise, the pg is being deleted and
will clean up the object contexts itself.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

osd, pg: clean up watchers on pg deletion and shutdown

Watchers and their object contexts need to be cleaned up so
they aren't used after the pg is gone. This happened if the
pool was deleted and the connection to the watcher was reset.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

mds: detach replay thread

Since we don't join it.

This fixes a leak of per-thread state.. namely, an ~8MB chunk of virtual
memory (and a handful of real pages).

Signed-off-by: Sage Weil <sage@newdream.net>