qa: move file_layout into workunits/misc

Signed-off-by: Sage Weil <sage@newdream.net>

radosgw_admin: fix cli test

Merge branch 'master' of ssh://ceph.newdream.net/git/ceph

Merge branch 'master' into rgw

qa: make kernel_untar_build.sh output more helpful

- rm -rv
- list remaining files on failure

to help debug #1230

Signed-off-by: Sage Weil <sage@newdream.net>

testceph: test file opens with extra slashes

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

testceph: expand test cases

Signed-off-by: Richard Sharpe <realrichardsharpe@gmail.com>
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

filepath: don't parse multiple slashes as multiple dname bits.

This causes all kinds of trouble if it occurs because most of the
code isn't prepared for it. So prevent that from happening except
on messages that were explicitly created that way.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Resetter: Check return values from Journaler.

And clean up the shutdown a bit.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Journaler: pay attention to return codes from read head.

Previously we ignored them, except for printing them out. This could
lead to bad things like creating new journals for non-existent MDSes
if you entered an invalid rank during --reset-journal.

Also assert that the stripe unit is valid
before using it as  a divisor.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

ReplicatedPG: Fix some obc leaks

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

PG: simplify merge_log, remove old objects

Previously, some old objects could escape removal during a log merge.
This patch also simplifies merge_log somewhat.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

PG: fix add_next_event and merge add_event

Previously, we would assume that we had an object at the prior_version
in the log event if we encounter it but don't see the object in missing.
Now, if prior_version < log_tail, we assume that we do not have the
object.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

filestore: allow FIEMAP to be disabled via conf

The ext4 fiemap ioctl is buggy in some older kernels.  Allow the admin to
disable it.

Fixes: #1222
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

qa: add file_layout.sh workunit

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: make cast explicit to suppress compiler warning.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Merge branch 'next'

testradospp: fix compilation

mon: Guard injectargs parsing to prevent crashes on bad input

While we're at it, reply to injectargs commands instead of
doing it silently.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mds: Explicitly initialize layout fields, and to the correct values.

We were previously encoding an fl_pg_preferred of 0, which did
horrible things to the kernel client since 0 is a valid osd to ask for!
To make such things easier to track down in the future, explicitly
fill in defaults when memsetting the struct here.

(There remain other places that memset the struct to zero without
a lot of checks. But we definitely don't want to force them all
to fill in the individual fields, as that's fragile, and since they
don't seem to be breaking anything yet I'm inclined to leave them as
they are.)

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

v0.30

debian: drop python-support version req

rados: encode bufferlist in watch-notify

rados: encode bufferlist in watch-notify

rgw: fix some issues with call to clone_range

rgw: fix clone_range

rgw: radosgw_admin --help typo

rgw: implement atomic multipart upload, atomic copy

rgw: atomic put

rgw: radosgw_admin user rm --purge-data

config: remove unused iat options

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

testceph: actually call closedir on opened directory.

Otherwise testceph hangs because it's still got things open!
Fixes #1228

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

qa: initial commit of file_layout.sh

This tests the file layout functionality. Right now you need to
check the comparisons yourself. Expect that to change shortly.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

filer: no need for tid_t here

Signed-off-by: Sage Weil <sage@newdream.net>

client: print cache size during trim for debug

Signed-off-by: Sage Weil <sage@newdream.net>

FileStore: fix error in ALIGN and ALIGN_UP

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

radsogw_admin: fix clitest

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

testrados_delete_pool_while_open: fix printf args

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

radosgw_admin: --help/-h

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

create testrados_delete_pool_while_open

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

PG: assemble_backlog - fix comparison error, tail is non-inclusive

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

client: fix trim_caps

We can't blindly remove caps from inodes because we need at least one cap
for any inode in our cache.  Try to trim non-essential caps (when there's
>1), otherwise try to drop referring dentries and indirectly release caps
that way.

Signed-off-by: Sage Weil <sage@newdream.net>

client: fix mds touch

client: prefer auth cap in caps_issued_mask()

If we have an auth cap, prefer to touch that one.  This helps consolidate
caps on a single mds and allows mds replicas to eventually recall their
state.

Signed-off-by: Sage Weil <sage@newdream.net>

client: touch cap on lookup even if we use the dentry lease

Touch the dir cap for the lease's mds even if we use the lease to traverse.
This makes the trim_caps() behave better because it keeps the dentry and
session cap LRUs in sync.

Signed-off-by: Sage Weil <sage@newdream.net>

client: touch dentries during path traversal

Signed-off-by: Sage Weil <sage@newdream.net>

testrados: add NUM_THREADS

We can now test librados access with multiple simultaneous threads from
a single process.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

.gitignore: update for test_libceph_build

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

global_init: make global_init_set_globals static

Now that the libraries have been de-globalized, this funtion should only
be used by global_init itself.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

CephFSInterface: de-globalize

CephFSInterface: de-globalize

liceph: add ceph_get_mount_context

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

Fix C_NotifyComplete::notify

C_NotifyComplete::notify was broken in an earlier change where I removed
a derr() without removing the if statement before it.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

debian: obsync depends on python-xattr

Signed-off-by: Sage Weil <sage@newdream.net>

cleanup

mon: order degraded before unfound

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: clean up pool/pg_stat_t structs slightly

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

FileStore: ensure that CLONE_RANGE ranges are aligned

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

PG: assemble_backlog - fix comparison error, tail is non-inclusive

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

ReplicatedPG: make sure that rollback target is not degraded

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

ReplicatedPG,OSD: clone srces need to be non-degraded

do_op: check that clone_range src sobjects are present on
replicas as well.  Also, clean up src_obc map if we encounter an error
before returning.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

libceph: build without globals

libceph, libosdc, libclient: depend on libcommon rather than libglobal.
No more globals in libceph.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

libceph: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

filestore: remove warning

Filer, Journaler: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ObjectCacher.cc: de-globalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

rgw: restructure permissions verification

apparently 100-continue was broken

filestore: don't bother doing fiemap on small ranges

Merge remote branch 'origin/osd_metrics'

update .gitignore

client/Client.cc: deglobalize

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

qa: wget -q for kenrel_untar_build.sh

Signed-off-by: Sage Weil <sage@newdream.net>

librgw: eliminate globals

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

update .gitignore

librados, librbd, librgw: remove dep on libglobal

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: reorder, cleanup prologger metrics

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: rip out old peer stat sharing

This never really worked, and has bitrotted, and the stats it's calculating
are mostly redundant anyway.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: rename peer_stat_lock -> stat_lock

It protects osd_stat too.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: instrument osdmap updates

Messages, epochs, dup epochs.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: clean up numpg*, hb*

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: instrument readable latency too

Time before a write is readable (not necessarily on disk).  Note that if we
get the commit first (e.g. writeahead journal) this value isn't calculated
or logged.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: remove lame rlsum/num

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: fix loadavg check

Use the syscall, stupid.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: instrument pushes, pull requests

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: add sub_op instrumentation

pull subop
push subop

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: fix up log_op_stats to actually work

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: rename push applied, committed completions

Be consistent.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: clean up client op stats

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: fix compilation

Revert "osd: make it compile on older boost libraries"

This reverts commit 116241ef106e6a88a84482f7f0f21219a6d29e33.

test_ceph_argparse: fix build

When this was merged, libcommon.a was not changed to libcommon.la

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

push global_init_set_globals into preinit callers

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

osd: make it compile on older boost libraries

Merge branch 'next'

Conflicts:
src/librbd.cc

Merge remote branch 'origin/stable' into next

Conflicts:
src/librbd.cc

create test_librados_build

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

librados: fix end_block calculation for aio_read

Conflicts:

src/librbd.cc

librados: fix end_block calculation for aio_read

Conflicts:

src/librbd.cc

librbd: track block_ofs instead of ofs in aio_sparse_read callback

Add ceph_argparse unit test

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

ceph_argparse_witharg: fix dashes in args

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>

common/Throttle: Remove unused return type on Throttle::get()

Signed-off-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Sage Weil <sage@newdream.net>