test_async_snap: loop a finite number of times.

Signed-off-by: Sage Weil <sage@newdream.net>

qa: btrfs: test_async_snap

Hammer on the btrfs async sync/snap create/remove ioctls.

Signed-off-by: Sage Weil <sage@newdream.net>

Use LIBEDIT_LIBS in Makefile.am

The editline library might require some additional libs, so
instead of hardcoding '-ledit' we should be using LIBEDIT_LIBS
in Makefile.am

Signed-off-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Sage Weil <sage@newdream.net>

Merge branch 'wip-dotnames'

clientN -> client.N

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

monN -> mon.N

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

mdsN -> mds.N

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osdN -> osd.N

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

msgr: print entity_name_t with a .

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix leak of src_obc references

We never moved the src_obc map from the local var into the RepGather
member!

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

test-rados-api: test tmap rm

Make sure we get ENOENT if the entry doesn't exist.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: error out current op only on tmap ENOENT

If an individual op in do_osd_ops() fails, set result, and continue to the
end of the loop.  Normally we'll return the error then, but if the ignore
error flag is set we'll continue.

Fix the tmap error handling so that we don't modify the tmap if the op
failed.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: some PgPriorSet comments, trivial cleanups, new FIXME

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: skip replay phase if osd_replay_window=0

The replay phase is only useful if there are clients waiting for ACK (but
not COMMIT) and expect ordering to be preserved across OSD failures.
Currently nobody does that except the client in sync io mode (shared
writers).  For installations where we don't care (e.g., using rados only),
we can disable replay altogether.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

ReplicatedPG: remove assert in remove_watcher

notif is not in notif->session->notifs.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

ReplicatedPG: clean up watched objects on role change

Also removes inconsistent watch handling in handle_notify_timeout.
We will need to handle watch timeouts elsewhere.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>

ceph-authtool: fix -C

Fixes: #1578
Signed-off-by: Sage Weil <sage@newdream.net>

ceph.spec.in: radosgw init script

Signed-off-by: Sage Weil <sage@newdream.net>

mon: auth: implement 'auth print-key <name>'

Signed-off-by: Sage Weil <sage@newdream.net>

mon: fix error return codes for 'auth ...'

Signed-off-by: Sage Weil <sage@newdream.net>

ceph.spec.in: add rbd python bindings

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

debian: make python-ceph depend on librbd

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

pybind: fix errors found by pyflakes

Unused imports, typos, and unused variables.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

debian: move ceph and ceph-authtool into ceph-common

Signed-off-by: Sage Weil <sage@newdream.net>

mon: auth: generate random key on 'auth add ...' if no keyring is provided

Signed-off-by: Sage Weil <sage@newdream.net>

debian: ceph-client-tools -> ceph-common, now required, w/ ceph-conf etc.

Signed-off-by: Sage Weil <sage@newdream.net>

osdmap: remove dead code

Signed-off-by: Sage Weil <sage@newdream.net>

Merge branch 'stable'

radosgw: fix content-length checks

Ignore content-length check if the REQUEST_URI has a trailing ?acl.

(The object_str.find() check was wrong because object_str has the ?whatever
stripped off, and because it returns std::string::npos on failure.)

Signed-off-by: Sage Weil <sage@newdream.net>

radosgw: tolerate race creating .rgw* metadata pools

There may be multiple people trying to create these pools.  That's just
fine.

Signed-off-by: Sage Weil <sage@newdream.net>

librbd: fix end_block calculation for read_iterate

All end block calculations match now.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

test_rbd.py: Add large read and write tests

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librbd: don't zero negative lengths

They are past the end of the buffer (and wrap around).

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rados.py: be paranoid about strings

This prevents bad conversions (i.e. from int to char *).

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd.py: remove useless version object

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd.py: be paranoid about strings

This prevents bad conversions (i.e. from int to char *).

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd.py: explicitly cast strings to c_char_p for type clarity

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd.py: pass 0 for the default order, not NULL

librbd expects the order pointer to be valid - if it points to 0, the
default order is used.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rados.py: make exceptions inherit from a base class

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rados.py: allow configuring rados through constructor options

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd.py: make Image a context manager

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rados.py: make Rados and Ioctx objects context managers

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd.py: make exceptions inherit from a base class

This way you can catch them all.
Also remove the unused NoData exceptions
- this was just useful for xattrs in librados.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

Use cpp_strerror or strerror_r when appropriate.

The only remaining uses of strerror are in single-threaded programs
that don't depend on libcommon.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd.py: add partial bindings for librbd

This is all the basic operations, but nothing with callbacks yet.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rados.py: raise specific error when write is too long

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd: init counter to 0 so errors don't say '-1% done'

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

ReplicatedPG, tmap: return -ENOENT if the key to remove does not exist

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librbd: use pre-existing helper methods for tmap operations

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

librbd: add debugging for failing to remove a header object

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

rbd: improve error messages

One of these was accidentally not negated.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>

mon: fix it again

Signed-off-by: Sage Weil <sage@newdream.net>

mon: fix dup pool creation race

Fixes: #1574
Signed-off-by: Sage Weil <sage@newdream.net>

init-ceph: more c* -> ceph-* cleanup

Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix warning more

Argh

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
Signed-off-by: Sage Weil <sage@newdream.net>

radosgw_admin: 'bucket list' or 'buckets list'

The plural looks weird to me.

Signed-off-by: Sage Weil <sage@newdream.net>

Makefile: include init-radosgw in tarball

Signed-off-by: Sage Weil <sage@newdream.net>

radosgw: shut down threadpool

Don't crash when run as a non-daemon without apache.. e.g. 'radosgw' with
no args.

Fixes: #1572
Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix warning

Signed-off-by: Sage Weil <sage@newdream.net>

debian: include radosgw init script in package

Signed-off-by: Sage Weil <sage@newdream.net>

radosgw: init script

Signed-off-by: Sage Weil <sage@newdream.net>

Merge branch 'stable'

filejournal: improve debug error messge

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd/mon: make max maps per MOSDMap message tunable

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: limit size of osdmap messages

Limit size of MOSDMap messages sent from OSDs.  Send many smaller
messages instead of a single large one.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: drop map_cache_keep_from

The purpose here was to avoid trimming cached maps prior to what we have
on disk.  However, now that we have the map_bl cache, this isn't needed:
anything after that epoch will come out of that cache.

Also, it was broken anyway--the value was never read.  So clean it out!

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

osd: limit size of OSDMap cache

If we get way way behind on our maps, we may end up with a really large
OSDMap cache because we currently on trim old maps based on
oldest_last_clean, which may be way in the past.  Avoid eating up gobs of
RAM by putting a ceiling on the cache size.  It'll mean more disk IO in
those situations, but it also means that we'll only load up the old maps
that we actually need (not every single one).

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rgw: only require content-length to be set on non-acl puts

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

osd: don't finish boot unless instance in map is really us

We were going BOOTING->ACTIVE as soon as we showed up in the map with the
same client_addr.  Also verify that we were up_from an epoch after when
we started or rebound, to avoid the case where we rebind to the same
ports for client_addr (but maybe not others) and get caught in a rebind
loop.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

init-ceph: don't umount btrfs by default

As an admin I find this very annoying, even though it isn't strictly
symmetrical with the mount behavior.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

- Fix atomic set.

rgw: only require content-length to be set on non-acl puts

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Revert "rgw: tolerate missing content-length header on canned ACL puts"

This reverts commit 05b06e807ef20c7fe1bd7c7abbeecd656d2280c0.

Don't be stupid when an empty content-length can only be on ACL puts!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

Merge remote branch 'gh/stable'

man: c* -> ceph-* rename: Regenerate manpages.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

doc: Fix reStructuredText formatting warnings.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

Makefile: c* -> ceph-* rename: missed cdebugpack in CLEANFILES.

make distcheck said:

  ERROR: files left in build directory after distclean:
  ./src/ceph-debugpack
  make[1]: *** [distcleancheck] Error 1

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

Makefile: Add missing header test/unit.h

Used to fail "make distcheck".

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

.gitignore: distcheck temporary directory.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

Makefile: include run-cli-tests-maybe-unset-ccache in dist tarball.

Apparently "make distcheck" has been broken for 3 months;
I find that hard to believe, but that's what it looks like.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

ceph-run: c* -> ceph-* rename: Use ceph-run as stderr prefix.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

ceph-debugpack: c* -> ceph-* rename: use "ceph-debugpack" as temp dir name too.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

udev: c* -> ceph-* rename: missed crbdnamer.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

debian: c* -> ceph-* rename: missed csyn, crbdnamer.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

rpm: c* -> ceph-* rename: missed crbdnamer, cdebugpack.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

Makefile: c* -> ceph-* rename: missed crun.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

doc: v0.35 is out, remove rpm build workaround.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

doc: Add index into the sidebar.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

doc: Search page is always in sidebar, no need to have it on page.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

doc: Add a space after sudoers "NOPASSWD:", as that seems more common.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

doc: Add index entries, remove glossary as it's practically replaced by the index.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

doc: Leftovers of the c* -> ceph-* rename.

Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>

rgw: tolerate missing content-length header on canned ACL puts

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>

mon: remember source client address in routed requests

when we resend_routed_requests, the source client address is lost.
This may cause problems. For example, if we resend an mds beacon
(boot) to a new monitor leader, the new mdsmap will contain a new
mds entry without IP address.

To reproduce this bug:
1. deploy a cluster with 3 mons.
2. let active mds send beacon to mon0; standby mds send becaon to mon2
3. gdb attach to mon2 to make it unresponsive and make standby mds laggy.
4. gdb attach to mon0 to make it unresponsive and make active mds laggy.
5. detach mon2, then the standby mds will become active.
6. ceph mds dump -o - shows the active mds address is :/0

Signed-off-by: Henry C Chang <henry_c_chang@tcloudcomputing.com>
Signed-off-by: Sage Weil <sage@newdream.net>

osd: fix race between handle_osd_ping and handle_osd_map

If handle_osd_map is in progress and handle_osd_ping doesn't have the
map lock, we can't call osdmap->get_inst() in send_still_alive().  Keep
the entity_inst_t in the failure_pending map so that we don't need to.

osd/OSDMap.h: In function 'entity_inst_t OSDMap::get_inst(int)', in thread '0x7fda6a46b710'
osd/OSDMap.h: 477: FAILED assert(is_up(osd))
 ceph version 0.24.1 (commit:e06fb657842379259826f3d9215101fc14575fbd)
 1: (OSD::send_still_alive(int)+0x1b9) [0x4dd1b9]
 2: (OSD::handle_osd_ping(MOSDPing*)+0x716) [0x4f6446]
 3: (OSD::heartbeat_dispatch(Message*)+0x36) [0x4f6666]
 4: (SimpleMessenger::dispatch_entry()+0x882) [0x46f002]
 5: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x465fac]
 6: (()+0x6a3a) [0x7fda7835aa3a]
 7: (clone()+0x6d) [0x7fda76f7777d]

Reported-by: Henry C Chang <henry_c_chang@tcloudcomputing.com>
Signed-off-by: Sage Weil <sage@newdream.net>

Makefile: drop useless all_sources thing

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

rename source files c* -> ceph-*

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

misc: use strncpy to prevent buffer overflow

ceph-elaster: add parentheses around macros

test_store: parse args, don't link librados

Signed-off-by: Sage Weil <sage@newdream.net>