Josh Durgin [Thu, 30 Aug 2012 00:30:17 +0000 (17:30 -0700)]
doc: clarify rbd man page (esp. layering)
* a clone's size can't be overridden
* note which commands require format 2
* clarify details of copy
* add examples for cloning
* add pool to map example for consistency
* fix a couple warnings and re-sync man page with rst
Tommi Virtanen [Thu, 30 Aug 2012 14:16:52 +0000 (10:16 -0400)]
upstart, ceph-create-keys: Make client.admin key generation automatic.
This should help simplify Chef etc deployments. Now (when using the
Upstart jobs), when a ceph-mon is started, ceph-create-admin-key is
triggered. If /etc/ceph/$cluster.client.admin.keyring already exists,
it does nothing; otherwise, it waits for ceph-mon to reach quorum, and
then does a "ceph auth get-or-create" to create the key, and writes it
atomically to disk.
The equivalent code can be removed from the Chef cookbook once this is
in.
Tommi Virtanen [Thu, 30 Aug 2012 14:21:29 +0000 (10:21 -0400)]
config: Add a per-name default keyring to front of keyring search path.
This lets us have e.g. /etc/ceph/ceph.client.admin.keyring that is
owned by root:admin and mode u=rw,g=r,o= without making every non-root
run of the command line tools complain and fail.
This is what the Chef cookbook has been doing for a while already.
Josh Durgin [Wed, 29 Aug 2012 00:24:47 +0000 (17:24 -0700)]
librbd: prevent racing clone and snap unprotect
If the following sequence of events occured,
a clone could be created of an unprotected snapshot:
1. A: begin clone - check that snap foo is protected
2. B: rbd unprotect snap foo
3. B: check that all pools have no clones of foo
4. B: unprotect snap foo
5. A: finish creating clone of foo, add it as a child
To stop this from happening, check at the beginning and end of
cloning that the parent snapshot is protected. If it is not,
or checking protection status fails (possibly because the parent
snapshot was removed), remove the clone and return an error.
Yehuda Sadeh [Tue, 28 Aug 2012 23:17:21 +0000 (16:17 -0700)]
rgw: clear usage map before reading usage
Fixes: #3057
Since we read usage in chunks we need to clear the
usage map before reading the next chunk, otherwise
we're going to aggregate the old data as well.
Sage Weil [Tue, 28 Aug 2012 22:14:41 +0000 (15:14 -0700)]
osd: fix waiting_for_disk assertion
If requeue is false, we won't have cleared out waiting_for_ondisk; adjust
assert placement as appropriate. Also, make sur we handle the requeue
and !op case properly (although I'm not sure offhand if/when it would
come up).
Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com>
Mike Ryan [Tue, 28 Aug 2012 18:57:03 +0000 (11:57 -0700)]
rados_bench: wait for completion callbacks before returning
If we don't wait for the callback, the finisher may cleanup the callback
context before the callback is actually invoked, causing a
use-after-free error.
Mike Ryan [Tue, 28 Aug 2012 18:57:03 +0000 (11:57 -0700)]
rados_bench: wait for completion callbacks before returning
If we don't wait for the callback, the finisher may cleanup the callback
context before the callback is actually invoked, causing a
use-after-free error.
Sage Weil [Mon, 27 Aug 2012 21:31:32 +0000 (14:31 -0700)]
osd: requeue dup ops inline with in-progress ops
We should requeue the dups along with the originals. This avoids
situations where, after requeue, the dups are reordered with respect to
each other. For example:
- client sends A, B, C
- osd receives A
- connection drops
- client sends A', B', C'
- osd puts A' in waiting_for_ondisk, starts B' and C'
- on_change() requeues everything
Final queue order (before this patch) is
A, B', C', A'
After this patch, the resulting queue order is
A, A', B', C'
Or somewhat more generally, it might be:
A, A', B, B', B'', C', C'', D'', ....
Fixes (another source of): #2947 Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com>
Yehuda Sadeh [Tue, 21 Aug 2012 22:05:38 +0000 (15:05 -0700)]
rgw: implement garbage collector
Add a garbage collector thread that is responsible for clean
up of clutter. When removing an object, store info about the
leftovers in a special gc map (via rgw objclass). A new
radosgw-admin commands to list objects in gc, and to run the
gc process manually. Also, gc processors can run in parallel,
however, each will handle a single gc shard (synchronized
using lock objclass).
Sage Weil [Fri, 24 Aug 2012 21:43:56 +0000 (14:43 -0700)]
mon: describe how pgs are stuck in 'health detail'
Showing the current state and saying it is stuck doesn't tell you how it
is stuck (e.g. stuck unclean, stuck inactive, etc.). Also include the
stuck duration.
Fixes: #2876 Signed-off-by: Sage Weil <sage@inktank.com>
Mike Ryan [Tue, 24 Jul 2012 03:45:31 +0000 (20:45 -0700)]
obj_bencher: remove all benchmark files matching a prefix
This is a fallback for when a user wishes to delete ALL benchmark files
matching a particular prefix. In the fast case, a metadata file tells us
enough to quickly delete the files in parallel. This is the slow case,
where each file's name must be checked against the prefix.
Sage Weil [Thu, 23 Aug 2012 20:26:32 +0000 (13:26 -0700)]
msg/Pipe: conditionally detect session reset
Lossless peers (osd<->osd, mds<->mds, mon<->mon) never reset sessions
to each other. In the osd and mds cases, there is no need to check for
session resets. More significantly, these checks can trigger with an
unfortunately sequence of socket failures. In particular,
- A sends connect request to B
- B accepts, increments connect_seq, then has a socket failure
before telling A
- A reconnects, stil with connect_seq == 0
- B sees connect_seq == 0 and thinks there was a reset
This warrants a closer look in the fs client <-> mds case, but for now,
in the cluster-internal communications, it is moot, since reset
detection is unnecessary.
In the monitor case: we do need to check with resets because the peers
reuse the same entity_addr_t's (nonce==0), which means that a daemon
restart is effectively a reset. In that case, use a different policy
that continues to check for resets.
Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com>
Sage Weil [Thu, 23 Aug 2012 20:27:26 +0000 (13:27 -0700)]
osd: prefer acting osds in calc_acting()
We currently prefer up osds, and then pull sequentially from peer_info
(strays we know about at the time). This adds an additional preference
for the current acting, which means we can avoid changes to acting when
they are largely useless.
In particular, I observed that we chose [5,3] and later (when recovery
completed) chose [5,1] because we had since heard about an eligible stray
on 1. That switch was basically a waste...
Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com>
Dan Mick [Mon, 20 Aug 2012 22:02:57 +0000 (15:02 -0700)]
rbd: force all exiting paths through main()/return
This properly destroys objects. In the process, remove usage_exit();
also kill error-handling in set_conf_param (never relevant for rbd.cc,
and if you call it with both pointers NULL, well...)
Also switch to EXIT_FAILURE for consistency.
Signed-off-by: Dan Mick <dan.mick@inktank.com> Reviewed-by: Josh Durgin <josh.durgin@inktank.com> Fixes: #2948
Sage Weil [Wed, 22 Aug 2012 04:12:33 +0000 (21:12 -0700)]
objecter: use ordered map<> for tracking tids to preserve order on resend
We are using a hash_map<> to map tids to Op*'s. In handle_osd_map(),
we will recalc_op_target() on each Op in a random (hash) order. These
will get put in a temp map<tid,Op*> to ensure they are resent in the
correct order, but their order on the session->ops list will be random.
Then later, if we reset an OSD connection, we will resend everything for
that session in ops order, which is be incorrect.
Fix this by explicitly reordering the requests to resend in
kick_requests(), much like we do in handle_osd_map(). This lets us
continue to use a hash_map<>, which is faster for reasonable numbers of
requests. A simpler but slower fix would be to just use map<> instead.
This is one of many bugs contributing to #2947.
Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com>
projects / ceph.git / log