Sage Weil [Wed, 23 Oct 2013 01:06:41 +0000 (18:06 -0700)]
osd/ReplicatedPG: handle is_whiteout in do_osd_ops()
Most of the time we handle whiteouts by returning ENOENT before we even
get this far. However, for a mixed read/write transaction (e.g., a guard)
or certain ops (like create exclusive) we need to deal with the
exists == true and whiteout flag set case explicitly.
Greg Farnum [Thu, 10 Oct 2013 00:48:57 +0000 (17:48 -0700)]
ReplicatedPG: promote: first draft pass at doing object promotion
This is not yet at all complete -- among other things, it will
retry forever on any object which doesn't exist in the underlying
pool. But it demonstrates the approach reasonably clearly.
Signed-off-by: Greg Farnum <greg@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com>y
Greg Farnum [Wed, 9 Oct 2013 23:16:36 +0000 (16:16 -0700)]
ReplicatedPG: copy: switch out the CopyCallback interface
The tuple was already unwieldy with 4 members; I didn't want to add
more. Instead, create a new CopyResults struct which contains all the
object info and completion data, and pass the retval and a CopyResults*
in the CopyCallbackResults tuple.
Signed-off-by: Greg Farnum <greg@inktank.com> Reviewed-by: Sage Weil <sage@inktank.com>
Sage Weil [Wed, 4 Dec 2013 17:11:02 +0000 (09:11 -0800)]
osd/ReplicatedPG: trim old hit_set objects on persist
Any time we persist a hit_set object, take the opportunity to remove any
old ones that we don't want any more.
Note that this means if the admin decreases the number of objects to track,
we won't remove them until the next time we persist something. We also
don't clean up if the HitSet tracking is disabled entirely.
Sage Weil [Sun, 6 Oct 2013 18:39:58 +0000 (11:39 -0700)]
osd/ReplicatedPG: use vectorized osd_op outdata for pg ops
This lets us put PGLS in a compound operation. Nothing does that yet, but
this would allow it.
Despite appearances, this is not a protocol change and does not require
a feature bit for clients: using the osd_ops vector mechanisms store all
the data in the same places as before, it just fills in some of the
already-decoded-but-empty data structures in the MOSDOpReply header.
<Greg note:> We may need a feature bit to let clients know they can send
compound PG ops to OSDs, though? Or maybe we can let it be covered
by supporting hitset ops.
Sage Weil [Thu, 3 Oct 2013 23:40:15 +0000 (16:40 -0700)]
osd/osd_types: include pg_hit_set_history_t in pg_info_t
Track metadata about the currently accumulating HitSet as well as
previously archived ones in the pg_info_t. This will not scale well for
extremely long histories, but does let us avoid explicitly sharing this
metadata during recovery or other normal update activity.
Sage Weil [Fri, 4 Oct 2013 23:07:20 +0000 (16:07 -0700)]
osd/ReplicatedPG: factor out simple_repop_{create,submit} helpers
This makes it easier to create repops correctly, and should help
prevent bugs like the one we remove here in process_copy_op (we were
serializing on the wrong object!)
Signed-off-by: Sage Weil <sage@inktank.com> Signed-off-by: Greg Farnum <greg@inktank.com>
Sage Weil [Thu, 10 Oct 2013 16:56:39 +0000 (09:56 -0700)]
osdc/Objecter: reimplement list_objects
Return to caller at the end of each PG. This allows the caller to look at
the [pg_]hash_position and get something meaningful.
If there are no objects in the PG, we skip it so that every callback has
*some* data (unless the pool is totally empty!). So the real difference
here is that we don't move on to the next PG just to reach max_entries.
This gives the client some data sooner, but may mean more callbacks into
client code.
Sage Weil [Sun, 6 Oct 2013 20:22:31 +0000 (13:22 -0700)]
osdc/Objecter: separate explicit pg target from current target
The pgid field is used to store the pg the op mapped to. We were just
setting it directly for PGLS. Instead, fill in a new base_pgid, and copy that
to pgid in recalc_op_target(), the same way we do when we map an object
name to a PG.
In particular, we take this opportunity to map a raw pgid to an actual
pgid. This means the base_pg could come from a raw hash value (although
it doesn't, yet).
Signed-off-by: Sage Weil <sage@inktank.com> Signed-off-by: Greg Farnum <greg@inktank.com>
Samuel Just [Wed, 6 Nov 2013 22:33:03 +0000 (14:33 -0800)]
ReplicatedPG: don't skip missing if sentries is empty on pgls
Formerly, if sentries is empty, we skip missing. In general,
we need to continue adding items from missing until we get
to next (returned from collection_list_partial) to avoid
missing any objects.
Fixes: #6633 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: David Zafman <david.zafman@inktank.com>
Samuel Just [Tue, 5 Nov 2013 23:40:29 +0000 (15:40 -0800)]
RadosModel: use sharedptr_registry for snaps_in_use
There might be two concurrent rollback ops each of which
adds snap x to snaps_in_use. Between when the first
completes and the second completes, snap x may be removed
since the first would have removed snap x from snaps_in_use.
Using sharedptr_registry here avoids this by ensuring that
the snap won't be removed from snaps_in_use until all refs
are gone.
This patch also adds size() to sharedptr_registry.
Fixes: #6719 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: David Zafman <david.zafman@inktank.com>
Samuel Just [Mon, 4 Nov 2013 19:25:31 +0000 (11:25 -0800)]
FileStore::_collection_move_rename: handle missing dst dir on replay
In case of a replay, a missing destination directory indicates that
the destination object and directory have been removed by a later
transaction. Thus, we need to remove the src object and return
0.
Fixes: #6714 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com>
Danny Al-Gaaf [Mon, 4 Nov 2013 22:30:47 +0000 (23:30 +0100)]
galois.c: fix compiler warning
galois_create_split_w8_tables() takes no parameter, remove '8' passed
to the function in one case.
osd/ErasureCodePluginJerasure/galois.c: In function 'galois_w32_region_multiply':
osd/ErasureCodePluginJerasure/galois.c:696:5: warning: call to function 'galois_create_split_w8_tables' without a real prototype [-Wunprototyped-calls]
In file included from osd/ErasureCodePluginJerasure/galois.c:53:0:
osd/ErasureCodePluginJerasure/galois.h:71:12: note: 'galois_create_split_w8_tables' was declared here
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Samuel Just [Mon, 4 Nov 2013 05:02:36 +0000 (21:02 -0800)]
OSD: allow project_pg_history to handle a missing map
If we get a peering message for an old map we don't have, we
can throwit out: the sending OSD will learn about the newer
maps and update itself accordingly, and we don't have the
information to know if the message is valid. This situation
can only happen if the sender was down for a long enough time
to create a map gap and its PGs have not yet advanced from
their boot-up maps to the current ones, so we can rely on it
Fixes: #6712 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com>
Samuel Just [Sun, 3 Nov 2013 19:06:10 +0000 (11:06 -0800)]
OSD: don't clear peering_wait_for_split in advance_map()
I really don't know why I added this... Ops can be discarded from the
waiting_for_pg queue if we aren't primary simply because there must have
been an exchange of peering events before subops will be sent within a
particular epoch. Thus, any events in the waiting_for_pg queue must be
client ops which should only be seen by the primary. Peering events, on
the other hand, should only be discarded if we are in a new interval,
and that check might as well be performed in the peering wq.
Fixes: #6681 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com>
Samuel Just [Sat, 2 Nov 2013 20:54:51 +0000 (13:54 -0700)]
ReplicatedPG::recover_backfill: adjust last_backfill to HEAD if snapdir
Otherwise, if last_backfill_started is a snapdir, we will fail to send a
transaction for a client IO creating the head object and removing the
snapdir object. The result will be that head will eventually be
backfilled, but the snapdir object will erroneously not be removed.
Fixes: #6685 Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com>
projects / ceph.git / log