Sage Weil [Sat, 8 Feb 2014 02:05:04 +0000 (18:05 -0800)]
osd/ReplicatedPG: do not flush omap objects to an EC base pool
The EC pool does not support omap content. If the caching/tiering agent
encounters such an object, just skip it. Use the OMAP object_info_t flag
for this.
Although legacy pools will have objects with omap that do not have this
flag set, no *cache* pools yet exist, so we do not need to worry about the
agent running across legacy content.
Sage Weil [Sat, 8 Feb 2014 02:03:19 +0000 (18:03 -0800)]
osd: add OMAP flag to object_info_t
Set a flag if we ever set or update OMAP content on an object. This gives
us an easy indicator for the cache agent (without actually querying the
ObjectStore) so that we can avoid trying to flush omap to EC pools.
Sage Weil [Mon, 3 Feb 2014 01:11:23 +0000 (17:11 -0800)]
osd/ReplicatedPG: do not choke on op-less flush OpContexts (from flush)
The agent initiates flush ops that don't have an OpRequest associated
with them. Make reply_ctx skip the actual reply message instead of
crashing if the flush request gets canceled (e.g., due to a race with
a write).
Sage Weil [Tue, 28 Jan 2014 01:57:53 +0000 (17:57 -0800)]
osd/ReplicatedPG: add slop to agent mode selection
We want to avoid a situation where the agent clicks on and off when the
system hovers around a utilization threshold. Particularly for trim,
the system can expend a lot of energy doing a minimal amount of work when
the effort level is low. To avoid this, enable when we are some amount
above the threshold, and do not turn off until we are the same amount below
the target.
Sage Weil [Tue, 28 Jan 2014 00:26:19 +0000 (16:26 -0800)]
osd/ReplicatedPG: initial agent to random hash position inside pg
When the agent starts, start at a random offset to ensure we get a more
uniform distribution of attention to all objects in the PG. Otherwise, we
will disproportionately examine objects at the "beginning" of the PG if we
are interrupted by peering or restarts or some other activity.
Note that if the agent_state is preserved, we do not forget our position,
which is also nice.
We *could* persist this position in the pg_info_t somewhere, but I am not
sure it is worth the effort.
Sage Weil [Fri, 24 Jan 2014 22:35:41 +0000 (14:35 -0800)]
osd/ReplicatedPG: basic flush and evict agent functionality
This is very basic flush and evict functionality for the tiering agent.
The flush policy is very simple: if we are above the threshold and the
object is dirty, and not super young, flush it. This is not too braindead
of a policy (although we could clearly do something smarter).
The evict policy is pretty simple: evict the object if it is clean and
we are over our full threshold. If we are in the middle mode, try to
estimate how cold the object is based on an accumulated histogram of
objects we have examined so far, and decide to evict based on our
position in that histogram relative to our "effort" level.
Caveats:
* the histograms are not refreshed
* we aren't taking temperature into consideration yet, although some of
the infrastructure is there.
Sage Weil [Tue, 4 Feb 2014 06:10:07 +0000 (22:10 -0800)]
osd/ReplicatedPG: add on_finish to OpContext
Add a callback hook for whenever an OpContext completes or cancels. We
are pretty sloppy here about the return values because our initial user
will not care, and it is unclear if future users will.
Sage Weil [Mon, 20 Jan 2014 18:28:43 +0000 (10:28 -0800)]
osd: add pg_pool_t::get_pg_num_divisor
A PG is not always an equally sized fraction of the total pool size due to
the use of ceph_stable_mod. Add a helper to return the fraction
(denominator) of a given pg based on the current pg_num value.
By default, disallow adjustment of primary affinity unless the user has
opted in by adjusting their monitor config. This will avoid some user
pain because inadvertantly setting the affinity will prevent older clients
from connecting to and using the cluster.
Sage Weil [Tue, 11 Feb 2014 17:25:04 +0000 (09:25 -0800)]
osd/OSDMap: apply primary_affinity to mapping
The behavior is a bit different for replicated and indep/erasure mode.
In the first case, we are rearranging the result. In the second case,
we can just set the primary argument to the right value.
Ilya Dryomov [Thu, 13 Feb 2014 17:38:10 +0000 (19:38 +0200)]
osdmaptool: fix --pool option for --test-map-object mode
Commit 7f1b12f2ef97 ("osdmaptool: add --test-map-pgs mode") broke
--pool for --test-map-object mode. Fix it, and improve --pool option
handling for both modes while at it (report strict_strtol() errors,
check if specified pool exists).
Loic Dachary [Thu, 13 Feb 2014 14:40:30 +0000 (15:40 +0100)]
mon: do not goto reply if a ruleset exists in pending
If the crush ruleset is found in pending, do not goto reply because it
does not exist yet. Wait for the pending proposal (and the ruleset) to
be accepted and then only return that it exists.
Basic tests and a test that create the conditions where an OSDMap
is pending with a ruleset that is not yet in the OSDMap. An attempt to
create a rule by the same name will return success and not create it again.
Loic Dachary [Sun, 9 Feb 2014 17:26:57 +0000 (18:26 +0100)]
mon: minor improvement to test-helpers.sh
* kill -9 daemons to better cope with stuck daemons
* reduce paxos propose interval to speedup tests
* look for erasure code plugins from the sources directory
Loic Dachary [Sun, 9 Feb 2014 20:18:45 +0000 (21:18 +0100)]
crush: add the dump_rule(int ruleset) method
And implement dump_rules() using dump_rule(). The indentiation and
variable names are intentionaly left as is to not confuse code being
moved around and the code changes.
Loic Dachary [Wed, 12 Feb 2014 15:58:23 +0000 (16:58 +0100)]
mon: remove racy tests from osd-pool-create.sh
Use the admin socket to create the conditions by which a pool creation
is made to wait for the next paxos proposal because the required crush
ruleset is pending.
It replaces a fragile time sensitive workaround that could fail because
of race conditions. It also has the benefit of increase the speed of the
test because there is no need to wait for a long time just to accomodate
the slowest machines.
and feed it to OSDMonitor::prepare_command_impl(). The pending OSDMap won't
be proposed because it short circuit PaxosService::dispatch. It will,
however, be proposed next time PaxosService::dispatch() gets a chance.
Loic Dachary [Wed, 12 Feb 2014 17:49:50 +0000 (18:49 +0100)]
mon: add --mon-advanced-debug-mode
When this flag is true, the mon is expected to provide functionalities
that are for developer oriented debug purposes only. It is meant to
be used by the developer and not the system administrator, because it
would allow a non-developer to break things in ways that would be very
difficult to diagnose. It should probably not be documented.
Yan, Zheng [Tue, 11 Feb 2014 05:32:23 +0000 (13:32 +0800)]
mds: remove xattr when null value is given to setxattr()
setxattr() should remove xattr if parameter 'value' is NULL. To
distinguish it from the zero-length value case, introduce a new
flag CEPH_XATTR_REMOVE for the setxattr request.
Sage Weil [Tue, 11 Feb 2014 17:24:29 +0000 (09:24 -0800)]
osd/OSDMap: fix _raw_to_up_osds for EC pools
When filtering down OSDs out of the result set (i.e., raw -> up), we can
shift positions for replicated pools but need to preserve position for
EC pools.
Sage Weil [Wed, 12 Feb 2014 21:18:04 +0000 (13:18 -0800)]
mon/OSDMonitor: fix legacy tunables warning
Warn on legacy tunables, not on non-optimal tunables. Optimal is a moving
target, but it is really the legacy defaults that we want to push people
off of.
Fixes: #7399 Signed-off-by: Sage Weil <sage@inktank.com>
projects / ceph.git / log