Erwan Velu [Wed, 13 Jun 2018 09:58:47 +0000 (11:58 +0200)]
qa/standalone/ceph-helpers.sh: Setup ulimit in setup()
If ulimit is set to a 1024 value, ceph-osd will segfault with the
following error :
filestore(td/smoke/0) error (24) Too many open files not handled on operation 0x55565d1fd004 (2182.1.0, or op 0, counting from 0)
This patch is about to insure that before setting up ceph daemons in tests, a valid ulimit value is setup.
Erwan Velu [Thu, 7 Jun 2018 13:56:45 +0000 (15:56 +0200)]
qa/standalone/ceph-helpers.sh: Thinner resolution in get_timeout_delays()
get_timeout_delays() is a generic function to compute delays for a long
period of time without saturating the CPU is busy loops.
It works pretty fine when the delay is short like having the following
series when requesting a 20seconds timeout : "0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ".
Here the maximum between two loops is 7.3 which is perfectly fine.
When the timeout reaches 300sec, the same code produces the following
series : "0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 "
In such example there is delays which are nearly 2 minutes !
That is not efficient as the expected event, between two loops, could
arrive just after this long sleep occurs making a minute+ sleep for
nothing. On a local system that could be ok while on a CI, if all jobs
run like CI the overall is pretty unefficient by generating useless CPU
waits.
This patch is about adding a maximum acceptable delay time between two
loops while keeping the same rampup behavior.
On the same 300 seconds delay example, with MAX_TIMEOUT set to 10, we
now have the following series: "0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3"
We can see that the long 12/25/51/102/95 values vanished and being
replaced by a series of 10 seconds. It's up to every test defining the
probability of having a soonish event to complete.
The MAX_TIMEOUT is set to 15seconds. Signed-off-by: Erwan Velu <erwan@redhat.com>
Sage Weil [Sat, 23 Jun 2018 22:05:51 +0000 (17:05 -0500)]
Merge PR #22479 into master
* refs/pull/22479/head:
mgr/devicehealth: add self-test
mgr/devicehealth: make config options instance attributes
mgr/devicehealth: active -> enable_monitoring
mgr/mgr_module: update docstring for get()
mgr/devicehealth: fix 'device get-health-metrics' when no metrics are stored
mgr/devicehealth: add some scraping infrastructure
osd: optionally query smart for a single devid
mgr/ActivePyModules: add get for 'device <devid>'
osd: key smart results by unique device id
mgr: enable devicehealth by default
mgr/devicehealth: rename old smart module to devicehealth
Sage Weil [Fri, 8 Jun 2018 19:59:34 +0000 (14:59 -0500)]
mgr/devicehealth: add some scraping infrastructure
- command to fetch smart info
- command to scrape a device and store the metrics
- command to scrape all devices (and store)
- command to dump stored metrics
- purging of old metrics
This is based on code originally written by Yaarit.
Signed-off-by: Yaarit Hatuka yaarithatuka@gmail.com Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 22 Jun 2018 18:00:35 +0000 (13:00 -0500)]
Merge PR #22586 into master
* refs/pull/22586/head:
global/signal_handler: one less frame of context
global/signal_handler: write crash dumps to /var/lib/ceph/crash/$uuid/
common/ceph_context: add "assert" and "abort" asok commands
common/assert: record assert info in g_assert_* globals
common: add crash_dir option
common/assert: get rid of duplicate log dump from assert handler
common/BackTrace: add dump()
log: do not discard recent after dumping it
debian,rpm: /var/lib/ceph/crash
Yunchuan Wen [Fri, 15 Jun 2018 05:59:30 +0000 (13:59 +0800)]
osd: update clone_overlap even the clone have been evicted
the clone_overlap is difference of range between head and clones.
if the clone have been evicted, the clone_overlap is still exist in the
snapset, so we should update the clone_overlap to make it sense.
Neha Ojha [Wed, 20 Jun 2018 17:20:58 +0000 (13:20 -0400)]
osd/PG: restrict async_recovery_targets to up osds
When an osd that is part of the acting set and not the up set, gets chosen
as an async_recovery_target, it gets removed from the acting set. Since this
osd is no longer in the up or acting set, it is classified as a stray in
the next peering cycle. This results in choose_acting() looping between two
proposed acting sets.
To avoid this, we will only choose up osds as async_recovery_targets.
Lenz Grimmer [Thu, 21 Jun 2018 13:34:01 +0000 (15:34 +0200)]
Merge pull request #22470 from votdev/feature_24436
mgr/dashboard: Replace RGW proxy controller
Reviewed-by: Patrick Nawracay <pnawracay@suse.com> Reviewed-by: Ricardo Dias <rdias@suse.com> Reviewed-by: Sebastian Wagner <swagner@suse.com> Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Volker Theile [Tue, 19 Jun 2018 11:19:10 +0000 (13:19 +0200)]
mgr/dashboard: Replace RGW proxy controller
Fixes: http://tracker.ceph.com/issues/24436
To fully support the role based authentication/authorization system it is necessary to replace the RGW proxy controller by separate controllers for RGW user and bucket.
The new civetweb 1.10 version in mimic and later is strict on control characters
being url encoded, making url validation more relaxed and passing these through
to rgw where the requisite url validation is done.
Jianpeng Ma [Wed, 20 Jun 2018 12:22:38 +0000 (20:22 +0800)]
os/bluestore: fix length overflow.
In fact, length of 'struct interval_t' and 'struct bluestore_pextent_t'
is uint32_t. But len of AllocatorLevel02::_mark_allocated is uint64_t.
So it may cause data overflow which cause bug.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
Qiaowei Ren [Tue, 19 Jun 2018 03:12:21 +0000 (11:12 +0800)]
common: fix enum redeclaration
This patch fixes the error about enum redeclaration during compile time.
The error is from the enum declaration in src/common/perf_counters.h:
enum unit_t : uint8_t
{
BYTES,
NONE
};
When external library also includes similar declaratiron, the following
error will appear:
error: redeclaration of enumerator 'NONE'
error: previous definition of 'NONE' ...
xie xingguo [Wed, 20 Jun 2018 01:04:19 +0000 (09:04 +0800)]
osd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone
If an osd is gone or moved out from the specific crush rule,
we should cancel any pg_upmap/pg_upmap_items still bound to
that osd too.
The original code does not work for the above case because
get_parent_of_type() will fail if that osd does not belong
to the crush_rule passed in and hence hits the assert below:
Sage Weil [Tue, 19 Jun 2018 18:51:09 +0000 (13:51 -0500)]
Merge PR #22596 into master
* refs/pull/22596/head:
os/bluestore: use vector instead of set for zombies
os/bluestore: reuse zombie OpSequencers by collection id
qa/suites/rados/objecstore/backends/objectstore: capture coredumps
os/bluestore: more debug output
os/bluestore: print cnode from _open_collections
os/bluestore: print cnode on fsck
qa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore
Sage Weil [Mon, 18 Jun 2018 12:32:08 +0000 (07:32 -0500)]
os/bluestore: reuse zombie OpSequencers by collection id
We can get a sequence that deletes and then recreates a collection where
the transaction removing the collection is delayed (due to pending IO on
its sequencer) but colleciton create is not (new sequencer).
Avoid any such reordering by recycling the old collection's sequencer if
the zombie_osr has not been reaped yet.
Fixes: http://tracker.ceph.com/issues/24550 Signed-off-by: Sage Weil <sage@redhat.com>
projects / ceph.git / log