Zack Cerza [Tue, 23 Oct 2018 22:43:51 +0000 (22:43 +0000)]
mgr/dashboard: Enable gzip compression
This is related to http://tracker.ceph.com/issues/36453. It is far from
a complete solution, but seems like a positive move.
I tested this change by first disabling my browser cache, and then used
the /docs endpoint to query /api/dashboard/health. Before compression:
Content-Length: 60748
Time: 615ms
After:
Content-Length: 7505
Time: 92ms
Then, I logged into the dashboard as normal and reloaded the page once I
was in. Some values for the reload operation before compression:
Total page load time: 58.48s
vendor.js Content-Length: 6486025
vendor.js time: 48.09s
After:
Total page load time: 14.55s
vendor.js Content-Length: 1143178
vendor.js time: 4.50s
Kefu Chai [Mon, 22 Oct 2018 08:41:45 +0000 (16:41 +0800)]
include/ceph_assert.h: do not pack assert params if WITH_ASAN
we pack the asset() params for smaller code size, but this creates a
inlined `assert_data_ctx` instance for every compilation unit which
call ceph_assert() defined in .h .
__PRETTY_FUNCTION__ is likely to be referenced by `assert_data_ctx`
sections which are included by different compiled object files. if the
ceph_assert() call is used by header file, then there will be multiple
`assert_data_ctx` sections sharing the same identifier. these sections are
defined as "COMDAT" group sections, i.e. common data sections. when linker
see multiple COMDAT sections with the same identifer, it will simply discard
the duplicated ones, and only keep a single copy of them. without enabling
ASan, GCC can always handle this problem just fine. but the dedup feature
does not work well with ASan. if ASan is enabled, and we link the objects
with the wrong order, some references will be pointing to the discarded
sections.
to address this issue, we could audit the link command line and inspect
all .o files to make sure they are properly ordered. but this is
non-trivial. as a workaround, in this change, the assert params are not
packed, and sent to the __ceph_assert_fail() overrides which accepts
unpacked params directly, so the COMDAT section is not created.
Jason Dillaman [Mon, 22 Oct 2018 14:44:40 +0000 (10:44 -0400)]
qa/tasks/qemu: use unique clone directory to avoid race with workunit
If there is a workunit task associated with the same client, the two
tasks will attempt to clone the suite repo to the same directory.
Worse, if it's parallel tasks, the two clones will clobber each
other.
Fixes: http://tracker.ceph.com/issues/36542 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Kefu Chai [Mon, 22 Oct 2018 06:06:38 +0000 (14:06 +0800)]
cmake,ceph.in: preload libasan if WITH_ASAN
we need to preload libasan.so as the python exectuable is not likely to
be compiled with ASan enabled.
see:
https://github.com/google/sanitizers/wiki/AddressSanitizerAsDso#asan-and-ld_preload
just to ease the use of ASan, for fine-tuned behaviour, use
`ASAN_OPTIONS`.
Kefu Chai [Mon, 22 Oct 2018 05:13:11 +0000 (13:13 +0800)]
cmake: should compile libzstd with -fPIC
otherwise we will have
/usr/bin/ld: libzstd/lib/libzstd.a(error_private.c.o): relocation
R_X86_64_32S against `.rodata' can not be used when making a shared
object; recompile with -fPIC
Kefu Chai [Mon, 22 Oct 2018 04:28:01 +0000 (12:28 +0800)]
cmake: pass cflags to disutils using CC instead of CFLAGS
in python's distutils.ccompiler, linker_exe is composed using CC instead
of LDFLAGS. the latter only effects how it builds (shared) library.
and put CMAKE_C_FLAGS into the cflags for the compiler for building
python C extensions, it's more consistent this way. more importantly,
if we build with ASan enabled, the canary program, a.k.a. rados_dummy.c,
won't link without proper CFLAGS.
without this change, rados.so fails to build with errors like:
/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_stack_free_10'
/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_report_exp_store8'
...
...
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
Link Error: RADOS library not found
make[3]: ***
[src/pybind/rados/CMakeFiles/cython_rados.dir/build.make:57:
src/pybind/rados/CMakeFiles/cython_rados] Error 1
Sage Weil [Sat, 20 Oct 2018 21:40:22 +0000 (16:40 -0500)]
Merge PR #24184 into master
* refs/pull/24184/head:
mgr/DaemonServer: remove any upmaps on merging PGs
mgr/DaemonServer: prevent merge if either pg is remapped|upmap
mgr/DaemonServer: move pending merge check for more consistent code
qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
mgr/DaemonServer: add option to bypass careful throttling for thrasher
PendingReleaseNotes: note about mgr/balancer/max_misplaced change
mgr/DaemonServer: remove stale/misleading check
mgr/DaemonServer: throttle pgp_num changes based on misplaced %
mgr/DaemonServer: block pg_num decrease(merge) until pgp_num is reduced
mgr/DaemonServer: adjust_pgs(): cosmetic change to debug output
mon/PGMap: add get_recovery_stats()
mgr/balancer: mgr/balancer/max_misplaced -> pg_max_misplaced
pybind/mgr/mgr_module: add get_option()
mgr/DaemonServer: allow pg_num increases that abort pending merges
mon/OSDMonitor: resent pre-nautilus client ops on aborted merge
mon/OSDMonitor: make pgp_num track pg_num more consistently
Reviewed-by: John Spray <john.spray@redhat.com> Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Sage Weil [Mon, 24 Sep 2018 18:53:46 +0000 (13:53 -0500)]
mgr/DaemonServer: remove any upmaps on merging PGs
Remove any pg_upmap[_items] on pgs that are merging to ensure that they
land on the same OSDs.
This is a bit sloppy: we *could* set the source upmap to match the target
upmap (vs potentially moving both PGs to a third location, and/or then
having the balancer move the resulting PG somewhere else again), but for
now assume upmaps are not a common case and Keep It Simple.
Alan Somers [Wed, 4 Oct 2017 22:31:23 +0000 (16:31 -0600)]
blkdev: overhaul API for better portability
* Turn the API into a class so it will work with GoogleMock
* Take file descriptors for all methods, instead of a mix of file
descriptors, path names, and canonical device names. It's more
consistent and it will work better with FreeBSD.
* Split get_device_by_fd into block_device_partition and
block_device_wholedisk
* Remove dead code
Tim Serong [Fri, 19 Oct 2018 08:25:45 +0000 (19:25 +1100)]
mgr/orchestrator: use result property in Completion classes
The _Completion class defined a get_result() method, but the orchestrator
CLI and Rook module both just expected a result property, so let's run with
the property version.
- Crypto plugin created and currently handles only AES_256_CBC
- Contents
-- Poller thread for QAT HW
-- Contiguous memory allocator using QAT's usdm driver
-- Do in-place crypto operation reducing mem usage.
-- Enable multi-QAT instance to allow parallel operations.
Sage Weil [Thu, 18 Oct 2018 22:15:08 +0000 (17:15 -0500)]
qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection
For EC pools we have a lot of shards, and 30% probability on each one
means we are very like to repeatedly fail backfill reservations.. long
enough that teuthology gives up waiting.
Brad Hubbard [Tue, 16 Oct 2018 01:57:05 +0000 (11:57 +1000)]
rpm: Use updated gperftools-libs at runtime
Due to ABI breakage in libtcmalloc.so.4 we need to specify the minimum
version to be used at runtime to be greater than or equal to the version
used at build time.
projects / ceph.git / log