Kefu Chai [Sun, 16 Jun 2019 10:08:50 +0000 (18:08 +0800)]
src: use spirit based ceph.conf parser
before this change, the ceph.conf parser was implemented using a
hand-written state machine. which works fine. but it's hard to
understand and hence difficult to maintain.
in this change, it's rewritten using boost::spirit. which well, is also
a state machine under the hood, but the declarative pattern matching
rules help the human readers to understand the machinary.
before this change, the state machine is able to skip the lines with
invalid syntax and spit a warning message for each of them. after this
change, the parser will just bail out when it fails to match the rules.
and the line continuation marker, i.e., "\" is simply skipped, so it's
allowed to have nothing after the backslash in the value of key-value
pair.
changes are listed as following:
* invalid UTF-8 key/value is now treated as an error
* backslash at the very end is now allowed
* repeated settings are tolerated, and no warnings will be printed.
the last one wins. before this change, we will print out warning
messages at seeing lines with duplicated keys.
* malformed section header is now an error
* a setting line which does not parse is now an error
* unmatched quotes in setting value is now an error
* "\" following an empty line is allowed now.
* the format of error message is changed, so the cli test is updated
accordingly.
* [global] section is not added by default. unless there is a single
line in the conf file. this is only for the convenience of testing.
Casey Bodley [Fri, 21 Jun 2019 19:18:55 +0000 (15:18 -0400)]
qa/rgw: fix swift warning message
File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/swift.py", line 245, in task
log.warning('Swift tests cannot run on rhel 7.6+, skipping client {client}'.format(client))
KeyError: 'client'
David Zafman [Wed, 22 May 2019 01:29:30 +0000 (18:29 -0700)]
mon: Improve health status for backfill_toofull and recovery_toofull
Treat backfull_toofull as a warning condition because it can resolve itself.
Includes test case for PG_BACKFILL_FULL
Includes test case for recovery_toofull / PG_RECOVERY_FULL
Fixes: https://tracker.ceph.com/issues/39555 Signed-off-by: David Zafman <dzafman@redhat.com>
Jason Dillaman [Wed, 19 Jun 2019 15:09:54 +0000 (11:09 -0400)]
rbd: use the ordered throttle for the export action
This allows exports to STDOUT to use multiple concurrent operations
and also fixes a potential race condition with concurrent callbacks
and file seeking.
Fixes: http://tracker.ceph.com/issues/40435 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 13 Jun 2019 19:22:54 +0000 (15:22 -0400)]
librbd: tweaks to improve throughput for journaled IO
Increase the per-object append flush bytes to 1MiB to mimic the 16MiB
in-memory writeback cache. Finally, default enable the new journal
writethrough until flush option.
Performance results comparing no journal, pre-change, and post-change:
no journal original modified
4K 40310.84 7223.73 27427.27
32K 21530.37 3256.39 10284.05
256K 3159.67 564.73 1627.16
4M 223.03 48.03 91.91
Fixes: http://tracker.ceph.com/issues/40072 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Kefu Chai [Sun, 9 Jun 2019 09:33:19 +0000 (17:33 +0800)]
debian/control: remove "libnl-3-dev" from build dependencies
what we need is "libnl-genl-3-dev", libnl-3-dev should be a dependency
of "libnl-genl-3-dev". the only reason we would have added it, is if
the package maintainer of "libnl-genl-3-dev" fails to do so. and
the packages of "libnl-genl-3-dev" on both debian and ubuntu just look
correct in this perspective.
Kefu Chai [Sun, 9 Jun 2019 09:12:27 +0000 (17:12 +0800)]
cmake: rewrite Findgenl to support components argument
* rename genl to nl:
"genl" is not very specific without more context. and libnl is actually
a library suite. so it would be better if we can rename the library to
be found to "libnl", and by following the naming convention of CMake,
in this change, Findgenl.cmake is renamed to Findnl.cmake
* Findnl.cmake: support the component argument, in our specific case,
what we want is "libnl-genl" library, which is one of the libraries
offered by libnl. so let's just make it a component.
* Findnl.cmake: should pass the names of the required variables instead of
their values to `find_package_handle_standard_args()`. before this
change, we pass `GENL_LIBRARIES` to this function. it was correct at the
first glance. but it is not able to handle the case where case where
libnl-genl is not installed. so the fix is to pass all the names of
required library paths to this function. in this change, their name
are concatenated to a single variable -- `nl_LIBRARIES`, and the
value of this variable is passed to
`find_package_handle_standard_args()`. and the error message would
be more specific this way if libnl-genl is not found:
Could NOT find nl (missing: nl_genl_LIBRARY)
* Findnl.cmake: add nl::<component> as imported library, it helps the
consumer of these libraries to import them -- no need to
specify the `target_include_directories()` separately anymore.
* move the find_package() code to where it is used. it helps to improve
the readability.
* remove `HAVE_GENL` variable: it's not used anywhere.
* drop the messages of "Not using foobar", they do not help.
Lenz Grimmer [Wed, 19 Jun 2019 13:07:39 +0000 (15:07 +0200)]
mgr/dashboard: Ensure E2E tests can be run independently (#28592)
mgr/dashboard: Ensure E2E tests can be run independently
Reviewed-by: Adam King <adking@redhat.com> Reviewed-by: Laura Paduano <lpaduano@suse.com> Reviewed-by: Nathan Weinberg <nweinber@redhat.com> Reviewed-by: Rafael Quintero <rquinter@redhat.com> Reviewed-by: Tiago Melo <tmelo@suse.com>
Sage Weil [Tue, 18 Jun 2019 15:02:40 +0000 (10:02 -0500)]
Merge PR #28229 into master
* refs/pull/28229/head:
os/bluestore: do garbage collection if blob count is too high.
common/perf_conters: make dump_formatted_xxx funcs as const.
os/bluestore: store extents for GC within WriteContext.
os/bluestore: GC class, make some members local.
os/bluestore: vector -> interval set in GC to track extents to collect.
tests/store_test: many-many spanning blobs test case
Patrick Nawracay [Mon, 17 Jun 2019 09:36:53 +0000 (11:36 +0200)]
mgr/dashboard: Ensure E2E tests can be run independently
Introduces the PageHelper.
- Reads class attributes from Helpers to reduce boilerplace code for
navigation.
- The PageHelper is supposed to be the new class for code that's reused
across all Helpers for page related code.
- The Helper class is by now meant to be used for non-page specific
helper code, like used in the `checkConsole` method.
- The Helper class will act as central registry to enable all other
tests to use helper functions of all derived PageHelper tests.
Example:
`Helper.pools.create('foobar', ...);`
Fixes: http://tracker.ceph.com/issues/40397 Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
xie xingguo [Tue, 18 Jun 2019 12:05:31 +0000 (20:05 +0800)]
mgr, osd: 'ceph osd df' by pool
Our test admin has been asking for this for the past few years:-)
Besides, this is also useful for operating on large Ceph clusters with
mutliple storage pools possibly spanning over all osds.
projects / ceph.git / log