J. Eric Ivancich [Fri, 29 Jan 2021 17:03:50 +0000 (12:03 -0500)]
rgw: add rgw-gap-list-comparator tool
The rgw-gap-list tool can produce a number of false positives when the
cluster is being used during its run. One technique to minimize the
number of false positives is to run the tool twice and look for the
objects that appear in both lists. The rgw-gap-list-comparator tool is
designed to do this comparison.
J. Eric Ivancich [Thu, 17 Dec 2020 23:21:36 +0000 (18:21 -0500)]
rgw: add rgw-gap-list tool
Due to a prior bug (pr: 38228) tail rados objects of some RGW objects
could have been incorrectly deleted. This tool is designed to look for
such cases. It essentially does the opposite of rgw-orphan-list,
looking for rados objects that RGW expects to be there, but which are
not to be found.
IMPORTANT: This is very experimental at this point in time, and any
"results" produced should be verified by other means.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com> Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
A new command-line option "--rgw-obj-fs" is added to
radosgw-admin. When used with the "bucket radoslist" subcommand, will
output lines with a rados object, bucket name, and object name,
separated by the field separator specified. Without this command-line
option, only the rados object is output, which is the previous
behavior.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Xuehan Xu [Wed, 27 Jan 2021 11:07:58 +0000 (19:07 +0800)]
crimson/osd: use do_with to capture MOSDPGPull::take_pulls()'s return
MOSDPGPull::take_pulls() returns a temporary object, so we need to use seastar::do_with
to create an object whose lives until seastar::do_with's inner continuation chain ends
Xuehan Xu [Wed, 27 Jan 2021 11:04:47 +0000 (19:04 +0800)]
crimson/osd: should capture seastar::do_with params by reference
The lambda within seastar::do_with need to use lvalue reference as its parameter,
otherwise it would be referencing a variable whose lifecycle ends when lambda finishes
runing
Sage Weil [Wed, 27 Jan 2021 21:44:21 +0000 (15:44 -0600)]
python-common: fix test_datetime_to_str_2 on non-UTC hosts
The old test parsed to a datetime without a tz, which was interpreted as
the local time zone when rendering back to a string. Specify that it's a
UTC datetime so that behavior is consistent regardless of the test host
timezone.
Paul Cuzner [Wed, 27 Jan 2021 20:40:58 +0000 (09:40 +1300)]
cephadm:add missing kernel_security property
The propery decorator had gone missing which meant
the dump of host facts was missing the kernel security
(LSM) settings. This patch just adds the @property
decorator back
Zac Dover [Wed, 27 Jan 2021 15:43:59 +0000 (01:43 +1000)]
doc/dev: add Ernesto Puerta as dashboard lead
This commit removes Lenz Grimmer's name and github
handle from the list of component leads and replaces
it with Ernesto Puerta's name and github handle.
Sebastian Wagner [Wed, 27 Jan 2021 11:57:07 +0000 (12:57 +0100)]
Merge pull request #38854 from adk3798/ots-monitoring
mgr/cephadm: force flag for ok-to-stop and ok-to-stop for monitoring stack
Reviewed-by: Juan Miguel Olmo MartÃnez <jolmomar@redhat.com> Reviewed-by: Kefu Chai <kchai@redhat.com> Reviewed-by: Michael Fritch <mfritch@suse.com> Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
`ShardServices::start_operation<T>()` spawns an operation,
calls `start()` on it and returns a future plus a smart
pointer controlling its life-time. Callers are responsible
to ensure the pointer doesn't go out-of-scope before
the entire execution is finished. This is error-prone.
`OSD::handle_osd_op()` forgets about its responsibility
which results in dangling `this` pointer of `ClientRequest`.
I believe the problem is much wider spread and the class
is just the tip of the iceberg.
In this commit `start_operation<T>()` is altered to extend
the life-time a bit. However, this isn't an ultimate solution
as callers are still able to e.g. put extra `this`-catching
lambdas on the returned future. A new `with_operation<T>()`-
like interface is expected as a follow-up.
Jason Dillaman [Tue, 26 Jan 2021 05:18:45 +0000 (00:18 -0500)]
librbd/migration: compute QCOW snapshot deltas via L2 table deltas
Since each entry in a L2 table represents a cluster block within the
QCOW image, we can compute deltas by comparing L2 tables. This fixes
an issue where snapshots were not being properly thin-provisioned
during the live-migration execute process.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Zac Dover [Fri, 22 Jan 2021 04:25:31 +0000 (14:25 +1000)]
doc/dev: improving teuthology docs
This commit is a proof of concept and a repair of
links to the "Integration Tests" section of the
Developer Guide.
This fixes the "reading a standalone test" link
from the "Running Tests in the Cloud" page
to the "Reading a Standalone Test" section in
the "Tests: Integration Tests" page.
also use updated build with sha1 of 1fadde94b08fab574b17637c2bebd2b1e7f9127b, the older version has an issue
where both libzbd and libzbd-dev packaged .pc file with the same path,
so they conflicted with each other. the new version addressed this
issue.
and install libzbd-dev instead of libzbd and libzbd-dev, as the package
name of the former could change if its so version is bumped, so use the
*-dev package is a safer choice from the perspective of testing the
dev build.
Kefu Chai [Tue, 19 Jan 2021 17:56:02 +0000 (01:56 +0800)]
pybind/mgr/dashboard: use _get_localized_key for composing store key
Just for keeping this 'encapsulated', there's _get_localized_key helper
in mgr_module (there's also a set_localized_store but it only deals with
the active manager)
As suggested by Ernesto Puerta <epuertat@redhat.com>
Kefu Chai [Tue, 19 Jan 2021 17:24:01 +0000 (01:24 +0800)]
pybind/mgr/cephadm/inventory: cast variable to expected type
to silence mypy warnings like:
cephadm/module.py:55: note: In module imported here,
cephadm/__init__.py:6: note: ... from here:
cephadm/inventory.py: note: In member "load" of class "SpecStore":
cephadm/inventory.py:130: error: Invalid index type "str" for "str"; expected type "Union[int, slice]"
cephadm/inventory.py:131: error: Invalid index type "str" for "str"; expected type "Union[int, slice]"
Kefu Chai [Tue, 19 Jan 2021 06:49:07 +0000 (14:49 +0800)]
pybind/mgr: correct annotation for BasePyOSDMap._apply_incremental()
it's incorrect. as the underlying C implementation,
"osdmap_apply_incremental()", actually expects an instance of
`BasePyOSDMapIncrementalType`. which is mapped to
`BasePyOSDMapIncremental` in Python. and this class is wrapped using
`OSDMapIncremental` in mgr_module.py.
Kefu Chai [Sun, 17 Jan 2021 08:36:35 +0000 (16:36 +0800)]
mgr/cephadm: use dict instead of defaultdict to silence mypy warning
otherwise we have:
1: cephadm/module.py:30: note: In module imported here:
1: cephadm/serve.py: note: In member "_check_for_strays" of class "CephadmServe":
1: cephadm/serve.py:395: error: Argument "default" to "get_metadata" of "MgrModule" has incompatible type "defaultdict[str, None]"; expected "Optional[Dict[str, str]]"
Kefu Chai [Sun, 17 Jan 2021 07:29:33 +0000 (15:29 +0800)]
mgr/progress: pass id to constructor of Event
to assure that the self.id is always valid, it helps to silence the
warnings from mypy like:
progress/module.py: note: In member "_refresh" of class "Event":
progress/module.py:46: error: Argument 1 to "update_progress_event" of "MgrModule" has incompatible type "Optional[str]"; expected "str"
projects / ceph.git / log