Sebastian Wagner [Wed, 27 Jan 2021 11:57:07 +0000 (12:57 +0100)]
Merge pull request #38854 from adk3798/ots-monitoring
mgr/cephadm: force flag for ok-to-stop and ok-to-stop for monitoring stack
Reviewed-by: Juan Miguel Olmo MartÃnez <jolmomar@redhat.com> Reviewed-by: Kefu Chai <kchai@redhat.com> Reviewed-by: Michael Fritch <mfritch@suse.com> Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
`ShardServices::start_operation<T>()` spawns an operation,
calls `start()` on it and returns a future plus a smart
pointer controlling its life-time. Callers are responsible
to ensure the pointer doesn't go out-of-scope before
the entire execution is finished. This is error-prone.
`OSD::handle_osd_op()` forgets about its responsibility
which results in dangling `this` pointer of `ClientRequest`.
I believe the problem is much wider spread and the class
is just the tip of the iceberg.
In this commit `start_operation<T>()` is altered to extend
the life-time a bit. However, this isn't an ultimate solution
as callers are still able to e.g. put extra `this`-catching
lambdas on the returned future. A new `with_operation<T>()`-
like interface is expected as a follow-up.
Jason Dillaman [Tue, 26 Jan 2021 05:18:45 +0000 (00:18 -0500)]
librbd/migration: compute QCOW snapshot deltas via L2 table deltas
Since each entry in a L2 table represents a cluster block within the
QCOW image, we can compute deltas by comparing L2 tables. This fixes
an issue where snapshots were not being properly thin-provisioned
during the live-migration execute process.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
also use updated build with sha1 of 1fadde94b08fab574b17637c2bebd2b1e7f9127b, the older version has an issue
where both libzbd and libzbd-dev packaged .pc file with the same path,
so they conflicted with each other. the new version addressed this
issue.
and install libzbd-dev instead of libzbd and libzbd-dev, as the package
name of the former could change if its so version is bumped, so use the
*-dev package is a safer choice from the perspective of testing the
dev build.
Kefu Chai [Tue, 19 Jan 2021 17:56:02 +0000 (01:56 +0800)]
pybind/mgr/dashboard: use _get_localized_key for composing store key
Just for keeping this 'encapsulated', there's _get_localized_key helper
in mgr_module (there's also a set_localized_store but it only deals with
the active manager)
As suggested by Ernesto Puerta <epuertat@redhat.com>
Kefu Chai [Tue, 19 Jan 2021 17:24:01 +0000 (01:24 +0800)]
pybind/mgr/cephadm/inventory: cast variable to expected type
to silence mypy warnings like:
cephadm/module.py:55: note: In module imported here,
cephadm/__init__.py:6: note: ... from here:
cephadm/inventory.py: note: In member "load" of class "SpecStore":
cephadm/inventory.py:130: error: Invalid index type "str" for "str"; expected type "Union[int, slice]"
cephadm/inventory.py:131: error: Invalid index type "str" for "str"; expected type "Union[int, slice]"
Kefu Chai [Tue, 19 Jan 2021 06:49:07 +0000 (14:49 +0800)]
pybind/mgr: correct annotation for BasePyOSDMap._apply_incremental()
it's incorrect. as the underlying C implementation,
"osdmap_apply_incremental()", actually expects an instance of
`BasePyOSDMapIncrementalType`. which is mapped to
`BasePyOSDMapIncremental` in Python. and this class is wrapped using
`OSDMapIncremental` in mgr_module.py.
Kefu Chai [Sun, 17 Jan 2021 08:36:35 +0000 (16:36 +0800)]
mgr/cephadm: use dict instead of defaultdict to silence mypy warning
otherwise we have:
1: cephadm/module.py:30: note: In module imported here:
1: cephadm/serve.py: note: In member "_check_for_strays" of class "CephadmServe":
1: cephadm/serve.py:395: error: Argument "default" to "get_metadata" of "MgrModule" has incompatible type "defaultdict[str, None]"; expected "Optional[Dict[str, str]]"
Kefu Chai [Sun, 17 Jan 2021 07:29:33 +0000 (15:29 +0800)]
mgr/progress: pass id to constructor of Event
to assure that the self.id is always valid, it helps to silence the
warnings from mypy like:
progress/module.py: note: In member "_refresh" of class "Event":
progress/module.py:46: error: Argument 1 to "update_progress_event" of "MgrModule" has incompatible type "Optional[str]"; expected "str"
Kefu Chai [Sun, 17 Jan 2021 01:35:36 +0000 (09:35 +0800)]
pybind/mgr/iostat: use correct types
for better readability and for the sake of correctness.
* do not cast denominator to float before diving by it
* use '//' for divding an int
* cast int to str as it will be used as an element in a list of str
Kefu Chai [Sun, 24 Jan 2021 06:58:51 +0000 (14:58 +0800)]
cephadm: refactor call() using asyncio.asyncio.StreamReader
simpler this way, also fix a couple issues:
* create a child watcher explicitly, see
https://bugs.python.org/issue35621
* use StringIO for collecting outputs for better performance,
instead of appending the lines to an existing str
* catch ValueError when reading from the stream reader,
because StreamReader.readline() could raise ValueError when
it reaches the buffer limit while looking for a separator.
in this case, we should try again, in hope that the spawned
process can feed the reader with more data which contains
the separator (i.e., b'\n').
* backport ThreadedChildWatcher from Python 3.8 so we can
run create_subprocess_exec() in non-main threads.
Lucian Petrut [Thu, 7 Jan 2021 13:14:00 +0000 (13:14 +0000)]
cmake,win32*.sh: Windows DLL support
At the moment, the Windows binaries can only be linked statically.
This is less than ideal since:
* the resulting binaries can be quite large, especially when
including debug symbols
* librados and librbd cannot be used directly by 3rd party apps
* the build duration is increased
In order to do a dynamic build, we'll:
* add an option to win32_build.sh
* fix link order
* dynamically link boost
* disable the "-pie" flag when using Mingw, which generates incorrect
executable entry points
* by default, cmake generates import libs for executables. The issue
is that for rados.exe, it generates librados.dll.a, thus overriding
the librados.dll import library, which breaks the build process.
We'll configure cmake to skip generating import libs for executables.
Jason Dillaman [Tue, 26 Jan 2021 03:17:25 +0000 (22:17 -0500)]
librbd/migration: allow retrieving QCOW L2 table from L2 cache
This will be used in the next commit where the list-snaps state
machine is tweaked to compate L2 tables between snapshot revisions
to determine deltas.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Tue, 26 Jan 2021 01:45:40 +0000 (20:45 -0500)]
librbd/migration: refactor lookup table handling for QCOW format
This will be used by a refactored list-snaps state machine to compute
the L2 table cluster for image offsets. The L1 and L2 lookup tables
can now be represented in a single structure.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
projects / ceph.git / log