From: Sebastian Wagner Date: Thu, 12 Sep 2019 10:16:03 +0000 (+0200) Subject: doc/orchestrator: Adapted to new Completions X-Git-Tag: v15.1.0~737^2~10 X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=a74d57bcb2381ca43cf05c0c432962b102c78c02;p=ceph.git doc/orchestrator: Adapted to new Completions Signed-off-by: Sebastian Wagner --- diff --git a/doc/mgr/orchestrator_modules.rst b/doc/mgr/orchestrator_modules.rst index 1d52a253dc42..faa1c675f296 100644 --- a/doc/mgr/orchestrator_modules.rst +++ b/doc/mgr/orchestrator_modules.rst @@ -114,9 +114,9 @@ Completions and batching ------------------------ All methods that read or modify the state of the system can potentially -be long running. To handle that, all such methods return a *completion* -object (a *ReadCompletion* or a *WriteCompletion*). Orchestrator modules -must implement the *wait* method: this takes a list of completions, and +be long running. To handle that, all such methods return a *Completion* +object. Orchestrator modules +must implement the *process* method: this takes a list of completions, and is responsible for checking if they're finished, and advancing the underlying operations as needed. @@ -125,24 +125,21 @@ for completions. This might involve running the underlying operations in threads, or batching the operations up before later executing in one go in the background. If implementing such a batching pattern, the module would do no work on any operation until it appeared in a list -of completions passed into *wait*. +of completions passed into *process*. -*WriteCompletion* objects have a two-stage execution. First they become -*persistent*, meaning that the write has made it to the orchestrator -itself, and been persisted there (e.g. a manifest file has been updated). -If ceph-mgr crashed at this point, the operation would still eventually take -effect. Second, the completion becomes *effective*, meaning that the operation has really happened (e.g. a service has actually been started). +Some operations need to show a progress. Those operations need to add +a *ProgressReference* to the completion. At some point, the progress reference +becomes *effective*, meaning that the operation has really happened +(e.g. a service has actually been started). -.. automethod:: Orchestrator.wait +.. automethod:: Orchestrator.process -.. autoclass:: _Completion +.. autoclass:: Completion :members: -.. autoclass:: ReadCompletion +.. autoclass:: ProgressReference :members: -.. autoclass:: WriteCompletion - :members: Placement --------- @@ -196,7 +193,7 @@ In detail, orchestrators need to explicitly deal with different kinds of errors: 5. Errors when actually executing commands The resulting Completion should contain an error string that assists in understanding the - problem. In addition, :func:`_Completion.is_errored` is set to ``True`` + problem. In addition, :func:`Completion.is_errored` is set to ``True`` 6. Invalid configuration in the orchestrator modules @@ -205,7 +202,7 @@ In detail, orchestrators need to explicitly deal with different kinds of errors: All other errors are unexpected orchestrator issues and thus should raise an exception that are then logged into the mgr log file. If there is a completion object at that point, -:func:`_Completion.result` may contain an error message. +:func:`Completion.result` may contain an error message. Excluded functionality diff --git a/src/pybind/mgr/orchestrator.py b/src/pybind/mgr/orchestrator.py index bd9be5957176..f335eec7f7b9 100644 --- a/src/pybind/mgr/orchestrator.py +++ b/src/pybind/mgr/orchestrator.py @@ -321,13 +321,16 @@ class ProgressReference(object): completion=None # type: Optional[Callable[[], Completion[float]]] ): """ - ProgressReference can be used within Completions: + ProgressReference can be used within Completions:: - +---------------+ +---------------------------------+ - | | then | | - | My Completion | +--> | on_complete=ProgressReference() | - | | | | - +---------------+ +---------------------------------+ + +---------------+ +---------------------------------+ + | | then | | + | My Completion | +--> | on_complete=ProgressReference() | + | | | | + +---------------+ +---------------------------------+ + + See :func:`Completion.with_progress` for an easy way to create + a progress reference """ super(ProgressReference, self).__init__() @@ -396,11 +399,34 @@ class ProgressReference(object): self._completion_has_result = True self.progress = 1 + class Completion(_Promise[T]): """ Combines multiple promises into one overall operation. - :ivar exception: Holds an exception object, if the completion errored. + Completions are composable by being able to + call one completion from another completion. I.e. making them re-usable + using Promises E.g.:: + + >>> return Orchestrator().get_hosts().then(self._create_osd) + + where ``get_hosts`` returns a Completion of list of hosts and + ``_create_osd`` takes a list of hosts. + + The concept behind this is to store the computation steps + explicit and then explicitly evaluate the chain: + + >>> p = Completion(on_complete=lambda x: x*2).then(on_complete=lambda x: str(x)) + ... p.finalize(2) + ... assert p.result = "4" + + or graphically:: + + +---------------+ +-----------------+ + | | then | | + | lambda x: x*x | +--> | lambda x: str(x)| + | | | | + +---------------+ +-----------------+ """ def __init__(self,