mds/quiesce: QuiesceDb.h and QuiesceDbManager with tests
Quiesce DB is one of the components of the "Consistent Snapshots" epic.
The solution is discussed in a slide deck available for viewing to @redhat users:
https://docs.google.com/presentation/d/1wE3-e9AAme7Q3qmeshUSthJoQGw7-fKTrtS9PsdAIVo/edit?usp=sharing
This commit is focusing on the replicated quiesce database maintained by the MDS rank cluster.
One of the major goals was to design the component in a way that can be easily tested
outside of the MDS infrastructure, which is why the communication layer
has been asbtracted out by introducing just two communication callbacks
that will need to be implemented by the infrastructure.
The most of the component code is delivered in a single coherent commit, along with the uint tests.
Other commits will be dedicated to integration with the MDS infrastructure and other changes
that can't be attributed to the core quiesce db code or its tests.
The quiesce db component is composed of the following major parts/actors:
* QuiesceDbManager is the main actor, implementing both the leader and the replica roles.
Normally, there will be an instance of the manager per MDS rank, although given the
decoupling of the infrastructure and the manager, one can run any number of instances
on a single node, which is how test are working.
* The manager interfaces to the infrastructure via two main APIs with the infrastructure
that provides communication and cluster configuration (actor 2) and the quiesce db
client that is responsible for the quiescing of the roots (actor 3)
** ClusterMembership is how manager is configured to be part of a (virtual) cluster.
This structure will deliver information about other peers, the leader and provide
two communication APIs: send_listing_to for db replication from the leader to replicas
and send_ack for reporting quiesce success from the agents.
** Client Interface consists of a QuisceMap notify callback and a dedicated manager
method to submit asynchronous acks following the agent (rank) quiesce progress.
The API of the quiesce db is described in the slide deck mentioned above. The full scope
of capabilities are encapsulated in a single QuiesceDbRequest structure. This should
simplify the implementation of other components that will have to propagate the functionality
to the administrator user of the volumes plugin.