version: 3
install:
- requirements: admin/doc-requirements.txt
+ - requirements: admin/doc-read-the-docs.txt
sphinx:
builder: dirhtml
configuration: doc/conf.py
- :term:`Ceph Monitor`
- :term:`Ceph OSD Daemon`
-.. ditaa:: +---------------+ +---------------+
+.. ditaa::
+
+ +---------------+ +---------------+
| OSDs | | Monitors |
+---------------+ +---------------+
file in a filesystem, which is stored on an :term:`Object Storage Device`. Ceph
OSD Daemons handle the read/write operations on the storage disks.
-.. ditaa:: /-----\ +-----+ +-----+
+.. ditaa::
+
+ /-----\ +-----+ +-----+
| obj |------>| {d} |------>| {s} |
\-----/ +-----+ +-----+
forth.
-.. ditaa:: /------+------------------------------+----------------\
+.. ditaa::
+
+ /------+------------------------------+----------------\
| ID | Binary Data | Metadata |
+------+------------------------------+----------------+
| 1234 | 0101010101010100110101010010 | name1 = value1 |
.. note:: The ``client.admin`` user must provide the user ID and
secret key to the user in a secure manner.
-.. ditaa:: +---------+ +---------+
+.. ditaa::
+
+ +---------+ +---------+
| Client | | Monitor |
+---------+ +---------+
| request to |
ticket and uses it to sign requests to OSDs and metadata servers throughout the
cluster.
-.. ditaa:: +---------+ +---------+
+.. ditaa::
+
+ +---------+ +---------+
| Client | | Monitor |
+---------+ +---------+
| authenticate |
subsequent to the initial authentication, is signed using a ticket that the
monitors, OSDs and metadata servers can verify with their shared secret.
-.. ditaa:: +---------+ +---------+ +-------+ +-------+
+.. ditaa::
+
+ +---------+ +---------+ +-------+ +-------+
| Client | | Monitor | | MDS | | OSD |
+---------+ +---------+ +-------+ +-------+
| request to | | |
and tertiary OSDs (as many OSDs as additional replicas), and responds to the
client once it has confirmed the object was stored successfully.
-.. ditaa::
+.. ditaa::
+
+----------+
| Client |
| |
pools. The pool's ``size`` or number of replicas, the CRUSH rule and the
number of placement groups determine how Ceph will place the data.
-.. ditaa::
+.. ditaa::
+
+--------+ Retrieves +---------------+
| Client |------------>| Cluster Map |
+--------+ +---------------+
come online. The following diagram depicts how CRUSH maps objects to placement
groups, and placement groups to OSDs.
-.. ditaa::
+.. ditaa::
+
/-----\ /-----\ /-----\ /-----\ /-----\
| obj | | obj | | obj | | obj | | obj |
\-----/ \-----/ \-----/ \-----/ \-----/
new OSD after rebalancing is complete.
-.. ditaa::
+.. ditaa::
+
+--------+ +--------+
Before | OSD 1 | | OSD 2 |
+--------+ +--------+
.. ditaa::
+
+-------------------+
name | NYAN |
+-------------------+
account.
.. ditaa::
+
+-------------------+
name | NYAN |
+-------------------+
.. ditaa::
+
Primary OSD
+-------------+
on **OSD 3**.
.. ditaa::
+
Primary OSD
+-------------+
will be the head of the new authoritative log.
.. ditaa::
+
+-------------+
| OSD 1 |
| (down) |
.. ditaa::
+
Primary OSD
+-------------+
to Ceph clients.
-.. ditaa::
+.. ditaa::
+
+-------------+
| Ceph Client |
+------+------+
you can create your own custom Ceph Clients. The following diagram depicts the
basic architecture.
-.. ditaa::
+.. ditaa::
+
+---------------------------------+
| Ceph Storage Cluster Protocol |
| (librados) |
synchronization/communication channel.
-.. ditaa:: +----------+ +----------+ +----------+ +---------------+
+.. ditaa::
+
+ +----------+ +----------+ +----------+ +---------------+
| Client 1 | | Client 2 | | Client 3 | | OSD:Object ID |
+----------+ +----------+ +----------+ +---------------+
| | | |
groups, and consequently doesn't improve performance very much. The following
diagram depicts the simplest form of striping:
-.. ditaa::
+.. ditaa::
+
+---------------+
| Client Data |
| Format |
stripe (``stripe unit 16``) in the first object in the new object set (``object
4`` in the diagram below).
-.. ditaa::
+.. ditaa::
+
+---------------+
| Client Data |
| Format |
architecture.
.. ditaa::
+
+--------------+ +----------------+ +-------------+
| Block Device | | Object Storage | | CephFS |
+--------------+ +----------------+ +-------------+
a Filesystem in User Space (FUSE).
.. ditaa::
+
+-----------------------+ +------------------------+
| CephFS Kernel Object | | CephFS FUSE |
+-----------------------+ +------------------------+
+import shutil
import sys
import os
'sphinx.ext.autodoc',
'sphinx.ext.graphviz',
'sphinx.ext.todo',
- 'sphinxcontrib.ditaa',
'breathe',
'edit_on_github',
]
-ditaa = 'ditaa'
+ditaa = shutil.which("ditaa")
+if ditaa is not None:
+ extensions += ['sphinxcontrib.ditaa']
+else:
+ extensions += ['plantweb.directive']
+ plantweb_defaults = {
+ 'engine': 'ditaa'
+ }
+
todo_include_todos = True
top_level = os.path.dirname(
# handles edit-on-github and old version warning display
def setup(app):
app.add_javascript('js/ceph.js')
+ if ditaa is None:
+ # add "ditaa" as an alias of "diagram"
+ from plantweb.directive import DiagramDirective
+ app.add_directive('ditaa', DiagramDirective)
# mocking ceph_module offered by ceph-mgr. `ceph_module` is required by
# mgr.mgr_module
of Cloud Platforms include OpenStack, CloudStack, OpenNebula, etc.
-.. ditaa:: +---------------------------------------------------+
+.. ditaa::
+
+ +---------------------------------------------------+
| libvirt |
+------------------------+--------------------------+
|
.. ditaa::
+
/------------------\ /----------------\
| Admin Node | | node1 |
| +-------->+ |
.. ditaa::
+
/------------------\ /----------------\
| Admin Node | | node1 |
| +-------->+ |
- The :term:`Ceph Monitor`, which maintains a master copy of the cluster map.
- The :term:`Ceph OSD Daemon` (OSD), which stores data as objects on a storage node.
-.. ditaa::
+.. ditaa::
+---------------------------------+
| Ceph Storage Cluster Protocol |
| (librados) |
can read or write data. The client app doesn't need to learn about the topology
of the cluster directly.
-.. ditaa::
+.. ditaa::
+--------+ Retrieves +---------------+
| Client |------------>| Cluster Map |
+--------+ +---------------+
initial connection.
-.. ditaa:: +---------+ +---------+
+.. ditaa::
+ +---------+ +---------+
| Client | | Monitor |
+---------+ +---------+
| |
- Snapshot pools, list snapshots, etc.
-.. ditaa:: +---------+ +---------+ +---------+
+.. ditaa::
+ +---------+ +---------+ +---------+
| Client | | Monitor | | OSD |
+---------+ +---------+ +---------+
| | |
operations. Ceph Monitors leverage the key/value store's snapshots and iterators
(using leveldb) to perform store-wide synchronization.
-.. ditaa::
-
+.. ditaa::
/-------------\ /-------------\
| Monitor | Write Changes | Paxos |
| cCCC +-------------->+ cCCC |
capacity is 95TB, not 99TB.
.. ditaa::
-
+--------+ +--------+ +--------+ +--------+ +--------+ +--------+
| Rack 1 | | Rack 2 | | Rack 3 | | Rack 4 | | Rack 5 | | Rack 6 |
| cCCC | | cF00 | | cCCC | | cCCC | | cCCC | | cCCC |
and the leader tells the requester to synchronize with a provider.
-.. ditaa:: +-----------+ +---------+ +----------+
+.. ditaa::
+ +-----------+ +---------+ +----------+
| Requester | | Leader | | Provider |
+-----------+ +---------+ +----------+
| | |
or by setting the value at runtime.
-.. ditaa:: +---------+ +---------+
+.. ditaa::
+ +---------+ +---------+
| OSD 1 | | OSD 2 |
+---------+ +---------+
| |
your Ceph configuration file, or by setting the value at runtime.
-.. ditaa:: +---------+ +---------+ +---------+
+.. ditaa::
+
+ +---------+ +---------+ +---------+
| OSD 1 | | OSD 2 | | Monitor |
+---------+ +---------+ +---------+
| | |
setting under the ``[osd]`` section of your Ceph configuration file, or by
setting the value at runtime.
-.. ditaa:: +---------+ +---------+ +-------+ +---------+
+.. ditaa::
+
+ +---------+ +---------+ +-------+ +---------+
| OSD 1 | | OSD 2 | | OSD 3 | | Monitor |
+---------+ +---------+ +-------+ +---------+
| | | |
file, or by setting the value at runtime.
-.. ditaa:: +---------+ +---------+
+.. ditaa::
+
+ +---------+ +---------+
| OSD 1 | | Monitor |
+---------+ +---------+
| |
#. One for sending data to other OSDs.
#. Two for heartbeating on each interface.
-.. ditaa::
+.. ditaa::
/---------------\
| OSD |
| +---+----------------+-----------+
to Ceph clients.
-.. ditaa::
+.. ditaa::
+-------------+
| Ceph Client |
+------+------+
.. note:: If an OSD is ``down`` and ``in``, there is a problem and the cluster
will not be in a healthy state.
-.. ditaa:: +----------------+ +----------------+
+.. ditaa::
+
+ +----------------+ +----------------+
| | | |
| OSD #n In | | OSD #n Up |
| | | |
(assuming a pool with 3 replicas of the PG).
-.. ditaa:: +---------+ +---------+ +-------+
+.. ditaa::
+
+ +---------+ +---------+ +-------+
| OSD 1 | | OSD 2 | | OSD 3 |
+---------+ +---------+ +-------+
| | |
status should be ``active+clean``, which means a Ceph client can begin writing
to the placement group.
-.. ditaa::
-
+.. ditaa::
+
/-----------\ /-----------\ /-----------\
| Creating |------>| Peering |------>| Active |
\-----------/ \-----------/ \-----------/
group will store objects on two OSDs, as shown below.
.. ditaa::
-
+-----------------------+ +-----------------------+
| Placement Group #1 | | Placement Group #2 |
| | | |
individuals or system actors such as applications, which use Ceph clients to
interact with the Ceph Storage Cluster daemons.
-.. ditaa:: +-----+
+.. ditaa::
+ +-----+
| {o} |
| |
+--+--+ /---------\ /---------\
- **Subuser:** The term 'subuser' reflects a user of the Swift interface. A subuser
is associated to a user .
-.. ditaa:: +---------+
+.. ditaa::
+ +---------+
| User |
+----+----+
|
or Ceph Block Device clients. The S3 and Swift APIs share a common namespace, so
you may write data with one API and retrieve it with the other.
-.. ditaa:: +------------------------+ +------------------------+
+.. ditaa::
+
+ +------------------------+ +------------------------+
| S3 compatible API | | Swift compatible API |
+------------------------+-+------------------------+
| radosgw |
- `Ruby`_
-.. ditaa:: +----------------------------+ +-----------------------------+
+.. ditaa::
+
+ +----------------------------+ +-----------------------------+
| | | |
| Create a Connection |------->| Create a Container |
| | | |
:abbr:`RADOS (Reliable Autonomic Distributed Object Store)` Block Devices (RBD)
interact with OSDs using kernel modules or the ``librbd`` library.
-.. ditaa:: +------------------------+ +------------------------+
+.. ditaa::
+
+ +------------------------+ +------------------------+
| Kernel Module | | librbd |
+------------------------+-+------------------------+
| RADOS Protocol |
illustrates how ``libvirt`` and QEMU use Ceph block devices via ``librbd``.
-.. ditaa:: +---------------------------------------------------+
+.. ditaa::
+
+ +---------------------------------------------------+
| libvirt |
+------------------------+--------------------------+
|
image each time it spins up a new virtual machine.
-.. ditaa:: +---------------------------------------------------+
+.. ditaa::
+
+ +---------------------------------------------------+
| QEMU |
+---------------------------------------------------+
| librbd |
following diagram depicts the CloudStack/Ceph technology stack.
-.. ditaa:: +---------------------------------------------------+
+.. ditaa::
+
+ +---------------------------------------------------+
| CloudStack |
+---------------------------------------------------+
| libvirt |
technology stack.
-.. ditaa:: +---------------------------------------------------+
+.. ditaa::
+
+ +---------------------------------------------------+
| OpenStack |
+---------------------------------------------------+
| libvirt |
For virtual machines, `qemu-guest-agent` can be used to automatically freeze
filesystems when creating a snapshot.
-.. ditaa:: +------------+ +-------------+
+.. ditaa::
+
+ +------------+ +-------------+
| {s} | | {s} c999 |
| Active |<-------*| Snapshot |
| Image | | of Image |
clones rapidly.
-.. ditaa:: +-------------+ +-------------+
+.. ditaa::
+
+ +-------------+ +-------------+
| {s} c999 | | {s} |
| Snapshot | Child refers | COW Clone |
| of Image |<------------*| of Snapshot |
create a snapshot of the image. You must protect the snapshot. Once you have
performed these steps, you can begin cloning the snapshot.
-.. ditaa:: +----------------------------+ +-----------------------------+
+.. ditaa::
+
+ +----------------------------+ +-----------------------------+
| | | |
| Create Block Device Image |------->| Create a Snapshot |
| | | |
Ceph OSD (Object Storage Daemon). The Ceph Metadata Server is also
required when running Ceph Filesystem clients.
-.. ditaa:: +---------------+ +------------+ +------------+ +---------------+
+.. ditaa::
+
+ +---------------+ +------------+ +------------+ +---------------+
| OSDs | | Monitors | | Managers | | MDSs |
+---------------+ +------------+ +------------+ +---------------+
Ceph Monitor and Ceph Manager to ``node2`` and ``node3`` to improve reliability and availability.
.. ditaa::
+
/------------------\ /----------------\
| ceph-deploy | | node1 |
| Admin Node | | cCCC |
-.. ditaa::
+.. ditaa::
+
/------------------\ /-----------------\
| admin-node | | node1 |
| +-------->+ cCCC |
Block Device.
-.. ditaa::
+.. ditaa::
+
/------------------\ /----------------\
| Admin Node | | ceph-client |
| +-------->+ cCCC |
projects / ceph.git / commitdiff