+++ /dev/null
-======================
-Building ``ceph.conf``
-======================
--- /dev/null
+========================
+Ceph Configuration Files
+========================
+When you start the Ceph service, the initialization process activates a series of daemons that run in the background.
+The hosts in a typical RADOS cluster run at least one of three processes or daemons:
+
+- RADOS (``ceph-osd``)
+- Monitor (``ceph-mon``)
+- Metadata Server (``ceph-mds``)
+
+Each process or daemon looks for a ``ceph.conf`` file that provides their configuration settings.
+The default ``ceph.conf`` locations in sequential order include:
+
+ 1. ``$CEPH_CONF`` (*i.e.,* the path following the ``$CEPH_CONF`` environment variable)
+ 2. ``-c path/path`` (*i.e.,* the ``-c`` command line argument)
+ 3. /etc/ceph/ceph.conf
+ 4. ``~/.ceph/config``
+ 5. ``./ceph.conf`` (*i.e.,* in the current working directory)
+
+The ``ceph.conf`` file provides the settings for each Ceph daemon. Once you have installed the Ceph packages on the OSD Cluster hosts, you need to create
+a ``ceph.conf`` file to configure your OSD cluster.
+
+Creating ``ceph.conf``
+----------------------
+The ``ceph.conf`` file defines:
+
+- Cluster Membership
+- Host Names
+- Paths to Hosts
+- Runtime Options
+
+You can add comments to the ``ceph.conf`` file by preceding comments with a semi-colon (;). For example::
+
+ ; <--A semi-colon precedes a comment
+ ; A comment may be anything, and always follows a semi-colon on each line.
+ ; We recommend that you provide comments in your configuration file(s).
+
+Configuration File Basics
+~~~~~~~~~~~~~~~~~~~~~~~~~
+The ``ceph.conf`` file configures each instance of the three common processes
+in a RADOS cluster.
+
++-----------------+--------------+--------------+-----------------+-------------------------------------------------+
+| Setting Scope | Process | Setting | Instance Naming | Description |
++=================+==============+==============+=================+=================================================+
+| All Modules | All | ``[global]`` | N/A | Settings affect all instances of all daemons. |
++-----------------+--------------+--------------+-----------------+-------------------------------------------------+
+| Groups | Group | ``[group]`` | Alphanumeric | Settings affect all instances within the group |
++-----------------+--------------+--------------+-----------------+-------------------------------------------------+
+| RADOS | ``ceph-osd`` | ``[osd]`` | Numeric | Settings affect RADOS instances only. |
++-----------------+--------------+--------------+-----------------+-------------------------------------------------+
+| Monitor | ``ceph-mon`` | ``[mon]`` | Alphanumeric | Settings affect monitor instances only. |
++-----------------+--------------+--------------+-----------------+-------------------------------------------------+
+| Metadata Server | ``ceph-mds`` | ``[mds]`` | Alphanumeric | Settings affect MDS instances only. |
++-----------------+--------------+--------------+-----------------+-------------------------------------------------+
+
+Metavariables
+~~~~~~~~~~~~~
+The configuration system supports certain 'metavariables,' which are typically used in ``[global]`` or process/daemon settings.
+If metavariables occur inside a configuration value, Ceph expands them into a concrete value--similar to how Bash shell expansion works.
+
+There are a few different metavariables:
+
++--------------+----------------------------------------------------------------------------------------------------------+
+| Metavariable | Description |
++==============+==========================================================================================================+
+| ``$host`` | Expands to the host name of the current daemon. |
++--------------+----------------------------------------------------------------------------------------------------------+
+| ``$type`` | Expands to one of ``mds``, ``osd``, or ``mon``, depending on the type of the current daemon. |
++--------------+----------------------------------------------------------------------------------------------------------+
+| ``$id`` | Expands to the daemon identifier. For ``osd.0``, this would be ``0``; for ``mds.a``, it would be ``a``. |
++--------------+----------------------------------------------------------------------------------------------------------+
+| ``$num`` | Same as ``$id``. |
++--------------+----------------------------------------------------------------------------------------------------------+
+| ``$name`` | Expands to ``$type.$id``. |
++--------------+----------------------------------------------------------------------------------------------------------+
+
+Global Settings
+~~~~~~~~~~~~~~~
+The Ceph configuration file supports a hierarchy of settings, where child settings inherit the settings of the parent.
+Global settings affect all instances of all processes in the cluster. Use the ``[global]`` setting for values that
+are common for all hosts in the cluster. You can override each ``[global]`` setting by:
+
+1. Changing the setting in a particular ``[group]``.
+2. Changing the setting in a particular process type (*e.g.,* ``[osd]``, ``[mon]``, ``[mds]`` ).
+3. Changing the setting in a particular process (*e.g.,* ``[osd.1]`` )
+
+Overriding a global setting affects all child processes, except those that you specifically override.
+
+For example::
+
+ [global]
+ ; Enable authentication between hosts within the cluster.
+ auth supported = cephx
+
+Group Settings
+~~~~~~~~~~~~~~
+Group settings affect all instances of all processes in a group. Use the ``[group]`` setting for values that
+are common for all hosts in a group within the cluster. Each group must have a name. For example::
+
+ [group primary]
+ addr = 10.9.8.7
+
+ [group secondary]
+ addr = 6.5.4.3
+
+
+Process/Daemon Settings
+~~~~~~~~~~~~~~~~~~~~~~~
+You can specify settings that apply to a particular type of process. When you specify settings under ``[osd]``, ``[mon]`` or ``[mds]`` without specifying a particular instance,
+the setting will apply to all OSDs, monitors or metadata daemons respectively.
+
+Instance Settings
+~~~~~~~~~~~~~~~~~
+You may specify settings for particular instances of an daemon. You may specify an instance by entering its type, delimited by a period (.) and
+by the instance ID. The instance ID for an OSD is always numeric, but it may be alphanumeric for monitors and metadata servers. ::
+
+ [osd.1]
+ ; settings affect osd.1 only.
+ [mon.a1]
+ ; settings affect mon.a1 only.
+ [mds.b2]
+ ; settings affect mds.b2 only.
+
+``host`` and ``addr`` Settings
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The `Hardware Recommendations <../install/hardware_recommendations>`_ section provides some hardware guidelines for configuring the cluster.
+It is possible for a single host to run multiple daemons. For example, a single host with multiple disks or RAIDs may run one ``ceph-osd``
+for each disk or RAID. Additionally, a host may run both a ``ceph-mon`` and an ``ceph-osd`` daemon on the same host. Ideally, you will have
+a host for a particular type of process. For example, one host may run ``ceph-osd`` daemons, another host may run a ``ceph-mds`` daemon,
+and other hosts may run ``ceph-mon`` daemons.
+
+Each host has a name identified by the ``host`` setting, and a network location (i.e., domain name or IP address) identified by the ``addr`` setting.
+For example::
+
+ [osd.1]
+ host = hostNumber1
+ addr = 150.140.130.120:1100
+ [osd.2]
+ host = hostNumber1
+ addr = 150.140.130.120:1102
+
+
+Monitor Configuration
+~~~~~~~~~~~~~~~~~~~~~
+Ceph typically deploys with 3 monitors to ensure high availability should a monitor instance crash. An odd number of monitors (3) ensures
+that the Paxos algorithm can determine which version of the cluster map is the most accurate.
+
+.. note:: You may deploy Ceph with a single monitor, but if the instance fails, the lack of a monitor may interrupt data service availability.
+
+Ceph monitors typically listen on port ``6789``.
+
+Example Configuration File
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. literalinclude:: demo-ceph.conf
+ :language: ini
+
+
+Configuration File Deployment Options
+-------------------------------------
+The most common way to deploy the ``ceph.conf`` file in a cluster is to have all hosts share the same configuration file.
+
+You may create a ``ceph.conf`` file for each host if you wish, or specify a particular ``ceph.conf`` file for a subset of hosts within the cluster. However, using per-host ``ceph.conf``
+configuration files imposes a maintenance burden as the cluster grows. In a typical deployment, an administrator creates a ``ceph.conf`` file on the Administration host and then copies that
+file to each OSD Cluster host.
+
+The current cluster deployment script, ``mkcephfs``, does not make copies of the ``ceph.conf``. You must copy the file manually.
+
+
+
+
--- /dev/null
+[global]
+ auth supported = cephx
+ keyring = /etc/ceph/$name.keyring
+
+[mon]
+ mon data = /srv/mon.$id
+
+[mds]
+
+[osd]
+ osd data = /srv/osd.$id
+ osd journal = /srv/osd.$id.journal
+ osd journal size = 1000
+
+[mon.a]
+ host = myserver01
+ mon addr = 10.0.0.101:6789
+
+[mon.b]
+ host = myserver02
+ mon addr = 10.0.0.102:6789
+
+[mon.c]
+ host = myserver03
+ mon addr = 10.0.0.103:6789
+
+[osd.0]
+ host = myserver01
+
+[osd.1]
+ host = myserver02
+
+[osd.2]
+ host = myserver03
+
+[mds.a]
+ host = myserver01
\ No newline at end of file
--- /dev/null
+============================
+Deploying Ceph Configuration
+============================
+Ceph's current deployment script does not copy the configuration file you created from the Administration host
+to the OSD Cluster hosts. Copy the configuration file you created (*i.e.,* ``mycluster.conf`` in the example below)
+from the Administration host to ``etc/ceph/ceph.conf`` on each OSD Cluster host.
+
+::
+
+ ssh myserver01 sudo tee /etc/ceph/ceph.conf <mycluster.conf
+ ssh myserver02 sudo tee /etc/ceph/ceph.conf <mycluster.conf
+ ssh myserver03 sudo tee /etc/ceph/ceph.conf <mycluster.conf
+
+
+The current deployment script doesn't copy the start services. Copy the ``start``
+services from the Administration host to each OSD Cluster host. ::
+
+ ssh myserver01 sudo /etc/init.d/ceph start
+ ssh myserver02 sudo /etc/init.d/ceph start
+ ssh myserver03 sudo /etc/init.d/ceph start
+
+The current deployment script may not create the default server directories. Create
+server directories for each instance of a Ceph daemon.
+
+Using the exemplary ``ceph.conf`` file, you would perform the following:
+
+On ``myserver01``::
+
+ mkdir srv/osd.0
+ mkdir srv/mon.a
+
+On ``myserver02``::
+
+ mkdir srv/osd.1
+ mkdir srv/mon.b
+
+On ``myserver03``::
+
+ mkdir srv/osd.2
+ mkdir srv/mon.c
+
+On ``myserver04``::
+
+ mkdir srv/osd.3
+
+.. important:: The ``host`` variable determines which host runs each instance of a Ceph daemon.
+
--- /dev/null
+================================
+Deploying Ceph with ``mkcephfs``
+================================
+
+Once you have copied your Ceph Configuration to the OSD Cluster hosts, you may deploy Ceph with the ``mkcephfs`` script.
+
+.. note:: ``mkcephfs`` is a quick bootstrapping tool. It does not handle more complex operations, such as upgrades.
+
+ For production environments, you will deploy Ceph using Chef cookbooks (coming soon!).
+
+To run ``mkcephfs``, execute the following::
+
+ $ mkcephfs -a -c <path>/ceph.conf -k mycluster.keyring
+
+The script adds an admin key to the ``mycluster.keyring``, which is analogous to a root password. Ceph should begin operating.
+You can check on the health of your Ceph cluster with the following::
+
+ ceph -k mycluster.keyring -c mycluster.conf health
+
===================
Deploying with Chef
===================
+
+Coming Soon!
\ No newline at end of file
==========================
Creating a Storage Cluster
==========================
+Ceph can run with a cluster containing thousands of Object Storage Devices (OSDs).
+A minimal system will have at least two OSDs for data replication. To configure OSD clusters, you must
+provide settings in the configuration file. Ceph provides default values for many settings, which you can
+override in the configuration file. Additionally, you can make runtime modification to the configuration
+using command-line utilities.
+When Ceph starts, it activates three daemons:
+- ``ceph-osd`` (mandatory)
+- ``ceph-mon`` (mandatory)
+- ``ceph-mds`` (mandatory for cephfs only)
+
+Each process, daemon or utility loads the host's configuration file. A process may have information about
+more than one daemon instance (*i.e.,* multiple contexts). A daemon or utility only has information
+about a single daemon instance (a single context).
+
+.. note:: Ceph can run on a single host for evaluation purposes.
+
+- :doc:`Ceph Configuration Files <ceph_conf>`
+ - :doc:`OSD Configuration Settings <osd_configuration_settings>`
+ - :doc:`Monitor Configuration Settings <mon_configuration_settings>`
+ - :doc:`Metadata Server Configuration Settings <mds_configuration_settings>`
+- :doc:`Deploying the Ceph Configuration <deploying_ceph_conf>`
+- :doc:`Deploying Ceph with mkcephfs <deploying_ceph_with_mkcephfs>`
+- :doc:`Deploying Ceph with Chef (coming soon) <deploying_with_chef>`
.. toctree::
:hidden:
-
- building_ceph_conf
- running_mkcephfs
- deploying_with_chef
\ No newline at end of file
+ Configuration <ceph_conf>
+ [osd] Settings <osd_configuration_settings>
+ [mon] Settings <mon_configuration_settings>
+ [mds] Settings <mds_configuration_settings>
+ Deploy Config <deploying_ceph_conf>
+ deploying_ceph_with_mkcephfs
+ Chef Coming Soon! <deploying_with_chef>
\ No newline at end of file
--- /dev/null
+======================================
+Metadata Server Configuration Settings
+======================================
+
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| Setting | Type | Default | Description |
++===================================+=========================+============+================================================+
+| ``mds_max_file_size`` | 64-bit Integer Unsigned | 1ULL << 40 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_cache_size`` | 32-bit Integer | 100000 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_cache_mid`` | Float | .7 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_mem_max`` | 32-bit Integer | 1048576 | // KB |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_dir_commit_ratio`` | Float | .5 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_dir_max_commit_size`` | 32-bit Integer | 90 | // MB |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_decay_halflife`` | Float | 5 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_beacon_interval`` | Float | 4 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_beacon_grace`` | Float | 15 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_blacklist_interval`` | Float | 24.0*60.0 | // how long to blacklist failed nodes |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_session_timeout`` | Float | 60 | // cap bits and leases time out if client idle |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_session_autoclose`` | Float | 300 | // autoclose idle session |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_reconnect_timeout`` | Float | 45 | // secs to wait for clients during mds restart |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_tick_interval`` | Float | 5 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_dirstat_min_interval`` | Float | 1 | //try to avoid propagating more often than x |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_scatter_nudge_interval`` | Float | 5 | // how quickly dirstat changes propagate up |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_client_prealloc_inos`` | 32-bit Integer | 1000 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_early_reply`` | Boolean | true | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_use_tmap`` | Boolean | true | // use trivialmap for dir updates |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_default_dir_hash`` | 32-bit Integer | | CEPH_STR_HASH_RJENKINS |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_log`` | Boolean | true | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_log_skip_corrupt_events`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_log_max_events`` | 32-bit Integer | -1 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_log_max_segments`` | 32-bit Integer | 30 | // segment size defined by FileLayout above |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_log_max_expiring`` | 32-bit Integer | 20 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_log_eopen_size`` | 32-bit Integer | 100 | // # open inodes per log entry |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_sample_interval`` | Float | 3.0 | // every 5 seconds |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_replicate_threshold`` | Float | 8000 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_unreplicate_threshold`` | Float | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_frag`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_split_size`` | 32-bit Integer | 10000 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_split_rd`` | Float | 25000 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_split_wr`` | Float | 10000 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_split_bits`` | 32-bit Integer | 3 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_merge_size`` | 32-bit Integer | 50 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_merge_rd`` | Float | 1000 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_merge_wr`` | Float | 1000 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_interval`` | 32-bit Integer | 10 | // seconds |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_fragment_interval`` | 32-bit Integer | 5 | // seconds |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_idle_threshold`` | Float | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_max`` | 32-bit Integer | -1 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_max_until`` | 32-bit Integer | -1 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_mode`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_min_rebalance`` | Float | .1 | // must be x above avg before we export |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_min_start`` | Float | .2 | // if we need less x. we don't do anything |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_need_min`` | Float | .8 | // take within this range of what we need |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_need_max`` | Float | 1.2 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_midchunk`` | Float | .3 | // any sub bigger than this taken in full |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_minchunk`` | Float | .001 | // never take anything smaller than this |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_target_removal_min`` | 32-bit Integer | 5 | // min bal iters before old target is removed |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_bal_target_removal_max`` | 32-bit Integer | 10 | // max bal iters before old target is removed |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_replay_interval`` | Float | 1.0 | // time to wait before starting replay again |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_shutdown_check`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_thrash_exports`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_thrash_fragments`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_dump_cache_on_map`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_dump_cache_after_rejoin`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_verify_scatter`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_debug_scatterstat`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_debug_frag`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_debug_auth_pins`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_debug_subtrees`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_kill_mdstable_at`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_kill_export_at`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_kill_import_at`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_kill_link_at`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_kill_rename_at`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_wipe_sessions`` | Boolean | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_wipe_ino_prealloc`` | Boolean | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_skip_ino`` | 32-bit Integer | 0 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``max_mds`` | 32-bit Integer | 1 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_standby_for_name`` | String | "" | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_standby_for_rank`` | 32-bit Integer | -1 | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+| ``mds_standby_replay`` | Boolean | false | |
++-----------------------------------+-------------------------+------------+------------------------------------------------+
+
+
+// make it (mds_session_timeout - mds_beacon_grace |
\ No newline at end of file
--- /dev/null
+==============================
+Monitor Configuration Settings
+==============================
+
+
+
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| Setting | Type | Default Value | Description |
++===================================+================+===============+===========================================================+
+| ``mon data`` | String | "" | |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon sync fs threshold`` | 32-bit Integer | 5 | // sync when writing this many objects; 0 to disable. |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon tick interval`` | 32-bit Integer | 5 | |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon subscribe interval`` | Double | 300 | |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon osd auto mark in`` | Boolean | false | // mark any booting osds 'in' |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon osd auto mark auto out in`` | Boolean | true | // mark booting auto-marked-out osds 'in' |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon osd auto mark new in`` | Boolean | true | // mark booting new osds 'in' |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon osd down out interval`` | 32-bit Integer | 300 | // seconds |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon lease`` | Float | 5 | // lease interval |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon lease renew interval`` | Float | 3 | // on leader | to renew the lease |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon lease ack timeout`` | Float | 10.0 | // on leader | if lease isn't acked by all peons |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon clock drift allowed`` | Float | .050 | // allowed clock drift between monitors |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon clock drift warn backoff`` | Float | 5 | // exponential backoff for clock drift warnings |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon accept timeout`` | Float | 10.0 | // on leader | if paxos update isn't accepted |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon pg create interval`` | Float | 30.0 | // no more than every 30s |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon pg stuck threshold`` | 32-bit Integer | 300 | // number of seconds after which pgs can be considered |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon osd full ratio`` | Float | .95 | // what % full makes an OSD "full" |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon osd nearfull ratio`` | Float | .85 | // what % full makes an OSD near full |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon globalid prealloc`` | 32-bit Integer | 100 | // how many globalids to prealloc |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon osd report timeout`` | 32-bit Integer | 900 | // grace period before declaring unresponsive OSDs dead |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon force standby active`` | Boolean | true | // should mons force standby-replay mds to be active |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon min osdmap epochs`` | 32-bit Integer | 500 | |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon max pgmap epochs`` | 32-bit Integer | 500 | |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon max log epochs`` | 32-bit Integer | 500 | |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon probe timeout`` | Double | 2.0 | |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+| ``mon slurp timeout`` | Double | 10.0 | |
++-----------------------------------+----------------+---------------+-----------------------------------------------------------+
+
+inactive | unclean | or stale (see doc/control.rst under dump stuck for more info |
\ No newline at end of file
--- /dev/null
+==========================
+osd Configuration Settings
+==========================
+
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| Setting | Type | Default Value | Description |
++=========================================+=====================+=======================+================================================+
+| ``osd auto upgrade tmap`` | Boolean | True | Uses ``tmap`` for ``omap`` on old objects. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd tmapput sets users tmap`` | Boolean | False | Uses ``tmap`` for debugging only. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd data`` | String | None | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd journal`` | String | None | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd journal size`` | 32-bit Int | 0 | The size of the journal in MBs. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd max write size`` | 32-bit Int | 90 | The size of the maximum x to write in MBs. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd balance reads`` | Boolean | False | Load balance reads? |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd shed reads`` | 32-bit Int | False (0) | Forward from primary to replica. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd shed reads min latency`` | Double | .01 | The minimum local latency. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd shed reads min latency diff`` | Double | 1.5 | Percentage difference from peer. 150% default. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd client message size cap`` | 64-bit Int Unsigned | 500*1024L*1024L | Client data allowed in-memory. 200MB default. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd stat refresh interval`` | 64-bit Int Unsigned | .5 | The status refresh interval in seconds. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd pg bits`` | 32-bit Int | 6 | Placement group bits per OSD. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd pgp bits`` | 32-bit Int | 4 | Placement group p bits per OSD? |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd pg layout`` | 32-bit Int | 2 | Placement Group bits ? per OSD? |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd min rep`` | 32-bit Int | 1 | Need a description. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd max rep`` | 32-bit Int | 10 | Need a description. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd min raid width`` | 32-bit Int | 3 | The minimum RAID width. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd max raid width`` | 32-bit Int | 2 | The maximum RAID width. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd pool default crush rule`` | 32-bit Int | 0 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd pool default size`` | 32-bit Int | 2 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd pool default pg num`` | 32-bit Int | 8 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd pool default pgp num`` | 32-bit Int | 8 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd map cache max`` | 32-bit Int | 250 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd map message max`` | 32-bit Int | 100 | // max maps per MOSDMap message |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd op threads`` | 32-bit Int | 2 | // 0 == no threading |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd disk threads`` | 32-bit Int | 1 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd recovery threads`` | 32-bit Int | 1 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd recover clone overlap`` | Boolean | false | // preserve clone overlap during rvry/migrat |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd backfill scan min`` | 32-bit Int | 64 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd backfill scan max`` | 32-bit Int | 512 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd op thread timeout`` | 32-bit Int | 30 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd backlog thread timeout`` | 32-bit Int | 60*60*1 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd recovery thread timeout`` | 32-bit Int | 30 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd snap trim thread timeout`` | 32-bit Int | 60*60*1 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd scrub thread timeout`` | 32-bit Int | 60 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd scrub finalize thread timeout`` | 32-bit Int | 60*10 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd remove thread timeout`` | 32-bit Int | 60*60 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd command thread timeout`` | 32-bit Int | 10*60 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd age`` | Float | .8 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd age time`` | 32-bit Int | 0 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd heartbeat interval`` | 32-bit Int | 1 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd mon heartbeat interval`` | 32-bit Int | 30 | // if no peers | ping monitor |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd heartbeat grace`` | 32-bit Int | 20 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd mon report interval max`` | 32-bit Int | 120 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd mon report interval min`` | 32-bit Int | 5 | // pg stats | failures | up thru | boot. |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd mon ack timeout`` | 32-bit Int | 30 | // time out a mon if it doesn't ack stats |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd min down reporters`` | 32-bit Int | 1 | // num OSDs needed to report a down OSD |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd min down reports`` | 32-bit Int | 3 | // num times a down OSD must be reported |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd default data pool replay window`` | 32-bit Int | 45 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd preserve trimmed log`` | Boolean | true | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd auto mark unfound lost`` | Boolean | false | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd recovery delay start`` | Float | 15 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd recovery max active`` | 32-bit Int | 5 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd recovery max chunk`` | 64-bit Int Unsigned | 1<<20 | // max size of push chunk |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd recovery forget lost objects`` | Boolean | false | // off for now |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd max scrubs`` | 32-bit Int | 1 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd scrub load threshold`` | Float | 0.5 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd scrub min interval`` | Float | 300 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd scrub max interval`` | Float | 60*60*24 | // once a day |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd auto weight`` | Boolean | false | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd class error timeout`` | Double | 60.0 | // seconds |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd class timeout`` | Double | 60*60.0 | // seconds |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd class dir`` | String | "/rados-classes" | CEPH LIBDIR |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd check for log corruption`` | Boolean | false | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd use stale snap`` | Boolean | false | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd rollback to cluster snap`` | String | "" | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd default notify timeout`` | 32-bit Int Unsigned | 30 | // default notify timeout in seconds |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd kill backfill at`` | 32-bit Int | 0 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd min pg log entries`` | 32-bit Int Unsigned | 1000 | // num entries to keep in pg log when trimming |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd op complaint time`` | Float | 30 | // how old in secs makes op complaint-worthy |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
+| ``osd command max records`` | 32-bit Int | 256 | |
++-----------------------------------------+---------------------+-----------------------+------------------------------------------------+
\ No newline at end of file
+++ /dev/null
-====================
-Running ``mkcephfs``
-====================
Build Ceph Packages
===================
-You can create installation packages from the latest code using ``dpkg-buildpackage`` for Debian/Ubuntu or ``rpmbuild`` for the RPM Package Manager.
+To build packages, you must clone the `Ceph`_ repository.
+You can create installation packages from the latest code using ``dpkg-buildpackage`` for Debian/Ubuntu
+or ``rpmbuild`` for the RPM Package Manager.
-Debian
-------
+.. tip:: When building on a multi-core CPU, use the ``-j`` and the number of cores * 2.
+ For example, use ``-j4`` for a dual-core processor to accelerate the build.
-To create ``.deb`` packages, ensure that you have installed the `build prerequisites <build_prerequisites>`_ and install ``debhelper``. The requirements for
-``debhelper`` include:
-- ``debhelper``
+Advanced Package Tool (APT)
+---------------------------
+To create ``.deb`` packages for Debian/Ubuntu, ensure that you have cloned the `Ceph`_ repository,
+installed the `build prerequisites`_ and installed ``debhelper``::
+
+ $ sudo apt-get install debhelper
+
+Once you have installed debhelper, you can build the packages:
+
+ $ sudo dpkg-buildpackage
+
+For multi-processor CPUs use the ``-j`` option to accelerate the build.
+
+RPM Package Manager
+-------------------
+
+To create ``.prm`` packages, ensure that you have cloned the `Ceph`_ repository,
+installed the `build prerequisites`_ and installed ``rpm-build`` and ``rpmdevtools``::
+
+ $ yum install rpm-build rpmdevtools
+
+Once you have installed the tools, setup an RPM compilation environment::
+
+ $ rpmdev-setuptree
+
+Fetch the source tarball for the RPM compilation environment::
+
+ $ wget -P ~/rpmbuild/SOURCES/ http://ceph.newdream.net/download/ceph-<version>.tar.gz
+
+Build the RPM packages::
+
+ $ rpmbuild -tb ~/rpmbuild/SOURCES/ceph-<version>.tar.gz
+
+For multi-processor CPUs use the ``-j`` option to accelerate the build.
+
+
+.. _build prerequisites: ../build_prerequisites
+.. _Ceph: ../cloning_the_ceph_source_code_repository
\ No newline at end of file
On Ubuntu, execute ``sudo apt-get install`` for each dependency that isn't installed on your host. ::
- $ sudo apt-get install autotools-dev autoconf automake cdbs
- gcc g++ git libboost-dev libedit-dev libssl-dev libtool
- libfcgi libfcgi-dev libfuse-dev linux-kernel-headers
- libcrypto++-dev libcrypto++ libexpat1-dev libgtkmm-2.4-dev
+ $ sudo apt-get install autotools-dev autoconf automake cdbs gcc g++ git libboost-dev libedit-dev libssl-dev libtool libfcgi libfcgi-dev libfuse-dev linux-kernel-headers libcrypto++-dev libcrypto++ libexpat1-dev libgtkmm-2.4-dev
On Debian/Squeeze, execute ``aptitude install`` for each dependency that isn't installed on your host. ::
- $ aptitude install autotools-dev autoconf automake cdbs
- gcc g++ git libboost-dev libedit-dev libssl-dev libtool
- libfcgi libfcgi-dev libfuse-dev linux-kernel-headers
- libcrypto++-dev libcrypto++ libexpat1-dev libgtkmm-2.4-dev
+ $ aptitude install autotools-dev autoconf automake cdbs gcc g++ git libboost-dev libedit-dev libssl-dev libtool libfcgi libfcgi-dev libfuse-dev linux-kernel-headers libcrypto++-dev libcrypto++ libexpat1-dev libgtkmm-2.4-dev
Ubuntu Requirements
Execute ``sudo apt-get install`` for each dependency that isn't installed on your host. ::
- $ sudo apt-get install uuid-dev libkeytutils-dev libgoogle-perftools-dev
- libatomic-ops-dev libaio-dev libgdata-common libgdata13
+ $ sudo apt-get install uuid-dev libkeytutils-dev libgoogle-perftools-dev libatomic-ops-dev libaio-dev libgdata-common libgdata13
Debian
------
Ceph utilizes Python's Sphinx documentation tool. For details on
the Sphinx documentation tool, refer to: `Sphinx <http://sphinx.pocoo.org>`_
Follow the directions at `Sphinx 1.1.3 <http://pypi.python.org/pypi/Sphinx>`_
-to install Sphinx. To run Sphinx, with `admin/build-doc`, at least the following are required:
+to install Sphinx. To run Sphinx, with ``admin/build-doc``, at least the following are required:
- ``python-dev``
- ``python-pip``
To install Ceph locally, you may also use::
$ make install
+
+If you install Ceph locally, ``make`` will place the executables in ``usr/local/bin``.
+You may add the ``ceph.conf`` file to the ``usr/local/bin`` directory to run an evaluation environment of Ceph from a single directory.
Building Ceph Documentation
===========================
- ``master``: The unstable development branch.
- ``stable``: The bugfix branch.
- ``next``: The release candidate branch.
+
+::
+
+ git checkout master
+
==========================
Downloading a Ceph Release
==========================
-As Ceph development progresses, the Ceph team releases new versions. You may download Ceph releases here:
+As Ceph development progresses, the Ceph team releases new versions of the source code. You may download source code for Ceph releases here:
`Ceph Releases <http://ceph.newdream.net/download/>`_
\ No newline at end of file
-====================
-Downloading Packages
-====================
+==================================
+Downloading Debian/Ubuntu Packages
+==================================
+We automatically build Debian/Ubuntu packages for any branches or tags that appear in
+the ``ceph.git`` `repository <http://github.com/ceph/ceph>`_. If you want to build your own packages
+(e.g., for RPM), see `Build Ceph Packages <../build_from_source/build_packages>`_.
-We automatically build Debian and Ubuntu packages for any branches or tags that appear in
-the ``ceph.git`` `repository <http://github.com/ceph/ceph>`_. We build packages for the following
-architectures:
+When you download release packages, you will receive the latest package build, which may be several weeks behind the current release
+or the most recent code. It may contain bugs that have already been fixed in the most recent versions of the code. Until packages
+contain only stable code, you should carefully consider the tradeoffs of installing from a package or retrieving the latest release
+or the most current source code and building Ceph.
-- ``amd64``
-- ``i386``
+When you execute the following commands to install the Debian/Ubuntu Ceph packages, replace ``{ARCH}`` with the
+architecture of your CPU (e.g., ``amd64`` or ``i386``), ``{DISTRO}`` with the code name of your operating system
+(e.g., ``precise``, rather than the OS version number) and ``{BRANCH}`` with the version of Ceph you want to
+run (e.g., ``master``, ``stable``, ``unstable``, ``v0.44``, etc.).
-For each architecture, we build packages for the following distributions:
+Adding Release Packages to APT
+------------------------------
+We provide stable release packages for Debian/Ubuntu, which are signed signed with the ``release.asc`` key.
+Click `here <http://ceph.newdream.net/debian/dists>`_ to see the distributions and branches supported.
+To install a release package, you must first add a release key. ::
-- Debian 7.0 (``wheezy``)
-- Debian 6.0 (``squeeze``)
-- Debian unstable (``sid``)
-- Ubuntu 12.04 (``precise``)
-- Ubuntu 11.10 (``oneiric``)
-- Ubuntu 11.04 (``natty``)
-- Ubuntu 10.10 (``maverick``)
+ $ wget -q -O- https://raw.github.com/ceph/ceph/master/keys/release.asc \ | sudo apt-key add -
-When you execute the following commands to install the Ceph packages, replace ``{ARCH}`` with the architecture of your CPU,
-``{DISTRO}`` with the code name of your operating system (e.g., ``wheezy``, rather than the version number) and
-``{BRANCH}`` with the version of Ceph you want to run (e.g., ``master``, ``stable``, ``unstable``, ``v0.44``, etc.). ::
+For Debian/Ubuntu releases, we use the Advanced Package Tool (APT). To retrieve the release packages and updates
+and install them with ``apt``, you must add a ``ceph.list`` file to your ``apt`` configuration with the following
+path::
- wget -q -O- https://raw.github.com/ceph/ceph/master/keys/autobuild.asc \
- | sudo apt-key add -
+ etc/apt/sources.list.d/ceph.list
+
+Open the file and add the following line::
- sudo tee /etc/apt/sources.list.d/ceph.list <<EOF
- deb http://ceph.newdream.net/debian-snapshot-{ARCH}/{BRANCH}/ {DISTRO} main
- deb-src http://ceph.newdream.net/debian-snapshot-{ARCH}/{BRANCH}/ {DISTRO} main
- EOF
+ deb http://ceph.newdream.net/debian/{BRANCH}/ {DISTRO} main
- sudo apt-get update
- sudo apt-get install ceph
+Remember to replace ``{BRANCH}`` with the branch you want to use and replace ``{DISTRO}`` with the Linux distribution for your host. Then,
+save the file.
+Adding Autobuild Packages to APT
+--------------------------------
+We provide unstable release packages for Debian/Ubuntu, which contain the latest code and bug fixes.
+The autobuild packages are signed signed with the ``autobuild.asc`` key. To install an autobuild package,
+you must first add an autobuild key::
+
+ wget -q -O- https://raw.github.com/ceph/ceph/master/keys/autobuild.asc \ | sudo apt-key add -
+
+For Debian/Ubuntu releases, we use the Advanced Package Tool (APT). To retrieve the autobuild packages and updates
+and install them with ``apt``, you must add a ``ceph.list`` file to your ``apt`` configuration with the following
+path::
+
+ etc/apt/sources.list.d/ceph.list
+
+Open the file and add the following lines::
+
+ deb http://ceph.newdream.net/debian-snapshot-amd64/{BRANCH}/ {DISTRO} main
+ deb-src http://ceph.newdream.net/debian-snapshot-amd64/{BRANCH}/ {DISTRO} main
+
+Remember to replace ``{BRANCH}`` with the branch you want to use and replace ``{DISTRO}`` with the Linux distribution for your host. Then,
+save the file.
+
+Downloading Packages
+--------------------
+Once you add either release or autobuild packages for Debian/Ubuntu, you may download them with ``apt`` as follows::
+
+ sudo apt-get update
-When you download packages, you will receive the latest package build, which may be several weeks behind the current release
-or the most recent code. It may contain bugs that have already been fixed in the most recent versions of the code. Until packages
-contain only stable code, you should carefully consider the tradeoffs of installing from a package or retrieving the latest release
-or the most current source code and building Ceph.
\ No newline at end of file
-===========================
-File System Recommendations
-===========================
-Ceph OSDs depend on the Extended Attributes (XATTRS) of the underlying file system for:
+=========================================
+Hard Disk and File System Recommendations
+=========================================
+
+.. important:: Disable disk caching and asynchronous write.
+
+Ceph aims for data safety, which means that when the application receives notice that data was
+written to the disk, that data was actually written to the disk and not still in a transient state in
+a buffer or cache pending a lazy write to the hard disk.
+
+For data safety, you should mount your file system with caching disabled. Your file system should be
+mounted with ``sync`` and NOT ``async``. For example, your ``fstab`` file would reflect ``sync``. ::
+
+ /dev/hda / xfs sync 0 0
+
+Use ``hdparm`` to disable write caching on the hard disk::
+
+ $ hdparm -W 0 /dev/hda 0
+
+
+Ceph OSDs depend on the Extended Attributes (XATTRs) of the underlying file system for:
- Internal object state
- Snapshot metadata
- RADOS Gateway Access Control Lists (ACLs).
Ceph OSDs rely heavily upon the stability and performance of the underlying file system. The
-underlying file system must provide sufficient capacity for XATTRS. File system candidates for
+underlying file system must provide sufficient capacity for XATTRs. File system candidates for
Ceph include B tree and B+ tree file systems such as:
- ``btrfs``
The RADOS Gateway's ACL and Ceph snapshots easily surpass the 4-kilobyte limit for XATTRs in ``ext4``,
causing the ``ceph-osd`` process to crash. So ``ext4`` is a poor file system choice if
- you intend to deploy the RADOS Gateway or use snapshots.
+ you intend to deploy the RADOS Gateway or use snapshots. Version 0.45 or newer uses ``leveldb`` to
+ bypass this limitation.
-.. tip:: Use `btrfs`
+.. tip:: Use ``xfs`` initially and ``btrfs`` when it is ready for production.
The Ceph team believes that the best performance and stability will come from ``btrfs.``
The ``btrfs`` file system has internal transactions that keep the local data set in a consistent state.
This makes OSDs based on ``btrfs`` simple to deploy, while providing scalability not
currently available from block-based file systems. The 64-kb XATTR limit for ``xfs``
XATTRS is enough to accommodate RDB snapshot metadata and RADOS Gateway ACLs. So ``xfs`` is the second-choice
- file system of the Ceph team. If you only plan to use RADOS and ``rbd`` without snapshots and without
- ``radosgw``, the ``ext4`` file system should work just fine.
+ file system of the Ceph team in the long run, but ``xfs`` is currently more stable than ``btrfs``.
+ If you only plan to use RADOS and ``rbd`` without snapshots and without ``radosgw``, the ``ext4``
+ file system should work just fine.
+
--- /dev/null
+====================
+Host Recommendations
+====================
+We recommend using a dedicated Administration host for larger deployments, particularly when you intend to build Ceph from source
+or build your own packages.
+
+.. important:: The Administration host must have ``root`` access to OSD Cluster hosts for installation and maintenance.
+
+If you are installing Ceph on a single host for the first time to learn more about it, your local host is effectively your Administration host.
+
+Enable Extended Attributes (XATTRs)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+If you are using a file system other than ``XFS`` or ``btrfs``, you need to enable extended attributes. ::
+
+ mount -t ext4 -o user_xattr /dev/hda mount/mount_point
+
+Install Build Prerequisites
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Install `build prerequisites`_ on your Administration host machine. If you want to
+build install packages on each OSD Cluster host, install the `build prerequisites`_
+on each OSD Cluster host.
+
+Configure SSH
+~~~~~~~~~~~~~
+Before you can install and configure Ceph on an OSD Cluster host you need to configure SSH on
+the Administration host and on each OSD Cluster host.
+
+You must be able to login via SSH from your Administration host to each OSD Cluster host
+as ``root`` using the short name to identify the OSD Cluster host. Your Administration host must be able
+to connect to each OSD Cluster host using the OSD Cluster host's short name, not its full domain name (e.g., ``shortname``
+not ``shortname.domain.com``).
+
+To connect to an OSD Cluster host from your Administration host using the OSD Cluster short name only,
+add a host configuration for the OSD Cluster host to your ``~/.ssh_config file``. You must add an entry
+for each host in your cluster. Set the user name to ``root`` or a username with root privileges. For example::
+
+ Host shortname1
+ Hostname shortname1.domain.com
+ User root
+ Host shortname2
+ Hostname shortname2.domain.com
+ User root
+
+You must be able to use ``sudo`` over SSH from the Administration host on each OSD Cluster host
+without ``sudo`` prompting you for a password.
+
+If you have a public key for your root password in ``.ssh/id_rsa.pub``, you must copy the key and append it
+to the contents of the ``.ssh/authorized_keys`` file on each OSD Cluster host. Create the ``.ssh/authorized_keys``
+file if it doesn't exist. When you open an SSH connection from the Administration host to an OSD Cluster host,
+SSH uses the private key in the home directory of the Administration host and tries to match it with a public
+key in the home directory of the OSD Cluster host. If the SSH keys match, SSH will log you in automatically.
+If the SSH keys do not match, SSH will prompt you for a password.
+
+To generate an SSH key pair on your Administration host, log in with ``root`` permissions, go to your ``/home`` directory and enter the following::
+
+ $ ssh-keygen
+
+Add Packages to OSD Cluster Hosts
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Add the packages you downloaded or built on the Administration host to each OSD Cluster host. Perform the same steps
+from `Installing Debian/Ubuntu Packages`_ for each OSD Cluster host. To expedite adding
+the ``etc/apt/sources.list.d/ceph.list`` file to each OSD Cluster host, consider using ``tee``.
+::
+
+ $ sudo tee /etc/apt/sources.list.d/ceph.list <<EOF
+
+A prompt will appear, and you can add lines to the ``ceph.list`` file. For release packages, enter::
+
+ > deb http://ceph.newdream.net/debian/{BRANCH}/ {DISTRO} main
+
+For snapshot packages, enter::
+
+ > deb http://ceph.newdream.net/debian-snapshot-amd64/{BRANCH}/ {DISTRO} main
+ > deb-src http://ceph.newdream.net/debian-snapshot-amd64/{BRANCH}/ {DISTRO} main
+
+For packages you built on your Administration host, consider making them accessible via HTTP, and enter::
+
+ > deb http://{adminhostname}.domainname.com/{package directory}
+
+Once you have added the package directories, close the file. ::
+
+ > EOF
+
+
+.. _build prerequisites: ../build_from_source/build_prerequisites
+.. _Installing Debian/Ubuntu Packages: ../download_packages
\ No newline at end of file
1. :doc:`Hardware Recommendations <hardware_recommendations>`
2. :doc:`File System Recommendations <file_system_recommendations>`
-3. :doc:`Download Ceph Packages <download_packages>`
-4. :doc:`Building Ceph from Source <building_ceph_from_source>`
-
+3. :doc:`Host Recommendations <host_recommendations>`
+4. :doc:`Download Ceph Packages <download_packages>`
+5. :doc:`Building Ceph from Source <building_ceph_from_source>`
+6. :doc:`Installing Packages <installing_packages>`
.. toctree::
:hidden:
Hardware Recs <hardware_recommendations>
File System Recs <file_system_recommendations>
+ Host Recs <host_recommendations>
Download Packages <download_packages>
Build From Source <building_ceph_from_source>
+ Install Packages <installing_packages>
--- /dev/null
+========================
+Installing Ceph Packages
+========================
+Once you have downloaded or built Ceph packages, you may install them on your Admin host and OSD Cluster hosts.
+
+
+.. important:: All hosts should be running the same package version.
+ To ensure that you are running the same version on each host with APT, you may
+ execute ``sudo apt-get update`` on each host before you install the packages.
+
+
+Installing Packages with APT
+----------------------------
+Once you download or build the packages and add your packages to APT
+(see `Downloading Debian/Ubuntu Packages <../download_packages>`_), you may install them as follows::
+
+ $ sudo apt-get install ceph
+
+
+Installing Packages with RPM
+----------------------------
+Once you have built your RPM packages, you may install them as follows::
+
+ rpm -i rpmbuild/RPMS/x86_64/ceph-*.rpm
+
+
+Proceed to Creating a Cluster
+-----------------------------
+Once you have prepared your hosts and installed Ceph pages, proceed to `Creating a Storage Cluster <../../create_cluster>`_.
\ No newline at end of file
Quick Start
===========
-Ceph is intended for large-scale deployments, but you may install Ceph on a single host. Quick start is intended for Ubuntu Linux distributions.
-
-
-Install from a Package
-----------------------
-
-1. Login to your server
-2. Make a directory for ceph packages
-3.
-
-
-
-
-
-
-Install from Source
--------------------
-
-
-
-
-
+Ceph is intended for large-scale deployments, but you may install Ceph on a single host. Quick start is intended for Debian/Ubuntu Linux distributions.
+
+1. Login to your host.
+2. Make a directory for Ceph packages. *e.g.,* ``$ mkdir ceph``
+3. `Get Ceph packages <../../install/download_packages>`_ and add them to your APT configuration file.
+4. Update and Install Ceph packages. See `Downloading Debian/Ubuntu Packages <../../install/download_packages>`_ and `Installing Packages <../../install/installing_packages>`_ for details.
+5. Add a ``ceph.conf`` file. See `Ceph Configuration Files <../../create_cluster/ceph_conf>`_ for details.
+6. Run Ceph. See `Deploying Ceph with mkcephfs <../../create_cluster/deploying_ceph_with_mkcephfs>`_
\ No newline at end of file
projects / ceph.git / commitdiff