From: John Wilkins Date: Thu, 7 Aug 2014 22:29:34 +0000 (-0700) Subject: doc: Incorporated user feedback to clarify upgrade doc. X-Git-Tag: v0.85~75 X-Git-Url: http://git.apps.os.sepia.ceph.com/?a=commitdiff_plain;h=8357cae0272393a7a14be073240012d8e6da2e9d;p=ceph.git doc: Incorporated user feedback to clarify upgrade doc. Signed-off-by: John Wilkins --- diff --git a/doc/install/upgrading-ceph.rst b/doc/install/upgrading-ceph.rst index 4790a52976234..a4475f7a47eaf 100644 --- a/doc/install/upgrading-ceph.rst +++ b/doc/install/upgrading-ceph.rst @@ -2,34 +2,70 @@ Upgrading Ceph ================ -You can upgrade daemons in your Ceph cluster while the cluster is -online and in service! The upgrade process is relatively simple: +Each release of Ceph may have additional steps. Refer to the release-specific +sections in this document and the `release notes`_ document to identify +release-specific procedures for your cluster before using the upgrade +procedures. -#. Use ``ceph-deploy`` to upgrade the packages for multiple hosts, - or login to each host and upgrade the Ceph package manually. -#. Restart each daemon. -#. Ensure your cluster is healthy. -.. important:: Once you upgrade a daemon, you cannot downgrade it. +Summary +======= -Certain types of daemons depend upon others. For example, Ceph Metadata Servers -and Ceph Object Gateways depend upon Ceph Monitors and Ceph OSD Daemons. We -recommend upgrading daemons in this order: +You can upgrade daemons in your Ceph cluster while the cluster is online and in +service! Certain types of daemons depend upon others. For example, Ceph Metadata +Servers and Ceph Object Gateways depend upon Ceph Monitors and Ceph OSD Daemons. +We recommend upgrading in this order: -#. Ceph Monitors (or Ceph OSD Daemons) -#. Ceph OSD Daemons (or Ceph Monitors) +#. `Ceph Deploy`_ +#. Ceph Monitors +#. Ceph OSD Daemons #. Ceph Metadata Servers #. Ceph Object Gateways As a general rule, we recommend upgrading all the daemons of a specific type -(e.g., all ``ceph-osd`` daemons, all ``ceph-mon`` daemons, etc.) to ensure that +(e.g., all ``ceph-mon`` daemons, all ``ceph-osd`` daemons, etc.) to ensure that they are all on the same release. We also recommend that you upgrade all the daemons in your cluster before you try to exercise new functionality in a release. -Each release of Ceph may have some additional steps. Refer to the following -sections to identify release-specific procedures for your cluster before -using the upgrade procedures. +The `Upgrade Procedures`_ are relatively simple, but please look at +distribution-specific sections before upgrading. The basic process involves +three steps: + +#. Use ``ceph-deploy`` on your admin node to upgrade the packages for + multiple hosts (using the ``ceph-deploy install`` command), or login to each + host and upgrade the Ceph package `manually`_. For example, when + `Upgrading Monitors`_, the ``ceph-deploy`` syntax might look like this:: + + ceph-deploy install --release {release-name} ceph-node1[ ceph-node2] + ceph-deploy install --release firefly mon1 mon2 mon3 + + **Note:** The ``ceph-deploy install`` command will upgrade the packages + in the specified node(s) from the old release to the release you specify. + There is no ``ceph-deploy upgrade`` command. + +#. Login in to each Ceph node and restart each Ceph daemon. + See `Operating a Cluster`_ for details. + +#. Ensure your cluster is healthy. See `Monitoring a Cluster`_ for details. + +.. important:: Once you upgrade a daemon, you cannot downgrade it. + + +Ceph Deploy +=========== + +Before upgrading Ceph daemons, upgrade the ``ceph-deploy`` tool. :: + + sudo pip install -U ceph-deploy + +Or:: + + sudo apt-get install ceph-deploy + +Or:: + + sudo yum install ceph-deploy python-pushy Argonaut to Bobtail @@ -306,22 +342,6 @@ See `v0.65`_ for details on the new command line interface. .. _v0.65: http://ceph.com/docs/master/release-notes/#v0-65 -Ceph Deploy ------------ - -Before upgrading Ceph daemons, upgrade the ``ceph-deploy`` tool. :: - - sudo pip install -U ceph-deploy - -Or:: - - sudo apt-get install ceph-deploy - -Or:: - - sudo yum install ceph-deploy python-pushy - - Monitor ------- @@ -335,6 +355,85 @@ succession) to minimize the possibility of downtime. .. important:: Do not run a mixed-version cluster for an extended period. + +Emperor to Firefly +================== + +If your existing cluster is running a version older than v0.67 Dumpling, please +first upgrade to the latest Dumpling release before upgrading to v0.80 Firefly. +Please refer to `Cuttlefish to Dumpling`_ and the `Firefly release notes`_ for +details. To upgrade from a post-Emperor point release, see the `Firefly release +notes`_ for details. + + +Ceph Config File Changes +------------------------ + +We recommand adding the following to the ``[mon]`` section of your +``ceph.conf`` prior to upgrade:: + + mon warn on legacy crush tunables = false + +This will prevent health warnings due to the use of legacy CRUSH placement. +Although it is possible to rebalance existing data across your cluster, we do +not normally recommend it for production environments as a large amount of data +will move and there is a significant performance impact from the rebalancing. + + +Upgrade Sequence +---------------- + +Replace any reference to older repositories with a reference to the +Firefly repository. For example, with ``apt`` perform the following:: + + sudo rm /etc/apt/sources.list.d/ceph.list + echo deb http://ceph.com/debian-firefly/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list + +With CentOS/Red Hat distributions, remove the old repository. :: + + sudo rm /etc/yum.repos.d/ceph.repo + +Then add a new ``ceph.repo`` repository entry with the following contents, but +replace ``{distro}`` with your distribution (e.g., ``el6``, ``rhel6``, +``rhel7``, etc.). + +.. code-block:: ini + + [ceph] + name=Ceph Packages and Backports $basearch + baseurl=http://ceph.com/rpm/{distro}/$basearch + enabled=1 + gpgcheck=1 + type=rpm-md + gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc + + +.. note:: Ensure you use the correct URL for your distribution. Check the + http://ceph.com/rpm directory for your distribution. + +.. note:: Since you can upgrade using ``ceph-deploy`` you will only need to add + the repository on Ceph Client nodes where you use the ``ceph`` command line + interface or the ``ceph-deploy`` tool. + + +Upgrade daemons in the following order: + +#. **Monitors:** If the ``ceph-mon`` daemons are not restarted prior to the + ``ceph-osd`` daemons, the monitors will not correctly register their new + capabilities with the cluster and new features may not be usable until + the monitors are restarted a second time. + +#. **OSDs** + +#. **MDSs:** If the ``ceph-mds`` daemon is restarted first, it will wait until + all OSDs have been upgraded before finishing its startup sequence. + +#. **Gateways:** Upgrade ``radosgw`` daemons together. There is a subtle change + in behavior for multipart uploads that prevents a multipart request that + was initiated with a new ``radosgw`` from being completed by an old + ``radosgw``. + + Upgrade Procedures ================== @@ -354,8 +453,8 @@ To upgrade monitors, perform the following steps: You may use ``ceph-deploy`` to address all monitor nodes at once. For example:: - ceph-deploy install --stable {stable release} ceph-node1[ ceph-node2] - ceph-deploy install --stable dumpling mon1 mon2 mon3 + ceph-deploy install --release {release-name} ceph-node1[ ceph-node2] + ceph-deploy install --release dumpling mon1 mon2 mon3 You may also use the package manager for your Linux distribution on each individual node. To upgrade packages manually on each Debian/Ubuntu @@ -370,11 +469,11 @@ To upgrade monitors, perform the following steps: sudo yum update && sudo yum install ceph -#. Restart each monitor. For Debian/Ubuntu distributions, use:: +#. Restart each monitor. For Ubuntu distributions, use:: sudo restart ceph-mon id={hostname} - For CentOS/Red Hat distributions, use:: + For CentOS/Red Hat/Debian distributions, use:: sudo /etc/init.d/ceph restart {mon-id} @@ -398,8 +497,8 @@ To upgrade a Ceph OSD Daemon, perform the following steps: You may use ``ceph-deploy`` to address all Ceph OSD Daemon nodes at once. For example:: - ceph-deploy install --stable {stable release} ceph-node1[ ceph-node2] - ceph-deploy install --stable dumpling mon1 mon2 mon3 + ceph-deploy install --release {release-name} ceph-node1[ ceph-node2] + ceph-deploy install --release dumpling mon1 mon2 mon3 You may also use the package manager on each node to upgrade packages manually. For Debian/Ubuntu hosts, perform the following steps on each @@ -414,7 +513,7 @@ To upgrade a Ceph OSD Daemon, perform the following steps: sudo yum update && sudo yum install ceph -#. Restart the OSD, where ``N`` is the OSD number. For Debian/Ubuntu, use:: +#. Restart the OSD, where ``N`` is the OSD number. For Ubuntu, use:: sudo restart ceph-osd id=N @@ -422,7 +521,7 @@ To upgrade a Ceph OSD Daemon, perform the following steps: sudo restart ceph-osd-all - For CentOS/Red Hat distributions, use:: + For CentOS/Red Hat/Debian distributions, use:: sudo /etc/init.d/ceph restart N @@ -444,8 +543,8 @@ To upgrade a Ceph Metadata Server, perform the following steps: address all Ceph Metadata Server nodes at once, or use the package manager on each node. For example:: - ceph-deploy install --stable {stable release} ceph-node1[ ceph-node2] - ceph-deploy install --stable dumpling mon1 mon2 mon3 + ceph-deploy install --release {release-name} ceph-node1[ ceph-node2] + ceph-deploy install --release dumpling mon1 mon2 mon3 To upgrade packages manually, perform the following steps on each Debian/Ubuntu host. :: @@ -459,11 +558,11 @@ To upgrade a Ceph Metadata Server, perform the following steps: sudo yum update && sudo yum install ceph-mds -#. Restart the metadata server. For Debian/Ubuntu, use:: +#. Restart the metadata server. For Ubuntu, use:: sudo restart ceph-mds id={hostname} - For CentOS/Red Hat distributions, use:: + For CentOS/Red Hat/Debian distributions, use:: sudo /etc/init.d/ceph restart mds.{hostname} @@ -531,7 +630,15 @@ to use the default paths. Under those directories, the keyring should be in a file named ``keyring``. + + + .. _Monitor Config Reference: ../../rados/configuration/mon-config-ref .. _Joao's blog post: http://ceph.com/dev-notes/cephs-new-monitor-changes .. _Ceph Authentication: ../../rados/operations/authentication/ .. _Ceph Authentication - Backward Compatibility: ../../rados/operations/authentication/#backward-compatibility +.. _manually: ../install-storage-cluster/ +.. _Operating a Cluster: ../../rados/operations/operating +.. _Monitoring a Cluster: ../../rados/operations/monitoring +.. _Firefly release notes: ../../release-notes/#v0-80-firefly +.. _release notes: ../../release-notes \ No newline at end of file