Using Hadoop with CephFS
========================
-The Ceph file system can be used in place of HDFS in a Hadoop installation
-by using the Ceph file system client Java package, and requires no changes to
-the Hadoop code base.
-
-The Apache Hadoop project is a framework for building data-intensive
-applications. Applications built for the Hadoop framework include MapReduce,
-HBase, Hive, Mahout, and many others. Data management in Hadoop is handled by
-a distributed file system, and the default file system supported by Hadoop is
-the Hadoop Distributed File System (HDFS). However, Hadoop is not restricted
-to using HDFS, and any alternative file system can be used with Hadoop by
-plugging in a different implementation of the Hadoop virtual file system
-layer.
+The Ceph file system can be used as a drop-in replacement for the Hadoop File
+System (HDFS). This page describes the installation and configuration process
+of using Ceph with Hadoop.
+
+Dependencies
+============
+
+* CephFS Java Interface
+* Hadoop CephFS Plugin
+
+.. important:: Currently requires Hadoop 1.1.X stable series
Installation
============
the file system are beyond the scope of this document. Please refer to the
Ceph documentation for installing Ceph.
-.. important:: The master branch is currently required for compatibility.
-
The remaining two requirements are a Hadoop installation, and the Ceph file
system Java packages, including the Java CephFS Hadoop plugin. The high-level
steps are two add the dependencies to the Hadoop installation ``CLASSPATH``,
CephFS Java Packages
--------------------
-* CephFS Java package is located
-* CephFS Hadoop plugin is located
+* CephFS Hadoop plugin (`hadoop-cephfs.jar <http://ceph.com/download/hadoop-cephfs.jar>`_)
Adding these dependencies to a Hadoop installation will depend on your
particular deployment. In general the dependencies must be present on each
|Property |Value |Notes |
| | | |
+=====================+==========================+============================+
-|fs.default.name |Ceph URI |ceph://[monaddr:port][/root]|
+|fs.default.name |Ceph URI |ceph://[monaddr:port]/ |
| | | |
| | | |
+---------------------+--------------------------+----------------------------+