+++ /dev/null
-=====================================================
- Configuring Ceph Object Gateway with Apache/FastCGI
-=====================================================
-
-Configuring a Ceph Object Gateway requires a running Ceph Storage Cluster.
-Since it contains an embedded webserver (civetweb), the Ceph Object Gateway
-does not require an external web server, but it can be configured to use
-Apache with the FastCGI module.
-
-.. note:: CGI can pose a security risk.
-
-The Ceph Object Gateway is a client of the Ceph Storage Cluster. As a
-Ceph Storage Cluster client, it requires:
-
-- A name for the gateway instance. We use ``gateway`` in this guide.
-- A storage cluster user name with appropriate permissions in a keyring.
-- Pools to store its data.
-- A data directory for the gateway instance.
-- An instance entry in the Ceph Configuration file.
-- A configuration file for the web server to interact with FastCGI.
-
-
-Create a User and Keyring
-=========================
-
-Each instance must have a user name and key to communicate with a Ceph Storage
-Cluster. In the following steps, we use an admin node to create a keyring.
-Then, we create a client user name and key. Next, we add the
-key to the Ceph Storage Cluster. Finally, we distribute the key ring to
-the node containing the gateway instance.
-
-.. topic:: Monitor Key CAPS
-
- When you provide CAPS to the key, you MUST provide read capability.
- However, you have the option of providing write capability for the monitor.
- This is an important choice. If you provide write capability to the key,
- the Ceph Object Gateway will have the ability to create pools automatically;
- however, it will create pools with either the default number of placement
- groups (not ideal) or the number of placement groups you specified in your
- Ceph configuration file. If you allow the Ceph Object Gateway to create
- pools automatically, ensure that you have reasonable defaults for the number
- of placement groups first. See `Pool Configuration`_ for details.
-
-
-See `User Management`_ for additional details on Ceph authentication.
-
-#. Generate a Ceph Object Gateway user name and key for each instance. For
- exemplary purposes, we will use the name ``gateway`` after ``client.radosgw``::
-
- sudo ceph auth get-or-create client.radosgw.gateway osd 'allow rwx' mon 'allow rwx' -o /etc/ceph/ceph.client.radosgw.keyring
-
-#. Distribute the keyring to the node with the gateway instance. ::
-
- sudo scp /etc/ceph/ceph.client.radosgw.keyring ceph@{hostname}:/home/ceph
- ssh {hostname}
- sudo mv ceph.client.radosgw.keyring /etc/ceph/ceph.client.radosgw.keyring
-
-
- .. note:: The 2nd step is optional if ``admin node`` is the ``gateway host``.
-
-Create Pools
-============
-
-Ceph Object Gateways require Ceph Storage Cluster pools to store specific
-gateway data. If the user you created has permissions, the gateway
-will create the pools automatically. However, you should ensure that you have
-set an appropriate default number of placement groups per pool into your Ceph
-configuration file.
-
-.. note:: Ceph Object Gateways have multiple pools, so don't make the number of
- PGs too high considering all of the pools assigned to the same CRUSH
- hierarchy, or performance may suffer.
-
-When configuring a gateway with the default region and zone, the naming
-convention for pools typically omits region and zone naming, but you can use any
-naming convention you prefer. For example:
-
-
-- ``.rgw.root``
-- ``.rgw.control``
-- ``.rgw.gc``
-- ``.rgw.buckets``
-- ``.rgw.buckets.index``
-- ``.rgw.buckets.extra``
-- ``.log``
-- ``.intent-log``
-- ``.usage``
-- ``.users``
-- ``.users.email``
-- ``.users.swift``
-- ``.users.uid``
-
-
-See `Configuration Reference - Pools`_ for details on the default pools for
-gateways. See `Pools`_ for details on creating pools. As already said, if
-write permission is given, Ceph Object Gateway will create pools automatically.
-To create a pool manually, execute the following::
-
- ceph osd pool create {poolname} {pg-num} {pgp-num} {replicated | erasure} [{erasure-code-profile}] {ruleset-name} {ruleset-number}
-
-
-.. tip:: Ceph supports multiple CRUSH hierarchies and CRUSH rulesets, enabling
- great flexibility in the way you configure your gateway. Pools such as
- ``rgw.buckets.index`` may benefit from a pool of SSDs for fast performance.
- Backing storage may benefit from the increased economy of erasure-coded
- storage, and/or the improved performance from cache tiering.
-
-When you have completed this step, execute the following to ensure that
-you have created all of the foregoing pools::
-
- rados lspools
-
-
-Add a Gateway Configuration to Ceph
-===================================
-
-Add the Ceph Object Gateway configuration to your Ceph Configuration file in
-``admin node``. The Ceph Object Gateway configuration requires you to
-identify the Ceph Object Gateway instance. Then, you must specify the host name
-where you installed the Ceph Object Gateway daemon, a keyring (for use with
-cephx), the socket path for FastCGI and a log file.
-
-For distros with Apache 2.2 and early versions of Apache 2.4 (RHEL 6, Ubuntu
-12.04, 14.04 etc), append the following configuration to ``/etc/ceph/ceph.conf``
-in your ``admin node``::
-
- [client.radosgw.gateway]
- host = {hostname}
- keyring = /etc/ceph/ceph.client.radosgw.keyring
- rgw socket path = ""
- log file = /var/log/radosgw/client.radosgw.gateway.log
- rgw frontends = fastcgi socket_port=9000 socket_host=0.0.0.0
- rgw print continue = false
-
-
-.. note:: Apache 2.2 and early versions of Apache 2.4 do not use Unix Domain
- Sockets but use localhost TCP.
-
-For distros with Apache 2.4.9 or later (RHEL 7, CentOS 7 etc), append the
-following configuration to ``/etc/ceph/ceph.conf`` in your ``admin node``::
-
- [client.radosgw.gateway]
- host = {hostname}
- keyring = /etc/ceph/ceph.client.radosgw.keyring
- rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
- log file = /var/log/radosgw/client.radosgw.gateway.log
- rgw print continue = false
-
-
-.. note:: ``Apache 2.4.9`` supports Unix Domain Socket (UDS) but as
- ``Ubuntu 14.04`` ships with ``Apache 2.4.7`` it doesn't have UDS support and
- has to be configured for use with localhost TCP. A bug has been filed for
- backporting UDS support in ``Apache 2.4.7`` for ``Ubuntu 14.04``.
- See: `Backport support for UDS in Ubuntu Trusty`_
-
-Here, ``{hostname}`` is the short hostname (output of command ``hostname -s``)
-of the node that is going to provide the gateway service i.e., the
-``gateway host``.
-
-The ``[client.radosgw.gateway]`` portion of the gateway instance identifies this
-portion of the Ceph configuration file as configuring a Ceph Storage Cluster
-client where the client type is a Ceph Object Gateway (i.e., ``radosgw``).
-
-
-.. note:: The last line in the configuration i.e., ``rgw print continue = false``
- is added to avoid issues with ``PUT`` operations.
-
-Once you finish the setup procedure, if you encounter issues with your
-configuration, you can add debugging to the ``[global]`` section of your Ceph
-configuration file and restart the gateway to help troubleshoot any
-configuration issues. For example::
-
- [global]
- #append the following in the global section.
- debug ms = 1
- debug rgw = 20
-
-
-Distribute updated Ceph configuration file
-==========================================
-
-The updated Ceph configuration file needs to be distributed to all Ceph cluster
-nodes from the ``admin node``.
-
-It involves the following steps:
-
-#. Pull the updated ``ceph.conf`` from ``/etc/ceph/`` to the root directory of
- the cluster in admin node (e.g. ``my-cluster`` directory). The contents of
- ``ceph.conf`` in ``my-cluster`` will get overwritten. To do so, execute the
- following::
-
- ceph-deploy --overwrite-conf config pull {hostname}
-
- Here, ``{hostname}`` is the short hostname of the Ceph admin node.
-
-#. Push the updated ``ceph.conf`` file from the admin node to all other nodes in
- the cluster including the ``gateway host``::
-
- ceph-deploy --overwrite-conf config push [HOST] [HOST...]
-
- Give the hostnames of the other Ceph nodes in place of ``[HOST] [HOST...]``.
-
-
-Copy ceph.client.admin.keyring from admin node to gateway host
-==============================================================
-
-As the ``gateway host`` can be a different node that is not part of the cluster,
-the ``ceph.client.admin.keyring`` needs to be copied from the ``admin node`` to
-the ``gateway host``. To do so, execute the following on ``admin node``::
-
- sudo scp /etc/ceph/ceph.client.admin.keyring ceph@{hostname}:/home/ceph
- ssh {hostname}
- sudo mv ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring
-
-
-.. note:: The above step need not be executed if ``admin node`` is the
- ``gateway host``.
-
-
-Create Data Directory
-=====================
-
-Deployment scripts may not create the default Ceph Object Gateway data
-directory. Create data directories for each instance of a ``radosgw``
-daemon (if you haven't done so already). The ``host`` variables in the
-Ceph configuration file determine which host runs each instance of a
-``radosgw`` daemon. The typical form specifies the ``radosgw`` daemon,
-the cluster name and the daemon ID.
-
-To create the directory on the ``gateway host``, execute the following::
-
- sudo mkdir -p /var/lib/ceph/radosgw/ceph-radosgw.gateway
-
-
-Adjust Socket Directory Permissions
-===================================
-
-On some distros, the ``radosgw`` daemon runs as the unprivileged ``apache``
-UID, and this UID must have write access to the location where it will write
-its socket file.
-
-To grant permissions to the default socket location, execute the following on
-the ``gateway host``::
-
- sudo chown apache:apache /var/run/ceph
-
-
-Change Log File Owner
-=====================
-
-On some distros, the ``radosgw`` daemon runs as the unprivileged ``apache`` UID,
-but the ``root`` user owns the log file by default. You must change it to the
-``apache`` user so that Apache can populate the log file. To do so, execute
-the following::
-
- sudo chown apache:apache /var/log/radosgw/client.radosgw.gateway.log
-
-
-Start radosgw service
-=====================
-
-The Ceph Object gateway daemon needs to be started. To do so, execute the
-following on the ``gateway host``:
-
-On Debian-based distros::
-
- sudo /etc/init.d/radosgw start
-
-On RPM-based distros::
-
- sudo /etc/init.d/ceph-radosgw start
-
-
-Create a Gateway Configuration file
-===================================
-
-On the host where you installed the Ceph Object Gateway i.e., ``gateway host``,
-create an ``rgw.conf`` file. Place the file in ``/etc/apache2/conf-available``
-directory for ``Debian-based`` distros and in ``/etc/httpd/conf.d`` directory
-for ``RPM-based`` distros. It is a Apache configuration file which is needed
-for the ``radosgw`` service. This file must be readable by the web server.
-
-Execute the following steps:
-
-#. Create the file:
-
- For Debian-based distros, execute::
-
- sudo vi /etc/apache2/conf-available/rgw.conf
-
- For RPM-based distros, execute::
-
- sudo vi /etc/httpd/conf.d/rgw.conf
-
-#. For distros with Apache 2.2 and early versions of Apache 2.4 that use
- localhost TCP and do not support Unix Domain Socket, add the following
- contents to the file::
-
- <VirtualHost *:80>
- ServerName localhost
- DocumentRoot /var/www/html
-
- ErrorLog /var/log/httpd/rgw_error.log
- CustomLog /var/log/httpd/rgw_access.log combined
-
- # LogLevel debug
-
- RewriteEngine On
-
- RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
-
- SetEnv proxy-nokeepalive 1
-
- ProxyPass / fcgi://localhost:9000/
-
- </VirtualHost>
-
- .. note:: For Debian-based distros replace ``/var/log/httpd/``
- with ``/var/log/apache2``.
-
-#. For distros with Apache 2.4.9 or later that support Unix Domain Socket,
- add the following contents to the file::
-
- <VirtualHost *:80>
- ServerName localhost
- DocumentRoot /var/www/html
-
- ErrorLog /var/log/httpd/rgw_error.log
- CustomLog /var/log/httpd/rgw_access.log combined
-
- # LogLevel debug
-
- RewriteEngine On
-
- RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
-
- SetEnv proxy-nokeepalive 1
-
- ProxyPass / unix:///var/run/ceph/ceph.radosgw.gateway.fastcgi.sock|fcgi://localhost:9000/
-
- </VirtualHost>
-
-
-Restart Apache
-==============
-
-The Apache service needs to be restarted to accept the new configuration.
-
-For Debian-based distros, run::
-
- sudo service apache2 restart
-
-For RPM-based distros, run::
-
- sudo service httpd restart
-
-Or::
-
- sudo systemctl restart httpd
-
-
-Using The Gateway
-=================
-
-To use the REST interfaces, first create an initial Ceph Object Gateway
-user for the S3 interface. Then, create a subuser for the Swift interface.
-See the `Admin Guide`_ for more details on user management.
-
-Create a radosgw user for S3 access
-------------------------------------
-
-A ``radosgw`` user needs to be created and granted access. The command
-``man radosgw-admin`` will provide information on additional command options.
-
-To create the user, execute the following on the ``gateway host``::
-
- sudo radosgw-admin user create --uid="testuser" --display-name="First User"
-
-The output of the command will be something like the following::
-
- {"user_id": "testuser",
- "display_name": "First User",
- "email": "",
- "suspended": 0,
- "max_buckets": 1000,
- "auid": 0,
- "subusers": [],
- "keys": [
- { "user": "testuser",
- "access_key": "I0PJDPCIYZ665MW88W9R",
- "secret_key": "dxaXZ8U90SXydYzyS5ivamEP20hkLSUViiaR+ZDA"}],
- "swift_keys": [],
- "caps": [],
- "op_mask": "read, write, delete",
- "default_placement": "",
- "placement_tags": [],
- "bucket_quota": { "enabled": false,
- "max_size_kb": -1,
- "max_objects": -1},
- "user_quota": { "enabled": false,
- "max_size_kb": -1,
- "max_objects": -1},
- "temp_url_keys": []}
-
-
-.. note:: The values of ``keys->access_key`` and ``keys->secret_key`` are
- needed for access validation.
-
-Create a Swift user
--------------------
-
-A Swift subuser needs to be created if this kind of access is needed. Creating
-a Swift user is a two step process. The first step is to create the user.
-The second is to create the secret key.
-
-Execute the following steps on the ``gateway host``:
-
-Create the Swift user::
-
- sudo radosgw-admin subuser create --uid=testuser --subuser=testuser:swift --access=full
-
-The output will be something like the following::
-
- { "user_id": "testuser",
- "display_name": "First User",
- "email": "",
- "suspended": 0,
- "max_buckets": 1000,
- "auid": 0,
- "subusers": [
- { "id": "testuser:swift",
- "permissions": "full-control"}],
- "keys": [
- { "user": "testuser:swift",
- "access_key": "3Y1LNW4Q6X0Y53A52DET",
- "secret_key": ""},
- { "user": "testuser",
- "access_key": "I0PJDPCIYZ665MW88W9R",
- "secret_key": "dxaXZ8U90SXydYzyS5ivamEP20hkLSUViiaR+ZDA"}],
- "swift_keys": [],
- "caps": [],
- "op_mask": "read, write, delete",
- "default_placement": "",
- "placement_tags": [],
- "bucket_quota": { "enabled": false,
- "max_size_kb": -1,
- "max_objects": -1},
- "user_quota": { "enabled": false,
- "max_size_kb": -1,
- "max_objects": -1},
- "temp_url_keys": []}
-
-Create the secret key::
-
- sudo radosgw-admin key create --subuser=testuser:swift --key-type=swift --gen-secret
-
-The output will be something like the following::
-
- { "user_id": "testuser",
- "display_name": "First User",
- "email": "",
- "suspended": 0,
- "max_buckets": 1000,
- "auid": 0,
- "subusers": [
- { "id": "testuser:swift",
- "permissions": "full-control"}],
- "keys": [
- { "user": "testuser:swift",
- "access_key": "3Y1LNW4Q6X0Y53A52DET",
- "secret_key": ""},
- { "user": "testuser",
- "access_key": "I0PJDPCIYZ665MW88W9R",
- "secret_key": "dxaXZ8U90SXydYzyS5ivamEP20hkLSUViiaR+ZDA"}],
- "swift_keys": [
- { "user": "testuser:swift",
- "secret_key": "244+fz2gSqoHwR3lYtSbIyomyPHf3i7rgSJrF\/IA"}],
- "caps": [],
- "op_mask": "read, write, delete",
- "default_placement": "",
- "placement_tags": [],
- "bucket_quota": { "enabled": false,
- "max_size_kb": -1,
- "max_objects": -1},
- "user_quota": { "enabled": false,
- "max_size_kb": -1,
- "max_objects": -1},
- "temp_url_keys": []}
-
-Access Verification
-===================
-
-You then need to verify if the created users are able to access the gateway.
-
-Test S3 access
---------------
-
-You need to write and run a Python test script for verifying S3 access. The S3
-access test script will connect to the ``radosgw``, create a new bucket and list
-all buckets. The values for ``aws_access_key_id`` and ``aws_secret_access_key``
-are taken from the values of ``access_key`` and ``secret_key`` returned by the
-``radosgw_admin`` command.
-
-Execute the following steps:
-
-#. You will need to install the ``python-boto`` package.
-
- For Debian-based distros, run::
-
- sudo apt-get install python-boto
-
- For RPM-based distros, run::
-
- sudo yum install python-boto
-
-#. Create the Python script::
-
- vi s3test.py
-
-#. Add the following contents to the file::
-
- import boto
- import boto.s3.connection
- access_key = 'I0PJDPCIYZ665MW88W9R'
- secret_key = 'dxaXZ8U90SXydYzyS5ivamEP20hkLSUViiaR+ZDA'
- conn = boto.connect_s3(
- aws_access_key_id = access_key,
- aws_secret_access_key = secret_key,
- host = '{hostname}',
- is_secure=False,
- calling_format = boto.s3.connection.OrdinaryCallingFormat(),
- )
- bucket = conn.create_bucket('my-new-bucket')
- for bucket in conn.get_all_buckets():
- print "{name}\t{created}".format(
- name = bucket.name,
- created = bucket.creation_date,
- )
-
- Replace ``{hostname}`` with the hostname of the host where you have
- configured the gateway service i.e., the ``gateway host``.
-
-#. Run the script::
-
- python s3test.py
-
- The output will be something like the following::
-
- my-new-bucket 2015-02-16T17:09:10.000Z
-
-Test swift access
------------------
-
-Swift access can be verified via the ``swift`` command line client. The command
-``man swift`` will provide more information on available command line options.
-
-To install ``swift`` client, execute the following:
-
- For Debian-based distros::
-
- sudo apt-get install python-setuptools
- sudo easy_install pip
- sudo pip install --upgrade setuptools
- sudo pip install --upgrade python-swiftclient
-
- For RPM-based distros::
-
- sudo yum install python-setuptools
- sudo easy_install pip
- sudo pip install --upgrade setuptools
- sudo pip install --upgrade python-swiftclient
-
-To test swift access, execute the following::
-
- swift -A http://{IP ADDRESS}/auth/1.0 -U testuser:swift -K β{swift_secret_key}β list
-
-Replace ``{IP ADDRESS}`` with the public IP address of the gateway server and
-``{swift_secret_key}`` with its value from the output of
-``radosgw-admin key create`` command executed for the ``swift`` user.
-
-For example::
-
- swift -A http://10.19.143.116/auth/1.0 -U testuser:swift -K β244+fz2gSqoHwR3lYtSbIyomyPHf3i7rgSJrF/IAβ list
-
-The output should be::
-
- my-new-bucket
-
-
-.. _Configuration Reference - Pools: ../config-ref#pools
-.. _Pool Configuration: ../../rados/configuration/pool-pg-config-ref/
-.. _Pools: ../../rados/operations/pools
-.. _User Management: ../../rados/operations/user-management
-.. _Backport support for UDS in Ubuntu Trusty: https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1411030
-.. _Admin Guide: ../admin
projects / ceph-ci.git / commitdiff