REST API has been defined to provide configuration and control interfaces for the pubsub
mechanisms. This API has two flavors, one is S3-compatible and one is not. The two flavors can be used
together, although it is recommended to use the S3-compatible one.
-We also have detailed `S3 bucket notification compatibility`_ document.
Events are stored as RGW objects in a special bucket, under a special user. Events cannot
be accessed directly, but need to be pulled and acked using the new REST API.
+.. toctree::
+ :maxdepth: 1
+
+ S3 Bucket Notification Compatibility <s3-notification-compatibility>
+
PubSub Tier Type Configuration
-------------------------------
- event-id: id of event to be acked
-.. _Multisite Configuration: ./multisite.rst
-.. _S3 bucket notification compatibility: ./s3_notification_compatibility.rst
+.. _Multisite Configuration: ../multisite
--- /dev/null
+=====================================
+S3 Bucket Notifications Compatibility
+=====================================
+
+Ceph's `PubSub module`_ follows `AWS S3 Bucket Notifications API`_. However, some differences exist, as listed below.
+
+Supported Destination
+----------------------
+
+AWS supports: **SNS**, **SQS** and **Lambda** as possible destinations (AWS internal destinations).
+Currently, we support: **HTTP/S** and **AMQP**. And also support pulling and acking of events stored in Ceph (as an intenal destination).
+
+We are using the SNS ARNs to represent these destinations.
+
+Notification Configuration XML
+------------------------------
+
+Following tags (and the tags inside them) are not supported:
+
++-----------------------------------+----------------------------------------------+
+| Tag | Remaks |
++===================================+==============================================+
+| ``<QueueConfiguration>`` | not needed, we treat all destinations as SNS |
++-----------------------------------+----------------------------------------------+
+| ``<CloudFunctionConfiguration>`` | not needed, we treat all destinations as SNS |
++-----------------------------------+----------------------------------------------+
+| ``<Filter>`` | object filtering not supported |
++-----------------------------------+----------------------------------------------+
+
+REST API Extension
+------------------
+
+Ceph's bucket notification API follows has the following extensions:
+
+- Deletion of a specific notification, or all notifications on a bucket, without deletion of the bucket
+- Getting the information on a specific notification (when some exists on a bucket)
+
+Unsupported Fields in the Event Record
+--------------------------------------
+
+The records sent for bucket notification follow format described in: `Event Message Structure`_.
+However, the following fields are sent empty:
+
++----------------------------------------+-------------------------------------------------------------+
+| Field | Description |
++========================================+=============================================================+
+| ``userIdentity.principalId`` | The identity of the user that triggered the event |
++----------------------------------------+-------------------------------------------------------------+
+| ``requestParameters.sourceIPAddress`` | The IP address of the client that triggered the event |
++----------------------------------------+-------------------------------------------------------------+
+| ``requestParameters.x-amz-request-id`` | The request id that triggered the event |
++----------------------------------------+-------------------------------------------------------------+
+| ``requestParameters.x-amz-id-2`` | The IP address of the RGW on which the event was triggered |
++----------------------------------------+-------------------------------------------------------------+
+| ``s3.object.size`` | The size of the object |
++----------------------------------------+-------------------------------------------------------------+
+
+Event Types
+-----------
+
++----------------------------------------------+-----------------+-------------------------------------------+
+| Event | Status | Remarks |
++==============================================+=================+===========================================+
+| ``s3:ObjectCreated:*`` | Supported | |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectCreated:Put`` | Supported | supported at ``s3:ObjectCreated:*`` level |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectCreated:Post`` | Not Supported | start of multi-part upload not supported |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectCreated:Copy`` | Supported | supported at ``s3:ObjectCreated:*`` level |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectCreated:CompleteMultipartUpload`` | Supported | supported at ``s3:ObjectCreated:*`` level |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectRemoved:*`` | Supported | |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectRemoved:Delete`` | Supported | supported at ``s3:ObjectRemoved:*`` level |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectRemoved:DeleteMarkerCreated`` | Supported | supported at ``s3:ObjectRemoved:*`` level |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectRestore:Post`` | Not Supported | not applicable to Ceph |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ObjectRestore:Complete`` | Not Supported | not applicable to Ceph |
++----------------------------------------------+-----------------+-------------------------------------------+
+| ``s3:ReducedRedundancyLostObject`` | Not Supported | not applicable to Ceph |
++----------------------------------------------+-----------------+-------------------------------------------+
+
+.. _AWS S3 Bucket Notifications API: https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html
+.. _Event Message Structure: https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html
+.. _`PubSub module`: ../pubsub-module
The following table describes the support status for current Amazon S3 functional features:
-+---------------------------------+-----------------+----------------------------------------+
-| Feature | Status | Remarks |
-+=================================+=================+========================================+
-| **List Buckets** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Delete Bucket** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Create Bucket** | Supported | Different set of canned ACLs |
-+---------------------------------+-----------------+----------------------------------------+
-| **Bucket Lifecycle** | Supported | Removing expired files is supported |
-+---------------------------------+-----------------+----------------------------------------+
-| **Policy (Buckets, Objects)** | Supported | ACLs & bucket policies are supported |
-+---------------------------------+-----------------+----------------------------------------+
-| **Bucket Website** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Bucket ACLs (Get, Put)** | Supported | Different set of canned ACLs |
-+---------------------------------+-----------------+----------------------------------------+
-| **Bucket Location** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Bucket Notification** | Supported | `S3 Notification Compatibility`_ |
-+---------------------------------+-----------------+----------------------------------------+
-| **Bucket Object Versions** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Get Bucket Info (HEAD)** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Bucket Request Payment** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Put Object** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Delete Object** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Get Object** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Object ACLs (Get, Put)** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Get Object Info (HEAD)** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **POST Object** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Copy Object** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Multipart Uploads** | Supported | |
-+---------------------------------+-----------------+----------------------------------------+
-| **Object Tagging** | Supported | See :ref:`tag_policy` for Policy verbs |
-+---------------------------------+-----------------+----------------------------------------+
-| **Storage Class** | Not Supported | Use **Bucket Location** as alternative |
-+--------------------------------------------------------------------------------------------+
++---------------------------------+-----------------+------------------------------------------+
+| Feature | Status | Remarks |
++=================================+=================+==========================================+
+| **List Buckets** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Delete Bucket** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Create Bucket** | Supported | Different set of canned ACLs |
++---------------------------------+-----------------+------------------------------------------+
+| **Bucket Lifecycle** | Supported | Removing expired files is supported |
++---------------------------------+-----------------+------------------------------------------+
+| **Policy (Buckets, Objects)** | Supported | ACLs & bucket policies are supported |
++---------------------------------+-----------------+------------------------------------------+
+| **Bucket Website** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Bucket ACLs (Get, Put)** | Supported | Different set of canned ACLs |
++---------------------------------+-----------------+------------------------------------------+
+| **Bucket Location** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Bucket Notification** | Supported | See `S3 Notification Compatibility`_ |
++---------------------------------+-----------------+------------------------------------------+
+| **Bucket Object Versions** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Get Bucket Info (HEAD)** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Bucket Request Payment** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Put Object** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Delete Object** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Get Object** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Object ACLs (Get, Put)** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Get Object Info (HEAD)** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **POST Object** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Copy Object** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Multipart Uploads** | Supported | |
++---------------------------------+-----------------+------------------------------------------+
+| **Object Tagging** | Supported | See :ref:`tag_policy` for Policy verbs |
++---------------------------------+-----------------+------------------------------------------+
+| **Storage Class** | Not Supported | Use **Bucket Location** as alternative |
++----------------------------------------------------------------------------------------------+
Unsupported Header Fields
+----------------------------+------------+
.. _Amazon S3 API: http://docs.aws.amazon.com/AmazonS3/latest/API/APIRest.html
-.. _S3 Notification Compatibility: ./s3_notification_compatibility.rst
+.. _S3 Notification Compatibility: ../s3_notification_compatibility
+++ /dev/null
-=====================================
-S3 Bucket Notifications Compatibility
-=====================================
-
-Notification Push Endpoints
----------------------------
-
-AWS supports internal endpoints: SNS, SQS and Lambda as possible push endpoints. Currently we support HTTP/S and AMQP.
-We are using the SNS ARNs to represent these endpoints.
-We also support pulling and acking of events as described in the `pubsub module`_ documentation.
-
-Notification configuration XML
-------------------------------
-
-Following tags (and the tags inside them) are not supported:
-
-- ``<QueueConfiguration>``: this is used for AWS SQS endpoints, we treat all endpoints as SNS
-- ``<CloudFunctionConfiguration>``: this is used for AWS Lambda endpoints, we treat all endpoints as SNS
-- ``<Filter>``: object filtering based on key name/prefix/suffix not supported
-
-REST API Extension
-------------------
-
-- We support deletion of a specific notification, or all notifications on a bucket, without deletion of the bucket
-- We support getting the information on a specific notification (when some exists on a bucket)
-
-Unsupported Fields in the Event Record
---------------------------------------
-
-The records sent for bucket notification follow the S3 format described in the `pubsub module`_ documentation.
-However, the following fields are sent empty:
-
-+----------------------------------------+-------------------------------------------------------------+
-| Field | Description |
-+========================================+=============================================================+
-| ``userIdentity.principalId`` | The identity of the user that triggered the event |
-+----------------------------------------+-------------------------------------------------------------+
-| ``requestParameters.sourceIPAddress`` | The IP address of the client that triggered the event |
-+----------------------------------------+-------------------------------------------------------------+
-| ``requestParameters.x-amz-request-id`` | The request id that triggered the event |
-+----------------------------------------+-------------------------------------------------------------+
-| ``requestParameters.x-amz-id-2`` | The IP address of the RGW on which the event was triggered |
-+----------------------------------------+-------------------------------------------------------------+
-| ``s3.object.size`` | The size of the object |
-+----------------------------------------+-------------------------------------------------------------+
-
-Event Types
------------
-
-+----------------------------------------------+-----------------+-------------------------------------------+
-| Event | Status | Remarks |
-+==============================================+=================+===========================================+
-| ``s3:ObjectCreated:*`` | Supported | |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectCreated:Put`` | Supported | supported at ``s3:ObjectCreated:*`` level |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectCreated:Post`` | Not Supported | start of multi-part upload not supported |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectCreated:Copy`` | Supported | supported at ``s3:ObjectCreated:*`` level |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectCreated:CompleteMultipartUpload`` | Supported | supported at ``s3:ObjectCreated:*`` level |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectRemoved:*`` | Supported | |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectRemoved:Delete`` | Supported | supported at ``s3:ObjectRemoved:*`` level |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectRemoved:DeleteMarkerCreated`` | Supported | supported at ``s3:ObjectRemoved:*`` level |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectRestore:Post`` | Not Supported | not applicable to Ceph |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ObjectRestore:Complete`` | Not Supported | not applicable to Ceph |
-+----------------------------------------------+-----------------+-------------------------------------------+
-| ``s3:ReducedRedundancyLostObject`` | Not Supported | not applicable to Ceph |
-+----------------------------------------------+-----------------+-------------------------------------------+
-
-.. _`pubsub module`: ./pubsub-module.rst
-