From: Kefu Chai Date: Wed, 19 May 2021 14:36:07 +0000 (+0800) Subject: doc/radosgw: use confval directive to define options X-Git-Tag: v17.1.0~1852^2 X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=8b16defd4fce54dad599d5f2b3d32d163dcf1e30;p=ceph.git doc/radosgw: use confval directive to define options less repeating this way Signed-off-by: Kefu Chai --- diff --git a/doc/radosgw/config-ref.rst b/doc/radosgw/config-ref.rst index c1478ec90a308..2ebbfec622358 100644 --- a/doc/radosgw/config-ref.rst +++ b/doc/radosgw/config-ref.rst @@ -13,338 +13,45 @@ specified in the command. Thus variables meant to be applied to all RGW instances or all radosgw-admin options can be put into the ``[global]`` or the ``[client]`` section to avoid specifying ``instance-name``. -``rgw_frontends`` - -:Description: Configures the HTTP frontend(s). The configuration for multiple - frontends can be provided in a comma-delimited list. Each frontend - configuration may include a list of options separated by spaces, - where each option is in the form "key=value" or "key". See - `HTTP Frontends`_ for more on supported options. - -:Type: String -:Default: ``beast port=7480`` - -``rgw_data`` - -:Description: Sets the location of the data files for Ceph RADOS Gateway. -:Type: String -:Default: ``/var/lib/ceph/radosgw/$cluster-$id`` - - -``rgw_enable_apis`` - -:Description: Enables the specified APIs. - - .. note:: Enabling the ``s3`` API is a requirement for - any ``radosgw`` instance that is meant to - participate in a `multi-site <../multisite>`_ - configuration. -:Type: String -:Default: ``s3, s3website, swift, swift_auth, admin, sts, iam, notifications`` All APIs. - - -``rgw_cache_enabled`` - -:Description: Whether the Ceph Object Gateway cache is enabled. -:Type: Boolean -:Default: ``true`` - - -``rgw_cache_lru_size`` - -:Description: The number of entries in the Ceph Object Gateway cache. -:Type: Integer -:Default: ``10000`` - - -``rgw_dns_name`` - -:Description: The DNS name of the served domain. See also the ``hostnames`` setting within regions. -:Type: String -:Default: None - - -``rgw_script_uri`` - -:Description: The alternative value for the ``SCRIPT_URI`` if not set - in the request. - -:Type: String -:Default: None - - -``rgw_request_uri`` - -:Description: The alternative value for the ``REQUEST_URI`` if not set - in the request. - -:Type: String -:Default: None - - -``rgw_print_continue`` - -:Description: Enable ``100-continue`` if it is operational. -:Type: Boolean -:Default: ``true`` - - -``rgw_remote_addr_param`` - -:Description: The remote address parameter. For example, the HTTP field - containing the remote address, or the ``X-Forwarded-For`` - address if a reverse proxy is operational. - -:Type: String -:Default: ``REMOTE_ADDR`` - - -``rgw_op_thread_timeout`` - -:Description: The timeout in seconds for open threads. -:Type: Integer -:Default: 600 - - -``rgw_op_thread_suicide_timeout`` - -:Description: The time ``timeout`` in seconds before a Ceph Object Gateway - process dies. Disabled if set to ``0``. - -:Type: Integer -:Default: ``0`` - - -``rgw_thread_pool_size`` - -:Description: The size of the thread pool. -:Type: Integer -:Default: 100 threads. - - -``rgw_num_control_oids`` - -:Description: The number of notification objects used for cache synchronization - between different ``rgw`` instances. - -:Type: Integer -:Default: ``8`` - - -``rgw_init_timeout`` - -:Description: The number of seconds before Ceph Object Gateway gives up on - initialization. - -:Type: Integer -:Default: ``30`` - - -``rgw_mime_types_file`` - -:Description: The path and location of the MIME-types file. Used for Swift - auto-detection of object types. - -:Type: String -:Default: ``/etc/mime.types`` - - -``rgw_s3_success_create_obj_status`` - -:Description: The alternate success status response for ``create-obj``. -:Type: Integer -:Default: ``0`` - - -``rgw_resolve_cname`` - -:Description: Whether ``rgw`` should use DNS CNAME record of the request - hostname field (if hostname is not equal to ``rgw dns name``). - -:Type: Boolean -:Default: ``false`` - - -``rgw_obj_stripe_size`` - -:Description: The size of an object stripe for Ceph Object Gateway objects. - See `Architecture`_ for details on striping. - -:Type: Integer -:Default: ``4 << 20`` - - -``rgw_extended_http_attrs`` - -:Description: Add new set of attributes that could be set on an entity - (user, bucket or object). These extra attributes can be set - through HTTP header fields when putting the entity or modifying - it using POST method. If set, these attributes will return as - HTTP fields when doing GET/HEAD on the entity. - -:Type: String -:Default: None -:Example: "content_foo, content_bar, x-foo-bar" - - -``rgw_exit_timeout_secs`` - -:Description: Number of seconds to wait for a process before exiting - unconditionally. - -:Type: Integer -:Default: ``120`` - - -``rgw_get_obj_window_size`` - -:Description: The window size in bytes for a single object request. -:Type: Integer -:Default: ``16 << 20`` - - -``rgw_get_obj_max_req_size`` - -:Description: The maximum request size of a single get operation sent to the - Ceph Storage Cluster. - -:Type: Integer -:Default: ``4 << 20`` - - -``rgw_relaxed_s3_bucket_names`` - -:Description: Enables relaxed S3 bucket names rules for US region buckets. -:Type: Boolean -:Default: ``false`` - - -``rgw_list_buckets_max_chunk`` - -:Description: The maximum number of buckets to retrieve in a single operation - when listing user buckets. - -:Type: Integer -:Default: ``1000`` - - -``rgw_override_bucket_index_max_shards`` - -:Description: Represents the number of shards for the bucket index object, - a value of zero indicates there is no sharding. It is not - recommended to set a value too large (e.g. thousand) as it - increases the cost for bucket listing. - This variable should be set in the client or global sections - so that it is automatically applied to radosgw-admin commands. - -:Type: Integer -:Default: ``0`` - - -``rgw_curl_wait_timeout_ms`` - -:Description: The timeout in milliseconds for certain ``curl`` calls. -:Type: Integer -:Default: ``1000`` - - -``rgw_copy_obj_progress`` - -:Description: Enables output of object progress during long copy operations. -:Type: Boolean -:Default: ``true`` - - -``rgw_copy_obj_progress_every_bytes`` - -:Description: The minimum bytes between copy progress output. -:Type: Integer -:Default: ``1024 * 1024`` - - -``rgw_admin_entry`` - -:Description: The entry point for an admin request URL. -:Type: String -:Default: ``admin`` - - -``rgw_content_length_compat`` - -:Description: Enable compatibility handling of FCGI requests with both ``CONTENT_LENGTH`` and ``HTTP_CONTENT_LENGTH`` set. -:Type: Boolean -:Default: ``false`` - - -``rgw_bucket_quota_ttl`` - -:Description: The amount of time in seconds cached quota information is - trusted. After this timeout, the quota information will be - re-fetched from the cluster. -:Type: Integer -:Default: ``600`` - - -``rgw_user_quota_bucket_sync_interval`` - -:Description: The amount of time in seconds bucket quota information is - accumulated before syncing to the cluster. During this time, - other RGW instances will not see the changes in bucket quota - stats from operations on this instance. -:Type: Integer -:Default: ``180`` - - -``rgw_user_quota_sync_interval`` - -:Description: The amount of time in seconds user quota information is - accumulated before syncing to the cluster. During this time, - other RGW instances will not see the changes in user quota stats - from operations on this instance. -:Type: Integer -:Default: ``180`` - - -``rgw_bucket_default_quota_max_objects`` - -:Description: Default max number of objects per bucket. Set on new users, - if no other quota is specified. Has no effect on existing users. - This variable should be set in the client or global sections - so that it is automatically applied to radosgw-admin commands. -:Type: Integer -:Default: ``-1`` - - -``rgw_bucket_default_quota_max_size`` - -:Description: Default max capacity per bucket, in bytes. Set on new users, - if no other quota is specified. Has no effect on existing users. -:Type: Integer -:Default: ``-1`` - - -``rgw_user_default_quota_max_objects`` - -:Description: Default max number of objects for a user. This includes all - objects in all buckets owned by the user. Set on new users, - if no other quota is specified. Has no effect on existing users. -:Type: Integer -:Default: ``-1`` - - -``rgw_user_default_quota_max_size`` - -:Description: The value for user max size quota in bytes set on new users, - if no other quota is specified. Has no effect on existing users. -:Type: Integer -:Default: ``-1`` - - -``rgw_verify_ssl`` - -:Description: Verify SSL certificates while making requests. -:Type: Boolean -:Default: ``true`` +.. confval:: rgw_frontends +.. confval:: rgw_data +.. confval:: rgw_enable_apis +.. confval:: rgw_cache_enabled +.. confval:: rgw_cache_lru_size +.. confval:: rgw_dns_name +.. confval:: rgw_script_uri +.. confval:: rgw_request_uri +.. confval:: rgw_print_continue +.. confval:: rgw_remote_addr_param +.. confval:: rgw_op_thread_timeout +.. confval:: rgw_op_thread_suicide_timeout +.. confval:: rgw_thread_pool_size +.. confval:: rgw_num_control_oids +.. confval:: rgw_init_timeout +.. confval:: rgw_mime_types_file +.. confval:: rgw_s3_success_create_obj_status +.. confval:: rgw_resolve_cname +.. confval:: rgw_obj_stripe_size +.. confval:: rgw_extended_http_attrs +.. confval:: rgw_exit_timeout_secs +.. confval:: rgw_get_obj_window_size +.. confval:: rgw_get_obj_max_req_size +.. confval:: rgw_relaxed_s3_bucket_names +.. confval:: rgw_list_buckets_max_chunk +.. confval:: rgw_override_bucket_index_max_shards +.. confval:: rgw_curl_wait_timeout_ms +.. confval:: rgw_copy_obj_progress +.. confval:: rgw_copy_obj_progress_every_bytes +.. confval:: rgw_admin_entry +.. confval:: rgw_content_length_compat +.. confval:: rgw_bucket_quota_ttl +.. confval:: rgw_user_quota_bucket_sync_interval +.. confval:: rgw_user_quota_sync_interval +.. confval:: rgw_bucket_default_quota_max_objects +.. confval:: rgw_bucket_default_quota_max_size +.. confval:: rgw_user_default_quota_max_objects +.. confval:: rgw_user_default_quota_max_size +.. confval:: rgw_verify_ssl Lifecycle Settings ================== @@ -359,27 +66,16 @@ index shard enumeration with a random ordered sequence. There are two options in particular to look at when looking to increase the aggressiveness of lifecycle processing: -``rgw_lc_max_worker`` - -:Description: This option specifies the number of lifecycle worker threads - to run in parallel, thereby processing bucket and index - shards simultaneously. - -:Type: Integer -:Default: ``3`` - -``rgw_lc_max_wp_worker`` - -:Description: This option specifies the number of threads in each lifecycle - workers work pool. This option can help accelerate processing each bucket. +.. confval:: rgw_lc_max_worker +.. confval:: rgw_lc_max_wp_worker These values can be tuned based upon your specific workload to further increase the aggressiveness of lifecycle processing. For a workload with a larger number of buckets (thousands) -you would look at increasing the ``rgw_lc_max_worker`` value from the default value of 3 whereas for a +you would look at increasing the :confval:`rgw_lc_max_worker` value from the default value of 3 whereas for a workload with a smaller number of buckets but higher number of objects (hundreds of thousands) -per bucket you would consider decreasing ``rgw_lc_max_wp_worker`` from the default value of 3. +per bucket you would consider decreasing :confval:`rgw_lc_max_wp_worker` from the default value of 3. -:NOTE: When looking to tune either of these specific values please validate the +.. note:: When looking to tune either of these specific values please validate the current Cluster performance and Ceph Object Gateway utilization before increasing. Garbage Collection Settings @@ -392,11 +88,13 @@ objects in the Ceph Storage cluster some time after the gateway deletes the objects from the bucket index. The process of purging the deleted object data from the Ceph Storage cluster is known as Garbage Collection or GC. -To view the queue of objects awaiting garbage collection, execute the following:: +To view the queue of objects awaiting garbage collection, execute the following + +.. prompt:: bash $ - $ radosgw-admin gc list + radosgw-admin gc list - Note: specify ``--include-all`` to list all entries, including unexpired +.. note:: specify ``--include-all`` to list all entries, including unexpired Garbage collection is a background activity that may execute continuously or during times of low loads, depending upon how the @@ -412,58 +110,20 @@ types of workloads, administrators can increase the priority of garbage collection operations relative to other operations with the following configuration parameters. - -``rgw_gc_max_objs`` - -:Description: The maximum number of objects that may be handled by - garbage collection in one garbage collection processing cycle. - Please do not change this value after the first deployment. - -:Type: Integer -:Default: ``32`` - - -``rgw_gc_obj_min_wait`` - -:Description: The minimum wait time before a deleted object may be removed - and handled by garbage collection processing. - -:Type: Integer -:Default: ``2 * 3600`` - - -``rgw_gc_processor_max_time`` - -:Description: The maximum time between the beginning of two consecutive garbage - collection processing cycles. - -:Type: Integer -:Default: ``3600`` - - -``rgw_gc_processor_period`` - -:Description: The cycle time for garbage collection processing. -:Type: Integer -:Default: ``3600`` - - -``rgw_gc_max_concurrent_io`` - -:Description: The maximum number of concurrent IO operations that the RGW garbage - collection thread will use when purging old data. -:Type: Integer -:Default: ``10`` - +.. confval:: rgw_gc_max_objs +.. confval:: rgw_gc_obj_min_wait +.. confval:: rgw_gc_processor_max_time +.. confval:: rgw_gc_processor_period +.. confval:: rgw_gc_max_concurrent_io :Tuning Garbage Collection for Delete Heavy Workloads: -As an initial step towards tuning Ceph Garbage Collection to be more aggressive the following options are suggested to be increased from their default configuration values: +As an initial step towards tuning Ceph Garbage Collection to be more aggressive the following options are suggested to be increased from their default configuration values:: -``rgw_gc_max_concurrent_io = 20`` -``rgw_gc_max_trim_chunk = 64`` + rgw_gc_max_concurrent_io = 20 + rgw_gc_max_trim_chunk = 64 -:NOTE: Modifying these values requires a restart of the RGW service. +.. note:: Modifying these values requires a restart of the RGW service. Once these values have been increased from default please monitor for performance of the cluster during Garbage Collection to verify no adverse performance issues due to the increased values. @@ -475,354 +135,55 @@ Multisite Settings You may include the following settings in your Ceph configuration file under each ``[client.radosgw.{instance-name}]`` instance. - -``rgw_zone`` - -:Description: The name of the zone for the gateway instance. If no zone is - set, a cluster-wide default can be configured with the command - ``radosgw-admin zone default``. -:Type: String -:Default: None - - -``rgw_zonegroup`` - -:Description: The name of the zonegroup for the gateway instance. If no - zonegroup is set, a cluster-wide default can be configured with - the command ``radosgw-admin zonegroup default``. -:Type: String -:Default: None - - -``rgw_realm`` - -:Description: The name of the realm for the gateway instance. If no realm is - set, a cluster-wide default can be configured with the command - ``radosgw-admin realm default``. -:Type: String -:Default: None - - -``rgw_run_sync_thread`` - -:Description: If there are other zones in the realm to sync from, spawn threads - to handle the sync of data and metadata. -:Type: Boolean -:Default: ``true`` - - -``rgw_data_log_window`` - -:Description: The data log entries window in seconds. -:Type: Integer -:Default: ``30`` - - -``rgw_data_log_changes_size`` - -:Description: The number of in-memory entries to hold for the data changes log. -:Type: Integer -:Default: ``1000`` - - -``rgw_data_log_obj_prefix`` - -:Description: The object name prefix for the data log. -:Type: String -:Default: ``data_log`` - - -``rgw_data_log_num_shards`` - -:Description: The number of shards (objects) on which to keep the - data changes log. - -:Type: Integer -:Default: ``128`` - - -``rgw_md_log_max_shards`` - -:Description: The maximum number of shards for the metadata log. -:Type: Integer -:Default: ``64`` - -.. important:: The values of ``rgw_data_log_num_shards`` and - ``rgw_md_log_max_shards`` should not be changed after sync has +.. confval:: rgw_zone +.. confval:: rgw_zonegroup +.. confval:: rgw_realm +.. confval:: rgw_run_sync_thread +.. confval:: rgw_data_log_window +.. confval:: rgw_data_log_changes_size +.. confval:: rgw_data_log_obj_prefix +.. confval:: rgw_data_log_num_shards +.. confval:: rgw_md_log_max_shards + +.. important:: The values of :confval:`rgw_data_log_num_shards` and + :confval:`rgw_md_log_max_shards` should not be changed after sync has started. S3 Settings =========== -``rgw_s3_auth_use_ldap`` - -:Description: Should S3 authentication use LDAP? -:Type: Boolean -:Default: ``false`` - +.. confval:: rgw_s3_auth_use_ldap Swift Settings ============== -``rgw_enforce_swift_acls`` - -:Description: Enforces the Swift Access Control List (ACL) settings. -:Type: Boolean -:Default: ``true`` - - -``rgw_swift_token_expiration`` - -:Description: The time in seconds for expiring a Swift token. -:Type: Integer -:Default: ``24 * 3600`` - - -``rgw_swift_url`` - -:Description: The URL for the Ceph Object Gateway Swift API. -:Type: String -:Default: None - - -``rgw_swift_url_prefix`` - -:Description: The URL prefix for the Swift API, to distinguish it from - the S3 API endpoint. The default is ``swift``, which - makes the Swift API available at the URL - ``http://host:port/swift/v1`` (or - ``http://host:port/swift/v1/AUTH_%(tenant_id)s`` if - ``rgw swift account in url`` is enabled). - - For compatibility, setting this configuration variable - to the empty string causes the default ``swift`` to be - used; if you do want an empty prefix, set this option to - ``/``. - - .. warning:: If you set this option to ``/``, you must - disable the S3 API by modifying ``rgw - enable apis`` to exclude ``s3``. It is not - possible to operate radosgw with ``rgw - swift url prefix = /`` and simultaneously - support both the S3 and Swift APIs. If you - do need to support both APIs without - prefixes, deploy multiple radosgw instances - to listen on different hosts (or ports) - instead, enabling some for S3 and some for - Swift. -:Default: ``swift`` -:Example: "/swift-testing" - - -``rgw_swift_auth_url`` - -:Description: Default URL for verifying v1 auth tokens (if not using internal - Swift auth). - -:Type: String -:Default: None - - -``rgw_swift_auth_entry`` - -:Description: The entry point for a Swift auth URL. -:Type: String -:Default: ``auth`` - - -``rgw_swift_account_in_url`` - -:Description: Whether or not the Swift account name should be included - in the Swift API URL. - - If set to ``false`` (the default), then the Swift API - will listen on a URL formed like - ``http://host:port//v1``, and the - account name (commonly a Keystone project UUID if - radosgw is configured with `Keystone integration - <../keystone>`_) will be inferred from request - headers. - - If set to ``true``, the Swift API URL will be - ``http://host:port//v1/AUTH_`` - (or - ``http://host:port//v1/AUTH_``) - instead, and the Keystone ``object-store`` endpoint must - accordingly be configured to include the - ``AUTH_%(tenant_id)s`` suffix. - - You **must** set this option to ``true`` (and update the - Keystone service catalog) if you want radosgw to support - publicly-readable containers and `temporary URLs - <../swift/tempurl>`_. -:Type: Boolean -:Default: ``false`` - - -``rgw_swift_versioning_enabled`` - -:Description: Enables the Object Versioning of OpenStack Object Storage API. - This allows clients to put the ``X-Versions-Location`` attribute - on containers that should be versioned. The attribute specifies - the name of container storing archived versions. It must be owned - by the same user that the versioned container due to access - control verification - ACLs are NOT taken into consideration. - Those containers cannot be versioned by the S3 object versioning - mechanism. - - A slightly different attribute, ``X-History-Location``, which is also understood by - `OpenStack Swift `_ - for handling ``DELETE`` operations, is currently not supported. -:Type: Boolean -:Default: ``false`` - - -``rgw_trust_forwarded_https`` - -:Description: When a proxy in front of radosgw is used for ssl termination, radosgw - does not know whether incoming http connections are secure. Enable - this option to trust the ``Forwarded`` and ``X-Forwarded-Proto`` headers - sent by the proxy when determining whether the connection is secure. - This is required for some features, such as server side encryption. - (Never enable this setting if you do not have a trusted proxy in front of - radosgw, or else malicious users will be able to set these headers in - any request.) -:Type: Boolean -:Default: ``false`` - - +.. confval:: rgw_enforce_swift_acls +.. confval:: rgw_swift_tenant_name +.. confval:: rgw_swift_token_expiration +.. confval:: rgw_swift_url +.. confval:: rgw_swift_url_prefix +.. confval:: rgw_swift_auth_url +.. confval:: rgw_swift_auth_entry +.. confval:: rgw_swift_account_in_url +.. confval:: rgw_swift_versioning_enabled +.. confval:: rgw_trust_forwarded_https Logging Settings ================ - -``rgw_log_nonexistent_bucket`` - -:Description: Enables Ceph Object Gateway to log a request for a non-existent - bucket. - -:Type: Boolean -:Default: ``false`` - - -``rgw_log_object_name`` - -:Description: The logging format for an object name. See ma npage - :manpage:`date` for details about format specifiers. - -:Type: Date -:Default: ``%Y-%m-%d-%H-%i-%n`` - - -``rgw_log_object_name_utc`` - -:Description: Whether a logged object name includes a UTC time. - If ``false``, it uses the local time. - -:Type: Boolean -:Default: ``false`` - - -``rgw_usage_max_shards`` - -:Description: The maximum number of shards for usage logging. -:Type: Integer -:Default: ``32`` - - -``rgw_usage_max_user_shards`` - -:Description: The maximum number of shards used for a single user's - usage logging. - -:Type: Integer -:Default: ``1`` - - -``rgw_enable_ops_log`` - -:Description: Enable logging for each successful Ceph Object Gateway operation. -:Type: Boolean -:Default: ``false`` - - -``rgw_enable_usage_log`` - -:Description: Enable the usage log. -:Type: Boolean -:Default: ``false`` - - -``rgw_ops_log_rados`` - -:Description: Whether the operations log should be written to the - Ceph Storage Cluster backend. - -:Type: Boolean -:Default: ``true`` - - -``rgw_ops_log_socket_path`` - -:Description: The Unix domain socket for writing operations logs. -:Type: String -:Default: None - - -``rgw_ops_log_data_backlog`` - -:Description: The maximum data backlog data size for operations logs written - to a Unix domain socket. - -:Type: Integer -:Default: ``5 << 20`` - - -``rgw_usage_log_flush_threshold`` - -:Description: The number of dirty merged entries in the usage log before - flushing synchronously. - -:Type: Integer -:Default: 1024 - - -``rgw_usage_log_tick_interval`` - -:Description: Flush pending usage log data every ``n`` seconds. -:Type: Integer -:Default: ``30`` - - -``rgw_log_http_headers`` - -:Description: Comma-delimited list of HTTP headers to include with ops - log entries. Header names are case insensitive, and use - the full header name with words separated by underscores. - -:Type: String -:Default: None -:Example: "http_x_forwarded_for, http_x_special_k" - - -``rgw_intent_log_object_name`` - -:Description: The logging format for the intent log object name. See the manpage - :manpage:`date` for details about format specifiers. - -:Type: Date -:Default: ``%Y-%m-%d-%i-%n`` - - -``rgw_intent_log_object_name_utc`` - -:Description: Whether the intent log object name uses UTC time. - If ``false``, it uses the local time. - -:Type: Boolean -:Default: ``false`` - - +.. confval:: rgw_log_nonexistent_bucket +.. confval:: rgw_log_object_name +.. confval:: rgw_log_object_name_utc +.. confval:: rgw_usage_max_shards +.. confval:: rgw_usage_max_user_shards +.. confval:: rgw_enable_ops_log +.. confval:: rgw_enable_usage_log +.. confval:: rgw_ops_log_rados +.. confval:: rgw_ops_log_socket_path +.. confval:: rgw_ops_log_data_backlog +.. confval:: rgw_usage_log_flush_threshold +.. confval:: rgw_usage_log_tick_interval +.. confval:: rgw_log_http_headers Keystone Settings ================= @@ -873,7 +234,7 @@ QoS settings .. versionadded:: Nautilus The ``civetweb`` frontend has a threading model that uses a thread per -connection and hence is automatically throttled by ``rgw_thread_pool_size`` +connection and hence is automatically throttled by :confval:`rgw_thread_pool_size` configurable when it comes to accepting connections. The newer ``beast`` frontend is not restricted by the thread pool size when it comes to accepting new connections, so a scheduler abstraction is introduced in the Nautilus release diff --git a/src/common/options/rgw.yaml.in b/src/common/options/rgw.yaml.in index 5c3ab21335ca8..bc25ad8d7db66 100644 --- a/src/common/options/rgw.yaml.in +++ b/src/common/options/rgw.yaml.in @@ -154,6 +154,12 @@ options: value overrides bucket_index_max_shards stored in the zone. Setting this value in the zone is preferred, because it applies globally to all radosgw daemons running in the zone. + fmt_desc: Represents the number of shards for the bucket index object, + a value of zero indicates there is no sharding. It is not + recommended to set a value too large (e.g. thousand) as it + increases the cost for bucket listing. + This variable should be set in the client or global sections + so that it is automatically applied to radosgw-admin commands. default: 0 services: - rgw @@ -221,8 +227,8 @@ options: type: str level: advanced desc: Alternative location for RGW configuration. - long_desc: 'If this is set, the different Ceph system configurables (such as the - keyring file will be located in the path that is specified here. ' + long_desc: If this is set, the different Ceph system configurables (such as the keyring file will be located in the path that is specified here. + fmt_desc: Sets the location of the data files for Ceph RADOS Gateway. default: /var/lib/ceph/radosgw/$cluster-$id services: - rgw @@ -233,6 +239,13 @@ options: type: str level: advanced desc: A list of set of RESTful APIs that rgw handles. + fmt_desc: | + Enables the specified APIs. + + .. note:: Enabling the ``s3`` API is a requirement for + any ``radosgw`` instance that is meant to + participate in a `multi-site <../multisite>`_ + configuration. default: s3, s3website, swift, swift_auth, admin, sts, iam, notifications services: - rgw @@ -245,6 +258,7 @@ options: requests. Metadata entries can be user info, bucket info, and bucket instance info. If not found in the cache, entries will be fetched from the backing RADOS store. + fmt_desc: Whether the Ceph Object Gateway cache is enabled. default: true services: - rgw @@ -256,6 +270,7 @@ options: level: advanced desc: Max number of items in RGW metadata cache. long_desc: When full, the RGW metadata cache evicts least recently used entries. + fmt_desc: The number of entries in the Ceph Object Gateway cache. default: 10000 services: - rgw @@ -268,6 +283,7 @@ options: desc: The host name that RGW uses. long_desc: This is Needed for virtual hosting of buckets to work properly, unless configured via zonegroup configuration. + fmt_desc: The DNS name of the served domain. See also the ``hostnames`` setting within regions. services: - rgw with_legacy: true @@ -304,6 +320,8 @@ options: desc: Multiple content length headers compatibility long_desc: Try to handle requests with abiguous multiple content length headers (Content-Length, Http-Content-Length). + fmt_desc: Enable compatibility handling of FCGI requests with both ``CONTENT_LENGTH`` + and ``HTTP_CONTENT_LENGTH`` set. default: false services: - rgw @@ -347,6 +365,9 @@ options: desc: Number of LCWorker tasks that will be run in parallel long_desc: Number of LCWorker tasks that will run in parallel--used to permit >1 bucket/index shards to be processed simultaneously + fmt_desc: This option specifies the number of lifecycle worker threads + to run in parallel, thereby processing bucket and index + shards simultaneously. default: 3 services: - rgw @@ -357,6 +378,8 @@ options: desc: Number of workpool threads per LCWorker long_desc: Number of threads in per-LCWorker workpools--used to accelerate per-bucket processing + fmt_desc: This option specifies the number of threads in each lifecycle + workers work pool. This option can help accelerate processing each bucket. default: 3 services: - rgw @@ -400,12 +423,16 @@ options: - name: rgw_script_uri type: str level: dev + fmt_desc: The alternative value for the ``SCRIPT_URI`` if not set + in the request. services: - rgw with_legacy: true - name: rgw_request_uri type: str level: dev + fmt_desc: The alternative value for the ``REQUEST_URI`` if not set + in the request. services: - rgw with_legacy: true @@ -425,6 +452,7 @@ options: desc: Swift-auth storage URL long_desc: Used in conjunction with rgw internal swift authentication. This affects the X-Storage-Url response header value. + fmt_desc: The URL for the Ceph Object Gateway Swift API. services: - rgw see_also: @@ -435,6 +463,31 @@ options: level: advanced desc: Swift URL prefix long_desc: The URL path prefix for swift requests. + fmt_desc: | + The URL prefix for the Swift API, to distinguish it from + the S3 API endpoint. The default is ``swift``, which + makes the Swift API available at the URL + ``http://host:port/swift/v1`` (or + ``http://host:port/swift/v1/AUTH_%(tenant_id)s`` if + ``rgw swift account in url`` is enabled). + + For compatibility, setting this configuration variable + to the empty string causes the default ``swift`` to be + used; if you do want an empty prefix, set this option to + ``/``. + + .. warning:: If you set this option to ``/``, you must + disable the S3 API by modifying ``rgw + enable apis`` to exclude ``s3``. It is not + possible to operate radosgw with ``rgw + swift url prefix = /`` and simultaneously + support both the S3 and Swift APIs. If you + do need to support both APIs without + prefixes, deploy multiple radosgw instances + to listen on different hosts (or ports) + instead, enabling some for S3 and some for + Swift. + example: /swift-testing default: swift services: - rgw @@ -453,6 +506,7 @@ options: level: advanced desc: Swift auth URL prefix long_desc: URL path prefix for internal swift auth requests. + fmt_desc: The entry point for a Swift auth URL. default: auth services: - rgw @@ -474,6 +528,27 @@ options: level: advanced desc: Swift account encoded in URL long_desc: Whether the swift account is encoded in the uri path (AUTH_). + fmt_desc: | + Whether or not the Swift account name should be included + in the Swift API URL. + If set to ``false`` (the default), then the Swift API + will listen on a URL formed like + ``http://host:port//v1``, and the + account name (commonly a Keystone project UUID if + radosgw is configured with `Keystone integration + <../keystone>`_) will be inferred from request + headers. + If set to ``true``, the Swift API URL will be + ``http://host:port//v1/AUTH_`` + (or + ``http://host:port//v1/AUTH_``) + instead, and the Keystone ``object-store`` endpoint must + accordingly be configured to include the + ``AUTH_%(tenant_id)s`` suffix. + You **must** set this option to ``true`` (and update the + Keystone service catalog) if you want radosgw to support + publicly-readable containers and `temporary URLs + <../swift/tempurl>`_. default: false services: - rgw @@ -839,6 +914,7 @@ options: type: str level: advanced desc: Path prefix to be used for accessing RGW RESTful admin API. + fmt_desc: The entry point for an admin request URL. default: admin services: - rgw @@ -849,6 +925,7 @@ options: desc: RGW enforce swift acls long_desc: Should RGW enforce special Swift-only ACLs. Swift has a special ACL that gives permission to access all objects in a container. + fmt_desc: Enforces the Swift Access Control List (ACL) settings. default: true services: - rgw @@ -857,6 +934,7 @@ options: type: int level: advanced desc: Expiration time (in seconds) for token generated through RGW Swift auth. + fmt_desc: The time in seconds for expiring a Swift token. default: 1_day services: - rgw @@ -867,6 +945,7 @@ options: desc: RGW support of 100-continue long_desc: Should RGW explicitly send 100 (continue) responses. This is mainly relevant when using FastCGI, as some FastCGI modules do not fully support this feature. + fmt_desc: Enable ``100-continue`` if it is operational. default: true services: - rgw @@ -890,6 +969,9 @@ options: not at the originator's address. Therefore it is sometimes possible to have the proxy add the originator's address in a separate HTTP header, which will allow RGW to log it correctly. + fmt_desc: The remote address parameter. For example, the HTTP field + containing the remote address, or the ``X-Forwarded-For`` + address if a reverse proxy is operational. default: REMOTE_ADDR services: - rgw @@ -900,6 +982,7 @@ options: type: int level: dev desc: Timeout for async rados coroutine operations. + fmt_desc: The timeout in seconds for open threads. default: 10_min services: - rgw @@ -908,6 +991,8 @@ options: type: int level: dev default: 0 + fmt_desc: The time ``timeout`` in seconds before a Ceph Object Gateway + process dies. Disabled if set to ``0``. services: - rgw with_legacy: true @@ -919,6 +1004,7 @@ options: when using either the civetweb, or the fastcgi frontends. The higher this number is, RGW will be able to deal with more concurrent requests at the cost of more resource utilization. + fmt_desc: The size of the thread pool. default: 512 services: - rgw @@ -932,6 +1018,8 @@ options: info that is being sent when metadata is modified (such as user or bucket information). A higher number of control objects allows better concurrency of these messages, at the cost of more resource utilization. + fmt_desc: The number of notification objects used for cache synchronization + between different ``rgw`` instances. default: 8 services: - rgw @@ -943,6 +1031,7 @@ options: long_desc: RGW can send requests to other RGW servers (e.g., in multi-site sync work). This configurable selects whether RGW should verify the certificate for the remote peer and host. + fmt_desc: Verify SSL certificates while making requests. default: true services: - rgw @@ -1082,6 +1171,9 @@ options: type: str level: advanced desc: Zone name + fmt_desc: The name of the zone for the gateway instance. If no zone is + set, a cluster-wide default can be configured with the command + ``radosgw-admin zone default``. services: - rgw see_also: @@ -1147,6 +1239,9 @@ options: type: str level: advanced desc: Zonegroup name + fmt_desc: The name of the zonegroup for the gateway instance. If no + zonegroup is set, a cluster-wide default can be configured with + the command ``radosgw-admin zonegroup default``. services: - rgw see_also: @@ -1177,6 +1272,9 @@ options: - name: rgw_realm type: str level: advanced + fmt_desc: The name of the realm for the gateway instance. If no realm is + set, a cluster-wide default can be configured with the command + ``radosgw-admin realm default``. services: - rgw with_legacy: true @@ -1228,6 +1326,8 @@ options: long_desc: This config option applies to the ops log. When this option is set, the ops log will log operations that are sent to non existing buckets. These operations inherently fail, and do not correspond to a specific user. + fmt_desc: Enables Ceph Object Gateway to log a request for a non-existent + bucket. default: false services: - rgw @@ -1241,6 +1341,8 @@ options: desc: Ops log object name format long_desc: Defines the format of the RADOS objects names that ops log uses to store ops log data + fmt_desc: The logging format for an object name. See ma npage + :manpage:`date` for details about format specifiers. default: '%Y-%m-%d-%H-%i-%n' services: - rgw @@ -1253,6 +1355,8 @@ options: desc: Should ops log object name based on UTC long_desc: If set, the names of the RADOS objects that hold the ops log data will be based on UTC time zone. If not set, it will use the local time zone. + fmt_desc: Whether a logged object name includes a UTC time. + If ``false``, it uses the local time. default: false services: - rgw @@ -1266,6 +1370,7 @@ options: desc: Number of shards for usage log. long_desc: The number of RADOS objects that RGW will use in order to store the usage log data. + fmt_desc: The maximum number of shards for usage logging. default: 32 services: - rgw @@ -1277,6 +1382,8 @@ options: level: advanced desc: Number of shards for single user in usage log long_desc: The number of shards that a single user will span over in the usage log. + fmt_desc: The maximum number of shards used for a single user's + usage logging. default: 1 services: - rgw @@ -1289,6 +1396,7 @@ options: type: bool level: advanced desc: Enable ops log + fmt_desc: Enable logging for each successful Ceph Object Gateway operation. default: false services: - rgw @@ -1302,7 +1410,7 @@ options: - name: rgw_enable_usage_log type: bool level: advanced - desc: Enable usage log + desc: Enable the usage log default: false services: - rgw @@ -1315,6 +1423,8 @@ options: level: advanced desc: Use RADOS for ops log long_desc: If set, RGW will store ops log information in RADOS. + fmt_desc: Whether the operations log should be written to the + Ceph Storage Cluster backend. default: true services: - rgw @@ -1328,6 +1438,7 @@ options: desc: Unix domain socket path for ops log. long_desc: Path to unix domain socket that RGW will listen for connection on. When connected, RGW will send ops log data through it. + fmt_desc: The Unix domain socket for writing operations logs. services: - rgw see_also: @@ -1343,6 +1454,8 @@ options: to send info through unix domain socket. When data backlog is higher than this, ops log entries will be lost. In order to avoid ops log information loss, the listener needs to clear data (by reading it) quickly enough. + fmt_desc: The maximum data backlog data size for operations logs written + to a Unix domain socket. default: 5_M services: - rgw @@ -1358,6 +1471,8 @@ options: before it will be flushed to the backend. Note that the usage log is periodically flushed, even if number of entries does not reach this threshold. A usage log entry corresponds to one or more operations on a single bucket.i + fmt_desc: The number of dirty merged entries in the usage log before + flushing synchronously. default: 1024 services: - rgw @@ -1372,6 +1487,7 @@ options: long_desc: The number of seconds between consecutive usage log flushes. The usage log will also flush itself to the backend if the number of pending entries reaches a certain threshold. + fmt_desc: Flush pending usage log data every ``n`` seconds. default: 30 services: - rgw @@ -1386,6 +1502,8 @@ options: long_desc: The time length (in seconds) that RGW will allow for its initialization. RGW process will give up and quit if initialization is not complete after this amount of time. + fmt_desc: The number of seconds before Ceph Object Gateway gives up on + initialization. default: 5_min services: - rgw @@ -1397,6 +1515,8 @@ options: long_desc: The mime types file is needed in Swift when uploading an object. If object's content type is not specified, RGW will use data from this file to assign a content type to the object. + fmt_desc: The path and location of the MIME-types file. Used for Swift + auto-detection of object types. default: /etc/mime.types services: - rgw @@ -1407,6 +1527,9 @@ options: desc: Number of shards for garbage collector data long_desc: The number of garbage collector data shards, is the number of RADOS objects that RGW will use to store the garbage collection information on. + fmt_desc: The maximum number of objects that may be handled by + garbage collection in one garbage collection processing cycle. + Please do not change this value after the first deployment. default: 32 services: - rgw @@ -1425,6 +1548,8 @@ options: purging a deleted object's data. RGW will not remove object immediately, as object could still have readers. A mechanism exists to increase the object's expiration time when it's being read. The recommended value of its lower limit is 30 minutes + fmt_desc: The minimum wait time before a deleted object may be removed + and handled by garbage collection processing. default: 2_hr services: - rgw @@ -1445,6 +1570,8 @@ options: seconds) that RGW is allowed to hold that lease. In the case where RGW goes down uncleanly, this is the amount of time where processing of that data shard will be blocked. + fmt_desc: The maximum time between the beginning of two consecutive garbage + collection processing cycles. default: 1_hr services: - rgw @@ -1461,6 +1588,7 @@ options: long_desc: The amount of time between the start of consecutive runs of the garbage collector threads. If garbage collector runs takes more than this period, it will not wait before running again. + fmt_desc: The cycle time for garbage collection processing. default: 1_hr services: - rgw @@ -1534,6 +1662,7 @@ options: desc: HTTP return code override for object creation long_desc: If not zero, this is the HTTP return code that will be returned on a successful S3 object creation. + fmt_desc: The alternate success status response for ``create-obj``. default: 0 services: - rgw @@ -1554,6 +1683,8 @@ options: that was sent to a host in another domain. If a CNAME record is configured for that domain it will use it instead. This gives user to have the ability of creating a unique domain of their own to point at data in their bucket. + fmt_desc: Whether ``rgw`` should use DNS CNAME record of the request + hostname field (if hostname is not equal to ``rgw dns name``). default: false services: - rgw @@ -1565,6 +1696,8 @@ options: long_desc: The size of an object stripe for RGW objects. This is the maximum size a backing RADOS object will have. RGW objects that are larger than this will span over multiple objects. + fmt_desc: The size of an object stripe for Ceph Object Gateway objects. + See `Architecture`_ for details on striping. default: 4_M services: - rgw @@ -1577,8 +1710,14 @@ options: long_desc: Add new set of attributes that could be set on an object. These extra attributes can be set through HTTP header fields when putting the objects. If set, these attributes will return as HTTP fields when doing GET/HEAD on the object. + fmt_desc: Add new set of attributes that could be set on an entity + (user, bucket or object). These extra attributes can be set + through HTTP header fields when putting the entity or modifying + it using POST method. If set, these attributes will return as + HTTP fields when doing GET/HEAD on the entity. services: - rgw + example: content_foo, content_bar, x-foo-bar with_legacy: true - name: rgw_exit_timeout_secs type: int @@ -1603,6 +1742,8 @@ options: level: advanced desc: RGW object read chunk size long_desc: The maximum request size of a single object read operation sent to RADOS + fmt_desc: The maximum request size of a single get operation sent to the + Ceph Storage Cluster. default: 4_M services: - rgw @@ -1612,6 +1753,7 @@ options: level: advanced desc: RGW enable relaxed S3 bucket names long_desc: RGW enable relaxed S3 bucket name rules for US region buckets. + fmt_desc: Enables relaxed S3 bucket names rules for US region buckets. default: false services: - rgw @@ -1633,6 +1775,8 @@ options: long_desc: When RGW fetches lists of user's buckets from the backend, this is the max number of entries it will try to retrieve in a single operation. Note that the backend may choose to return a smaller number of entries. + fmt_desc: The maximum number of buckets to retrieve in a single operation + when listing user buckets. default: 1000 services: - rgw @@ -1644,6 +1788,7 @@ options: long_desc: The number of shards the RGW metadata log entries will reside in. This affects the metadata sync parallelism as a shard can only be processed by a single RGW at a time + fmt_desc: The maximum number of shards for the metadata log. default: 64 services: - rgw @@ -1663,6 +1808,7 @@ options: type: int level: dev default: 1000 + fmt_desc: The timeout in milliseconds for certain ``curl`` calls. services: - rgw with_legacy: true @@ -1690,7 +1836,8 @@ options: type: bool level: advanced desc: Send progress report through copy operation - long_desc: 'If true, RGW will send progress information when copy operation is executed. ' + long_desc: If true, RGW will send progress information when copy operation is executed. + fmt_desc: Enables output of object progress during long copy operations. default: true services: - rgw @@ -1699,6 +1846,7 @@ options: type: size level: advanced desc: Send copy-object progress info after these many bytes + fmt_desc: The minimum bytes between copy progress output. default: 1_M services: - rgw @@ -1733,6 +1881,7 @@ options: long_desc: The data log keeps information about buckets that have objectst that were modified within a specific timeframe. The sync process then knows which buckets are needed to be scanned for data sync. + fmt_desc: The data log entries window in seconds. default: 30 services: - rgw @@ -1743,6 +1892,7 @@ options: desc: Max size of pending changes in data log long_desc: RGW will trigger update to the data log if the number of pending entries reached this number. + fmt_dsec: The number of in-memory entries to hold for the data changes log. default: 1000 services: - rgw @@ -1754,6 +1904,8 @@ options: long_desc: The number of shards the RGW data log entries will reside in. This affects the data sync parallelism as a shard can only be processed by a single RGW at a time. + fmt_desc: The number of shards (objects) on which to keep the + data changes log. default: 128 services: - rgw @@ -1762,6 +1914,7 @@ options: type: str level: dev default: data_log + fmt_desc: The object name prefix for the data log. services: - rgw with_legacy: true @@ -1770,6 +1923,9 @@ options: level: advanced desc: Bucket quota stats cache TTL long_desc: Length of time for bucket stats to be cached within RGW instance. + fmt_desc: The amount of time in seconds cached quota information is + trusted. After this timeout, the quota information will be + re-fetched from the cluster. default: 10_min services: - rgw @@ -1801,6 +1957,10 @@ options: desc: Default quota for max objects in a bucket long_desc: The default quota configuration for max number of objects in a bucket. A negative number means 'unlimited'. + fmt_desc: Default max number of objects per bucket. Set on new users, + if no other quota is specified. Has no effect on existing users. + This variable should be set in the client or global sections + so that it is automatically applied to radosgw-admin commands. default: -1 services: - rgw @@ -1811,6 +1971,8 @@ options: desc: Default quota for total size in a bucket long_desc: The default quota configuration for total size of objects in a bucket. A negative number means 'unlimited'. + fmt_desc: Default max capacity per bucket, in bytes. Set on new users, + if no other quota is specified. Has no effect on existing users. default: -1 services: - rgw @@ -1832,6 +1994,11 @@ options: long_desc: A comma delimited list of frontends configuration. Each configuration contains the type of the frontend followed by an optional space delimited set of key=value config parameters. + fmt_desc: Configures the HTTP frontend(s). The configuration for multiple + frontends can be provided in a comma-delimited list. Each frontend + configuration may include a list of options separated by spaces, + where each option is in the form "key=value" or "key". See + `HTTP Frontends`_ for more on supported options. default: beast port=7480 services: - rgw @@ -1862,6 +2029,10 @@ options: level: advanced desc: User quota bucket sync interval long_desc: Time period for accumulating modified buckets before syncing these stats. + fmt_desc: The amount of time in seconds bucket quota information is + accumulated before syncing to the cluster. During this time, + other RGW instances will not see the changes in bucket quota + stats from operations on this instance. default: 3_min services: - rgw @@ -1872,6 +2043,10 @@ options: desc: User quota sync interval long_desc: Time period for accumulating modified buckets before syncing entire user stats. + fmt_desc: The amount of time in seconds user quota information is + accumulated before syncing to the cluster. During this time, + other RGW instances will not see the changes in user quota stats + from operations on this instance. default: 1_day services: - rgw @@ -1900,6 +2075,9 @@ options: desc: User quota max objects long_desc: The default quota configuration for total number of objects for a single user. A negative number means 'unlimited'. + fmt_desc: Default max number of objects for a user. This includes all + objects in all buckets owned by the user. Set on new users, + if no other quota is specified. Has no effect on existing users. default: -1 services: - rgw @@ -1910,6 +2088,8 @@ options: desc: User quota max size long_desc: The default quota configuration for total size of objects for a single user. A negative number means 'unlimited'. + fmt_desc: The value for user max size quota in bytes set on new users, + if no other quota is specified. Has no effect on existing users. default: -1 services: - rgw @@ -2014,6 +2194,10 @@ options: desc: List of HTTP headers to log long_desc: A comma delimited list of HTTP headers to log when seen, ignores case (e.g., http_x_forwarded_for). + fmt_desc: Comma-delimited list of HTTP headers to include with ops + log entries. Header names are case insensitive, and use + the full header name with words separated by underscores. + example: http_x_forwarded_for, http_x_special_k services: - rgw with_legacy: true @@ -2043,6 +2227,8 @@ options: type: bool level: advanced desc: Should run sync thread + fmt_desc: If there are other zones in the realm to sync from, spawn threads + to handle the sync of data and metadata. default: true services: - rgw @@ -2193,6 +2379,19 @@ options: type: bool level: advanced desc: Enable Swift versioning + fmt_desc: | + Enables the Object Versioning of OpenStack Object Storage API. + This allows clients to put the ``X-Versions-Location`` attribute + on containers that should be versioned. The attribute specifies + the name of container storing archived versions. It must be owned + by the same user that the versioned container due to access + control verification - ACLs are NOT taken into consideration. + Those containers cannot be versioned by the S3 object versioning + mechanism. + + A slightly different attribute, ``X-History-Location``, which is also understood by + `OpenStack Swift `_ + for handling ``DELETE`` operations, is currently not supported. default: false services: - rgw @@ -2274,6 +2473,14 @@ options: server side encryption. (Never enable this setting if you do not have a trusted proxy in front of radosgw, or else malicious users will be able to set these headers in any request.) + fmt_desc: When a proxy in front of radosgw is used for ssl termination, radosgw + does not know whether incoming http connections are secure. Enable + this option to trust the ``Forwarded`` and ``X-Forwarded-Proto`` headers + sent by the proxy when determining whether the connection is secure. + This is required for some features, such as server side encryption. + (Never enable this setting if you do not have a trusted proxy in front of + radosgw, or else malicious users will be able to set these headers in + any request.) default: false services: - rgw