<https://docs.aws.amazon.com/cli/latest/reference/s3api/select-object-content.html>`_
It passes the authentication and permission process as an incoming message
-(POST). **RGWSelectObj_ObjStore_S3::send_response_data** is the “entry point”,
+(POST). **RGWSelectObj_ObjStore_S3::send_response_data** is the “entry point”,
it handles each fetched chunk according to input object-key.
**send_response_data** is first handling the input query, it extracts the query
and other CLI parameters.
-Per each new fetched chunk (~4m), RGW executes an s3-select query on it. The
+Per each new fetched chunk (~4m), RGW executes an s3-select query on it. The
current implementation supports CSV objects and since chunks are randomly
“cutting” the CSV rows in the middle, those broken-lines (first or last per
chunk) are skipped while processing the query. Those “broken” lines are
Error Handling
~~~~~~~~~~~~~~
- | Upon an error being detected, RGW returns 400-Bad-Request and a specific error message sends back to the client.
- | Currently, there are 2 main types of error.
- |
- | **Syntax error**: the s3select parser rejects user requests that are not aligned with parser syntax definitions, as
- | described in this documentation.
- | Upon Syntax Error, the engine creates an error message that points to the location of the error.
- | RGW sends back the error message in a specific error response.
- |
- | **Processing Time error**: the runtime engine may detect errors that occur only on processing time, for that type of
- | error, a different error message would describe that.
- | RGW sends back the error message in a specific error response.
+Upon an error being detected, RGW returns 400-Bad-Request and a specific error message sends back to the client.
+Currently, there are 2 main types of error.
+**Syntax error**: the s3select parser rejects user requests that are not aligned with parser syntax definitions, as
+described in this documentation.
+Upon Syntax Error, the engine creates an error message that points to the location of the error.
+RGW sends back the error message in a specific error response.
+
+**Processing Time error**: the runtime engine may detect errors that occur only on processing time, for that type of
+error, a different error message would describe that.
+RGW sends back the error message in a specific error response.
.. _s3-select-feature-table:
Features Support
----------------
- | Currently only part of `AWS select command <https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-select.html>`_ is implemented, table below describes what is currently supported.
- | The following table describes the current implementation for s3-select functionalities:
+Currently only part of `AWS select command
+<https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-select.html>`_
+is implemented, table below describes what is currently supported.
+
+The following table describes the current implementation for s3-select
+functionalities:
+---------------------------------+-----------------+-----------------------------------------------------------------------+
| Feature | Detailed | Example / Description |
NULL
~~~~
-| NULL is a legit value in ceph-s3select systems similar to other DB systems, i.e. systems needs to handle the case where a value is NULL.
-| The definition of NULL in our context, is missing/unknown, in that sense **NULL can not produce a value on ANY arithmetic operations** ( a + NULL will produce NULL value).
-| The Same is with arithmetic comparison, **any comparison to NULL is NULL**, i.e. unknown.
-| Below is a truth table contains the NULL use-case.
+NULL is a legit value in ceph-s3select systems similar to other DB systems, i.e. systems needs to handle the case where a value is NULL.
+
+The definition of NULL in our context, is missing/unknown, in that sense **NULL can not produce a value on ANY arithmetic operations** ( a + NULL will produce NULL value).
+
+The Same is with arithmetic comparison, **any comparison to NULL is NULL**, i.e. unknown.
+Below is a truth table contains the NULL use-case.
+---------------------------------+-----------------------------+
| A is NULL | Result (NULL=UNKNOWN) |
Timestamp Functions
~~~~~~~~~~~~~~~~~~~
- | The timestamp functionalities as described in `AWS-specs <https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-date.html>`_ is fully implemented.
-
- | ``to_timestamp( string )`` : The casting operator converts string to timestamp basic type.
- | to_timestamp operator is able to convert the following ``YYYY-MM-DDTHH:mm:ss.SSSSSS+/-HH:mm`` , ``YYYY-MM-DDTHH:mm:ss.SSSSSSZ`` , ``YYYY-MM-DDTHH:mm:ss+/-HH:mm`` , ``YYYY-MM-DDTHH:mm:ssZ`` , ``YYYY-MM-DDTHH:mm+/-HH:mm`` , ``YYYY-MM-DDTHH:mmZ`` , ``YYYY-MM-DDT`` or ``YYYYT`` string formats into timestamp.
- | Where time (or part of it) is missing in the string format, zero's are replacing the missing parts. And for missing month and day, 1 is default value for them.
- | Timezone part is in format ``+/-HH:mm`` or ``Z`` , where the letter "Z" indicates Coordinated Universal Time (UTC). Value of timezone can range between -12:00 and +14:00.
-
- | ``extract(date-part from timestamp)`` : The function extracts date-part from input timestamp and returns it as integer.
- | Supported date-part : year, month, week, day, hour, minute, second, timezone_hour, timezone_minute.
-
- | ``date_add(date-part, quantity, timestamp)`` : The function adds quantity (integer) to date-part of timestamp and returns result as timestamp. It also includes timezone in calculation.
- | Supported data-part : year, month, day, hour, minute, second.
-
- | ``date_diff(date-part, timestamp, timestamp)`` : The function returns an integer, a calculated result for difference between 2 timestamps according to date-part. It includes timezone in calculation.
- | supported date-part : year, month, day, hour, minute, second.
-
- | ``utcnow()`` : return timestamp of current time.
-
- | ``to_string(timestamp, format_pattern)`` : returns a string representation of the input timestamp in the given input string format.
+The timestamp functionalities as described in `AWS-specs
+<https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-date.html>`_
+is fully implemented.
+
+ ``to_timestamp( string )`` : The casting operator converts string to timestamp
+ basic type. to_timestamp operator is able to convert the following
+ ``YYYY-MM-DDTHH:mm:ss.SSSSSS+/-HH:mm`` , ``YYYY-MM-DDTHH:mm:ss.SSSSSSZ`` ,
+ ``YYYY-MM-DDTHH:mm:ss+/-HH:mm`` , ``YYYY-MM-DDTHH:mm:ssZ`` ,
+ ``YYYY-MM-DDTHH:mm+/-HH:mm`` , ``YYYY-MM-DDTHH:mmZ`` , ``YYYY-MM-DDT`` or
+ ``YYYYT`` string formats into timestamp. Where time (or part of it) is
+ missing in the string format, zero's are replacing the missing parts. And for
+ missing month and day, 1 is default value for them. Timezone part is in
+ format ``+/-HH:mm`` or ``Z`` , where the letter "Z" indicates Coordinated
+ Universal Time (UTC). Value of timezone can range between -12:00 and +14:00.
+
+ ``extract(date-part from timestamp)`` : The function extracts date-part from
+ input timestamp and returns it as integer. Supported date-part : year, month,
+ week, day, hour, minute, second, timezone_hour, timezone_minute.
+
+ ``date_add(date-part, quantity, timestamp)`` : The function adds quantity
+ (integer) to date-part of timestamp and returns result as timestamp. It also
+ includes timezone in calculation. Supported data-part : year, month, day,
+ hour, minute, second.
+
+ ``date_diff(date-part, timestamp, timestamp)`` : The function returns an
+ integer, a calculated result for difference between 2 timestamps according to
+ date-part. It includes timezone in calculation. supported date-part : year,
+ month, day, hour, minute, second.
+
+ ``utcnow()`` : return timestamp of current time.
+
+ ``to_string(timestamp, format_pattern)`` : returns a string representation of
+ the input timestamp in the given input string format.
to_string parameters
~~~~~~~~~~~~~~~~~~~~
Aggregation Functions
~~~~~~~~~~~~~~~~~~~~~
- | ``count()`` : return integer according to number of rows matching condition(if such exist).
+``count()`` : return integer according to number of rows matching condition(if such exist).
- | ``sum(expression)`` : return a summary of expression per all rows matching condition(if such exist).
+``sum(expression)`` : return a summary of expression per all rows matching condition(if such exist).
- | ``avg(expression)`` : return a average of expression per all rows matching condition(if such exist).
+``avg(expression)`` : return a average of expression per all rows matching condition(if such exist).
- | ``max(expression)`` : return the maximal result for all expressions matching condition(if such exist).
+``max(expression)`` : return the maximal result for all expressions matching condition(if such exist).
- | ``min(expression)`` : return the minimal result for all expressions matching condition(if such exist).
+``min(expression)`` : return the minimal result for all expressions matching condition(if such exist).
String Functions
~~~~~~~~~~~~~~~~
- | ``substring(string,from,to)`` : substring( string ``from`` start [ ``for`` length ] )
- | return a string extract from input string according to from,to inputs.
- | ``substring(string from )``
- | ``substring(string from for)``
+``substring(string,from,to)`` : substring( string ``from`` start [ ``for`` length ] )
+return a string extract from input string according to from,to inputs.
+``substring(string from )``
+``substring(string from for)``
- | ``char_length`` : return a number of characters in string (``character_length`` does the same).
+``char_length`` : return a number of characters in string (``character_length`` does the same).
- | ``trim`` : trim ( [[``leading`` | ``trailing`` | ``both`` remove_chars] ``from``] string )
- | trims leading/trailing(or both) characters from target string, the default is blank character.
+``trim`` : trim ( [[``leading`` | ``trailing`` | ``both`` remove_chars] ``from``] string )
+trims leading/trailing(or both) characters from target string, the default is blank character.
- | ``upper\lower`` : converts characters into lowercase/uppercase.
+``upper\lower`` : converts characters into lowercase/uppercase.
SQL Limit Operator
~~~~~~~~~~~~~~~~~~
- | The SQL LIMIT operator is used to limit the number of rows processed by the query.
- | Upon reaching the limit set by the user, the RGW stops fetching additional chunks.
- | TODO : add examples, for aggregation and non-aggregation queries.
+The SQL LIMIT operator is used to limit the number of rows processed by the query.
+Upon reaching the limit set by the user, the RGW stops fetching additional chunks.
+TODO : add examples, for aggregation and non-aggregation queries.
Alias
~~~~~
- | **Alias** programming-construct is an essential part of s3-select language, it enables much better programming especially with objects containing many columns or in the case of complex queries.
+**Alias** programming-construct is an essential part of s3-select language, it enables much better programming especially with objects containing many columns or in the case of complex queries.
- | Upon parsing the statement containing alias construct, it replaces alias with reference to correct projection column, on query execution time the reference is evaluated as any other expression.
+Upon parsing the statement containing alias construct, it replaces alias with reference to correct projection column, on query execution time the reference is evaluated as any other expression.
- | There is a risk that self(or cyclic) reference may occur causing stack-overflow(endless-loop), for that concern upon evaluating an alias, it is validated for cyclic reference.
+There is a risk that self(or cyclic) reference may occur causing stack-overflow(endless-loop), for that concern upon evaluating an alias, it is validated for cyclic reference.
- | Alias also maintains a result cache, meaning that successive uses of a given alias do not evaluate the expression again. The result is instead returned from the cache.
+Alias also maintains a result cache, meaning that successive uses of a given alias do not evaluate the expression again. The result is instead returned from the cache.
- | With each new row the cache is invalidated as the results may then differ.
+With each new row the cache is invalidated as the results may then differ.
Testing
~~~~~~~
- | ``s3select`` contains several testing frameworks which provide a large coverage for its functionalities.
+``s3select`` contains several testing frameworks which provide a large coverage for its functionalities.
- | (1) Tests comparison against a trusted engine, meaning, C/C++ compiler is a trusted expression evaluator,
- | since the syntax for arithmetical and logical expressions are identical (s3select compare to C)
- | the framework runs equal expressions and validates their results.
- | A dedicated expression generator produces different sets of expressions per each new test session.
+(1) Tests comparison against a trusted engine, meaning, C/C++ compiler is a trusted expression evaluator,
+since the syntax for arithmetical and logical expressions are identical (s3select compare to C)
+the framework runs equal expressions and validates their results.
+A dedicated expression generator produces different sets of expressions per each new test session.
- | (2) Compares results of queries whose syntax is different but which are semantically equivalent.
- | This kind of test validates that different runtime flows produce an identical result
- | on each run with a different, random dataset.
+(2) Compares results of queries whose syntax is different but which are semantically equivalent.
+This kind of test validates that different runtime flows produce an identical result
+on each run with a different, random dataset.
- | For example, on a dataset which contains a random numbers(1-1000)
- | the following queries will produce identical results.
- | ``select count(*) from s3object where char_length(_3)=3;``
- | ``select count(*) from s3object where cast(_3 as int)>99 and cast(_3 as int)<1000;``
+For example, on a dataset which contains a random numbers(1-1000)
+the following queries will produce identical results.
+``select count(*) from s3object where char_length(_3)=3;``
+``select count(*) from s3object where cast(_3 as int)>99 and cast(_3 as int)<1000;``
- | (3) Constant dataset, the conventional way of testing. A query is processing a constant dataset, its result is validated against constant results.
+(3) Constant dataset, the conventional way of testing. A query is processing a constant dataset, its result is validated against constant results.
Additional Syntax Support
~~~~~~~~~~~~~~~~~~~~~~~~~
- | S3select syntax supports table-alias ``select s._1 from s3object s where s._2 = ‘4’;``
- |
- | S3select syntax supports case insensitive ``Select SUM(Cast(_1 as int)) FROM S3Object;``
- |
- | S3select syntax supports statements without closing semicolon ``select count(*) from s3object``
+S3select syntax supports table-alias ``select s._1 from s3object s where s._2 = ‘4’;``
+
+S3select syntax supports case insensitive ``Select SUM(Cast(_1 as int)) FROM S3Object;``
+
+S3select syntax supports statements without closing semicolon ``select count(*) from s3object``
Sending Query to RGW
--------------------
- | Any HTTP client can send an ``s3-select`` request to RGW, which must be compliant with `AWS Request syntax <https://docs.aws.amazon.com/AmazonS3/latest/API/API_SelectObjectContent.html#API_SelectObjectContent_RequestSyntax>`_.
+Any HTTP client can send an ``s3-select`` request to RGW, which must be compliant with `AWS Request syntax <https://docs.aws.amazon.com/AmazonS3/latest/API/API_SelectObjectContent.html#API_SelectObjectContent_RequestSyntax>`_.
- | When sending an ``s3-select`` request to RGW using AWS CLI, clients must follow `AWS command reference <https://docs.aws.amazon.com/cli/latest/reference/s3api/select-object-content.html>`_.
- | Below is an example:
+When sending an ``s3-select`` request to RGW using AWS CLI, clients must follow `AWS command reference <https://docs.aws.amazon.com/cli/latest/reference/s3api/select-object-content.html>`_.
+Below is an example:
::
Input Serialization
~~~~~~~~~~~~~~~~~~~
- | **FileHeaderInfo** -> (string)
- | Describes the first line of input. Valid values are:
- |
- | **NONE** : The first line is not a header.
- | **IGNORE** : The first line is a header, but you can't use the header values to indicate the column in an expression.
- | it's possible to use column position (such as _1, _2, …) to indicate the column (``SELECT s._1 FROM S3OBJECT s``).
- | **USE** : First line is a header, and you can use the header value to identify a column in an expression (``SELECT column_name FROM S3OBJECT``).
- |
- | **QuoteEscapeCharacter** -> (string)
- | A single character used for escaping the quotation mark character inside an already escaped value.
- |
- | **RecordDelimiter** -> (string)
- | A single character is used to separate individual records in the input. Instead of the default value, you can specify an arbitrary delimiter.
- |
- | **FieldDelimiter** -> (string)
- | A single character is used to separate individual fields in a record. You can specify an arbitrary delimiter.
+**FileHeaderInfo** -> (string)
+Describes the first line of input. Valid values are:
+
+**NONE** : The first line is not a header.
+**IGNORE** : The first line is a header, but you can't use the header values to indicate the column in an expression.
+it's possible to use column position (such as _1, _2, …) to indicate the column (``SELECT s._1 FROM S3OBJECT s``).
+**USE** : First line is a header, and you can use the header value to identify a column in an expression (``SELECT column_name FROM S3OBJECT``).
+
+**QuoteEscapeCharacter** -> (string)
+A single character used for escaping the quotation mark character inside an already escaped value.
+
+**RecordDelimiter** -> (string)
+A single character is used to separate individual records in the input. Instead of the default value, you can specify an arbitrary delimiter.
+
+**FieldDelimiter** -> (string)
+A single character is used to separate individual fields in a record. You can specify an arbitrary delimiter.
Output Serialization
~~~~~~~~~~~~~~~~~~~~
-| **AWS CLI example**
-
- | aws s3api select-object-content \
- | --bucket "mybucket" \
- | --key keyfile1 \
- | --expression "SELECT * FROM s3object s" \
- | --expression-type 'SQL' \
- | --request-progress '{"Enabled": false}' \
- | --input-serialization '{"CSV": {"FieldDelimiter": ","}, "CompressionType": "NONE"}' \
- | --output-serialization '{"CSV": {"FieldDelimiter": ":", "RecordDelimiter":"\\t", "QuoteFields": "ALWAYS"}}' /dev/stdout
- |
- | **QuoteFields** -> (string)
- | Indicates whether to use quotation marks around output fields.
- | **ALWAYS**: Always use quotation marks for output fields.
- | **ASNEEDED** (not implemented): Use quotation marks for output fields when needed.
- |
- | **RecordDelimiter** -> (string)
- | A single character is used to separate individual records in the output. Instead of the default value, you can specify an
- | arbitrary delimiter.
- |
- | **FieldDelimiter** -> (string)
- | The value used to separate individual fields in a record. You can specify an arbitrary delimiter.
+**AWS CLI example**
+
+ aws s3api select-object-content \
+ --bucket "mybucket" \
+ --key keyfile1 \
+ --expression "SELECT * FROM s3object s" \
+ --expression-type 'SQL' \
+ --request-progress '{"Enabled": false}' \
+ --input-serialization '{"CSV": {"FieldDelimiter": ","}, "CompressionType": "NONE"}' \
+ --output-serialization '{"CSV": {"FieldDelimiter": ":", "RecordDelimiter":"\\t", "QuoteFields": "ALWAYS"}}' /dev/stdout
+
+ **QuoteFields** -> (string)
+ Indicates whether to use quotation marks around output fields.
+ **ALWAYS**: Always use quotation marks for output fields.
+ **ASNEEDED** (not implemented): Use quotation marks for output fields when needed.
+
+ **RecordDelimiter** -> (string)
+ A single character is used to separate individual records in the output. Instead of the default value, you can specify an
+ arbitrary delimiter.
+
+ **FieldDelimiter** -> (string)
+ The value used to separate individual fields in a record. You can specify an arbitrary delimiter.
Scan Range Option
~~~~~~~~~~~~~~~~~
- | The scan range option to AWS-CLI enables the client to scan and process only a selected part of the object.
- | This option reduces input/output operations and bandwidth by skipping parts of the object that are not of interest.
- | TODO : different data-sources (CSV, JSON, Parquet)
+ The scan range option to AWS-CLI enables the client to scan and process only a selected part of the object.
+ This option reduces input/output operations and bandwidth by skipping parts of the object that are not of interest.
+ TODO : different data-sources (CSV, JSON, Parquet)
CSV Parsing Behavior
--------------------
- | The ``s3-select`` engine contains a CSV parser, which parses s3-objects as follows.
- | - Each row ends with ``row-delimiter``.
- | - ``field-separator`` separates adjacent columns, successive instances of ``field separator`` define a NULL column.
- | - ``quote-character`` overrides ``field separator``, meaning that ``field separator`` is treated like any character between quotes.
- | - ``escape character`` disables interpretation of special characters, except for ``row delimiter``.
+ The ``s3-select`` engine contains a CSV parser, which parses s3-objects as follows.
+ - Each row ends with ``row-delimiter``.
+ - ``field-separator`` separates adjacent columns, successive instances of ``field separator`` define a NULL column.
+ - ``quote-character`` overrides ``field separator``, meaning that ``field separator`` is treated like any character between quotes.
+ - ``escape character`` disables interpretation of special characters, except for ``row delimiter``.
- | Below are examples of CSV parsing rules.
+ Below are examples of CSV parsing rules.
+---------------------------------+-----------------+-----------------------------------------------------------------------+
| Feature | Description | input ==> tokens |
JSON
--------------------
- | A JSON reader has been integrated with the ``s3select-engine``, which allows the client to use SQL statements to scan and extract information from JSON documents.
- | It should be noted that the data readers and parsers for CSV, Parquet, and JSON documents are separated from the SQL engine itself, so all of these readers use the same SQL engine.
+A JSON reader has been integrated with the ``s3select-engine``, which allows the client to use SQL statements to scan and extract information from JSON documents.
+It should be noted that the data readers and parsers for CSV, Parquet, and JSON documents are separated from the SQL engine itself, so all of these readers use the same SQL engine.
- | It's important to note that values in a JSON document can be nested in various ways, such as within objects or arrays.
- | These objects and arrays can be nested within each other without any limitations.
- | When using SQL to query a specific value in a JSON document, the client must specify the location of the value
- | via a path in the SELECT statement.
+It's important to note that values in a JSON document can be nested in various ways, such as within objects or arrays.
+These objects and arrays can be nested within each other without any limitations.
+When using SQL to query a specific value in a JSON document, the client must specify the location of the value
+via a path in the SELECT statement.
- | The SQL engine processes the SELECT statement in a row-based fashion.
- | It uses the columns specified in the statement to perform its projection calculation, and each row contains values for these columns.
- | In other words, the SQL engine processes each row one at a time (and aggregates results), using the values in the columns to perform SQL calculations.
- | However, the generic structure of a JSON document does not have a row-and-column structure like CSV or Parquet.
- | Instead, it is the SQL statement itself that defines the rows and columns when querying a JSON document.
+The SQL engine processes the SELECT statement in a row-based fashion.
+It uses the columns specified in the statement to perform its projection calculation, and each row contains values for these columns.
+In other words, the SQL engine processes each row one at a time (and aggregates results), using the values in the columns to perform SQL calculations.
+However, the generic structure of a JSON document does not have a row-and-column structure like CSV or Parquet.
+Instead, it is the SQL statement itself that defines the rows and columns when querying a JSON document.
- | When querying JSON documents using SQL, the FROM clause in the SELECT statement defines the row boundaries.
- | A row in a JSON document should be similar to how the row delimiter is used to define rows when querying CSV objects, and how row groups are used to define rows when querying Parquet objects.
- | The statement "SELECT ... FROM s3object[*].aaa.bb.cc" instructs the reader to search for the path "aaa.bb.cc" and defines the row boundaries based on the occurrence of this path.
- | A row begins when the reader encounters the path, and it ends when the reader exits the innermost part of the path, which in this case is the object "cc".
+When querying JSON documents using SQL, the FROM clause in the SELECT statement defines the row boundaries.
+A row in a JSON document should be similar to how the row delimiter is used to define rows when querying CSV objects, and how row groups are used to define rows when querying Parquet objects.
+The statement "SELECT ... FROM s3object[*].aaa.bb.cc" instructs the reader to search for the path "aaa.bb.cc" and defines the row boundaries based on the occurrence of this path.
+A row begins when the reader encounters the path, and it ends when the reader exits the innermost part of the path, which in this case is the object "cc".
- | NOTE : The semantics of querying JSON document may change and may not be the same as the current methodology described.
+NOTE : The semantics of querying JSON document may change and may not be the same as the current methodology described.
- | TODO : relevant example for object and array values.
+TODO : relevant example for object and array values.
A JSON Query Example
--------------------
BOTO3
-----
- | using BOTO3 is "natural" and easy due to AWS-cli support.
+using BOTO3 is "natural" and easy due to AWS-cli support.
::
Error Response
~~~~~~~~~~~~~~
- | <?xml version="1.0" encoding="UTF-8"?>
- | <Error>
- | <Code>NoSuchKey</Code>
- | <Message>The resource you requested does not exist</Message>
- | <Resource>/mybucket/myfoto.jpg</Resource>
- | <RequestId>4442587FB7D0A2F9</RequestId>
- | </Error>
+::
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <Error>
+ <Code>NoSuchKey</Code>
+ <Message>The resource you requested does not exist</Message>
+ <Resource>/mybucket/myfoto.jpg</Resource>
+ <RequestId>4442587FB7D0A2F9</RequestId>
+ </Error>
Report Response
~~~~~~~~~~~~~~~
- | HTTP/1.1 200
- | <?xml version="1.0" encoding="UTF-8"?>
- | <Payload>
- | <Records>
- | <Payload>blob</Payload>
- | </Records>
- | <Stats>
- | <Details>
- | <BytesProcessed>long</BytesProcessed>
- | <BytesReturned>long</BytesReturned>
- | <BytesScanned>long</BytesScanned>
- | </Details>
- | </Stats>
- | <Progress>
- | <Details>
- | <BytesProcessed>long</BytesProcessed>
- | <BytesReturned>long</BytesReturned>
- | <BytesScanned>long</BytesScanned>
- | </Details>
- | </Progress>
- | <Cont>
- | </Cont>
- | <End>
- | </End>
- | </Payload>
+::
+
+ HTTP/1.1 200
+ <?xml version="1.0" encoding="UTF-8"?>
+ <Payload>
+ <Records>
+ <Payload>blob</Payload>
+ </Records>
+ <Stats>
+ <Details>
+ <BytesProcessed>long</BytesProcessed>
+ <BytesReturned>long</BytesReturned>
+ <BytesScanned>long</BytesScanned>
+ </Details>
+ </Stats>
+ <Progress>
+ <Details>
+ <BytesProcessed>long</BytesProcessed>
+ <BytesReturned>long</BytesReturned>
+ <BytesScanned>long</BytesScanned>
+ </Details>
+ </Progress>
+ <Cont>
+ </Cont>
+ <End>
+ </End>
+ </Payload>
Response Description
~~~~~~~~~~~~~~~~~~~~
- | For CEPH S3 Select, responses can be messages of the following types:
- |
- | **Records message**: Can contain a single record, partial records, or multiple records. Depending on the size of the result, a response can contain one or more of these messages.
- |
- | **Error message**: Upon an error being detected, RGW returns 400 Bad Request, and a specific error message sends back to the client, according to its type.
- |
- | **Continuation message**: Ceph S3 periodically sends this message to keep the TCP connection open.
- | These messages appear in responses at random. The client must detect the message type and process it accordingly.
- |
- | **Progress message**: Ceph S3 periodically sends this message if requested. It contains information about the progress of a query that has started but has not yet been completed.
- |
- | **Stats message**: Ceph S3 sends this message at the end of the request. It contains statistics about the query.
- |
- | **End message**: Indicates that the request is complete, and no more messages will be sent. You should not assume that request is complete until the client receives an End message.
+For CEPH S3 Select, responses can be messages of the following types:
+
+**Records message**: Can contain a single record, partial records, or multiple records. Depending on the size of the result, a response can contain one or more of these messages.
+
+**Error message**: Upon an error being detected, RGW returns 400 Bad Request, and a specific error message sends back to the client, according to its type.
+
+**Continuation message**: Ceph S3 periodically sends this message to keep the TCP connection open.
+These messages appear in responses at random. The client must detect the message type and process it accordingly.
+
+**Progress message**: Ceph S3 periodically sends this message if requested. It contains information about the progress of a query that has started but has not yet been completed.
+
+**Stats message**: Ceph S3 sends this message at the end of the request. It contains statistics about the query.
+
+**End message**: Indicates that the request is complete, and no more messages will be sent. You should not assume that request is complete until the client receives an End message.
projects / ceph.git / commitdiff