rgw/s3: remove 'aws-chunked' from Content-Encoding response

PutObject stores some of the generic http request headers in object
attrs so they can be returned as response headers in Get/HeadObject

S3 has its own `aws-chunked` value for the `Content-Encoding` header,
which it says does _not_ get stored with the object or returned with
Get/HeadObject

we've been storing this header with objects forever, so omitting the
value on PutObject doesn't fix the issue for existing objects. instead,
add the necessary filtering to Get/HeadObject

Fixes: https://tracker.ceph.com/issues/21128
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4a802b864a6f9d45506197dfb1bc23cf852e51f8)

diff --git a/src/rgw/rgw_rest_s3.cc b/src/rgw/rgw_rest_s3.cc
index 7e413ed5b58a0330f315bee7d05452887c629a2a..0ff827f22e62441c4a251e8b5dd3aeb1911d3201 100644 (file)
--- a/src/rgw/rgw_rest_s3.cc
+++ b/src/rgw/rgw_rest_s3.cc
@@ -369,6 +369,25 @@ inline bool str_has_cntrl(const char* s) {
   return str_has_cntrl(_s);
 }
 
+// remove any aws-chunked entries from the comma-separated list
+static std::string content_encoding_without_aws_chunked(std::string_view value)
+{
+  std::string result;
+  result.reserve(value.size());
+
+  for (std::string_view part : ceph::split(value, ", ")) {
+    if (part == "aws-chunked") {
+      continue;
+    }
+    if (!result.empty()) {
+      result += ", ";
+    }
+    result += part;
+  }
+
+  return result;
+}
+
 int RGWGetObj_ObjStore_S3::send_response_data(bufferlist& bl, off_t bl_ofs,
                                              off_t bl_len)
 {
@@ -652,7 +671,20 @@ int RGWGetObj_ObjStore_S3::send_response_data(bufferlist& bl, off_t bl_ofs,
             --len;
             s.resize(len);
           }
-          response_attrs[aiter->second] = s;
+
+          if (aiter->first == RGW_ATTR_CONTENT_ENC) {
+            // Amazon S3 stores the resulting object without the aws-chunked
+            // value in the content-encoding header. If aws-chunked is the only
+            // value that you pass in the content-encoding header, S3 considers
+            // the content-encoding header empty and does not return this header
+            // when your retrieve the object.
+            std::string encoding = content_encoding_without_aws_chunked(s);
+            if (!encoding.empty()) {
+              response_attrs[aiter->second] = std::move(encoding);
+            }
+          } else {
+            response_attrs[aiter->second] = std::move(s);
+          }
         }
       } else if (iter->first.compare(RGW_ATTR_CONTENT_TYPE) == 0) {
         /* Special handling for content_type. */