os/FileStore: force any new xattr into omap on E2BIG

If we have a huge xattr (or many little ones), the _fgetattrs() for the
inline_set will fail with E2BIG.  The conditions later where we decide
whether to clean up the old xattr will then also fail.  We *will* put
the xattr in omap, but the non-omap version isn't cleaned up.

Fix this by setting a flag if we get E2BIG that the inline_set is known
to be incomplete.  In that case, take the conservative step of assuming
the xattr might be present and chain_fremovexattr().  Ignore any error
because it might not be there.

This is clearly harmless in the general case because it won't be there.
If it is, we will hopefully remove enough xattrs that the E2BIG
condition will go away (usually by removing some really big chained
xattr).

See original bug #7779.  With this in place, we can repair objects in
the broken state if we know the rados attr(s) that are responsible.
Usually that is user.rgw.manifset, and a rados get + set of the attr
will repair things.

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
Signed-off-by: Sage Weil <sage@redhat.com>

diff --git a/src/os/FileStore.cc b/src/os/FileStore.cc
index f490107ecacb6c9596065b6e6ff6cab0e5928de5..b38cc014b3a9505df574b32aca8913889742c8fd 100644 (file)
--- a/src/os/FileStore.cc
+++ b/src/os/FileStore.cc
@@ -3741,6 +3741,7 @@ int FileStore::_setattrs(coll_t cid, const ghobject_t& oid, map<string,bufferptr
   map<string, bufferptr> inline_to_set;
   FDRef fd;
   int spill_out = -1;
+  bool incomplete_inline = false;
 
   int r = lfn_open(cid, oid, false, &fd);
   if (r < 0) {
@@ -3755,8 +3756,11 @@ int FileStore::_setattrs(coll_t cid, const ghobject_t& oid, map<string,bufferptr
     spill_out = 1;
 
   r = _fgetattrs(**fd, inline_set);
+  incomplete_inline = (r == -E2BIG);
   assert(!m_filestore_fail_eio || r != -EIO);
-  dout(15) << "setattrs " << cid << "/" << oid << dendl;
+  dout(15) << "setattrs " << cid << "/" << oid
+          << (incomplete_inline ? " (incomplete_inline, forcing omap)" : "")
+          << dendl;
 
   for (map<string,bufferptr>::iterator p = aset.begin();
        p != aset.end();
@@ -3764,6 +3768,12 @@ int FileStore::_setattrs(coll_t cid, const ghobject_t& oid, map<string,bufferptr
     char n[CHAIN_XATTR_MAX_NAME_LEN];
     get_attrname(p->first.c_str(), n, CHAIN_XATTR_MAX_NAME_LEN);
 
+    if (incomplete_inline) {
+      chain_fremovexattr(**fd, n); // ignore any error
+      omap_set[p->first].push_back(p->second);
+      continue;
+    }
+
     if (p->second.length() > m_filestore_max_inline_xattr_size) {
        if (inline_set.count(p->first)) {
          inline_set.erase(p->first);