rgw:gc: use XXHash for gc queue selection

We've observed significant imbalance in gc queue loading, most
likely due to poor ceph_str_hash_linux dispersion for common
key forms.

XXHash64 should be an effective and fast replacement for all
current RGW users of ceph_str_hash_linux (which also include
bucket-index sharding and bucket-lifecycle shard selection).

Unlike the other users, gc queue selection appears safe to
"just switch" as only defer-gc operations (which are non-critical
and of brief temporal locality) make any sort of rendezvous
on specific enqueued tags.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

diff --git a/src/rgw/rgw_gc.cc b/src/rgw/rgw_gc.cc
index 98be34bf4c282148ace13a45ca92d05b8b76ce82..b7b89e7b8697741ab2b50b823e3806a1b197e496 100644 (file)
--- a/src/rgw/rgw_gc.cc
+++ b/src/rgw/rgw_gc.cc
@@ -17,6 +17,7 @@
 
 #include <list> // XXX
 #include <sstream>
+#include "xxhash.h"
 
 #define dout_context g_ceph_context
 #define dout_subsys ceph_subsys_rgw
@@ -60,7 +61,7 @@ void RGWGC::finalize()
 
 int RGWGC::tag_index(const string& tag)
 {
-  return rgw_shard_id(tag, max_objs);
+  return rgw_shards_mod(XXH64(tag.c_str(), tag.size(), seed), max_objs);
 }
 
 int RGWGC::send_chain(cls_rgw_obj_chain& chain, const string& tag)

diff --git a/src/rgw/rgw_gc.h b/src/rgw/rgw_gc.h
index 5dda753fd54839ef08e6dc33ab6c42017d446d77..9eafb5fbd875ce735ee1c24beb6b470c747d3b9b 100644 (file)
--- a/src/rgw/rgw_gc.h
+++ b/src/rgw/rgw_gc.h
@@ -25,6 +25,8 @@ class RGWGC : public DoutPrefixProvider {
   string *obj_names;
   std::atomic<bool> down_flag = { false };
 
+  static constexpr uint64_t seed = 8675309;
+
   int tag_index(const string& tag);
 
   class GCWorker : public Thread {