erasure-code: set max_size to chunk_count() instead of 20

The ruleset created for an erasure coded pool has max_size set to a
fixed value of 20, which may be incorrect when more than 20 chunks are
needed and lead to obscure errors. Set it to the number of chunks,
i.e. k+m most of the time.

In a cluster with few OSDs (9 for instance), setting max_size to 20
causes performance problems when injecting a new crushmap. The monitor
will call CrushTester::test which tries 1024 mappins for all sizes
ranging from min_size to max_size. Each attempt to map more OSDs than
available will exhaust all retries (50 by default) and it takes a
significant amount of time. In a cluster with 9 OSDs, testing one such
ruleset can take up to 5 seconds.

Since the test blocks the monitor leader, a few erasure coded rulesets
will block the monitor long enough to exceed the timeouts and trigger an
election.

http://tracker.ceph.com/issues/10363 Fixes: #10363

Signed-off-by: Loic Dachary <ldachary@redhat.com>

diff --git a/src/crush/CrushWrapper.h b/src/crush/CrushWrapper.h
index b17f9a14f14f4b81420a943a3ad4016f6ef92091..c7667c5989bac64985dd19ea5d083520c275e8ca 100644 (file)
--- a/src/crush/CrushWrapper.h
+++ b/src/crush/CrushWrapper.h
@@ -709,6 +709,11 @@ public:
     ruleno = crush_add_rule(crush, n, ruleno);
     return ruleno;
   }
+  int set_rule_mask_max_size(unsigned ruleno, int max_size) {
+    crush_rule *r = get_rule(ruleno);
+    if (IS_ERR(r)) return -1;
+    return r->mask.max_size = max_size;
+  }
   int set_rule_step(unsigned ruleno, unsigned step, int op, int arg1, int arg2) {
     if (!crush) return -ENOENT;
     crush_rule *n = get_rule(ruleno);

diff --git a/src/erasure-code/isa/ErasureCodeIsa.cc b/src/erasure-code/isa/ErasureCodeIsa.cc
index d5ba3ff1d19aae29a8272852af1d10dd4bc9d3ad..f9e079306f5f63046e42d349ad14518831b336f7 100644 (file)
--- a/src/erasure-code/isa/ErasureCodeIsa.cc
+++ b/src/erasure-code/isa/ErasureCodeIsa.cc
@@ -55,8 +55,10 @@ ErasureCodeIsa::create_ruleset(const string &name,
 
   if (ruleid < 0)
     return ruleid;
-  else
+  else {
+    crush.set_rule_mask_max_size(ruleid, get_chunk_count());
     return crush.get_rule_mask_ruleset(ruleid);
+  }
 }
 
 // -----------------------------------------------------------------------------

diff --git a/src/erasure-code/jerasure/ErasureCodeJerasure.cc b/src/erasure-code/jerasure/ErasureCodeJerasure.cc
index f3944e83c0faaf8fd2af696850a331b1580e223c..4720d21858bbe627986bf5003d638cc709a612fb 100644 (file)
--- a/src/erasure-code/jerasure/ErasureCodeJerasure.cc
+++ b/src/erasure-code/jerasure/ErasureCodeJerasure.cc
@@ -46,8 +46,10 @@ int ErasureCodeJerasure::create_ruleset(const string &name,
                                        "indep", pg_pool_t::TYPE_ERASURE, ss);
   if (ruleid < 0)
     return ruleid;
-  else
+  else {
+    crush.set_rule_mask_max_size(ruleid, get_chunk_count());
     return crush.get_rule_mask_ruleset(ruleid);
+  }
 }
 
 void ErasureCodeJerasure::init(const map<string,string> &parameters)

diff --git a/src/erasure-code/lrc/ErasureCodeLrc.cc b/src/erasure-code/lrc/ErasureCodeLrc.cc
index b198350527f3f821808def752e36664480d574c4..2f90ea8f86dc83cc7bc4195daf269d801136c78d 100644 (file)
--- a/src/erasure-code/lrc/ErasureCodeLrc.cc
+++ b/src/erasure-code/lrc/ErasureCodeLrc.cc
@@ -64,7 +64,7 @@ int ErasureCodeLrc::create_ruleset(const string &name,
 
   int steps = 4 + ruleset_steps.size();
   int min_rep = 3;
-  int max_rep = 30;
+  int max_rep = get_chunk_count();
   int ret;
   ret = crush.add_rule(steps, ruleset, pg_pool_t::TYPE_ERASURE,
                  min_rep, max_rep, rno);