From: Loic Dachary <ldachary@redhat.com>
Date: Thu, 18 Dec 2014 00:25:54 +0000 (+0100)
Subject: erasure-code: set max_size to chunk_count() instead of 20
X-Git-Tag: v0.93~262^2~2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=8b64fe93b088a3a33d357869c47e6bf928c3f0e4;p=ceph.git

erasure-code: set max_size to chunk_count() instead of 20

The ruleset created for an erasure coded pool has max_size set to a
fixed value of 20, which may be incorrect when more than 20 chunks are
needed and lead to obscure errors. Set it to the number of chunks,
i.e. k+m most of the time.

In a cluster with few OSDs (9 for instance), setting max_size to 20
causes performance problems when injecting a new crushmap. The monitor
will call CrushTester::test which tries 1024 mappins for all sizes
ranging from min_size to max_size. Each attempt to map more OSDs than
available will exhaust all retries (50 by default) and it takes a
significant amount of time. In a cluster with 9 OSDs, testing one such
ruleset can take up to 5 seconds.

Since the test blocks the monitor leader, a few erasure coded rulesets
will block the monitor long enough to exceed the timeouts and trigger an
election.

http://tracker.ceph.com/issues/10363 Fixes: #10363

Signed-off-by: Loic Dachary <ldachary@redhat.com>
---

diff --git a/src/crush/CrushWrapper.h b/src/crush/CrushWrapper.h
index b17f9a14f14f..c7667c5989ba 100644
--- a/src/crush/CrushWrapper.h
+++ b/src/crush/CrushWrapper.h
@@ -709,6 +709,11 @@ public:
     ruleno = crush_add_rule(crush, n, ruleno);
     return ruleno;
   }
+  int set_rule_mask_max_size(unsigned ruleno, int max_size) {
+    crush_rule *r = get_rule(ruleno);
+    if (IS_ERR(r)) return -1;
+    return r->mask.max_size = max_size;
+  }
   int set_rule_step(unsigned ruleno, unsigned step, int op, int arg1, int arg2) {
     if (!crush) return -ENOENT;
     crush_rule *n = get_rule(ruleno);
diff --git a/src/erasure-code/isa/ErasureCodeIsa.cc b/src/erasure-code/isa/ErasureCodeIsa.cc
index d5ba3ff1d19a..f9e079306f5f 100644
--- a/src/erasure-code/isa/ErasureCodeIsa.cc
+++ b/src/erasure-code/isa/ErasureCodeIsa.cc
@@ -55,8 +55,10 @@ ErasureCodeIsa::create_ruleset(const string &name,
 
   if (ruleid < 0)
     return ruleid;
-  else
+  else {
+    crush.set_rule_mask_max_size(ruleid, get_chunk_count());
     return crush.get_rule_mask_ruleset(ruleid);
+  }
 }
 
 // -----------------------------------------------------------------------------
diff --git a/src/erasure-code/jerasure/ErasureCodeJerasure.cc b/src/erasure-code/jerasure/ErasureCodeJerasure.cc
index f3944e83c0fa..4720d21858bb 100644
--- a/src/erasure-code/jerasure/ErasureCodeJerasure.cc
+++ b/src/erasure-code/jerasure/ErasureCodeJerasure.cc
@@ -46,8 +46,10 @@ int ErasureCodeJerasure::create_ruleset(const string &name,
 					"indep", pg_pool_t::TYPE_ERASURE, ss);
   if (ruleid < 0)
     return ruleid;
-  else
+  else {
+    crush.set_rule_mask_max_size(ruleid, get_chunk_count());
     return crush.get_rule_mask_ruleset(ruleid);
+  }
 }
 
 void ErasureCodeJerasure::init(const map<string,string> &parameters)
diff --git a/src/erasure-code/lrc/ErasureCodeLrc.cc b/src/erasure-code/lrc/ErasureCodeLrc.cc
index b198350527f3..2f90ea8f86dc 100644
--- a/src/erasure-code/lrc/ErasureCodeLrc.cc
+++ b/src/erasure-code/lrc/ErasureCodeLrc.cc
@@ -64,7 +64,7 @@ int ErasureCodeLrc::create_ruleset(const string &name,
 
   int steps = 4 + ruleset_steps.size();
   int min_rep = 3;
-  int max_rep = 30;
+  int max_rep = get_chunk_count();
   int ret;
   ret = crush.add_rule(steps, ruleset, pg_pool_t::TYPE_ERASURE,
 		  min_rep, max_rep, rno);