From: Takeshi Miyamae Date: Wed, 18 Feb 2015 11:42:50 +0000 (+0900) Subject: erasure code: add shec's documentation / change default layout X-Git-Tag: suse_latest~40^2 X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=refs%2Fpull%2F3755%2Fhead;p=ceph.git erasure code: add shec's documentation / change default layout Signed-off-by: Takeshi Miyamae --- diff --git a/doc/rados/operations/erasure-code-isa.rst b/doc/rados/operations/erasure-code-isa.rst index 241c7faeef9c..9d4ff7f52f0a 100644 --- a/doc/rados/operations/erasure-code-isa.rst +++ b/doc/rados/operations/erasure-code-isa.rst @@ -2,14 +2,14 @@ ISA erasure code plugin ======================= -The *isa* plugin is encapsulates the `ISA +The *isa* plugin encapsulates the `ISA `_ library. It only runs on Intel processors. Create an isa profile ===================== -To create a new *jerasure* erasure code profile:: +To create a new *isa* erasure code profile:: ceph osd erasure-code-profile set {name} \ plugin=isa \ diff --git a/doc/rados/operations/erasure-code-shec.rst b/doc/rados/operations/erasure-code-shec.rst new file mode 100644 index 000000000000..2d71ad7a49ac --- /dev/null +++ b/doc/rados/operations/erasure-code-shec.rst @@ -0,0 +1,133 @@ +======================== +SHEC erasure code plugin +======================== + +The *shec* plugin encapsulates the `multiple SHEC +`_ +library. It allows ceph to recover data more efficiently than Reed Solomon codes. + +Create an SHEC profile +====================== + +To create a new *shec* erasure code profile:: + + ceph osd erasure-code-profile set {name} \ + plugin=shec \ + [k={data-chunks}] \ + [m={coding-chunks}] \ + [c={durability-estimator}] \ + [ruleset-root={root}] \ + [ruleset-failure-domain={bucket-type}] \ + [directory={directory}] \ + [--force] + +Where: + +``k={data-chunks}`` + +:Description: Each object is split in **data-chunks** parts, + each stored on a different OSD. + +:Type: Integer +:Required: No. +:Default: 4 + +``m={coding-chunks}`` + +:Description: Compute **coding-chunks** for each object and store them on + different OSDs. The number of **coding-chunks** does not necessarily + equal the number of OSDs that can be down without losing data. + +:Type: Integer +:Required: No. +:Default: 3 + +``c={durability-estimator}`` + +:Description: The number of parity chunks each of which includes each data chunk in its + calculation range. The number is used as a **durability estimator**. + For instance, if c=2, 2 OSDs can be down without losing data. + +:Type: Integer +:Required: No. +:Default: 2 + +``ruleset-root={root}`` + +:Description: The name of the crush bucket used for the first step of + the ruleset. For intance **step take default**. + +:Type: String +:Required: No. +:Default: default + +``ruleset-failure-domain={bucket-type}`` + +:Description: Ensure that no two chunks are in a bucket with the same + failure domain. For instance, if the failure domain is + **host** no two chunks will be stored on the same + host. It is used to create a ruleset step such as **step + chooseleaf host**. + +:Type: String +:Required: No. +:Default: host + +``directory={directory}`` + +:Description: Set the **directory** name from which the erasure code + plugin is loaded. + +:Type: String +:Required: No. +:Default: /usr/lib/ceph/erasure-code + +``--force`` + +:Description: Override an existing profile by the same name. + +:Type: String +:Required: No. + +Brief description of SHEC's layouts +=================================== + +Space Efficiency +---------------- + +Space efficiency is a ratio of data chunks to all ones in a object and +represented as k/(k+m). +In order to improve space efficiency, you should increase k or decrease m. + +:: + + space efficiency of SHEC(4,3,2) = 4/(4+3) = 0.57 + SHEC(5,3,2) or SHEC(4,2,2) improves SHEC(4,3,2)'s space efficiency + +Durability +---------- + +The third parameter of SHEC (=c) is a durability estimator, which approximates +the number of OSDs that can be down without losing data. + +``durability estimator of SHEC(4,3,2) = 2`` + +Recovery Efficiency +------------------- + +Describing calculation of recovery efficiency is beyond the scope of this document, +but at least increasing m without increasing c achieves improvement of recovery efficiency. +(However, we must pay attention to the sacrifice of space efficiency in this case.) + +``SHEC(4,2,2) -> SHEC(4,3,2) : achieves improvement of recovery efficiency`` + +Erasure code profile examples +============================= + +:: + + $ ceph osd erasure-code-profile set SHECprofile \ + plugin=shec \ + k=8 m=4 c=3 \ + ruleset-failure-domain=host + $ ceph osd pool create shecpool 256 256 erasure SHECprofile diff --git a/src/erasure-code/shec/ErasureCodeShec.h b/src/erasure-code/shec/ErasureCodeShec.h index 937d72f34f20..7eaccf5e7fd9 100644 --- a/src/erasure-code/shec/ErasureCodeShec.h +++ b/src/erasure-code/shec/ErasureCodeShec.h @@ -52,11 +52,11 @@ public: ErasureCodeShecTableCache &_tcache) : tcache(_tcache), k(0), - DEFAULT_K(2), + DEFAULT_K(4), m(0), - DEFAULT_M(1), + DEFAULT_M(3), c(0), - DEFAULT_C(1), + DEFAULT_C(2), w(0), DEFAULT_W(8), technique(_technique), diff --git a/src/test/erasure-code/TestErasureCodeShec.cc b/src/test/erasure-code/TestErasureCodeShec.cc index 8d3a5f6a2067..25325f64d2d0 100644 --- a/src/test/erasure-code/TestErasureCodeShec.cc +++ b/src/test/erasure-code/TestErasureCodeShec.cc @@ -785,9 +785,9 @@ TEST(ErasureCodeShec, init_29) EXPECT_TRUE(shec->matrix != NULL); EXPECT_EQ(0, r); //k,m,c are default values - EXPECT_EQ(2u, shec->k); - EXPECT_EQ(1u, shec->m); - EXPECT_EQ(1u, shec->c); + EXPECT_EQ(4u, shec->k); + EXPECT_EQ(3u, shec->m); + EXPECT_EQ(2u, shec->c); delete shec; delete parameters;