From: Sage Weil <sage@redhat.com>
Date: Thu, 27 Feb 2020 15:30:27 +0000 (-0600)
Subject: compressor/lz4: rebuild if buffer is not contiguous
X-Git-Tag: v14.2.10~54^2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=c1d8ae5b3ac66cfc08b565de8963e2d3ff4e2c48;p=ceph.git

compressor/lz4: rebuild if buffer is not contiguous

In older versions of lz4 (specifically < 1.8.2) bit errors
can be introduced when compressing from fragmented memory.  The lz4
bug was fixed by this lz4 commit:

commit af127334670a5e7b710bbd6adb71aa7c3ef0cd72
Author: Yann Collet <cyan@fb.com>
Date:   Sat May 5 18:24:11 2018 -0700

    fixed frametest error

    The error can be reproduced using following command :
    ./frametest -v -i100000000 -s1659 -t31096808

    It's actually a bug in the stream LZ4 API,
    when starting a new stream
    and providing a first chunk to complete with size < MINMATCH.
    In which case, the chunk becomes a dictionary.
    No hash was generated and stored,
    but the chunk is accessible as default position 0 points to dictStart,
    and position 0 is still within MAX_DISTANCE.
    Then, next attempt to read 32-bits from position 0 fails.

    The issue would have been mitigated by starting from index 64 KB,
    effectively eliminating position 0 as too far away.

    The proper fix is to eliminate such "dictionary" as too small.
    Which is what this patch does.

This is a workaround to rebuild our input buffer into a continguos buffer
if it is not already contiguous.

Fixes: https://tracker.ceph.com/issues/39525
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8e80f5f678915f18bdf873502283bf3e2020f904)
---

diff --git a/src/compressor/lz4/LZ4Compressor.h b/src/compressor/lz4/LZ4Compressor.h
index 8189f18f4c3a..6e604d5cc975 100644
--- a/src/compressor/lz4/LZ4Compressor.h
+++ b/src/compressor/lz4/LZ4Compressor.h
@@ -36,6 +36,18 @@ class LZ4Compressor : public Compressor {
   }
 
   int compress(const bufferlist &src, bufferlist &dst) override {
+    // older versions of liblz4 introduce bit errors when compressing
+    // fragmented buffers.  this was fixed in lz4 commit
+    // af127334670a5e7b710bbd6adb71aa7c3ef0cd72, which first
+    // appeared in v1.8.2.
+    //
+    // workaround: rebuild if not contiguous.
+    if (!src.is_contiguous()) {
+      bufferlist new_src = src;
+      new_src.rebuild();
+      return compress(new_src, dst);
+    }
+
 #ifdef HAVE_QATZIP
     if (qat_enabled)
       return qat_accel.compress(src, dst);