os/bluestore/BlockDevice: disable readahead of fd_buffered

Readahead can break our mix of aio/dio and buffered io.  Although the
dio path will invalidate pages before and after initiating the IO, and we
do not issue racing buffered reads, the readahead code might.  That would
result in pages in the page cache that are not coherent with an aio write.

This is easy to reproduce with ceph_test_objectstore's Synthetic case
when bluestore_default_buffered_read=true.

Signed-off-by: Sage Weil <sage@redhat.com>

diff --git a/src/os/bluestore/BlockDevice.cc b/src/os/bluestore/BlockDevice.cc
index 6f2f436c3c9dbbf139420037c45580c4feab93de..968a78ff8dc8db89d673ac170c6cd52555b160b5 100644 (file)
--- a/src/os/bluestore/BlockDevice.cc
+++ b/src/os/bluestore/BlockDevice.cc
@@ -95,6 +95,15 @@ int BlockDevice::open(string p)
     assert(0 == "non-aio not supported");
   }
 
+  // disable readahead as it will wreak havoc on our mix of
+  // directio/aio and buffered io.
+  r = posix_fadvise(fd_buffered, 0, 0, POSIX_FADV_RANDOM);
+  if (r < 0) {
+    r = -errno;
+    derr << __func__ << " open got: " << cpp_strerror(r) << dendl;
+    goto out_fail;
+  }
+
   r = _lock();
   if (r < 0) {
     derr << __func__ << " failed to lock " << path << ": " << cpp_strerror(r)