From: luo rixin <luorixin@huawei.com>
Date: Wed, 14 Sep 2022 11:50:01 +0000 (+0800)
Subject: os/bluestore: use direct write in BlueStore::_write_bdev_label
X-Git-Tag: v18.1.0~1117^2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=afdb5e4271dcabb8aaa7f5c797c31dcbb2e387f6;p=ceph.git

os/bluestore: use direct write in BlueStore::_write_bdev_label

On AArch64 with kernel page size 64K, it occurs occasionally
"OSD::init(): unable to read osd superblock" when deploying osd.
As bluestore use direct write to write the superblock at 0x2000~1000
and BlueStore::_write_bdev_label use buffer write to write label at
0x0~1000, The OS flush the buffer write algined to page size, it will
overwrite the superblock(0x2000~1000). Use driect write to avoid
overwriting the superblock.

Fixes: https://tracker.ceph.com/issues/57537
Signed-off-by: luo rixin <luorixin@huawei.com>
---

diff --git a/src/os/bluestore/BlueStore.cc b/src/os/bluestore/BlueStore.cc
index 89748853babb..313512bcb3a1 100644
--- a/src/os/bluestore/BlueStore.cc
+++ b/src/os/bluestore/BlueStore.cc
@@ -5375,13 +5375,14 @@ int BlueStore::_write_bdev_label(CephContext *cct,
   z.zero();
   bl.append(std::move(z));
 
-  int fd = TEMP_FAILURE_RETRY(::open(path.c_str(), O_WRONLY|O_CLOEXEC));
+  int fd = TEMP_FAILURE_RETRY(::open(path.c_str(), O_WRONLY|O_CLOEXEC|O_DIRECT));
   if (fd < 0) {
     fd = -errno;
     derr << __func__ << " failed to open " << path << ": " << cpp_strerror(fd)
 	 << dendl;
     return fd;
   }
+  bl.rebuild_aligned_size_and_memory(BDEV_LABEL_BLOCK_SIZE, BDEV_LABEL_BLOCK_SIZE, IOV_MAX);
   int r = bl.write_fd(fd);
   if (r < 0) {
     derr << __func__ << " failed to write to " << path