common/buffer_seastar: fix alien threads memory

The underlying raw_seastar_foreign_ptr::ptr is allocated from seastar.
This ptr is wrapped with seastar::foreign_ptr:
```
/// \c foreign_ptr<> wraps smart pointers -- \ref seastar::shared_ptr<>,
/// or similar, and remembers on what core this happened.
/// When the \c foreign_ptr<> object is destroyed, it sends a message to
/// the original core so that the wrapped object can be safely destroyed.
```

The issue is that once the pointer is de-allocated from an alien thread
it is unable to send a message to the original core.
Fix this issue by making use of seastar::alien integration with non-seastar applications.
In case ~raw_seastar_foreign_ptr() will be called from an alien thread, we will submit *and wait*
for the memory to be released from the origin core.

Fixes: https://tracker.ceph.com/issues/64086
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit c714c3c4f92994413e34a5900492c8987c85669e)

diff --git a/src/common/buffer_seastar.cc b/src/common/buffer_seastar.cc
index bc529c937ab2f66997f16eef1619e43bbcc0594a..fa040a4661c25e28d4bd807c0715f82af8af4fce 100644 (file)
--- a/src/common/buffer_seastar.cc
+++ b/src/common/buffer_seastar.cc
@@ -14,6 +14,8 @@
 
 #include <seastar/core/sharded.hh>
 #include <seastar/net/packet.hh>
+#include <seastar/core/reactor.hh>
+#include <seastar/core/alien.hh>
 
 #include "include/buffer_raw.h"
 #include "buffer_seastar.h"
@@ -24,9 +26,21 @@ namespace ceph::buffer {
 
 class raw_seastar_foreign_ptr : public raw {
   seastar::foreign_ptr<temporary_buffer> ptr;
+  seastar::alien::instance& alien;
  public:
   raw_seastar_foreign_ptr(temporary_buffer&& buf)
-    : raw(buf.get_write(), buf.size()), ptr(std::move(buf)) {}
+    : raw(buf.get_write(), buf.size()), ptr(std::move(buf)),
+      alien(seastar::engine().alien()) {}
+
+  ~raw_seastar_foreign_ptr() {
+    if (!seastar::engine_is_ready()) {
+      // we should let a seastar reactor destroy this memory, we are alien.
+      seastar::alien::run_on(alien, ptr.get_owner_shard(),
+      [_ptr = std::move(ptr)]() mutable noexcept {
+        _ptr.reset();
+      });
+    }
+  }
 };
 
 class raw_seastar_local_ptr : public raw {