rgw: D3N Cache changes for Upstream

rgw: D3N Cache changes for Upstream

Upstreaming / rebase of #24500

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: change io_ctx pool per storage class

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Changing free() to delete()

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: Addressing review comments

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: Fixing seg fault

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Moving CacheRequest out of librados

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: cache initialization fix

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix missing spawn.hpp compile errors

resolves compilation errors similar to:
```
[ 15%] Building CXX object src/tools/immutable_object_cache/CMakeFiles/ceph_immutable_object_cache_lib.dir/CacheController.cc.o
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_common.h:31,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_rados.h:17,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/IoCtxImpl.h:30,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/RadosClient.h:35,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOSImpl.h:27,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOS.cc:37:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
   31 | #include <spawn/spawn.hpp>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
src/neorados/CMakeFiles/neorados_api_obj.dir/build.make:62: recipe for target 'src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o' failed
make[3]: *** [src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o] Error 1
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>
Resolving merge conflict

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Removing all logs and unnecessary comments

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Cache Read and Write working

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Initial Commit L1 Cache

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
post re-rebase merge, update fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>
fixup! post re-rebase merge, update fixes

rgw: continuation of rgwcache branch rebase

Signed-off-by: Mark Kogan <mkogan@redhat.com>
RGW: DataCache: post merge fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>
fixes of segmentation fault caused by oid

Signed-off-by: E. Ugur Kaynar <ukaynar@bu.edu>
rgw: fixes for segmentation faults and configuration processing

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: data cache first commit

Signed-off-by: Mania Abdi <mania.abdi287@gmail.com>
rgw: cleanup addressing PR comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: cleanup addressing PR comments, continuation.

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: pr cleanup addressing second review round

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Addressing review comments, removing all D3N code from librados

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: for compilation err from removal of mydout() helper

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rge: addressing review comments

rgw: move d3n datacache into separate files
rgw: 1st part of datacache rebranding to d3n

fix forward declaration compile err (only with clang):
../src/rgw/rgw_cache.h:396:4: error: member access into incomplete type 'struct get_obj_data'
  d->data_lock.lock();
   ^
../src/rgw/rgw_cache.h:365:8: note: forward declaration of 'get_obj_data'
struct get_obj_data;
       ^

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: addressing review comments, datacache rebranding to d3n cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Cleaning up unused D3N cache code

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: cont. cleaning up of rgw_obj_data()

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Removing redundant code, fix for multipart S3 objects

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix: incorrect content len on multipart get

in s3tests_boto3.functional.test_s3:test_multipart_copy_versioned
when d3n cache is disabled

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault reading from cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault in multisite

sync on secondary site

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault in multisite teuthology tests, cont.

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Adding drain to wait for all AIO reads to complete

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix for using read() by liabio or posix io per config

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: improve persistent data cache directory handling

on start create the persistent datacache directory if necessary
and add an option to evict it's content if already exists

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix possible segfault during eviction

Signed-off-by: Mark Kogan <mkogan@redhat.com>
Co-authored-by: Mania Abdi <mania.abdi287@gmail.com>
Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu>
Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing latest review comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: add debug logs for cache in/out flow

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: move the L2 cache functionality to separate PR

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing review comments

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address java_s3tests teuthology issues

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: do not handle compressed objects fro now

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: l2 cleanup and log fixups + post dpp

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: thread dpp thru get_obj_iterate_cb() and related

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: bypass reading versioned objects from cache

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: cleanup and fix s3tests

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: allow to enable cache only on beast

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: delete the content of the cache directory on rgw start

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: re-enable d3n cache with civetweb frontend

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: rebase post zipper 10

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address teuthoogy valgrind leaks detected

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: mitigating valgrind leaks

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: remove rgw_d3n_l1_libaio_read option

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip segfault fix

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: mitigate libaio SIGEV_THREAD cb race

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: change libaio signaling mechanism

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip cont. libaio cb thread race

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip libaio cb thread race

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: libaio cleanups and edge case handling fixes

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: narrow the libaio locking scope

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: add libaio req ordering mechanism

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix lock regression

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing reviwe comments and cleasnup

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: libaio locks cleanup

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: refactor libaio abstraction to share the ioc implementation

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing latest review comments and cleanup

Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address review comments, cont.

Signed-off-by: Mark Kogan <mkogan@redhat.com>
Co-authored-by: Mania Abdi <mania.abdi287@gmail.com>
Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu>
Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Ali Maredia <amaredia@redhat.com>
Co-authored-by: Feng Hualong <hualong.feng@intel.com>