rgw/cache: implementation of `put` using yield context
as completion token and adding throttling to `put_async`
in the cache driver api. Also added a test case to the
ssd driver unit test for `put_async`.
Samarah [Wed, 3 Jan 2024 15:52:25 +0000 (15:52 +0000)]
rgw: this commit squashes commits performing cleanup in various files
like d4n filter, policy, cache backend, aio and process_env.
d4n/filter: Remove unnecessary erasure of attrs in read op's `prepare`
method
d4n: Remove `dout` definitions from header files
rgw: Remove unnecessary includes in `rgw_process_env.h`
rgw: Remove `cache_read_op` from `rgw_aio.h`
rgw: Return POSIX error codes and perform minor cleanup for D4N and
Redis files
rgw/cache: this commit squashes commits for removing CephContext* from
ssd, redis driver and related test files.
rgw/cache: removing CephContext* from ssd driver and redis driver.
It can be accessed in every api using dpp->get_cct(), where dpp
is of type DoutPrefixProvider*.
rgw/test: Remove `cct` parameter from cache driver initialization call
redis driver test file.
rgw/qa: removing cct parameter from ssd driver test file.
Samarah [Tue, 31 Oct 2023 14:21:06 +0000 (14:21 +0000)]
rgw/d4n: this commit squashes the following commits related to LFUDA Policy.
rgw/d4n: Make minor updates, improve `LFUDAPolicy::erase` method, and
fix `localWeight` policy sets
rgw/d4n: Update `LFUDA::find_victim` method, add locks in policy code,
and remove setting of attrs in filter's `put_async` calls
d4n/policy: Update logic for minimum average weight calculation
d4n/policy: Remove iteration in `eviction`
d4n/policy: this commit squashes the following changes to policy and filter driver files.
d4n/policy: Update policy with correct directory usage; create LFUDAEntry struct + entries map; add `optional_yield` to `insert` method
d4n/policy: Add `version` to Entry struct/policy methods and calls in
the D4N filter; add error checking and fix logic in LFUDA code
rgw/cache: this commit squashes following commits related to changes in
redis driver and ssd driver.
rgw/redis: Add `del`, `put_async`, and `calculate_free_space` methods to RedisDriver
rgw/redis: Work on RedisDriver `put_async` and `get_async` methods
rgw/cache: Add error-checking to `SSDDriver::set_attr`
rgw/redis: Remove `calculate_free_space` method from RedisDriver, fix
`free_space` calculations, clean up async code
test/d4n: this commit squashes the following commits related to tests for d4n.
test/redis: Add temporary `boost::redis` unit test program for experimentation
test/rgw: Add/update unit test programs for D4N classes + RedisDriver; switch to `boost::redis` usage
test/d4n: Begin `cpp_redis` removal from D4N filter test program; needs
to be fixed
test/d4n: Temporarily remove D4N filter test program until D4N workflows
are fully implemented
rgw/d4n: this commit squashes the following commits related to
clean up and removal of cpp_redis.
d4n/filter: Add `optional_yield` to CacheDriver calls in D4N Filter and
make minor updates to several filter methods
rgw/d4n: fix compilation issue.
rgw/cache: Add `del` method to CacheDriver and SSDDriver
cmake/d4n: Remove unnecessary D4N lines
rgw: Add `io_context` to D4N Filter and RedisDriver, remove `cpp_redis`
library from RedisDriver, and perform minor cleanup
d4n: Remove `cpp_redis` library from D4N directory and policy; update calls in filter; move Entry struct to base CachePolicy class
build/cpp_redis: Remove `cpp_redis` library
rgw/d4n: including <boost/asio/detached.hpp> wheerever needed.
rgw/d4n : fixes to d4n filter, policy, directory and redis driver files
for compilation errors.
rgw/d4n: copying over read op params of the filter driver to
the read op params of 'next' in prepare() method, to be used
while storing objects to the backend store.
rgw/cache: moving LRU related data structures and interfaces to the Policy Manager.
Replacing the related calls in d4n filter driver with that of the cachepolicy in the
Policy Manager and adding cache block to the block directory
once the block is written to the cache backend and the
policy manager data structure.
rgw/cache: this commit removes indexing from ssd and redis
driver and squashes the following commits.
rgw/cache: modifications to the ssd cache driver after removing
indexing from the cache backend.
rgw/cache: modifications to redis cache driver, after
removing indexing from cache backend.
rgw/cache: modifications to policy driver code, commenting out
some pieces which refer to cache apis. Those apis have been
removed as part of modification to remove indexing from the
cache backend. These need to appropriately re-instated, once
we finalize which module will contain the data structures
related to cache policy.
rgw/cache: this commit squashes commits related to removing
cache_aio_read()/cache_aio_write() from cache api, ssd driver
and redis driver.
rgw/cache: removing cache_aio_read()/cache_aio_write()
interfaces from the Cache API. And moving cache_read_op
from rgw_aio.cc to ssd cache driver.
rgw/cache: changes related to cache_aio_read()/cache_aio_write()
in redis cache driver.
rgw/cache: implementation of put_async method in ssd driver
and redis driver and squashes the following commits.
rgw/cache: implementation of async put. the call
does not take into account throttling for now.
rgw/cache: dummy implementation of put_async in redis
driver to fix compilation error.
rgw/cache: This commit squashes the following commits
related to xattrs in ssd driver.
rgw/cache: code to handle xattrs, which are stored as
file xattrs.
rgw/cache: correcting ssd driver get_attrs.
rgw/cache: fixing set_attrs in ssd cache driver.
Samarah [Thu, 25 May 2023 18:58:26 +0000 (14:58 -0400)]
rgw/d4n: This commit squashes the following commits related to
d4n policy and filter driver.
RGW: Add cache driver base class - add d4n filter driver.
RGW: Update cache and policy files; add RedisDriver into D4N filter
RGW: Add D4N classes and unit testing; update cpp_redis submodule
QA: Add D4N teuthology suite
RGW: Improve D4N readability and structure
RGW: Add base D4N policy and test
RGW: Add GWF policy to D4N
RGW: Add global weight property
RGW: Added D4N namespace
RGW: Update policy driver interface
RGW: Update unit tests
RGW: Add Address struct to all files
RGW: Update D4N names and structure
RGW: Update structure and fix errors
RGW: Add more features to policy driver
RGW: Access local weight in policy code
RGW: Work on D4N workflows
RGW: Add object directory class and remove copy_value method
RGW: Establish object directory in read workflow
RGW: Update cache and policy files; add RedisDriver into D4N filter
RGW: Switch out D4N cache methods with Redis driver methods
RGW: Update RedisDriver to match new CacheDriver structure; define set_attrs method
RGW: Fix D4N read workflow crashes
RGW: Update D4N files to match CacheDriver changes
RGW: fix key_exists method for RedisDriver and clean up rgw_sal_d4n.cc
RGW: Use correct get_block method
RGW: Make CachePolicy a virtual class
RGW: Initialize localWeight if not found and develop find_victim method
RGW: Debugging network failure
RGW: Rebase and debugging network failure
RGW: Update RedisDriver::list_entries and usage in D4N policy driver
RGW: Fix network failure issue; add entries and entry methods
RGW: Update D4N test files to match rebase
RGW: Update D4N policy and RedisDriver with entries
RGW: make localWeight an attribute
rgw/cache: commit to fix compilation issues.
Samarah [Thu, 8 Jun 2023 17:10:58 +0000 (13:10 -0400)]
rgw/cache: This commit squashes the following commits related to list_entries cache api.
RGW: Add list_entries method to CacheDriver and to Redis driver.
RGW: Change CacheDriver::list_entries output type
RGW: Update RedisDriver::list_entries and usage in D4N policy driver
rgw/cache: This commit squashes modifications to base class.
1. Rename namespace from rgw::cal to rgw::cache
2. Remove Partition from the CacheDriver constructor.
3. Removing rgw_cache_driver.cc since all methods
are pure virtual in rgw_cache_driver.h.
4. Moving structures Partition and Entry out of
class CacheDriver. And modifying the signatures of methods
get_current_partition_info() and get_free_space().
5. modifying the cache driver base class.
projects / ceph.git / log