From: Nitzan Mordechai Date: Thu, 17 Jul 2025 06:17:00 +0000 (+0000) Subject: mgr: replace TTLCache with MgrMapCache and protect api with readonly X-Git-Tag: v21.0.1~61^2~4 X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=403340bcf8c2122afff708519f176baa6b646fc1;p=ceph.git mgr: replace TTLCache with MgrMapCache and protect api with readonly This patch removes the old TTLCache implementation and introduces a new generic MgrMapCache driven by a runtime toggle: - Add `mgr_map_cache_enabled` config option in global.yaml - Swap out `ttl_cache` for `api_cache` (MgrMapCache) in ActivePyModules - Update cacheable_get_python() and get_python() to use LFU‐based api_cache - add new get_mutable parameter to the get api call to get a copy. - Invalidate api_cache on notify_all events - Remove all TTLCache headers, sources, and tests - Include MgrMapCache.cc in CMakeLists and update BaseMgrModule bindings - Improve logging around cache hits, misses, and state changes - ActivePyModules * Remove unused update_cache_metrics() * Log cache hits/misses inline and only insert into cache when enabled+cacheable (with proper Py_INCREF) * Switch get_python() to use PyFormatterRO for cacheable routes, PyFormatter otherwise - MgrMapCache/LFUCache * Add can_read_cache()/can_write_cache() helpers and use const& for key parameters * Guard perf counter increments and improve debug logging - PyFormatter * Add PyFormatterRO subclass that freezes dicts/lists into read-only proxies on the fly - Python mgr_module * Simplify get() to return raw result This change ensures immutable JSON output on cache hits and tightens up cache logic. mgr/cli: add cache flush command with proper status reporting Allow operators to invalidate individual mgr Python caches at runtime without restarting the manager. Introduces a new CLI command: $ ceph mgr cli cache flush which returns success or a clear error if the named cache entry doesn’t exist or isn’t cacheable. This makes it easy to drop stale cached maps (e.g. osd_map, mon_map) on demand. Fixes: https://tracker.ceph.com/issues/72447 Signed-off-by: Nitzan Mordechai mgr: add new unit tests for MgrMapCache - Guard against null perf‐counter before calling inc(), preventing crashes - Add “foo” to allowed_keys list (for test coverage) - Rename and refocus the CMake test target from TTLCache to MgrMapCache - Introduce test_mgrmapcache.cc with LFUCache tests. - Remove the obsolete test_ttlcache.cc Fixes: https://tracker.ceph.com/issues/72447 Signed-off-by: Nitzan Mordechai mgr/test_cache: add new tests adding new unit-test for mgrcache Fixes: https://tracker.ceph.com/issues/72447 Signed-off-by: Nitzan Mordechai --- diff --git a/PendingReleaseNotes b/PendingReleaseNotes index 2eb6e6be2e8..80693534172 100644 --- a/PendingReleaseNotes +++ b/PendingReleaseNotes @@ -110,6 +110,11 @@ behavior is controlled by the new ``mgr_stats_period_autotune`` (default: ``true``) and ``mgr_stats_period_autotune_queue_threshold`` (default: ``100``) config options. +* MGR: The Manager cache system has been redesigned from a TTL-based (time-to-live) + approach to an event-driven invalidation strategy. The cache now automatically + invalidates entries when underlying cluster maps are updated. The cache is enabled + by default. The previous `mgr_ttl_cache_expire_seconds` configuration option has + been removed and replaced with `mgr_map_cache_enabled` (default: true). >=20.0.0 diff --git a/doc/mgr/administrator.rst b/doc/mgr/administrator.rst index a6a7047577c..6d60e897b58 100644 --- a/doc/mgr/administrator.rst +++ b/doc/mgr/administrator.rst @@ -107,20 +107,29 @@ daemon as failed using ``ceph mgr fail ``. Performance and Scalability --------------------------- - -All the Manager modules share a cache that can be enabled with -``ceph config set mgr mgr_ttl_cache_expire_seconds ``, where seconds -is the time to live of the cached python objects. - -It is recommended to enable the cache with a 10 seconds TTL when there are 500+ -osds or 10k+ pgs as internal structures might increase in size, and cause latency -issues when requesting large structures. As an example, an OSDMap with 1000 osds -has a approximate size of 4MiB. With heavy load, on a 3000 osd cluster there has -been a 1.5x improvement enabling the cache. - -Furthermore, you can run ``ceph daemon mgr.${MGRNAME} perf dump`` to retrieve -perf counters of a mgr module. In ``mgr.cache_hit`` and ``mgr.cache_miss`` -you'll find the hit/miss ratio of the mgr cache. +Manager modules share a cache that is enabled by default. The cache +uses an event-driven invalidation strategy, automatically updating when cluster +maps change to ensure modules always work with current data while maximizing +performance. + +The cache is particularly beneficial for clusters with 500+ OSDs or 10k+ PGs +as internal structures increase in size, which may result in latency issues when +requesting large structures. As an example, an OSDMap with 1000 OSDs has an +approximate size of 4MiB. With heavy load, on a 3000 OSD cluster, the response +latency for cached requests reduces by approximately 50% when the cache +is enabled. + +The cache automatically invalidates entries when the underlying cluster maps +(such as OSDMap, PGMap, or MonMap) are updated. If needed, you can manually +flush specific cached maps using ``ceph mgr cli cache flush [map-name]``. + +To disable the cache (not recommended for large clusters), run: +``ceph config set mgr mgr_map_cache_enabled false`` + +You can run ``ceph daemon mgr.${MGRNAME} perf dump`` to retrieve +perf counters of a Manager module. In ``mgr.cache_hit`` and ``mgr.cache_miss`` +you'll find the hit/miss ratio of the Manager cache, which can help verify the +cache is operating effectively. The Manager includes a ThreadMonitor that tracks CPU usage and memory consumption for each enabled module. This monitoring can be configured with diff --git a/doc/mgr/cli_api.rst b/doc/mgr/cli_api.rst index 86dfd281392..54162a7abe9 100644 --- a/doc/mgr/cli_api.rst +++ b/doc/mgr/cli_api.rst @@ -52,3 +52,11 @@ For example, run the following command to benchmark the command to get .. prompt:: bash # ceph mgr cli_benchmark 100 10 get osd_map + + +Flush the saved cache map for the Ceph Manager CLI. Use this command to refresh the cache +and ensure the most up-to-date information is used: + +.. prompt:: bash # + + ceph mgr cli cache flush \ No newline at end of file diff --git a/qa/mgr_ttl_cache/disable.yaml b/qa/mgr_ttl_cache/disable.yaml deleted file mode 100644 index bbd78d53f2f..00000000000 --- a/qa/mgr_ttl_cache/disable.yaml +++ /dev/null @@ -1,5 +0,0 @@ -overrides: - ceph: - conf: - mgr: - mgr ttl cache expire seconds: 0 diff --git a/qa/mgr_ttl_cache/enable.yaml b/qa/mgr_ttl_cache/enable.yaml deleted file mode 100644 index 2c1c0e0533e..00000000000 --- a/qa/mgr_ttl_cache/enable.yaml +++ /dev/null @@ -1,5 +0,0 @@ -overrides: - ceph: - conf: - mgr: - mgr ttl cache expire seconds: 5 diff --git a/qa/suites/rados/mgr/mgr_ttl_cache/.qa b/qa/suites/rados/mgr/mgr_ttl_cache/.qa deleted file mode 120000 index a602a0353e7..00000000000 --- a/qa/suites/rados/mgr/mgr_ttl_cache/.qa +++ /dev/null @@ -1 +0,0 @@ -../.qa/ \ No newline at end of file diff --git a/qa/suites/rados/mgr/mgr_ttl_cache/disable.yaml b/qa/suites/rados/mgr/mgr_ttl_cache/disable.yaml deleted file mode 120000 index d7db486dd9f..00000000000 --- a/qa/suites/rados/mgr/mgr_ttl_cache/disable.yaml +++ /dev/null @@ -1 +0,0 @@ -.qa/mgr_ttl_cache/disable.yaml \ No newline at end of file diff --git a/qa/suites/rados/mgr/mgr_ttl_cache/enable.yaml b/qa/suites/rados/mgr/mgr_ttl_cache/enable.yaml deleted file mode 120000 index 18286a656ba..00000000000 --- a/qa/suites/rados/mgr/mgr_ttl_cache/enable.yaml +++ /dev/null @@ -1 +0,0 @@ -.qa/mgr_ttl_cache/enable.yaml \ No newline at end of file diff --git a/qa/tasks/mgr/test_cache.py b/qa/tasks/mgr/test_cache.py index 7a05de47c17..8dd2a005c81 100644 --- a/qa/tasks/mgr/test_cache.py +++ b/qa/tasks/mgr/test_cache.py @@ -1,83 +1,148 @@ import json +import logging +import uuid +import threading +from concurrent.futures import ThreadPoolExecutor from .mgr_test_case import MgrTestCase +log = logging.getLogger(__name__) + class TestCache(MgrTestCase): def setUp(self): + log.info("TestCache setup") super(TestCache, self).setUp() + log.info("Setting up mgrs") self.setup_mgrs() + log.info("Loading cli_api module") self._load_module("cli_api") - self.ttl = 10 - self.enable_cache(self.ttl) + log.info("Enabling cache") + self.enable_cache() - def tearDown(self): - self.disable_cache() + def enable_cache(self, on=True): + cache_set = 'true' if on else 'false' + self.mgr_cluster.mon_manager.raw_cluster_cmd('config', 'set', 'mgr', 'mgr_map_cache_enabled', cache_set) def get_hit_miss_ratio(self): - perf_dump_command = f"daemon mgr.{self.mgr_cluster.get_active_id()} perf dump" - perf_dump_res = self.cluster_cmd(perf_dump_command) - perf_dump = json.loads(perf_dump_res) - h = perf_dump["mgr"]["cache_hit"] - m = perf_dump["mgr"]["cache_miss"] - return int(h), int(m) - - def enable_cache(self, ttl): - set_ttl = f"config set mgr mgr_ttl_cache_expire_seconds {ttl}" - self.cluster_cmd(set_ttl) - - def disable_cache(self): - set_ttl = "config set mgr mgr_ttl_cache_expire_seconds 0" - self.cluster_cmd(set_ttl) - - + perf_dump = self.mgr_cluster.mon_manager.raw_cluster_cmd('daemon', f'mgr.{self.mgr_cluster.get_active_id()}', 'perf', 'dump') + pd = json.loads(perf_dump) + return int(pd["mgr"]["cache_hit"]), int(pd["mgr"]["cache_miss"]) + + def flush_cache_map(self, what): + self.mgr_cluster.mon_manager.raw_cluster_cmd('mgr', 'cli', 'cache', 'flush', what) + + def create_pool(self, pool_name): + self.mgr_cluster.mon_manager.raw_cluster_cmd('osd', 'pool', 'create', pool_name, '1', '--yes-i-really-mean-it') + + def remove_pool(self, pool_name): + self.config_set('mon', 'mon_allow_pool_delete', 'true') + self.mgr_cluster.mon_manager.raw_cluster_cmd('osd', 'pool', 'rm', pool_name, pool_name, '--yes-i-really-really-mean-it-not-faking') + + def osd_epoch(self): + m = json.loads(self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map")) + return int(m["epoch"]) + + def bump_osdmap(self): + pool = f"foo_{uuid.uuid4().hex[:8]}" + self.create_pool(pool) + start = self.osd_epoch() + def ok(): + return self.osd_epoch() > start + self.wait_until_true(ok, 30) + self.remove_pool(pool) + + # Init cache def test_init_cache(self): - get_ttl = "config get mgr mgr_ttl_cache_expire_seconds" - res = self.cluster_cmd(get_ttl) - self.assertEqual(int(res), 10) - - def test_health_not_cached(self): - get_health = "mgr api get health" - - h_start, m_start = self.get_hit_miss_ratio() - self.cluster_cmd(get_health) - h, m = self.get_hit_miss_ratio() - - self.assertEqual(h, h_start) - self.assertEqual(m, m_start) - + res = self.mgr_cluster.mon_manager.raw_cluster_cmd('config', 'get', 'mgr', 'mgr_map_cache_enabled') + self.assertEqual(res.strip().lower(), "true") + + # Disabled bypass + def test_disabled_bypass(self): + self.enable_cache(False) + h0, m0 = self.get_hit_miss_ratio() + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") + h1, m1 = self.get_hit_miss_ratio() + self.assertEqual((h1, m1), (h0, m0)) + self.enable_cache(True) + + # Non-cacheable key ignored (health) + def test_non_cacheable_stays_uncached(self): + h0, m0 = self.get_hit_miss_ratio() + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "health") + h1, m1 = self.get_hit_miss_ratio() + self.assertEqual((h1, m1), (h0, m0)) + + # Cache hit after warm + def test_osdmap_hit_after_warm(self): + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") + h0, m0 = self.get_hit_miss_ratio() + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") + h1, m1 = self.get_hit_miss_ratio() + self.assertGreater(h1, h0) + self.assertEqual(m1, m0) + + # Invalidate on osdmap change → miss then hit + def test_invalidate_on_osdmap_change(self): + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") # warm + h0, m0 = self.get_hit_miss_ratio() + self.bump_osdmap() # should invalidate + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") # miss + h1, m1 = self.get_hit_miss_ratio() + self.assertGreater(m1, m0) + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") # hit + h2, m2 = self.get_hit_miss_ratio() + self.assertGreater(h2, h1) + self.assertEqual(m2, m1) + + # Concurrency: many reads → one miss, rest hits + def test_concurrent_reads_single_miss(self): + self.enable_cache(True) + self.bump_osdmap() + h0, m0 = self.get_hit_miss_ratio() + N = 8 + def read_once(_): + return self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") + with ThreadPoolExecutor(max_workers=N) as ex: + list(ex.map(read_once, range(N))) + h1, m1 = self.get_hit_miss_ratio() + # Allow either 1 miss or small race overfill; assert lower bound + self.assertGreaterEqual(h1 - h0, N - 1) + self.assertGreaterEqual(m1 - m0, 1) + + # Another cacheable key (mon_status) behaves like osd_map + def test_mon_status_cached(self): + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "mon_status") # warm + h0, m0 = self.get_hit_miss_ratio() + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "mon_status") + h1, m1 = self.get_hit_miss_ratio() + self.assertGreater(h1, h0) + self.assertEqual(m1, m0) + + # Stress invalidate while reading (race safety) + def test_race_read_vs_invalidate(self): + stop = False + def reader(): + while not stop: + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") + t = threading.Thread(target=reader) + t.start() + try: + for _ in range(3): + self.bump_osdmap() # triggers invalidation in mgr + finally: + stop = True + t.join() + # If we reach here without exceptions or crashes, pass + self.assertTrue(True) + + # test get api def test_osdmap(self): - get_osdmap = "mgr api get osd_map" - # store in cache - self.cluster_cmd(get_osdmap) + self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") # get from cache - res = self.cluster_cmd(get_osdmap) + res = self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "cli", "get", "osd_map") osd_map = json.loads(res) self.assertIn("osds", osd_map) self.assertGreater(len(osd_map["osds"]), 0) self.assertIn("epoch", osd_map) - - - - def test_hit_miss_ratio(self): - get_osdmap = "mgr api get osd_map" - - hit_start, miss_start = self.get_hit_miss_ratio() - - def wait_miss(): - self.cluster_cmd(get_osdmap) - _, m = self.get_hit_miss_ratio() - return m == miss_start + 1 - - # Miss, add osd_map to cache - self.wait_until_true(wait_miss, self.ttl + 5) - h, m = self.get_hit_miss_ratio() - self.assertEqual(h, hit_start) - self.assertEqual(m, miss_start+1) - - # Hit, get osd_map from cache - self.cluster_cmd(get_osdmap) - h, m = self.get_hit_miss_ratio() - self.assertEqual(h, hit_start+1) - self.assertEqual(m, miss_start+1) diff --git a/src/common/options/global.yaml.in b/src/common/options/global.yaml.in index 2673fded73a..997c3c1c6a8 100644 --- a/src/common/options/global.yaml.in +++ b/src/common/options/global.yaml.in @@ -6730,6 +6730,15 @@ options: desc: Enable / disable the Manager op tracker default: true with_legacy: true +- name: mgr_map_cache_enabled + type: bool + level: dev + desc: Enable the manager's map cache for API calls + default: true + services: + - mgr + flags: + - runtime - name: mgr_num_op_tracker_shard type: uint level: advanced @@ -7006,13 +7015,6 @@ options: services: - rgw - osd -- name: mgr_ttl_cache_expire_seconds - type: uint - level: dev - desc: Set the Manager cache time to live in seconds; set to 0 to disable the cache. - default: 0 - services: - - mgr - name: objectstore_debug_throw_on_failed_txc type: bool level: dev diff --git a/src/mgr/ActivePyModules.cc b/src/mgr/ActivePyModules.cc index 93ad567def4..ba11fa2d21c 100644 --- a/src/mgr/ActivePyModules.cc +++ b/src/mgr/ActivePyModules.cc @@ -29,7 +29,7 @@ #include "osd/OSDMap.h" #include "osd/osd_types.h" #include "mgr/MgrContext.h" -#include "mgr/TTLCache.h" +#include "mgr/MgrMapCache.h" #include "mgr/mgr_perf_counters.h" #include "messages/MMgrReport.h" // for class PerfCounterType @@ -197,41 +197,55 @@ PyObject *ActivePyModules::get_daemon_status_python( return f.get(); } -void ActivePyModules::update_cache_metrics() { - auto hit_miss_ratio = ttl_cache.get_hit_miss_ratio(); - perfcounter->set(l_mgr_cache_hit, hit_miss_ratio.first); - perfcounter->set(l_mgr_cache_miss, hit_miss_ratio.second); +int ActivePyModules::ceph_cache_map_erase(std::string_view what) +{ + if (!api_cache.exists(what)) { + dout(10) << " what: " << what << " not in cache" << dendl; + return -ENOENT; + } else if (!api_cache.is_cacheable(what)) { + dout(10) << " what: " << what << " not cacheable" << dendl; + return -EINVAL; + } + dout(10) << " what: " << what << dendl; + api_cache.erase(what); + return 0; } -PyObject *ActivePyModules::cacheable_get_python(const std::string &what) +PyObject *ActivePyModules::cacheable_get_python(std::string_view what, const bool get_mutable) { - uint64_t ttl_seconds = g_conf().get_val("mgr_ttl_cache_expire_seconds"); - if(ttl_seconds > 0) { - ttl_cache.set_ttl(ttl_seconds); - try{ - PyObject* cached = ttl_cache.get(what); - update_cache_metrics(); + const bool use_cache = + !get_mutable && + api_cache.is_enabled() && + api_cache.is_cacheable(what); + if (use_cache) { + PyObject* cached = api_cache.get(what); + if (cached) { + dout(20) << ": api cache hit for " << what << " hit/miss " + << api_cache.get_hits() << "/" << api_cache.get_misses() + << dendl; return cached; - } catch (std::out_of_range& e) {} + } } - PyObject *obj = get_python(what); - if(ttl_seconds && ttl_cache.is_cacheable(what)) { - ttl_cache.insert(what, obj); + PyObject *obj = get_python(what, get_mutable); + if (use_cache && obj) { + api_cache.insert(what, obj); } - update_cache_metrics(); return obj; } -PyObject *ActivePyModules::get_python(const std::string &what) +PyObject *ActivePyModules::get_python(std::string_view what, const bool get_mutable) { - uint64_t ttl_seconds = g_conf().get_val("mgr_ttl_cache_expire_seconds"); - - PyFormatter pf; - PyJSONFormatter jf; - // Use PyJSONFormatter if TTL cache is enabled. - Formatter &f = ttl_seconds ? (Formatter&)jf : (Formatter&)pf; - + const bool use_cache = + !get_mutable && + api_cache.is_enabled() && + api_cache.is_cacheable(what) && + PyGILState_Check(); + + PyFormatter py_formatter; + PyFormatterRO py_formatter_ro; + PyFormatter &f = use_cache ? (PyFormatter&)py_formatter_ro : + py_formatter; if (what == "fs_map") { without_gil_t no_gil; cluster_state.with_fsmap([&](const FSMap &fsmap) { @@ -404,7 +418,7 @@ PyObject *ActivePyModules::get_python(const std::string &what) } else if (what.size() > 7 && what.substr(0, 7) == "device ") { without_gil_t no_gil; - string devid = what.substr(7); + string devid(what.substr(7)); if (!daemon_state.with_device(devid, [&] (const DeviceState& dev) { with_gil_t with_gil{no_gil}; @@ -535,11 +549,8 @@ PyObject *ActivePyModules::get_python(const std::string &what) derr << "Python module requested unknown data '" << what << "'" << dendl; Py_RETURN_NONE; } - if(ttl_seconds) { - return jf.get(); - } else { - return pf.get(); - } + + return f.get(); } void ActivePyModules::start_one(PyModuleRef py_module) @@ -594,6 +605,9 @@ void ActivePyModules::notify_all(const std::string ¬ify_type, const std::string ¬ify_id) { std::lock_guard l(lock); + + // invalidate api cache for this notify type + api_cache.invalidate(notify_type); dout(10) << __func__ << ": notify_all " << notify_type << dendl; for (auto& [name, module] : modules) { @@ -1563,6 +1577,7 @@ void ActivePyModules::get_progress_events(std::map *e void ActivePyModules::config_notify() { std::lock_guard l(lock); + api_cache.invalidate("config"); for (auto& [name, module] : modules) { // Send all python calls down a Finisher to avoid blocking // C++ code, and avoid any potential lock cycles. diff --git a/src/mgr/ActivePyModules.h b/src/mgr/ActivePyModules.h index 151be14926c..3a5a129ef9c 100644 --- a/src/mgr/ActivePyModules.h +++ b/src/mgr/ActivePyModules.h @@ -28,7 +28,7 @@ #include "mon/mon_types.h" #include "mon/ConfigMap.h" #include "mgr/MDSPerfMetricTypes.h" -#include "mgr/TTLCache.h" +#include "mgr/MgrMapCache.h" #include "DaemonState.h" #include "ClusterState.h" @@ -38,6 +38,7 @@ #include #include #include +#include class health_check_map_t; class DaemonServer; @@ -66,8 +67,8 @@ class ActivePyModules LogChannelRef clog, audit_clog; Objecter &objecter; Finisher &finisher; - TTLCache ttl_cache; ThreadMonitor* m_thread_monitor = nullptr; + MgrMapCache api_cache; public: Finisher cmd_finisher; private: @@ -92,8 +93,11 @@ public: // FIXME: wrap for send_command? MonClient &get_monc() {return monc;} Objecter &get_objecter() {return objecter;} - PyObject *cacheable_get_python(const std::string &what); - PyObject *get_python(const std::string &what); + PyObject *cacheable_get_python(std::string_view what, + const bool get_mutable = false); + PyObject *get_python(std::string_view what, + const bool get_mutable = false); + int ceph_cache_map_erase(std::string_view what); PyObject *get_server_python(const std::string &hostname); PyObject *list_servers_python(); PyObject *get_metadata_python( diff --git a/src/mgr/BaseMgrModule.cc b/src/mgr/BaseMgrModule.cc index 5be199e490e..8e34159aec7 100644 --- a/src/mgr/BaseMgrModule.cc +++ b/src/mgr/BaseMgrModule.cc @@ -384,11 +384,31 @@ static PyObject* ceph_state_get(BaseMgrModule *self, PyObject *args) { char *what = NULL; - if (!PyArg_ParseTuple(args, "s:ceph_state_get", &what)) { + int get_mutable = 0; // Default to False + dout(10) << __func__ << " called" << dendl; + if (!PyArg_ParseTuple(args, "s|i:ceph_state_get", &what, &get_mutable)) { + derr << __func__ << " Invalid args!" << dendl; return NULL; } + dout(10) << __func__ << " what: " << what << " mutable: " << get_mutable << dendl; + return self->py_modules->cacheable_get_python(what, (bool)get_mutable); +} - return self->py_modules->cacheable_get_python(what); +static PyObject* +ceph_cache_map_erase(BaseMgrModule *self, PyObject *args) +{ + char *what = NULL; + dout(10) << __func__ << " called" << dendl; + if (!PyArg_ParseTuple(args, "s:ceph_cache_map_erase", &what)) { + derr << __func__ << " Invalid args!" << dendl; + Py_RETURN_FALSE; + } + dout(10) << __func__ << " what: " << what << dendl; + if (self->py_modules->ceph_cache_map_erase(what) < 0) { + dout(10) << __func__ << " failed to erase cache map entry: " << what << dendl; + Py_RETURN_FALSE; + } + Py_RETURN_TRUE; } @@ -1588,6 +1608,9 @@ PyMethodDef BaseMgrModule_methods[] = { {"_ceph_get", (PyCFunction)ceph_state_get, METH_VARARGS, "Get a cluster object"}, + {"_ceph_erase", (PyCFunction)ceph_cache_map_erase, METH_VARARGS, + "Erase a cached python map"}, + {"_ceph_notify_all", (PyCFunction)ceph_notify_all, METH_VARARGS, "notify all modules"}, diff --git a/src/mgr/BaseMgrStandbyModule.cc b/src/mgr/BaseMgrStandbyModule.cc index 8ac3c3b678e..7f45620faa4 100644 --- a/src/mgr/BaseMgrStandbyModule.cc +++ b/src/mgr/BaseMgrStandbyModule.cc @@ -173,7 +173,8 @@ static PyObject* ceph_standby_state_get(BaseMgrStandbyModule *self, PyObject *args) { char *whatc = NULL; - if (!PyArg_ParseTuple(args, "s:ceph_state_get", &whatc)) { + int get_mutable = 0; + if (!PyArg_ParseTuple(args, "s|i:ceph_state_get", &whatc, &get_mutable)) { return NULL; } std::string what(whatc); diff --git a/src/mgr/CMakeLists.txt b/src/mgr/CMakeLists.txt index 61d02cb884c..a8893e78028 100644 --- a/src/mgr/CMakeLists.txt +++ b/src/mgr/CMakeLists.txt @@ -36,6 +36,7 @@ if(WITH_MGR) StandbyPyModules.cc ThreadMonitor.cc mgr_commands.cc + MgrMapCache.cc MgrOpRequest.cc ${CMAKE_SOURCE_DIR}/src/common/TrackedOp.cc $) diff --git a/src/mgr/Mgr.cc b/src/mgr/Mgr.cc index 50fb01fce48..174682e74f8 100644 --- a/src/mgr/Mgr.cc +++ b/src/mgr/Mgr.cc @@ -622,10 +622,12 @@ Dispatcher::dispatch_result_t Mgr::ms_dispatch2(const ref_t& m) case CEPH_MSG_FS_MAP: handle_fs_map(ref_cast(m)); py_module_registry->notify_all("fs_map", ""); + py_module_registry->notify_all("mds_metadata", ""); return Dispatcher::ACKNOWLEDGED(); case CEPH_MSG_OSD_MAP: handle_osd_map(); py_module_registry->notify_all("osd_map", ""); + py_module_registry->notify_all("osd_metadata", ""); // Continuous subscribe, so that we can generate notifications // for our MgrPyModules @@ -762,6 +764,7 @@ bool Mgr::got_mgr_map(const MgrMap& m) cluster_state.with_mgrmap([&](const MgrMap& m) { old_modules = m.modules; }); + py_module_registry->notify_all("mgr_map", ""); if (m.modules != old_modules) { derr << "mgrmap module list changed to (" << m.modules << "), respawn" << dendl; @@ -805,12 +808,14 @@ void Mgr::handle_mgr_digest(ref_t m) dout(10) << m->mon_status_json.length() << dendl; dout(10) << m->health_json.length() << dendl; cluster_state.load_digest(m.get()); - //no users: py_module_registry->notify_all("mon_status", ""); + py_module_registry->notify_all("mon_status", ""); py_module_registry->notify_all("health", ""); // Hack: use this as a tick/opportunity to prompt python-land that // the pgmap might have changed since last time we were here. py_module_registry->notify_all("pg_summary", ""); + py_module_registry->notify_all("pg_stats", ""); + py_module_registry->notify_all("pg_dump", ""); dout(10) << "done." << dendl; m.reset(); diff --git a/src/mgr/MgrMapCache.cc b/src/mgr/MgrMapCache.cc new file mode 100644 index 00000000000..de5c5cadb82 --- /dev/null +++ b/src/mgr/MgrMapCache.cc @@ -0,0 +1,237 @@ +// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:nil -*- +// vim: ts=8 sw=2 sts=2 expandtab + +/* + * Ceph - scalable distributed file system + * + * Copyright (C) 2026 IBM + * + * This is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License version 2.1, as published by the Free Software + * Foundation. See file COPYING. + * + */ + +#include "mgr/MgrMapCache.h" + +#include +#include + +#include "common/config_proxy.h" +#include "common/debug.h" +#include "global/global_context.h" + +#define dout_context g_ceph_context +#define dout_subsys ceph_subsys_mgr +#undef dout_prefix +#define dout_prefix *_dout << "api cache " << __func__ << " " + +static const std::unordered_set mgr_cache_keys = { + "osd_map", "pg_dump", "pg_stats", "mon_status", "mgr_map", + "osd_metadata", "mds_metadata", "config" +}; + +template +MgrMapCache::~MgrMapCache() { + g_conf().remove_observer(this); +} + +template +bool LFUCache::extract(std::string_view k, Value* out) noexcept { + std::unique_lock l(m); + auto it = cache_data.find(k); + if (it == cache_data.end()) return false; + *out = it->second.val; + cache_data.erase(it); + return true; +} + +template +void LFUCache::drain(std::vector& out) noexcept { + std::unique_lock l(m); + out.reserve(cache_data.size()); + for (auto& kv : cache_data) out.push_back(kv.second.val); + cache_data.clear(); + hits.store(0); + misses.store(0); +} + +template +Value LFUCache::get(std::string_view k) { + std::shared_lock l(m); + if (!is_enabled()) throw std::out_of_range("cache disabled"); + auto it = cache_data.find(k); + if (it == cache_data.end()) { + throw std::out_of_range(std::string(k)); + } + it->second.hits.fetch_add(1, std::memory_order_relaxed); + mark_hit(); + return it->second.val; +} + +template +bool LFUCache::try_get(std::string_view k, Value* out, bool count_hit) noexcept { + std::shared_lock l(m); + auto it = cache_data.find(k); + if (it == cache_data.end()) { + if (count_hit) { + mark_miss(); + } + return false; + } + if (count_hit) { + it->second.hits.fetch_add(1, std::memory_order_relaxed); + mark_hit(); + } + *out = it->second.val; + return true; +} + +template +typename LFUCache::InsertRes +LFUCache::insert(std::string_view key, Value value) { + if (!can_write_cache(key)) { + return InsertRes{false}; + } + + std::unique_lock l(m); + // Re-check enabled after acquiring lock: state may have flipped while waiting. + if (!enabled.load(std::memory_order_relaxed)) { + return InsertRes{false}; + } + + auto it = cache_data.find(key); + if (it != cache_data.end()) { + InsertRes res{true}; + res.replaced = std::move(it->second.val); + it->second.val = std::move(value); + return res; + } + + // New insert counts as a miss (cache didn't have it) + mark_miss(); + + InsertRes res{true}; + if (cache_data.size() >= capacity && !cache_data.empty()) { + auto min_it = std::min_element(cache_data.begin(), + cache_data.end(), + [](const auto& a, const auto& b) { + return a.second.hits.load(std::memory_order_relaxed) < + b.second.hits.load(std::memory_order_relaxed); + }); + res.evicted = std::move(min_it->second.val); + cache_data.erase(min_it); + } + + // Allocate std::string only here, when we actually need to store a new key. + cache_data.emplace(std::string(key), Entry(std::move(value))); + return res; +} + +template +MgrMapCache::MgrMapCache(uint16_t size) + : CacheImp(mgr_cache_keys, size, g_conf().get_val("mgr_map_cache_enabled")) { + dout(20) << ": creating cache with size " << size << dendl; + g_conf().add_observer(this); +} + +template +void MgrMapCache::handle_conf_change( + const ConfigProxy& conf, const std::set& changed) { + if (changed.count("mgr_map_cache_enabled")) { + this->set_enabled(conf.get_val("mgr_map_cache_enabled")); + } +} + +MgrMapCache::MgrMapCache(uint16_t size) + : CacheImp(mgr_cache_keys, size, g_conf().get_val("mgr_map_cache_enabled")) { + dout(20) << ": creating cache with size " << size << dendl; + g_conf().add_observer(this); +} + +MgrMapCache::~MgrMapCache() { + g_conf().remove_observer(this); + this->clear(); +} + +PyObject* MgrMapCache::get(std::string_view k) { + if (!this->is_enabled() || + !this->is_cacheable(k) || + !PyGILState_Check()) + return nullptr; + std::shared_lock l(this->m); + auto it = this->cache_data.find(k); + if (it == this->cache_data.end()) { + this->mark_miss(); + return nullptr; + } + PyObject* o = it->second.val; + Py_INCREF(o); // INCREF under lock: no window for erase() to drop refcount to zero + it->second.hits.fetch_add(1, std::memory_order_relaxed); + this->mark_hit(); + return o; +} + +void MgrMapCache::insert(std::string_view key, PyObject* value) { + if (!this->can_write_cache(key) || !PyGILState_Check()) { + return; + } + + Py_INCREF(value); + auto res = CacheImp::insert(key, value); + + if (!res.inserted) { + // Cache was disabled between our check and lock acquisition; undo the INCREF. + Py_DECREF(value); + return; + } + + auto schedule_decref = [](PyObject* obj) { + if (!obj) return; + if (Py_AddPendingCall(+[](void* p){ Py_DECREF((PyObject*)p); return 0; }, obj) != 0) { + PyGILState_STATE st = PyGILState_Ensure(); + Py_DECREF(obj); + PyGILState_Release(st); + } + }; + if (res.replaced.has_value()) schedule_decref(res.replaced.value()); + if (res.evicted.has_value()) schedule_decref(res.evicted.value()); + + dout(20) << ": inserted key: " << key << " py count: " + << Py_REFCNT(value) << " hit/miss:" + << CacheImp::get_hits() << "/" + << CacheImp::get_misses() << dendl; +} + +void MgrMapCache::erase(std::string_view key) noexcept { + if (!this->is_cacheable(key)) return; + PyObject* o = nullptr; + if (!this->extract(key, &o)) return; + + Py_AddPendingCall(+[](void* p){ Py_DECREF((PyObject*)p); return 0; }, o); + dout(20) << ": erased key: " << key + << " hit/miss:" + << CacheImp::get_hits() << "/" + << CacheImp::get_misses() << dendl; +} + +void MgrMapCache::clear() noexcept { + std::vector to_drop; + this->drain(to_drop); + if (to_drop.empty()) return; + PyGILState_STATE st = PyGILState_Ensure(); + for (auto* o : to_drop) Py_DECREF(o); + PyGILState_Release(st); + dout(20) << ": Cache cleared" << dendl; +} + +void MgrMapCache::handle_conf_change( + const ConfigProxy& conf, const std::set& changed) { + if (changed.count("mgr_map_cache_enabled")) { + this->set_enabled(conf.get_val("mgr_map_cache_enabled")); + } +} + +// Explicit instantiation for unit tests +template class LFUCache; diff --git a/src/mgr/MgrMapCache.h b/src/mgr/MgrMapCache.h new file mode 100644 index 00000000000..04b167c396e --- /dev/null +++ b/src/mgr/MgrMapCache.h @@ -0,0 +1,189 @@ +// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:nil -*- +// vim: ts=8 sw=2 sts=2 expandtab + +/* + * Ceph - scalable distributed file system + * + * Copyright (C) 2026 IBM + * + * This is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License version 2.1, as published by the Free Software + * Foundation. See file COPYING. + * + */ +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +#include "common/config_obs.h" +#include "common/perf_counters.h" +#include "mgr/mgr_perf_counters.h" +#include "PyUtil.h" + +// Transparent hash so unordered containers accept std::string_view lookups +// without constructing a std::string. +struct StringViewHash { + using is_transparent = void; + size_t operator()(std::string_view sv) const noexcept { + return std::hash{}(sv); + } +}; + +template +class LFUCache { + struct Entry { + Value val; + mutable std::atomic hits{0}; + + Entry() = default; + Entry(Value v) : val(std::move(v)), hits(0) {} + Entry(const Entry& o) : val(o.val), hits(o.hits.load()) {} + Entry(Entry&& o) noexcept : val(std::move(o.val)), hits(o.hits.load()) {} + }; + std::atomic hits{0}, misses{0}; + const std::unordered_set> allowed_keys; + +protected: + std::unordered_map> cache_data; + const size_t capacity; + std::atomic enabled{true}; + mutable std::shared_mutex m; + + void mark_miss() { + misses++; + if (perfcounter) + perfcounter->inc(l_mgr_cache_miss); + } + + void mark_hit() { + hits++; + if (perfcounter) + perfcounter->inc(l_mgr_cache_hit); + } + +public: + explicit LFUCache(std::unordered_set keys, + size_t cap = UINT16_MAX, + bool ena = true) + : allowed_keys(keys.begin(), keys.end()), capacity{cap}, enabled{ena} {} + virtual ~LFUCache() = default; + + void set_enabled(bool e) { + enabled.store(e); + if (!e) { + clear(); + } + } + + bool is_enabled() const noexcept { + return enabled.load(); + } + + size_t size() const { + std::shared_lock l(m); + return cache_data.size(); + } + + bool is_cacheable(std::string_view key) const noexcept { + return allowed_keys.find(key) != allowed_keys.end(); + } + + bool can_read_cache(std::string_view key) const noexcept { + return is_enabled() && is_cacheable(key) && exists(key); + } + + bool can_write_cache(std::string_view key) const noexcept { + return is_enabled() && is_cacheable(key); + } + + struct InsertRes { + bool inserted{false}; // false means the cache rejected the write + std::optional replaced; // set if an existing key was overwritten + std::optional evicted; // set if an entry was evicted to make room + }; + + bool try_get(std::string_view k, Value* out, bool count_hit = true) noexcept; + InsertRes insert(std::string_view key, Value value); + + bool erase(std::string_view key) { + std::unique_lock l(m); + auto it = cache_data.find(key); + if (it == cache_data.end()) return false; + cache_data.erase(it); + return true; + } + + bool extract(std::string_view k, Value* out) noexcept; + void drain(std::vector& out) noexcept; + + virtual void clear() { + std::unique_lock l(m); + cache_data.clear(); + hits.store(0); + misses.store(0); + } + + bool exists(std::string_view key) const noexcept { + std::shared_lock l(m); + return cache_data.find(key) != cache_data.end(); + } + + uint64_t get_hits() const noexcept { + return hits.load(); + } + + uint64_t get_misses() const noexcept { + return misses.load(); + } + + Value get(std::string_view k); +}; + + +// ---------- MgrMapCache generic ---------- +template +class MgrMapCache : public LFUCache, + public md_config_obs_t { + using CacheImp = LFUCache; +public: + explicit MgrMapCache(uint16_t sz = UINT16_MAX); + ~MgrMapCache(); + bool try_get(std::string_view k, Value* out, bool count_hit = true) noexcept { + return CacheImp::try_get(k, out, count_hit); + } + void insert(std::string_view k, Value v) { CacheImp::insert(k, v); } + bool extract(std::string_view k, Value* out) noexcept { return CacheImp::extract(k, out); } + void erase(std::string_view k) noexcept { Value v{}; (void)CacheImp::extract(k, &v); } + void clear() noexcept { CacheImp::clear(); } +private: + std::vector get_tracked_keys() const noexcept override { return {"mgr_map_cache_enabled"}; } + void handle_conf_change(const ConfigProxy& conf, const std::set& changed) override; +}; + +// ------- Full template specialization for PyObject*. with GIL rules ---------- +template <> +class MgrMapCache : public LFUCache, + public md_config_obs_t { + using CacheImp = LFUCache; +public: + MgrMapCache(uint16_t size = UINT16_MAX); + ~MgrMapCache(); + bool try_get(std::string_view k, PyObject** out, bool count_hit = true) noexcept = delete; + PyObject* get(std::string_view key); + void erase(std::string_view key) noexcept; + void clear() noexcept override; + void insert(std::string_view key, PyObject* value); + void invalidate(std::string_view key) { + erase(key); + } +private: + std::vector get_tracked_keys() const noexcept override { return {"mgr_map_cache_enabled"}; } + void handle_conf_change(const ConfigProxy& conf, const std::set& changed) override; +}; diff --git a/src/mgr/PyFormatter.cc b/src/mgr/PyFormatter.cc index 7c333308ab5..9461cceab0a 100644 --- a/src/mgr/PyFormatter.cc +++ b/src/mgr/PyFormatter.cc @@ -18,6 +18,13 @@ #include "PyFormatter.h" #include +#include "common/debug.h" + +#define dout_context g_ceph_context +#define dout_subsys ceph_subsys_mgr +#undef dout_prefix +#define dout_prefix *_dout << __func__ << " " + #define LARGE_SIZE 1024 @@ -131,16 +138,3 @@ void PyFormatter::finish_pending_streams() pending_streams.clear(); } - -PyObject* PyJSONFormatter::get() -{ - if(json_formatter::stack_size()) { - close_section(); - } - ceph_assert(!json_formatter::stack_size()); - std::ostringstream ss; - flush(ss); - std::string s = ss.str(); - PyObject* obj = PyBytes_FromStringAndSize(s.c_str(), s.size()); - return obj; -} diff --git a/src/mgr/PyFormatter.h b/src/mgr/PyFormatter.h index e5ee0c29600..287fc0f55ec 100644 --- a/src/mgr/PyFormatter.h +++ b/src/mgr/PyFormatter.h @@ -115,7 +115,7 @@ public: ceph_abort(); } - PyObject *get() + virtual PyObject *get() { finish_pending_streams(); @@ -125,13 +125,12 @@ public: void finish_pending_streams(); -private: +protected: PyObject *root; PyObject *cursor; std::stack stack; - void dump_pyobject(std::string_view name, PyObject *p); - +private: class PendingStream { public: PyObject *cursor; @@ -143,22 +142,128 @@ private: }; -class PyJSONFormatter : public JSONFormatter { +class PyFormatterRO : public PyFormatter { public: - PyObject *get(); - PyJSONFormatter (const PyJSONFormatter&) = default; - PyJSONFormatter(bool pretty=false, bool is_array=false) : JSONFormatter(pretty) { - if(is_array) { - open_array_section(""); - } else { - open_object_section(""); + using PyFormatter::PyFormatter; + PyObject* get() override { + finish_pending_streams(); + if (!is_converted_to_readonly) { + convert_to_readonly(); } -} + Py_INCREF(root); + return root; + } + void reset() override { + PyFormatter::reset(); + is_converted_to_readonly = false; + } private: - using json_formatter = JSONFormatter; - template void add_value(std::string_view name, T val); - void add_value(std::string_view name, std::string_view val, bool quoted); + bool is_converted_to_readonly = false; + + /// Convert entire data structure to read-only once + void convert_to_readonly() { + PyObject* readonly_root = make_immutable(root); + + if (readonly_root != root) { + Py_DECREF(root); + root = readonly_root; + cursor = root; + } + is_converted_to_readonly = true; + } + + /// Recursively convert object to immutable equivalent + PyObject* make_immutable(PyObject* obj) { + if (PyList_Check(obj)) { + return convert_list_to_tuple(obj); + } + if (PyDict_Check(obj)) { + return convert_dict_to_proxy(obj); + } + if (PySet_Check(obj)) { + return convert_set_to_frozenset(obj); + } + if (PyTuple_Check(obj)) { + return convert_tuple_contents(obj); + } + // Already immutable + Py_INCREF(obj); + return obj; + } + + PyObject* convert_list_to_tuple(PyObject* list) { + Py_ssize_t size = PyList_Size(list); + PyObject* tuple = PyTuple_New(size); + + for (Py_ssize_t i = 0; i < size; i++) { + PyObject* item = PyList_GetItem(list, i); + PyObject* immutable_item = make_immutable(item); + if (!immutable_item) { + Py_DECREF(tuple); + Py_INCREF(list); + return list; + } + PyTuple_SET_ITEM(tuple, i, immutable_item); + } + return tuple; + } + + PyObject* convert_dict_to_proxy(PyObject* dict) { + PyObject* immutable_dict = PyDict_New(); + PyObject* key, *value; + Py_ssize_t pos = 0; + + while (PyDict_Next(dict, &pos, &key, &value)) { + PyObject* immutable_value = make_immutable(value); + if (!immutable_value) { + Py_DECREF(immutable_dict); + Py_INCREF(dict); + return dict; + } + if (PyDict_SetItem(immutable_dict, key, immutable_value) < 0) { + Py_DECREF(immutable_value); + Py_DECREF(immutable_dict); + Py_INCREF(dict); + return dict; + } + Py_DECREF(immutable_value); + } + return immutable_dict; + } + + PyObject* convert_set_to_frozenset(PyObject* set) { + PyObject* frozenset = PyFrozenSet_New(set); + if (frozenset) { + return frozenset; + } + + // Fallback + Py_INCREF(set); + return set; + } + + PyObject* convert_tuple_contents(PyObject* tuple) { + Py_ssize_t size = PyTuple_Size(tuple); + PyObject* new_tuple = PyTuple_New(size); + if (!new_tuple) { + Py_INCREF(tuple); + return tuple; + } + + for (Py_ssize_t i = 0; i < size; i++) { + PyObject* item = PyTuple_GetItem(tuple, i); + PyObject* immutable_item = make_immutable(item); + + if (!immutable_item) { + Py_DECREF(new_tuple); + Py_INCREF(tuple); + return tuple; + } + PyTuple_SET_ITEM(new_tuple, i, immutable_item); + } + return new_tuple; + } }; #endif diff --git a/src/mgr/TTLCache.cc b/src/mgr/TTLCache.cc deleted file mode 100644 index 45260bf4a77..00000000000 --- a/src/mgr/TTLCache.cc +++ /dev/null @@ -1,127 +0,0 @@ -#include "TTLCache.h" - -#include -#include -#include - -#include "PyUtil.h" - -template -void TTLCacheBase::insert(Key key, Value value) { - auto now = std::chrono::steady_clock::now(); - if (!ttl) return; - int16_t random_ttl_offset = - ttl * ttl_spread_ratio * (2l * rand() / float(RAND_MAX) - 1); - // in order not to have spikes of misses we increase or decrease by 25% of - // the ttl - int16_t spreaded_ttl = ttl + random_ttl_offset; - auto expiration_date = now + std::chrono::seconds(spreaded_ttl); - cache::insert(key, {value, expiration_date}); -} - -template Value TTLCacheBase::get(Key key) { - if (!exists(key)) { - throw_key_not_found(key); - } - if (expired(key)) { - erase(key); - throw_key_not_found(key); - } - Value value = {get_value(key)}; - return value; -} - -template PyObject* TTLCache::get(Key key) { - if (!this->exists(key)) { - this->throw_key_not_found(key); - } - if (this->expired(key)) { - this->erase(key); - this->throw_key_not_found(key); - } - PyObject* cached_value = this->get_value(key); - Py_INCREF(cached_value); - return cached_value; -} - -template -void TTLCacheBase::erase(Key key) { - cache::erase(key); -} - -template void TTLCache::insert(Key key, PyObject* value) { - if (!this->get_ttl()) return; - Py_INCREF(value); - this->TTLCacheBase::insert(key, value); -} - -template void TTLCache::clear() { - for (auto& entry : this->content) { - PyObject* v = nullptr; - v = std::get<0>(entry.second); - if (v != nullptr) { - Py_DECREF(v); - } - } - this->ttl_base::clear(); -} - -template void TTLCache::erase(Key key) { - auto stored_value = this->cache::get(key, false); - PyObject* cached_value = std::get<0>(stored_value); - if (cached_value != nullptr) { - Py_DECREF(cached_value); - } - ttl_base::erase(key); -} - -template -bool TTLCacheBase::expired(Key key) { - ttl_time_point expiration_date = get_value_time_point(key); - auto now = std::chrono::steady_clock::now(); - if (now >= expiration_date) { - return true; - } else { - return false; - } -} - -template void TTLCacheBase::clear() { - cache::clear(); -} - -template -Value TTLCacheBase::get_value(Key key, bool count_hit) { - value_type stored_value = cache::get(key, count_hit); - Value value = std::get<0>(stored_value); - return value; -} - -template -ttl_time_point TTLCacheBase::get_value_time_point(Key key) { - value_type stored_value = cache::get(key, false); - ttl_time_point tp = std::get<1>(stored_value); - return tp; -} - -template -void TTLCacheBase::set_ttl(uint16_t ttl) { - this->ttl = ttl; -} - -template -bool TTLCacheBase::exists(Key key) { - return cache::exists(key); -} - -template -void TTLCacheBase::throw_key_not_found(Key key) { - cache::throw_key_not_found(key); -} - -template -PyObject* TTLCache::get_value(Key key, bool count_hit) { - auto stored_value = cache::get(key, count_hit); - PyObject* value = std::get<0>(stored_value); - return value; -} diff --git a/src/mgr/TTLCache.h b/src/mgr/TTLCache.h deleted file mode 100644 index 35e96c365c7..00000000000 --- a/src/mgr/TTLCache.h +++ /dev/null @@ -1,135 +0,0 @@ -#pragma once - -#include -#include -#include -#include -#include -#include -#include -#include - -#include "PyUtil.h" - -template class Cache { - private: - std::atomic hits, misses; - - protected: - unsigned int capacity; - Cache(unsigned int size = UINT16_MAX) : hits{0}, misses{0}, capacity{size} {}; - std::map content; - std::vector allowed_keys = {"osd_map", "pg_dump", "pg_stats"}; - - void mark_miss() { - misses++; - } - - void mark_hit() { - hits++; - } - - unsigned int get_misses() { return misses; } - unsigned int get_hits() { return hits; } - void throw_key_not_found(Key key) { - std::stringstream ss; - ss << "Key " << key << " couldn't be found\n"; - throw std::out_of_range(ss.str()); - } - - public: - void insert(Key key, Value value) { - mark_miss(); - if (content.size() < capacity) { - content.insert({key, value}); - } - } - Value get(Key key, bool count_hit = true) { - if (count_hit) { - mark_hit(); - } - return content[key]; - } - void erase(Key key) { content.erase(content.find(key)); } - void clear() { content.clear(); } - bool exists(Key key) { return content.find(key) != content.end(); } - std::pair get_hit_miss_ratio() { - return std::make_pair(hits.load(), misses.load()); - } - bool is_cacheable(Key key) { - for (auto k : allowed_keys) { - if (key == k) return true; - } - return false; - } - int size() { return content.size(); } - - //return the keys vector - std::vector keys() { - std::vector keys; - for (auto const& [k, v] : content) { - keys.push_back(k); - } - return keys; - } - ~Cache(){}; -}; - -using ttl_time_point = std::chrono::time_point; -template -class TTLCacheBase : public Cache> { - private: - uint16_t ttl; - float ttl_spread_ratio; - using value_type = std::pair; - using cache = Cache; - - protected: - Value get_value(Key key, bool count_hit = true); - ttl_time_point get_value_time_point(Key key); - bool exists(Key key); - bool expired(Key key); - void finish_get(Key key); - void finish_erase(Key key); - void throw_key_not_found(Key key); - - public: - TTLCacheBase(uint16_t ttl_ = 0, uint16_t size = UINT16_MAX, - float spread = 0.25) - : Cache(size), ttl{ttl_}, ttl_spread_ratio{spread} {} - ~TTLCacheBase(){}; - void insert(Key key, Value value); - Value get(Key key); - void erase(Key key); - void clear(); - uint16_t get_ttl() { return ttl; }; - void set_ttl(uint16_t ttl); -}; - -template -class TTLCache : public TTLCacheBase { - public: - TTLCache(uint16_t ttl_ = 0, uint16_t size = UINT16_MAX, float spread = 0.25) - : TTLCacheBase(ttl_, size, spread) {} - ~TTLCache(){}; -}; - -template -class TTLCache : public TTLCacheBase { - public: - TTLCache(uint16_t ttl_ = 0, uint16_t size = UINT16_MAX, float spread = 0.25) - : TTLCacheBase(ttl_, size, spread) {} - ~TTLCache(){ clear(); }; - void insert(Key key, PyObject* value); - PyObject* get(Key key); - void erase(Key key); - void clear(); - PyObject* get_value(Key key, bool count_hit = true); - private: - using ttl_base = TTLCacheBase; - using value_type = std::pair; - using cache = Cache; -}; - -#include "TTLCache.cc" - diff --git a/src/pybind/mgr/ceph_module.pyi b/src/pybind/mgr/ceph_module.pyi index 3f37576c4b6..920f80f1eb1 100644 --- a/src/pybind/mgr/ceph_module.pyi +++ b/src/pybind/mgr/ceph_module.pyi @@ -42,7 +42,8 @@ class BasePyCRUSH(object): class BaseMgrStandbyModule(object): def __init__(self, capsule): pass - def _ceph_get(self, data_name: str) -> Dict[str, Any]: ... + def _ceph_get(self, data_name: str, mutable: bool = False) -> Dict[str, Any]: ... + def _ceph_erase(self, data_name: str) -> None: ... def _ceph_get_mgr_id(self):... def _ceph_get_module_option(self, key, prefix=None):... def _ceph_get_option(self, key):... @@ -69,7 +70,8 @@ class BaseMgrModule(object): def _ceph_lookup_release_name(self, release: int) -> str: ... def _ceph_cluster_log(self, channel: str, priority: int, message: str) -> None: ... def _ceph_get_context(self) -> object: ... - def _ceph_get(self, data_name: str) -> Any: ... + def _ceph_get(self, data_name: str, mutable: bool = False) -> Any: ... + def _ceph_erase(self, data_name: str) -> None: ... def _ceph_notify_all(self, what: str, tag: str) -> None: ... def _ceph_get_server(self, hostname: Optional[str]) -> Union[ServerInfoT, List[ServerInfoT]]: ... diff --git a/src/pybind/mgr/cli_api/module.py b/src/pybind/mgr/cli_api/module.py index 08dac550a30..c3ff9d924d4 100755 --- a/src/pybind/mgr/cli_api/module.py +++ b/src/pybind/mgr/cli_api/module.py @@ -119,6 +119,19 @@ class CLI(MgrModule, metaclass=MgrAPIReflector): } return HandleCommandResult(stdout=pretty_json(stats)) + @CLICommand('mgr cli cache flush') + def erase_cache(self, what: str) -> HandleCommandResult: + """ + Erase a cached map by its name. + """ + r = self.erase(what) + if r is False: + return HandleCommandResult( + errno.EINVAL, + stderr=f"no cached map named {what}" + ) + return HandleCommandResult(stdout=f"Cache map {what} erased successfully") + class BenchmarkException(Exception): pass diff --git a/src/pybind/mgr/mgr_module.py b/src/pybind/mgr/mgr_module.py index 3a27154fc17..192eefbf96e 100644 --- a/src/pybind/mgr/mgr_module.py +++ b/src/pybind/mgr/mgr_module.py @@ -1172,6 +1172,9 @@ class MgrModule(ceph_module.BaseMgrModule, MgrModuleLoggingMixin): def have_enough_osds(self) -> bool: # wait until we have enough OSDs to allow the pool to be healthy ready = 0 + self.log.debug("checking for enough OSDs") + self.log.debug(f'osds returned from osd_map: {self.get("osd_map")["osds"]}') + self.log.debug(f'osd_map: {self.get("osd_map")}') for osd in self.get("osd_map")["osds"]: if osd["up"] and osd["in"]: ready += 1 @@ -1491,7 +1494,7 @@ class MgrModule(ceph_module.BaseMgrModule, MgrModuleLoggingMixin): self._rados = None @API.expose - def get(self, data_name: str) -> Any: + def get(self, data_name: str, mutable: bool = False) -> Any: """ Called by the plugin to fetch named cluster-wide objects from ceph-mgr. @@ -1502,16 +1505,29 @@ class MgrModule(ceph_module.BaseMgrModule, MgrModuleLoggingMixin): pool_stats, pg_ready, osd_ping_times, mgr_map, mgr_ips, modified_config_options, service_map, mds_metadata, have_local_config_map, osd_pool_stats, pg_status. + :param bool mutable: If True, returns a mutable copy of the data that can + be modified safely. If False (default), returns read-only cached + data (in case cached enabled) for better performance and cache protection. Note: All these structures have their own JSON representations: experiment or look at the C++ ``dump()`` methods to learn about them. """ - obj = self._ceph_get(data_name) - if isinstance(obj, bytes): - obj = json.loads(obj) + return self._ceph_get(data_name, mutable) - return obj + @API.expose + def erase(self, data_name: str) -> None: + """ + Called by the plugin to erase cache entries for named + cluster-wide objects from ceph-mgr. + :param str data_name: Valid things to erase are osd_map, mon_map, + fs_map, pg_summary, io_rate, pg_dump, df, osd_stats, + health, mon_status, devices, pg_stats, pool_stats, + pg_ready, osd_ping_times, mgr_map, mgr_ips, + modified_config_options, service_map, mds_metadata, + have_local_config_map, osd_pool_stats, pg_status. + """ + return self._ceph_erase(data_name) def _stattype_to_str(self, stattype: int) -> str: diff --git a/src/pybind/mgr/tests/__init__.py b/src/pybind/mgr/tests/__init__.py index 8ae6ea54b46..5539ccd2de8 100644 --- a/src/pybind/mgr/tests/__init__.py +++ b/src/pybind/mgr/tests/__init__.py @@ -97,7 +97,7 @@ if 'UNITTEST' in os.environ: }) return val - def _ceph_get(self, data_name): + def _ceph_get(self, data_name, mutable=False): return self.mock_store_get('_ceph_get', data_name, mock.MagicMock()) def _ceph_send_command(self, res, svc_type, svc_id, command, tag, inbuf, *, one_shot=False): diff --git a/src/test/mgr/CMakeLists.txt b/src/test/mgr/CMakeLists.txt index d28346aa8ef..543bdee869a 100644 --- a/src/test/mgr/CMakeLists.txt +++ b/src/test/mgr/CMakeLists.txt @@ -73,10 +73,12 @@ target_link_libraries(unittest_mgr_servicemap ceph-common ) -# unittest_mgr_ttlcache -add_executable(unittest_mgr_ttlcache test_ttlcache.cc) -add_ceph_unittest(unittest_mgr_ttlcache) -target_link_libraries(unittest_mgr_ttlcache ceph-common +# unittest_mgr_map_cache +add_executable(unittest_mgr_map_cache + test_mgrmapcache.cc + ${CMAKE_SOURCE_DIR}/src/mgr/MgrMapCache.cc) +add_ceph_unittest(unittest_mgr_map_cache) +target_link_libraries(unittest_mgr_map_cache ceph-common Python3::Python ${CMAKE_DL_LIBS} ${GSSAPI_LIBRARIES}) #scripts diff --git a/src/test/mgr/test_mgrmapcache.cc b/src/test/mgr/test_mgrmapcache.cc new file mode 100644 index 00000000000..c9ff871849d --- /dev/null +++ b/src/test/mgr/test_mgrmapcache.cc @@ -0,0 +1,221 @@ +// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:nil -*- +// vim: ts=8 sw=2 sts=2 expandtab + +/* + * Ceph - scalable distributed file system + * + * Copyright (C) 2026 IBM + * + * This is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License version 2.1, as published by the Free Software + * Foundation. See file COPYING. + * + */ + +#include +#include +#include + +#include "common/perf_counters.h" + +#include "mgr/mgr_perf_counters.h" +#include "mgr/MgrMapCache.h" + +#include "gtest/gtest.h" + +PerfCounters *perfcounter = nullptr; + +using namespace std; + +TEST(LFUCache, Get) { + LFUCache c{{"foo"}, 100}; + c.insert("foo", 1); + int foo = c.get("foo"); + ASSERT_EQ(foo, 1); +} + +TEST(LFUCache, Erase) { + LFUCache c{{"foo"}, 100}; + c.insert("foo", 1); + int foo; + if (!c.try_get("foo", &foo)) { + FAIL(); + } + ASSERT_EQ(foo, 1); + c.erase("foo"); + try{ + foo = c.get("foo"); + FAIL(); + } catch (std::out_of_range& e) { + SUCCEED(); + } +} + +TEST(LFUCache, Clear) { + LFUCache c{{"osd_map", "pg_dump", "pg_stats"}, 100}; + c.insert("osd_map", 1); + c.insert("pg_dump", 2); + c.insert("pg_stats", 3); + ASSERT_EQ(c.size(), 3); + c.clear(); + ASSERT_EQ(c.size(), 0); + try{ + c.get("osd_map"); + FAIL(); + } catch (std::out_of_range& e) { + SUCCEED(); + } +} + +TEST(LFUCache, NotEnabled) { + LFUCache c{{"foo"}, 100}; + c.insert("foo", 1); + int foo = c.get("foo"); + ASSERT_EQ(foo, 1); + c.set_enabled(false); + try{ + foo = c.get("foo"); + FAIL(); + } catch (std::out_of_range& e) { + SUCCEED(); + } +} + +TEST(LFUCache, SizeLimit) { + LFUCache c{{"foo", "osd_map", "pg_dump", "pg_stats", "mon_status"}, 4, true}; + c.insert("foo", 1); + c.insert("osd_map", 2); + c.insert("pg_dump", 3); + c.insert("pg_stats", 4); + c.get("foo"); // foo hits 1 + c.get("pg_dump"); // pg_dump hits 1 + for (int i = 0; i < 100; ++i) { + c.get("pg_stats"); // pg_stats hits 100 + } + c.insert("mon_status", 5); // This should evict "osd_map" since it has the least hits + ASSERT_EQ(c.size(), 4); + int foo = c.get("foo"); + int pg_dump = c.get("pg_dump"); + int pg_stats = c.get("pg_stats"); + int mon_status = c.get("mon_status"); + try { + c.get("osd_map"); // This should throw an exception since it was evicted + FAIL(); // If nothing throws, this will fail + } catch (std::out_of_range& e) { + ASSERT_EQ(foo, 1); + ASSERT_EQ(pg_dump, 3); + ASSERT_EQ(pg_stats, 4); + ASSERT_EQ(mon_status, 5); + SUCCEED(); + } +} + +TEST(LFUCache, HitRatio) { + LFUCache c{{"osd_map", "pg_dump", "pg_stats"}, 100, true}; + c.insert("osd_map", 1); + c.insert("pg_dump", 2); + c.insert("pg_stats", 3); + c.get("osd_map"); // hits 1 + c.get("osd_map"); // hits 2 + c.get("osd_map"); // hits 3 + c.get("pg_dump"); // hits 4 + std::pair hit_miss_ratio = {c.get_hits(), c.get_misses()}; + ASSERT_EQ(std::get<1>(hit_miss_ratio), 3); + ASSERT_EQ(std::get<0>(hit_miss_ratio), 4); +} + +TEST(LFUCache, InsertResNewKey) { + LFUCache c{{"foo"}, 100}; + auto res = c.insert("foo", 42); + ASSERT_TRUE(res.inserted); + ASSERT_FALSE(res.replaced.has_value()); + ASSERT_FALSE(res.evicted.has_value()); +} + +TEST(LFUCache, InsertResReplaced) { + LFUCache c{{"foo"}, 100}; + c.insert("foo", 1); + auto res = c.insert("foo", 2); + ASSERT_TRUE(res.inserted); + ASSERT_TRUE(res.replaced.has_value()); + ASSERT_EQ(res.replaced.value(), 1); + ASSERT_FALSE(res.evicted.has_value()); + // new value is live in cache + int v; + ASSERT_TRUE(c.try_get("foo", &v)); + ASSERT_EQ(v, 2); +} + +TEST(LFUCache, InsertResEvicted) { + LFUCache c{{"foo", "osd_map", "pg_dump", "pg_stats"}, 3}; + c.insert("osd_map", 10); + c.insert("pg_dump", 20); + c.insert("pg_stats", 30); + // Give pg_dump and pg_stats hits so osd_map has the clear minimum (0 hits) + c.get("pg_dump"); + c.get("pg_stats"); + auto res = c.insert("foo", 99); + ASSERT_TRUE(res.inserted); + ASSERT_FALSE(res.replaced.has_value()); + ASSERT_TRUE(res.evicted.has_value()); + ASSERT_EQ(res.evicted.value(), 10); +} + +TEST(LFUCache, InsertResRejectedWhenDisabled) { + LFUCache c{{"foo"}, 100}; + c.set_enabled(false); + auto res = c.insert("foo", 1); + ASSERT_FALSE(res.inserted); + ASSERT_FALSE(res.replaced.has_value()); + ASSERT_FALSE(res.evicted.has_value()); + ASSERT_EQ(c.size(), 0); +} + +TEST(LFUCache, InsertResRejectedForUnknownKey) { + LFUCache c{{"foo"}, 100}; + auto res = c.insert("not_an_allowed_key", 1); + ASSERT_FALSE(res.inserted); + ASSERT_EQ(c.size(), 0); +} + +TEST(LFUCache, InsertResRejectedAfterDisableDuringWait) { + // Simulate the post-lock enabled re-check: disable the cache, then call + // insert directly on LFUCache (bypassing the outer can_write_cache guard) + // by re-enabling just long enough to pass can_write_cache, then disabling. + // We approximate this by disabling between two sequential inserts and + // verifying the second is rejected. + LFUCache c{{"foo"}, 100}; + auto res1 = c.insert("foo", 1); + ASSERT_TRUE(res1.inserted); + c.set_enabled(false); + auto res2 = c.insert("foo", 2); + ASSERT_FALSE(res2.inserted); + // cache was cleared by set_enabled(false) + ASSERT_EQ(c.size(), 0); +} + +TEST(LFUCache, ConcurrentReads) { + LFUCache c{{"foo"}, 100, true}; + c.insert("foo", 42); + + std::atomic success_count{0}; + std::vector threads; + + // Launch 10 threads doing 1000 reads each + for (int i = 0; i < 10; ++i) { + threads.emplace_back([&c, &success_count]() { + for (int j = 0; j < 1000; ++j) { + int val; + if (c.try_get("foo", &val)) { + success_count++; + } + } + }); + } + + for (auto& t : threads) t.join(); + + ASSERT_EQ(success_count, 10000); + ASSERT_EQ(c.get_hits(), 10000); +} diff --git a/src/test/mgr/test_pyformatter.cc b/src/test/mgr/test_pyformatter.cc index 42131a15425..b092c5dca6a 100644 --- a/src/test/mgr/test_pyformatter.cc +++ b/src/test/mgr/test_pyformatter.cc @@ -373,6 +373,229 @@ TEST(PyFormatter, EmptySectionName) /* End Negative Tests */ +/* + * Tests for PyFormatterRO (read-only-on-the-fly builder): + * - lists -> tuples, sets -> frozensets + * - dicts remain dicts (JSON-friendly) + * - works with nested structures + * - can be json.dumps()-ed safely (for structures without sets) + */ + +using namespace ceph; + +class PyFormatterROTestHelper : public PyFormatterRO { +public: + using PyFormatterRO::PyFormatterRO; + + // Expose the protected method for testing + void test_dump_pyobject(std::string_view name, PyObject *p) { + dump_pyobject(name, p); + } +}; + +struct PyRuntime : public ::testing::Test { + static void SetUpTestSuite() { + if (!Py_IsInitialized()) { + Py_Initialize(); + } + } + static void TearDownTestSuite() { + if (Py_IsInitialized()) { + Py_Finalize(); + } + } + void SetUp() override { + gstate = PyGILState_Ensure(); + ASSERT_TRUE(Py_IsInitialized()); + } + void TearDown() override { + PyGILState_Release(gstate); + } +private: + PyGILState_STATE gstate{}; +}; + +TEST_F(PyRuntime, PyFormatterRO_FreezesContainers) { + PyFormatterROTestHelper ro; // default root is a dict + + // Build values + PyObject* src_list = PyList_New(0); + PyList_Append(src_list, PyLong_FromLong(1)); + PyList_Append(src_list, PyLong_FromLong(2)); + + PyObject* src_set = PySet_New(nullptr); + PySet_Add(src_set, PyLong_FromLong(7)); + PySet_Add(src_set, PyLong_FromLong(8)); + + PyObject* src_dict = PyDict_New(); + PyDict_SetItemString(src_dict, "k", PyLong_FromLong(42)); + + // dump_pyobject steals the reference, do not Py_DECREF afterward + ro.test_dump_pyobject("alist", src_list); + ro.test_dump_pyobject("aset", src_set); + ro.test_dump_pyobject("adict", src_dict); + + PyObject* root = ro.get(); // new ref + ASSERT_TRUE(PyDict_Check(root)); + + // alist -> tuple + { + PyObject* alist = PyDict_GetItemString(root, "alist"); // borrowed + ASSERT_NE(alist, nullptr); + EXPECT_TRUE(PyTuple_Check(alist)); + EXPECT_EQ(PyTuple_Size(alist), 2); + } + + // aset -> frozenset + { + PyObject* aset = PyDict_GetItemString(root, "aset"); // borrowed + ASSERT_NE(aset, nullptr); + EXPECT_TRUE(PyFrozenSet_Check(aset)); + EXPECT_EQ(PySet_Size(aset), 2); + } + + // adict stays dict + { + PyObject* adict = PyDict_GetItemString(root, "adict"); // borrowed + ASSERT_NE(adict, nullptr); + EXPECT_TRUE(PyDict_Check(adict)); + EXPECT_EQ(PyDict_Size(adict), 1); + } + + Py_DECREF(root); +} + +TEST_F(PyRuntime, PyFormatterRO_NestedFreezingAndJsonDump) { + PyFormatterROTestHelper ro; + + // Build: outer = [ {"k1": 1, "k2": 2}, [3, 4] ] + PyObject* first_dict = PyDict_New(); + PyDict_SetItemString(first_dict, "k1", PyLong_FromLong(1)); + PyDict_SetItemString(first_dict, "k2", PyLong_FromLong(2)); + + PyObject* second_list = PyList_New(0); + PyList_Append(second_list, PyLong_FromLong(3)); + PyList_Append(second_list, PyLong_FromLong(4)); + + PyObject* outer_list = PyList_New(0); + PyList_Append(outer_list, first_dict); + PyList_Append(outer_list, second_list); + + // Temps no longer needed + Py_DECREF(first_dict); + Py_DECREF(second_list); + + // dump_pyobject steals the reference, do not Py_DECREF afterward + ro.test_dump_pyobject("outer", outer_list); + + PyObject* root = ro.get(); // new ref + ASSERT_TRUE(PyDict_Check(root)); + + PyObject* outer = PyDict_GetItemString(root, "outer"); + ASSERT_NE(outer, nullptr); + ASSERT_TRUE(PyTuple_Check(outer)); + ASSERT_EQ(PyTuple_Size(outer), 2); + + PyObject* outer0 = PyTuple_GetItem(outer, 0); + PyObject* outer1 = PyTuple_GetItem(outer, 1); + + // Element 0 remains a dict + ASSERT_TRUE(PyDict_Check(outer0)); + EXPECT_EQ(PyDict_Size(outer0), 2); + + // Element 1 is the forzened list -> tuple + ASSERT_TRUE(PyTuple_Check(outer1)); + ASSERT_EQ(PyTuple_Size(outer1), 2); + EXPECT_TRUE(PyLong_Check(PyTuple_GetItem(outer1, 0))); + EXPECT_TRUE(PyLong_Check(PyTuple_GetItem(outer1, 1))); + + // json.dumps should work (dict + tuple are serializable; no sets here) + PyObject* json_mod = PyImport_ImportModule("json"); + ASSERT_NE(json_mod, nullptr); + PyObject* dumps = PyObject_GetAttrString(json_mod, "dumps"); + ASSERT_NE(dumps, nullptr); + + PyObject* args = PyTuple_Pack(1, root); + PyObject* s = PyObject_CallObject(dumps, args); + Py_DECREF(args); + Py_DECREF(dumps); + Py_DECREF(json_mod); + + ASSERT_NE(s, nullptr); + EXPECT_TRUE(PyUnicode_Check(s)); + Py_DECREF(s); + + Py_DECREF(root); +} + +TEST_F(PyRuntime, PyFormatterRO_ReadonlyBehavior) { + PyFormatterROTestHelper ro; + + // Insert three things + PyObject* src_list = PyList_New(0); + PyList_Append(src_list, PyLong_FromLong(1)); + PyList_Append(src_list, PyLong_FromLong(2)); + + PyObject* src_set = PySet_New(nullptr); + PySet_Add(src_set, PyLong_FromLong(7)); + PySet_Add(src_set, PyLong_FromLong(8)); + + PyObject* src_dict = PyDict_New(); + PyDict_SetItemString(src_dict, "k", PyLong_FromLong(42)); + + // dump_pyobject steals the reference, do not Py_DECREF afterward + ro.test_dump_pyobject("alist", src_list); + ro.test_dump_pyobject("aset", src_set); + ro.test_dump_pyobject("adict", src_dict); + + PyObject* root = ro.get(); // new ref + ASSERT_TRUE(PyDict_Check(root)); + + // alist -> tuple; mutation must fail + { + PyObject* alist = PyDict_GetItemString(root, "alist"); // borrowed + ASSERT_NE(alist, nullptr); + ASSERT_TRUE(PyTuple_Check(alist)); + + PyObject* ninety_nine = PyLong_FromLong(99); + int rc = PySequence_SetItem(alist, 0, ninety_nine); + Py_DECREF(ninety_nine); + //print the rc and error + EXPECT_EQ(rc, -1); + EXPECT_TRUE(PyErr_Occurred()); + + PyErr_Clear(); + } + + // aset -> frozenset; adding must fail (no 'add' method) + { + PyObject* aset = PyDict_GetItemString(root, "aset"); // borrowed + ASSERT_NE(aset, nullptr); + ASSERT_TRUE(PyFrozenSet_Check(aset)); + + PyObject* nine = PyLong_FromLong(9); + PyObject* res = PyObject_CallMethod(aset, "add", "O", nine); + Py_DECREF(nine); + EXPECT_EQ(res, nullptr); // no method on frozenset + EXPECT_TRUE(PyErr_Occurred()); // AttributeError + PyErr_Clear(); + } + + // adict stays dict (mutable) — allowed + { + PyObject* adict = PyDict_GetItemString(root, "adict"); // borrowed + ASSERT_NE(adict, nullptr); + ASSERT_TRUE(PyDict_Check(adict)); + + int rc = PyDict_SetItemString(adict, "k2", PyLong_FromLong(5)); + EXPECT_EQ(rc, 0); + EXPECT_EQ(PyDict_Size(adict), 2); + } + + Py_DECREF(root); +} + + int main(int argc, char* argv[]) { @@ -380,4 +603,4 @@ main(int argc, char* argv[]) ::testing::AddGlobalTestEnvironment(new PythonEnv); return RUN_ALL_TESTS(); -} +} \ No newline at end of file diff --git a/src/test/mgr/test_ttlcache.cc b/src/test/mgr/test_ttlcache.cc deleted file mode 100644 index 2719aa49a38..00000000000 --- a/src/test/mgr/test_ttlcache.cc +++ /dev/null @@ -1,76 +0,0 @@ -#include - -#include "gtest/gtest.h" -#include "mgr/TTLCache.h" - -using namespace std; - -TEST(TTLCache, Get) -{ - TTLCache c{100}; - c.insert("foo", 1); - int foo = c.get("foo"); - ASSERT_EQ(foo, 1); -} - -TEST(TTLCache, Erase) -{ - TTLCache c{100}; - c.insert("foo", 1); - int foo = c.get("foo"); - ASSERT_EQ(foo, 1); - c.erase("foo"); - try { - foo = c.get("foo"); - FAIL(); - } catch (std::out_of_range& e) { - SUCCEED(); - } -} - -TEST(TTLCache, Clear) -{ - TTLCache c{100}; - c.insert("foo", 1); - c.insert("foo2", 2); - c.clear(); - ASSERT_FALSE(c.size()); -} - -TEST(TTLCache, NoTTL) -{ - TTLCache c{100}; - c.insert("foo", 1); - int foo = c.get("foo"); - ASSERT_EQ(foo, 1); - c.set_ttl(0); - c.insert("foo2", 2); - try { - foo = c.get("foo2"); - FAIL(); - } catch (std::out_of_range& e) { - SUCCEED(); - } -} - -TEST(TTLCache, SizeLimit) -{ - TTLCache c{100, 2}; - c.insert("foo", 1); - c.insert("foo2", 2); - c.insert("foo3", 3); - ASSERT_EQ(c.size(), 2); -} - -TEST(TTLCache, HitRatio) -{ - TTLCache c{100}; - c.insert("foo", 1); - c.insert("foo2", 2); - c.insert("foo3", 3); - c.get("foo2"); - c.get("foo3"); - std::pair hit_miss_ratio = c.get_hit_miss_ratio(); - ASSERT_EQ(std::get<1>(hit_miss_ratio), 3); - ASSERT_EQ(std::get<0>(hit_miss_ratio), 2); -}