]> git-server-git.apps.pok.os.sepia.ceph.com Git - rocksdb.git/commit
Fix a leak of open Blob files (#13106)
authorPeter Dillinger <peterd@meta.com>
Thu, 31 Oct 2024 22:29:30 +0000 (15:29 -0700)
committerPeter Dillinger <peterd@meta.com>
Fri, 1 Nov 2024 15:34:20 +0000 (08:34 -0700)
commitab03f7fb531d70c95da8b9ad7e73be8f3f183edd
tree77b1df99c4972986c1c5abaf05bef9badddab87d
parentfb6fa230c779facde440f3787c0c34bee1582662
Fix a leak of open Blob files (#13106)

Summary:
An earlier change (https://github.com/facebook/rocksdb/commit/b34cef57b798520791312f2f40681c4d12d5d33c) removed apparently unused functionality where an obsolete blob file number is passed for removal from TableCache, which manages SST files. This was actually relying on broken/fragile abstractions wherein TableCache and BlobFileCache share the same Cache and using the TableCache interface to manipulate blob file caching. No unit test was actually checking for removal of obsolete blob files from the cache (which is somewhat tricky to check and a second order correctness requirement).

Here we fix the leak and add a DEBUG+ASAN-only check in DB::Close() that no obsolete files are lingering in the table/blob file cache.

Fixes https://github.com/facebook/rocksdb/issues/13066

Important follow-up (FIXME): The added check discovered some apparent cases of leaked (into table_cache) SST file readers that would stick around until DB::Close(). Need to enable that check, diagnose, and fix.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/13106

Test Plan:
added a check that is called during DB::Close in ASAN builds (to minimize paying the cost in all unit tests). Without the fix, the check failed in at least these tests:

```
db_blob_basic_test DBBlobBasicTest.DynamicallyWarmCacheDuringFlush
db_blob_compaction_test DBBlobCompactionTest.CompactionReadaheadMerge
db_blob_compaction_test DBBlobCompactionTest.MergeBlobWithBase
db_blob_compaction_test DBBlobCompactionTest.CompactionDoNotFillCache
db_blob_compaction_test DBBlobCompactionTest.SkipUntilFilter
db_blob_compaction_test DBBlobCompactionTest.CompactionFilter
db_blob_compaction_test DBBlobCompactionTest.CompactionReadaheadFilter
db_blob_compaction_test DBBlobCompactionTest.CompactionReadaheadGarbageCollection
```

Reviewed By: ltamasi

Differential Revision: D65296123

Pulled By: pdillinger

fbshipit-source-id: 2276d76482beb2c75c9010bc1bec070bb23a24c0
db/blob/blob_file_cache.cc
db/blob/blob_file_cache.h
db/column_family.h
db/db_impl/db_impl.cc
db/db_impl/db_impl.h
db/db_impl/db_impl_debug.cc
db/table_cache.cc
db/version_builder.cc
db/version_set.h
unreleased_history/bug_fixes/blob_file_leak.md [new file with mode: 0644]