]> git-server-git.apps.pok.os.sepia.ceph.com Git - rocksdb.git/commit
[RocksDB] Reduce memory footprint of the blockbased table hash index.
authorHaobo Xu <haobo@fb.com>
Fri, 13 Jun 2014 02:03:22 +0000 (19:03 -0700)
committerHaobo Xu <haobo@fb.com>
Thu, 19 Jun 2014 01:16:07 +0000 (18:16 -0700)
commit0f0076ed5abd1540dd0b7af5d51cde529c178d52
treee03ee6113301d821da89817309dc8b59a0e8fcca
parent3525aac9e5714be6635361dee5e320854ed814fc
[RocksDB] Reduce memory footprint of the blockbased table hash index.

Summary:
Currently, the in-memory hash index of blockbased table uses a precise hash map to track the prefix to block range mapping. In some use cases, especially when prefix itself is big, the memory overhead becomes a problem. This diff introduces a fixed hash bucket array that does not store the prefix and allows prefix collision, which is similar to the plaintable hash index, in order to reduce the memory consumption.
Just a quick draft, still testing and refining.

Test Plan: unit test and shadow testing

Reviewers: dhruba, kailiu, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19047
include/rocksdb/table.h
table/block.cc
table/block.h
table/block_based_table_builder.cc
table/block_based_table_reader.cc
table/block_prefix_index.cc [new file with mode: 0644]
table/block_prefix_index.h [new file with mode: 0644]
table/table_test.cc