Sage Weil [Wed, 2 Dec 2015 21:38:46 +0000 (16:38 -0500)]
db/log_reader: treat bad record length or checksum as kEof
If we are in kTolerateCorruptedTailRecords, treat these errors as the
end of the log. This is particularly important for recycled logs, where
we will regularly see corrupted headers (bad length or checksum) when
replaying a log. If we are aligned with a block boundary or get lucky,
we will land on an old header and see the log nubmer mismatch, but more
commonly we will land midway through some previously and effectively
see noise.
Sage Weil [Wed, 2 Dec 2015 13:53:26 +0000 (08:53 -0500)]
env: add EnvMirror
This is an Env implementation that mirrors all storage-related methods on
two different backend Env's and verifies that they return the same
results (return status and read results). This is useful for implementing
a new Env and verifying its correctness.
sdong [Tue, 1 Dec 2015 06:16:49 +0000 (22:16 -0800)]
Fix DBCompactionTestWithParam.CompactionTrigger in non-jemalloc build.
Summary: DBCompactionTestWithParam.CompactionTrigger fails in non-jemalloc build, after the skip list memtable change. Fix it by making mem table flush trigger by number of entries.
Test Plan: Run the test using both of jemalloc and non-jemalloc build.
sdong [Tue, 1 Dec 2015 05:32:59 +0000 (21:32 -0800)]
Revert previous behavior of internal_key_skipped_count
Summary: With recent commit 33e0c93826fd45640b24324a642ca03d8b9ef123, db iterator skips perf context counter internal_key_skipped_count when blindly issuing internal Next(). Now increment the counter by one when issuing this Next()
Test Plan: Run all existing tests
Reviewers: rven, yhchiang, IslamAbdelRahman, kradhakrishnan, igor, anthony
sdong [Mon, 30 Nov 2015 19:41:53 +0000 (11:41 -0800)]
Fix DBTest.SuggestCompactRangeTest for disable jemalloc case
Summary: DBTest.SuggestCompactRangeTest fails for the case when jemalloc is disabled, including ASAN and valgrind builds. It is caused by the improvement of skip list, which allocates different size of nodes for a new records. Fix it by using a special mem table that triggers a flush by number of entries. In that way the behavior will be consistent for all allocators.
Test Plan: Run the test with both of DISABLE_JEMALLOC=1 and 0
sdong [Tue, 24 Nov 2015 23:53:42 +0000 (15:53 -0800)]
DB to only flush the column family with the largest memtable while option.db_write_buffer_size is hit
Summary: When option.db_write_buffer_size is hit, we currently flush all column families. Move to flush the column family with the largest active memt table instead. In this way, we can avoid too many small files in some cases.
Test Plan: Modify test DBTest.SharedWriteBuffer to work with the updated behavior
Reviewers: kradhakrishnan, yhchiang, rven, anthony, IslamAbdelRahman, igor
sdong [Mon, 30 Nov 2015 18:30:22 +0000 (10:30 -0800)]
Initialize options.row_cache
Summary: options.row_cache should already been initialized as null by default. Still try to set it following current convention, because one valgrind failure reports a failure related to it.
projects / rocksdb.git / log