]> git-server-git.apps.pok.os.sepia.ceph.com Git - rocksdb.git/log
rocksdb.git
3 years agodb: VersionEdit can understand the format introduced in PR 3488. ceph-reef-v7.8.fb
Radoslaw Zarzynski [Tue, 21 Aug 2018 19:47:48 +0000 (21:47 +0200)]
db: VersionEdit can understand the format introduced in PR 3488.

RocksDB's PR 3488 introduced a new format of `VersionEdit`
encoding which is not understandable for older versions.
Thus, the change broke forward compatibility and has been
reverted in PR 3762. Later, PR 3765 reiterated the concept
but in a way that does not provide compatibility with the very
short-living format of PR 3488.

This change tries to address the issue of accessing DBs
in format of 3488 by ignoring the special entries.

Fixes: http://tracker.ceph.com/issues/25146
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit ffed5e68515620e1de0b66d96d2ec15cef1c82a7)
(cherry picked from commit c540de6f709b66efd41436694f72d6f7986a325b)

3 years agoUpdate HISTORY.md and version to 7.8.3
akankshamahajan [Tue, 29 Nov 2022 14:51:03 +0000 (06:51 -0800)]
Update HISTORY.md and version to 7.8.3

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

3 years agoUpdate HISTORY for reverts
Peter Dillinger [Mon, 28 Nov 2022 17:37:05 +0000 (09:37 -0800)]
Update HISTORY for reverts

3 years agoRevert "Improve / refactor anonymous mmap capabilities (#10810)"
Peter Dillinger [Mon, 28 Nov 2022 17:34:16 +0000 (09:34 -0800)]
Revert "Improve / refactor anonymous mmap capabilities (#10810)"

This reverts commit 8367f0d2d76de0f7d096cc65f5f9ebfb907d551a.

3 years agoRevert "Fix include of windows.h in mmap.h (#10885)"
Peter Dillinger [Mon, 28 Nov 2022 17:33:56 +0000 (09:33 -0800)]
Revert "Fix include of windows.h in mmap.h (#10885)"

This reverts commit 49b7f219de87e4429067666cd92f826fe202f2f1.

3 years agoupdate version.h for 7.8.2
Andrew Kryczka [Mon, 28 Nov 2022 07:14:56 +0000 (23:14 -0800)]
update version.h for 7.8.2

3 years agobatch latest fixes into 7.8.2
Andrew Kryczka [Mon, 28 Nov 2022 07:12:09 +0000 (23:12 -0800)]
batch latest fixes into 7.8.2

3 years agoFix CompactionIterator flag for penultimate level output (#10967)
Andrew Kryczka [Tue, 22 Nov 2022 00:14:03 +0000 (16:14 -0800)]
Fix CompactionIterator flag for penultimate level output (#10967)

Summary:
We were not resetting it in non-debug mode so it could be true once and then stay true for future keys where it should be false. This PR adds the reset logic.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10967

Test Plan:
- built `db_bench` with DEBUG_LEVEL=0
- ran benchmark: `TEST_TMPDIR=/dev/shm/prefix ./db_bench -benchmarks=fillrandom -compaction_style=1 -preserve_internal_time_seconds=100 -preclude_last_level_data_seconds=10 -write_buffer_size=1048576 -target_file_size_base=1048576 -subcompactions=8 -duration=120`
- compared "output_to_penultimate_level: X bytes + last: Y bytes" lines in LOG output
  - Before this fix, Y was always zero
  - After this fix, Y gradually increased throughout the benchmark

Reviewed By: riversand963

Differential Revision: D41417726

Pulled By: ajkr

fbshipit-source-id: ace1e9a289e751a5b0c2fbaa8addd4eda5525329

3 years agoFix flush picking non-consecutive memtables (#10921)
Andrew Kryczka [Fri, 4 Nov 2022 22:55:54 +0000 (15:55 -0700)]
Fix flush picking non-consecutive memtables (#10921)

Summary:
Prevents `MemTableList::PickMemtablesToFlush()` from picking non-consecutive memtables. It leads to wrong ordering in L0 if the files are committed, or an error like below if force_consistency_checks=true catches it:

```
Corruption: force_consistency_checks: VersionBuilder: L0 file https://github.com/facebook/rocksdb/issues/25 with seqno 320416 368066 vs. file https://github.com/facebook/rocksdb/issues/24 with seqno 336037 352068
```

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10921

Test Plan: fix the expectation in the existing test of this behavior

Reviewed By: riversand963

Differential Revision: D41046935

Pulled By: ajkr

fbshipit-source-id: 783696bff56115063d5dc5856dfaed6a9881d1ab

3 years agoPrevent iterating over range tombstones beyond `iterate_upper_bound` (#10966) (#10985)
Changyu Bi [Thu, 24 Nov 2022 01:29:25 +0000 (17:29 -0800)]
Prevent iterating over range tombstones beyond `iterate_upper_bound` (#10966) (#10985)

Summary:
Currently, `iterate_upper_bound` is not checked for range tombstone keys in MergingIterator. This may impact performance when there is a large number of range tombstones right after `iterate_upper_bound`. This PR fixes this issue by checking `iterate_upper_bound` in MergingIterator for range tombstone keys.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10966

Test Plan:
- added unit test
- stress test: `python3 tools/db_crashtest.py whitebox --simple --verify_iterator_with_expected_state_one_in=5 --delrangepercent=5 --prefixpercent=18 --writepercent=48 --readpercen=15 --duration=36000 --range_deletion_width=100`
- ran different stress tests over sandcastle
- Falcon team ran some test traffic and saw reduced CPU usage on processing range tombstones.