]> git-server-git.apps.pok.os.sepia.ceph.com Git - rocksdb.git/commitdiff
Branch cut 9.1.fb
authorakankshamahajan <akankshamahajan@fb.com>
Mon, 25 Mar 2024 15:54:51 +0000 (08:54 -0700)
committerakankshamahajan <akankshamahajan@fb.com>
Mon, 25 Mar 2024 15:54:51 +0000 (08:54 -0700)
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

22 files changed:
HISTORY.md
unreleased_history/behavior_changes/generic_rate_limiter_burst.md [deleted file]
unreleased_history/behavior_changes/max_successive_merges_io.md [deleted file]
unreleased_history/bug_fixes/blockcachetier_blob_as_mergebase.md [deleted file]
unreleased_history/bug_fixes/blockcachetier_tablecachemiss.md [deleted file]
unreleased_history/bug_fixes/data_race_in_wal_manager.md [deleted file]
unreleased_history/bug_fixes/recycle_logs_point_in_time_recovery.md [deleted file]
unreleased_history/new_features/get_merge_operands_continue_callback.md [deleted file]
unreleased_history/new_features/new_ingest_files_behavior.md [deleted file]
unreleased_history/new_features/retry_on_corruption.md [deleted file]
unreleased_history/new_features/temperature_handling.md [deleted file]
unreleased_history/new_features/wbwi_get_entity_from_batch.md [deleted file]
unreleased_history/new_features/write_time_and_timed_put.md [deleted file]
unreleased_history/performance_improvements/java_api_multiget.md [deleted file]
unreleased_history/public_api_changes/best_effort_atomic.md [deleted file]
unreleased_history/public_api_changes/bottommost_temperature.md [deleted file]
unreleased_history/public_api_changes/new_block_cache_perf_counters.md [deleted file]
unreleased_history/public_api_changes/new_remote_compaction_api.md [deleted file]
unreleased_history/public_api_changes/optional_skip_udt_based_validation_write_committed.md [deleted file]
unreleased_history/public_api_changes/perf_level_for_blocking_events.md [deleted file]
unreleased_history/public_api_changes/rate_limiter_burst.md [deleted file]
unreleased_history/public_api_changes/write-memtable-time-new-perf-level.md [deleted file]

index 7d2bc89e117957f6a2f253e5dbd0f8697a3abb7b..058f326c618da2798a0e785c22c77cf9fb17a2a3 100644 (file)
@@ -1,6 +1,51 @@
 # Rocksdb Change Log
 > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
 
+## 9.1.0 (03/22/2024)
+### New Features
+* Added an option, `GetMergeOperandsOptions::continue_cb`, to give users the ability to end `GetMergeOperands()`'s lookup process before all merge operands were found.
+* *Add sanity checks for ingesting external files that currently checks if the user key comparator used to create the file is compatible with the column family's user key comparator.
+*Support ingesting external files for column family that has user-defined timestamps in memtable only enabled.
+* On file systems that support storage level data checksum and reconstruction, retry SST block reads for point lookups, scans, and flush and compaction if there's a checksum mismatch on the initial read.
+* Some enhancements and fixes to experimental Temperature handling features, including new `default_write_temperature` CF option and opening an `SstFileWriter` with a temperature.
+* `WriteBatchWithIndex` now supports wide-column point lookups via the `GetEntityFromBatch` API. See the API comments for more details.
+* *Implement experimental features: API `Iterator::GetProperty("rocksdb.iterator.write-time")` to allow users to get data's approximate write unix time and write data with a specific write time via `WriteBatch::TimedPut` API.
+
+### Public API Changes
+* Best-effort recovery (`best_efforts_recovery == true`) may now be used together with atomic flush (`atomic_flush == true`). The all-or-nothing recovery guarantee for atomically flushed data will be upheld.
+* Remove deprecated option `bottommost_temperature`, already replaced by `last_level_temperature`
+* Added new PerfContext counters for block cache bytes read - block_cache_index_read_byte, block_cache_filter_read_byte, block_cache_compression_dict_read_byte, and block_cache_read_byte.
+* Deprecate experimental Remote Compaction APIs - StartV2() and WaitForCompleteV2() and introduce Schedule() and Wait(). The new APIs essentially does the same thing as the old APIs. They allow taking externally generated unique id to wait for remote compaction to complete.
+* *For API `WriteCommittedTransaction::GetForUpdate`, if the column family enables user-defined timestamp, it was mandated that argument `do_validate` cannot be false, and UDT based validation has to be done with a user set read timestamp. It's updated to make the UDT based validation optional if user sets `do_validate` to false and does not set a read timestamp. With this, `GetForUpdate` skips UDT based validation and it's users' responsibility to enforce the UDT invariant. SO DO NOT skip this UDT-based validation if users do not have ways to enforce the UDT invariant. Ways to enforce the invariant on the users side include manage a monotonically increasing timestamp, commit transactions in a single thread etc.
+* Defined a new PerfLevel `kEnableWait` to measure time spent by user threads blocked in RocksDB other than mutex, such as a write thread waiting to be added to a write group, a write thread delayed or stalled etc.
+* `RateLimiter`'s API no longer requires the burst size to be the refill size. Users of `NewGenericRateLimiter()` can now provide burst size in `single_burst_bytes`. Implementors of `RateLimiter::SetSingleBurstBytes()` need to adapt their implementations to match the changed API doc.
+* Add `write_memtable_time` to the newly introduced PerfLevel `kEnableWait`.
+
+### Behavior Changes
+* `RateLimiter`s created by `NewGenericRateLimiter()` no longer modify the refill period when `SetSingleBurstBytes()` is called.
+* Merge writes will only keep merge operand count within `ColumnFamilyOptions::max_successive_merges` when the key's merge operands are all found in memory, unless `strict_max_successive_merges` is explicitly set.
+
+### Bug Fixes
+* Fixed `kBlockCacheTier` reads to return `Status::Incomplete` when I/O is needed to fetch a merge chain's base value from a blob file.
+* Fixed `kBlockCacheTier` reads to return `Status::Incomplete` on table cache miss rather than incorrectly returning an empty value.
+* Fixed a data race in WalManager that may affect how frequent PurgeObsoleteWALFiles() runs.
+* Re-enable the recycle_log_file_num option in DBOptions for kPointInTimeRecovery WAL recovery mode, which was previously disabled due to a bug in the recovery logic. This option is incompatible with WriteOptions::disableWAL. A Status::InvalidArgument() will be returned if disableWAL is specified.
+
+### Performance Improvements
+* Java API `multiGet()` variants now take advantage of the underlying batched `multiGet()` performance improvements.
+Before
+```
+Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
+MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 64 thrpt 25 6315.541 ± 8.106 ops/s
+MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 1024 thrpt 25 6975.468 ± 68.964 ops/s
+```
+After
+```
+Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
+MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 64 thrpt 25 7046.739 ± 13.299 ops/s
+MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 1024 thrpt 25 7654.521 ± 60.121 ops/s
+```
+
 ## 9.0.0 (02/16/2024)
 ### New Features
 * Provide support for FSBuffer for point lookups. Also added support for scans and compactions that don't go through prefetching.
diff --git a/unreleased_history/behavior_changes/generic_rate_limiter_burst.md b/unreleased_history/behavior_changes/generic_rate_limiter_burst.md
deleted file mode 100644 (file)
index e45b56e..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* `RateLimiter`s created by `NewGenericRateLimiter()` no longer modify the refill period when `SetSingleBurstBytes()` is called.
diff --git a/unreleased_history/behavior_changes/max_successive_merges_io.md b/unreleased_history/behavior_changes/max_successive_merges_io.md
deleted file mode 100644 (file)
index c331b40..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* Merge writes will only keep merge operand count within `ColumnFamilyOptions::max_successive_merges` when the key's merge operands are all found in memory, unless `strict_max_successive_merges` is explicitly set.
diff --git a/unreleased_history/bug_fixes/blockcachetier_blob_as_mergebase.md b/unreleased_history/bug_fixes/blockcachetier_blob_as_mergebase.md
deleted file mode 100644 (file)
index 13ee51a..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* Fixed `kBlockCacheTier` reads to return `Status::Incomplete` when I/O is needed to fetch a merge chain's base value from a blob file.
diff --git a/unreleased_history/bug_fixes/blockcachetier_tablecachemiss.md b/unreleased_history/bug_fixes/blockcachetier_tablecachemiss.md
deleted file mode 100644 (file)
index 298b69b..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* Fixed `kBlockCacheTier` reads to return `Status::Incomplete` on table cache miss rather than incorrectly returning an empty value.
diff --git a/unreleased_history/bug_fixes/data_race_in_wal_manager.md b/unreleased_history/bug_fixes/data_race_in_wal_manager.md
deleted file mode 100644 (file)
index e561072..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* Fixed a data race in WalManager that may affect how frequent PurgeObsoleteWALFiles() runs.
\ No newline at end of file
diff --git a/unreleased_history/bug_fixes/recycle_logs_point_in_time_recovery.md b/unreleased_history/bug_fixes/recycle_logs_point_in_time_recovery.md
deleted file mode 100644 (file)
index d11ba21..0000000
+++ /dev/null
@@ -1 +0,0 @@
-Re-enable the recycle_log_file_num option in DBOptions for kPointInTimeRecovery WAL recovery mode, which was previously disabled due to a bug in the recovery logic. This option is incompatible with WriteOptions::disableWAL. A Status::InvalidArgument() will be returned if disableWAL is specified.
diff --git a/unreleased_history/new_features/get_merge_operands_continue_callback.md b/unreleased_history/new_features/get_merge_operands_continue_callback.md
deleted file mode 100644 (file)
index 69ad144..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* Added an option, `GetMergeOperandsOptions::continue_cb`, to give users the ability to end `GetMergeOperands()`'s lookup process before all merge operands were found.
diff --git a/unreleased_history/new_features/new_ingest_files_behavior.md b/unreleased_history/new_features/new_ingest_files_behavior.md
deleted file mode 100644 (file)
index ec50847..0000000
+++ /dev/null
@@ -1,2 +0,0 @@
-*Add sanity checks for ingesting external files that currently checks if the user key comparator used to create the file is compatible with the column family's user key comparator.
-*Support ingesting external files for column family that has user-defined timestamps in memtable only enabled.
\ No newline at end of file
diff --git a/unreleased_history/new_features/retry_on_corruption.md b/unreleased_history/new_features/retry_on_corruption.md
deleted file mode 100644 (file)
index 807c1b5..0000000
+++ /dev/null
@@ -1 +0,0 @@
-On file systems that support storage level data checksum and reconstruction, retry SST block reads for point lookups, scans, and flush and compaction if there's a checksum mismatch on the initial read.
diff --git a/unreleased_history/new_features/temperature_handling.md b/unreleased_history/new_features/temperature_handling.md
deleted file mode 100644 (file)
index 2001fd8..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* Some enhancements and fixes to experimental Temperature handling features, including new `default_write_temperature` CF option and opening an `SstFileWriter` with a temperature.
diff --git a/unreleased_history/new_features/wbwi_get_entity_from_batch.md b/unreleased_history/new_features/wbwi_get_entity_from_batch.md
deleted file mode 100644 (file)
index 3eab941..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* `WriteBatchWithIndex` now supports wide-column point lookups via the `GetEntityFromBatch` API. See the API comments for more details.
diff --git a/unreleased_history/new_features/write_time_and_timed_put.md b/unreleased_history/new_features/write_time_and_timed_put.md
deleted file mode 100644 (file)
index fd6654d..0000000
+++ /dev/null
@@ -1 +0,0 @@
-*Implement experimental features: API `Iterator::GetProperty("rocksdb.iterator.write-time")` to allow users to get data's approximate write unix time and write data with a specific write time via `WriteBatch::TimedPut` API.
\ No newline at end of file
diff --git a/unreleased_history/performance_improvements/java_api_multiget.md b/unreleased_history/performance_improvements/java_api_multiget.md
deleted file mode 100644 (file)
index 1f60fe7..0000000
+++ /dev/null
@@ -1,14 +0,0 @@
-Java API `multiGet()` variants now take advantage of the underlying batched `multiGet()` performance improvements.
-
-Before
-```
-Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
-MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 64 thrpt 25 6315.541 ± 8.106 ops/s
-MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 1024 thrpt 25 6975.468 ± 68.964 ops/s
-```
-After
-```
-Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
-MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 64 thrpt 25 7046.739 ± 13.299 ops/s
-MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 1024 thrpt 25 7654.521 ± 60.121 ops/s
-```
\ No newline at end of file
diff --git a/unreleased_history/public_api_changes/best_effort_atomic.md b/unreleased_history/public_api_changes/best_effort_atomic.md
deleted file mode 100644 (file)
index 5f610ec..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* Best-effort recovery (`best_efforts_recovery == true`) may now be used together with atomic flush (`atomic_flush == true`). The all-or-nothing recovery guarantee for atomically flushed data will be upheld.
diff --git a/unreleased_history/public_api_changes/bottommost_temperature.md b/unreleased_history/public_api_changes/bottommost_temperature.md
deleted file mode 100644 (file)
index 998a154..0000000
+++ /dev/null
@@ -1 +0,0 @@
-Remove deprecated option `bottommost_temperature`, already replaced by `last_level_temperature`
diff --git a/unreleased_history/public_api_changes/new_block_cache_perf_counters.md b/unreleased_history/public_api_changes/new_block_cache_perf_counters.md
deleted file mode 100644 (file)
index 70fe323..0000000
+++ /dev/null
@@ -1 +0,0 @@
-Added new PerfContext counters for block cache bytes read - block_cache_index_read_byte, block_cache_filter_read_byte, block_cache_compression_dict_read_byte, and block_cache_read_byte.
diff --git a/unreleased_history/public_api_changes/new_remote_compaction_api.md b/unreleased_history/public_api_changes/new_remote_compaction_api.md
deleted file mode 100644 (file)
index eac106c..0000000
+++ /dev/null
@@ -1 +0,0 @@
-Deprecate experimental Remote Compaction APIs - StartV2() and WaitForCompleteV2() and introduce Schedule() and Wait(). The new APIs essentially does the same thing as the old APIs. They allow taking externally generated unique id to wait for remote compaction to complete.
diff --git a/unreleased_history/public_api_changes/optional_skip_udt_based_validation_write_committed.md b/unreleased_history/public_api_changes/optional_skip_udt_based_validation_write_committed.md
deleted file mode 100644 (file)
index 9a85a20..0000000
+++ /dev/null
@@ -1 +0,0 @@
-*For API `WriteCommittedTransaction::GetForUpdate`, if the column family enables user-defined timestamp, it was mandated that argument `do_validate` cannot be false, and UDT based validation has to be done with a user set read timestamp. It's updated to make the UDT based validation optional if user sets `do_validate` to false and does not set a read timestamp. With this, `GetForUpdate` skips UDT based validation and it's users' responsibility to enforce the UDT invariant. SO DO NOT skip this UDT-based validation if users do not have ways to enforce the UDT invariant. Ways to enforce the invariant on the users side include manage a monotonically increasing timestamp, commit transactions in a single thread etc.
\ No newline at end of file
diff --git a/unreleased_history/public_api_changes/perf_level_for_blocking_events.md b/unreleased_history/public_api_changes/perf_level_for_blocking_events.md
deleted file mode 100644 (file)
index d7dbc10..0000000
+++ /dev/null
@@ -1 +0,0 @@
-Defined a new PerfLevel `kEnableWait` to measure time spent by user threads blocked in RocksDB other than mutex, such as a write thread waiting to be added to a write group, a write thread delayed or stalled etc.
diff --git a/unreleased_history/public_api_changes/rate_limiter_burst.md b/unreleased_history/public_api_changes/rate_limiter_burst.md
deleted file mode 100644 (file)
index ffa9fc9..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* `RateLimiter`'s API no longer requires the burst size to be the refill size. Users of `NewGenericRateLimiter()` can now provide burst size in `single_burst_bytes`. Implementors of `RateLimiter::SetSingleBurstBytes()` need to adapt their implementations to match the changed API doc.
diff --git a/unreleased_history/public_api_changes/write-memtable-time-new-perf-level.md b/unreleased_history/public_api_changes/write-memtable-time-new-perf-level.md
deleted file mode 100644 (file)
index 08f5628..0000000
+++ /dev/null
@@ -1 +0,0 @@
-* Add `write_memtable_time` to the newly introduced PerfLevel `kEnableWait`.
\ No newline at end of file