]> git-server-git.apps.pok.os.sepia.ceph.com Git - rocksdb.git/log
rocksdb.git
10 years agoBug of trivial move of dynamic level
sdong [Wed, 15 Apr 2015 02:58:52 +0000 (19:58 -0700)]
Bug of trivial move of dynamic level

Summary: D36669 introduces a bug that trivial moved data is not going to specific level but the next level, which will incorrectly be level 1 for level 0 compaciton if base level is not level 1. Fixing it by appreciating the output level

Test Plan: Run all tests

Reviewers: MarkCallaghan, rven, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D37119

Conflicts:
db/db_test.cc

10 years agoFix bad performance in debug mode
Igor Canadi [Mon, 13 Apr 2015 22:58:45 +0000 (15:58 -0700)]
Fix bad performance in debug mode

Summary:
See github issue 574: https://github.com/facebook/rocksdb/issues/574

Basically when we're running in DEBUG mode we're calling `usleep(0)` on
every mutex lock. I bisected the issue to
https://reviews.facebook.net/D36963. Instead of calling sleep(0), this
diff just avoids calling SleepForMicroseconds() when delay is not set.

Test Plan:
    bpl=10485760;overlap=10;mcz=2;del=300000000;levels=2;ctrig=10000000; delay=10000000; stop=10000000; wbn=30; mbc=20; mb=1073741824;wbs=268435456; dds=1; sync=0; r=100000; t=1; vs=800; bs=65536; cs=1048576; of=500000; si=1000000; ./db_bench --benchmarks=fillrandom --disable_seek_compaction=1 --mmap_read=0 --statistics=1 --histogram=1 --num=$r --threads=$t --value_size=$vs --block_size=$bs --cache_size=$cs --bloom_bits=10 --cache_numshardbits=4 --open_files=$of --verify_checksum=1 --db=/tmp/rdb10test --sync=$sync --disable_wal=1 --compression_type=snappy --stats_interval=$si --compression_ratio=0.5 --disable_data_sync=$dds --write_buffer_size=$wbs --target_file_size_base=$mb --max_write_buffer_number=$wbn --max_background_compactions=$mbc --level0_file_num_compaction_trigger=$ctrig --level0_slowdown_writes_trigger=$delay --level0_stop_writes_trigger=$stop --num_levels=$levels --delete_obsolete_files_period_micros=$del --min_level_to_compress=$mcz --max_grandparent_overlap_factor=$overlap --stats_per_interval=1 --max_bytes_for_level_base=$bpl --memtablerep=vector --use_existing_db=0 --disable_auto_compactions=1 --source_compaction_factor=10000000 | grep ops

Before:
fillrandom   :     117.525 micros/op 8508 ops/sec;    6.6 MB/s
After:
fillrandom   :       1.283 micros/op 779502 ops/sec;  606.6 MB/s

Reviewers: rven, yhchiang, sdong

Reviewed By: sdong

Subscribers: meyering, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36963

10 years agoFixed xfunc related compile errors in ROCKSDB_LITE
Yueh-Hsuan Chiang [Fri, 10 Apr 2015 04:05:09 +0000 (21:05 -0700)]
Fixed xfunc related compile errors in ROCKSDB_LITE

Summary:
Fixed xfunc related compile errors in ROCKSDB_LITE

Now make OPT=-DROCKSDB_LITE shared_lib -j32 would work

Test Plan:
make clean
make OPT=-DROCKSDB_LITE shared_lib -j32
make clean
make OPT=-DROCKSDB_LITE static_lib -j32

Reviewers: sdong, igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36825

10 years agoFix a compile error in ROCKSDB_LITE in db/db_impl.cc
Yueh-Hsuan Chiang [Fri, 10 Apr 2015 00:05:17 +0000 (17:05 -0700)]
Fix a compile error in ROCKSDB_LITE in db/db_impl.cc

Summary:
Fix a compile error in ROCKSDB_LITE in db/db_impl.cc
related to internal_stats.

Test Plan: make OPT=-DROCKSDB_LITE shared_lib

Reviewers: sdong, igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36819

10 years agoFix a compilation error in ROCKSDB_LITE in db/internal_stats.h
Yueh-Hsuan Chiang [Thu, 9 Apr 2015 23:39:03 +0000 (16:39 -0700)]
Fix a compilation error in ROCKSDB_LITE in db/internal_stats.h

Summary:
Fix a compilation error in ROCKSDB_LITE in db/internal_stats.h

Other compilation errors will be fixed in a separate diff.

Test Plan: make OPT=-DROCKSDB_LITE

Reviewers: sdong, igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36807

10 years agoFix level size overflow for options_.level_compaction_dynamic_level_bytes=true
sdong [Fri, 3 Apr 2015 05:24:50 +0000 (22:24 -0700)]
Fix level size overflow for options_.level_compaction_dynamic_level_bytes=true

Summary: Int is used for level size targets when options_.level_compaction_dynamic_level_bytes=true, which will cause overflow when database grows big. Fix it.

Test Plan: Add a new unit test which fails without the fix.

Reviewers: rven, yhchiang, MarkCallaghan, igor

Reviewed By: igor

Subscribers: leveldb, dhruba, yoshinorim

Differential Revision: https://reviews.facebook.net/D36453

10 years agoFix crash caused by opening an empty DB in readonly mode
Venkatesh Radhakrishnan [Wed, 1 Apr 2015 23:55:08 +0000 (16:55 -0700)]
Fix crash caused by opening an empty DB in readonly mode

Summary:
This diff fixes a crash found when an empty database is opened in readonly mode.
We now check the number of levels before we open the DB as a compacted DB.

Test Plan: DBTest.EmptyCompactedDB

Reviewers: igor, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D36327

Conflicts:
db/db_test.cc

10 years agoTSAN: avoid new link failure with -pg v3.10
Jim Meyering [Wed, 25 Mar 2015 21:40:41 +0000 (14:40 -0700)]
TSAN: avoid new link failure with -pg

Summary:
* Makefile (COMPILE_WITH_TSAN): Avoid a link failure by disabling
-pg when building with TSAN enabled.
Now that "make check" builds all $(PROGRAMS), it is linking
a few programs that were not normally linked before.
For example, this would fail to link with the following diagnostic:

  COMPILE_WITH_TSAN=1 make -j40 log_and_apply_bench

    CCLD     log_and_apply_bench
  ld: /usr/lib/../lib64/gcrt1.o: relocation R_X86_64_32S against `__libc_csu_fini' can not be used when making a shared object; recompile with -fPIC
  /usr/lib/../lib64/gcrt1.o: error adding symbols: Bad value
  collect2: error: ld returned 1 exit status
  Makefile:511: recipe for target 'log_and_apply_bench' failed
  make: *** [log_and_apply_bench] Error 1

Since removing -pg is sufficient to get past this link
failure, and no one cares about profiling TSAN-enabled
binaries anyway, we will refrain from linking with -pg
when TSAN testing is enabled.  Use a new variable, "pg"
which is set to "-pg" in most cases, but that is made
empty when COMPILE_WITH_TSAN is set.

Test Plan:
  Now, this succeeds:

    rm -f log_and_apply_bench
    COMPILE_WITH_TSAN=1 make -j40 log_and_apply_bench

Reviewers: igor.sugak, rven, sdong, ljin, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35943

10 years agoAdd a missing section title in HISTORY.md
Yueh-Hsuan Chiang [Wed, 25 Mar 2015 21:14:26 +0000 (14:14 -0700)]
Add a missing section title in HISTORY.md

Summary:
Add a missing section title in HISTORY.md

Test Plan:
no code change

10 years agoFixed a typo in RocksDBSample.java
Yueh-Hsuan Chiang [Wed, 25 Mar 2015 18:08:08 +0000 (11:08 -0700)]
Fixed a typo in RocksDBSample.java

Summary:
Fixed a typo in RocksDBSample.java

Test Plan:
make clean
make rocksdbjava -j32
make jtest

10 years agobuild: cause the "check" rule to depend on $(PROGRAMS)
Jim Meyering [Wed, 25 Mar 2015 05:09:40 +0000 (22:09 -0700)]
build: cause the "check" rule to depend on $(PROGRAMS)

Summary:
* Makefile (check): Cause "make check" to build all $(PROGRAMS),
so that it verifies that the few benchmark-only source files that
not already built via "make check" do compile and link successfully.

Test Plan:
run "make clean; make check", and verify that
table/table_reader_bench.cc is now compiled.
Before, it was not, which led to an incomplete fix
for a build break.

Reviewers: ljin, igor.sugak, rven, igor, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35883

10 years agoMerge branch 'master' of github.com:facebook/rocksdb
Anurag Indu [Wed, 25 Mar 2015 00:00:41 +0000 (17:00 -0700)]
Merge branch 'master' of github.com:facebook/rocksdb

10 years agoFix build
Anurag Indu [Tue, 24 Mar 2015 23:53:49 +0000 (16:53 -0700)]
Fix build

Test Plan: Running make all

Reviewers: sdong

Reviewed By: sdong

Subscribers: rven, yhchiang, igor, meyering, dhruba

Differential Revision: https://reviews.facebook.net/D35889

10 years agoUpdate HISTORY.md for 3.10.0
Yueh-Hsuan Chiang [Tue, 24 Mar 2015 23:39:39 +0000 (16:39 -0700)]
Update HISTORY.md for 3.10.0

Summary: Update HISTORY.md for 3.10.0

Test Plan: no code chagne.

Reviewers: sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35871

10 years agoMerge branch 'master' of github.com:facebook/rocksdb
Anurag Indu [Tue, 24 Mar 2015 23:27:39 +0000 (16:27 -0700)]
Merge branch 'master' of github.com:facebook/rocksdb

10 years agoFixing build issue
Anurag Indu [Tue, 24 Mar 2015 22:06:09 +0000 (15:06 -0700)]
Fixing build issue

Summary: Fixing issues with get context function.

Test Plan: Run make commit-prereq

Reviewers: sdong, meyering, yhchiang

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35853

10 years agoMerge pull request #555 from pshareghi/staticLZ4
Yueh-Hsuan Chiang [Tue, 24 Mar 2015 22:39:42 +0000 (15:39 -0700)]
Merge pull request #555 from pshareghi/staticLZ4

[RocksJava] Added static lz4 support for roccksjavastatic

10 years agoMerge pull request #556 from fyrz/RocksJava-Maven-Fix
Yueh-Hsuan Chiang [Tue, 24 Mar 2015 21:58:00 +0000 (14:58 -0700)]
Merge pull request #556 from fyrz/RocksJava-Maven-Fix

[RocksJava] Maven build fix

10 years agoFix compile error when NROCKSDB_THREAD_STATUS is not used.
Yueh-Hsuan Chiang [Tue, 24 Mar 2015 21:50:32 +0000 (14:50 -0700)]
Fix compile error when NROCKSDB_THREAD_STATUS is not used.

Summary: Fix compile error when NROCKSDB_THREAD_STATUS is not used.

Test Plan: make dbg OPT=-DNROCKSDB_THREAD_STATUS -j32

Reviewers: sdong, igor, rven

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35847

10 years ago[RocksJava] Maven build fix
fyrz [Tue, 24 Mar 2015 21:52:30 +0000 (22:52 +0100)]
[RocksJava] Maven build fix

10 years agoAdded static lz4 support for roccksjavastatic
Pooya Shareghi [Tue, 24 Mar 2015 17:40:00 +0000 (10:40 -0700)]
Added static lz4 support for roccksjavastatic

Usage:
make clean rocksdbjavastatic

Dependency:
LZ4

To install LZ4:
git clone https://github.com/Cyan4973/lz4.git && \
cd lz4/lib && \
make install

Alternativel installation method:
curl -O https://codeload.github.com/Cyan4973/lz4/tar.gz/r127
mv r127 lz4-r127.tar.gz
tar xvzf lz4-r127.tar.gz
cd lz4-r127/lib && \
make install

Signed-off-by: Pooya Shareghi <shareghi@gmail.com>
10 years agoMerge pull request #545 from fyrz/RocksJava-Level-Compression
Yueh-Hsuan Chiang [Tue, 24 Mar 2015 19:14:09 +0000 (12:14 -0700)]
Merge pull request #545 from fyrz/RocksJava-Level-Compression

[RocksJava] Add compression per level to API