Add Statistics.getHistogramString() to print more detailed outputs of a histogram

Summary:
Provide a way for users to know more detailed ditribution of a histogram metrics. Example outputs:

Manually add statement
  fprintf(stdout, "%s\n", dbstats->getHistogramString(SST_READ_MICROS).c_str());
Will print out something like:

Count: 989151  Average: 1.7659  StdDev: 1.52
Min: 0.0000  Median: 1.2071  Max: 860.0000
Percentiles: P50: 1.21 P75: 1.70 P99: 5.12 P99.9: 13.67 P99.99: 21.70
------------------------------------------------------
[       0,       1 )   390839  39.513%  39.513% ########
[       1,       2 )   500918  50.641%  90.154% ##########
[       2,       3 )    79358   8.023%  98.177% ##
[       3,       4 )     6297   0.637%  98.813%
[       4,       5 )     1712   0.173%  98.986%
[       5,       6 )     1134   0.115%  99.101%
[       6,       7 )     1222   0.124%  99.224%
[       7,       8 )     1529   0.155%  99.379%
[       8,       9 )     1264   0.128%  99.507%
[       9,      10 )      988   0.100%  99.607%
[      10,      12 )     1378   0.139%  99.746%
[      12,      14 )     1828   0.185%  99.931%
[      14,      16 )      410   0.041%  99.972%
[      16,      18 )       72   0.007%  99.980%
[      18,      20 )       67   0.007%  99.986%
[      20,      25 )      106   0.011%  99.997%
[      25,      30 )       24   0.002%  99.999%
[      30,      35 )        1   0.000% 100.000%
[     250,     300 )        2   0.000% 100.000%
[     300,     350 )        1   0.000% 100.000%
[     800,     900 )        1   0.000% 100.000%

Test Plan: Manually add a print in db_bench and make sure it prints out as expected. Will add some codes to cover the function

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D43611

diff --git a/HISTORY.md b/HISTORY.md
index 2f20ecbe3b9fe886ef5d47c21a873a8263db7bb6..28f2b64f0626751d72764cdaf32b4c17ea6d0096 100644 (file)
--- a/HISTORY.md
+++ b/HISTORY.md
@@ -7,6 +7,7 @@
 * RollbackToSavePoint() in WriteBatch/WriteBatchWithIndex
 * Add NewCompactOnDeletionCollectorFactory() in utilities/table_properties_collectors, which allows rocksdb to mark a SST file as need-compaction when it observes at least D deletion entries in any N consecutive entries in that SST file.  Note that this feature depends on an experimental NeedCompact() API --- the result of this API will not persist after DB restart.
 * Add DBOptions::delete_scheduler. Use NewDeleteScheduler() in include/rocksdb/delete_scheduler.h to create a DeleteScheduler that can be shared among multiple RocksDB instances to control the file deletion rate of SST files that exist in the first db_path.
+* Add statistics::getHistogramString() to print detailed distribution of a histogram metric.
 
 ### Public API Changes
 * Deprecated WriteOptions::timeout_hint_us. We no longer support write timeout. If you really need this option, talk to us and we might consider returning it.

diff --git a/include/rocksdb/statistics.h b/include/rocksdb/statistics.h
index 61a9958aec0bbc289d3affcc61f95f76918bdaf1..79c77766a404464ec93211c118aa4a36029725a1 100644 (file)
--- a/include/rocksdb/statistics.h
+++ b/include/rocksdb/statistics.h
@@ -295,7 +295,7 @@ class Statistics {
   virtual uint64_t getTickerCount(uint32_t tickerType) const = 0;
   virtual void histogramData(uint32_t type,
                              HistogramData* const data) const = 0;
-
+  virtual std::string getHistogramString(uint32_t type) const { return ""; }
   virtual void recordTick(uint32_t tickerType, uint64_t count = 0) = 0;
   virtual void setTickerCount(uint32_t tickerType, uint64_t count) = 0;
   virtual void measureTime(uint32_t histogramType, uint64_t time) = 0;

diff --git a/util/statistics.cc b/util/statistics.cc
index ba7670bb4cdd4dad48ffef985eaf13296e0264b2..f06589a17b1e28d3133111d4dac4984e15c7e154 100644 (file)
--- a/util/statistics.cc
+++ b/util/statistics.cc
@@ -50,6 +50,12 @@ void StatisticsImpl::histogramData(uint32_t histogramType,
   histograms_[histogramType].Data(data);
 }
 
+std::string StatisticsImpl::getHistogramString(uint32_t histogramType) const {
+  assert(enable_internal_stats_ ? histogramType < INTERNAL_HISTOGRAM_ENUM_MAX
+                                : histogramType < HISTOGRAM_ENUM_MAX);
+  return histograms_[histogramType].ToString();
+}
+
 void StatisticsImpl::setTickerCount(uint32_t tickerType, uint64_t count) {
   assert(
     enable_internal_stats_ ?

diff --git a/util/statistics.h b/util/statistics.h
index c56900accbfec9a0decbfd7b96a4f3bc547f2240..55914f59ed62e04296629cf8f16cb5326209fb7d 100644 (file)
--- a/util/statistics.h
+++ b/util/statistics.h
@@ -37,6 +37,7 @@ class StatisticsImpl : public Statistics {
   virtual uint64_t getTickerCount(uint32_t ticker_type) const override;
   virtual void histogramData(uint32_t histogram_type,
                              HistogramData* const data) const override;
+  std::string getHistogramString(uint32_t histogram_type) const override;
 
   virtual void setTickerCount(uint32_t ticker_type, uint64_t count) override;
   virtual void recordTick(uint32_t ticker_type, uint64_t count) override;