Compare commits

...
Author SHA1 Message Date
Siying Dong 7dc9d706be Fix "*/" outside comment warning in env_hdfs.h 2018-03-02 14:44:34 -08:00
Andrew Kryczka ce58659f58 fix for non-fbcode linux 2018-03-02 13:57:30 -08:00
Andrew Kryczka 7f92556ad7 Comment out unused variables 2018-03-02 13:57:30 -08:00
Yi Wu 1209b6db5c Blob DB: remove existing garbage collection implementation
Summary:
Red diff to remove existing implementation of garbage collection. The current approach is reference counting kind of approach and require a lot of effort to get the size counter right on compaction and deletion. I'm going to go with a simple mark-sweep kind of approach and will send another PR for that.

CompactionEventListener was added solely for blob db and it adds complexity and overhead to compaction iterator. Removing it as well.
Closes https://github.com/facebook/rocksdb/pull/3551

Differential Revision: D7130190

Pulled By: yiwu-arbug

fbshipit-source-id: c3a375ad2639a3f6ed179df6eda602372cc5b8df
2018-03-02 12:57:23 -08:00
Adam Retter 2ac988c67e Add TransactionDB and OptimisticTransactionDB to the Java API
Summary:
Closes https://github.com/facebook/rocksdb/issues/697
Closes https://github.com/facebook/rocksdb/issues/1151
Closes https://github.com/facebook/rocksdb/pull/1298

Differential Revision: D7131402

Pulled By: sagar0

fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 10:34:13 -08:00
Maysam Yabandeh d060421c77 Fix a leak in prepared_section_completed_
Summary:
The zeroed entries were not removed from prepared_section_completed_ map. This patch adds a unit test to show the problem and fixes that by refactoring the code. The new code is more efficient since i) it uses two separate mutex to avoid contention between commit and prepare threads, ii) it uses a sorted vector for maintaining uniq log entires with prepare which avoids a very large heap with many duplicate entries.
Closes https://github.com/facebook/rocksdb/pull/3545

Differential Revision: D7106071

Pulled By: maysamyabandeh

fbshipit-source-id: b3ae17cb6cd37ef10b6b35e0086c15c758768a48
2018-03-01 20:41:56 -08:00
Yi Wu bf937cf15b Add "rocksdb.live-sst-files-size" DB property
Summary:
Add "rocksdb.live-sst-files-size" DB property which only include files of latest version. Existing "rocksdb.total-sst-files-size" include files from all versions and thus include files that's obsolete but not yet deleted. I'm going to use this new property to cap blob db sst + blob files size.
Closes https://github.com/facebook/rocksdb/pull/3548

Differential Revision: D7116939

Pulled By: yiwu-arbug

fbshipit-source-id: c6a52e45ce0f24ef78708156e1a923c1dd6bc79a
2018-03-01 18:01:10 -08:00
leviathan1995 ec5843dca9 Comment typo
Summary: Closes https://github.com/facebook/rocksdb/pull/3546

Differential Revision: D7111708

Pulled By: ajkr

fbshipit-source-id: 522a4a00eb3e34c73afcb86c1f75cd2e90e7608d
2018-02-28 09:56:45 -08:00
Andrew Kryczka 3ae0047278 skip CompactRange flush based on memtable contents
Summary:
CompactRange has a call to Flush because we guarantee that, at the time it's called, all existing keys in the range will be pushed through the user's compaction filter. However, previously the flush was done blindly, so it'd happen even if the memtable does not contain keys in the range specified by the user. This caused unnecessarily many L0 files to be created, leading to write stalls in some cases. This PR checks the memtable's contents, and decides to flush only if it overlaps with `CompactRange`'s range.

- Move the memtable overlap check logic from `ExternalSstFileIngestionJob` to `ColumnFamilyData::RangesOverlapWithMemtables`
- Reuse the above logic in `CompactRange` and skip flushing if no overlap
Closes https://github.com/facebook/rocksdb/pull/3520

Differential Revision: D7018897

Pulled By: ajkr

fbshipit-source-id: a3c6b1cfae56687b49dd89ccac7c948e53545934
2018-02-27 17:12:44 -08:00
Siying Dong c287c098a4 Update comments in DB::Close()
Summary: Closes https://github.com/facebook/rocksdb/pull/3543

Differential Revision: D7093251

Pulled By: siying

fbshipit-source-id: 4066b82c95ecb65866c5842d68ab13ab9f85d567
2018-02-27 12:42:31 -08:00
Istvan Szukacs d6336563e5 Adding CentOS 7 Vagrantfile & build script
Summary:
I have updated the Vagrantfile to have an entry for CentOS 7. Also created a simple build script which is pretty similar to the one in Beringei.

How to test:
```
vagrant up centos7
```
Todo:

Implement -j X for the build.
Closes https://github.com/facebook/rocksdb/pull/3530

Differential Revision: D7090739

Pulled By: ajkr

fbshipit-source-id: 9f9eda5b507568993543d08de7ce168dfc12282e
2018-02-26 15:27:17 -08:00
Zhongyi Xie ad05cbb182 DB:Open should fail on tmpfs when use_direct_reads=true
Summary:
Before:

> $ TEST_TMPDIR=/dev/shm ./db_bench -use_direct_reads=true -benchmarks=readrandomwriterandom -num=10000000 -reads=100000 -write_buffer_size=1048576 -target_file_size_base=1048576 -max_bytes_for_level_base=4194304 -max_background_jobs=12 -readwritepercent=50 -key_size=16 -value_size=48 -threads=32
DB path: [/dev/shm/dbbench]
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
db_bench: tpp.c:84: __pthread_tpp_change_priority: Assertion `new_prio == -1 || (new_prio >= fifo_min_prio && new_prio <= fifo_max_prio)' failed.
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument
put error: IO error: While open a file for random read: /dev/shm/dbbench/000007.sst: Invalid argument

After:
> TEST_TMPDIR=/dev/shm ./db_bench -use_direct_reads=true -benchmarks=readrandomwriterandom -num=10000000 -reads=100000 -write_buffer_size=1048576 -target_file_size_base=1048576 -max_bytes_for_level_base=4194304 -max_background_jobs=12 -readwritepercent=50 -key_size=16 -value_size=48 -threads=32
Initializing RocksDB Options from the specified file
Initializing RocksDB Options from command-line flags
open error: Not implemented: Direct I/O is not supported by the specified DB.
Closes https://github.com/facebook/rocksdb/pull/3539

Differential Revision: D7082658

Pulled By: miasantreble

fbshipit-source-id: f9d9c6ec3b5e9e049cab52154940ee101ba4d342
2018-02-26 14:58:06 -08:00
Dmitri Smirnov 7eb292da14 Fix a memory leak in WindowsThread
Summary:
_endthreadex does not return and thus objects
  for stack destructors do not run. This creates a memory leak.
  We remove the calls since _enthreadex called automatically after the
  threadproc returns i.e. thread exits.
Closes https://github.com/facebook/rocksdb/pull/3542

Differential Revision: D7088713

Pulled By: ajkr

fbshipit-source-id: 749ecafc6a9572f587f76e516547e07734349a54
2018-02-26 13:46:12 -08:00
236 changed files with 10405 additions and 1879 deletions

No files matched your search

+8
View File
@@ -2,6 +2,14 @@
## Unreleased
### Public API Change
* RocksDBOptionsParser::Parse()'s `ignore_unknown_options` argument will only be effective if the option file shows it is generated using a higher version of RocksDB than the current version.
* Remove CompactionEventListener.
### New Features
* Avoid unnecessarily flushing in `CompactRange()` when the range specified by the user does not overlap unflushed memtables.
* Add "rocksdb.live-sst-files-size" DB property to return total bytes of all SST files belong to the latest LSM tree.
### Bug Fixes
* Fix a leak in prepared_section_completed_ where the zeroed entries would not removed from the map.
## 5.12.0 (2/14/2018)
### Public API Change
Vendored
+5
View File
@@ -14,6 +14,11 @@ Vagrant.configure("2") do |config|
box.vm.box = "chef/centos-6.5"
end
config.vm.define "centos7" do |box|
box.vm.box = "centos/7"
box.vm.provision "shell", path: "build_tools/setup_centos7.sh"
end
config.vm.define "FreeBSD10" do |box|
box.vm.guest = :freebsd
box.vm.box = "robin/freebsd-10"
+43
View File
@@ -0,0 +1,43 @@
#!/bin/bash
set -e
ROCKSDB_VERSION="5.10.3"
ZSTD_VERSION="1.1.3"
echo "This script configures CentOS with everything needed to build and run RocksDB"
yum update -y && yum install epel-release -y
yum install -y \
wget \
gcc-c++ \
snappy snappy-devel \
zlib zlib-devel \
bzip2 bzip2-devel \
lz4-devel \
libasan \
gflags
mkdir -pv /usr/local/rocksdb-${ROCKSDB_VERSION}
ln -sfT /usr/local/rocksdb-${ROCKSDB_VERSION} /usr/local/rocksdb
wget -qO /tmp/zstd-${ZSTD_VERSION}.tar.gz https://github.com/facebook/zstd/archive/v${ZSTD_VERSION}.tar.gz
wget -qO /tmp/rocksdb-${ROCKSDB_VERSION}.tar.gz https://github.com/facebook/rocksdb/archive/v${ROCKSDB_VERSION}.tar.gz
cd /tmp
tar xzvf zstd-${ZSTD_VERSION}.tar.gz
tar xzvf rocksdb-${ROCKSDB_VERSION}.tar.gz -C /usr/local/
echo "Installing ZSTD..."
pushd zstd-${ZSTD_VERSION}
make && make install
popd
echo "Compiling RocksDB..."
cd /usr/local/rocksdb
chown -R vagrant:vagrant /usr/local/rocksdb/
sudo -u vagrant make static_lib
cd examples/
sudo -u vagrant make all
sudo -u vagrant ./c_simple_example
+3 -3
View File
@@ -40,9 +40,9 @@ static int DecodeValue(void* v) {
const std::string kLRU = "lru";
const std::string kClock = "clock";
void dumbDeleter(const Slice& key, void* value) {}
void dumbDeleter(const Slice& /*key*/, void* /*value*/) {}
void eraseDeleter(const Slice& key, void* value) {
void eraseDeleter(const Slice& /*key*/, void* value) {
Cache* cache = reinterpret_cast<Cache*>(value);
cache->Erase("foo");
}
@@ -470,7 +470,7 @@ class Value {
};
namespace {
void deleter(const Slice& key, void* value) {
void deleter(const Slice& /*key*/, void* value) {
delete static_cast<Value *>(value);
}
} // namespace
+1 -1
View File
@@ -586,7 +586,7 @@ Status ClockCacheShard::Insert(const Slice& key, uint32_t hash, void* value,
size_t charge,
void (*deleter)(const Slice& key, void* value),
Cache::Handle** out_handle,
Cache::Priority priority) {
Cache::Priority /*priority*/) {
CleanupContext context;
HashTable::accessor accessor;
char* key_data = new char[key.size()];
+1 -1
View File
@@ -53,7 +53,7 @@ Status ShardedCache::Insert(const Slice& key, void* value, size_t charge,
->Insert(key, hash, value, charge, deleter, handle, priority);
}
Cache::Handle* ShardedCache::Lookup(const Slice& key, Statistics* stats) {
Cache::Handle* ShardedCache::Lookup(const Slice& key, Statistics* /*stats*/) {
uint32_t hash = HashSlice(key);
return GetShard(Shard(hash))->Lookup(key, hash);
}
+4 -4
View File
@@ -61,10 +61,10 @@ TableBuilder* NewTableBuilder(
Status BuildTable(
const std::string& dbname, Env* env, const ImmutableCFOptions& ioptions,
const MutableCFOptions& mutable_cf_options, const EnvOptions& env_options,
TableCache* table_cache, InternalIterator* iter,
std::unique_ptr<InternalIterator> range_del_iter, FileMetaData* meta,
const InternalKeyComparator& internal_comparator,
const MutableCFOptions& /*mutable_cf_options*/,
const EnvOptions& env_options, TableCache* table_cache,
InternalIterator* iter, std::unique_ptr<InternalIterator> range_del_iter,
FileMetaData* meta, const InternalKeyComparator& internal_comparator,
const std::vector<std::unique_ptr<IntTblPropCollectorFactory>>*
int_tbl_prop_collector_factories,
uint32_t column_family_id, const std::string& column_family_name,
+8 -8
View File
@@ -252,7 +252,7 @@ struct rocksdb_comparator_t : public Comparator {
// No-ops since the C binding does not support key shortening methods.
virtual void FindShortestSeparator(std::string*,
const Slice&) const override {}
virtual void FindShortSuccessor(std::string* key) const override {}
virtual void FindShortSuccessor(std::string* /*key*/) const override {}
};
struct rocksdb_filterpolicy_t : public FilterPolicy {
@@ -367,7 +367,7 @@ struct rocksdb_mergeoperator_t : public MergeOperator {
virtual bool PartialMergeMulti(const Slice& key,
const std::deque<Slice>& operand_list,
std::string* new_value,
Logger* logger) const override {
Logger* /*logger*/) const override {
size_t operand_count = operand_list.size();
std::vector<const char*> operand_pointers(operand_count);
std::vector<size_t> operand_sizes(operand_count);
@@ -2166,8 +2166,8 @@ void rocksdb_options_set_level0_stop_writes_trigger(
opt->rep.level0_stop_writes_trigger = n;
}
void rocksdb_options_set_max_mem_compaction_level(rocksdb_options_t* opt,
int n) {}
void rocksdb_options_set_max_mem_compaction_level(rocksdb_options_t* /*opt*/,
int /*n*/) {}
void rocksdb_options_set_wal_recovery_mode(rocksdb_options_t* opt,int mode) {
opt->rep.wal_recovery_mode = static_cast<WALRecoveryMode>(mode);
@@ -2231,8 +2231,8 @@ void rocksdb_options_set_manifest_preallocation_size(
}
// noop
void rocksdb_options_set_purge_redundant_kvs_while_flush(rocksdb_options_t* opt,
unsigned char v) {}
void rocksdb_options_set_purge_redundant_kvs_while_flush(
rocksdb_options_t* /*opt*/, unsigned char /*v*/) {}
void rocksdb_options_set_use_direct_reads(rocksdb_options_t* opt,
unsigned char v) {
@@ -2402,7 +2402,7 @@ void rocksdb_options_set_table_cache_numshardbits(
}
void rocksdb_options_set_table_cache_remove_scan_count_limit(
rocksdb_options_t* opt, int v) {
rocksdb_options_t* /*opt*/, int /*v*/) {
// this option is deprecated
}
@@ -2973,7 +2973,7 @@ rocksdb_sstfilewriter_t* rocksdb_sstfilewriter_create(
rocksdb_sstfilewriter_t* rocksdb_sstfilewriter_create_with_comparator(
const rocksdb_envoptions_t* env, const rocksdb_options_t* io_options,
const rocksdb_comparator_t* comparator) {
const rocksdb_comparator_t* /*comparator*/) {
rocksdb_sstfilewriter_t* writer = new rocksdb_sstfilewriter_t;
writer->rep = new SstFileWriter(env->rep, io_options->rep);
return writer;
+67
View File
@@ -31,6 +31,7 @@
#include "monitoring/thread_status_util.h"
#include "options/options_helper.h"
#include "table/block_based_table_factory.h"
#include "table/merging_iterator.h"
#include "util/autovector.h"
#include "util/compression.h"
@@ -845,6 +846,10 @@ uint64_t ColumnFamilyData::GetTotalSstFilesSize() const {
return VersionSet::GetTotalSstFilesSize(dummy_versions_);
}
uint64_t ColumnFamilyData::GetLiveSstFilesSize() const {
return current_->GetSstFilesSize();
}
MemTable* ColumnFamilyData::ConstructNewMemtable(
const MutableCFOptions& mutable_cf_options, SequenceNumber earliest_seq) {
return new MemTable(internal_comparator_, ioptions_, mutable_cf_options,
@@ -881,6 +886,68 @@ bool ColumnFamilyData::RangeOverlapWithCompaction(
smallest_user_key, largest_user_key, level);
}
Status ColumnFamilyData::RangesOverlapWithMemtables(
const autovector<Range>& ranges, SuperVersion* super_version,
bool* overlap) {
assert(overlap != nullptr);
*overlap = false;
// Create an InternalIterator over all unflushed memtables
Arena arena;
ReadOptions read_opts;
read_opts.total_order_seek = true;
MergeIteratorBuilder merge_iter_builder(&internal_comparator_, &arena);
merge_iter_builder.AddIterator(
super_version->mem->NewIterator(read_opts, &arena));
super_version->imm->AddIterators(read_opts, &merge_iter_builder);
ScopedArenaIterator memtable_iter(merge_iter_builder.Finish());
std::vector<InternalIterator*> memtable_range_del_iters;
auto* active_range_del_iter =
super_version->mem->NewRangeTombstoneIterator(read_opts);
if (active_range_del_iter != nullptr) {
memtable_range_del_iters.push_back(active_range_del_iter);
}
super_version->imm->AddRangeTombstoneIterators(read_opts,
&memtable_range_del_iters);
RangeDelAggregator range_del_agg(internal_comparator_, {} /* snapshots */,
false /* collapse_deletions */);
Status status;
{
std::unique_ptr<InternalIterator> memtable_range_del_iter(
NewMergingIterator(&internal_comparator_,
memtable_range_del_iters.empty()
? nullptr
: &memtable_range_del_iters[0],
static_cast<int>(memtable_range_del_iters.size())));
status = range_del_agg.AddTombstones(std::move(memtable_range_del_iter));
}
for (size_t i = 0; i < ranges.size() && status.ok() && !*overlap; ++i) {
auto* vstorage = super_version->current->storage_info();
auto* ucmp = vstorage->InternalComparator()->user_comparator();
InternalKey range_start(ranges[i].start, kMaxSequenceNumber,
kValueTypeForSeek);
memtable_iter->Seek(range_start.Encode());
status = memtable_iter->status();
ParsedInternalKey seek_result;
if (status.ok()) {
if (memtable_iter->Valid() &&
!ParseInternalKey(memtable_iter->key(), &seek_result)) {
status = Status::Corruption("DB have corrupted keys");
}
}
if (status.ok()) {
if (memtable_iter->Valid() &&
ucmp->Compare(seek_result.user_key, ranges[i].limit) <= 0) {
*overlap = true;
} else if (range_del_agg.IsRangeOverlapped(ranges[i].start,
ranges[i].limit)) {
*overlap = true;
}
}
}
return status;
}
const int ColumnFamilyData::kCompactAllLevels = -1;
const int ColumnFamilyData::kCompactToBaseLevel = -2;
+11
View File
@@ -244,6 +244,7 @@ class ColumnFamilyData {
void SetCurrent(Version* _current);
uint64_t GetNumLiveVersions() const; // REQUIRE: DB mutex held
uint64_t GetTotalSstFilesSize() const; // REQUIRE: DB mutex held
uint64_t GetLiveSstFilesSize() const; // REQUIRE: DB mutex held
void SetMemtable(MemTable* new_mem) {
uint64_t memtable_id = last_memtable_id_.fetch_add(1) + 1;
new_mem->SetID(memtable_id);
@@ -274,6 +275,16 @@ class ColumnFamilyData {
const Slice& largest_user_key,
int level) const;
// Check if the passed ranges overlap with any unflushed memtables
// (immutable or mutable).
//
// @param super_version A referenced SuperVersion that will be held for the
// duration of this function.
//
// Thread-safe
Status RangesOverlapWithMemtables(const autovector<Range>& ranges,
SuperVersion* super_version, bool* overlap);
// A flag to tell a manual compaction is to compact all levels together
// instead of a specific level.
static const int kCompactAllLevels;
+5 -4
View File
@@ -1168,13 +1168,14 @@ TEST_F(ColumnFamilyTest, MemtableNotSupportSnapshot) {
#endif // !ROCKSDB_LITE
class TestComparator : public Comparator {
int Compare(const rocksdb::Slice& a, const rocksdb::Slice& b) const override {
int Compare(const rocksdb::Slice& /*a*/,
const rocksdb::Slice& /*b*/) const override {
return 0;
}
const char* Name() const override { return "Test"; }
void FindShortestSeparator(std::string* start,
const rocksdb::Slice& limit) const override {}
void FindShortSuccessor(std::string* key) const override {}
void FindShortestSeparator(std::string* /*start*/,
const rocksdb::Slice& /*limit*/) const override {}
void FindShortSuccessor(std::string* /*key*/) const override {}
};
static TestComparator third_comparator;
+4 -5
View File
@@ -37,8 +37,7 @@ class FlushedFileCollector : public EventListener {
FlushedFileCollector() {}
~FlushedFileCollector() {}
virtual void OnFlushCompleted(
DB* db, const FlushJobInfo& info) override {
virtual void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
std::lock_guard<std::mutex> lock(mutex_);
flushed_files_.push_back(info.file_path);
}
@@ -257,9 +256,9 @@ TEST_F(CompactFilesTest, CapturingPendingFiles) {
TEST_F(CompactFilesTest, CompactionFilterWithGetSv) {
class FilterWithGet : public CompactionFilter {
public:
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value,
bool* value_changed) const override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
if (db_ == nullptr) {
return true;
}
+23 -22
View File
@@ -32,55 +32,56 @@ class CompactedDBImpl : public DBImpl {
override;
using DBImpl::Put;
virtual Status Put(const WriteOptions& options,
ColumnFamilyHandle* column_family, const Slice& key,
const Slice& value) override {
virtual Status Put(const WriteOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice& /*key*/, const Slice& /*value*/) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
using DBImpl::Merge;
virtual Status Merge(const WriteOptions& options,
ColumnFamilyHandle* column_family, const Slice& key,
const Slice& value) override {
virtual Status Merge(const WriteOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice& /*key*/, const Slice& /*value*/) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
using DBImpl::Delete;
virtual Status Delete(const WriteOptions& options,
ColumnFamilyHandle* column_family,
const Slice& key) override {
virtual Status Delete(const WriteOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice& /*key*/) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
virtual Status Write(const WriteOptions& options,
WriteBatch* updates) override {
virtual Status Write(const WriteOptions& /*options*/,
WriteBatch* /*updates*/) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
using DBImpl::CompactRange;
virtual Status CompactRange(const CompactRangeOptions& options,
ColumnFamilyHandle* column_family,
const Slice* begin, const Slice* end) override {
virtual Status CompactRange(const CompactRangeOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice* /*begin*/,
const Slice* /*end*/) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
virtual Status DisableFileDeletions() override {
return Status::NotSupported("Not supported in compacted db mode.");
}
virtual Status EnableFileDeletions(bool force) override {
virtual Status EnableFileDeletions(bool /*force*/) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
virtual Status GetLiveFiles(std::vector<std::string>&,
uint64_t* manifest_file_size,
bool flush_memtable = true) override {
uint64_t* /*manifest_file_size*/,
bool /*flush_memtable*/ = true) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
using DBImpl::Flush;
virtual Status Flush(const FlushOptions& options,
ColumnFamilyHandle* column_family) override {
virtual Status Flush(const FlushOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
using DB::IngestExternalFile;
virtual Status IngestExternalFile(
ColumnFamilyHandle* column_family,
const std::vector<std::string>& external_files,
const IngestExternalFileOptions& ingestion_options) override {
ColumnFamilyHandle* /*column_family*/,
const std::vector<std::string>& /*external_files*/,
const IngestExternalFileOptions& /*ingestion_options*/) override {
return Status::NotSupported("Not supported in compacted db mode.");
}
+3 -51
View File
@@ -12,31 +12,6 @@
namespace rocksdb {
#ifndef ROCKSDB_LITE
CompactionEventListener::CompactionListenerValueType fromInternalValueType(
ValueType vt) {
switch (vt) {
case kTypeDeletion:
return CompactionEventListener::CompactionListenerValueType::kDelete;
case kTypeValue:
return CompactionEventListener::CompactionListenerValueType::kValue;
case kTypeMerge:
return CompactionEventListener::CompactionListenerValueType::
kMergeOperand;
case kTypeSingleDeletion:
return CompactionEventListener::CompactionListenerValueType::
kSingleDelete;
case kTypeRangeDeletion:
return CompactionEventListener::CompactionListenerValueType::kRangeDelete;
case kTypeBlobIndex:
return CompactionEventListener::CompactionListenerValueType::kBlobIndex;
default:
assert(false);
return CompactionEventListener::CompactionListenerValueType::kInvalid;
}
}
#endif // ROCKSDB_LITE
CompactionIterator::CompactionIterator(
InternalIterator* input, const Comparator* cmp, MergeHelper* merge_helper,
SequenceNumber last_sequence, std::vector<SequenceNumber>* snapshots,
@@ -44,7 +19,6 @@ CompactionIterator::CompactionIterator(
const SnapshotChecker* snapshot_checker, Env* env,
bool expect_valid_internal_key, RangeDelAggregator* range_del_agg,
const Compaction* compaction, const CompactionFilter* compaction_filter,
CompactionEventListener* compaction_listener,
const std::atomic<bool>* shutting_down,
const SequenceNumber preserve_deletes_seqnum)
: CompactionIterator(
@@ -53,21 +27,18 @@ CompactionIterator::CompactionIterator(
expect_valid_internal_key, range_del_agg,
std::unique_ptr<CompactionProxy>(
compaction ? new CompactionProxy(compaction) : nullptr),
compaction_filter, compaction_listener, shutting_down,
preserve_deletes_seqnum) {}
compaction_filter, shutting_down, preserve_deletes_seqnum) {}
CompactionIterator::CompactionIterator(
InternalIterator* input, const Comparator* cmp, MergeHelper* merge_helper,
SequenceNumber last_sequence, std::vector<SequenceNumber>* snapshots,
SequenceNumber /*last_sequence*/, std::vector<SequenceNumber>* snapshots,
SequenceNumber earliest_write_conflict_snapshot,
const SnapshotChecker* snapshot_checker, Env* env,
bool expect_valid_internal_key, RangeDelAggregator* range_del_agg,
std::unique_ptr<CompactionProxy> compaction,
const CompactionFilter* compaction_filter,
CompactionEventListener* compaction_listener,
const std::atomic<bool>* shutting_down,
const SequenceNumber preserve_deletes_seqnum
)
const SequenceNumber preserve_deletes_seqnum)
: input_(input),
cmp_(cmp),
merge_helper_(merge_helper),
@@ -79,9 +50,6 @@ CompactionIterator::CompactionIterator(
range_del_agg_(range_del_agg),
compaction_(std::move(compaction)),
compaction_filter_(compaction_filter),
#ifndef ROCKSDB_LITE
compaction_listener_(compaction_listener),
#endif // ROCKSDB_LITE
shutting_down_(shutting_down),
preserve_deletes_seqnum_(preserve_deletes_seqnum),
ignore_snapshots_(false),
@@ -293,28 +261,12 @@ void CompactionIterator::NextFromInput() {
(snapshot_checker_ == nullptr ||
snapshot_checker_->IsInSnapshot(ikey_.sequence, kMaxSequenceNumber));
#ifndef ROCKSDB_LITE
if (compaction_listener_) {
compaction_listener_->OnCompaction(compaction_->level(), ikey_.user_key,
fromInternalValueType(ikey_.type),
value_, ikey_.sequence, true);
}
#endif // !ROCKSDB_LITE
// Apply the compaction filter to the first committed version of the user
// key.
if (current_key_committed_) {
InvokeFilterIfNeeded(&need_skip, &skip_until);
}
} else {
#ifndef ROCKSDB_LITE
if (compaction_listener_) {
compaction_listener_->OnCompaction(compaction_->level(), ikey_.user_key,
fromInternalValueType(ikey_.type),
value_, ikey_.sequence, false);
}
#endif // ROCKSDB_LITE
// Update the current key to reflect the new sequence number/type without
// copying the user key.
// TODO(rven): Compaction filter does not process keys in this path
+1 -8
View File
@@ -20,8 +20,6 @@
namespace rocksdb {
class CompactionEventListener;
class CompactionIterator {
public:
// A wrapper around Compaction. Has a much smaller interface, only what
@@ -32,7 +30,7 @@ class CompactionIterator {
: compaction_(compaction) {}
virtual ~CompactionProxy() = default;
virtual int level(size_t compaction_input_level = 0) const {
virtual int level(size_t /*compaction_input_level*/ = 0) const {
return compaction_->level();
}
virtual bool KeyNotExistsBeyondOutputLevel(
@@ -69,7 +67,6 @@ class CompactionIterator {
RangeDelAggregator* range_del_agg,
const Compaction* compaction = nullptr,
const CompactionFilter* compaction_filter = nullptr,
CompactionEventListener* compaction_listener = nullptr,
const std::atomic<bool>* shutting_down = nullptr,
const SequenceNumber preserve_deletes_seqnum = 0);
@@ -83,7 +80,6 @@ class CompactionIterator {
RangeDelAggregator* range_del_agg,
std::unique_ptr<CompactionProxy> compaction,
const CompactionFilter* compaction_filter = nullptr,
CompactionEventListener* compaction_listener = nullptr,
const std::atomic<bool>* shutting_down = nullptr,
const SequenceNumber preserve_deletes_seqnum = 0);
@@ -147,9 +143,6 @@ class CompactionIterator {
RangeDelAggregator* range_del_agg_;
std::unique_ptr<CompactionProxy> compaction_;
const CompactionFilter* compaction_filter_;
#ifndef ROCKSDB_LITE
CompactionEventListener* compaction_listener_;
#endif // !ROCKSDB_LITE
const std::atomic<bool>* shutting_down_;
const SequenceNumber preserve_deletes_seqnum_;
bool bottommost_level_;
+19 -16
View File
@@ -19,15 +19,15 @@ namespace rocksdb {
// Expects no merging attempts.
class NoMergingMergeOp : public MergeOperator {
public:
bool FullMergeV2(const MergeOperationInput& merge_in,
MergeOperationOutput* merge_out) const override {
bool FullMergeV2(const MergeOperationInput& /*merge_in*/,
MergeOperationOutput* /*merge_out*/) const override {
ADD_FAILURE();
return false;
}
bool PartialMergeMulti(const Slice& key,
const std::deque<Slice>& operand_list,
std::string* new_value,
Logger* logger) const override {
bool PartialMergeMulti(const Slice& /*key*/,
const std::deque<Slice>& /*operand_list*/,
std::string* /*new_value*/,
Logger* /*logger*/) const override {
ADD_FAILURE();
return false;
}
@@ -126,7 +126,7 @@ class LoggingForwardVectorIterator : public InternalIterator {
keys_.begin();
}
virtual void SeekForPrev(const Slice& target) override { assert(false); }
virtual void SeekForPrev(const Slice& /*target*/) override { assert(false); }
virtual void Next() override {
assert(Valid());
@@ -158,9 +158,12 @@ class FakeCompaction : public CompactionIterator::CompactionProxy {
public:
FakeCompaction() = default;
virtual int level(size_t compaction_input_level) const override { return 0; }
virtual int level(size_t /*compaction_input_level*/) const override {
return 0;
}
virtual bool KeyNotExistsBeyondOutputLevel(
const Slice& user_key, std::vector<size_t>* level_ptrs) const override {
const Slice& /*user_key*/,
std::vector<size_t>* /*level_ptrs*/) const override {
return is_bottommost_level || key_not_exists_beyond_output_level;
}
virtual bool bottommost_level() const override { return is_bottommost_level; }
@@ -245,7 +248,7 @@ class CompactionIteratorTest : public testing::TestWithParam<bool> {
iter_.get(), cmp_, merge_helper_.get(), last_sequence, &snapshots_,
earliest_write_conflict_snapshot, snapshot_checker_.get(),
Env::Default(), false, range_del_agg_.get(), std::move(compaction),
filter, nullptr, &shutting_down_));
filter, &shutting_down_));
}
void AddSnapshot(SequenceNumber snapshot,
@@ -365,9 +368,9 @@ TEST_P(CompactionIteratorTest, RangeDeletionWithSnapshots) {
TEST_P(CompactionIteratorTest, CompactionFilterSkipUntil) {
class Filter : public CompactionFilter {
virtual Decision FilterV2(int level, const Slice& key, ValueType t,
virtual Decision FilterV2(int /*level*/, const Slice& key, ValueType t,
const Slice& existing_value,
std::string* new_value,
std::string* /*new_value*/,
std::string* skip_until) const override {
std::string k = key.ToString();
std::string v = existing_value.ToString();
@@ -548,10 +551,10 @@ TEST_P(CompactionIteratorTest, ShuttingDownInMerge) {
TEST_P(CompactionIteratorTest, SingleMergeOperand) {
class Filter : public CompactionFilter {
virtual Decision FilterV2(int level, const Slice& key, ValueType t,
virtual Decision FilterV2(int /*level*/, const Slice& key, ValueType t,
const Slice& existing_value,
std::string* new_value,
std::string* skip_until) const override {
std::string* /*new_value*/,
std::string* /*skip_until*/) const override {
std::string k = key.ToString();
std::string v = existing_value.ToString();
@@ -602,7 +605,7 @@ TEST_P(CompactionIteratorTest, SingleMergeOperand) {
bool PartialMergeMulti(const Slice& key,
const std::deque<Slice>& operand_list,
std::string* new_value,
Logger* logger) const override {
Logger* /*logger*/) const override {
std::string string_key = key.ToString();
EXPECT_TRUE(string_key == "a" || string_key == "b");
+2 -13
View File
@@ -762,24 +762,13 @@ void CompactionJob::ProcessKeyValueCompaction(SubcompactionState* sub_compact) {
input->SeekToFirst();
}
// we allow only 1 compaction event listener. Used by blob storage
CompactionEventListener* comp_event_listener = nullptr;
#ifndef ROCKSDB_LITE
for (auto& celitr : cfd->ioptions()->listeners) {
comp_event_listener = celitr->GetCompactionEventListener();
if (comp_event_listener != nullptr) {
break;
}
}
#endif // ROCKSDB_LITE
Status status;
sub_compact->c_iter.reset(new CompactionIterator(
input.get(), cfd->user_comparator(), &merge, versions_->LastSequence(),
&existing_snapshots_, earliest_write_conflict_snapshot_,
snapshot_checker_, env_, false, range_del_agg.get(),
sub_compact->compaction, compaction_filter, comp_event_listener,
shutting_down_, preserve_deletes_seqnum_));
sub_compact->compaction, compaction_filter, shutting_down_,
preserve_deletes_seqnum_));
auto c_iter = sub_compact->c_iter.get();
c_iter->SeekToFirst();
if (c_iter->Valid() &&
+1 -1
View File
@@ -426,7 +426,7 @@ class CompactionJobStatsChecker : public EventListener {
// Once a compaction completed, this function will verify the returned
// CompactionJobInfo with the oldest CompactionJobInfo added earlier
// in "expected_stats_" which has not yet being used for verification.
virtual void OnCompactionCompleted(DB *db, const CompactionJobInfo& ci) {
virtual void OnCompactionCompleted(DB* /*db*/, const CompactionJobInfo& ci) {
if (verify_next_comp_io_stats_) {
ASSERT_GT(ci.stats.file_write_nanos, 0);
ASSERT_GT(ci.stats.file_range_sync_nanos, 0);
+5 -4
View File
@@ -199,7 +199,7 @@ void CompactionPicker::GetRange(const std::vector<CompactionInputFiles>& inputs,
assert(initialized);
}
bool CompactionPicker::ExpandInputsToCleanCut(const std::string& cf_name,
bool CompactionPicker::ExpandInputsToCleanCut(const std::string& /*cf_name*/,
VersionStorageInfo* vstorage,
CompactionInputFiles* inputs) {
// This isn't good compaction
@@ -309,7 +309,7 @@ Compaction* CompactionPicker::CompactFiles(
Status CompactionPicker::GetCompactionInputsFromFileNumbers(
std::vector<CompactionInputFiles>* input_files,
std::unordered_set<uint64_t>* input_set, const VersionStorageInfo* vstorage,
const CompactionOptions& compact_options) const {
const CompactionOptions& /*compact_options*/) const {
if (input_set->size() == 0U) {
return Status::InvalidArgument(
"Compaction must include at least one file.");
@@ -1612,8 +1612,9 @@ Compaction* FIFOCompactionPicker::PickCompaction(
Compaction* FIFOCompactionPicker::CompactRange(
const std::string& cf_name, const MutableCFOptions& mutable_cf_options,
VersionStorageInfo* vstorage, int input_level, int output_level,
uint32_t output_path_id, const InternalKey* begin, const InternalKey* end,
InternalKey** compaction_end, bool* manual_conflict) {
uint32_t /*output_path_id*/, const InternalKey* /*begin*/,
const InternalKey* /*end*/, InternalKey** compaction_end,
bool* /*manual_conflict*/) {
assert(input_level == 0);
assert(output_level == 0);
*compaction_end = nullptr;
+14 -12
View File
@@ -267,27 +267,29 @@ class NullCompactionPicker : public CompactionPicker {
virtual ~NullCompactionPicker() {}
// Always return "nullptr"
Compaction* PickCompaction(const std::string& cf_name,
const MutableCFOptions& mutable_cf_options,
VersionStorageInfo* vstorage,
LogBuffer* log_buffer) override {
Compaction* PickCompaction(const std::string& /*cf_name*/,
const MutableCFOptions& /*mutable_cf_options*/,
VersionStorageInfo* /*vstorage*/,
LogBuffer* /*log_buffer*/) override {
return nullptr;
}
// Always return "nullptr"
Compaction* CompactRange(const std::string& cf_name,
const MutableCFOptions& mutable_cf_options,
VersionStorageInfo* vstorage, int input_level,
int output_level, uint32_t output_path_id,
const InternalKey* begin, const InternalKey* end,
InternalKey** compaction_end,
bool* manual_conflict) override {
Compaction* CompactRange(const std::string& /*cf_name*/,
const MutableCFOptions& /*mutable_cf_options*/,
VersionStorageInfo* /*vstorage*/,
int /*input_level*/, int /*output_level*/,
uint32_t /*output_path_id*/,
const InternalKey* /*begin*/,
const InternalKey* /*end*/,
InternalKey** /*compaction_end*/,
bool* /*manual_conflict*/) override {
return nullptr;
}
// Always returns false.
virtual bool NeedsCompaction(
const VersionStorageInfo* vstorage) const override {
const VersionStorageInfo* /*vstorage*/) const override {
return false;
}
};
+3 -1
View File
@@ -20,7 +20,9 @@ namespace rocksdb {
class CountingLogger : public Logger {
public:
using Logger::Logv;
virtual void Logv(const char* format, va_list ap) override { log_count++; }
virtual void Logv(const char* /*format*/, va_list /*ap*/) override {
log_count++;
}
size_t log_count;
};
+1 -1
View File
@@ -204,7 +204,7 @@ void UniversalCompactionPicker::SortedRun::DumpSizeInfo(
std::vector<UniversalCompactionPicker::SortedRun>
UniversalCompactionPicker::CalculateSortedRuns(
const VersionStorageInfo& vstorage, const ImmutableCFOptions& ioptions,
const VersionStorageInfo& vstorage, const ImmutableCFOptions& /*ioptions*/,
const MutableCFOptions& mutable_cf_options) {
std::vector<UniversalCompactionPicker::SortedRun> ret;
for (FileMetaData* f : vstorage.LevelFiles(0)) {
+9 -9
View File
@@ -188,10 +188,10 @@ class DoubleComparator : public Comparator {
return -1;
}
}
virtual void FindShortestSeparator(std::string* start,
const Slice& limit) const override {}
virtual void FindShortestSeparator(std::string* /*start*/,
const Slice& /*limit*/) const override {}
virtual void FindShortSuccessor(std::string* key) const override {}
virtual void FindShortSuccessor(std::string* /*key*/) const override {}
};
class HashComparator : public Comparator {
@@ -211,10 +211,10 @@ class HashComparator : public Comparator {
return -1;
}
}
virtual void FindShortestSeparator(std::string* start,
const Slice& limit) const override {}
virtual void FindShortestSeparator(std::string* /*start*/,
const Slice& /*limit*/) const override {}
virtual void FindShortSuccessor(std::string* key) const override {}
virtual void FindShortSuccessor(std::string* /*key*/) const override {}
};
class TwoStrComparator : public Comparator {
@@ -243,10 +243,10 @@ class TwoStrComparator : public Comparator {
}
return a2.compare(b2);
}
virtual void FindShortestSeparator(std::string* start,
const Slice& limit) const override {}
virtual void FindShortestSeparator(std::string* /*start*/,
const Slice& /*limit*/) const override {}
virtual void FindShortSuccessor(std::string* key) const override {}
virtual void FindShortSuccessor(std::string* /*key*/) const override {}
};
} // namespace
+4 -3
View File
@@ -862,8 +862,9 @@ class TestEnv : public EnvWrapper {
CloseHelper();
}
}
virtual void Logv(const char *format, va_list ap) override { };
protected:
virtual void Logv(const char* /*format*/, va_list /*ap*/) override{};
protected:
virtual Status CloseImpl() override {
return CloseHelper();
}
@@ -879,7 +880,7 @@ class TestEnv : public EnvWrapper {
int GetCloseCount() { return close_count; }
virtual Status NewLogger(const std::string& fname,
virtual Status NewLogger(const std::string& /*fname*/,
shared_ptr<Logger>* result) {
result->reset(new TestLogger(this));
return Status::OK();
+1 -1
View File
@@ -47,7 +47,7 @@ class DBBlockCacheTest : public DBTestBase {
return options;
}
void InitTable(const Options& options) {
void InitTable(const Options& /*options*/) {
std::string value(kValueSize, 'a');
for (size_t i = 0; i < kNumBlocks; i++) {
ASSERT_OK(Put(ToString(i), value.c_str()));
+25 -23
View File
@@ -56,9 +56,9 @@ INSTANTIATE_TEST_CASE_P(
class KeepFilter : public CompactionFilter {
public:
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value, bool* value_changed) const
override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
cfilter_count++;
return false;
}
@@ -68,9 +68,9 @@ class KeepFilter : public CompactionFilter {
class DeleteFilter : public CompactionFilter {
public:
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value, bool* value_changed) const
override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
cfilter_count++;
return true;
}
@@ -80,9 +80,9 @@ class DeleteFilter : public CompactionFilter {
class DeleteISFilter : public CompactionFilter {
public:
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value,
bool* value_changed) const override {
virtual bool Filter(int /*level*/, const Slice& key, const Slice& /*value*/,
std::string* /*new_value*/,
bool* /*value_changed*/) const override {
cfilter_count++;
int i = std::stoi(key.ToString());
if (i > 5 && i <= 105) {
@@ -100,8 +100,10 @@ class DeleteISFilter : public CompactionFilter {
// zero-padded to length 10.
class SkipEvenFilter : public CompactionFilter {
public:
virtual Decision FilterV2(int level, const Slice& key, ValueType value_type,
const Slice& existing_value, std::string* new_value,
virtual Decision FilterV2(int /*level*/, const Slice& key,
ValueType /*value_type*/,
const Slice& /*existing_value*/,
std::string* /*new_value*/,
std::string* skip_until) const override {
cfilter_count++;
int i = std::stoi(key.ToString());
@@ -123,9 +125,9 @@ class SkipEvenFilter : public CompactionFilter {
class DelayFilter : public CompactionFilter {
public:
explicit DelayFilter(DBTestBase* d) : db_test(d) {}
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value,
bool* value_changed) const override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
db_test->env_->addon_time_.fetch_add(1000);
return true;
}
@@ -140,9 +142,9 @@ class ConditionalFilter : public CompactionFilter {
public:
explicit ConditionalFilter(const std::string* filtered_value)
: filtered_value_(filtered_value) {}
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value,
bool* value_changed) const override {
virtual bool Filter(int /*level*/, const Slice& /*key*/, const Slice& value,
std::string* /*new_value*/,
bool* /*value_changed*/) const override {
return value.ToString() == *filtered_value_;
}
@@ -156,9 +158,9 @@ class ChangeFilter : public CompactionFilter {
public:
explicit ChangeFilter() {}
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value, bool* value_changed) const
override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* new_value,
bool* value_changed) const override {
assert(new_value != nullptr);
*new_value = NEW_VALUE;
*value_changed = true;
@@ -247,7 +249,7 @@ class DelayFilterFactory : public CompactionFilterFactory {
public:
explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {}
virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
const CompactionFilter::Context& context) override {
const CompactionFilter::Context& /*context*/) override {
return std::unique_ptr<CompactionFilter>(new DelayFilter(db_test));
}
@@ -263,7 +265,7 @@ class ConditionalFilterFactory : public CompactionFilterFactory {
: filtered_value_(filtered_value.ToString()) {}
virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
const CompactionFilter::Context& context) override {
const CompactionFilter::Context& /*context*/) override {
return std::unique_ptr<CompactionFilter>(
new ConditionalFilter(&filtered_value_));
}
@@ -281,7 +283,7 @@ class ChangeFilterFactory : public CompactionFilterFactory {
explicit ChangeFilterFactory() {}
virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
const CompactionFilter::Context& context) override {
const CompactionFilter::Context& /*context*/) override {
return std::unique_ptr<CompactionFilter>(new ChangeFilter());
}
+59 -1
View File
@@ -53,7 +53,7 @@ class FlushedFileCollector : public EventListener {
FlushedFileCollector() {}
~FlushedFileCollector() {}
virtual void OnFlushCompleted(DB* db, const FlushJobInfo& info) override {
virtual void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
std::lock_guard<std::mutex> lock(mutex_);
flushed_files_.push_back(info.file_path);
}
@@ -3317,6 +3317,64 @@ TEST_F(DBCompactionTest, CompactRangeSkipFlushAfterDelay) {
rocksdb::SyncPoint::GetInstance()->DisableProcessing();
}
TEST_F(DBCompactionTest, CompactRangeFlushOverlappingMemtable) {
// Verify memtable only gets flushed if it contains data overlapping the range
// provided to `CompactRange`. Tests all kinds of overlap/non-overlap.
const int kNumEndpointKeys = 5;
std::string keys[kNumEndpointKeys] = {"a", "b", "c", "d", "e"};
Options options = CurrentOptions();
options.disable_auto_compactions = true;
Reopen(options);
// One extra iteration for nullptr, which means left side of interval is
// unbounded.
for (int i = 0; i <= kNumEndpointKeys; ++i) {
Slice begin;
Slice* begin_ptr;
if (i == 0) {
begin_ptr = nullptr;
} else {
begin = keys[i - 1];
begin_ptr = &begin;
}
// Start at `i` so right endpoint comes after left endpoint. One extra
// iteration for nullptr, which means right side of interval is unbounded.
for (int j = std::max(0, i - 1); j <= kNumEndpointKeys; ++j) {
Slice end;
Slice* end_ptr;
if (j == kNumEndpointKeys) {
end_ptr = nullptr;
} else {
end = keys[j];
end_ptr = &end;
}
ASSERT_OK(Put("b", "val"));
ASSERT_OK(Put("d", "val"));
CompactRangeOptions compact_range_opts;
ASSERT_OK(db_->CompactRange(compact_range_opts, begin_ptr, end_ptr));
uint64_t get_prop_tmp, num_memtable_entries = 0;
ASSERT_TRUE(db_->GetIntProperty(DB::Properties::kNumEntriesImmMemTables,
&get_prop_tmp));
num_memtable_entries += get_prop_tmp;
ASSERT_TRUE(db_->GetIntProperty(DB::Properties::kNumEntriesActiveMemTable,
&get_prop_tmp));
num_memtable_entries += get_prop_tmp;
if (begin_ptr == nullptr || end_ptr == nullptr ||
(i <= 4 && j >= 1 && (begin != "c" || end != "c"))) {
// In this case `CompactRange`'s range overlapped in some way with the
// memtable's range, so flush should've happened. Then "b" and "d" won't
// be in the memtable.
ASSERT_EQ(0, num_memtable_entries);
} else {
ASSERT_EQ(2, num_memtable_entries);
// flush anyways to prepare for next iteration
db_->Flush(FlushOptions());
}
}
}
}
INSTANTIATE_TEST_CASE_P(DBCompactionTestWithParam, DBCompactionTestWithParam,
::testing::Values(std::make_tuple(1, true),
std::make_tuple(1, false),
+15 -14
View File
@@ -675,8 +675,9 @@ Status DBImpl::SetDBOptions(
}
// return the same level if it cannot be moved
int DBImpl::FindMinimumEmptyLevelFitting(ColumnFamilyData* cfd,
const MutableCFOptions& mutable_cf_options, int level) {
int DBImpl::FindMinimumEmptyLevelFitting(
ColumnFamilyData* cfd, const MutableCFOptions& /*mutable_cf_options*/,
int level) {
mutex_.AssertHeld();
const auto* vstorage = cfd->current()->storage_info();
int minimum_level = level;
@@ -897,7 +898,7 @@ struct IterState {
bool background_purge;
};
static void CleanupIteratorState(void* arg1, void* arg2) {
static void CleanupIteratorState(void* arg1, void* /*arg2*/) {
IterState* state = reinterpret_cast<IterState*>(arg1);
if (state->super_version->Unref()) {
@@ -2335,31 +2336,31 @@ Status DBImpl::GetDbIdentity(std::string& identity) const {
}
// Default implementation -- returns not supported status
Status DB::CreateColumnFamily(const ColumnFamilyOptions& cf_options,
const std::string& column_family_name,
ColumnFamilyHandle** handle) {
Status DB::CreateColumnFamily(const ColumnFamilyOptions& /*cf_options*/,
const std::string& /*column_family_name*/,
ColumnFamilyHandle** /*handle*/) {
return Status::NotSupported("");
}
Status DB::CreateColumnFamilies(
const ColumnFamilyOptions& cf_options,
const std::vector<std::string>& column_family_names,
std::vector<ColumnFamilyHandle*>* handles) {
const ColumnFamilyOptions& /*cf_options*/,
const std::vector<std::string>& /*column_family_names*/,
std::vector<ColumnFamilyHandle*>* /*handles*/) {
return Status::NotSupported("");
}
Status DB::CreateColumnFamilies(
const std::vector<ColumnFamilyDescriptor>& column_families,
std::vector<ColumnFamilyHandle*>* handles) {
const std::vector<ColumnFamilyDescriptor>& /*column_families*/,
std::vector<ColumnFamilyHandle*>* /*handles*/) {
return Status::NotSupported("");
}
Status DB::DropColumnFamily(ColumnFamilyHandle* column_family) {
Status DB::DropColumnFamily(ColumnFamilyHandle* /*column_family*/) {
return Status::NotSupported("");
}
Status DB::DropColumnFamilies(
const std::vector<ColumnFamilyHandle*>& column_families) {
const std::vector<ColumnFamilyHandle*>& /*column_families*/) {
return Status::NotSupported("");
}
@@ -2832,7 +2833,7 @@ Status DBImpl::IngestExternalFile(
// Figure out if we need to flush the memtable first
if (status.ok()) {
bool need_flush = false;
status = ingestion_job.NeedsFlush(&need_flush);
status = ingestion_job.NeedsFlush(&need_flush, cfd->GetSuperVersion());
TEST_SYNC_POINT_CALLBACK("DBImpl::IngestExternalFile:NeedFlush",
&need_flush);
if (status.ok() && need_flush) {
+24 -17
View File
@@ -14,7 +14,6 @@
#include <limits>
#include <list>
#include <map>
#include <queue>
#include <set>
#include <string>
#include <utility>
@@ -433,6 +432,8 @@ class DBImpl : public DB {
uint64_t TEST_FindMinLogContainingOutstandingPrep();
uint64_t TEST_FindMinPrepLogReferencedByMemTable();
size_t TEST_PreparedSectionCompletedSize();
size_t TEST_LogsWithPrepSize();
int TEST_BGCompactionsAllowed() const;
int TEST_BGFlushesAllowed() const;
@@ -1298,27 +1299,33 @@ class DBImpl : public DB {
// Indicate DB was opened successfully
bool opened_successfully_;
// minimum log number still containing prepared data.
// REQUIRES: logs_with_prep_mutex_ held
//
// sorted list of log numbers still containing prepared data.
// this is used by FindObsoleteFiles to determine which
// flushed logs we must keep around because they still
// contain prepared data which has not been flushed or rolled back
std::priority_queue<uint64_t, std::vector<uint64_t>, std::greater<uint64_t>>
min_log_with_prep_;
// contain prepared data which has not been committed or rolled back
struct LogCnt {
uint64_t log; // the log number
uint64_t cnt; // number of prepared sections in the log
};
std::vector<LogCnt> logs_with_prep_;
std::mutex logs_with_prep_mutex_;
// to be used in conjunction with min_log_with_prep_.
// REQUIRES: prepared_section_completed_mutex_ held
//
// to be used in conjunction with logs_with_prep_.
// once a transaction with data in log L is committed or rolled back
// rather than removing the value from the heap we add that value
// to prepared_section_completed_ which maps LOG -> instance_count
// since a log could contain multiple prepared sections
// rather than updating logs_with_prep_ directly we keep track of that
// in prepared_section_completed_ which maps LOG -> instance_count. This helps
// avoiding contention between a commit thread and the prepare threads.
//
// when trying to determine the minimum log still active we first
// consult min_log_with_prep_. while that root value maps to
// a value > 0 in prepared_section_completed_ we decrement the
// instance_count for that log and pop the root value in
// min_log_with_prep_. This will work the same as a min_heap
// where we are deleteing arbitrary elements and the up heaping.
// consult logs_with_prep_. while that root value maps to
// an equal value in prepared_section_completed_ we erase the log from
// both logs_with_prep_ and prepared_section_completed_.
std::unordered_map<uint64_t, uint64_t> prepared_section_completed_;
std::mutex prep_heap_mutex_;
std::mutex prepared_section_completed_mutex_;
// Callback for compaction to check if a key is visible to a snapshot.
// REQUIRES: mutex held
@@ -1367,8 +1374,8 @@ class DBImpl : public DB {
return Env::WLTH_SHORT;
}
// When set, we use a seprate queue for writes that dont write to memtable. In
// 2PC these are the writes at Prepare phase.
// When set, we use a separate queue for writes that dont write to memtable.
// In 2PC these are the writes at Prepare phase.
const bool two_write_queues_;
const bool manual_wal_flush_;
// Increase the sequence number after writing each batch, whether memtable is
+12 -2
View File
@@ -291,7 +291,17 @@ Status DBImpl::CompactRange(const CompactRangeOptions& options,
bool exclusive = options.exclusive_manual_compaction;
bool flush_needed = true;
if (!options.allow_write_stall) {
if (begin != nullptr && end != nullptr) {
// TODO(ajkr): We could also optimize away the flush in certain cases where
// one/both sides of the interval are unbounded. But it requires more
// changes to RangesOverlapWithMemtables.
Range range(*begin, *end);
SuperVersion* super_version = cfd->GetReferencedSuperVersion(&mutex_);
cfd->RangesOverlapWithMemtables({range}, super_version, &flush_needed);
CleanupSuperVersion(super_version);
}
if (!options.allow_write_stall && flush_needed) {
InstrumentedMutexLock l(&mutex_);
uint64_t orig_active_memtable_id = cfd->mem()->GetID();
WriteStallCondition write_stall_condition = WriteStallCondition::kNormal;
@@ -854,7 +864,7 @@ int DBImpl::NumberLevels(ColumnFamilyHandle* column_family) {
return cfh->cfd()->NumberLevels();
}
int DBImpl::MaxMemCompactionLevel(ColumnFamilyHandle* column_family) {
int DBImpl::MaxMemCompactionLevel(ColumnFamilyHandle* /*column_family*/) {
return 0;
}
+6
View File
@@ -186,6 +186,12 @@ uint64_t DBImpl::TEST_FindMinLogContainingOutstandingPrep() {
return FindMinLogContainingOutstandingPrep();
}
size_t DBImpl::TEST_PreparedSectionCompletedSize() {
return prepared_section_completed_.size();
}
size_t DBImpl::TEST_LogsWithPrepSize() { return logs_with_prep_.size(); }
uint64_t DBImpl::TEST_FindMinPrepLogReferencedByMemTable() {
return FindMinPrepLogReferencedByMemTable();
}
+42 -39
View File
@@ -48,58 +48,61 @@ uint64_t DBImpl::FindMinPrepLogReferencedByMemTable() {
return min_log;
}
// TODO(myabandeh): Avoid using locks
void DBImpl::MarkLogAsHavingPrepSectionFlushed(uint64_t log) {
assert(log != 0);
std::lock_guard<std::mutex> lock(prep_heap_mutex_);
std::lock_guard<std::mutex> lock(prepared_section_completed_mutex_);
auto it = prepared_section_completed_.find(log);
assert(it != prepared_section_completed_.end());
it->second += 1;
if (UNLIKELY(it == prepared_section_completed_.end())) {
prepared_section_completed_[log] = 1;
} else {
it->second += 1;
}
}
// TODO(myabandeh): Avoid using locks
void DBImpl::MarkLogAsContainingPrepSection(uint64_t log) {
assert(log != 0);
std::lock_guard<std::mutex> lock(prep_heap_mutex_);
min_log_with_prep_.push(log);
auto it = prepared_section_completed_.find(log);
if (it == prepared_section_completed_.end()) {
prepared_section_completed_[log] = 0;
std::lock_guard<std::mutex> lock(logs_with_prep_mutex_);
auto rit = logs_with_prep_.rbegin();
bool updated = false;
// Most probabely the last log is the one that is being marked for
// having a prepare section; so search from the end.
for (; rit != logs_with_prep_.rend() && rit->log >= log; ++rit) {
if (rit->log == log) {
rit->cnt++;
updated = true;
break;
}
}
if (!updated) {
// We are either at the start, or at a position with rit->log < log
logs_with_prep_.insert(rit.base(), {log, 1});
}
}
uint64_t DBImpl::FindMinLogContainingOutstandingPrep() {
if (!allow_2pc()) {
return 0;
}
std::lock_guard<std::mutex> lock(prep_heap_mutex_);
uint64_t min_log = 0;
// first we look in the prepared heap where we keep
// track of transactions that have been prepared (written to WAL)
// but not yet committed.
while (!min_log_with_prep_.empty()) {
min_log = min_log_with_prep_.top();
auto it = prepared_section_completed_.find(min_log);
// value was marked as 'deleted' from heap
if (it != prepared_section_completed_.end() && it->second > 0) {
it->second -= 1;
min_log_with_prep_.pop();
// back to squere one...
min_log = 0;
continue;
} else {
// found a valid value
break;
std::lock_guard<std::mutex> lock(logs_with_prep_mutex_);
auto it = logs_with_prep_.begin();
// start with the smallest log
for (; it != logs_with_prep_.end();) {
auto min_log = it->log;
{
std::lock_guard<std::mutex> lock2(prepared_section_completed_mutex_);
auto completed_it = prepared_section_completed_.find(min_log);
if (completed_it == prepared_section_completed_.end() ||
completed_it->second < it->cnt) {
return min_log;
}
assert(completed_it != prepared_section_completed_.end() &&
completed_it->second == it->cnt);
prepared_section_completed_.erase(completed_it);
}
// erase from beigning in vector is not efficient but this function is not
// on the fast path.
it = logs_with_prep_.erase(it);
}
return min_log;
// no such log found
return 0;
}
uint64_t DBImpl::MinLogNumberToKeep() {
+23
View File
@@ -356,6 +356,29 @@ Status DBImpl::Recover(
assert(s.IsIOError());
return s;
}
// Verify compatibility of env_options_ and filesystem
{
unique_ptr<RandomAccessFile> idfile;
EnvOptions customized_env(env_options_);
customized_env.use_direct_reads |=
immutable_db_options_.use_direct_io_for_flush_and_compaction;
s = env_->NewRandomAccessFile(IdentityFileName(dbname_), &idfile,
customized_env);
if (!s.ok()) {
const char* error_msg = s.ToString().c_str();
// Check if unsupported Direct I/O is the root cause
customized_env.use_direct_reads = false;
s = env_->NewRandomAccessFile(IdentityFileName(dbname_), &idfile,
customized_env);
if (s.ok()) {
return Status::InvalidArgument(
"Direct I/O is not supported by the specified DB.");
} else {
return Status::InvalidArgument(
"Found options incompatible with filesystem", error_msg);
}
}
}
}
Status s = versions_->Recover(column_families, read_only);
+1 -1
View File
@@ -107,7 +107,7 @@ Status DBImplReadOnly::NewIterators(
}
Status DB::OpenForReadOnly(const Options& options, const std::string& dbname,
DB** dbptr, bool error_if_log_file_exist) {
DB** dbptr, bool /*error_if_log_file_exist*/) {
*dbptr = nullptr;
// Try to first open DB as fully compacted DB
+30 -29
View File
@@ -36,46 +36,47 @@ class DBImplReadOnly : public DBImpl {
std::vector<Iterator*>* iterators) override;
using DBImpl::Put;
virtual Status Put(const WriteOptions& options,
ColumnFamilyHandle* column_family, const Slice& key,
const Slice& value) override {
virtual Status Put(const WriteOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice& /*key*/, const Slice& /*value*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
using DBImpl::Merge;
virtual Status Merge(const WriteOptions& options,
ColumnFamilyHandle* column_family, const Slice& key,
const Slice& value) override {
virtual Status Merge(const WriteOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice& /*key*/, const Slice& /*value*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
using DBImpl::Delete;
virtual Status Delete(const WriteOptions& options,
ColumnFamilyHandle* column_family,
const Slice& key) override {
virtual Status Delete(const WriteOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice& /*key*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
using DBImpl::SingleDelete;
virtual Status SingleDelete(const WriteOptions& options,
ColumnFamilyHandle* column_family,
const Slice& key) override {
virtual Status SingleDelete(const WriteOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice& /*key*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
virtual Status Write(const WriteOptions& options,
WriteBatch* updates) override {
virtual Status Write(const WriteOptions& /*options*/,
WriteBatch* /*updates*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
using DBImpl::CompactRange;
virtual Status CompactRange(const CompactRangeOptions& options,
ColumnFamilyHandle* column_family,
const Slice* begin, const Slice* end) override {
virtual Status CompactRange(const CompactRangeOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice* /*begin*/,
const Slice* /*end*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
using DBImpl::CompactFiles;
virtual Status CompactFiles(
const CompactionOptions& compact_options,
ColumnFamilyHandle* column_family,
const std::vector<std::string>& input_file_names,
const int output_level, const int output_path_id = -1) override {
const CompactionOptions& /*compact_options*/,
ColumnFamilyHandle* /*column_family*/,
const std::vector<std::string>& /*input_file_names*/,
const int /*output_level*/, const int /*output_path_id*/ = -1) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
@@ -83,18 +84,18 @@ class DBImplReadOnly : public DBImpl {
return Status::NotSupported("Not supported operation in read only mode.");
}
virtual Status EnableFileDeletions(bool force) override {
virtual Status EnableFileDeletions(bool /*force*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
virtual Status GetLiveFiles(std::vector<std::string>&,
uint64_t* manifest_file_size,
bool flush_memtable = true) override {
uint64_t* /*manifest_file_size*/,
bool /*flush_memtable*/ = true) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
using DBImpl::Flush;
virtual Status Flush(const FlushOptions& options,
ColumnFamilyHandle* column_family) override {
virtual Status Flush(const FlushOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
@@ -105,9 +106,9 @@ class DBImplReadOnly : public DBImpl {
using DB::IngestExternalFile;
virtual Status IngestExternalFile(
ColumnFamilyHandle* column_family,
const std::vector<std::string>& external_files,
const IngestExternalFileOptions& ingestion_options) override {
ColumnFamilyHandle* /*column_family*/,
const std::vector<std::string>& /*external_files*/,
const IngestExternalFileOptions& /*ingestion_options*/) override {
return Status::NotSupported("Not supported operation in read only mode.");
}
+1 -1
View File
@@ -1175,7 +1175,7 @@ Status DBImpl::ScheduleFlushes(WriteContext* context) {
}
#ifndef ROCKSDB_LITE
void DBImpl::NotifyOnMemTableSealed(ColumnFamilyData* cfd,
void DBImpl::NotifyOnMemTableSealed(ColumnFamilyData* /*cfd*/,
const MemTableInfo& mem_table_info) {
if (immutable_db_options_.listeners.size() == 0U) {
return;
+3 -3
View File
@@ -50,7 +50,7 @@ class DBIteratorTest : public DBTestBase,
class FlushBlockEveryKeyPolicy : public FlushBlockPolicy {
public:
virtual bool Update(const Slice& key, const Slice& value) override {
virtual bool Update(const Slice& /*key*/, const Slice& /*value*/) override {
if (!start_) {
start_ = true;
return false;
@@ -70,8 +70,8 @@ class FlushBlockEveryKeyPolicyFactory : public FlushBlockPolicyFactory {
}
FlushBlockPolicy* NewFlushBlockPolicy(
const BlockBasedTableOptions& table_options,
const BlockBuilder& data_block_builder) const override {
const BlockBasedTableOptions& /*table_options*/,
const BlockBuilder& /*data_block_builder*/) const override {
return new FlushBlockEveryKeyPolicy;
}
};
+1 -1
View File
@@ -119,7 +119,7 @@ class TestPrefixExtractor : public SliceTransform {
return separator(key) != nullptr;
}
virtual bool InRange(const Slice& key) const override { return false; }
virtual bool InRange(const Slice& /*key*/) const override { return false; }
private:
const char* separator(const Slice& key) const {
+58 -5
View File
@@ -14,6 +14,7 @@
#include "db/db_test_util.h"
#include "port/stack_trace.h"
#include "rocksdb/listener.h"
#include "rocksdb/options.h"
#include "rocksdb/perf_context.h"
#include "rocksdb/perf_level.h"
@@ -985,8 +986,9 @@ class CountingUserTblPropCollector : public TablePropertiesCollector {
return Status::OK();
}
Status AddUserKey(const Slice& user_key, const Slice& value, EntryType type,
SequenceNumber seq, uint64_t file_size) override {
Status AddUserKey(const Slice& /*user_key*/, const Slice& /*value*/,
EntryType /*type*/, SequenceNumber /*seq*/,
uint64_t /*file_size*/) override {
++count_;
return Status::OK();
}
@@ -1027,8 +1029,9 @@ class CountingDeleteTabPropCollector : public TablePropertiesCollector {
public:
const char* Name() const override { return "CountingDeleteTabPropCollector"; }
Status AddUserKey(const Slice& user_key, const Slice& value, EntryType type,
SequenceNumber seq, uint64_t file_size) override {
Status AddUserKey(const Slice& /*user_key*/, const Slice& /*value*/,
EntryType type, SequenceNumber /*seq*/,
uint64_t /*file_size*/) override {
if (type == kEntryDelete) {
num_deletes_++;
}
@@ -1055,7 +1058,7 @@ class CountingDeleteTabPropCollectorFactory
: public TablePropertiesCollectorFactory {
public:
virtual TablePropertiesCollector* CreateTablePropertiesCollector(
TablePropertiesCollectorFactory::Context context) override {
TablePropertiesCollectorFactory::Context /*context*/) override {
return new CountingDeleteTabPropCollector();
}
const char* Name() const override {
@@ -1383,6 +1386,56 @@ TEST_F(DBPropertiesTest, EstimateOldestKeyTime) {
Close();
}
TEST_F(DBPropertiesTest, SstFilesSize) {
struct TestListener : public EventListener {
void OnCompactionCompleted(DB* db,
const CompactionJobInfo& /*info*/) override {
assert(callback_triggered == false);
assert(size_before_compaction > 0);
callback_triggered = true;
uint64_t total_sst_size = 0;
uint64_t live_sst_size = 0;
bool ok = db->GetIntProperty(DB::Properties::kTotalSstFilesSize,
&total_sst_size);
ASSERT_TRUE(ok);
// total_sst_size include files before and after compaction.
ASSERT_GT(total_sst_size, size_before_compaction);
ok =
db->GetIntProperty(DB::Properties::kLiveSstFilesSize, &live_sst_size);
ASSERT_TRUE(ok);
// live_sst_size only include files after compaction.
ASSERT_GT(live_sst_size, 0);
ASSERT_LT(live_sst_size, size_before_compaction);
}
uint64_t size_before_compaction = 0;
bool callback_triggered = false;
};
std::shared_ptr<TestListener> listener = std::make_shared<TestListener>();
Options options;
options.disable_auto_compactions = true;
options.listeners.push_back(listener);
Reopen(options);
for (int i = 0; i < 10; i++) {
ASSERT_OK(Put("key" + ToString(i), std::string(1000, 'v')));
}
ASSERT_OK(Flush());
for (int i = 0; i < 5; i++) {
ASSERT_OK(Delete("key" + ToString(i)));
}
ASSERT_OK(Flush());
uint64_t sst_size;
bool ok = db_->GetIntProperty(DB::Properties::kTotalSstFilesSize, &sst_size);
ASSERT_TRUE(ok);
ASSERT_GT(sst_size, 0);
listener->size_before_compaction = sst_size;
// Compact to clean all keys and trigger listener.
ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr));
ASSERT_TRUE(listener->callback_triggered);
}
#endif // ROCKSDB_LITE
} // namespace rocksdb
+81 -71
View File
@@ -750,9 +750,9 @@ TEST_F(DBTest, FlushSchedule) {
namespace {
class KeepFilter : public CompactionFilter {
public:
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value,
bool* value_changed) const override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
return false;
}
@@ -782,9 +782,9 @@ class KeepFilterFactory : public CompactionFilterFactory {
class DelayFilter : public CompactionFilter {
public:
explicit DelayFilter(DBTestBase* d) : db_test(d) {}
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value,
bool* value_changed) const override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
db_test->env_->addon_time_.fetch_add(1000);
return true;
}
@@ -799,7 +799,7 @@ class DelayFilterFactory : public CompactionFilterFactory {
public:
explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {}
virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
const CompactionFilter::Context& context) override {
const CompactionFilter::Context& /*context*/) override {
return std::unique_ptr<CompactionFilter>(new DelayFilter(db_test));
}
@@ -2247,17 +2247,17 @@ class ModelDB : public DB {
return Write(o, &batch);
}
using DB::Get;
virtual Status Get(const ReadOptions& options, ColumnFamilyHandle* cf,
const Slice& key, PinnableSlice* value) override {
virtual Status Get(const ReadOptions& /*options*/, ColumnFamilyHandle* /*cf*/,
const Slice& key, PinnableSlice* /*value*/) override {
return Status::NotSupported(key);
}
using DB::MultiGet;
virtual std::vector<Status> MultiGet(
const ReadOptions& options,
const std::vector<ColumnFamilyHandle*>& column_family,
const ReadOptions& /*options*/,
const std::vector<ColumnFamilyHandle*>& /*column_family*/,
const std::vector<Slice>& keys,
std::vector<std::string>* values) override {
std::vector<std::string>* /*values*/) override {
std::vector<Status> s(keys.size(),
Status::NotSupported("Not implemented."));
return s;
@@ -2266,9 +2266,9 @@ class ModelDB : public DB {
#ifndef ROCKSDB_LITE
using DB::IngestExternalFile;
virtual Status IngestExternalFile(
ColumnFamilyHandle* column_family,
const std::vector<std::string>& external_files,
const IngestExternalFileOptions& options) override {
ColumnFamilyHandle* /*column_family*/,
const std::vector<std::string>& /*external_files*/,
const IngestExternalFileOptions& /*options*/) override {
return Status::NotSupported("Not implemented.");
}
@@ -2278,22 +2278,22 @@ class ModelDB : public DB {
using DB::GetPropertiesOfAllTables;
virtual Status GetPropertiesOfAllTables(
ColumnFamilyHandle* column_family,
TablePropertiesCollection* props) override {
ColumnFamilyHandle* /*column_family*/,
TablePropertiesCollection* /*props*/) override {
return Status();
}
virtual Status GetPropertiesOfTablesInRange(
ColumnFamilyHandle* column_family, const Range* range, std::size_t n,
TablePropertiesCollection* props) override {
ColumnFamilyHandle* /*column_family*/, const Range* /*range*/,
std::size_t /*n*/, TablePropertiesCollection* /*props*/) override {
return Status();
}
#endif // ROCKSDB_LITE
using DB::KeyMayExist;
virtual bool KeyMayExist(const ReadOptions& options,
ColumnFamilyHandle* column_family, const Slice& key,
std::string* value,
virtual bool KeyMayExist(const ReadOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice& /*key*/, std::string* /*value*/,
bool* value_found = nullptr) override {
if (value_found != nullptr) {
*value_found = false;
@@ -2301,8 +2301,9 @@ class ModelDB : public DB {
return true; // Not Supported directly
}
using DB::NewIterator;
virtual Iterator* NewIterator(const ReadOptions& options,
ColumnFamilyHandle* column_family) override {
virtual Iterator* NewIterator(
const ReadOptions& options,
ColumnFamilyHandle* /*column_family*/) override {
if (options.snapshot == nullptr) {
KVMap* saved = new KVMap;
*saved = map_;
@@ -2314,9 +2315,9 @@ class ModelDB : public DB {
}
}
virtual Status NewIterators(
const ReadOptions& options,
const std::vector<ColumnFamilyHandle*>& column_family,
std::vector<Iterator*>* iterators) override {
const ReadOptions& /*options*/,
const std::vector<ColumnFamilyHandle*>& /*column_family*/,
std::vector<Iterator*>* /*iterators*/) override {
return Status::NotSupported("Not supported yet");
}
virtual const Snapshot* GetSnapshot() override {
@@ -2329,7 +2330,7 @@ class ModelDB : public DB {
delete reinterpret_cast<const ModelSnapshot*>(snapshot);
}
virtual Status Write(const WriteOptions& options,
virtual Status Write(const WriteOptions& /*options*/,
WriteBatch* batch) override {
class Handler : public WriteBatch::Handler {
public:
@@ -2337,7 +2338,8 @@ class ModelDB : public DB {
virtual void Put(const Slice& key, const Slice& value) override {
(*map_)[key.ToString()] = value.ToString();
}
virtual void Merge(const Slice& key, const Slice& value) override {
virtual void Merge(const Slice& /*key*/,
const Slice& /*value*/) override {
// ignore merge for now
// (*map_)[key.ToString()] = value.ToString();
}
@@ -2351,62 +2353,65 @@ class ModelDB : public DB {
}
using DB::GetProperty;
virtual bool GetProperty(ColumnFamilyHandle* column_family,
const Slice& property, std::string* value) override {
virtual bool GetProperty(ColumnFamilyHandle* /*column_family*/,
const Slice& /*property*/,
std::string* /*value*/) override {
return false;
}
using DB::GetIntProperty;
virtual bool GetIntProperty(ColumnFamilyHandle* column_family,
const Slice& property, uint64_t* value) override {
virtual bool GetIntProperty(ColumnFamilyHandle* /*column_family*/,
const Slice& /*property*/,
uint64_t* /*value*/) override {
return false;
}
using DB::GetMapProperty;
virtual bool GetMapProperty(
ColumnFamilyHandle* column_family, const Slice& property,
std::map<std::string, std::string>* value) override {
ColumnFamilyHandle* /*column_family*/, const Slice& /*property*/,
std::map<std::string, std::string>* /*value*/) override {
return false;
}
using DB::GetAggregatedIntProperty;
virtual bool GetAggregatedIntProperty(const Slice& property,
uint64_t* value) override {
virtual bool GetAggregatedIntProperty(const Slice& /*property*/,
uint64_t* /*value*/) override {
return false;
}
using DB::GetApproximateSizes;
virtual void GetApproximateSizes(ColumnFamilyHandle* column_family,
const Range* range, int n, uint64_t* sizes,
uint8_t include_flags
virtual void GetApproximateSizes(ColumnFamilyHandle* /*column_family*/,
const Range* /*range*/, int n,
uint64_t* sizes,
uint8_t /*include_flags*/
= INCLUDE_FILES) override {
for (int i = 0; i < n; i++) {
sizes[i] = 0;
}
}
using DB::GetApproximateMemTableStats;
virtual void GetApproximateMemTableStats(ColumnFamilyHandle* column_family,
const Range& range,
uint64_t* const count,
uint64_t* const size) override {
virtual void GetApproximateMemTableStats(
ColumnFamilyHandle* /*column_family*/, const Range& /*range*/,
uint64_t* const count, uint64_t* const size) override {
*count = 0;
*size = 0;
}
using DB::CompactRange;
virtual Status CompactRange(const CompactRangeOptions& options,
ColumnFamilyHandle* column_family,
const Slice* start, const Slice* end) override {
virtual Status CompactRange(const CompactRangeOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/,
const Slice* /*start*/,
const Slice* /*end*/) override {
return Status::NotSupported("Not supported operation.");
}
virtual Status SetDBOptions(
const std::unordered_map<std::string, std::string>& new_options)
const std::unordered_map<std::string, std::string>& /*new_options*/)
override {
return Status::NotSupported("Not supported operation.");
}
using DB::CompactFiles;
virtual Status CompactFiles(const CompactionOptions& compact_options,
ColumnFamilyHandle* column_family,
const std::vector<std::string>& input_file_names,
const int output_level,
const int output_path_id = -1) override {
virtual Status CompactFiles(
const CompactionOptions& /*compact_options*/,
ColumnFamilyHandle* /*column_family*/,
const std::vector<std::string>& /*input_file_names*/,
const int /*output_level*/, const int /*output_path_id*/ = -1) override {
return Status::NotSupported("Not supported operation.");
}
@@ -2419,24 +2424,25 @@ class ModelDB : public DB {
}
Status EnableAutoCompaction(
const std::vector<ColumnFamilyHandle*>& column_family_handles) override {
const std::vector<ColumnFamilyHandle*>& /*column_family_handles*/)
override {
return Status::NotSupported("Not supported operation.");
}
using DB::NumberLevels;
virtual int NumberLevels(ColumnFamilyHandle* column_family) override {
virtual int NumberLevels(ColumnFamilyHandle* /*column_family*/) override {
return 1;
}
using DB::MaxMemCompactionLevel;
virtual int MaxMemCompactionLevel(
ColumnFamilyHandle* column_family) override {
ColumnFamilyHandle* /*column_family*/) override {
return 1;
}
using DB::Level0StopWriteTrigger;
virtual int Level0StopWriteTrigger(
ColumnFamilyHandle* column_family) override {
ColumnFamilyHandle* /*column_family*/) override {
return -1;
}
@@ -2445,7 +2451,8 @@ class ModelDB : public DB {
virtual Env* GetEnv() const override { return nullptr; }
using DB::GetOptions;
virtual Options GetOptions(ColumnFamilyHandle* column_family) const override {
virtual Options GetOptions(
ColumnFamilyHandle* /*column_family*/) const override {
return options_;
}
@@ -2453,8 +2460,8 @@ class ModelDB : public DB {
virtual DBOptions GetDBOptions() const override { return options_; }
using DB::Flush;
virtual Status Flush(const rocksdb::FlushOptions& options,
ColumnFamilyHandle* column_family) override {
virtual Status Flush(const rocksdb::FlushOptions& /*options*/,
ColumnFamilyHandle* /*column_family*/) override {
Status ret;
return ret;
}
@@ -2464,39 +2471,42 @@ class ModelDB : public DB {
#ifndef ROCKSDB_LITE
virtual Status DisableFileDeletions() override { return Status::OK(); }
virtual Status EnableFileDeletions(bool force) override {
virtual Status EnableFileDeletions(bool /*force*/) override {
return Status::OK();
}
virtual Status GetLiveFiles(std::vector<std::string>&, uint64_t* size,
bool flush_memtable = true) override {
virtual Status GetLiveFiles(std::vector<std::string>&, uint64_t* /*size*/,
bool /*flush_memtable*/ = true) override {
return Status::OK();
}
virtual Status GetSortedWalFiles(VectorLogPtr& files) override {
virtual Status GetSortedWalFiles(VectorLogPtr& /*files*/) override {
return Status::OK();
}
virtual Status DeleteFile(std::string name) override { return Status::OK(); }
virtual Status DeleteFile(std::string /*name*/) override {
return Status::OK();
}
virtual Status GetUpdatesSince(
rocksdb::SequenceNumber, unique_ptr<rocksdb::TransactionLogIterator>*,
const TransactionLogIterator::ReadOptions& read_options =
const TransactionLogIterator::ReadOptions& /*read_options*/ =
TransactionLogIterator::ReadOptions()) override {
return Status::NotSupported("Not supported in Model DB");
}
virtual void GetColumnFamilyMetaData(
ColumnFamilyHandle* column_family,
ColumnFamilyMetaData* metadata) override {}
ColumnFamilyHandle* /*column_family*/,
ColumnFamilyMetaData* /*metadata*/) override {}
#endif // ROCKSDB_LITE
virtual Status GetDbIdentity(std::string& identity) const override {
virtual Status GetDbIdentity(std::string& /*identity*/) const override {
return Status::OK();
}
virtual SequenceNumber GetLatestSequenceNumber() const override { return 0; }
virtual bool SetPreserveDeletesSequenceNumber(SequenceNumber seqnum) override {
virtual bool SetPreserveDeletesSequenceNumber(
SequenceNumber /*seqnum*/) override {
return true;
}
@@ -4776,7 +4786,7 @@ class DelayedMergeOperator : public MergeOperator {
public:
explicit DelayedMergeOperator(DBTest* d) : db_test_(d) {}
virtual bool FullMergeV2(const MergeOperationInput& merge_in,
virtual bool FullMergeV2(const MergeOperationInput& /*merge_in*/,
MergeOperationOutput* merge_out) const override {
db_test_->env_->addon_time_.fetch_add(1000);
merge_out->new_value = "";
+8 -9
View File
@@ -498,9 +498,9 @@ TEST_F(DBTest2, WalFilterTest) {
apply_option_at_record_index_(apply_option_for_record_index),
current_record_index_(0) {}
virtual WalProcessingOption LogRecord(const WriteBatch& batch,
WriteBatch* new_batch,
bool* batch_changed) const override {
virtual WalProcessingOption LogRecord(
const WriteBatch& /*batch*/, WriteBatch* /*new_batch*/,
bool* /*batch_changed*/) const override {
WalFilter::WalProcessingOption option_to_return;
if (current_record_index_ == apply_option_at_record_index_) {
@@ -874,11 +874,10 @@ TEST_F(DBTest2, WalFilterTestWithColumnFamilies) {
cf_name_id_map_ = cf_name_id_map;
}
virtual WalProcessingOption LogRecordFound(unsigned long long log_number,
const std::string& log_file_name,
const WriteBatch& batch,
WriteBatch* new_batch,
bool* batch_changed) override {
virtual WalProcessingOption LogRecordFound(
unsigned long long log_number, const std::string& /*log_file_name*/,
const WriteBatch& batch, WriteBatch* /*new_batch*/,
bool* /*batch_changed*/) override {
class LogRecordBatchHandler : public WriteBatch::Handler {
private:
const std::map<uint32_t, uint64_t> & cf_log_number_map_;
@@ -1231,7 +1230,7 @@ class CompactionStallTestListener : public EventListener {
public:
CompactionStallTestListener() : compacted_files_cnt_(0) {}
void OnCompactionCompleted(DB* db, const CompactionJobInfo& ci) override {
void OnCompactionCompleted(DB* /*db*/, const CompactionJobInfo& ci) override {
ASSERT_EQ(ci.cf_name, "default");
ASSERT_EQ(ci.base_input_level, 0);
ASSERT_EQ(ci.compaction_reason, CompactionReason::kLevelL0FilesNum);
+6 -5
View File
@@ -1143,17 +1143,18 @@ UpdateStatus DBTestBase::updateInPlaceSmallerVarintSize(char* prevValue,
}
}
UpdateStatus DBTestBase::updateInPlaceLargerSize(char* prevValue,
uint32_t* prevSize,
UpdateStatus DBTestBase::updateInPlaceLargerSize(char* /*prevValue*/,
uint32_t* /*prevSize*/,
Slice delta,
std::string* newValue) {
*newValue = std::string(delta.size(), 'c');
return UpdateStatus::UPDATED;
}
UpdateStatus DBTestBase::updateInPlaceNoAction(char* prevValue,
uint32_t* prevSize, Slice delta,
std::string* newValue) {
UpdateStatus DBTestBase::updateInPlaceNoAction(char* /*prevValue*/,
uint32_t* /*prevSize*/,
Slice /*delta*/,
std::string* /*newValue*/) {
return UpdateStatus::UPDATE_FAILED;
}
+1 -1
View File
@@ -187,7 +187,7 @@ class SpecialSkipListFactory : public MemTableRepFactory {
using MemTableRepFactory::CreateMemTableRep;
virtual MemTableRep* CreateMemTableRep(
const MemTableRep::KeyComparator& compare, Allocator* allocator,
const SliceTransform* transform, Logger* logger) override {
const SliceTransform* transform, Logger* /*logger*/) override {
return new SpecialMemTableRep(
allocator, factory_.CreateMemTableRep(compare, allocator, transform, 0),
num_entries_flush_);
+7 -7
View File
@@ -56,9 +56,9 @@ void VerifyCompactionResult(
class KeepFilter : public CompactionFilter {
public:
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value, bool* value_changed) const
override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
return false;
}
@@ -88,9 +88,9 @@ class KeepFilterFactory : public CompactionFilterFactory {
class DelayFilter : public CompactionFilter {
public:
explicit DelayFilter(DBTestBase* d) : db_test(d) {}
virtual bool Filter(int level, const Slice& key, const Slice& value,
std::string* new_value,
bool* value_changed) const override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*value*/, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
db_test->env_->addon_time_.fetch_add(1000);
return true;
}
@@ -105,7 +105,7 @@ class DelayFilterFactory : public CompactionFilterFactory {
public:
explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {}
virtual std::unique_ptr<CompactionFilter> CreateCompactionFilter(
const CompactionFilter::Context& context) override {
const CompactionFilter::Context& /*context*/) override {
return std::unique_ptr<CompactionFilter>(new DelayFilter(db_test));
}
+1 -1
View File
@@ -159,7 +159,7 @@ class DeleteFileTest : public testing::Test {
}
// An empty job to guard all jobs are processed
static void GuardFinish(void* arg) {
static void GuardFinish(void* /*arg*/) {
TEST_SYNC_POINT("DeleteFileTest::GuardFinish");
}
};
+10 -54
View File
@@ -132,11 +132,15 @@ Status ExternalSstFileIngestionJob::Prepare(
return status;
}
Status ExternalSstFileIngestionJob::NeedsFlush(bool* flush_needed) {
SuperVersion* super_version = cfd_->GetSuperVersion();
Status ExternalSstFileIngestionJob::NeedsFlush(bool* flush_needed,
SuperVersion* super_version) {
autovector<Range> ranges;
for (const IngestedFileInfo& file_to_ingest : files_to_ingest_) {
ranges.emplace_back(file_to_ingest.smallest_user_key,
file_to_ingest.largest_user_key);
}
Status status =
IngestedFilesOverlapWithMemtables(super_version, flush_needed);
cfd_->RangesOverlapWithMemtables(ranges, super_version, flush_needed);
if (status.ok() && *flush_needed &&
!ingestion_options_.allow_blocking_flush) {
status = Status::InvalidArgument("External file requires flush");
@@ -148,11 +152,12 @@ Status ExternalSstFileIngestionJob::NeedsFlush(bool* flush_needed) {
// nonmem_write_thread_
Status ExternalSstFileIngestionJob::Run() {
Status status;
SuperVersion* super_version = cfd_->GetSuperVersion();
#ifndef NDEBUG
// We should never run the job with a memtable that is overlapping
// with the files we are ingesting
bool need_flush = false;
status = NeedsFlush(&need_flush);
status = NeedsFlush(&need_flush, super_version);
assert(status.ok() && need_flush == false);
#endif
@@ -167,7 +172,6 @@ Status ExternalSstFileIngestionJob::Run() {
// It is safe to use this instead of LastAllocatedSequence since we are
// the only active writer, and hence they are equal
const SequenceNumber last_seqno = versions_->LastSequence();
SuperVersion* super_version = cfd_->GetSuperVersion();
edit_.SetColumnFamily(cfd_->GetID());
// The levels that the files will be ingested into
@@ -375,54 +379,6 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
return status;
}
Status ExternalSstFileIngestionJob::IngestedFilesOverlapWithMemtables(
SuperVersion* sv, bool* overlap) {
*overlap = false;
// Create an InternalIterator over all memtables
Arena arena;
ReadOptions ro;
ro.total_order_seek = true;
MergeIteratorBuilder merge_iter_builder(&cfd_->internal_comparator(), &arena);
merge_iter_builder.AddIterator(sv->mem->NewIterator(ro, &arena));
sv->imm->AddIterators(ro, &merge_iter_builder);
ScopedArenaIterator memtable_iter(merge_iter_builder.Finish());
std::vector<InternalIterator*> memtable_range_del_iters;
auto* active_range_del_iter = sv->mem->NewRangeTombstoneIterator(ro);
if (active_range_del_iter != nullptr) {
memtable_range_del_iters.push_back(active_range_del_iter);
}
sv->imm->AddRangeTombstoneIterators(ro, &memtable_range_del_iters);
RangeDelAggregator range_del_agg(cfd_->internal_comparator(),
{} /* snapshots */,
false /* collapse_deletions */);
Status status;
{
std::unique_ptr<InternalIterator> memtable_range_del_iter(
NewMergingIterator(&cfd_->internal_comparator(),
memtable_range_del_iters.empty()
? nullptr
: &memtable_range_del_iters[0],
static_cast<int>(memtable_range_del_iters.size())));
status = range_del_agg.AddTombstones(std::move(memtable_range_del_iter));
}
if (status.ok()) {
for (IngestedFileInfo& f : files_to_ingest_) {
status = IngestedFileOverlapWithIteratorRange(&f, memtable_iter.get(),
overlap);
if (!status.ok() || *overlap == true) {
break;
}
if (range_del_agg.IsRangeOverlapped(f.smallest_user_key,
f.largest_user_key)) {
*overlap = true;
break;
}
}
}
return status;
}
Status ExternalSstFileIngestionJob::AssignLevelAndSeqnoForIngestedFile(
SuperVersion* sv, bool force_global_seqno, CompactionStyle compaction_style,
IngestedFileInfo* file_to_ingest, SequenceNumber* assigned_seqno) {
+6 -6
View File
@@ -84,8 +84,12 @@ class ExternalSstFileIngestionJob {
// Check if we need to flush the memtable before running the ingestion job
// This will be true if the files we are ingesting are overlapping with any
// key range in the memtable.
// REQUIRES: Mutex held
Status NeedsFlush(bool* flush_needed);
//
// @param super_version A referenced SuperVersion that will be held for the
// duration of this function.
//
// Thread-safe
Status NeedsFlush(bool* flush_needed, SuperVersion* super_version);
// Will execute the ingestion job and prepare edit() to be applied.
// REQUIRES: Mutex held
@@ -110,10 +114,6 @@ class ExternalSstFileIngestionJob {
Status GetIngestedFileInfo(const std::string& external_file,
IngestedFileInfo* file_to_ingest);
// Check if the files we are ingesting overlap with any memtable.
// REQUIRES: Mutex held
Status IngestedFilesOverlapWithMemtables(SuperVersion* sv, bool* overlap);
// Assign `file_to_ingest` the appropriate sequence number and the lowest
// possible level that it can be ingested to according to compaction_style.
// REQUIRES: Mutex held
+5 -4
View File
@@ -396,8 +396,9 @@ class SstFileWriterCollector : public TablePropertiesCollector {
return Status::OK();
}
Status AddUserKey(const Slice& user_key, const Slice& value, EntryType type,
SequenceNumber seq, uint64_t file_size) override {
Status AddUserKey(const Slice& /*user_key*/, const Slice& /*value*/,
EntryType /*type*/, SequenceNumber /*seq*/,
uint64_t /*file_size*/) override {
++count_;
return Status::OK();
}
@@ -417,7 +418,7 @@ class SstFileWriterCollectorFactory : public TablePropertiesCollectorFactory {
explicit SstFileWriterCollectorFactory(std::string prefix)
: prefix_(prefix), num_created_(0) {}
virtual TablePropertiesCollector* CreateTablePropertiesCollector(
TablePropertiesCollectorFactory::Context context) override {
TablePropertiesCollectorFactory::Context /*context*/) override {
num_created_++;
return new SstFileWriterCollector(prefix_);
}
@@ -1798,7 +1799,7 @@ TEST_F(ExternalSSTFileTest, FileWithCFInfo) {
class TestIngestExternalFileListener : public EventListener {
public:
void OnExternalFileIngested(DB* db,
void OnExternalFileIngested(DB* /*db*/,
const ExternalFileIngestionInfo& info) override {
ingested_files.push_back(info);
}
+3 -3
View File
@@ -36,10 +36,10 @@ class IntComparator : public Comparator {
const char* Name() const override { return "IntComparator"; }
void FindShortestSeparator(std::string* start,
const Slice& limit) const override {}
void FindShortestSeparator(std::string* /*start*/,
const Slice& /*limit*/) const override {}
void FindShortSuccessor(std::string* key) const override {}
void FindShortSuccessor(std::string* /*key*/) const override {}
};
class FileIndexerTest : public testing::Test {
+1 -1
View File
@@ -104,7 +104,7 @@ class ForwardLevelIterator : public InternalIterator {
file_iter_->Seek(internal_key);
valid_ = file_iter_->Valid();
}
void SeekForPrev(const Slice& internal_key) override {
void SeekForPrev(const Slice& /*internal_key*/) override {
status_ = Status::NotSupported("ForwardLevelIterator::SeekForPrev()");
valid_ = false;
}
+1 -1
View File
@@ -55,7 +55,7 @@ class ForwardIterator : public InternalIterator {
ColumnFamilyData* cfd, SuperVersion* current_sv = nullptr);
virtual ~ForwardIterator();
void SeekForPrev(const Slice& target) override {
void SeekForPrev(const Slice& /*target*/) override {
status_ = Status::NotSupported("ForwardIterator::SeekForPrev()");
valid_ = false;
}
+73 -55
View File
@@ -233,6 +233,7 @@ static const std::string estimate_live_data_size = "estimate-live-data-size";
static const std::string min_log_number_to_keep = "min-log-number-to-keep";
static const std::string base_level = "base-level";
static const std::string total_sst_files_size = "total-sst-files-size";
static const std::string live_sst_files_size = "live-sst-files-size";
static const std::string estimate_pending_comp_bytes =
"estimate-pending-compaction-bytes";
static const std::string aggregated_table_properties =
@@ -307,6 +308,8 @@ const std::string DB::Properties::kMinLogNumberToKeep =
rocksdb_prefix + min_log_number_to_keep;
const std::string DB::Properties::kTotalSstFilesSize =
rocksdb_prefix + total_sst_files_size;
const std::string DB::Properties::kLiveSstFilesSize =
rocksdb_prefix + live_sst_files_size;
const std::string DB::Properties::kBaseLevel = rocksdb_prefix + base_level;
const std::string DB::Properties::kEstimatePendingCompactionBytes =
rocksdb_prefix + estimate_pending_comp_bytes;
@@ -405,6 +408,8 @@ const std::unordered_map<std::string, DBPropertyInfo>
{false, nullptr, &InternalStats::HandleBaseLevel, nullptr}},
{DB::Properties::kTotalSstFilesSize,
{false, nullptr, &InternalStats::HandleTotalSstFilesSize, nullptr}},
{DB::Properties::kLiveSstFilesSize,
{false, nullptr, &InternalStats::HandleLiveSstFilesSize, nullptr}},
{DB::Properties::kEstimatePendingCompactionBytes,
{false, nullptr, &InternalStats::HandleEstimatePendingCompactionBytes,
nullptr}},
@@ -442,7 +447,7 @@ bool InternalStats::GetStringProperty(const DBPropertyInfo& property_info,
}
bool InternalStats::GetMapProperty(const DBPropertyInfo& property_info,
const Slice& property,
const Slice& /*property*/,
std::map<std::string, std::string>* value) {
assert(value != nullptr);
assert(property_info.handle_map != nullptr);
@@ -494,7 +499,7 @@ bool InternalStats::HandleCompressionRatioAtLevelPrefix(std::string* value,
return true;
}
bool InternalStats::HandleLevelStats(std::string* value, Slice suffix) {
bool InternalStats::HandleLevelStats(std::string* value, Slice /*suffix*/) {
char buf[1000];
const auto* vstorage = cfd_->current()->storage_info();
snprintf(buf, sizeof(buf),
@@ -527,35 +532,36 @@ bool InternalStats::HandleCFMapStats(
return true;
}
bool InternalStats::HandleCFStats(std::string* value, Slice suffix) {
bool InternalStats::HandleCFStats(std::string* value, Slice /*suffix*/) {
DumpCFStats(value);
return true;
}
bool InternalStats::HandleCFStatsNoFileHistogram(std::string* value,
Slice suffix) {
Slice /*suffix*/) {
DumpCFStatsNoFileHistogram(value);
return true;
}
bool InternalStats::HandleCFFileHistogram(std::string* value, Slice suffix) {
bool InternalStats::HandleCFFileHistogram(std::string* value,
Slice /*suffix*/) {
DumpCFFileHistogram(value);
return true;
}
bool InternalStats::HandleDBStats(std::string* value, Slice suffix) {
bool InternalStats::HandleDBStats(std::string* value, Slice /*suffix*/) {
DumpDBStats(value);
return true;
}
bool InternalStats::HandleSsTables(std::string* value, Slice suffix) {
bool InternalStats::HandleSsTables(std::string* value, Slice /*suffix*/) {
auto* current = cfd_->current();
*value = current->DebugString(true, true);
return true;
}
bool InternalStats::HandleAggregatedTableProperties(std::string* value,
Slice suffix) {
Slice /*suffix*/) {
std::shared_ptr<const TableProperties> tp;
auto s = cfd_->current()->GetAggregatedTableProperties(&tp);
if (!s.ok()) {
@@ -582,34 +588,34 @@ bool InternalStats::HandleAggregatedTablePropertiesAtLevel(std::string* value,
return true;
}
bool InternalStats::HandleNumImmutableMemTable(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleNumImmutableMemTable(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
*value = cfd_->imm()->NumNotFlushed();
return true;
}
bool InternalStats::HandleNumImmutableMemTableFlushed(uint64_t* value,
DBImpl* db,
Version* version) {
DBImpl* /*db*/,
Version* /*version*/) {
*value = cfd_->imm()->NumFlushed();
return true;
}
bool InternalStats::HandleMemTableFlushPending(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleMemTableFlushPending(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
// Return number of mem tables that are ready to flush (made immutable)
*value = (cfd_->imm()->IsFlushPending() ? 1 : 0);
return true;
}
bool InternalStats::HandleNumRunningFlushes(uint64_t* value, DBImpl* db,
Version* version) {
Version* /*version*/) {
*value = db->num_running_flushes();
return true;
}
bool InternalStats::HandleCompactionPending(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleCompactionPending(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
// 1 if the system already determines at least one compaction is needed.
// 0 otherwise,
const auto* vstorage = cfd_->current()->storage_info();
@@ -618,70 +624,74 @@ bool InternalStats::HandleCompactionPending(uint64_t* value, DBImpl* db,
}
bool InternalStats::HandleNumRunningCompactions(uint64_t* value, DBImpl* db,
Version* version) {
Version* /*version*/) {
*value = db->num_running_compactions_;
return true;
}
bool InternalStats::HandleBackgroundErrors(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleBackgroundErrors(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
// Accumulated number of errors in background flushes or compactions.
*value = GetBackgroundErrorCount();
return true;
}
bool InternalStats::HandleCurSizeActiveMemTable(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleCurSizeActiveMemTable(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
// Current size of the active memtable
*value = cfd_->mem()->ApproximateMemoryUsage();
return true;
}
bool InternalStats::HandleCurSizeAllMemTables(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleCurSizeAllMemTables(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
// Current size of the active memtable + immutable memtables
*value = cfd_->mem()->ApproximateMemoryUsage() +
cfd_->imm()->ApproximateUnflushedMemTablesMemoryUsage();
return true;
}
bool InternalStats::HandleSizeAllMemTables(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleSizeAllMemTables(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
*value = cfd_->mem()->ApproximateMemoryUsage() +
cfd_->imm()->ApproximateMemoryUsage();
return true;
}
bool InternalStats::HandleNumEntriesActiveMemTable(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleNumEntriesActiveMemTable(uint64_t* value,
DBImpl* /*db*/,
Version* /*version*/) {
// Current number of entires in the active memtable
*value = cfd_->mem()->num_entries();
return true;
}
bool InternalStats::HandleNumEntriesImmMemTables(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleNumEntriesImmMemTables(uint64_t* value,
DBImpl* /*db*/,
Version* /*version*/) {
// Current number of entries in the immutable memtables
*value = cfd_->imm()->current()->GetTotalNumEntries();
return true;
}
bool InternalStats::HandleNumDeletesActiveMemTable(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleNumDeletesActiveMemTable(uint64_t* value,
DBImpl* /*db*/,
Version* /*version*/) {
// Current number of entires in the active memtable
*value = cfd_->mem()->num_deletes();
return true;
}
bool InternalStats::HandleNumDeletesImmMemTables(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleNumDeletesImmMemTables(uint64_t* value,
DBImpl* /*db*/,
Version* /*version*/) {
// Current number of entries in the immutable memtables
*value = cfd_->imm()->current()->GetTotalNumDeletes();
return true;
}
bool InternalStats::HandleEstimateNumKeys(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleEstimateNumKeys(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
// Estimate number of entries in the column family:
// Use estimated entries in tables + total entries in memtables.
const auto* vstorage = cfd_->current()->storage_info();
@@ -697,77 +707,85 @@ bool InternalStats::HandleEstimateNumKeys(uint64_t* value, DBImpl* db,
}
bool InternalStats::HandleNumSnapshots(uint64_t* value, DBImpl* db,
Version* version) {
Version* /*version*/) {
*value = db->snapshots().count();
return true;
}
bool InternalStats::HandleOldestSnapshotTime(uint64_t* value, DBImpl* db,
Version* version) {
Version* /*version*/) {
*value = static_cast<uint64_t>(db->snapshots().GetOldestSnapshotTime());
return true;
}
bool InternalStats::HandleNumLiveVersions(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleNumLiveVersions(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
*value = cfd_->GetNumLiveVersions();
return true;
}
bool InternalStats::HandleCurrentSuperVersionNumber(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleCurrentSuperVersionNumber(uint64_t* value,
DBImpl* /*db*/,
Version* /*version*/) {
*value = cfd_->GetSuperVersionNumber();
return true;
}
bool InternalStats::HandleIsFileDeletionsEnabled(uint64_t* value, DBImpl* db,
Version* version) {
Version* /*version*/) {
*value = db->IsFileDeletionsEnabled();
return true;
}
bool InternalStats::HandleBaseLevel(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleBaseLevel(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
const auto* vstorage = cfd_->current()->storage_info();
*value = vstorage->base_level();
return true;
}
bool InternalStats::HandleTotalSstFilesSize(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleTotalSstFilesSize(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
*value = cfd_->GetTotalSstFilesSize();
return true;
}
bool InternalStats::HandleLiveSstFilesSize(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
*value = cfd_->GetLiveSstFilesSize();
return true;
}
bool InternalStats::HandleEstimatePendingCompactionBytes(uint64_t* value,
DBImpl* db,
Version* version) {
DBImpl* /*db*/,
Version* /*version*/) {
const auto* vstorage = cfd_->current()->storage_info();
*value = vstorage->estimated_compaction_needed_bytes();
return true;
}
bool InternalStats::HandleEstimateTableReadersMem(uint64_t* value, DBImpl* db,
bool InternalStats::HandleEstimateTableReadersMem(uint64_t* value,
DBImpl* /*db*/,
Version* version) {
*value = (version == nullptr) ? 0 : version->GetMemoryUsageByTableReaders();
return true;
}
bool InternalStats::HandleEstimateLiveDataSize(uint64_t* value, DBImpl* db,
Version* version) {
bool InternalStats::HandleEstimateLiveDataSize(uint64_t* value, DBImpl* /*db*/,
Version* /*version*/) {
const auto* vstorage = cfd_->current()->storage_info();
*value = vstorage->EstimateLiveDataSize();
return true;
}
bool InternalStats::HandleMinLogNumberToKeep(uint64_t* value, DBImpl* db,
Version* version) {
Version* /*version*/) {
*value = db->MinLogNumberToKeep();
return true;
}
bool InternalStats::HandleActualDelayedWriteRate(uint64_t* value, DBImpl* db,
Version* version) {
Version* /*version*/) {
const WriteController& wc = db->write_controller();
if (!wc.NeedsDelay()) {
*value = 0;
@@ -778,7 +796,7 @@ bool InternalStats::HandleActualDelayedWriteRate(uint64_t* value, DBImpl* db,
}
bool InternalStats::HandleIsWriteStopped(uint64_t* value, DBImpl* db,
Version* version) {
Version* /*version*/) {
*value = db->write_controller().IsStopped() ? 1 : 0;
return true;
}
+1
View File
@@ -467,6 +467,7 @@ class InternalStats {
Version* version);
bool HandleBaseLevel(uint64_t* value, DBImpl* db, Version* version);
bool HandleTotalSstFilesSize(uint64_t* value, DBImpl* db, Version* version);
bool HandleLiveSstFilesSize(uint64_t* value, DBImpl* db, Version* version);
bool HandleEstimatePendingCompactionBytes(uint64_t* value, DBImpl* db,
Version* version);
bool HandleEstimateTableReadersMem(uint64_t* value, DBImpl* db,
+9 -8
View File
@@ -46,11 +46,11 @@ class EventListenerTest : public DBTestBase {
};
struct TestPropertiesCollector : public rocksdb::TablePropertiesCollector {
virtual rocksdb::Status AddUserKey(const rocksdb::Slice& key,
const rocksdb::Slice& value,
rocksdb::EntryType type,
rocksdb::SequenceNumber seq,
uint64_t file_size) override {
virtual rocksdb::Status AddUserKey(const rocksdb::Slice& /*key*/,
const rocksdb::Slice& /*value*/,
rocksdb::EntryType /*type*/,
rocksdb::SequenceNumber /*seq*/,
uint64_t /*file_size*/) override {
return Status::OK();
}
virtual rocksdb::Status Finish(
@@ -73,7 +73,7 @@ struct TestPropertiesCollector : public rocksdb::TablePropertiesCollector {
class TestPropertiesCollectorFactory : public TablePropertiesCollectorFactory {
public:
virtual TablePropertiesCollector* CreateTablePropertiesCollector(
TablePropertiesCollectorFactory::Context context) override {
TablePropertiesCollectorFactory::Context /*context*/) override {
return new TestPropertiesCollector;
}
const char* Name() const override { return "TestTablePropertiesCollector"; }
@@ -425,7 +425,7 @@ TEST_F(EventListenerTest, DisableBGCompaction) {
class TestCompactionReasonListener : public EventListener {
public:
void OnCompactionCompleted(DB* db, const CompactionJobInfo& ci) override {
void OnCompactionCompleted(DB* /*db*/, const CompactionJobInfo& ci) override {
std::lock_guard<std::mutex> lock(mutex_);
compaction_reasons_.push_back(ci.compaction_reason);
}
@@ -807,7 +807,8 @@ class BackgroundErrorListener : public EventListener {
public:
BackgroundErrorListener(SpecialEnv* env) : env_(env), counter_(0) {}
void OnBackgroundError(BackgroundErrorReason reason, Status* bg_error) override {
void OnBackgroundError(BackgroundErrorReason /*reason*/,
Status* bg_error) override {
if (counter_ == 0) {
// suppress the first error and disable write-dropping such that a retry
// can succeed.
+3 -3
View File
@@ -46,9 +46,9 @@ class DestroyAllCompactionFilter : public CompactionFilter {
public:
DestroyAllCompactionFilter() {}
virtual bool Filter(int level, const Slice& key, const Slice& existing_value,
std::string* new_value,
bool* value_changed) const override {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& existing_value, std::string* /*new_value*/,
bool* /*value_changed*/) const override {
return existing_value.ToString() == "destroy";
}
+1 -1
View File
@@ -236,7 +236,7 @@ int MemTable::KeyComparator::operator()(const char* prefix_len_key,
return comparator.CompareKeySeq(a, key);
}
void MemTableRep::InsertConcurrently(KeyHandle handle) {
void MemTableRep::InsertConcurrently(KeyHandle /*handle*/) {
#ifndef ROCKSDB_LITE
throw std::runtime_error("concurrent insert not supported");
#else
+2 -2
View File
@@ -154,7 +154,7 @@ bool MemTableListVersion::GetFromList(
}
Status MemTableListVersion::AddRangeTombstoneIterators(
const ReadOptions& read_opts, Arena* arena,
const ReadOptions& read_opts, Arena* /*arena*/,
RangeDelAggregator* range_del_agg) {
assert(range_del_agg != nullptr);
for (auto& m : memlist_) {
@@ -300,7 +300,7 @@ void MemTableList::PickMemtablesToFlush(autovector<MemTable*>* ret) {
}
void MemTableList::RollbackMemtableFlush(const autovector<MemTable*>& mems,
uint64_t file_number) {
uint64_t /*file_number*/) {
AutoThreadOperationStageUpdater stage_updater(
ThreadStatus::STAGE_MEMTABLE_ROLLBACK);
assert(!mems.empty());
+1 -1
View File
@@ -504,7 +504,7 @@ void runTest(int argc, const std::string& dbname, const bool use_ttl = false) {
}
} // namespace
int main(int argc, char *argv[]) {
int main(int argc, char* /*argv*/ []) {
//TODO: Make this test like a general rocksdb unit-test
rocksdb::port::InstallStackTraceHandler();
runTest(argc, test::TmpDir() + "/merge_testdb");
+1 -1
View File
@@ -327,7 +327,7 @@ class TestPlainTableFactory : public PlainTableFactory {
const TableReaderOptions& table_reader_options,
unique_ptr<RandomAccessFileReader>&& file, uint64_t file_size,
unique_ptr<TableReader>* table,
bool prefetch_index_and_filter_in_cache) const override {
bool /*prefetch_index_and_filter_in_cache*/) const override {
TableProperties* props = nullptr;
auto s =
ReadTableProperties(file.get(), file_size, kPlainTableMagicNumber,
+3 -3
View File
@@ -126,10 +126,10 @@ class TestKeyComparator : public Comparator {
return "TestKeyComparator";
}
virtual void FindShortestSeparator(std::string* start,
const Slice& limit) const override {}
virtual void FindShortestSeparator(std::string* /*start*/,
const Slice& /*limit*/) const override {}
virtual void FindShortSuccessor(std::string* key) const override {}
virtual void FindShortSuccessor(std::string* /*key*/) const override {}
};
namespace {
+2 -2
View File
@@ -19,8 +19,8 @@ class SnapshotChecker {
class DisableGCSnapshotChecker : public SnapshotChecker {
public:
virtual ~DisableGCSnapshotChecker() {}
virtual bool IsInSnapshot(SequenceNumber sequence,
SequenceNumber snapshot_sequence) const {
virtual bool IsInSnapshot(SequenceNumber /*sequence*/,
SequenceNumber /*snapshot_sequence*/) const {
// By returning false, we prevent all the values from being GCed
return false;
}
+2 -2
View File
@@ -30,7 +30,7 @@ namespace rocksdb {
namespace {
template <class T>
static void DeleteEntry(const Slice& key, void* value) {
static void DeleteEntry(const Slice& /*key*/, void* value) {
T* typed_value = reinterpret_cast<T*>(value);
delete typed_value;
}
@@ -41,7 +41,7 @@ static void UnrefEntry(void* arg1, void* arg2) {
cache->Release(h);
}
static void DeleteTableReader(void* arg1, void* arg2) {
static void DeleteTableReader(void* arg1, void* /*arg2*/) {
TableReader* table_reader = reinterpret_cast<TableReader*>(arg1);
delete table_reader;
}
+2 -2
View File
@@ -12,8 +12,8 @@
namespace rocksdb {
Status InternalKeyPropertiesCollector::InternalAdd(const Slice& key,
const Slice& value,
uint64_t file_size) {
const Slice& /*value*/,
uint64_t /*file_size*/) {
ParsedInternalKey ikey;
if (!ParseInternalKey(key, &ikey)) {
return Status::InvalidArgument("Invalid internal key");
+1 -1
View File
@@ -73,7 +73,7 @@ class InternalKeyPropertiesCollectorFactory
: public IntTblPropCollectorFactory {
public:
virtual IntTblPropCollector* CreateIntTblPropCollector(
uint32_t column_family_id) override {
uint32_t /*column_family_id*/) override {
return new InternalKeyPropertiesCollector();
}
+10 -9
View File
@@ -82,8 +82,9 @@ class RegularKeysStartWithA: public TablePropertiesCollector {
return Status::OK();
}
Status AddUserKey(const Slice& user_key, const Slice& value, EntryType type,
SequenceNumber seq, uint64_t file_size) override {
Status AddUserKey(const Slice& user_key, const Slice& /*value*/,
EntryType type, SequenceNumber /*seq*/,
uint64_t file_size) override {
// simply asssume all user keys are not empty.
if (user_key.data()[0] == 'A') {
++count_;
@@ -133,7 +134,7 @@ class RegularKeysStartWithABackwardCompatible
return Status::OK();
}
Status Add(const Slice& user_key, const Slice& value) override {
Status Add(const Slice& user_key, const Slice& /*value*/) override {
// simply asssume all user keys are not empty.
if (user_key.data()[0] == 'A') {
++count_;
@@ -161,8 +162,8 @@ class RegularKeysStartWithAInternal : public IntTblPropCollector {
return Status::OK();
}
Status InternalAdd(const Slice& user_key, const Slice& value,
uint64_t file_size) override {
Status InternalAdd(const Slice& user_key, const Slice& /*value*/,
uint64_t /*file_size*/) override {
// simply asssume all user keys are not empty.
if (user_key.data()[0] == 'A') {
++count_;
@@ -193,7 +194,7 @@ class RegularKeysStartWithAFactory : public IntTblPropCollectorFactory,
}
}
virtual IntTblPropCollector* CreateIntTblPropCollector(
uint32_t column_family_id) override {
uint32_t /*column_family_id*/) override {
return new RegularKeysStartWithAInternal();
}
const char* Name() const override { return "RegularKeysStartWithA"; }
@@ -203,7 +204,7 @@ class RegularKeysStartWithAFactory : public IntTblPropCollectorFactory,
class FlushBlockEveryThreePolicy : public FlushBlockPolicy {
public:
virtual bool Update(const Slice& key, const Slice& value) override {
virtual bool Update(const Slice& /*key*/, const Slice& /*value*/) override {
return (++count_ % 3U == 0);
}
@@ -220,8 +221,8 @@ class FlushBlockEveryThreePolicyFactory : public FlushBlockPolicyFactory {
}
FlushBlockPolicy* NewFlushBlockPolicy(
const BlockBasedTableOptions& table_options,
const BlockBuilder& data_block_builder) const override {
const BlockBasedTableOptions& /*table_options*/,
const BlockBuilder& /*data_block_builder*/) const override {
return new FlushBlockEveryThreePolicy;
}
};
+6 -4
View File
@@ -269,16 +269,18 @@ void TransactionLogIteratorImpl::UpdateCurrentWriteBatch(const Slice& record) {
return Status::OK();
}
Status PutCF(uint32_t cf, const Slice& key, const Slice& val) override {
Status PutCF(uint32_t /*cf*/, const Slice& /*key*/,
const Slice& /*val*/) override {
return Status::OK();
}
Status DeleteCF(uint32_t cf, const Slice& key) override {
Status DeleteCF(uint32_t /*cf*/, const Slice& /*key*/) override {
return Status::OK();
}
Status SingleDeleteCF(uint32_t cf, const Slice& key) override {
Status SingleDeleteCF(uint32_t /*cf*/, const Slice& /*key*/) override {
return Status::OK();
}
Status MergeCF(uint32_t cf, const Slice& key, const Slice& val) override {
Status MergeCF(uint32_t /*cf*/, const Slice& /*key*/,
const Slice& /*val*/) override {
return Status::OK();
}
Status MarkBeginPrepare() override { return Status::OK(); }
+1 -1
View File
@@ -199,7 +199,7 @@ class VersionBuilder::Rep {
}
}
void CheckConsistencyForDeletes(VersionEdit* edit, uint64_t number,
void CheckConsistencyForDeletes(VersionEdit* /*edit*/, uint64_t number,
int level) {
#ifdef NDEBUG
if (!base_vstorage_->force_consistency_checks()) {
+1 -1
View File
@@ -198,7 +198,7 @@ static bool GetInternalKey(Slice* input, InternalKey* dst) {
}
}
bool VersionEdit::GetLevel(Slice* input, int* level, const char** msg) {
bool VersionEdit::GetLevel(Slice* input, int* level, const char** /*msg*/) {
uint32_t v;
if (GetVarint32(input, &v)) {
*level = v;
+10 -1
View File
@@ -891,6 +891,15 @@ void Version::GetColumnFamilyMetaData(ColumnFamilyMetaData* cf_meta) {
}
}
uint64_t Version::GetSstFilesSize() {
uint64_t sst_files_size = 0;
for (int level = 0; level < storage_info_.num_levels_; level++) {
for (const auto& file_meta : storage_info_.LevelFiles(level)) {
sst_files_size += file_meta->fd.GetFileSize();
}
}
return sst_files_size;
}
uint64_t VersionStorageInfo::GetEstimatedActiveKeys() const {
// Estimation will be inaccurate when:
@@ -2898,7 +2907,7 @@ void VersionSet::LogAndApplyCFHelper(VersionEdit* edit) {
}
void VersionSet::LogAndApplyHelper(ColumnFamilyData* cfd,
VersionBuilder* builder, Version* v,
VersionBuilder* builder, Version* /*v*/,
VersionEdit* edit, InstrumentedMutex* mu) {
mu->AssertHeld();
assert(!edit->IsColumnFamilyManipulation());
+3 -1
View File
@@ -613,6 +613,8 @@ class Version {
void GetColumnFamilyMetaData(ColumnFamilyMetaData* cf_meta);
uint64_t GetSstFilesSize();
private:
Env* env_;
friend class VersionSet;
@@ -870,7 +872,7 @@ class VersionSet {
struct LogReporter : public log::Reader::Reporter {
Status* status;
virtual void Corruption(size_t bytes, const Status& s) override {
virtual void Corruption(size_t /*bytes*/, const Status& s) override {
if (this->status->ok()) *this->status = s;
}
};
+3 -1
View File
@@ -76,7 +76,9 @@ class CountingLogger : public Logger {
public:
CountingLogger() : log_count(0) {}
using Logger::Logv;
virtual void Logv(const char* format, va_list ap) override { log_count++; }
virtual void Logv(const char* /*format*/, va_list /*ap*/) override {
log_count++;
}
int log_count;
};
+1 -1
View File
@@ -73,7 +73,7 @@ class WalManagerTest : public testing::Test {
}
// NOT thread safe
void RollTheLog(bool archived) {
void RollTheLog(bool /*archived*/) {
current_log_number_++;
std::string fname = ArchivedLogFileName(dbname_, current_log_number_);
unique_ptr<WritableFile> file;
+3 -3
View File
@@ -186,7 +186,7 @@ WriteBatch::~WriteBatch() { delete save_points_; }
WriteBatch::Handler::~Handler() { }
void WriteBatch::Handler::LogData(const Slice& blob) {
void WriteBatch::Handler::LogData(const Slice& /*blob*/) {
// If the user has not specified something to do with blobs, then we ignore
// them.
}
@@ -569,7 +569,7 @@ void WriteBatchInternal::SetSequence(WriteBatch* b, SequenceNumber seq) {
EncodeFixed64(&b->rep_[0], seq);
}
size_t WriteBatchInternal::GetFirstOffset(WriteBatch* b) {
size_t WriteBatchInternal::GetFirstOffset(WriteBatch* /*b*/) {
return WriteBatchInternal::kHeader;
}
@@ -1228,7 +1228,7 @@ class MemTableInserter : public WriteBatch::Handler {
return PutCFImpl(column_family_id, key, value, kTypeValue);
}
Status DeleteImpl(uint32_t column_family_id, const Slice& key,
Status DeleteImpl(uint32_t /*column_family_id*/, const Slice& key,
const Slice& value, ValueType delete_type) {
Status ret_status;
MemTable* mem = cf_mems_->GetMemTable();
+16 -16
View File
@@ -437,7 +437,7 @@ TEST_F(WriteBatchTest, DISABLED_ManyUpdates) {
struct NoopHandler : public WriteBatch::Handler {
uint32_t num_seen = 0;
char expected_char = 'A';
virtual Status PutCF(uint32_t column_family_id, const Slice& key,
virtual Status PutCF(uint32_t /*column_family_id*/, const Slice& key,
const Slice& value) override {
EXPECT_EQ(kKeyValueSize, key.size());
EXPECT_EQ(kKeyValueSize, value.size());
@@ -452,22 +452,22 @@ TEST_F(WriteBatchTest, DISABLED_ManyUpdates) {
++num_seen;
return Status::OK();
}
virtual Status DeleteCF(uint32_t column_family_id,
const Slice& key) override {
virtual Status DeleteCF(uint32_t /*column_family_id*/,
const Slice& /*key*/) override {
ADD_FAILURE();
return Status::OK();
}
virtual Status SingleDeleteCF(uint32_t column_family_id,
const Slice& key) override {
virtual Status SingleDeleteCF(uint32_t /*column_family_id*/,
const Slice& /*key*/) override {
ADD_FAILURE();
return Status::OK();
}
virtual Status MergeCF(uint32_t column_family_id, const Slice& key,
const Slice& value) override {
virtual Status MergeCF(uint32_t /*column_family_id*/, const Slice& /*key*/,
const Slice& /*value*/) override {
ADD_FAILURE();
return Status::OK();
}
virtual void LogData(const Slice& blob) override { ADD_FAILURE(); }
virtual void LogData(const Slice& /*blob*/) override { ADD_FAILURE(); }
virtual bool Continue() override { return num_seen < kNumUpdates; }
} handler;
@@ -492,7 +492,7 @@ TEST_F(WriteBatchTest, DISABLED_LargeKeyValue) {
struct NoopHandler : public WriteBatch::Handler {
int num_seen = 0;
virtual Status PutCF(uint32_t column_family_id, const Slice& key,
virtual Status PutCF(uint32_t /*column_family_id*/, const Slice& key,
const Slice& value) override {
EXPECT_EQ(kKeyValueSize, key.size());
EXPECT_EQ(kKeyValueSize, value.size());
@@ -503,22 +503,22 @@ TEST_F(WriteBatchTest, DISABLED_LargeKeyValue) {
++num_seen;
return Status::OK();
}
virtual Status DeleteCF(uint32_t column_family_id,
const Slice& key) override {
virtual Status DeleteCF(uint32_t /*column_family_id*/,
const Slice& /*key*/) override {
ADD_FAILURE();
return Status::OK();
}
virtual Status SingleDeleteCF(uint32_t column_family_id,
const Slice& key) override {
virtual Status SingleDeleteCF(uint32_t /*column_family_id*/,
const Slice& /*key*/) override {
ADD_FAILURE();
return Status::OK();
}
virtual Status MergeCF(uint32_t column_family_id, const Slice& key,
const Slice& value) override {
virtual Status MergeCF(uint32_t /*column_family_id*/, const Slice& /*key*/,
const Slice& /*value*/) override {
ADD_FAILURE();
return Status::OK();
}
virtual void LogData(const Slice& blob) override { ADD_FAILURE(); }
virtual void LogData(const Slice& /*blob*/) override { ADD_FAILURE(); }
virtual bool Continue() override { return num_seen < 2; }
} handler;
+2 -4
View File
@@ -54,9 +54,7 @@ class WriteCallbackTestWriteCallback1 : public WriteCallback {
class WriteCallbackTestWriteCallback2 : public WriteCallback {
public:
Status Callback(DB *db) override {
return Status::Busy();
}
Status Callback(DB* /*db*/) override { return Status::Busy(); }
bool AllowWriteBatching() override { return true; }
};
@@ -74,7 +72,7 @@ class MockWriteCallback : public WriteCallback {
was_called_.store(other.was_called_.load());
}
Status Callback(DB* db) override {
Status Callback(DB* /*db*/) override {
was_called_.store(true);
if (should_fail_) {
return Status::Busy();
+2 -1
View File
@@ -456,7 +456,8 @@ void WriteThread::EnterAsMemTableWriter(Writer* leader,
last_writer->sequence + WriteBatchInternal::Count(last_writer->batch) - 1;
}
void WriteThread::ExitAsMemTableWriter(Writer* self, WriteGroup& write_group) {
void WriteThread::ExitAsMemTableWriter(Writer* /*self*/,
WriteGroup& write_group) {
Writer* leader = write_group.leader;
Writer* last_writer = write_group.last_writer;
+10 -4
View File
@@ -844,7 +844,9 @@ static void decodeCTRParameters(const char *prefix, size_t blockSize, uint64_t &
// CreateNewPrefix initialized an allocated block of prefix memory
// for a new file.
Status CTREncryptionProvider::CreateNewPrefix(const std::string& fname, char *prefix, size_t prefixLength) {
Status CTREncryptionProvider::CreateNewPrefix(const std::string& /*fname*/,
char* prefix,
size_t prefixLength) {
// Create & seed rnd.
Random rnd((uint32_t)Env::Default()->NowMicros());
// Fill entire prefix block with random values.
@@ -873,7 +875,9 @@ Status CTREncryptionProvider::CreateNewPrefix(const std::string& fname, char *pr
// in plain text.
// Returns the amount of space (starting from the start of the prefix)
// that has been initialized.
size_t CTREncryptionProvider::PopulateSecretPrefixPart(char *prefix, size_t prefixLength, size_t blockSize) {
size_t CTREncryptionProvider::PopulateSecretPrefixPart(char* /*prefix*/,
size_t /*prefixLength*/,
size_t /*blockSize*/) {
// Nothing to do here, put in custom data in override when needed.
return 0;
}
@@ -898,8 +902,10 @@ Status CTREncryptionProvider::CreateCipherStream(const std::string& fname, const
// CreateCipherStreamFromPrefix creates a block access cipher stream for a file given
// given name and options. The given prefix is already decrypted.
Status CTREncryptionProvider::CreateCipherStreamFromPrefix(const std::string& fname, const EnvOptions& options,
uint64_t initialCounter, const Slice& iv, const Slice& prefix, unique_ptr<BlockAccessCipherStream>* result) {
Status CTREncryptionProvider::CreateCipherStreamFromPrefix(
const std::string& /*fname*/, const EnvOptions& /*options*/,
uint64_t initialCounter, const Slice& iv, const Slice& /*prefix*/,
unique_ptr<BlockAccessCipherStream>* result) {
(*result) = unique_ptr<BlockAccessCipherStream>(new CTRCipherStream(cipher_, iv.data(), initialCounter));
return Status::OK();
}
+5 -5
View File
@@ -611,13 +611,13 @@ Status NewHdfsEnv(Env** hdfs_env, const std::string& fsname) {
// dummy placeholders used when HDFS is not available
namespace rocksdb {
Status HdfsEnv::NewSequentialFile(const std::string& fname,
unique_ptr<SequentialFile>* result,
const EnvOptions& options) {
return Status::NotSupported("Not compiled with hdfs support");
Status HdfsEnv::NewSequentialFile(const std::string& /*fname*/,
unique_ptr<SequentialFile>* /*result*/,
const EnvOptions& /*options*/) {
return Status::NotSupported("Not compiled with hdfs support");
}
Status NewHdfsEnv(Env** hdfs_env, const std::string& fsname) {
Status NewHdfsEnv(Env** /*hdfs_env*/, const std::string& /*fsname*/) {
return Status::NotSupported("Not compiled with hdfs support");
}
}
+16 -12
View File
@@ -1255,33 +1255,36 @@ TEST_P(EnvPosixTestWithParam, WritableFileWrapper) {
inc(0);
}
Status Append(const Slice& data) override { inc(1); return Status::OK(); }
Status Truncate(uint64_t size) override { return Status::OK(); }
Status Append(const Slice& /*data*/) override {
inc(1);
return Status::OK();
}
Status Truncate(uint64_t /*size*/) override { return Status::OK(); }
Status Close() override { inc(2); return Status::OK(); }
Status Flush() override { inc(3); return Status::OK(); }
Status Sync() override { inc(4); return Status::OK(); }
Status Fsync() override { inc(5); return Status::OK(); }
void SetIOPriority(Env::IOPriority pri) override { inc(6); }
void SetIOPriority(Env::IOPriority /*pri*/) override { inc(6); }
uint64_t GetFileSize() override { inc(7); return 0; }
void GetPreallocationStatus(size_t* block_size,
size_t* last_allocated_block) override {
void GetPreallocationStatus(size_t* /*block_size*/,
size_t* /*last_allocated_block*/) override {
inc(8);
}
size_t GetUniqueId(char* id, size_t max_size) const override {
size_t GetUniqueId(char* /*id*/, size_t /*max_size*/) const override {
inc(9);
return 0;
}
Status InvalidateCache(size_t offset, size_t length) override {
Status InvalidateCache(size_t /*offset*/, size_t /*length*/) override {
inc(10);
return Status::OK();
}
protected:
Status Allocate(uint64_t offset, uint64_t len) override {
Status Allocate(uint64_t /*offset*/, uint64_t /*len*/) override {
inc(11);
return Status::OK();
}
Status RangeSync(uint64_t offset, uint64_t nbytes) override {
Status RangeSync(uint64_t /*offset*/, uint64_t /*nbytes*/) override {
inc(12);
return Status::OK();
}
@@ -1489,8 +1492,9 @@ class TestEnv : public EnvWrapper {
CloseHelper();
}
}
virtual void Logv(const char *format, va_list ap) override { };
protected:
virtual void Logv(const char* /*format*/, va_list /*ap*/) override{};
protected:
virtual Status CloseImpl() override {
return CloseHelper();
}
@@ -1506,7 +1510,7 @@ class TestEnv : public EnvWrapper {
int GetCloseCount() { return close_count; }
virtual Status NewLogger(const std::string& fname,
virtual Status NewLogger(const std::string& /*fname*/,
shared_ptr<Logger>* result) {
result->reset(new TestLogger(this));
return Status::OK();
+2 -2
View File
@@ -448,7 +448,7 @@ PosixMmapReadableFile::~PosixMmapReadableFile() {
}
Status PosixMmapReadableFile::Read(uint64_t offset, size_t n, Slice* result,
char* scratch) const {
char* /*scratch*/) const {
Status s;
if (offset > length_) {
*result = Slice();
@@ -941,7 +941,7 @@ size_t PosixWritableFile::GetUniqueId(char* id, size_t max_size) const {
*/
PosixRandomRWFile::PosixRandomRWFile(const std::string& fname, int fd,
const EnvOptions& options)
const EnvOptions& /*options*/)
: filename_(fname), fd_(fd) {}
PosixRandomRWFile::~PosixRandomRWFile() {
+1 -1
View File
@@ -202,7 +202,7 @@ class PosixMmapFile : public WritableFile {
// Means Close() will properly take care of truncate
// and it does not need any additional information
virtual Status Truncate(uint64_t size) override { return Status::OK(); }
virtual Status Truncate(uint64_t /*size*/) override { return Status::OK(); }
virtual Status Close() override;
virtual Status Append(const Slice& data) override;
virtual Status Flush() override;
+7 -7
View File
@@ -447,8 +447,8 @@ MockEnv::~MockEnv() {
// Partial implementation of the Env interface.
Status MockEnv::NewSequentialFile(const std::string& fname,
unique_ptr<SequentialFile>* result,
const EnvOptions& soptions) {
unique_ptr<SequentialFile>* result,
const EnvOptions& /*soptions*/) {
auto fn = NormalizePath(fname);
MutexLock lock(&mutex_);
if (file_map_.find(fn) == file_map_.end()) {
@@ -464,8 +464,8 @@ Status MockEnv::NewSequentialFile(const std::string& fname,
}
Status MockEnv::NewRandomAccessFile(const std::string& fname,
unique_ptr<RandomAccessFile>* result,
const EnvOptions& soptions) {
unique_ptr<RandomAccessFile>* result,
const EnvOptions& /*soptions*/) {
auto fn = NormalizePath(fname);
MutexLock lock(&mutex_);
if (file_map_.find(fn) == file_map_.end()) {
@@ -482,7 +482,7 @@ Status MockEnv::NewRandomAccessFile(const std::string& fname,
Status MockEnv::NewRandomRWFile(const std::string& fname,
unique_ptr<RandomRWFile>* result,
const EnvOptions& soptions) {
const EnvOptions& /*soptions*/) {
auto fn = NormalizePath(fname);
MutexLock lock(&mutex_);
if (file_map_.find(fn) == file_map_.end()) {
@@ -525,8 +525,8 @@ Status MockEnv::NewWritableFile(const std::string& fname,
return Status::OK();
}
Status MockEnv::NewDirectory(const std::string& name,
unique_ptr<Directory>* result) {
Status MockEnv::NewDirectory(const std::string& /*name*/,
unique_ptr<Directory>* result) {
result->reset(new MockEnvDirectory());
return Status::OK();
}
+61 -48
View File
@@ -245,7 +245,7 @@ static const Status notsup;
class HdfsEnv : public Env {
public:
explicit HdfsEnv(const std::string& fsname) {
explicit HdfsEnv(const std::string& /*fsname*/) {
fprintf(stderr, "You have not build rocksdb with HDFS support\n");
fprintf(stderr, "Please see hdfs/README for details\n");
abort();
@@ -258,112 +258,125 @@ class HdfsEnv : public Env {
unique_ptr<SequentialFile>* result,
const EnvOptions& options) override;
virtual Status NewRandomAccessFile(const std::string& fname,
unique_ptr<RandomAccessFile>* result,
const EnvOptions& options) override {
virtual Status NewRandomAccessFile(const std::string& /*fname*/,
unique_ptr<RandomAccessFile>* /*result*/,
const EnvOptions& /*options*/) override {
return notsup;
}
virtual Status NewWritableFile(const std::string& fname,
unique_ptr<WritableFile>* result,
const EnvOptions& options) override {
virtual Status NewWritableFile(const std::string& /*fname*/,
unique_ptr<WritableFile>* /*result*/,
const EnvOptions& /*options*/) override {
return notsup;
}
virtual Status NewDirectory(const std::string& name,
unique_ptr<Directory>* result) override {
virtual Status NewDirectory(const std::string& /*name*/,
unique_ptr<Directory>* /*result*/) override {
return notsup;
}
virtual Status FileExists(const std::string& fname) override {
virtual Status FileExists(const std::string& /*fname*/) override {
return notsup;
}
virtual Status GetChildren(const std::string& path,
std::vector<std::string>* result) override {
virtual Status GetChildren(const std::string& /*path*/,
std::vector<std::string>* /*result*/) override {
return notsup;
}
virtual Status DeleteFile(const std::string& fname) override {
virtual Status DeleteFile(const std::string& /*fname*/) override {
return notsup;
}
virtual Status CreateDir(const std::string& name) override { return notsup; }
virtual Status CreateDirIfMissing(const std::string& name) override {
virtual Status CreateDir(const std::string& /*name*/) override {
return notsup;
}
virtual Status DeleteDir(const std::string& name) override { return notsup; }
virtual Status GetFileSize(const std::string& fname,
uint64_t* size) override {
virtual Status CreateDirIfMissing(const std::string& /*name*/) override {
return notsup;
}
virtual Status GetFileModificationTime(const std::string& fname,
uint64_t* time) override {
virtual Status DeleteDir(const std::string& /*name*/) override {
return notsup;
}
virtual Status RenameFile(const std::string& src,
const std::string& target) override {
virtual Status GetFileSize(const std::string& /*fname*/,
uint64_t* /*size*/) override {
return notsup;
}
virtual Status LinkFile(const std::string& src,
const std::string& target) override {
virtual Status GetFileModificationTime(const std::string& /*fname*/,
uint64_t* /*time*/) override {
return notsup;
}
virtual Status LockFile(const std::string& fname, FileLock** lock) override {
virtual Status RenameFile(const std::string& /*src*/,
const std::string& /*target*/) override {
return notsup;
}
virtual Status UnlockFile(FileLock* lock) override { return notsup; }
virtual Status NewLogger(const std::string& fname,
shared_ptr<Logger>* result) override {
virtual Status LinkFile(const std::string& /*src*/,
const std::string& /*target*/) override {
return notsup;
}
virtual void Schedule(void (*function)(void* arg), void* arg,
Priority pri = LOW, void* tag = nullptr,
void (*unschedFunction)(void* arg) = 0) override {}
virtual Status LockFile(const std::string& /*fname*/,
FileLock** /*lock*/) override {
return notsup;
}
virtual int UnSchedule(void* tag, Priority pri) override { return 0; }
virtual Status UnlockFile(FileLock* /*lock*/) override { return notsup; }
virtual void StartThread(void (*function)(void* arg), void* arg) override {}
virtual Status NewLogger(const std::string& /*fname*/,
shared_ptr<Logger>* /*result*/) override {
return notsup;
}
virtual void Schedule(void (* /*function*/)(void* arg), void* /*arg*/,
Priority /*pri*/ = LOW, void* /*tag*/ = nullptr,
void (* /*unschedFunction*/)(void* arg) = 0) override {}
virtual int UnSchedule(void* /*tag*/, Priority /*pri*/) override { return 0; }
virtual void StartThread(void (* /*function*/)(void* arg),
void* /*arg*/) override {}
virtual void WaitForJoin() override {}
virtual unsigned int GetThreadPoolQueueLen(
Priority pri = LOW) const override {
Priority /*pri*/ = LOW) const override {
return 0;
}
virtual Status GetTestDirectory(std::string* path) override { return notsup; }
virtual Status GetTestDirectory(std::string* /*path*/) override {
return notsup;
}
virtual uint64_t NowMicros() override { return 0; }
virtual void SleepForMicroseconds(int micros) override {}
virtual void SleepForMicroseconds(int /*micros*/) override {}
virtual Status GetHostName(char* name, uint64_t len) override {
virtual Status GetHostName(char* /*name*/, uint64_t /*len*/) override {
return notsup;
}
virtual Status GetCurrentTime(int64_t* unix_time) override { return notsup; }
virtual Status GetAbsolutePath(const std::string& db_path,
std::string* outputpath) override {
virtual Status GetCurrentTime(int64_t* /*unix_time*/) override {
return notsup;
}
virtual void SetBackgroundThreads(int number, Priority pri = LOW) override {}
virtual int GetBackgroundThreads(Priority pri = LOW) override { return 0; }
virtual void IncBackgroundThreadsIfNeeded(int number, Priority pri) override {
virtual Status GetAbsolutePath(const std::string& /*db_path*/,
std::string* /*outputpath*/) override {
return notsup;
}
virtual std::string TimeToString(uint64_t number) override { return ""; }
virtual void SetBackgroundThreads(int /*number*/,
Priority /*pri*/ = LOW) override {}
virtual int GetBackgroundThreads(Priority /*pri*/ = LOW) override {
return 0;
}
virtual void IncBackgroundThreadsIfNeeded(int /*number*/,
Priority /*pri*/) override {}
virtual std::string TimeToString(uint64_t /*number*/) override { return ""; }
virtual uint64_t GetThreadID() const override {
return 0;
+2 -1
View File
@@ -216,7 +216,8 @@ class Cache {
// Mark the last inserted object as being a raw data block. This will be used
// in tests. The default implementation does nothing.
virtual void TEST_mark_as_data_block(const Slice& key, size_t charge) {}
virtual void TEST_mark_as_data_block(const Slice& /*key*/,
size_t /*charge*/) {}
private:
// No copying allowed
+7 -5
View File
@@ -94,8 +94,10 @@ class CompactionFilter {
// be used by a single thread that is doing the compaction run, and this
// call does not need to be thread-safe. However, multiple filters may be
// in existence and operating concurrently.
virtual bool Filter(int level, const Slice& key, const Slice& existing_value,
std::string* new_value, bool* value_changed) const {
virtual bool Filter(int /*level*/, const Slice& /*key*/,
const Slice& /*existing_value*/,
std::string* /*new_value*/,
bool* /*value_changed*/) const {
return false;
}
@@ -108,8 +110,8 @@ class CompactionFilter {
// may not realize there is a write conflict and may allow a Transaction to
// Commit that should have failed. Instead, it is better to implement any
// Merge filtering inside the MergeOperator.
virtual bool FilterMergeOperand(int level, const Slice& key,
const Slice& operand) const {
virtual bool FilterMergeOperand(int /*level*/, const Slice& /*key*/,
const Slice& /*operand*/) const {
return false;
}
@@ -154,7 +156,7 @@ class CompactionFilter {
// MergeOperator.
virtual Decision FilterV2(int level, const Slice& key, ValueType value_type,
const Slice& existing_value, std::string* new_value,
std::string* skip_until) const {
std::string* /*skip_until*/) const {
switch (value_type) {
case ValueType::kValue: {
bool value_changed = false;
+15 -8
View File
@@ -93,8 +93,8 @@ static const int kMinorVersion = __ROCKSDB_MINOR__;
// A range of keys
struct Range {
Slice start; // Included in the range
Slice limit; // Not included in the range
Slice start;
Slice limit;
Range() { }
Range(const Slice& s, const Slice& l) : start(s), limit(l) { }
@@ -174,7 +174,8 @@ class DB {
// Close the DB by releasing resources, closing files etc. This should be
// called before calling the desctructor so that the caller can get back a
// status in case there are any errors. This will not fsync the WAL files.
// If syncing is required, the caller must first call SyncWAL.
// If syncing is required, the caller must first call SyncWAL(), or Write()
// using an empty write batch with WriteOptions.sync=true.
// Regardless of the return status, the DB must be freed. If the return
// status is NotSupported(), then the DB implementation does cleanup in the
// destructor
@@ -575,6 +576,10 @@ class DB {
// WARNING: may slow down online queries if there are too many files.
static const std::string kTotalSstFilesSize;
// "rocksdb.live-sst-files-size" - returns total size (bytes) of all SST
// files belong to the latest LSM tree.
static const std::string kLiveSstFilesSize;
// "rocksdb.base-level" - returns number of level to which L0 data will be
// compacted.
static const std::string kBaseLevel;
@@ -650,6 +655,7 @@ class DB {
// "rocksdb.estimate-live-data-size"
// "rocksdb.min-log-number-to-keep"
// "rocksdb.total-sst-files-size"
// "rocksdb.live-sst-files-size"
// "rocksdb.base-level"
// "rocksdb.estimate-pending-compaction-bytes"
// "rocksdb.num-running-compactions"
@@ -880,7 +886,7 @@ class DB {
// Flush the WAL memory buffer to the file. If sync is true, it calls SyncWAL
// afterwards.
virtual Status FlushWAL(bool sync) {
virtual Status FlushWAL(bool /*sync*/) {
return Status::NotSupported("FlushWAL not implemented");
}
// Sync the wal. Note that Write() followed by SyncWAL() is not exactly the
@@ -1134,13 +1140,14 @@ class DB {
ColumnFamilyHandle* column_family, const Range* range, std::size_t n,
TablePropertiesCollection* props) = 0;
virtual Status SuggestCompactRange(ColumnFamilyHandle* column_family,
const Slice* begin, const Slice* end) {
virtual Status SuggestCompactRange(ColumnFamilyHandle* /*column_family*/,
const Slice* /*begin*/,
const Slice* /*end*/) {
return Status::NotSupported("SuggestCompactRange() is not implemented.");
}
virtual Status PromoteL0(ColumnFamilyHandle* column_family,
int target_level) {
virtual Status PromoteL0(ColumnFamilyHandle* /*column_family*/,
int /*target_level*/) {
return Status::NotSupported("PromoteL0() is not implemented.");
}
+28 -27
View File
@@ -181,9 +181,9 @@ class Env {
// returns non-OK.
//
// The returned file will only be accessed by one thread at a time.
virtual Status ReopenWritableFile(const std::string& fname,
unique_ptr<WritableFile>* result,
const EnvOptions& options) {
virtual Status ReopenWritableFile(const std::string& /*fname*/,
unique_ptr<WritableFile>* /*result*/,
const EnvOptions& /*options*/) {
return Status::NotSupported();
}
@@ -198,9 +198,9 @@ class Env {
// *result and returns OK. On failure returns non-OK.
//
// The returned file will only be accessed by one thread at a time.
virtual Status NewRandomRWFile(const std::string& fname,
unique_ptr<RandomRWFile>* result,
const EnvOptions& options) {
virtual Status NewRandomRWFile(const std::string& /*fname*/,
unique_ptr<RandomRWFile>* /*result*/,
const EnvOptions& /*options*/) {
return Status::NotSupported("RandomRWFile is not implemented in this Env");
}
@@ -268,12 +268,13 @@ class Env {
const std::string& target) = 0;
// Hard Link file src to target.
virtual Status LinkFile(const std::string& src, const std::string& target) {
virtual Status LinkFile(const std::string& /*src*/,
const std::string& /*target*/) {
return Status::NotSupported("LinkFile is not supported for this Env");
}
virtual Status AreFilesSame(const std::string& first,
const std::string& second, bool* res) {
virtual Status AreFilesSame(const std::string& /*first*/,
const std::string& /*second*/, bool* /*res*/) {
return Status::NotSupported("AreFilesSame is not supported for this Env");
}
@@ -324,7 +325,7 @@ class Env {
// Arrange to remove jobs for given arg from the queue_ if they are not
// already scheduled. Caller is expected to have exclusive lock on arg.
virtual int UnSchedule(void* arg, Priority pri) { return 0; }
virtual int UnSchedule(void* /*arg*/, Priority /*pri*/) { return 0; }
// Start a new thread, invoking "function(arg)" within the new thread.
// When "function(arg)" returns, the thread will be destroyed.
@@ -334,7 +335,7 @@ class Env {
virtual void WaitForJoin() {}
// Get thread pool queue length for specific thread pool.
virtual unsigned int GetThreadPoolQueueLen(Priority pri = LOW) const {
virtual unsigned int GetThreadPoolQueueLen(Priority /*pri*/ = LOW) const {
return 0;
}
@@ -388,7 +389,7 @@ class Env {
virtual void IncBackgroundThreadsIfNeeded(int number, Priority pri) = 0;
// Lower IO priority for threads from the specified pool.
virtual void LowerThreadPoolIOPriority(Priority pool = LOW) {}
virtual void LowerThreadPoolIOPriority(Priority /*pool*/ = LOW) {}
// Converts seconds-since-Jan-01-1970 to a printable string
virtual std::string TimeToString(uint64_t time) = 0;
@@ -432,7 +433,7 @@ class Env {
const ImmutableDBOptions& db_options) const;
// Returns the status of all threads that belong to the current Env.
virtual Status GetThreadList(std::vector<ThreadStatus>* thread_list) {
virtual Status GetThreadList(std::vector<ThreadStatus>* /*thread_list*/) {
return Status::NotSupported("Not supported.");
}
@@ -498,14 +499,14 @@ class SequentialFile {
// Remove any kind of caching of data from the offset to offset+length
// of this file. If the length is 0, then it refers to the end of file.
// If the system is not caching the file contents, then this is a noop.
virtual Status InvalidateCache(size_t offset, size_t length) {
virtual Status InvalidateCache(size_t /*offset*/, size_t /*length*/) {
return Status::NotSupported("InvalidateCache not supported.");
}
// Positioned Read for direct I/O
// If Direct I/O enabled, offset, n, and scratch should be properly aligned
virtual Status PositionedRead(uint64_t offset, size_t n, Slice* result,
char* scratch) {
virtual Status PositionedRead(uint64_t /*offset*/, size_t /*n*/,
Slice* /*result*/, char* /*scratch*/) {
return Status::NotSupported();
}
};
@@ -531,7 +532,7 @@ class RandomAccessFile {
char* scratch) const = 0;
// Readahead the file starting from offset by n bytes for caching.
virtual Status Prefetch(uint64_t offset, size_t n) {
virtual Status Prefetch(uint64_t /*offset*/, size_t /*n*/) {
return Status::OK();
}
@@ -550,14 +551,14 @@ class RandomAccessFile {
// a single varint.
//
// Note: these IDs are only valid for the duration of the process.
virtual size_t GetUniqueId(char* id, size_t max_size) const {
virtual size_t GetUniqueId(char* /*id*/, size_t /*max_size*/) const {
return 0; // Default implementation to prevent issues with backwards
// compatibility.
};
enum AccessPattern { NORMAL, RANDOM, SEQUENTIAL, WILLNEED, DONTNEED };
virtual void Hint(AccessPattern pattern) {}
virtual void Hint(AccessPattern /*pattern*/) {}
// Indicates the upper layers if the current RandomAccessFile implementation
// uses direct IO.
@@ -570,7 +571,7 @@ class RandomAccessFile {
// Remove any kind of caching of data from the offset to offset+length
// of this file. If the length is 0, then it refers to the end of file.
// If the system is not caching the file contents, then this is a noop.
virtual Status InvalidateCache(size_t offset, size_t length) {
virtual Status InvalidateCache(size_t /*offset*/, size_t /*length*/) {
return Status::NotSupported("InvalidateCache not supported.");
}
};
@@ -621,9 +622,7 @@ class WritableFile {
// before closing. It is not always possible to keep track of the file
// size due to whole pages writes. The behavior is undefined if called
// with other writes to follow.
virtual Status Truncate(uint64_t size) {
return Status::OK();
}
virtual Status Truncate(uint64_t /*size*/) { return Status::OK(); }
virtual Status Close() = 0;
virtual Status Flush() = 0;
virtual Status Sync() = 0; // sync data
@@ -690,7 +689,7 @@ class WritableFile {
}
// For documentation, refer to RandomAccessFile::GetUniqueId()
virtual size_t GetUniqueId(char* id, size_t max_size) const {
virtual size_t GetUniqueId(char* /*id*/, size_t /*max_size*/) const {
return 0; // Default implementation to prevent issues with backwards
}
@@ -698,7 +697,7 @@ class WritableFile {
// of this file. If the length is 0, then it refers to the end of file.
// If the system is not caching the file contents, then this is a noop.
// This call has no effect on dirty pages in the cache.
virtual Status InvalidateCache(size_t offset, size_t length) {
virtual Status InvalidateCache(size_t /*offset*/, size_t /*length*/) {
return Status::NotSupported("InvalidateCache not supported.");
}
@@ -708,7 +707,9 @@ class WritableFile {
// This asks the OS to initiate flushing the cached data to disk,
// without waiting for completion.
// Default implementation does nothing.
virtual Status RangeSync(uint64_t offset, uint64_t nbytes) { return Status::OK(); }
virtual Status RangeSync(uint64_t /*offset*/, uint64_t /*nbytes*/) {
return Status::OK();
}
// PrepareWrite performs any necessary preparation for a write
// before the write actually occurs. This allows for pre-allocation
@@ -735,7 +736,7 @@ class WritableFile {
}
// Pre-allocates space for a file.
virtual Status Allocate(uint64_t offset, uint64_t len) {
virtual Status Allocate(uint64_t /*offset*/, uint64_t /*len*/) {
return Status::OK();
}
+3 -2
View File
@@ -50,7 +50,7 @@ class FilterBitsBuilder {
#pragma warning(push)
#pragma warning(disable : 4702) // unreachable code
#endif
virtual int CalculateNumEntry(const uint32_t space) {
virtual int CalculateNumEntry(const uint32_t /*space*/) {
#ifndef ROCKSDB_LITE
throw std::runtime_error("CalculateNumEntry not Implemented");
#else
@@ -122,7 +122,8 @@ class FilterPolicy {
// Get the FilterBitsReader, which is ONLY used for full filter block
// It contains interface to tell if key can be in filter
// The input slice should NOT be deleted by FilterPolicy
virtual FilterBitsReader* GetFilterBitsReader(const Slice& contents) const {
virtual FilterBitsReader* GetFilterBitsReader(
const Slice& /*contents*/) const {
return nullptr;
}
};
+2 -32
View File
@@ -227,30 +227,6 @@ struct ExternalFileIngestionInfo {
TableProperties table_properties;
};
// A call-back function to RocksDB which will be called when the compaction
// iterator is compacting values. It is meant to be returned from
// EventListner::GetCompactionEventListner() at the beginning of compaction
// job.
class CompactionEventListener {
public:
enum CompactionListenerValueType {
kValue,
kMergeOperand,
kDelete,
kSingleDelete,
kRangeDelete,
kBlobIndex,
kInvalid,
};
virtual void OnCompaction(int level, const Slice& key,
CompactionListenerValueType value_type,
const Slice& existing_value,
const SequenceNumber& sn, bool is_new) = 0;
virtual ~CompactionEventListener() = default;
};
// EventListener class contains a set of call-back functions that will
// be called when specific RocksDB event happens such as flush. It can
// be used as a building block for developing custom features such as
@@ -379,8 +355,8 @@ class EventListener {
// returns. Otherwise, RocksDB may be blocked.
// @param handle is a pointer to the column family handle to be deleted
// which will become a dangling pointer after the deletion.
virtual void OnColumnFamilyHandleDeletionStarted(ColumnFamilyHandle* handle) {
}
virtual void OnColumnFamilyHandleDeletionStarted(
ColumnFamilyHandle* /*handle*/) {}
// A call-back function for RocksDB which will be called after an external
// file is ingested using IngestExternalFile.
@@ -413,12 +389,6 @@ class EventListener {
// returns. Otherwise, RocksDB may be blocked.
virtual void OnStallConditionsChanged(const WriteStallInfo& /*info*/) {}
// Factory method to return CompactionEventListener. If multiple listeners
// provides CompactionEventListner, only the first one will be used.
virtual CompactionEventListener* GetCompactionEventListener() {
return nullptr;
}
virtual ~EventListener() {}
};
+3 -3
View File
@@ -97,7 +97,7 @@ class MemTableRep {
//
// Currently only skip-list based memtable implement the interface. Other
// implementations will fallback to Insert() by default.
virtual void InsertWithHint(KeyHandle handle, void** hint) {
virtual void InsertWithHint(KeyHandle handle, void** /*hint*/) {
// Ignore the hint by default.
Insert(handle);
}
@@ -149,8 +149,8 @@ class MemTableRep {
virtual void Get(const LookupKey& k, void* callback_args,
bool (*callback_func)(void* arg, const char* entry));
virtual uint64_t ApproximateNumEntries(const Slice& start_ikey,
const Slice& end_key) {
virtual uint64_t ApproximateNumEntries(const Slice& /*start_ikey*/,
const Slice& /*end_key*/) {
return 0;
}
+8 -9
View File
@@ -66,11 +66,9 @@ class MergeOperator {
// internal corruption. This will be treated as an error by the library.
//
// Also make use of the *logger for error messages.
virtual bool FullMerge(const Slice& key,
const Slice* existing_value,
const std::deque<std::string>& operand_list,
std::string* new_value,
Logger* logger) const {
virtual bool FullMerge(const Slice& /*key*/, const Slice* /*existing_value*/,
const std::deque<std::string>& /*operand_list*/,
std::string* /*new_value*/, Logger* /*logger*/) const {
// deprecated, please use FullMergeV2()
assert(false);
return false;
@@ -145,9 +143,10 @@ class MergeOperator {
// If there is corruption in the data, handle it in the FullMergeV2() function
// and return false there. The default implementation of PartialMerge will
// always return false.
virtual bool PartialMerge(const Slice& key, const Slice& left_operand,
const Slice& right_operand, std::string* new_value,
Logger* logger) const {
virtual bool PartialMerge(const Slice& /*key*/, const Slice& /*left_operand*/,
const Slice& /*right_operand*/,
std::string* /*new_value*/,
Logger* /*logger*/) const {
return false;
}
@@ -196,7 +195,7 @@ class MergeOperator {
// during a point lookup, thereby helping in limiting the number of levels to
// read from.
// Doesn't help with iterators.
virtual bool ShouldMerge(const std::vector<Slice>& operands) const {
virtual bool ShouldMerge(const std::vector<Slice>& /*operands*/) const {
return false;
}
};
+1 -1
View File
@@ -45,7 +45,7 @@ class RateLimiter {
// Request for token for bytes. If this request can not be satisfied, the call
// is blocked. Caller is responsible to make sure
// bytes <= GetSingleBurstBytes()
virtual void Request(const int64_t bytes, const Env::IOPriority pri) {
virtual void Request(const int64_t /*bytes*/, const Env::IOPriority /*pri*/) {
assert(false);
}
+1 -1
View File
@@ -177,7 +177,7 @@ class PinnableSlice : public Slice, public Cleanable {
}
}
void remove_prefix(size_t n) {
void remove_prefix(size_t /*n*/) {
assert(0); // Not implemented
}
+2 -2
View File
@@ -58,7 +58,7 @@ class SliceTransform {
virtual bool InDomain(const Slice& key) const = 0;
// This is currently not used and remains here for backward compatibility.
virtual bool InRange(const Slice& dst) const { return false; }
virtual bool InRange(const Slice& /*dst*/) const { return false; }
// Transform(s)=Transform(`prefix`) for any s with `prefix` as a prefix.
//
@@ -83,7 +83,7 @@ class SliceTransform {
// "abcd,e", the file can be filtered out and the key will be invisible.
//
// i.e., an implementation always returning false is safe.
virtual bool SameResultWhenAppended(const Slice& prefix) const {
virtual bool SameResultWhenAppended(const Slice& /*prefix*/) const {
return false;
}
};
+1 -1
View File
@@ -597,7 +597,7 @@ class Statistics {
virtual uint64_t getTickerCount(uint32_t tickerType) const = 0;
virtual void histogramData(uint32_t type,
HistogramData* const data) const = 0;
virtual std::string getHistogramString(uint32_t type) const { return ""; }
virtual std::string getHistogramString(uint32_t /*type*/) const { return ""; }
virtual void recordTick(uint32_t tickerType, uint64_t count = 0) = 0;
virtual void setTickerCount(uint32_t tickerType, uint64_t count) = 0;
virtual uint64_t getAndResetTickerCount(uint32_t tickerType) = 0;
+2 -2
View File
@@ -467,8 +467,8 @@ class TableFactory {
// RocksDB prints configurations at DB Open().
virtual std::string GetPrintableTableOptions() const = 0;
virtual Status GetOptionString(std::string* opt_string,
const std::string& delimiter) const {
virtual Status GetOptionString(std::string* /*opt_string*/,
const std::string& /*delimiter*/) const {
return Status::NotSupported(
"The table factory doesn't implement GetOptionString().");
}
+1 -1
View File
@@ -80,7 +80,7 @@ class GeoDB : public StackableDB {
// GeoDB owns the pointer `DB* db` now. You should not delete it or
// use it after the invocation of GeoDB
// GeoDB(DB* db, const GeoDBOptions& options) : StackableDB(db) {}
GeoDB(DB* db, const GeoDBOptions& options) : StackableDB(db) {}
GeoDB(DB* db, const GeoDBOptions& /*options*/) : StackableDB(db) {}
virtual ~GeoDB() {}
// Insert a new object into the location database. The object is
@@ -62,7 +62,7 @@ class OptimisticTransactionDB {
protected:
// To Create an OptimisticTransactionDB, call Open()
explicit OptimisticTransactionDB(DB* db) {}
explicit OptimisticTransactionDB(DB* /*db*/) {}
OptimisticTransactionDB() {}
private:
+5 -5
View File
@@ -233,9 +233,9 @@ class Transaction {
// An overload of the the above method that receives a PinnableSlice
// For backward compatiblity a default implementation is provided
virtual Status GetForUpdate(const ReadOptions& options,
ColumnFamilyHandle* column_family,
ColumnFamilyHandle* /*column_family*/,
const Slice& key, PinnableSlice* pinnable_val,
bool exclusive = true) {
bool /*exclusive*/ = true) {
if (pinnable_val == nullptr) {
std::string* null_str = nullptr;
return GetForUpdate(options, key, null_str);
@@ -440,8 +440,8 @@ class Transaction {
virtual bool IsDeadlockDetect() const { return false; }
virtual std::vector<TransactionID> GetWaitingTxns(uint32_t* column_family_id,
std::string* key) const {
virtual std::vector<TransactionID> GetWaitingTxns(
uint32_t* /*column_family_id*/, std::string* /*key*/) const {
assert(false);
return std::vector<TransactionID>();
}
@@ -469,7 +469,7 @@ class Transaction {
uint64_t GetId() { return id_; }
protected:
explicit Transaction(const TransactionDB* db) {}
explicit Transaction(const TransactionDB* /*db*/) {}
Transaction() : log_number_(0), txn_state_(STARTED) {}
// the log in which the prepared section for this txn resides
+8 -10
View File
@@ -44,8 +44,8 @@ class WalFilter {
// @params cf_name_id_map column_family_name to column_family_id map
virtual void ColumnFamilyLogNumberMap(
const std::map<uint32_t, uint64_t>& cf_lognumber_map,
const std::map<std::string, uint32_t>& cf_name_id_map) {}
const std::map<uint32_t, uint64_t>& /*cf_lognumber_map*/,
const std::map<std::string, uint32_t>& /*cf_name_id_map*/) {}
// LogRecord is invoked for each log record encountered for all the logs
// during replay on logs on recovery. This method can be used to:
@@ -75,11 +75,9 @@ class WalFilter {
// @returns Processing option for the current record.
// Please see WalProcessingOption enum above for
// details.
virtual WalProcessingOption LogRecordFound(unsigned long long log_number,
const std::string& log_file_name,
const WriteBatch& batch,
WriteBatch* new_batch,
bool* batch_changed) {
virtual WalProcessingOption LogRecordFound(
unsigned long long /*log_number*/, const std::string& /*log_file_name*/,
const WriteBatch& batch, WriteBatch* new_batch, bool* batch_changed) {
// Default implementation falls back to older function for compatibility
return LogRecord(batch, new_batch, batch_changed);
}
@@ -87,9 +85,9 @@ class WalFilter {
// Please see the comments for LogRecord above. This function is for
// compatibility only and contains a subset of parameters.
// New code should use the function above.
virtual WalProcessingOption LogRecord(const WriteBatch& batch,
WriteBatch* new_batch,
bool* batch_changed) const {
virtual WalProcessingOption LogRecord(const WriteBatch& /*batch*/,
WriteBatch* /*new_batch*/,
bool* /*batch_changed*/) const {
return WalProcessingOption::kContinueProcessing;
}
Loaded 100 of 236 files, more files were not shown because too many files have changed in this diff. Show more