Fix EnvLibrados and add to CI (#9088 )

Summary: This feature was not part of any common or CI build, so no surprise it broke. Now we can at least ensure compilation. I don't know how to run the test successfully (missing config file) so it is bypassed for now. Fixes https://github.com/facebook/rocksdb/issues/9078 Pull Request resolved: https://github.com/facebook/rocksdb/pull/9088 Test Plan: CI Reviewed By: mrambacher Differential Revision: D32009467 Pulled By: pdillinger fbshipit-source-id: 3e0d1e5fde7f0ece703d48a81479e1cc7392c25c
update HISTORY.md and version.h for 6.25.3
2026-11-06 11:02:36 -05:00 · 2021-10-29 09:21:24 -07:00 · 2021-10-14 10:46:37 -07:00 · 2021-10-14 10:45:19 -07:00 · 2021-10-14 10:44:46 -07:00 · 2021-10-11 16:42:46 -07:00
318 changed files with 2258 additions and 12374 deletions
@@ -387,16 +387,6 @@ jobs:
      - run: CC=gcc-10 CXX=g++-10 V=1 SKIP_LINK=1 ROCKSDB_CXX_STANDARD=c++20 make -j16 all | .circleci/cat_ignore_eagain # Linking broken because libgflags compiled with newer ABI
      - post-steps

-  build-linux-gcc-11-no_test_run:
-    machine:
-      image: ubuntu-2004:202010-01
-    resource_class: xlarge
-    steps:
-      - pre-steps
-      - run: sudo apt-get update -y && sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test && sudo apt-get install gcc-11 g++-11 libgflags-dev
-      - run: CC=gcc-11 CXX=g++-11 V=1 SKIP_LINK=1 make -j16 all | .circleci/cat_ignore_eagain # Linking broken because libgflags compiled with newer ABI
-      - post-steps
-
  # This job is only to make sure the microbench tests are able to run, the benchmark result is not meaningful as the CI host is changing.
  build-linux-microbench:
    machine:
@@ -797,7 +787,6 @@ workflows:
      - build-linux-gcc-8-no_test_run
      - build-linux-gcc-9-no_test_run
      - build-linux-gcc-10-cxx20-no_test_run
-      - build-linux-gcc-11-no_test_run
      - build-linux-arm-cmake-no_test_run
  build-macos:
    jobs:
@@ -713,7 +713,7 @@ set(SOURCES
        env/file_system_tracer.cc
        env/fs_remap.cc
        env/mock_env.cc
-        env/unique_id_gen.cc
+        env/unique_id.cc
        file/delete_scheduler.cc
        file/file_prefetch_buffer.cc
        file/file_util.cc
@@ -807,7 +807,6 @@ set(SOURCES
        table/table_factory.cc
        table/table_properties.cc
        table/two_level_iterator.cc
-        table/unique_id.cc
        test_util/sync_point.cc
        test_util/sync_point_impl.cc
        test_util/testutil.cc
@@ -852,8 +851,6 @@ set(SOURCES
        utilities/blob_db/blob_db_impl_filesnapshot.cc
        utilities/blob_db/blob_dump_tool.cc
        utilities/blob_db/blob_file.cc
-        utilities/cache_dump_load.cc
-        utilities/cache_dump_load_impl.cc
        utilities/cassandra/cassandra_compaction_filter.cc
        utilities/cassandra/format.cc
        utilities/cassandra/merge_operator.cc
@@ -906,7 +903,6 @@ set(SOURCES
        utilities/transactions/write_unprepared_txn.cc
        utilities/transactions/write_unprepared_txn_db.cc
        utilities/ttl/db_ttl_impl.cc
-        utilities/wal_filter.cc
        utilities/write_batch_with_index/write_batch_with_index.cc
        utilities/write_batch_with_index/write_batch_with_index_internal.cc)

@@ -1,47 +1,18 @@
-# Rocksdb Change Lo
-## 6.26.1 (2021-11-18)
+# Rocksdb Change Log
+## 6.25.3 (2021-10-14)
 ### Bug Fixes
-* Fix builds for some platforms.
-
-## 6.26.0 (2021-10-20)
-### Bug Fixes
-* Fixes a bug in directed IO mode when calling MultiGet() for blobs in the same blob file. The bug is caused by not sorting the blob read requests by file offsets.
-* Fix the incorrect disabling of SST rate limited deletion when the WAL and DB are in different directories. Only WAL rate limited deletion should be disabled if its in a different directory.
-* Fix `DisableManualCompaction()` to cancel compactions even when they are waiting on automatic compactions to drain due to `CompactRangeOptions::exclusive_manual_compactions == true`.
-* Fix contract of `Env::ReopenWritableFile()` and `FileSystem::ReopenWritableFile()` to specify any existing file must not be deleted or truncated.
 * Fixed bug in calls to `IngestExternalFiles()` with files for multiple column families. The bug could have introduced a delay in ingested file keys becoming visible after `IngestExternalFiles()` returned. Furthermore, mutations to ingested file keys while they were invisible could have been dropped (not necessarily immediately).
 * Fixed a possible race condition impacting users of `WriteBufferManager` who constructed it with `allow_stall == true`. The race condition led to undefined behavior (in our experience, typically a process crash).
 * Fixed a bug where stalled writes would remain stalled forever after the user calls `WriteBufferManager::SetBufferSize()` with `new_size == 0` to dynamically disable memory limiting.
-* Make `DB::close()` thread-safe.
-* Fix a bug in atomic flush where one bg flush thread will wait forever for a preceding bg flush thread to commit its result to MANIFEST but encounters an error which is mapped to a soft error (DB not stopped).

-### New Features
-* Print information about blob files when using "ldb list_live_files_metadata"
-* Provided support for SingleDelete with user defined timestamp.
-* Experimental new function DB::GetLiveFilesStorageInfo offers essentially a unified version of other functions like GetLiveFiles, GetLiveFilesChecksumInfo, and GetSortedWalFiles. Checkpoints and backups could show small behavioral changes and/or improved performance as they now use this new API.
-* Add remote compaction read/write bytes statistics: `REMOTE_COMPACT_READ_BYTES`, `REMOTE_COMPACT_WRITE_BYTES`.
-* Introduce an experimental feature to dump out the blocks from block cache and insert them to the secondary cache to reduce the cache warmup time (e.g., used while migrating DB instance). More information are in `class CacheDumper` and `CacheDumpedLoader` at `rocksdb/utilities/cache_dump_load.h` Note that, this feature is subject to the potential change in the future, it is still experimental.
-* Introduced a new BlobDB configuration option `blob_garbage_collection_force_threshold`, which can be used to trigger compactions targeting the SST files which reference the oldest blob files when the ratio of garbage in those blob files meets or exceeds the specified threshold. This can reduce space amplification with skewed workloads where the affected SST files might not otherwise get picked up for compaction.
-* Added EXPERIMENTAL support for table file (SST) unique identifiers that are stable and universally unique, available with new function `GetUniqueIdFromTableProperties`. Only SST files from RocksDB >= 6.24 support unique IDs.
-* Added `GetMapProperty()` support for "rocksdb.dbstats" (`DB::Properties::kDBStats`). As a map property, it includes DB-level internal stats accumulated over the DB's lifetime, such as user write related stats and uptime.
+## 6.25.2 (2021-10-11)
+### Bug Fixes
+* Fix `DisableManualCompaction()` to cancel compactions even when they are waiting on automatic compactions to drain due to `CompactRangeOptions::exclusive_manual_compactions == true`.
+* Fix contract of `Env::ReopenWritableFile()` and `FileSystem::ReopenWritableFile()` to specify any existing file must not be deleted or truncated.

-### Public API change
-* Made SystemClock extend the Customizable class and added a CreateFromString method.  Implementations need to be registered with the ObjectRegistry and to implement a Name() method in order to be created via this method.
-* Made SliceTransform extend the Customizable class and added a CreateFromString method.  Implementations need to be registered with the ObjectRegistry and to implement a Name() method in order to be created via this method.  The Capped and Prefixed transform classes return a short name (no length); use GetId for the fully qualified name.
-* Made FileChecksumGenFactory, SstPartitionerFactory, TablePropertiesCollectorFactory, and WalFilter extend the Customizable class and added a CreateFromString method.
-* Some fields of SstFileMetaData are deprecated for compatibility with new base class FileStorageInfo.
-* Add `file_temperature` to `IngestExternalFileArg` such that when ingesting SST files, we are able to indicate the temperature of the this batch of files.
-* If `DB::Close()` failed with a non aborted status, calling `DB::Close()` again will return the original status instead of Status::OK.
-* Add CacheTier to advanced_options.h to describe the cache tier we used. Add a `lowest_used_cache_tier` option to `DBOptions` (immutable) and pass it to BlockBasedTableReader. By default it is `CacheTier::kNonVolatileBlockTier`, which means, we always use both block cache (kVolatileTier) and secondary cache (kNonVolatileBlockTier). By set it to `CacheTier::kVolatileTier`, the DB will not use the secondary cache.
-* Even when options.max_compaction_bytes is hit, compaction output files are only cut when it aligns with grandparent files' boundaries. options.max_compaction_bytes could be slightly violated with the change, but the violation is no more than one target SST file size, which is usually much smaller.
-
-### Performance Improvements
-* Improved CPU efficiency of building block-based table (SST) files (#9039 and #9040).
-
-### Java API Changes
-* Add Java API bindings for new integrated BlobDB options
-* `keyMayExist()` supports ByteBuffer.
-* Fix multiget throwing Null Pointer Exception for num of keys > 70k (https://github.com/facebook/rocksdb/issues/8039).
+## 6.25.1 (2021-09-28)
+### Bug Fixes
+* Fixes a bug in directed IO mode when calling MultiGet() for blobs in the same blob file. The bug is caused by not sorting the blob read requests by file offsets.

 ## 6.25.0 (2021-09-20)
 ### Bug Fixes
@@ -81,10 +52,6 @@
 * Extended `FlushJobInfo` and `CompactionJobInfo` in listener.h to provide information about the blob files generated by a flush/compaction and garbage collected during compaction in Integrated BlobDB. Added struct members `blob_file_addition_infos` and `blob_file_garbage_infos` that contain this information.
 * Extended parameter `output_file_names` of `CompactFiles` API to also include paths of the blob files generated by the compaction in Integrated BlobDB.
 * Most `BackupEngine` functions now return `IOStatus` instead of `Status`. Most existing code should be compatible with this change but some calls might need to be updated.
-* Add a new field `level_at_creation` in `TablePropertiesCollectorFactory::Context` to capture the level at creating the SST file (i.e, table), of which the properties are being collected.
-
-### Miscellaneous
-* Add a paranoid check where in case FileSystem layer doesn't fill the buffer but returns succeed, checksum is unlikely to match even if buffer contains a previous block. The byte modified is not useful anyway, so it isn't expected to change any behavior when FileSystem is satisfying its contract.

 ## 6.24.0 (2021-08-20)
 ### Bug Fixes
@@ -2035,7 +2035,7 @@ ZLIB_SHA256 ?= c3e5e9fdd5004dcb542feda5ee4f0ff0744628baf8ed2dd5d66f8ca1197cb1a1
 ZLIB_DOWNLOAD_BASE ?= http://zlib.net
 BZIP2_VER ?= 1.0.8
 BZIP2_SHA256 ?= ab5a03176ee106d3f0fa90e381da478ddae405918153cca248e682cd0c4a2269
-BZIP2_DOWNLOAD_BASE ?= http://sourceware.org/pub/bzip2
+BZIP2_DOWNLOAD_BASE ?= https://sourceware.org/pub/bzip2
 SNAPPY_VER ?= 1.1.8
 SNAPPY_SHA256 ?= 16b677f07832a612b0836178db7f374e414f94657c138e6993cbfc5dcc58651f
 SNAPPY_DOWNLOAD_BASE ?= https://github.com/google/snappy/archive
@@ -225,7 +225,7 @@ cpp_library(
        "env/fs_remap.cc",
        "env/io_posix.cc",
        "env/mock_env.cc",
-        "env/unique_id_gen.cc",
+        "env/unique_id.cc",
        "file/delete_scheduler.cc",
        "file/file_prefetch_buffer.cc",
        "file/file_util.cc",
@@ -327,7 +327,6 @@ cpp_library(
        "table/table_factory.cc",
        "table/table_properties.cc",
        "table/two_level_iterator.cc",
-        "table/unique_id.cc",
        "test_util/sync_point.cc",
        "test_util/sync_point_impl.cc",
        "test_util/transaction_test_util.cc",
@@ -371,8 +370,6 @@ cpp_library(
        "utilities/blob_db/blob_db_impl_filesnapshot.cc",
        "utilities/blob_db/blob_dump_tool.cc",
        "utilities/blob_db/blob_file.cc",
-        "utilities/cache_dump_load.cc",
-        "utilities/cache_dump_load_impl.cc",
        "utilities/cassandra/cassandra_compaction_filter.cc",
        "utilities/cassandra/format.cc",
        "utilities/cassandra/merge_operator.cc",
@@ -438,7 +435,6 @@ cpp_library(
        "utilities/transactions/write_unprepared_txn.cc",
        "utilities/transactions/write_unprepared_txn_db.cc",
        "utilities/ttl/db_ttl_impl.cc",
-        "utilities/wal_filter.cc",
        "utilities/write_batch_with_index/write_batch_with_index.cc",
        "utilities/write_batch_with_index/write_batch_with_index_internal.cc",
    ],
@@ -551,7 +547,7 @@ cpp_library(
        "env/fs_remap.cc",
        "env/io_posix.cc",
        "env/mock_env.cc",
-        "env/unique_id_gen.cc",
+        "env/unique_id.cc",
        "file/delete_scheduler.cc",
        "file/file_prefetch_buffer.cc",
        "file/file_util.cc",
@@ -653,7 +649,6 @@ cpp_library(
        "table/table_factory.cc",
        "table/table_properties.cc",
        "table/two_level_iterator.cc",
-        "table/unique_id.cc",
        "test_util/sync_point.cc",
        "test_util/sync_point_impl.cc",
        "test_util/transaction_test_util.cc",
@@ -697,8 +692,6 @@ cpp_library(
        "utilities/blob_db/blob_db_impl_filesnapshot.cc",
        "utilities/blob_db/blob_dump_tool.cc",
        "utilities/blob_db/blob_file.cc",
-        "utilities/cache_dump_load.cc",
-        "utilities/cache_dump_load_impl.cc",
        "utilities/cassandra/cassandra_compaction_filter.cc",
        "utilities/cassandra/format.cc",
        "utilities/cassandra/merge_operator.cc",
@@ -764,7 +757,6 @@ cpp_library(
        "utilities/transactions/write_unprepared_txn.cc",
        "utilities/transactions/write_unprepared_txn_db.cc",
        "utilities/ttl/db_ttl_impl.cc",
-        "utilities/wal_filter.cc",
        "utilities/write_batch_with_index/write_batch_with_index.cc",
        "utilities/write_batch_with_index/write_batch_with_index_internal.cc",
    ],
@@ -850,11 +842,9 @@ cpp_library(
        "db_stress_tool/db_stress_common.cc",
        "db_stress_tool/db_stress_driver.cc",
        "db_stress_tool/db_stress_gflags.cc",
-        "db_stress_tool/db_stress_listener.cc",
        "db_stress_tool/db_stress_shared_state.cc",
        "db_stress_tool/db_stress_test_base.cc",
        "db_stress_tool/db_stress_tool.cc",
-        "db_stress_tool/expected_state.cc",
        "db_stress_tool/no_batched_ops_stress.cc",
        "test_util/testutil.cc",
        "tools/block_cache_analyzer/block_cache_trace_analyzer.cc",
@@ -113,7 +113,7 @@ LzLabs is using RocksDB as a storage engine in their multi-database distributed
 [YugabyteDB](https://www.yugabyte.com/) is an open source, high performance, distributed SQL database that uses RocksDB as its storage layer. For more information, please see https://github.com/yugabyte/yugabyte-db/.

 ## ArangoDB
-[ArangoDB](https://www.arangodb.com/) is a native multi-model database with flexible data models for documents, graphs, and key-values, for building high performance applications using a convenient SQL-like query language or JavaScript extensions. It uses RocksDB as its storage engine.
+[ArangoDB](https://www.arangodb.com/) is a native multi-model database with flexible data models for documents, graphs, and key-values, for building high performance applications using a convenient SQL-like query language or JavaScript extensions. It uses RocksDB as its sotrage engine.

 ## Milvus
 [Milvus](https://milvus.io/) is an open source vector database for unstructured data. It uses RocksDB not only as one of the supported kv storage engines, but also as a message queue.
@@ -606,8 +606,8 @@ EOF
 fi

 # TODO(tec): Fix -Wshorten-64-to-32 errors on FreeBSD and enable the warning.
-# -Wshorten-64-to-32 breaks compilation on FreeBSD aarch64 and i386
-if ! { [ "$TARGET_OS" = FreeBSD ] && [ "$TARGET_ARCHITECTURE" = arm64 -o "$TARGET_ARCHITECTURE" = i386 ]; }; then
+# -Wshorten-64-to-32 breaks compilation on FreeBSD i386
+if ! [ "$TARGET_OS" = FreeBSD -a "$TARGET_ARCHITECTURE" = i386 ]; then
  # Test whether -Wshorten-64-to-32 is available
  $CXX $PLATFORM_CXXFLAGS -x c++ - -o /dev/null -Wshorten-64-to-32 2>/dev/null  <<EOF
    int main() {}
@@ -18,4 +18,3 @@ KERNEL_HEADERS_BASE=/mnt/gvfs/third-party2/kernel-headers/32b8a2407b634df3f8f948
 BINUTILS_BASE=/mnt/gvfs/third-party2/binutils/08634589372fa5f237bfd374e8c644a8364e78c1/2.32/platform009/ba86d1f/
 VALGRIND_BASE=/mnt/gvfs/third-party2/valgrind/6ae525939ad02e5e676855082fbbc7828dbafeac/3.15.0/platform009/7f3b187
 LUA_BASE=/mnt/gvfs/third-party2/lua/162efd9561a3d21f6869f4814011e9cf1b3ff4dc/5.3.4/platform009/a6271c4
-BENCHMARK_BASE=/mnt/gvfs/third-party2/benchmark/bce8d9564eaf161700aa3a20b1051564acf555fb/1.5.5/platform009/7f3b187
@@ -63,13 +63,6 @@ else
 fi
 CFLAGS+=" -DGFLAGS=gflags"

-BENCHMARK_INCLUDE=" -I $BENCHMARK_BASE/include/"
-if test -z $PIC_BUILD; then
-  BENCHMARK_LIBS=" $BENCHMARK_BASE/lib/libbenchmark.a"
-else
-  BENCHMARK_LIBS=" $BENCHMARK_BASE/lib/libbenchmark_pic.a"
-fi
-
 # location of jemalloc
 JEMALLOC_INCLUDE=" -I $JEMALLOC_BASE/include/"
 JEMALLOC_LIB=" $JEMALLOC_BASE/lib/libjemalloc.a"
@@ -111,7 +104,7 @@ BINUTILS="$BINUTILS_BASE/bin"
 AR="$BINUTILS/ar"
 AS="$BINUTILS/as"

-DEPS_INCLUDE="$SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $ZSTD_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE $TBB_INCLUDE $LIBURING_INCLUDE $BENCHMARK_INCLUDE"
+DEPS_INCLUDE="$SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $ZSTD_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE $TBB_INCLUDE $LIBURING_INCLUDE"

 STDLIBS="-L $GCC_BASE/lib64"

@@ -157,7 +150,7 @@ CFLAGS+=" $DEPS_INCLUDE"
 CFLAGS+=" -DROCKSDB_PLATFORM_POSIX -DROCKSDB_LIB_IO_POSIX -DROCKSDB_FALLOCATE_PRESENT -DROCKSDB_MALLOC_USABLE_SIZE -DROCKSDB_RANGESYNC_PRESENT -DROCKSDB_SCHED_GETCPU_PRESENT -DROCKSDB_SUPPORT_THREAD_LOCAL -DHAVE_SSE42 -DROCKSDB_IOURING_PRESENT"
 CXXFLAGS+=" $CFLAGS"

-EXEC_LDFLAGS=" $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $NUMA_LIB $TBB_LIBS $LIBURING_LIBS $BENCHMARK_LIBS"
+EXEC_LDFLAGS=" $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $NUMA_LIB $TBB_LIBS $LIBURING_LIBS"
 EXEC_LDFLAGS+=" -Wl,--dynamic-linker,/usr/local/fbcode/platform009/lib/ld.so"
 EXEC_LDFLAGS+=" $LIBUNWIND"
 EXEC_LDFLAGS+=" -Wl,-rpath=/usr/local/fbcode/platform009/lib"
@@ -168,7 +161,7 @@ EXEC_LDFLAGS+=" -ldl"
 PLATFORM_LDFLAGS="$LIBGCC_LIBS $GLIBC_LIBS $STDLIBS -lgcc -lstdc++"
 PLATFORM_LDFLAGS+=" -B$BINUTILS"

-EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $TBB_LIBS $LIBURING_LIBS $BENCHMARK_LIBS"
+EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $TBB_LIBS $LIBURING_LIBS"

 VALGRIND_VER="$VALGRIND_BASE/bin/"

@@ -38,9 +38,7 @@ if [ "$CLANG_FORMAT_DIFF" ]; then
  fi
 else
  # First try directly executing the possibilities
-  if clang-format-diff --help &> /dev/null < /dev/null; then
-    CLANG_FORMAT_DIFF=clang-format-diff
-  elif clang-format-diff.py --help &> /dev/null < /dev/null; then
+  if clang-format-diff.py --help &> /dev/null < /dev/null; then
    CLANG_FORMAT_DIFF=clang-format-diff.py
  elif $REPO_ROOT/clang-format-diff.py --help &> /dev/null < /dev/null; then
    CLANG_FORMAT_DIFF=$REPO_ROOT/clang-format-diff.py
@@ -1204,7 +1204,7 @@ FBCODE_STRESS_CRASH_TEST_COMMANDS="[
            {
                'name':'Run RocksDB whitebox crash tests',
                'timeout': 86400,
-                'shell':'cd $WORKING_DIR; mkdir /dev/shm/rocksdb_fbcode_crash_test && TEST_TMPDIR=\$(mktemp -d --tmpdir=/dev/shm/rocksdb_fbcode_crash_test) python3 rocksdb/src/tools/db_crashtest.py --stress_cmd=buck-out/dbg/gen/rocks/tools/rocks_db_stress -secondary_cache_uri=\"$SECONDARY_CACHE_URI\" --env_uri=$ENV_URI $EXTRA_DB_STRESS_ARGS -logtostderr=false $TEST_TYPE || $CONTRUN_NAME=db_stress_fbcode $TASK_CREATION_TOOL',
+                'shell':'cd $WORKING_DIR; mkdir /dev/shm/rocksdb_whitebox_crash_test && TEST_TMPDIR=\$(mktemp -d --tmpdir=/dev/shm/rocksdb_whitebox_crash_test) python3 rocksdb/src/tools/db_crashtest.py --stress_cmd=buck-out/dbg/gen/rocks/tools/rocks_db_stress -secondary_cache_uri=\"$SECONDARY_CACHE_URI\" --env_uri=$ENV_URI -checkpoint_one_in=0 -backup_one_in=0 -cache_size=134217728 -cache_numshardbits=4 $EXTRA_DB_STRESS_ARGS -logtostderr=false whitebox || $CONTRUN_NAME=db_stress_fbcode $TASK_CREATION_TOOL',
                'user':'root',
                $PARSER
            },
@@ -15,11 +15,9 @@
 #include "rocksdb/cache.h"
 #include "rocksdb/io_status.h"
 #include "rocksdb/sst_file_manager.h"
-#include "rocksdb/utilities/cache_dump_load.h"
 #include "test_util/testharness.h"
 #include "util/coding.h"
 #include "util/random.h"
-#include "utilities/cache_dump_load_impl.h"
 #include "utilities/fault_injection_fs.h"

 namespace ROCKSDB_NAMESPACE {
@@ -695,7 +693,7 @@ TEST_F(DBSecondaryCacheTest, TestSecondaryCacheCorrectness1) {
  }

  ASSERT_OK(Flush());
-  // After Flush is successful, RocksDB will do the paranoid check for the new
+  // After Flush is successful, RocksDB do the paranoid check for the new
  // SST file. Meta blocks are always cached in the block cache and they
  // will not be evicted. When block_2 is cache miss and read out, it is
  // inserted to the block cache. Note that, block_1 is never successfully
@@ -789,7 +787,7 @@ TEST_F(DBSecondaryCacheTest, TestSecondaryCacheCorrectness2) {
  }

  ASSERT_OK(Flush());
-  // After Flush is successful, RocksDB will do the paranoid check for the new
+  // After Flush is successful, RocksDB do the paranoid check for the new
  // SST file. Meta blocks are always cached in the block cache and they
  // will not be evicted. When block_2 is cache miss and read out, it is
  // inserted to the block cache. Thefore, block_1 is evicted from block
@@ -883,7 +881,7 @@ TEST_F(DBSecondaryCacheTest, NoSecondaryCacheInsertion) {
  }

  ASSERT_OK(Flush());
-  // After Flush is successful, RocksDB will do the paranoid check for the new
+  // After Flush is successful, RocksDB do the paranoid check for the new
  // SST file. Meta blocks are always cached in the block cache and they
  // will not be evicted. Now, block cache is large enough, it cache
  // both block_1 and block_2. When first time read block_1 and block_2
@@ -985,7 +983,7 @@ TEST_F(DBSecondaryCacheTest, SecondaryCacheFailureTest) {
  }

  ASSERT_OK(Flush());
-  // After Flush is successful, RocksDB will do the paranoid check for the new
+  // After Flush is successful, RocksDB do the paranoid check for the new
  // SST file. Meta blocks are always cached in the block cache and they
  // will not be evicted. When block_2 is cache miss and read out, it is
  // inserted to the block cache. Note that, block_1 is never successfully
@@ -1184,668 +1182,6 @@ TEST_F(DBSecondaryCacheTest, TestSecondaryCacheMultiGet) {
  Destroy(options);
 }

-class LRUCacheWithStat : public LRUCache {
- public:
-  LRUCacheWithStat(
-      size_t _capacity, int _num_shard_bits, bool _strict_capacity_limit,
-      double _high_pri_pool_ratio,
-      std::shared_ptr<MemoryAllocator> _memory_allocator = nullptr,
-      bool _use_adaptive_mutex = kDefaultToAdaptiveMutex,
-      CacheMetadataChargePolicy _metadata_charge_policy =
-          kDontChargeCacheMetadata,
-      const std::shared_ptr<SecondaryCache>& _secondary_cache = nullptr)
-      : LRUCache(_capacity, _num_shard_bits, _strict_capacity_limit,
-                 _high_pri_pool_ratio, _memory_allocator, _use_adaptive_mutex,
-                 _metadata_charge_policy, _secondary_cache) {
-    insert_count_ = 0;
-    lookup_count_ = 0;
-  }
-  ~LRUCacheWithStat() {}
-
-  Status Insert(const Slice& key, void* value, size_t charge, DeleterFn deleter,
-                Handle** handle, Priority priority) override {
-    insert_count_++;
-    return LRUCache::Insert(key, value, charge, deleter, handle, priority);
-  }
-  Status Insert(const Slice& key, void* value, const CacheItemHelper* helper,
-                size_t chargge, Handle** handle = nullptr,
-                Priority priority = Priority::LOW) override {
-    insert_count_++;
-    return LRUCache::Insert(key, value, helper, chargge, handle, priority);
-  }
-  Handle* Lookup(const Slice& key, Statistics* stats) override {
-    lookup_count_++;
-    return LRUCache::Lookup(key, stats);
-  }
-  Handle* Lookup(const Slice& key, const CacheItemHelper* helper,
-                 const CreateCallback& create_cb, Priority priority, bool wait,
-                 Statistics* stats = nullptr) override {
-    lookup_count_++;
-    return LRUCache::Lookup(key, helper, create_cb, priority, wait, stats);
-  }
-
-  uint32_t GetInsertCount() { return insert_count_; }
-  uint32_t GetLookupcount() { return lookup_count_; }
-  void ResetCount() {
-    insert_count_ = 0;
-    lookup_count_ = 0;
-  }
-
- private:
-  uint32_t insert_count_;
-  uint32_t lookup_count_;
-};
-
-#ifndef ROCKSDB_LITE
-
-TEST_F(DBSecondaryCacheTest, LRUCacheDumpLoadBasic) {
-  LRUCacheOptions cache_opts(1024 * 1024, 0, false, 0.5, nullptr,
-                             kDefaultToAdaptiveMutex, kDontChargeCacheMetadata);
-  LRUCacheWithStat* tmp_cache = new LRUCacheWithStat(
-      cache_opts.capacity, cache_opts.num_shard_bits,
-      cache_opts.strict_capacity_limit, cache_opts.high_pri_pool_ratio,
-      cache_opts.memory_allocator, cache_opts.use_adaptive_mutex,
-      cache_opts.metadata_charge_policy, cache_opts.secondary_cache);
-  std::shared_ptr<Cache> cache(tmp_cache);
-  BlockBasedTableOptions table_options;
-  table_options.block_cache = cache;
-  table_options.block_size = 4 * 1024;
-  Options options = GetDefaultOptions();
-  options.create_if_missing = true;
-  options.table_factory.reset(NewBlockBasedTableFactory(table_options));
-  options.env = fault_env_.get();
-  DestroyAndReopen(options);
-  fault_fs_->SetFailGetUniqueId(true);
-
-  Random rnd(301);
-  const int N = 256;
-  std::vector<std::string> value;
-  char buf[1000];
-  memset(buf, 'a', 1000);
-  value.resize(N);
-  for (int i = 0; i < N; i++) {
-    // std::string p_v = rnd.RandomString(1000);
-    std::string p_v(buf, 1000);
-    value[i] = p_v;
-    ASSERT_OK(Put(Key(i), p_v));
-  }
-  ASSERT_OK(Flush());
-  Compact("a", "z");
-
-  // do th eread for all the key value pairs, so all the blocks should be in
-  // cache
-  uint32_t start_insert = tmp_cache->GetInsertCount();
-  uint32_t start_lookup = tmp_cache->GetLookupcount();
-  std::string v;
-  for (int i = 0; i < N; i++) {
-    v = Get(Key(i));
-    ASSERT_EQ(v, value[i]);
-  }
-  uint32_t dump_insert = tmp_cache->GetInsertCount() - start_insert;
-  uint32_t dump_lookup = tmp_cache->GetLookupcount() - start_lookup;
-  ASSERT_EQ(63,
-            static_cast<int>(dump_insert));  // the insert in the block cache
-  ASSERT_EQ(256,
-            static_cast<int>(dump_lookup));  // the lookup in the block cache
-  // We have enough blocks in the block cache
-
-  CacheDumpOptions cd_options;
-  cd_options.clock = fault_env_->GetSystemClock().get();
-  std::string dump_path = db_->GetName() + "/cache_dump";
-  std::unique_ptr<CacheDumpWriter> dump_writer;
-  Status s = NewToFileCacheDumpWriter(fault_fs_, FileOptions(), dump_path,
-                                      &dump_writer);
-  ASSERT_OK(s);
-  std::unique_ptr<CacheDumper> cache_dumper;
-  s = NewDefaultCacheDumper(cd_options, cache, std::move(dump_writer),
-                            &cache_dumper);
-  ASSERT_OK(s);
-  std::vector<DB*> db_list;
-  db_list.push_back(db_);
-  s = cache_dumper->SetDumpFilter(db_list);
-  ASSERT_OK(s);
-  s = cache_dumper->DumpCacheEntriesToWriter();
-  ASSERT_OK(s);
-  cache_dumper.reset();
-
-  // we have a new cache it is empty, then, before we do the Get, we do the
-  // dumpload
-  std::shared_ptr<TestSecondaryCache> secondary_cache =
-      std::make_shared<TestSecondaryCache>(2048 * 1024);
-  cache_opts.secondary_cache = secondary_cache;
-  tmp_cache = new LRUCacheWithStat(
-      cache_opts.capacity, cache_opts.num_shard_bits,
-      cache_opts.strict_capacity_limit, cache_opts.high_pri_pool_ratio,
-      cache_opts.memory_allocator, cache_opts.use_adaptive_mutex,
-      cache_opts.metadata_charge_policy, cache_opts.secondary_cache);
-  std::shared_ptr<Cache> cache_new(tmp_cache);
-  table_options.block_cache = cache_new;
-  table_options.block_size = 4 * 1024;
-  options.create_if_missing = true;
-  options.table_factory.reset(NewBlockBasedTableFactory(table_options));
-  options.env = fault_env_.get();
-
-  // start to load the data to new block cache
-  start_insert = secondary_cache->num_inserts();
-  start_lookup = secondary_cache->num_lookups();
-  std::unique_ptr<CacheDumpReader> dump_reader;
-  s = NewFromFileCacheDumpReader(fault_fs_, FileOptions(), dump_path,
-                                 &dump_reader);
-  ASSERT_OK(s);
-  std::unique_ptr<CacheDumpedLoader> cache_loader;
-  s = NewDefaultCacheDumpedLoader(cd_options, table_options, secondary_cache,
-                                  std::move(dump_reader), &cache_loader);
-  ASSERT_OK(s);
-  s = cache_loader->RestoreCacheEntriesToSecondaryCache();
-  ASSERT_OK(s);
-  uint32_t load_insert = secondary_cache->num_inserts() - start_insert;
-  uint32_t load_lookup = secondary_cache->num_lookups() - start_lookup;
-  // check the number we inserted
-  ASSERT_EQ(64, static_cast<int>(load_insert));
-  ASSERT_EQ(0, static_cast<int>(load_lookup));
-  ASSERT_OK(s);
-
-  Reopen(options);
-
-  // After load, we do the Get again
-  start_insert = secondary_cache->num_inserts();
-  start_lookup = secondary_cache->num_lookups();
-  uint32_t cache_insert = tmp_cache->GetInsertCount();
-  uint32_t cache_lookup = tmp_cache->GetLookupcount();
-  for (int i = 0; i < N; i++) {
-    v = Get(Key(i));
-    ASSERT_EQ(v, value[i]);
-  }
-  uint32_t final_insert = secondary_cache->num_inserts() - start_insert;
-  uint32_t final_lookup = secondary_cache->num_lookups() - start_lookup;
-  // no insert to secondary cache
-  ASSERT_EQ(0, static_cast<int>(final_insert));
-  // lookup the secondary to get all blocks
-  ASSERT_EQ(64, static_cast<int>(final_lookup));
-  uint32_t block_insert = tmp_cache->GetInsertCount() - cache_insert;
-  uint32_t block_lookup = tmp_cache->GetLookupcount() - cache_lookup;
-  // Check the new block cache insert and lookup, should be no insert since all
-  // blocks are from the secondary cache.
-  ASSERT_EQ(0, static_cast<int>(block_insert));
-  ASSERT_EQ(256, static_cast<int>(block_lookup));
-
-  fault_fs_->SetFailGetUniqueId(false);
-  Destroy(options);
-}
-
-TEST_F(DBSecondaryCacheTest, LRUCacheDumpLoadWithFilter) {
-  LRUCacheOptions cache_opts(1024 * 1024, 0, false, 0.5, nullptr,
-                             kDefaultToAdaptiveMutex, kDontChargeCacheMetadata);
-  LRUCacheWithStat* tmp_cache = new LRUCacheWithStat(
-      cache_opts.capacity, cache_opts.num_shard_bits,
-      cache_opts.strict_capacity_limit, cache_opts.high_pri_pool_ratio,
-      cache_opts.memory_allocator, cache_opts.use_adaptive_mutex,
-      cache_opts.metadata_charge_policy, cache_opts.secondary_cache);
-  std::shared_ptr<Cache> cache(tmp_cache);
-  BlockBasedTableOptions table_options;
-  table_options.block_cache = cache;
-  table_options.block_size = 4 * 1024;
-  Options options = GetDefaultOptions();
-  options.create_if_missing = true;
-  options.table_factory.reset(NewBlockBasedTableFactory(table_options));
-  options.env = fault_env_.get();
-  std::string dbname1 = test::PerThreadDBPath("db_1");
-  ASSERT_OK(DestroyDB(dbname1, options));
-  DB* db1 = nullptr;
-  ASSERT_OK(DB::Open(options, dbname1, &db1));
-  std::string dbname2 = test::PerThreadDBPath("db_2");
-  ASSERT_OK(DestroyDB(dbname2, options));
-  DB* db2 = nullptr;
-  ASSERT_OK(DB::Open(options, dbname2, &db2));
-  fault_fs_->SetFailGetUniqueId(true);
-
-  // write the KVs to db1
-  Random rnd(301);
-  const int N = 256;
-  std::vector<std::string> value1;
-  WriteOptions wo;
-  char buf[1000];
-  memset(buf, 'a', 1000);
-  value1.resize(N);
-  for (int i = 0; i < N; i++) {
-    std::string p_v(buf, 1000);
-    value1[i] = p_v;
-    ASSERT_OK(db1->Put(wo, Key(i), p_v));
-  }
-  ASSERT_OK(db1->Flush(FlushOptions()));
-  Slice bg("a");
-  Slice ed("b");
-  ASSERT_OK(db1->CompactRange(CompactRangeOptions(), &bg, &ed));
-
-  // Write the KVs to DB2
-  std::vector<std::string> value2;
-  memset(buf, 'b', 1000);
-  value2.resize(N);
-  for (int i = 0; i < N; i++) {
-    std::string p_v(buf, 1000);
-    value2[i] = p_v;
-    ASSERT_OK(db2->Put(wo, Key(i), p_v));
-  }
-  ASSERT_OK(db2->Flush(FlushOptions()));
-  ASSERT_OK(db2->CompactRange(CompactRangeOptions(), &bg, &ed));
-
-  // do th eread for all the key value pairs, so all the blocks should be in
-  // cache
-  uint32_t start_insert = tmp_cache->GetInsertCount();
-  uint32_t start_lookup = tmp_cache->GetLookupcount();
-  ReadOptions ro;
-  std::string v;
-  for (int i = 0; i < N; i++) {
-    ASSERT_OK(db1->Get(ro, Key(i), &v));
-    ASSERT_EQ(v, value1[i]);
-  }
-  for (int i = 0; i < N; i++) {
-    ASSERT_OK(db2->Get(ro, Key(i), &v));
-    ASSERT_EQ(v, value2[i]);
-  }
-  uint32_t dump_insert = tmp_cache->GetInsertCount() - start_insert;
-  uint32_t dump_lookup = tmp_cache->GetLookupcount() - start_lookup;
-  ASSERT_EQ(128,
-            static_cast<int>(dump_insert));  // the insert in the block cache
-  ASSERT_EQ(512,
-            static_cast<int>(dump_lookup));  // the lookup in the block cache
-  // We have enough blocks in the block cache
-
-  CacheDumpOptions cd_options;
-  cd_options.clock = fault_env_->GetSystemClock().get();
-  std::string dump_path = db1->GetName() + "/cache_dump";
-  std::unique_ptr<CacheDumpWriter> dump_writer;
-  Status s = NewToFileCacheDumpWriter(fault_fs_, FileOptions(), dump_path,
-                                      &dump_writer);
-  ASSERT_OK(s);
-  std::unique_ptr<CacheDumper> cache_dumper;
-  s = NewDefaultCacheDumper(cd_options, cache, std::move(dump_writer),
-                            &cache_dumper);
-  ASSERT_OK(s);
-  std::vector<DB*> db_list;
-  db_list.push_back(db1);
-  s = cache_dumper->SetDumpFilter(db_list);
-  ASSERT_OK(s);
-  s = cache_dumper->DumpCacheEntriesToWriter();
-  ASSERT_OK(s);
-  cache_dumper.reset();
-
-  // we have a new cache it is empty, then, before we do the Get, we do the
-  // dumpload
-  std::shared_ptr<TestSecondaryCache> secondary_cache =
-      std::make_shared<TestSecondaryCache>(2048 * 1024);
-  cache_opts.secondary_cache = secondary_cache;
-  tmp_cache = new LRUCacheWithStat(
-      cache_opts.capacity, cache_opts.num_shard_bits,
-      cache_opts.strict_capacity_limit, cache_opts.high_pri_pool_ratio,
-      cache_opts.memory_allocator, cache_opts.use_adaptive_mutex,
-      cache_opts.metadata_charge_policy, cache_opts.secondary_cache);
-  std::shared_ptr<Cache> cache_new(tmp_cache);
-  table_options.block_cache = cache_new;
-  table_options.block_size = 4 * 1024;
-  options.create_if_missing = true;
-  options.table_factory.reset(NewBlockBasedTableFactory(table_options));
-  options.env = fault_env_.get();
-
-  // Start the cache loading process
-  start_insert = secondary_cache->num_inserts();
-  start_lookup = secondary_cache->num_lookups();
-  std::unique_ptr<CacheDumpReader> dump_reader;
-  s = NewFromFileCacheDumpReader(fault_fs_, FileOptions(), dump_path,
-                                 &dump_reader);
-  ASSERT_OK(s);
-  std::unique_ptr<CacheDumpedLoader> cache_loader;
-  s = NewDefaultCacheDumpedLoader(cd_options, table_options, secondary_cache,
-                                  std::move(dump_reader), &cache_loader);
-  ASSERT_OK(s);
-  s = cache_loader->RestoreCacheEntriesToSecondaryCache();
-  ASSERT_OK(s);
-  uint32_t load_insert = secondary_cache->num_inserts() - start_insert;
-  uint32_t load_lookup = secondary_cache->num_lookups() - start_lookup;
-  // check the number we inserted
-  ASSERT_EQ(64, static_cast<int>(load_insert));
-  ASSERT_EQ(0, static_cast<int>(load_lookup));
-  ASSERT_OK(s);
-
-  ASSERT_OK(db1->Close());
-  delete db1;
-  ASSERT_OK(DB::Open(options, dbname1, &db1));
-
-  // After load, we do the Get again. To validate the cache, we do not allow any
-  // I/O, so we set the file system to false.
-  IOStatus error_msg = IOStatus::IOError("Retryable IO Error");
-  fault_fs_->SetFilesystemActive(false, error_msg);
-  start_insert = secondary_cache->num_inserts();
-  start_lookup = secondary_cache->num_lookups();
-  uint32_t cache_insert = tmp_cache->GetInsertCount();
-  uint32_t cache_lookup = tmp_cache->GetLookupcount();
-  for (int i = 0; i < N; i++) {
-    ASSERT_OK(db1->Get(ro, Key(i), &v));
-    ASSERT_EQ(v, value1[i]);
-  }
-  uint32_t final_insert = secondary_cache->num_inserts() - start_insert;
-  uint32_t final_lookup = secondary_cache->num_lookups() - start_lookup;
-  // no insert to secondary cache
-  ASSERT_EQ(0, static_cast<int>(final_insert));
-  // lookup the secondary to get all blocks
-  ASSERT_EQ(64, static_cast<int>(final_lookup));
-  uint32_t block_insert = tmp_cache->GetInsertCount() - cache_insert;
-  uint32_t block_lookup = tmp_cache->GetLookupcount() - cache_lookup;
-  // Check the new block cache insert and lookup, should be no insert since all
-  // blocks are from the secondary cache.
-  ASSERT_EQ(0, static_cast<int>(block_insert));
-  ASSERT_EQ(256, static_cast<int>(block_lookup));
-  fault_fs_->SetFailGetUniqueId(false);
-  fault_fs_->SetFilesystemActive(true);
-  delete db1;
-  delete db2;
-  ASSERT_OK(DestroyDB(dbname1, options));
-  ASSERT_OK(DestroyDB(dbname2, options));
-}
-
-// Test the option not to use the secondary cache in a certain DB.
-TEST_F(DBSecondaryCacheTest, TestSecondaryCacheOptionBasic) {
-  LRUCacheOptions opts(4 * 1024, 0, false, 0.5, nullptr,
-                       kDefaultToAdaptiveMutex, kDontChargeCacheMetadata);
-  std::shared_ptr<TestSecondaryCache> secondary_cache(
-      new TestSecondaryCache(2048 * 1024));
-  opts.secondary_cache = secondary_cache;
-  std::shared_ptr<Cache> cache = NewLRUCache(opts);
-  BlockBasedTableOptions table_options;
-  table_options.block_cache = cache;
-  table_options.block_size = 4 * 1024;
-  Options options = GetDefaultOptions();
-  options.create_if_missing = true;
-  options.table_factory.reset(NewBlockBasedTableFactory(table_options));
-  options.env = fault_env_.get();
-  fault_fs_->SetFailGetUniqueId(true);
-  options.lowest_used_cache_tier = CacheTier::kVolatileTier;
-
-  // Set the file paranoid check, so after flush, the file will be read
-  // all the blocks will be accessed.
-  options.paranoid_file_checks = true;
-  DestroyAndReopen(options);
-  std::string session_id;
-  ASSERT_OK(db_->GetDbSessionId(session_id));
-  secondary_cache->SetDbSessionId(session_id);
-  Random rnd(301);
-  const int N = 6;
-  for (int i = 0; i < N; i++) {
-    std::string p_v = rnd.RandomString(1007);
-    ASSERT_OK(Put(Key(i), p_v));
-  }
-
-  ASSERT_OK(Flush());
-
-  for (int i = 0; i < N; i++) {
-    std::string p_v = rnd.RandomString(1007);
-    ASSERT_OK(Put(Key(i + 70), p_v));
-  }
-
-  ASSERT_OK(Flush());
-
-  // Flush will trigger the paranoid check and read blocks. But only block cache
-  // will be read. No operations for secondary cache.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  Compact("a", "z");
-
-  // Compaction will also insert and evict blocks, no operations to the block
-  // cache. No operations for secondary cache.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  std::string v = Get(Key(0));
-  ASSERT_EQ(1007, v.size());
-
-  // Check the data in first block. Cache miss, direclty read from SST file.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  v = Get(Key(5));
-  ASSERT_EQ(1007, v.size());
-
-  // Check the second block.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  v = Get(Key(5));
-  ASSERT_EQ(1007, v.size());
-
-  // block cache hit
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  v = Get(Key(70));
-  ASSERT_EQ(1007, v.size());
-
-  // Check the first block in the second SST file. Cache miss and trigger SST
-  // file read. No operations for secondary cache.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  v = Get(Key(75));
-  ASSERT_EQ(1007, v.size());
-
-  // Check the second block in the second SST file. Cache miss and trigger SST
-  // file read. No operations for secondary cache.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  Destroy(options);
-}
-
-// We disable the secondary cache in DBOptions at first. Close and reopen the DB
-// with new options, which set the lowest_used_cache_tier to
-// kNonVolatileBlockTier. So secondary cache will be used.
-TEST_F(DBSecondaryCacheTest, TestSecondaryCacheOptionChange) {
-  LRUCacheOptions opts(4 * 1024, 0, false, 0.5, nullptr,
-                       kDefaultToAdaptiveMutex, kDontChargeCacheMetadata);
-  std::shared_ptr<TestSecondaryCache> secondary_cache(
-      new TestSecondaryCache(2048 * 1024));
-  opts.secondary_cache = secondary_cache;
-  std::shared_ptr<Cache> cache = NewLRUCache(opts);
-  BlockBasedTableOptions table_options;
-  table_options.block_cache = cache;
-  table_options.block_size = 4 * 1024;
-  Options options = GetDefaultOptions();
-  options.create_if_missing = true;
-  options.table_factory.reset(NewBlockBasedTableFactory(table_options));
-  options.env = fault_env_.get();
-  fault_fs_->SetFailGetUniqueId(true);
-  options.lowest_used_cache_tier = CacheTier::kVolatileTier;
-
-  // Set the file paranoid check, so after flush, the file will be read
-  // all the blocks will be accessed.
-  options.paranoid_file_checks = true;
-  DestroyAndReopen(options);
-  std::string session_id;
-  ASSERT_OK(db_->GetDbSessionId(session_id));
-  secondary_cache->SetDbSessionId(session_id);
-  Random rnd(301);
-  const int N = 6;
-  for (int i = 0; i < N; i++) {
-    std::string p_v = rnd.RandomString(1007);
-    ASSERT_OK(Put(Key(i), p_v));
-  }
-
-  ASSERT_OK(Flush());
-
-  for (int i = 0; i < N; i++) {
-    std::string p_v = rnd.RandomString(1007);
-    ASSERT_OK(Put(Key(i + 70), p_v));
-  }
-
-  ASSERT_OK(Flush());
-
-  // Flush will trigger the paranoid check and read blocks. But only block cache
-  // will be read.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  Compact("a", "z");
-
-  // Compaction will also insert and evict blocks, no operations to the block
-  // cache.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  std::string v = Get(Key(0));
-  ASSERT_EQ(1007, v.size());
-
-  // Check the data in first block. Cache miss, direclty read from SST file.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  v = Get(Key(5));
-  ASSERT_EQ(1007, v.size());
-
-  // Check the second block.
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  v = Get(Key(5));
-  ASSERT_EQ(1007, v.size());
-
-  // block cache hit
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-
-  // Change the option to enable secondary cache after we Reopen the DB
-  options.lowest_used_cache_tier = CacheTier::kNonVolatileBlockTier;
-  Reopen(options);
-
-  v = Get(Key(70));
-  ASSERT_EQ(1007, v.size());
-
-  // Enable the secondary cache, trigger lookup of the first block in second SST
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 1u);
-
-  v = Get(Key(75));
-  ASSERT_EQ(1007, v.size());
-
-  // trigger lookup of the second block in second SST
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 2u);
-  Destroy(options);
-}
-
-// Two DB test. We create 2 DBs sharing the same block cache and secondary
-// cache. We diable the secondary cache option for DB2.
-TEST_F(DBSecondaryCacheTest, TestSecondaryCacheOptionTwoDB) {
-  LRUCacheOptions opts(4 * 1024, 0, false, 0.5, nullptr,
-                       kDefaultToAdaptiveMutex, kDontChargeCacheMetadata);
-  std::shared_ptr<TestSecondaryCache> secondary_cache(
-      new TestSecondaryCache(2048 * 1024));
-  opts.secondary_cache = secondary_cache;
-  std::shared_ptr<Cache> cache = NewLRUCache(opts);
-  BlockBasedTableOptions table_options;
-  table_options.block_cache = cache;
-  table_options.block_size = 4 * 1024;
-  Options options = GetDefaultOptions();
-  options.create_if_missing = true;
-  options.table_factory.reset(NewBlockBasedTableFactory(table_options));
-  options.env = fault_env_.get();
-  options.paranoid_file_checks = true;
-  std::string dbname1 = test::PerThreadDBPath("db_t_1");
-  ASSERT_OK(DestroyDB(dbname1, options));
-  DB* db1 = nullptr;
-  ASSERT_OK(DB::Open(options, dbname1, &db1));
-  std::string dbname2 = test::PerThreadDBPath("db_t_2");
-  ASSERT_OK(DestroyDB(dbname2, options));
-  DB* db2 = nullptr;
-  Options options2 = options;
-  options2.lowest_used_cache_tier = CacheTier::kVolatileTier;
-  ASSERT_OK(DB::Open(options2, dbname2, &db2));
-  fault_fs_->SetFailGetUniqueId(true);
-
-  // Set the file paranoid check, so after flush, the file will be read
-  // all the blocks will be accessed.
-  std::string session_id;
-  ASSERT_OK(db1->GetDbSessionId(session_id));
-  secondary_cache->SetDbSessionId(session_id);
-
-  WriteOptions wo;
-  Random rnd(301);
-  const int N = 6;
-  for (int i = 0; i < N; i++) {
-    std::string p_v = rnd.RandomString(1007);
-    ASSERT_OK(db1->Put(wo, Key(i), p_v));
-  }
-
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 0u);
-  ASSERT_OK(db1->Flush(FlushOptions()));
-
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 2u);
-
-  for (int i = 0; i < N; i++) {
-    std::string p_v = rnd.RandomString(1007);
-    ASSERT_OK(db2->Put(wo, Key(i), p_v));
-  }
-
-  // No change in the secondary cache, since it is disabled in DB2
-  ASSERT_EQ(secondary_cache->num_inserts(), 0u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 2u);
-  ASSERT_OK(db2->Flush(FlushOptions()));
-  ASSERT_EQ(secondary_cache->num_inserts(), 1u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 2u);
-
-  Slice bg("a");
-  Slice ed("b");
-  ASSERT_OK(db1->CompactRange(CompactRangeOptions(), &bg, &ed));
-  ASSERT_OK(db2->CompactRange(CompactRangeOptions(), &bg, &ed));
-
-  ASSERT_EQ(secondary_cache->num_inserts(), 1u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 2u);
-
-  ReadOptions ro;
-  std::string v;
-  ASSERT_OK(db1->Get(ro, Key(0), &v));
-  ASSERT_EQ(1007, v.size());
-
-  // DB 1 has lookup block 1 and it is miss in block cache, trigger secondary
-  // cache lookup
-  ASSERT_EQ(secondary_cache->num_inserts(), 1u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 3u);
-
-  ASSERT_OK(db1->Get(ro, Key(5), &v));
-  ASSERT_EQ(1007, v.size());
-
-  // DB 1 lookup the second block and it is miss in block cache, trigger
-  // secondary cache lookup
-  ASSERT_EQ(secondary_cache->num_inserts(), 1u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 4u);
-
-  ASSERT_OK(db2->Get(ro, Key(0), &v));
-  ASSERT_EQ(1007, v.size());
-
-  // For db2, it is not enabled with secondary cache, so no search in the
-  // secondary cache
-  ASSERT_EQ(secondary_cache->num_inserts(), 1u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 4u);
-
-  ASSERT_OK(db2->Get(ro, Key(5), &v));
-  ASSERT_EQ(1007, v.size());
-
-  // For db2, it is not enabled with secondary cache, so no search in the
-  // secondary cache
-  ASSERT_EQ(secondary_cache->num_inserts(), 1u);
-  ASSERT_EQ(secondary_cache->num_lookups(), 4u);
-
-  fault_fs_->SetFailGetUniqueId(false);
-  fault_fs_->SetFilesystemActive(true);
-  delete db1;
-  delete db2;
-  ASSERT_OK(DestroyDB(dbname1, options));
-  ASSERT_OK(DestroyDB(dbname2, options));
-}
-
-#endif  // ROCKSDB_LITE
-
 }  // namespace ROCKSDB_NAMESPACE

 int main(int argc, char** argv) {
@@ -12,6 +12,7 @@
 #include <string>
 #include "db/db_impl/db_impl.h"
 #include "db/db_iter.h"
+#include "db/dbformat.h"
 #include "db/range_del_aggregator.h"
 #include "memory/arena.h"
 #include "options/cf_options.h"
@@ -33,7 +34,7 @@ class Version;
 // the same as the inner DBIter.
 class ArenaWrappedDBIter : public Iterator {
 public:
-  ~ArenaWrappedDBIter() override {
+  virtual ~ArenaWrappedDBIter() {
    if (db_iter_ != nullptr) {
      db_iter_->~DBIter();
    } else {
@@ -14,7 +14,6 @@
 #include "db/dbformat.h"
 #include "test_util/testharness.h"
 #include "test_util/testutil.h"
-#include "util/vector_iterator.h"

 namespace ROCKSDB_NAMESPACE {

@@ -68,7 +67,7 @@ TEST(BlobCountingIteratorTest, CountBlobs) {

  assert(keys.size() == values.size());

-  VectorIterator input(keys, values);
+  test::VectorIterator input(keys, values);
  BlobGarbageMeter blob_garbage_meter;

  BlobCountingIterator blob_counter(&input, &blob_garbage_meter);
@@ -308,7 +307,7 @@ TEST(BlobCountingIteratorTest, CorruptBlobIndex) {

  assert(keys.size() == values.size());

-  VectorIterator input(keys, values);
+  test::VectorIterator input(keys, values);
  BlobGarbageMeter blob_garbage_meter;

  BlobCountingIterator blob_counter(&input, &blob_garbage_meter);
@@ -39,10 +39,9 @@ class TestFileNumberGenerator {

 class BlobFileBuilderTest : public testing::Test {
 protected:
-  BlobFileBuilderTest() {
-    mock_env_.reset(MockEnv::Create(Env::Default()));
-    fs_ = mock_env_->GetFileSystem().get();
-    clock_ = mock_env_->GetSystemClock().get();
+  BlobFileBuilderTest() : mock_env_(Env::Default()) {
+    fs_ = mock_env_.GetFileSystem().get();
+    clock_ = mock_env_.GetSystemClock().get();
  }

  void VerifyBlobFile(uint64_t blob_file_number,
@@ -109,7 +108,7 @@ class BlobFileBuilderTest : public testing::Test {
    ASSERT_EQ(footer.expiration_range, ExpirationRange());
  }

-  std::unique_ptr<Env> mock_env_;
+  MockEnv mock_env_;
  FileSystem* fs_;
  SystemClock* clock_;
  FileOptions file_options_;
@@ -124,11 +123,11 @@ TEST_F(BlobFileBuilderTest, BuildAndCheckOneFile) {

  Options options;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileBuilderTest_BuildAndCheckOneFile"),
      0);
  options.enable_blob_files = true;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;

  ImmutableOptions immutable_options(options);
  MutableCFOptions mutable_cf_options(options);
@@ -207,12 +206,12 @@ TEST_F(BlobFileBuilderTest, BuildAndCheckMultipleFiles) {

  Options options;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileBuilderTest_BuildAndCheckMultipleFiles"),
      0);
  options.enable_blob_files = true;
  options.blob_file_size = value_size;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;

  ImmutableOptions immutable_options(options);
  MutableCFOptions mutable_cf_options(options);
@@ -294,12 +293,11 @@ TEST_F(BlobFileBuilderTest, InlinedValues) {

  Options options;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
-                            "BlobFileBuilderTest_InlinedValues"),
+      test::PerThreadDBPath(&mock_env_, "BlobFileBuilderTest_InlinedValues"),
      0);
  options.enable_blob_files = true;
  options.min_blob_size = 1024;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;

  ImmutableOptions immutable_options(options);
  MutableCFOptions mutable_cf_options(options);
@@ -349,11 +347,10 @@ TEST_F(BlobFileBuilderTest, Compression) {

  Options options;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(), "BlobFileBuilderTest_Compression"),
-      0);
+      test::PerThreadDBPath(&mock_env_, "BlobFileBuilderTest_Compression"), 0);
  options.enable_blob_files = true;
  options.blob_compression_type = kSnappyCompression;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;

  ImmutableOptions immutable_options(options);
  MutableCFOptions mutable_cf_options(options);
@@ -432,12 +429,11 @@ TEST_F(BlobFileBuilderTest, CompressionError) {

  Options options;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
-                            "BlobFileBuilderTest_CompressionError"),
+      test::PerThreadDBPath(&mock_env_, "BlobFileBuilderTest_CompressionError"),
      0);
  options.enable_blob_files = true;
  options.blob_compression_type = kSnappyCompression;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  ImmutableOptions immutable_options(options);
  MutableCFOptions mutable_cf_options(options);

@@ -510,12 +506,11 @@ TEST_F(BlobFileBuilderTest, Checksum) {

  Options options;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(), "BlobFileBuilderTest_Checksum"),
-      0);
+      test::PerThreadDBPath(&mock_env_, "BlobFileBuilderTest_Checksum"), 0);
  options.enable_blob_files = true;
  options.file_checksum_gen_factory =
      std::make_shared<DummyFileChecksumGenFactory>();
-  options.env = mock_env_.get();
+  options.env = &mock_env_;

  ImmutableOptions immutable_options(options);
  MutableCFOptions mutable_cf_options(options);
@@ -580,12 +575,12 @@ class BlobFileBuilderIOErrorTest
    : public testing::Test,
      public testing::WithParamInterface<std::string> {
 protected:
-  BlobFileBuilderIOErrorTest() : sync_point_(GetParam()) {
-    mock_env_.reset(MockEnv::Create(Env::Default()));
-    fs_ = mock_env_->GetFileSystem().get();
-  }
+  BlobFileBuilderIOErrorTest()
+      : mock_env_(Env::Default()),
+        fs_(mock_env_.GetFileSystem().get()),
+        sync_point_(GetParam()) {}

-  std::unique_ptr<Env> mock_env_;
+  MockEnv mock_env_;
  FileSystem* fs_;
  FileOptions file_options_;
  std::string sync_point_;
@@ -607,12 +602,11 @@ TEST_P(BlobFileBuilderIOErrorTest, IOError) {

  Options options;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
-                            "BlobFileBuilderIOErrorTest_IOError"),
+      test::PerThreadDBPath(&mock_env_, "BlobFileBuilderIOErrorTest_IOError"),
      0);
  options.enable_blob_files = true;
  options.blob_file_size = value_size;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;

  ImmutableOptions immutable_options(options);
  MutableCFOptions mutable_cf_options(options);
@@ -84,18 +84,17 @@ void WriteBlobFile(uint32_t column_family_id,

 class BlobFileCacheTest : public testing::Test {
 protected:
-  BlobFileCacheTest() { mock_env_.reset(MockEnv::Create(Env::Default())); }
+  BlobFileCacheTest() : mock_env_(Env::Default()) {}

-  std::unique_ptr<Env> mock_env_;
+  MockEnv mock_env_;
 };

 TEST_F(BlobFileCacheTest, GetBlobFileReader) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.statistics = CreateDBStatistics();
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
-                            "BlobFileCacheTest_GetBlobFileReader"),
+      test::PerThreadDBPath(&mock_env_, "BlobFileCacheTest_GetBlobFileReader"),
      0);
  options.enable_blob_files = true;

@@ -136,10 +135,10 @@ TEST_F(BlobFileCacheTest, GetBlobFileReader) {

 TEST_F(BlobFileCacheTest, GetBlobFileReader_Race) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.statistics = CreateDBStatistics();
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileCacheTest_GetBlobFileReader_Race"),
      0);
  options.enable_blob_files = true;
@@ -188,10 +187,10 @@ TEST_F(BlobFileCacheTest, GetBlobFileReader_Race) {

 TEST_F(BlobFileCacheTest, GetBlobFileReader_IOError) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.statistics = CreateDBStatistics();
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileCacheTest_GetBlobFileReader_IOError"),
      0);
  options.enable_blob_files = true;
@@ -222,10 +221,10 @@ TEST_F(BlobFileCacheTest, GetBlobFileReader_IOError) {

 TEST_F(BlobFileCacheTest, GetBlobFileReader_CacheFull) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.statistics = CreateDBStatistics();
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileCacheTest_GetBlobFileReader_CacheFull"),
      0);
  options.enable_blob_files = true;
@@ -22,17 +22,12 @@ class BlobFileCompletionCallback {
      ErrorHandler* error_handler, EventLogger* event_logger,
      const std::vector<std::shared_ptr<EventListener>>& listeners,
      const std::string& dbname)
-      : event_logger_(event_logger), listeners_(listeners), dbname_(dbname) {
-#ifndef ROCKSDB_LITE
-    sst_file_manager_ = sst_file_manager;
-    mutex_ = mutex;
-    error_handler_ = error_handler;
-#else
-    (void)sst_file_manager;
-    (void)mutex;
-    (void)error_handler;
-#endif  // ROCKSDB_LITE
-  }
+      : sst_file_manager_(sst_file_manager),
+        mutex_(mutex),
+        error_handler_(error_handler),
+        event_logger_(event_logger),
+        listeners_(listeners),
+        dbname_(dbname) {}

  void OnBlobFileCreationStarted(const std::string& file_name,
                                 const std::string& column_family_name,
@@ -89,11 +84,9 @@ class BlobFileCompletionCallback {
  }

 private:
-#ifndef ROCKSDB_LITE
  SstFileManager* sst_file_manager_;
  InstrumentedMutex* mutex_;
  ErrorHandler* error_handler_;
-#endif  // ROCKSDB_LITE
  EventLogger* event_logger_;
  std::vector<std::shared_ptr<EventListener>> listeners_;
  std::string dbname_;
@@ -358,18 +358,11 @@ void BlobFileReader::MultiGetBlob(
    const autovector<uint64_t>& value_sizes, autovector<Status*>& statuses,
    autovector<PinnableSlice*>& values, uint64_t* bytes_read) const {
  const size_t num_blobs = user_keys.size();
-  assert(num_blobs > 0);
  assert(num_blobs == offsets.size());
  assert(num_blobs == value_sizes.size());
  assert(num_blobs == statuses.size());
  assert(num_blobs == values.size());

-#ifndef NDEBUG
-  for (size_t i = 0; i < offsets.size() - 1; ++i) {
-    assert(offsets[i] <= offsets[i + 1]);
-  }
-#endif  // !NDEBUG
-
  std::vector<FSReadRequest> read_reqs(num_blobs);
  autovector<uint64_t> adjustments;
  uint64_t total_len = 0;
@@ -44,7 +44,6 @@ class BlobFileReader {
                 CompressionType compression_type, PinnableSlice* value,
                 uint64_t* bytes_read) const;

-  // offsets must be sorted in ascending order by caller.
  void MultiGetBlob(
      const ReadOptions& read_options,
      const autovector<std::reference_wrapper<const Slice>>& user_keys,
@@ -134,15 +134,16 @@ void WriteBlobFile(const ImmutableOptions& immutable_options,

 class BlobFileReaderTest : public testing::Test {
 protected:
-  BlobFileReaderTest() { mock_env_.reset(MockEnv::Create(Env::Default())); }
-  std::unique_ptr<Env> mock_env_;
+  BlobFileReaderTest() : mock_env_(Env::Default()) {}
+
+  MockEnv mock_env_;
 };

 TEST_F(BlobFileReaderTest, CreateReaderAndGetBlob) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileReaderTest_CreateReaderAndGetBlob"),
      0);
  options.enable_blob_files = true;
@@ -399,10 +400,9 @@ TEST_F(BlobFileReaderTest, Malformed) {
  // detect the error when we open it for reading

  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(), "BlobFileReaderTest_Malformed"),
-      0);
+      test::PerThreadDBPath(&mock_env_, "BlobFileReaderTest_Malformed"), 0);
  options.enable_blob_files = true;

  ImmutableOptions immutable_options(options);
@@ -452,9 +452,9 @@ TEST_F(BlobFileReaderTest, Malformed) {

 TEST_F(BlobFileReaderTest, TTL) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(), "BlobFileReaderTest_TTL"), 0);
+      test::PerThreadDBPath(&mock_env_, "BlobFileReaderTest_TTL"), 0);
  options.enable_blob_files = true;

  ImmutableOptions immutable_options(options);
@@ -486,9 +486,9 @@ TEST_F(BlobFileReaderTest, TTL) {

 TEST_F(BlobFileReaderTest, ExpirationRangeInHeader) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileReaderTest_ExpirationRangeInHeader"),
      0);
  options.enable_blob_files = true;
@@ -525,9 +525,9 @@ TEST_F(BlobFileReaderTest, ExpirationRangeInHeader) {

 TEST_F(BlobFileReaderTest, ExpirationRangeInFooter) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileReaderTest_ExpirationRangeInFooter"),
      0);
  options.enable_blob_files = true;
@@ -564,9 +564,9 @@ TEST_F(BlobFileReaderTest, ExpirationRangeInFooter) {

 TEST_F(BlobFileReaderTest, IncorrectColumnFamily) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileReaderTest_IncorrectColumnFamily"),
      0);
  options.enable_blob_files = true;
@@ -602,10 +602,9 @@ TEST_F(BlobFileReaderTest, IncorrectColumnFamily) {

 TEST_F(BlobFileReaderTest, BlobCRCError) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(), "BlobFileReaderTest_BlobCRCError"),
-      0);
+      test::PerThreadDBPath(&mock_env_, "BlobFileReaderTest_BlobCRCError"), 0);
  options.enable_blob_files = true;

  ImmutableOptions immutable_options(options);
@@ -661,10 +660,9 @@ TEST_F(BlobFileReaderTest, Compression) {
  }

  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(), "BlobFileReaderTest_Compression"),
-      0);
+      test::PerThreadDBPath(&mock_env_, "BlobFileReaderTest_Compression"), 0);
  options.enable_blob_files = true;

  ImmutableOptions immutable_options(options);
@@ -728,9 +726,9 @@ TEST_F(BlobFileReaderTest, UncompressionError) {
  }

  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileReaderTest_UncompressionError"),
      0);
  options.enable_blob_files = true;
@@ -787,13 +785,13 @@ class BlobFileReaderIOErrorTest
    : public testing::Test,
      public testing::WithParamInterface<std::string> {
 protected:
-  BlobFileReaderIOErrorTest() : sync_point_(GetParam()) {
-    mock_env_.reset(MockEnv::Create(Env::Default()));
-    fault_injection_env_.reset(new FaultInjectionTestEnv(mock_env_.get()));
-  }
+  BlobFileReaderIOErrorTest()
+      : mock_env_(Env::Default()),
+        fault_injection_env_(&mock_env_),
+        sync_point_(GetParam()) {}

-  std::unique_ptr<Env> mock_env_;
-  std::unique_ptr<FaultInjectionTestEnv> fault_injection_env_;
+  MockEnv mock_env_;
+  FaultInjectionTestEnv fault_injection_env_;
  std::string sync_point_;
 };

@@ -809,9 +807,9 @@ TEST_P(BlobFileReaderIOErrorTest, IOError) {
  // Simulates an I/O error during the specified step

  Options options;
-  options.env = fault_injection_env_.get();
+  options.env = &fault_injection_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(fault_injection_env_.get(),
+      test::PerThreadDBPath(&fault_injection_env_,
                            "BlobFileReaderIOErrorTest_IOError"),
      0);
  options.enable_blob_files = true;
@@ -833,8 +831,8 @@ TEST_P(BlobFileReaderIOErrorTest, IOError) {
                &blob_offset, &blob_size);

  SyncPoint::GetInstance()->SetCallBack(sync_point_, [this](void* /* arg */) {
-    fault_injection_env_->SetFilesystemActive(false,
-                                              Status::IOError(sync_point_));
+    fault_injection_env_.SetFilesystemActive(false,
+                                             Status::IOError(sync_point_));
  });
  SyncPoint::GetInstance()->EnableProcessing();

@@ -872,11 +870,10 @@ class BlobFileReaderDecodingErrorTest
    : public testing::Test,
      public testing::WithParamInterface<std::string> {
 protected:
-  BlobFileReaderDecodingErrorTest() : sync_point_(GetParam()) {
-    mock_env_.reset(MockEnv::Create(Env::Default()));
-  }
+  BlobFileReaderDecodingErrorTest()
+      : mock_env_(Env::Default()), sync_point_(GetParam()) {}

-  std::unique_ptr<Env> mock_env_;
+  MockEnv mock_env_;
  std::string sync_point_;
 };

@@ -888,9 +885,9 @@ INSTANTIATE_TEST_CASE_P(BlobFileReaderTest, BlobFileReaderDecodingErrorTest,

 TEST_P(BlobFileReaderDecodingErrorTest, DecodingError) {
  Options options;
-  options.env = mock_env_.get();
+  options.env = &mock_env_;
  options.cf_paths.emplace_back(
-      test::PerThreadDBPath(mock_env_.get(),
+      test::PerThreadDBPath(&mock_env_,
                            "BlobFileReaderDecodingErrorTest_DecodingError"),
      0);
  options.enable_blob_files = true;
@@ -15,6 +15,7 @@

 #include "db/blob/blob_file_builder.h"
 #include "db/compaction/compaction_iterator.h"
+#include "db/dbformat.h"
 #include "db/event_helpers.h"
 #include "db/internal_stats.h"
 #include "db/merge_helper.h"
@@ -2751,15 +2751,6 @@ double rocksdb_options_get_blob_gc_age_cutoff(rocksdb_options_t* opt) {
  return opt->rep.blob_garbage_collection_age_cutoff;
 }

-void rocksdb_options_set_blob_gc_force_threshold(rocksdb_options_t* opt,
-                                                 double val) {
-  opt->rep.blob_garbage_collection_force_threshold = val;
-}
-
-double rocksdb_options_get_blob_gc_force_threshold(rocksdb_options_t* opt) {
-  return opt->rep.blob_garbage_collection_force_threshold;
-}
-
 void rocksdb_options_set_num_levels(rocksdb_options_t* opt, int n) {
  opt->rep.num_levels = n;
 }
@@ -1793,11 +1793,8 @@ int main(int argc, char** argv) {
    rocksdb_options_set_enable_blob_gc(o, 1);
    CheckCondition(1 == rocksdb_options_get_enable_blob_gc(o));

-    rocksdb_options_set_blob_gc_age_cutoff(o, 0.5);
-    CheckCondition(0.5 == rocksdb_options_get_blob_gc_age_cutoff(o));
-
-    rocksdb_options_set_blob_gc_force_threshold(o, 0.75);
-    CheckCondition(0.75 == rocksdb_options_get_blob_gc_force_threshold(o));
+    rocksdb_options_set_blob_gc_age_cutoff(o, 0.75);
+    CheckCondition(0.75 == rocksdb_options_get_blob_gc_age_cutoff(o));

    // Create a copy that should be equal to the original.
    rocksdb_options_t* copy;
@@ -29,7 +29,6 @@
 #include "db/version_set.h"
 #include "db/write_controller.h"
 #include "file/sst_file_manager_impl.h"
-#include "logging/logging.h"
 #include "monitoring/thread_status_util.h"
 #include "options/options_helper.h"
 #include "port/port.h"
@@ -349,18 +348,12 @@ ColumnFamilyOptions SanitizeOptions(const ImmutableDBOptions& db_options,
  }

  if (result.level_compaction_dynamic_level_bytes) {
-    if (result.compaction_style != kCompactionStyleLevel) {
-      ROCKS_LOG_WARN(db_options.info_log.get(),
-                     "level_compaction_dynamic_level_bytes only makes sense"
-                     "for level-based compaction");
-      result.level_compaction_dynamic_level_bytes = false;
-    } else if (result.cf_paths.size() > 1U) {
-      // we don't yet know how to make both of this feature and multiple
-      // DB path work.
-      ROCKS_LOG_WARN(db_options.info_log.get(),
-                     "multiple cf_paths/db_paths and"
-                     "level_compaction_dynamic_level_bytes"
-                     "can't be used together");
+    if (result.compaction_style != kCompactionStyleLevel ||
+        result.cf_paths.size() > 1U) {
+      // 1. level_compaction_dynamic_level_bytes only makes sense for
+      //    level-based compaction.
+      // 2. we don't yet know how to make both of this feature and multiple
+      //    DB path work.
      result.level_compaction_dynamic_level_bytes = false;
    }
  }
@@ -1358,19 +1351,12 @@ Status ColumnFamilyData::ValidateOptions(
    }
  }

-  if (cf_options.enable_blob_garbage_collection) {
-    if (cf_options.blob_garbage_collection_age_cutoff < 0.0 ||
-        cf_options.blob_garbage_collection_age_cutoff > 1.0) {
-      return Status::InvalidArgument(
-          "The age cutoff for blob garbage collection should be in the range "
-          "[0.0, 1.0].");
-    }
-    if (cf_options.blob_garbage_collection_force_threshold < 0.0 ||
-        cf_options.blob_garbage_collection_force_threshold > 1.0) {
-      return Status::InvalidArgument(
-          "The garbage ratio threshold for forcing blob garbage collection "
-          "should be in the range [0.0, 1.0].");
-    }
+  if (cf_options.enable_blob_garbage_collection &&
+      (cf_options.blob_garbage_collection_age_cutoff < 0.0 ||
+       cf_options.blob_garbage_collection_age_cutoff > 1.0)) {
+    return Status::InvalidArgument(
+        "The age cutoff for blob garbage collection should be in the range "
+        "[0.0, 1.0].");
  }

  if (cf_options.compaction_style == kCompactionStyleFIFO &&
@@ -3407,30 +3407,6 @@ TEST(ColumnFamilyTest, ValidateBlobGCCutoff) {
                  .IsInvalidArgument());
 }

-TEST(ColumnFamilyTest, ValidateBlobGCForceThreshold) {
-  DBOptions db_options;
-
-  ColumnFamilyOptions cf_options;
-  cf_options.enable_blob_garbage_collection = true;
-
-  cf_options.blob_garbage_collection_force_threshold = -0.5;
-  ASSERT_TRUE(ColumnFamilyData::ValidateOptions(db_options, cf_options)
-                  .IsInvalidArgument());
-
-  cf_options.blob_garbage_collection_force_threshold = 0.0;
-  ASSERT_OK(ColumnFamilyData::ValidateOptions(db_options, cf_options));
-
-  cf_options.blob_garbage_collection_force_threshold = 0.5;
-  ASSERT_OK(ColumnFamilyData::ValidateOptions(db_options, cf_options));
-
-  cf_options.blob_garbage_collection_force_threshold = 1.0;
-  ASSERT_OK(ColumnFamilyData::ValidateOptions(db_options, cf_options));
-
-  cf_options.blob_garbage_collection_force_threshold = 1.5;
-  ASSERT_TRUE(ColumnFamilyData::ValidateOptions(db_options, cf_options)
-                  .IsInvalidArgument());
-}
-
 }  // namespace ROCKSDB_NAMESPACE

 #ifdef ROCKSDB_UNITTESTS_WITH_CUSTOM_OBJECTS_FROM_STATIC_LIBS
@@ -10,24 +10,22 @@
 #include <string>
 #include <vector>

-#include "db/dbformat.h"
 #include "rocksdb/comparator.h"
 #include "test_util/testharness.h"
 #include "test_util/testutil.h"
-#include "util/vector_iterator.h"

 namespace ROCKSDB_NAMESPACE {

 // A vector iterator which does its own bounds checking. This is for testing the
 // optimizations in the clipping iterator where we bypass the bounds checking if
 // the input iterator has already performed it.
-class BoundsCheckingVectorIterator : public VectorIterator {
+class BoundsCheckingVectorIterator : public test::VectorIterator {
 public:
  BoundsCheckingVectorIterator(const std::vector<std::string>& keys,
                               const std::vector<std::string>& values,
                               const Slice* start, const Slice* end,
                               const Comparator* cmp)
-      : VectorIterator(keys, values, cmp), start_(start), end_(end), cmp_(cmp) {
+      : VectorIterator(keys, values), start_(start), end_(end), cmp_(cmp) {
    assert(cmp_);
  }

@@ -107,7 +105,7 @@ TEST_P(ClippingIteratorTest, Clip) {
      use_bounds_checking_vec_it
          ? new BoundsCheckingVectorIterator(input_keys, input_values, &start,
                                             &end, BytewiseComparator())
-          : new VectorIterator(input_keys, input_values, BytewiseComparator()));
+          : new test::VectorIterator(input_keys, input_values));

  ClippingIterator clip(input.get(), &start, &end, BytewiseComparator());

@@ -11,7 +11,6 @@
 #include "db/blob/blob_file_builder.h"
 #include "db/blob/blob_index.h"
 #include "db/snapshot_checker.h"
-#include "logging/logging.h"
 #include "port/likely.h"
 #include "rocksdb/listener.h"
 #include "table/internal_iterator.h"
@@ -544,21 +543,9 @@ void CompactionIterator::NextFromInput() {
      // we can choose how to handle such a combinations of operations.  We will
      // try to compact out as much as we can in these cases.
      // We will report counts on these anomalous cases.
-      //
-      // Note: If timestamp is enabled, then record will be eligible for
-      // deletion, only if, along with above conditions (Rule 1 and Rule 2)
-      // full_history_ts_low_ is specified and timestamp for that key is less
-      // than *full_history_ts_low_. If it's not eligible for deletion, then we
-      // will output the SingleDelete. For Optimization 3 also, if
-      // full_history_ts_low_ is specified and timestamp for the key is less
-      // than *full_history_ts_low_ then only optimization will be applied.

      // The easiest way to process a SingleDelete during iteration is to peek
      // ahead at the next key.
-      const bool is_timestamp_eligible_for_gc =
-          (timestamp_size_ == 0 ||
-           (full_history_ts_low_ && cmp_with_history_ts_low_ < 0));
-
      ParsedInternalKey next_ikey;
      AdvanceInputIter();

@@ -567,7 +554,7 @@ void CompactionIterator::NextFromInput() {
      if (input_.Valid() &&
          ParseInternalKey(input_.key(), &next_ikey, allow_data_in_errors_)
              .ok() &&
-          cmp_->EqualWithoutTimestamp(ikey_.user_key, next_ikey.user_key)) {
+          cmp_->Equal(ikey_.user_key, next_ikey.user_key)) {
 #ifndef NDEBUG
        const Compaction* c =
            compaction_ ? compaction_->real_compaction() : nullptr;
@@ -575,6 +562,7 @@ void CompactionIterator::NextFromInput() {
        TEST_SYNC_POINT_CALLBACK(
            "CompactionIterator::NextFromInput:SingleDelete:1",
            const_cast<Compaction*>(c));
+
        // Check whether the next key belongs to the same snapshot as the
        // SingleDelete.
        if (prev_snapshot == 0 ||
@@ -582,20 +570,15 @@ void CompactionIterator::NextFromInput() {
          TEST_SYNC_POINT_CALLBACK(
              "CompactionIterator::NextFromInput:SingleDelete:2", nullptr);
          if (next_ikey.type == kTypeSingleDeletion) {
-            // We encountered two SingleDeletes for same key in a row. This
-            // could be due to unexpected user input. Skip the first
-            // SingleDelete and let the next iteration decide how to handle the
-            // second SingleDelete
+            // We encountered two SingleDeletes in a row.  This could be due to
+            // unexpected user input.
+            // Skip the first SingleDelete and let the next iteration decide how
+            // to handle the second SingleDelete

            // First SingleDelete has been skipped since we already called
            // input_.Next().
            ++iter_stats_.num_record_drop_obsolete;
            ++iter_stats_.num_single_del_mismatch;
-          } else if (!is_timestamp_eligible_for_gc) {
-            // We cannot drop the SingleDelete as timestamp is enabled, and
-            // timestamp of this key is greater than or equal to
-            // *full_history_ts_low_. We will output the SingleDelete.
-            valid_ = true;
          } else if (has_outputted_key_ ||
                     DefinitelyInSnapshot(ikey_.sequence,
                                          earliest_write_conflict_snapshot_)) {
@@ -652,8 +635,7 @@ void CompactionIterator::NextFromInput() {
        if (compaction_ != nullptr &&
            DefinitelyInSnapshot(ikey_.sequence, earliest_snapshot_) &&
            compaction_->KeyNotExistsBeyondOutputLevel(ikey_.user_key,
-                                                       &level_ptrs_) &&
-            is_timestamp_eligible_for_gc) {
+                                                       &level_ptrs_)) {
          // Key doesn't exist outside of this range.
          // Can compact out this SingleDelete.
          ++iter_stats_.num_record_drop_obsolete;
@@ -1005,12 +987,8 @@ void CompactionIterator::PrepareOutput() {
        ikeyNotNeededForIncrementalSnapshot() && bottommost_level_ &&
        DefinitelyInSnapshot(ikey_.sequence, earliest_snapshot_) &&
        ikey_.type != kTypeMerge) {
-      assert(ikey_.type != kTypeDeletion);
-      assert(ikey_.type != kTypeSingleDeletion ||
-             (timestamp_size_ || full_history_ts_low_));
-      if (ikey_.type == kTypeDeletion ||
-          (ikey_.type == kTypeSingleDeletion &&
-           (!timestamp_size_ || !full_history_ts_low_))) {
+      assert(ikey_.type != kTypeDeletion && ikey_.type != kTypeSingleDeletion);
+      if (ikey_.type == kTypeDeletion || ikey_.type == kTypeSingleDeletion) {
        ROCKS_LOG_FATAL(info_log_,
                        "Unexpected key type %d for seq-zero optimization",
                        ikey_.type);
@@ -3,17 +3,15 @@
 //  COPYING file in the root directory) and Apache 2.0 License
 //  (found in the LICENSE.Apache file in the root directory).

-#include "db/compaction/compaction_iterator.h"

 #include <string>
 #include <vector>

-#include "db/dbformat.h"
+#include "db/compaction/compaction_iterator.h"
 #include "port/port.h"
 #include "test_util/testharness.h"
 #include "test_util/testutil.h"
 #include "util/string_util.h"
-#include "util/vector_iterator.h"
 #include "utilities/merge_operators.h"

 namespace ROCKSDB_NAMESPACE {
@@ -88,7 +86,7 @@ class FilterAllKeysCompactionFilter : public CompactionFilter {
  const char* Name() const override { return "AllKeysCompactionFilter"; }
 };

-class LoggingForwardVectorIterator : public VectorIterator {
+class LoggingForwardVectorIterator : public InternalIterator {
 public:
  struct Action {
    enum class Type {
@@ -110,19 +108,22 @@ class LoggingForwardVectorIterator : public VectorIterator {

  LoggingForwardVectorIterator(const std::vector<std::string>& keys,
                               const std::vector<std::string>& values)
-      : VectorIterator(keys, values) {
-    current_ = keys_.size();
+      : keys_(keys), values_(values), current_(keys.size()) {
+    assert(keys_.size() == values_.size());
  }

+  bool Valid() const override { return current_ < keys_.size(); }
+
  void SeekToFirst() override {
    log.emplace_back(Action::Type::SEEK_TO_FIRST);
-    VectorIterator::SeekToFirst();
+    current_ = 0;
  }
  void SeekToLast() override { assert(false); }

  void Seek(const Slice& target) override {
    log.emplace_back(Action::Type::SEEK, target.ToString());
-    VectorIterator::Seek(target);
+    current_ = std::lower_bound(keys_.begin(), keys_.end(), target.ToString()) -
+               keys_.begin();
  }

  void SeekForPrev(const Slice& /*target*/) override { assert(false); }
@@ -130,20 +131,27 @@ class LoggingForwardVectorIterator : public VectorIterator {
  void Next() override {
    assert(Valid());
    log.emplace_back(Action::Type::NEXT);
-    VectorIterator::Next();
+    current_++;
  }
  void Prev() override { assert(false); }

  Slice key() const override {
    assert(Valid());
-    return VectorIterator::key();
+    return Slice(keys_[current_]);
  }
  Slice value() const override {
    assert(Valid());
-    return VectorIterator::value();
+    return Slice(values_[current_]);
  }

+  Status status() const override { return Status::OK(); }
+
  std::vector<Action> log;
+
+ private:
+  std::vector<std::string> keys_;
+  std::vector<std::string> values_;
+  size_t current_;
 };

 class FakeCompaction : public CompactionIterator::CompactionProxy {
@@ -236,7 +244,7 @@ class CompactionIteratorTest : public testing::TestWithParam<bool> {
      bool key_not_exists_beyond_output_level = false,
      const std::string* full_history_ts_low = nullptr) {
    std::unique_ptr<InternalIterator> unfragmented_range_del_iter(
-        new VectorIterator(range_del_ks, range_del_vs, &icmp_));
+        new test::VectorIterator(range_del_ks, range_del_vs));
    auto tombstone_list = std::make_shared<FragmentedRangeTombstoneList>(
        std::move(unfragmented_range_del_iter), icmp_);
    std::unique_ptr<FragmentedRangeTombstoneIterator> range_del_iter(
@@ -1250,101 +1258,6 @@ TEST_P(CompactionIteratorTsGcTest, RewriteTs) {
  }
 }

-TEST_P(CompactionIteratorTsGcTest, SingleDeleteNoKeyEligibleForGC) {
-  constexpr char user_key[][2] = {{'a', '\0'}, {'b', '\0'}};
-  const std::vector<std::string> input_keys = {
-      test::KeyStr(/*ts=*/104, user_key[0], /*seq=*/4, kTypeSingleDeletion),
-      test::KeyStr(/*ts=*/103, user_key[0], /*seq=*/3, kTypeValue),
-      test::KeyStr(/*ts=*/102, user_key[1], /*seq=*/2, kTypeValue)};
-  const std::vector<std::string> input_values = {"", "a3", "b2"};
-  std::string full_history_ts_low;
-  // All keys' timestamps are newer than or equal to 102, thus none of them
-  // will be eligible for GC.
-  PutFixed64(&full_history_ts_low, 102);
-  const std::vector<std::string>& expected_keys = input_keys;
-  const std::vector<std::string>& expected_values = input_values;
-  const std::vector<std::pair<bool, bool>> params = {
-      {false, false}, {false, true}, {true, true}};
-  for (const std::pair<bool, bool>& param : params) {
-    const bool bottommost_level = param.first;
-    const bool key_not_exists_beyond_output_level = param.second;
-    RunTest(input_keys, input_values, expected_keys, expected_values,
-            /*last_committed_seq=*/kMaxSequenceNumber,
-            /*merge_operator=*/nullptr, /*compaction_filter=*/nullptr,
-            bottommost_level,
-            /*earliest_write_conflict_snapshot=*/kMaxSequenceNumber,
-            key_not_exists_beyond_output_level, &full_history_ts_low);
-  }
-}
-
-TEST_P(CompactionIteratorTsGcTest, SingleDeleteDropTombstones) {
-  constexpr char user_key[] = "a";
-  const std::vector<std::string> input_keys = {
-      test::KeyStr(/*ts=*/103, user_key, /*seq=*/4, kTypeSingleDeletion),
-      test::KeyStr(/*ts=*/102, user_key, /*seq=*/3, kTypeValue),
-      test::KeyStr(/*ts=*/101, user_key, /*seq=*/2, kTypeSingleDeletion),
-      test::KeyStr(/*ts=*/100, user_key, /*seq=*/1, kTypeValue)};
-  const std::vector<std::string> input_values = {"", "a2", "", "a0"};
-  const std::vector<std::string> expected_keys = {input_keys[0], input_keys[1]};
-  const std::vector<std::string> expected_values = {"", "a2"};
-
-  // Take a snapshot at seq 2.
-  AddSnapshot(2);
-  {
-    const std::vector<std::pair<bool, bool>> params = {
-        {false, false}, {false, true}, {true, true}};
-    for (const std::pair<bool, bool>& param : params) {
-      const bool bottommost_level = param.first;
-      const bool key_not_exists_beyond_output_level = param.second;
-      std::string full_history_ts_low;
-      PutFixed64(&full_history_ts_low, 102);
-      RunTest(input_keys, input_values, expected_keys, expected_values,
-              /*last_committed_seq=*/kMaxSequenceNumber,
-              /*merge_operator=*/nullptr, /*compaction_filter=*/nullptr,
-              bottommost_level,
-              /*earliest_write_conflict_snapshot=*/kMaxSequenceNumber,
-              key_not_exists_beyond_output_level, &full_history_ts_low);
-    }
-  }
-}
-
-TEST_P(CompactionIteratorTsGcTest, SingleDeleteAllKeysOlderThanThreshold) {
-  constexpr char user_key[][2] = {{'a', '\0'}, {'b', '\0'}};
-  const std::vector<std::string> input_keys = {
-      test::KeyStr(/*ts=*/103, user_key[0], /*seq=*/4, kTypeSingleDeletion),
-      test::KeyStr(/*ts=*/102, user_key[0], /*seq=*/3, kTypeValue),
-      test::KeyStr(/*ts=*/104, user_key[1], /*seq=*/5, kTypeValue)};
-  const std::vector<std::string> input_values = {"", "a2", "b5"};
-  std::string full_history_ts_low;
-  PutFixed64(&full_history_ts_low, std::numeric_limits<uint64_t>::max());
-  {
-    // With a snapshot at seq 3, both the deletion marker and the key at 3 must
-    // be preserved.
-    AddSnapshot(3);
-    const std::vector<std::string> expected_keys = {
-        input_keys[0], input_keys[1], input_keys[2]};
-    const std::vector<std::string> expected_values = {"", "a2", "b5"};
-    RunTest(input_keys, input_values, expected_keys, expected_values,
-            /*last_committed_seq=*/kMaxSequenceNumber,
-            /*merge_operator=*/nullptr, /*compaction_filter=*/nullptr,
-            /*bottommost_level=*/false,
-            /*earliest_write_conflict_snapshot=*/kMaxSequenceNumber,
-            /*key_not_exists_beyond_output_level=*/false, &full_history_ts_low);
-    ClearSnapshots();
-  }
-  {
-    // No snapshot.
-    const std::vector<std::string> expected_keys = {input_keys[2]};
-    const std::vector<std::string> expected_values = {"b5"};
-    RunTest(input_keys, input_values, expected_keys, expected_values,
-            /*last_committed_seq=*/kMaxSequenceNumber,
-            /*merge_operator=*/nullptr, /*compaction_filter=*/nullptr,
-            /*bottommost_level=*/false,
-            /*earliest_write_conflict_snapshot=*/kMaxSequenceNumber,
-            /*key_not_exists_beyond_output_level=*/false, &full_history_ts_low);
-  }
-}
-
 INSTANTIATE_TEST_CASE_P(CompactionIteratorTsGcTestInstance,
                        CompactionIteratorTsGcTest,
                        testing::Values(true, false));
@@ -109,8 +109,6 @@ const char* GetCompactionReasonString(CompactionReason compaction_reason) {
      return "PeriodicCompaction";
    case CompactionReason::kChangeTemperature:
      return "ChangeTemperature";
-    case CompactionReason::kForcedBlobGC:
-      return "ForcedBlobGC";
    case CompactionReason::kNumOfReasons:
      // fall through
    default:
@@ -219,7 +217,6 @@ struct CompactionJob::SubcompactionState {
        &compaction->column_family_data()->internal_comparator();
    const std::vector<FileMetaData*>& grandparents = compaction->grandparents();

-    bool grandparant_file_switched = false;
    // Scan to find earliest grandparent file that contains key.
    while (grandparent_index < grandparents.size() &&
           icmp->Compare(internal_key,
@@ -227,7 +224,6 @@ struct CompactionJob::SubcompactionState {
               0) {
      if (seen_key) {
        overlapped_bytes += grandparents[grandparent_index]->fd.GetFileSize();
-        grandparant_file_switched = true;
      }
      assert(grandparent_index + 1 >= grandparents.size() ||
             icmp->Compare(
@@ -237,8 +233,8 @@ struct CompactionJob::SubcompactionState {
    }
    seen_key = true;

-    if (grandparant_file_switched && overlapped_bytes + curr_file_size >
-                                         compaction->max_compaction_bytes()) {
+    if (overlapped_bytes + curr_file_size >
+        compaction->max_compaction_bytes()) {
      // Too much overlap for current output; start new output
      overlapped_bytes = 0;
      return true;
@@ -1117,9 +1113,8 @@ CompactionJob::ProcessKeyValueCompactionWithCompactionService(
  sub_compact->num_output_records = compaction_result.num_output_records;
  sub_compact->approx_size = compaction_input.approx_size;  // is this used?
  sub_compact->total_bytes = compaction_result.total_bytes;
-  RecordTick(stats_, REMOTE_COMPACT_READ_BYTES, compaction_result.bytes_read);
-  RecordTick(stats_, REMOTE_COMPACT_WRITE_BYTES,
-             compaction_result.bytes_written);
+  IOSTATS_ADD(bytes_written, compaction_result.bytes_written);
+  IOSTATS_ADD(bytes_read, compaction_result.bytes_read);
  return CompactionServiceJobStatus::kSuccess;
 }
 #endif  // !ROCKSDB_LITE
@@ -2253,12 +2248,6 @@ std::string CompactionServiceCompactionJob::GetTableFileName(
  return MakeTableFileName(output_path_, file_number);
 }

-void CompactionServiceCompactionJob::RecordCompactionIOStats() {
-  compaction_result_->bytes_read += IOSTATS(bytes_read);
-  compaction_result_->bytes_written += IOSTATS(bytes_written);
-  CompactionJob::RecordCompactionIOStats();
-}
-
 CompactionServiceCompactionJob::CompactionServiceCompactionJob(
    int job_id, Compaction* compaction, const ImmutableDBOptions& db_options,
    const MutableDBOptions& mutable_db_options, const FileOptions& file_options,
@@ -2349,6 +2338,9 @@ Status CompactionServiceCompactionJob::Run() {
  // Finish up all book-keeping to unify the subcompaction results
  AggregateStatistics();
  UpdateCompactionStats();
+
+  compaction_result_->bytes_written = IOSTATS(bytes_written);
+  compaction_result_->bytes_read = IOSTATS(bytes_read);
  RecordCompactionIOStats();

  LogFlush(db_options_.info_log);
@@ -20,6 +20,7 @@
 #include "db/blob/blob_file_completion_callback.h"
 #include "db/column_family.h"
 #include "db/compaction/compaction_iterator.h"
+#include "db/dbformat.h"
 #include "db/flush_scheduler.h"
 #include "db/internal_stats.h"
 #include "db/job_context.h"
@@ -116,7 +117,7 @@ class CompactionJob {
  void AggregateStatistics();
  void UpdateCompactionStats();
  void LogCompaction();
-  virtual void RecordCompactionIOStats();
+  void RecordCompactionIOStats();
  void CleanupCompaction();

  // Call compaction filter. Then iterate through input and compact the
@@ -303,10 +304,10 @@ struct CompactionServiceResult {
  std::string output_path;

  // some statistics about the compaction
-  uint64_t num_output_records = 0;
-  uint64_t total_bytes = 0;
-  uint64_t bytes_read = 0;
-  uint64_t bytes_written = 0;
+  uint64_t num_output_records;
+  uint64_t total_bytes;
+  uint64_t bytes_read;
+  uint64_t bytes_written;
  CompactionJobStats stats;

  // serialization interface to read and write the object
@@ -346,9 +347,6 @@ class CompactionServiceCompactionJob : private CompactionJob {

  IOStatus io_status() const { return CompactionJob::io_status(); }

- protected:
-  void RecordCompactionIOStats() override;
-
 private:
  // Get table file name in output_path
  std::string GetTableFileName(uint64_t file_number) override;
@@ -1293,16 +1293,13 @@ TEST_F(CompactionJobTimestampTest, GCDisabled) {
  auto file1 =
      mock::MakeMockFile({{KeyStr("a", 10, ValueType::kTypeValue, 100), "a10"},
                          {KeyStr("a", 9, ValueType::kTypeValue, 99), "a9"},
-                          {KeyStr("b", 8, ValueType::kTypeValue, 98), "b8"},
-                          {KeyStr("d", 7, ValueType::kTypeValue, 97), "d7"}});
-
+                          {KeyStr("b", 8, ValueType::kTypeValue, 98), "b8"}});
  AddMockFile(file1);

  auto file2 = mock::MakeMockFile(
-      {{KeyStr("b", 6, ValueType::kTypeDeletionWithTimestamp, 96), ""},
-       {KeyStr("c", 5, ValueType::kTypeDeletionWithTimestamp, 95), ""},
-       {KeyStr("c", 4, ValueType::kTypeValue, 94), "c5"},
-       {KeyStr("d", 3, ValueType::kTypeSingleDeletion, 93), ""}});
+      {{KeyStr("b", 7, ValueType::kTypeDeletionWithTimestamp, 97), ""},
+       {KeyStr("c", 6, ValueType::kTypeDeletionWithTimestamp, 96), ""},
+       {KeyStr("c", 5, ValueType::kTypeValue, 95), "c5"}});
  AddMockFile(file2);

  SetLastSequence(10);
@@ -1311,11 +1308,9 @@ TEST_F(CompactionJobTimestampTest, GCDisabled) {
      {{KeyStr("a", 10, ValueType::kTypeValue, 100), "a10"},
       {KeyStr("a", 9, ValueType::kTypeValue, 99), "a9"},
       {KeyStr("b", 8, ValueType::kTypeValue, 98), "b8"},
-       {KeyStr("b", 6, ValueType::kTypeDeletionWithTimestamp, 96), ""},
-       {KeyStr("c", 5, ValueType::kTypeDeletionWithTimestamp, 95), ""},
-       {KeyStr("c", 4, ValueType::kTypeValue, 94), "c5"},
-       {KeyStr("d", 7, ValueType::kTypeValue, 97), "d7"},
-       {KeyStr("d", 3, ValueType::kTypeSingleDeletion, 93), ""}});
+       {KeyStr("b", 7, ValueType::kTypeDeletionWithTimestamp, 97), ""},
+       {KeyStr("c", 6, ValueType::kTypeDeletionWithTimestamp, 96), ""},
+       {KeyStr("c", 5, ValueType::kTypeValue, 95), "c5"}});
  const auto& files = cfd_->current()->storage_info()->LevelFiles(0);
  RunCompaction({files}, expected_results);
 }
@@ -1353,21 +1348,19 @@ TEST_F(CompactionJobTimestampTest, AllKeysExpired) {

  auto file1 = mock::MakeMockFile(
      {{KeyStr("a", 5, ValueType::kTypeDeletionWithTimestamp, 100), ""},
-       {KeyStr("b", 6, ValueType::kTypeSingleDeletion, 99), ""},
-       {KeyStr("c", 7, ValueType::kTypeValue, 98), "c7"}});
+       {KeyStr("b", 6, ValueType::kTypeValue, 99), "b6"}});
  AddMockFile(file1);

  auto file2 = mock::MakeMockFile(
-      {{KeyStr("a", 4, ValueType::kTypeValue, 97), "a4"},
-       {KeyStr("b", 3, ValueType::kTypeValue, 96), "b3"},
-       {KeyStr("c", 2, ValueType::kTypeDeletionWithTimestamp, 95), ""},
-       {KeyStr("c", 1, ValueType::kTypeValue, 94), "c1"}});
+      {{KeyStr("a", 4, ValueType::kTypeValue, 98), "a4"},
+       {KeyStr("b", 3, ValueType::kTypeDeletionWithTimestamp, 97), ""},
+       {KeyStr("b", 2, ValueType::kTypeValue, 96), "b2"}});
  AddMockFile(file2);

-  SetLastSequence(7);
+  SetLastSequence(6);

  auto expected_results =
-      mock::MakeMockFile({{KeyStr("c", 0, ValueType::kTypeValue, 0), "c7"}});
+      mock::MakeMockFile({{KeyStr("b", 0, ValueType::kTypeValue, 0), "b6"}});
  const auto& files = cfd_->current()->storage_info()->LevelFiles(0);

  full_history_ts_low_ = encode_u64_ts_(std::numeric_limits<uint64_t>::max());
@@ -15,11 +15,9 @@
 #include <string>
 #include <utility>
 #include <vector>
-
 #include "db/column_family.h"
 #include "file/filename.h"
 #include "logging/log_buffer.h"
-#include "logging/logging.h"
 #include "monitoring/statistics.h"
 #include "test_util/sync_point.h"
 #include "util/random.h"
@@ -554,14 +552,10 @@ void CompactionPicker::GetGrandparents(
  InternalKey start, limit;
  GetRange(inputs, output_level_inputs, &start, &limit);
  // Compute the set of grandparent files that overlap this compaction
-  // (parent == level+1; grandparent == level+2 or the first
-  // level after that has overlapping files)
-  for (int level = output_level_inputs.level + 1; level < NumberLevels();
-       level++) {
-    vstorage->GetOverlappingInputs(level, &start, &limit, grandparents);
-    if (!grandparents->empty()) {
-      break;
-    }
+  // (parent == level+1; grandparent == level+2)
+  if (output_level_inputs.level + 1 < NumberLevels()) {
+    vstorage->GetOverlappingInputs(output_level_inputs.level + 1, &start,
+                                   &limit, grandparents);
  }
 }

@@ -13,10 +13,8 @@
 #include <cinttypes>
 #include <string>
 #include <vector>
-
 #include "db/column_family.h"
 #include "logging/log_buffer.h"
-#include "logging/logging.h"
 #include "util/string_util.h"

 namespace ROCKSDB_NAMESPACE {
@@ -31,9 +31,6 @@ bool LevelCompactionPicker::NeedsCompaction(
  if (!vstorage->FilesMarkedForCompaction().empty()) {
    return true;
  }
-  if (!vstorage->FilesMarkedForForcedBlobGC().empty()) {
-    return true;
-  }
  for (int i = 0; i <= vstorage->MaxInputLevel(); i++) {
    if (vstorage->CompactionScore(i) >= 1) {
      return true;
@@ -251,13 +248,6 @@ void LevelCompactionBuilder::SetupInitialFiles() {
    compaction_reason_ = CompactionReason::kPeriodicCompaction;
    return;
  }
-
-  // Forced blob garbage collection
-  PickFileToCompact(vstorage_->FilesMarkedForForcedBlobGC(), false);
-  if (!start_level_inputs_.empty()) {
-    compaction_reason_ = CompactionReason::kForcedBlobGC;
-    return;
-  }
 }

 bool LevelCompactionBuilder::SetupOtherL0FilesIfNeeded() {
@@ -733,221 +733,6 @@ TEST_F(CompactionPickerTest, UniversalPeriodicCompaction6) {
  ASSERT_EQ(4, compaction->output_level());
 }

-TEST_F(CompactionPickerTest, UniversalIncrementalSpace1) {
-  const uint64_t kFileSize = 100000;
-
-  mutable_cf_options_.max_compaction_bytes = 555555;
-  mutable_cf_options_.compaction_options_universal.incremental = true;
-  mutable_cf_options_.compaction_options_universal
-      .max_size_amplification_percent = 30;
-  UniversalCompactionPicker universal_compaction_picker(ioptions_, &icmp_);
-
-  NewVersionStorage(5, kCompactionStyleUniversal);
-
-  Add(0, 1U, "150", "200", kFileSize, 0, 500, 550);
-  Add(2, 2U, "010", "080", kFileSize, 0, 200, 251);
-  Add(3, 5U, "310", "380", kFileSize, 0, 200, 251);
-  Add(3, 6U, "410", "880", kFileSize, 0, 200, 251);
-  Add(3, 7U, "910", "980", 1, 0, 200, 251);
-  Add(4, 10U, "201", "250", kFileSize, 0, 101, 150);
-  Add(4, 11U, "301", "350", kFileSize, 0, 101, 150);
-  Add(4, 12U, "401", "450", kFileSize, 0, 101, 150);
-  Add(4, 13U, "501", "750", kFileSize, 0, 101, 150);
-  Add(4, 14U, "801", "850", kFileSize, 0, 101, 150);
-  Add(4, 15U, "901", "950", kFileSize, 0, 101, 150);
-  //  Add(4, 15U, "960", "970", kFileSize, 0, 101, 150);
-
-  UpdateVersionStorageInfo();
-
-  std::unique_ptr<Compaction> compaction(
-      universal_compaction_picker.PickCompaction(
-          cf_name_, mutable_cf_options_, mutable_db_options_, vstorage_.get(),
-          &log_buffer_));
-  ASSERT_TRUE(compaction);
-  ASSERT_EQ(4, compaction->output_level());
-  ASSERT_EQ(3, compaction->start_level());
-  ASSERT_EQ(2U, compaction->num_input_files(0));
-  ASSERT_EQ(5U, compaction->input(0, 0)->fd.GetNumber());
-  ASSERT_EQ(6U, compaction->input(0, 1)->fd.GetNumber());
-  // ASSERT_EQ(4U, compaction->num_input_files(1));
-  ASSERT_EQ(11U, compaction->input(1, 0)->fd.GetNumber());
-  ASSERT_EQ(12U, compaction->input(1, 1)->fd.GetNumber());
-  ASSERT_EQ(13U, compaction->input(1, 2)->fd.GetNumber());
-  ASSERT_EQ(14U, compaction->input(1, 3)->fd.GetNumber());
-}
-
-TEST_F(CompactionPickerTest, UniversalIncrementalSpace2) {
-  const uint64_t kFileSize = 100000;
-
-  mutable_cf_options_.max_compaction_bytes = 400000;
-  mutable_cf_options_.compaction_options_universal.incremental = true;
-  mutable_cf_options_.compaction_options_universal
-      .max_size_amplification_percent = 30;
-  UniversalCompactionPicker universal_compaction_picker(ioptions_, &icmp_);
-
-  NewVersionStorage(5, kCompactionStyleUniversal);
-
-  Add(0, 1U, "150", "200", kFileSize, 0, 500, 550);
-  Add(1, 2U, "010", "080", kFileSize, 0, 200, 251);
-  Add(2, 5U, "310", "380", kFileSize, 0, 200, 251);
-  Add(2, 6U, "410", "880", kFileSize, 0, 200, 251);
-  Add(2, 7U, "910", "980", kFileSize, 0, 200, 251);
-  Add(4, 10U, "201", "250", kFileSize, 0, 101, 150);
-  Add(4, 11U, "301", "350", kFileSize, 0, 101, 150);
-  Add(4, 12U, "401", "450", kFileSize, 0, 101, 150);
-  Add(4, 13U, "501", "750", kFileSize, 0, 101, 150);
-  Add(4, 14U, "801", "850", kFileSize, 0, 101, 150);
-  Add(4, 15U, "901", "950", kFileSize, 0, 101, 150);
-
-  UpdateVersionStorageInfo();
-
-  std::unique_ptr<Compaction> compaction(
-      universal_compaction_picker.PickCompaction(
-          cf_name_, mutable_cf_options_, mutable_db_options_, vstorage_.get(),
-          &log_buffer_));
-  ASSERT_TRUE(compaction);
-  ASSERT_EQ(4, compaction->output_level());
-  ASSERT_EQ(2, compaction->start_level());
-  ASSERT_EQ(1U, compaction->num_input_files(0));
-  ASSERT_EQ(7U, compaction->input(0, 0)->fd.GetNumber());
-  ASSERT_EQ(1U, compaction->num_input_files(1));
-  ASSERT_EQ(15U, compaction->input(1, 0)->fd.GetNumber());
-}
-
-TEST_F(CompactionPickerTest, UniversalIncrementalSpace3) {
-  // Test bottom level files falling between gaps between two upper level
-  // files
-  const uint64_t kFileSize = 100000;
-
-  mutable_cf_options_.max_compaction_bytes = 300000;
-  mutable_cf_options_.compaction_options_universal.incremental = true;
-  mutable_cf_options_.compaction_options_universal
-      .max_size_amplification_percent = 30;
-  UniversalCompactionPicker universal_compaction_picker(ioptions_, &icmp_);
-
-  NewVersionStorage(5, kCompactionStyleUniversal);
-
-  Add(0, 1U, "150", "200", kFileSize, 0, 500, 550);
-  Add(2, 2U, "010", "080", kFileSize, 0, 200, 251);
-  Add(3, 5U, "000", "180", kFileSize, 0, 200, 251);
-  Add(3, 6U, "181", "190", kFileSize, 0, 200, 251);
-  Add(3, 7U, "710", "810", kFileSize, 0, 200, 251);
-  Add(3, 8U, "820", "830", kFileSize, 0, 200, 251);
-  Add(3, 9U, "900", "991", kFileSize, 0, 200, 251);
-  Add(4, 10U, "201", "250", kFileSize, 0, 101, 150);
-  Add(4, 11U, "301", "350", kFileSize, 0, 101, 150);
-  Add(4, 12U, "401", "450", kFileSize, 0, 101, 150);
-  Add(4, 13U, "501", "750", kFileSize, 0, 101, 150);
-  Add(4, 14U, "801", "850", kFileSize, 0, 101, 150);
-  Add(4, 15U, "901", "950", kFileSize, 0, 101, 150);
-
-  UpdateVersionStorageInfo();
-
-  std::unique_ptr<Compaction> compaction(
-      universal_compaction_picker.PickCompaction(
-          cf_name_, mutable_cf_options_, mutable_db_options_, vstorage_.get(),
-          &log_buffer_));
-  ASSERT_TRUE(compaction);
-  ASSERT_EQ(4, compaction->output_level());
-  ASSERT_EQ(2, compaction->start_level());
-  ASSERT_EQ(1U, compaction->num_input_files(0));
-  ASSERT_EQ(2U, compaction->input(0, 0)->fd.GetNumber());
-  ASSERT_EQ(2U, compaction->num_input_files(1));
-  ASSERT_EQ(5U, compaction->input(1, 0)->fd.GetNumber());
-  ASSERT_EQ(6U, compaction->input(1, 1)->fd.GetNumber());
-  ASSERT_EQ(0, compaction->num_input_files(2));
-}
-
-TEST_F(CompactionPickerTest, UniversalIncrementalSpace4) {
-  // Test compaction candidates always cover many files.
-  const uint64_t kFileSize = 100000;
-
-  mutable_cf_options_.max_compaction_bytes = 3200000;
-  mutable_cf_options_.compaction_options_universal.incremental = true;
-  mutable_cf_options_.compaction_options_universal
-      .max_size_amplification_percent = 30;
-  UniversalCompactionPicker universal_compaction_picker(ioptions_, &icmp_);
-
-  NewVersionStorage(5, kCompactionStyleUniversal);
-
-  Add(0, 1U, "150", "200", kFileSize, 0, 500, 550);
-  Add(2, 2U, "010", "080", kFileSize, 0, 200, 251);
-
-  // Generate files like following:
-  // L3: (1101, 1180) (1201, 1280) ... (7901, 7908)
-  // L4: (1130, 1150) (1160, 1210) (1230, 1250) (1260 1310) ... (7960, 8010)
-  for (int i = 11; i < 79; i++) {
-    Add(3, 100 + i * 3, ToString(i * 100).c_str(),
-        ToString(i * 100 + 80).c_str(), kFileSize, 0, 200, 251);
-    // Add a tie breaker
-    if (i == 66) {
-      Add(3, 10000U, "6690", "6699", kFileSize, 0, 200, 251);
-    }
-
-    Add(4, 100 + i * 3 + 1, ToString(i * 100 + 30).c_str(),
-        ToString(i * 100 + 50).c_str(), kFileSize, 0, 200, 251);
-    Add(4, 100 + i * 3 + 2, ToString(i * 100 + 60).c_str(),
-        ToString(i * 100 + 110).c_str(), kFileSize, 0, 200, 251);
-  }
-  UpdateVersionStorageInfo();
-
-  std::unique_ptr<Compaction> compaction(
-      universal_compaction_picker.PickCompaction(
-          cf_name_, mutable_cf_options_, mutable_db_options_, vstorage_.get(),
-          &log_buffer_));
-  ASSERT_TRUE(compaction);
-  ASSERT_EQ(4, compaction->output_level());
-  ASSERT_EQ(3, compaction->start_level());
-  ASSERT_EQ(6U, compaction->num_input_files(0));
-  ASSERT_EQ(100 + 62U * 3, compaction->input(0, 0)->fd.GetNumber());
-  ASSERT_EQ(10000U, compaction->input(0, 5)->fd.GetNumber());
-  ASSERT_EQ(11, compaction->num_input_files(1));
-}
-
-TEST_F(CompactionPickerTest, UniversalIncrementalSpace5) {
-  // Test compaction candidates always cover many files with some single
-  // files larger than size threshold.
-  const uint64_t kFileSize = 100000;
-
-  mutable_cf_options_.max_compaction_bytes = 3200000;
-  mutable_cf_options_.compaction_options_universal.incremental = true;
-  mutable_cf_options_.compaction_options_universal
-      .max_size_amplification_percent = 30;
-  UniversalCompactionPicker universal_compaction_picker(ioptions_, &icmp_);
-
-  NewVersionStorage(5, kCompactionStyleUniversal);
-
-  Add(0, 1U, "150", "200", kFileSize, 0, 500, 550);
-  Add(2, 2U, "010", "080", kFileSize, 0, 200, 251);
-
-  // Generate files like following:
-  // L3: (1101, 1180) (1201, 1280) ... (7901, 7908)
-  // L4: (1130, 1150) (1160, 1210) (1230, 1250) (1260 1310) ... (7960, 8010)
-  for (int i = 11; i < 70; i++) {
-    Add(3, 100 + i * 3, ToString(i * 100).c_str(),
-        ToString(i * 100 + 80).c_str(),
-        i % 10 == 9 ? kFileSize * 100 : kFileSize, 0, 200, 251);
-
-    Add(4, 100 + i * 3 + 1, ToString(i * 100 + 30).c_str(),
-        ToString(i * 100 + 50).c_str(), kFileSize, 0, 200, 251);
-    Add(4, 100 + i * 3 + 2, ToString(i * 100 + 60).c_str(),
-        ToString(i * 100 + 110).c_str(), kFileSize, 0, 200, 251);
-  }
-  UpdateVersionStorageInfo();
-
-  std::unique_ptr<Compaction> compaction(
-      universal_compaction_picker.PickCompaction(
-          cf_name_, mutable_cf_options_, mutable_db_options_, vstorage_.get(),
-          &log_buffer_));
-  ASSERT_TRUE(compaction);
-  ASSERT_EQ(4, compaction->output_level());
-  ASSERT_EQ(3, compaction->start_level());
-  ASSERT_EQ(6U, compaction->num_input_files(0));
-  ASSERT_EQ(100 + 14 * 3, compaction->input(0, 0)->fd.GetNumber());
-  ASSERT_EQ(100 + 19 * 3, compaction->input(0, 5)->fd.GetNumber());
-  ASSERT_EQ(13, compaction->num_input_files(1));
-}
-
 TEST_F(CompactionPickerTest, NeedsCompactionFIFO) {
  NewVersionStorage(1, kCompactionStyleFIFO);
  const int kFileCount =
@@ -15,11 +15,9 @@
 #include <queue>
 #include <string>
 #include <utility>
-
 #include "db/column_family.h"
 #include "file/filename.h"
 #include "logging/log_buffer.h"
-#include "logging/logging.h"
 #include "monitoring/statistics.h"
 #include "test_util/sync_point.h"
 #include "util/random.h"
@@ -89,14 +87,6 @@ class UniversalCompactionBuilder {
  // Pick Universal compaction to limit space amplification.
  Compaction* PickCompactionToReduceSizeAmp();

-  // Try to pick incremental compaction to reduce space amplification.
-  // It will return null if it cannot find a fanout within the threshold.
-  // Fanout is defined as
-  //    total size of files to compact at output level
-  //  --------------------------------------------------
-  //    total size of files to compact at other levels
-  Compaction* PickIncrementalForReduceSizeAmp(double fanout_threshold);
-
  Compaction* PickDeleteTriggeredCompaction();

  // Form a compaction from the sorted run indicated by start_index to the
@@ -112,14 +102,12 @@ class UniversalCompactionBuilder {
  // because some files are being compacted.
  Compaction* PickPeriodicCompaction();

-  // Used in universal compaction when the allow_trivial_move
+  // Used in universal compaction when the enabled_trivial_move
  // option is set. Checks whether there are any overlapping files
  // in the input. Returns true if the input files are non
  // overlapping.
  bool IsInputFilesNonOverlapping(Compaction* c);

-  uint64_t GetMaxOverlappingBytes() const;
-
  const ImmutableOptions& ioptions_;
  const InternalKeyComparator* icmp_;
  double score_;
@@ -724,19 +712,6 @@ Compaction* UniversalCompactionBuilder::PickCompactionToReduceSortedRuns(
                     cf_name_.c_str(), file_num_buf);
  }

-  std::vector<FileMetaData*> grandparents;
-  // Include grandparents for potential file cutting in incremental
-  // mode. It is for aligning file cutting boundaries across levels,
-  // so that subsequent compactions can pick files with aligned
-  // buffer.
-  // Single files are only picked up in incremental mode, so that
-  // there is no need for full range.
-  if (mutable_cf_options_.compaction_options_universal.incremental &&
-      first_index_after < sorted_runs_.size() &&
-      sorted_runs_[first_index_after].level > 1) {
-    grandparents = vstorage_->LevelFiles(sorted_runs_[first_index_after].level);
-  }
-
  CompactionReason compaction_reason;
  if (max_number_of_files_to_compact == UINT_MAX) {
    compaction_reason = CompactionReason::kUniversalSizeRatio;
@@ -748,14 +723,14 @@ Compaction* UniversalCompactionBuilder::PickCompactionToReduceSortedRuns(
      std::move(inputs), output_level,
      MaxFileSizeForLevel(mutable_cf_options_, output_level,
                          kCompactionStyleUniversal),
-      GetMaxOverlappingBytes(), path_id,
+      LLONG_MAX, path_id,
      GetCompressionType(ioptions_, vstorage_, mutable_cf_options_, start_level,
                         1, enable_compression),
      GetCompressionOptions(mutable_cf_options_, vstorage_, start_level,
                            enable_compression),
      Temperature::kUnknown,
-      /* max_subcompactions */ 0, grandparents, /* is manual */ false, score_,
-      false /* deletion_compaction */, compaction_reason);
+      /* max_subcompactions */ 0, /* grandparents */ {}, /* is manual */ false,
+      score_, false /* deletion_compaction */, compaction_reason);
 }

 // Look at overall size amplification. If size amplification
@@ -811,8 +786,6 @@ Compaction* UniversalCompactionBuilder::PickCompactionToReduceSizeAmp() {
  for (size_t loop = start_index; loop + 1 < sorted_runs_.size(); loop++) {
    sr = &sorted_runs_[loop];
    if (sr->being_compacted) {
-      // TODO with incremental compaction is supported, we might want to
-      // schedule some incremental compactions in parallel if needed.
      char file_num_buf[kFormatFileNumberBufSize];
      sr->Dump(file_num_buf, sizeof(file_num_buf), true);
      ROCKS_LOG_BUFFER(
@@ -846,250 +819,16 @@ Compaction* UniversalCompactionBuilder::PickCompactionToReduceSizeAmp() {
        " earliest-file-size %" PRIu64,
        cf_name_.c_str(), candidate_size, earliest_file_size);
  }
-  // Since incremental compaction can't include more than second last
-  // level, it can introduce penalty, compared to full compaction. We
-  // hard code the pentalty to be 80%. If we end up with a compaction
-  // fanout higher than 80% of full level compactions, we fall back
-  // to full level compaction.
-  // The 80% threshold is arbitrary and can be adjusted or made
-  // configurable in the future.
-  // This also prevent the case when compaction falls behind and we
-  // need to compact more levels for compactions to catch up.
-  if (mutable_cf_options_.compaction_options_universal.incremental) {
-    double fanout_threshold = static_cast<double>(earliest_file_size) /
-                              static_cast<double>(candidate_size) * 1.8;
-    Compaction* picked = PickIncrementalForReduceSizeAmp(fanout_threshold);
-    if (picked != nullptr) {
-      // As the feature is still incremental, picking incremental compaction
-      // might fail and we will fall bck to compacting full level.
-      return picked;
-    }
-  }
  return PickCompactionToOldest(start_index,
                                CompactionReason::kUniversalSizeAmplification);
 }

-Compaction* UniversalCompactionBuilder::PickIncrementalForReduceSizeAmp(
-    double fanout_threshold) {
-  // Try find all potential compactions with total size just over
-  // options.max_compaction_size / 2, and take the one with the lowest
-  // fanout (defined in declaration of the function).
-  // This is done by having a sliding window of the files at the second
-  // lowest level, and keep expanding while finding overlapping in the
-  // last level. Once total size exceeds the size threshold, calculate
-  // the fanout value. And then shrinking from the small side of the
-  // window. Keep doing it until the end.
-  // Finally, we try to include upper level files if they fall into
-  // the range.
-  //
-  // Note that it is a similar problem as leveled compaction's
-  // kMinOverlappingRatio priority, but instead of picking single files
-  // we expand to a target compaction size. The reason is that in
-  // leveled compaction, actual fanout value tends to high, e.g. 10, so
-  // even with single file in down merging level, the extra size
-  // compacted in boundary files is at a lower ratio. But here users
-  // often have size of second last level size to be 1/4, 1/3 or even
-  // 1/2 of the bottommost level, so picking single file in second most
-  // level will cause significant waste, which is not desirable.
-  //
-  // This algorithm has lots of room to improve to pick more efficient
-  // compactions.
-  assert(sorted_runs_.size() >= 2);
-  int second_last_level = sorted_runs_[sorted_runs_.size() - 2].level;
-  if (second_last_level == 0) {
-    // Can't split Level 0.
-    return nullptr;
-  }
-  int output_level = sorted_runs_.back().level;
-  const std::vector<FileMetaData*>& bottom_files =
-      vstorage_->LevelFiles(output_level);
-  const std::vector<FileMetaData*>& files =
-      vstorage_->LevelFiles(second_last_level);
-  assert(!bottom_files.empty());
-  assert(!files.empty());
-
-  //  std::unordered_map<uint64_t, uint64_t> file_to_order;
-
-  int picked_start_idx = 0;
-  int picked_end_idx = 0;
-  double picked_fanout = fanout_threshold;
-
-  // Use half target compaction bytes as anchor to stop growing second most
-  // level files, and reserve growing space for more overlapping bottom level,
-  // clean cut, files from other levels, etc.
-  uint64_t comp_thres_size = mutable_cf_options_.max_compaction_bytes / 2;
-  int start_idx = 0;
-  int bottom_end_idx = 0;
-  int bottom_start_idx = 0;
-  uint64_t non_bottom_size = 0;
-  uint64_t bottom_size = 0;
-  bool end_bottom_size_counted = false;
-  for (int end_idx = 0; end_idx < static_cast<int>(files.size()); end_idx++) {
-    FileMetaData* end_file = files[end_idx];
-
-    // Include bottom most level files smaller than the current second
-    // last level file.
-    int num_skipped = 0;
-    while (bottom_end_idx < static_cast<int>(bottom_files.size()) &&
-           icmp_->Compare(bottom_files[bottom_end_idx]->largest,
-                          end_file->smallest) < 0) {
-      if (!end_bottom_size_counted) {
-        bottom_size += bottom_files[bottom_end_idx]->fd.file_size;
-      }
-      bottom_end_idx++;
-      end_bottom_size_counted = false;
-      num_skipped++;
-    }
-
-    if (num_skipped > 1) {
-      // At least a file in the bottom most level falls into the file gap. No
-      // reason to include the file. We cut the range and start a new sliding
-      // window.
-      start_idx = end_idx;
-    }
-
-    if (start_idx == end_idx) {
-      // new sliding window.
-      non_bottom_size = 0;
-      bottom_size = 0;
-      bottom_start_idx = bottom_end_idx;
-      end_bottom_size_counted = false;
-    }
-
-    non_bottom_size += end_file->fd.file_size;
-
-    // Include all overlapping files in bottom level.
-    while (bottom_end_idx < static_cast<int>(bottom_files.size()) &&
-           icmp_->Compare(bottom_files[bottom_end_idx]->smallest,
-                          end_file->largest) < 0) {
-      if (!end_bottom_size_counted) {
-        bottom_size += bottom_files[bottom_end_idx]->fd.file_size;
-        end_bottom_size_counted = true;
-      }
-      if (icmp_->Compare(bottom_files[bottom_end_idx]->largest,
-                         end_file->largest) > 0) {
-        // next level file cross large boundary of current file.
-        break;
-      }
-      bottom_end_idx++;
-      end_bottom_size_counted = false;
-    }
-
-    if ((non_bottom_size + bottom_size > comp_thres_size ||
-         end_idx == static_cast<int>(files.size()) - 1) &&
-        non_bottom_size > 0) {  // Do we alow 0 size file at all?
-      // If it is a better compaction, remember it in picked* variables.
-      double fanout = static_cast<double>(bottom_size) /
-                      static_cast<double>(non_bottom_size);
-      if (fanout < picked_fanout) {
-        picked_start_idx = start_idx;
-        picked_end_idx = end_idx;
-        picked_fanout = fanout;
-      }
-      // Shrink from the start end to under comp_thres_size
-      while (non_bottom_size + bottom_size > comp_thres_size &&
-             start_idx <= end_idx) {
-        non_bottom_size -= files[start_idx]->fd.file_size;
-        start_idx++;
-        if (start_idx < static_cast<int>(files.size())) {
-          while (bottom_start_idx <= bottom_end_idx &&
-                 icmp_->Compare(bottom_files[bottom_start_idx]->largest,
-                                files[start_idx]->smallest) < 0) {
-            bottom_size -= bottom_files[bottom_start_idx]->fd.file_size;
-            bottom_start_idx++;
-          }
-        }
-      }
-    }
-  }
-
-  if (picked_fanout >= fanout_threshold) {
-    assert(picked_fanout == fanout_threshold);
-    return nullptr;
-  }
-
-  std::vector<CompactionInputFiles> inputs;
-  CompactionInputFiles bottom_level_inputs;
-  CompactionInputFiles second_last_level_inputs;
-  second_last_level_inputs.level = second_last_level;
-  bottom_level_inputs.level = output_level;
-  for (int i = picked_start_idx; i <= picked_end_idx; i++) {
-    if (files[i]->being_compacted) {
-      return nullptr;
-    }
-    second_last_level_inputs.files.push_back(files[i]);
-  }
-  assert(!second_last_level_inputs.empty());
-  if (!picker_->ExpandInputsToCleanCut(cf_name_, vstorage_,
-                                       &second_last_level_inputs,
-                                       /*next_smallest=*/nullptr)) {
-    return nullptr;
-  }
-  // We might be able to avoid this binary search if we save and expand
-  // from bottom_start_idx and bottom_end_idx, but for now, we use
-  // SetupOtherInputs() for simplicity.
-  int parent_index = -1;  // Create and use bottom_start_idx?
-  if (!picker_->SetupOtherInputs(cf_name_, mutable_cf_options_, vstorage_,
-                                 &second_last_level_inputs,
-                                 &bottom_level_inputs, &parent_index,
-                                 /*base_index=*/-1)) {
-    return nullptr;
-  }
-
-  // Try to include files in upper levels if they fall into the range.
-  // Since we need to go from lower level up and this is in the reverse
-  // order, compared to level order, we first write to an reversed
-  // data structure and finally copy them to compaction inputs.
-  InternalKey smallest, largest;
-  picker_->GetRange(second_last_level_inputs, &smallest, &largest);
-  std::vector<CompactionInputFiles> inputs_reverse;
-  for (auto it = ++(++sorted_runs_.rbegin()); it != sorted_runs_.rend(); it++) {
-    SortedRun& sr = *it;
-    if (sr.level == 0) {
-      break;
-    }
-    std::vector<FileMetaData*> level_inputs;
-    vstorage_->GetCleanInputsWithinInterval(sr.level, &smallest, &largest,
-                                            &level_inputs);
-    if (!level_inputs.empty()) {
-      inputs_reverse.push_back({});
-      inputs_reverse.back().level = sr.level;
-      inputs_reverse.back().files = level_inputs;
-      picker_->GetRange(inputs_reverse.back(), &smallest, &largest);
-    }
-  }
-  for (auto it = inputs_reverse.rbegin(); it != inputs_reverse.rend(); it++) {
-    inputs.push_back(*it);
-  }
-
-  inputs.push_back(second_last_level_inputs);
-  inputs.push_back(bottom_level_inputs);
-
-  // TODO support multi paths?
-  uint32_t path_id = 0;
-  return new Compaction(
-      vstorage_, ioptions_, mutable_cf_options_, mutable_db_options_,
-      std::move(inputs), output_level,
-      MaxFileSizeForLevel(mutable_cf_options_, output_level,
-                          kCompactionStyleUniversal),
-      GetMaxOverlappingBytes(), path_id,
-      GetCompressionType(ioptions_, vstorage_, mutable_cf_options_,
-                         output_level, 1, true /* enable_compression */),
-      GetCompressionOptions(mutable_cf_options_, vstorage_, output_level,
-                            true /* enable_compression */),
-      Temperature::kUnknown,
-      /* max_subcompactions */ 0, /* grandparents */ {}, /* is manual */ false,
-      score_, false /* deletion_compaction */,
-      CompactionReason::kUniversalSizeAmplification);
-}
-
 // Pick files marked for compaction. Typically, files are marked by
 // CompactOnDeleteCollector due to the presence of tombstones.
 Compaction* UniversalCompactionBuilder::PickDeleteTriggeredCompaction() {
  CompactionInputFiles start_level_inputs;
  int output_level;
  std::vector<CompactionInputFiles> inputs;
-  std::vector<FileMetaData*> grandparents;

  if (vstorage_->num_levels() == 1) {
    // This is single level universal. Since we're basically trying to reclaim
@@ -1196,9 +935,6 @@ Compaction* UniversalCompactionBuilder::PickDeleteTriggeredCompaction() {
      if (picker_->FilesRangeOverlapWithCompaction(inputs, output_level)) {
        return nullptr;
      }
-
-      picker_->GetGrandparents(vstorage_, start_level_inputs,
-                               output_level_inputs, &grandparents);
    } else {
      inputs.push_back(start_level_inputs);
    }
@@ -1216,13 +952,13 @@ Compaction* UniversalCompactionBuilder::PickDeleteTriggeredCompaction() {
      std::move(inputs), output_level,
      MaxFileSizeForLevel(mutable_cf_options_, output_level,
                          kCompactionStyleUniversal),
-      /* max_grandparent_overlap_bytes */ GetMaxOverlappingBytes(), path_id,
+      /* max_grandparent_overlap_bytes */ LLONG_MAX, path_id,
      GetCompressionType(ioptions_, vstorage_, mutable_cf_options_,
                         output_level, 1),
      GetCompressionOptions(mutable_cf_options_, vstorage_, output_level),
      Temperature::kUnknown,
-      /* max_subcompactions */ 0, grandparents, /* is manual */ false, score_,
-      false /* deletion_compaction */,
+      /* max_subcompactions */ 0, /* grandparents */ {}, /* is manual */ false,
+      score_, false /* deletion_compaction */,
      CompactionReason::kFilesMarkedForCompaction);
 }

@@ -1290,7 +1026,7 @@ Compaction* UniversalCompactionBuilder::PickCompactionToOldest(
      std::move(inputs), output_level,
      MaxFileSizeForLevel(mutable_cf_options_, output_level,
                          kCompactionStyleUniversal),
-      GetMaxOverlappingBytes(), path_id,
+      LLONG_MAX, path_id,
      GetCompressionType(ioptions_, vstorage_, mutable_cf_options_,
                         output_level, 1, true /* enable_compression */),
      GetCompressionOptions(mutable_cf_options_, vstorage_, output_level,
@@ -1365,17 +1101,6 @@ Compaction* UniversalCompactionBuilder::PickPeriodicCompaction() {

  return c;
 }
-
-uint64_t UniversalCompactionBuilder::GetMaxOverlappingBytes() const {
-  if (!mutable_cf_options_.compaction_options_universal.incremental) {
-    return port::kMaxUint64;
-  } else {
-    // Try to align cutting boundary with files at the next level if the
-    // file isn't end up with 1/2 of target size, or it would overlap
-    // with two full size files at the next level.
-    return mutable_cf_options_.target_file_size_base / 2 * 3;
-  }
-}
 }  // namespace ROCKSDB_NAMESPACE

 #endif  // !ROCKSDB_LITE
@@ -310,9 +310,6 @@ TEST_P(CompactionServiceTest, BasicCompactions) {
  Options options = CurrentOptions();
  ReopenWithCompactionService(&options);

-  Statistics* primary_statistics = GetPrimaryStatistics();
-  Statistics* compactor_statistics = GetCompactorStatistics();
-
  for (int i = 0; i < 20; i++) {
    for (int j = 0; j < 10; j++) {
      int key_id = i * 10 + j;
@@ -340,24 +337,14 @@ TEST_P(CompactionServiceTest, BasicCompactions) {
    }
  }
  auto my_cs = GetCompactionService();
+  Statistics* compactor_statistics = GetCompactorStatistics();
  ASSERT_GE(my_cs->GetCompactionNum(), 1);

  // make sure the compaction statistics is only recorded on the remote side
-  ASSERT_GE(compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES), 1);
-  ASSERT_GE(compactor_statistics->getTickerCount(COMPACT_READ_BYTES), 1);
-  ASSERT_EQ(primary_statistics->getTickerCount(COMPACT_WRITE_BYTES), 0);
-  // even with remote compaction, primary host still needs to read SST files to
-  // `verify_table()`.
-  ASSERT_GE(primary_statistics->getTickerCount(COMPACT_READ_BYTES), 1);
-  // all the compaction write happens on the remote side
-  ASSERT_EQ(primary_statistics->getTickerCount(REMOTE_COMPACT_WRITE_BYTES),
-            compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES));
-  ASSERT_GE(primary_statistics->getTickerCount(REMOTE_COMPACT_READ_BYTES), 1);
-  ASSERT_GT(primary_statistics->getTickerCount(COMPACT_READ_BYTES),
-            primary_statistics->getTickerCount(REMOTE_COMPACT_READ_BYTES));
-  // compactor is already the remote side, which doesn't have remote
-  ASSERT_EQ(compactor_statistics->getTickerCount(REMOTE_COMPACT_READ_BYTES), 0);
-  ASSERT_EQ(compactor_statistics->getTickerCount(REMOTE_COMPACT_WRITE_BYTES),
+  ASSERT_GE(
+      compactor_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY), 1);
+  Statistics* primary_statistics = GetPrimaryStatistics();
+  ASSERT_EQ(primary_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY),
            0);

  // Test failed compaction
@@ -695,10 +682,10 @@ TEST_P(CompactionServiceTest, FallbackLocalAuto) {
  auto my_cs = GetCompactionService();
  Statistics* compactor_statistics = GetCompactorStatistics();
  Statistics* primary_statistics = GetPrimaryStatistics();
-  uint64_t compactor_write_bytes =
-      compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES);
-  uint64_t primary_write_bytes =
-      primary_statistics->getTickerCount(COMPACT_WRITE_BYTES);
+  uint64_t compactor_new_key =
+      compactor_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY);
+  uint64_t primary_new_key =
+      primary_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY);

  my_cs->OverrideStartStatus(CompactionServiceJobStatus::kUseLocal);

@@ -732,12 +719,11 @@ TEST_P(CompactionServiceTest, FallbackLocalAuto) {
  ASSERT_EQ(my_cs->GetCompactionNum(), 0);

  // make sure the compaction statistics is only recorded on the local side
-  ASSERT_EQ(compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES),
-            compactor_write_bytes);
-  ASSERT_GT(primary_statistics->getTickerCount(COMPACT_WRITE_BYTES),
-            primary_write_bytes);
-  ASSERT_EQ(primary_statistics->getTickerCount(REMOTE_COMPACT_READ_BYTES), 0);
-  ASSERT_EQ(primary_statistics->getTickerCount(REMOTE_COMPACT_WRITE_BYTES), 0);
+  ASSERT_EQ(
+      compactor_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY),
+      compactor_new_key);
+  ASSERT_GT(primary_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY),
+            primary_new_key);
 }

 TEST_P(CompactionServiceTest, FallbackLocalManual) {
@@ -751,10 +737,10 @@ TEST_P(CompactionServiceTest, FallbackLocalManual) {
  auto my_cs = GetCompactionService();
  Statistics* compactor_statistics = GetCompactorStatistics();
  Statistics* primary_statistics = GetPrimaryStatistics();
-  uint64_t compactor_write_bytes =
-      compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES);
-  uint64_t primary_write_bytes =
-      primary_statistics->getTickerCount(COMPACT_WRITE_BYTES);
+  uint64_t compactor_new_key =
+      compactor_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY);
+  uint64_t primary_new_key =
+      primary_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY);

  // re-enable remote compaction
  my_cs->ResetOverride();
@@ -767,32 +753,31 @@ TEST_P(CompactionServiceTest, FallbackLocalManual) {
  ASSERT_OK(db_->CompactRange(CompactRangeOptions(), &start, &end));
  ASSERT_GE(my_cs->GetCompactionNum(), comp_num + 1);
  // make sure the compaction statistics is only recorded on the remote side
-  ASSERT_GT(compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES),
-            compactor_write_bytes);
-  ASSERT_EQ(primary_statistics->getTickerCount(REMOTE_COMPACT_WRITE_BYTES),
-            compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES));
-  ASSERT_EQ(primary_statistics->getTickerCount(COMPACT_WRITE_BYTES),
-            primary_write_bytes);
+  ASSERT_GT(
+      compactor_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY),
+      compactor_new_key);
+  ASSERT_EQ(primary_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY),
+            primary_new_key);

  // return run local again with API WaitForComplete
  my_cs->OverrideWaitStatus(CompactionServiceJobStatus::kUseLocal);
  start_str = Key(120);
  start = start_str;
  comp_num = my_cs->GetCompactionNum();
-  compactor_write_bytes =
-      compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES);
-  primary_write_bytes = primary_statistics->getTickerCount(COMPACT_WRITE_BYTES);
+  compactor_new_key =
+      compactor_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY);
+  primary_new_key =
+      primary_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY);

  ASSERT_OK(db_->CompactRange(CompactRangeOptions(), &start, nullptr));
  ASSERT_EQ(my_cs->GetCompactionNum(),
            comp_num);  // no remote compaction is run
  // make sure the compaction statistics is only recorded on the local side
-  ASSERT_EQ(compactor_statistics->getTickerCount(COMPACT_WRITE_BYTES),
-            compactor_write_bytes);
-  ASSERT_GT(primary_statistics->getTickerCount(COMPACT_WRITE_BYTES),
-            primary_write_bytes);
-  ASSERT_EQ(primary_statistics->getTickerCount(REMOTE_COMPACT_WRITE_BYTES),
-            compactor_write_bytes);
+  ASSERT_EQ(
+      compactor_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY),
+      compactor_new_key);
+  ASSERT_GT(primary_statistics->getTickerCount(COMPACTION_KEY_DROP_NEWER_ENTRY),
+            primary_new_key);

  // verify result after 2 manual compactions
  VerifyTestData();
@@ -8,24 +8,7 @@

 #include <algorithm>

-#include "rocksdb/utilities/customizable_util.h"
-#include "rocksdb/utilities/object_registry.h"
-#include "rocksdb/utilities/options_type.h"
-
 namespace ROCKSDB_NAMESPACE {
-static std::unordered_map<std::string, OptionTypeInfo>
-    sst_fixed_prefix_type_info = {
-#ifndef ROCKSDB_LITE
-        {"length",
-         {0, OptionType::kSizeT, OptionVerificationType::kNormal,
-          OptionTypeFlags::kNone}},
-#endif  // ROCKSDB_LITE
-};
-
-SstPartitionerFixedPrefixFactory::SstPartitionerFixedPrefixFactory(size_t len)
-    : len_(len) {
-  RegisterOptions("Length", &len_, &sst_fixed_prefix_type_info);
-}

 PartitionerResult SstPartitionerFixedPrefix::ShouldPartition(
    const PartitionerRequest& request) {
@@ -58,33 +41,4 @@ std::shared_ptr<SstPartitionerFactory> NewSstPartitionerFixedPrefixFactory(
  return std::make_shared<SstPartitionerFixedPrefixFactory>(prefix_len);
 }

-#ifndef ROCKSDB_LITE
-namespace {
-static int RegisterSstPartitionerFactories(ObjectLibrary& library,
-                                           const std::string& /*arg*/) {
-  library.Register<SstPartitionerFactory>(
-      SstPartitionerFixedPrefixFactory::kClassName(),
-      [](const std::string& /*uri*/,
-         std::unique_ptr<SstPartitionerFactory>* guard,
-         std::string* /* errmsg */) {
-        guard->reset(new SstPartitionerFixedPrefixFactory(0));
-        return guard->get();
-      });
-  return 1;
-}
-}  // namespace
-#endif  // ROCKSDB_LITE
-
-Status SstPartitionerFactory::CreateFromString(
-    const ConfigOptions& options, const std::string& value,
-    std::shared_ptr<SstPartitionerFactory>* result) {
-#ifndef ROCKSDB_LITE
-  static std::once_flag once;
-  std::call_once(once, [&]() {
-    RegisterSstPartitionerFactories(*(ObjectLibrary::Default().get()), "");
-  });
-#endif  // ROCKSDB_LITE
-  return LoadSharedObject<SstPartitionerFactory>(options, value, nullptr,
-                                                 result);
-}
 }  // namespace ROCKSDB_NAMESPACE
@@ -6,7 +6,6 @@
 #ifndef ROCKSDB_LITE

 #include "db/db_impl/db_impl.h"
-#include "db/db_test_util.h"
 #include "rocksdb/db.h"
 #include "rocksdb/env.h"
 #include "table/cuckoo/cuckoo_table_factory.h"
@@ -134,7 +133,6 @@ TEST_F(CuckooTableDBTest, Flush) {

  TablePropertiesCollection ptc;
  ASSERT_OK(reinterpret_cast<DB*>(dbfull())->GetPropertiesOfAllTables(&ptc));
-  VerifySstUniqueIds(ptc);
  ASSERT_EQ(1U, ptc.size());
  ASSERT_EQ(3U, ptc.begin()->second->num_entries);
  ASSERT_EQ("1", FilesPerLevel());
@@ -151,7 +149,6 @@ TEST_F(CuckooTableDBTest, Flush) {
  ASSERT_OK(dbfull()->TEST_FlushMemTable());

  ASSERT_OK(reinterpret_cast<DB*>(dbfull())->GetPropertiesOfAllTables(&ptc));
-  VerifySstUniqueIds(ptc);
  ASSERT_EQ(2U, ptc.size());
  auto row = ptc.begin();
  ASSERT_EQ(3U, row->second->num_entries);
@@ -169,7 +166,6 @@ TEST_F(CuckooTableDBTest, Flush) {
  ASSERT_OK(Delete("key4"));
  ASSERT_OK(dbfull()->TEST_FlushMemTable());
  ASSERT_OK(reinterpret_cast<DB*>(dbfull())->GetPropertiesOfAllTables(&ptc));
-  VerifySstUniqueIds(ptc);
  ASSERT_EQ(3U, ptc.size());
  row = ptc.begin();
  ASSERT_EQ(3U, row->second->num_entries);
@@ -194,7 +190,6 @@ TEST_F(CuckooTableDBTest, FlushWithDuplicateKeys) {

  TablePropertiesCollection ptc;
  ASSERT_OK(reinterpret_cast<DB*>(dbfull())->GetPropertiesOfAllTables(&ptc));
-  VerifySstUniqueIds(ptc);
  ASSERT_EQ(1U, ptc.size());
  ASSERT_EQ(2U, ptc.begin()->second->num_entries);
  ASSERT_EQ("1", FilesPerLevel());
@@ -1130,15 +1130,9 @@ TEST_F(DBBasicTest, DBCloseFlushError) {
  ASSERT_OK(Put("key3", "value3"));
  fault_injection_env->SetFilesystemActive(false);
  Status s = dbfull()->Close();
-  ASSERT_NE(s, Status::OK());
-  // retry should return the same error
-  s = dbfull()->Close();
-  ASSERT_NE(s, Status::OK());
  fault_injection_env->SetFilesystemActive(true);
-  // retry close() is no-op even the system is back. Could be improved if
-  // Close() is retry-able: #9029
-  s = dbfull()->Close();
  ASSERT_NE(s, Status::OK());
+
  Destroy(options);
 }

@@ -2597,21 +2591,6 @@ TEST_F(DBBasicTest, ManifestChecksumMismatch) {
  ASSERT_TRUE(s.IsCorruption());
 }

-TEST_F(DBBasicTest, ConcurrentlyCloseDB) {
-  Options options = CurrentOptions();
-  DestroyAndReopen(options);
-  std::vector<std::thread> workers;
-  for (int i = 0; i < 10; i++) {
-    workers.push_back(std::thread([&]() {
-      auto s = db_->Close();
-      ASSERT_OK(s);
-    }));
-  }
-  for (auto& w : workers) {
-    w.join();
-  }
-}
-
 #ifndef ROCKSDB_LITE
 class DBBasicTestTrackWal : public DBTestBase,
                            public testing::WithParamInterface<bool> {
@@ -1820,8 +1820,8 @@ TEST_F(DBBloomFilterTest, DynamicBloomFilterUpperBound) {
      ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_PREFIX_USEFUL), 0);
    }
    ASSERT_OK(dbfull()->SetOptions({{"prefix_extractor", "fixed:5"}}));
-    ASSERT_EQ(dbfull()->GetOptions().prefix_extractor->AsString(),
-              "rocksdb.FixedPrefix.5");
+    ASSERT_EQ(0, strcmp(dbfull()->GetOptions().prefix_extractor->Name(),
+                        "rocksdb.FixedPrefix.5"));
    {
      // BF changed, [abcdxx00, abce) is a valid bound, will trigger BF read
      Slice upper_bound("abce");
@@ -1940,8 +1940,8 @@ TEST_F(DBBloomFilterTest, DynamicBloomFilterMultipleSST) {
    ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_PREFIX_CHECKED), 1);

    ASSERT_OK(dbfull()->SetOptions({{"prefix_extractor", "capped:3"}}));
-    ASSERT_EQ(dbfull()->GetOptions().prefix_extractor->AsString(),
-              "rocksdb.CappedPrefix.3");
+    ASSERT_EQ(0, strcmp(dbfull()->GetOptions().prefix_extractor->Name(),
+                        "rocksdb.CappedPrefix.3"));
    read_options.iterate_upper_bound = &upper_bound;
    std::unique_ptr<Iterator> iter(db_->NewIterator(read_options));
    ASSERT_EQ(CountIter(iter, "foo"), 2);
@@ -1974,8 +1974,8 @@ TEST_F(DBBloomFilterTest, DynamicBloomFilterMultipleSST) {
    }

    ASSERT_OK(dbfull()->SetOptions({{"prefix_extractor", "fixed:2"}}));
-    ASSERT_EQ(dbfull()->GetOptions().prefix_extractor->AsString(),
-              "rocksdb.FixedPrefix.2");
+    ASSERT_EQ(0, strcmp(dbfull()->GetOptions().prefix_extractor->Name(),
+                        "rocksdb.FixedPrefix.2"));
    // third SST with fixed:2 BF
    ASSERT_OK(Put("foo6", "bar6"));
    ASSERT_OK(Put("foo7", "bar7"));
@@ -2022,8 +2022,8 @@ TEST_F(DBBloomFilterTest, DynamicBloomFilterMultipleSST) {
      ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_PREFIX_USEFUL), 3);
    }
    ASSERT_OK(dbfull()->SetOptions({{"prefix_extractor", "capped:3"}}));
-    ASSERT_EQ(dbfull()->GetOptions().prefix_extractor->AsString(),
-              "rocksdb.CappedPrefix.3");
+    ASSERT_EQ(0, strcmp(dbfull()->GetOptions().prefix_extractor->Name(),
+                        "rocksdb.CappedPrefix.3"));
    {
      std::unique_ptr<Iterator> iter_all(db_->NewIterator(read_options));
      ASSERT_EQ(CountIter(iter_all, "foo"), 6);
@@ -2063,8 +2063,9 @@ TEST_F(DBBloomFilterTest, DynamicBloomFilterNewColumnFamily) {
    // create a new CF and set prefix_extractor dynamically
    options.prefix_extractor.reset(NewCappedPrefixTransform(3));
    CreateColumnFamilies({"ramen_dojo_" + std::to_string(iteration)}, options);
-    ASSERT_EQ(dbfull()->GetOptions(handles_[2]).prefix_extractor->AsString(),
-              "rocksdb.CappedPrefix.3");
+    ASSERT_EQ(0,
+              strcmp(dbfull()->GetOptions(handles_[2]).prefix_extractor->Name(),
+                     "rocksdb.CappedPrefix.3"));
    ASSERT_OK(Put(2, "foo3", "bar3"));
    ASSERT_OK(Put(2, "foo4", "bar4"));
    ASSERT_OK(Put(2, "foo5", "bar5"));
@@ -2080,8 +2081,9 @@ TEST_F(DBBloomFilterTest, DynamicBloomFilterNewColumnFamily) {
    }
    ASSERT_OK(
        dbfull()->SetOptions(handles_[2], {{"prefix_extractor", "fixed:2"}}));
-    ASSERT_EQ(dbfull()->GetOptions(handles_[2]).prefix_extractor->AsString(),
-              "rocksdb.FixedPrefix.2");
+    ASSERT_EQ(0,
+              strcmp(dbfull()->GetOptions(handles_[2]).prefix_extractor->Name(),
+                     "rocksdb.FixedPrefix.2"));
    {
      std::unique_ptr<Iterator> iter(
          db_->NewIterator(read_options, handles_[2]));
@@ -2146,8 +2148,8 @@ TEST_F(DBBloomFilterTest, DynamicBloomFilterOptions) {
    ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_PREFIX_USEFUL), 0);

    ASSERT_OK(dbfull()->SetOptions({{"prefix_extractor", "capped:3"}}));
-    ASSERT_EQ(dbfull()->GetOptions().prefix_extractor->AsString(),
-              "rocksdb.CappedPrefix.3");
+    ASSERT_EQ(0, strcmp(dbfull()->GetOptions().prefix_extractor->Name(),
+                        "rocksdb.CappedPrefix.3"));
    {
      std::unique_ptr<Iterator> iter(db_->NewIterator(read_options));
      // "fp*" should be skipped
@@ -4979,7 +4979,7 @@ TEST_P(DBCompactionDirectIOTest, DirectIO) {
  options.create_if_missing = true;
  options.disable_auto_compactions = true;
  options.use_direct_io_for_flush_and_compaction = GetParam();
-  options.env = MockEnv::Create(Env::Default());
+  options.env = new MockEnv(Env::Default());
  Reopen(options);
  bool readahead = false;
  SyncPoint::GetInstance()->SetCallBack(
@@ -5255,7 +5255,6 @@ TEST_F(DBCompactionTest, ManualCompactionMax) {
  generate_sst_func();
  uint64_t total_size = (l1_avg_size * 10) + (l2_avg_size * 100);
  opts.max_compaction_bytes = total_size / num_split;
-  opts.target_file_size_base = total_size / num_split;
  Reopen(opts);
  num_compactions.store(0);
  ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr));
@@ -5263,7 +5262,6 @@ TEST_F(DBCompactionTest, ManualCompactionMax) {

  // very small max_compaction_bytes, it should still move forward
  opts.max_compaction_bytes = l1_avg_size / 2;
-  opts.target_file_size_base = l1_avg_size / 2;
  DestroyAndReopen(opts);
  generate_sst_func();
  num_compactions.store(0);
@@ -5277,8 +5275,7 @@ TEST_F(DBCompactionTest, ManualCompactionMax) {
  generate_sst_func();
  total_size = (l1_avg_size * 10) + (l2_avg_size * 100);
  Status s = db_->SetOptions(
-      {{"max_compaction_bytes", std::to_string(total_size / num_split)},
-       {"target_file_size_base", std::to_string(total_size / num_split)}});
+      {{"max_compaction_bytes", std::to_string(total_size / num_split)}});
  ASSERT_OK(s);

  num_compactions.store(0);
@@ -13,9 +13,9 @@
 #if !defined(ROCKSDB_LITE)

 #include "db/db_test_util.h"
+#include "env/mock_env.h"
 #include "port/port.h"
 #include "port/stack_trace.h"
-#include "rocksdb/env.h"
 #include "util/random.h"

 namespace ROCKSDB_NAMESPACE {
@@ -30,7 +30,7 @@ TEST_F(DBTestDynamicLevel, DynamicLevelMaxBytesBase) {
    return;
  }
  // Use InMemoryEnv, or it would be too slow.
-  std::unique_ptr<Env> env(NewMemEnv(env_));
+  std::unique_ptr<Env> env(new MockEnv(env_));

  const int kNKeys = 1000;
  int keys[kNKeys];
@@ -6,67 +6,23 @@

 #ifndef ROCKSDB_LITE

+#include <stdint.h>
 #include <algorithm>
-#include <cstdint>
-#include <memory>
+#include <cinttypes>
 #include <string>
-#include <vector>
-
 #include "db/db_impl/db_impl.h"
 #include "db/job_context.h"
 #include "db/version_set.h"
 #include "file/file_util.h"
 #include "file/filename.h"
-#include "logging/logging.h"
 #include "port/port.h"
 #include "rocksdb/db.h"
 #include "rocksdb/env.h"
-#include "rocksdb/metadata.h"
-#include "rocksdb/types.h"
 #include "test_util/sync_point.h"
-#include "util/file_checksum_helper.h"
 #include "util/mutexlock.h"

 namespace ROCKSDB_NAMESPACE {

-Status DBImpl::FlushForGetLiveFiles() {
-  mutex_.AssertHeld();
-
-  // flush all dirty data to disk.
-  Status status;
-  if (immutable_db_options_.atomic_flush) {
-    autovector<ColumnFamilyData*> cfds;
-    SelectColumnFamiliesForAtomicFlush(&cfds);
-    mutex_.Unlock();
-    status =
-        AtomicFlushMemTables(cfds, FlushOptions(), FlushReason::kGetLiveFiles);
-    if (status.IsColumnFamilyDropped()) {
-      status = Status::OK();
-    }
-    mutex_.Lock();
-  } else {
-    for (auto cfd : *versions_->GetColumnFamilySet()) {
-      if (cfd->IsDropped()) {
-        continue;
-      }
-      cfd->Ref();
-      mutex_.Unlock();
-      status = FlushMemTable(cfd, FlushOptions(), FlushReason::kGetLiveFiles);
-      TEST_SYNC_POINT("DBImpl::GetLiveFiles:1");
-      TEST_SYNC_POINT("DBImpl::GetLiveFiles:2");
-      mutex_.Lock();
-      cfd->UnrefAndTryDelete();
-      if (!status.ok() && !status.IsColumnFamilyDropped()) {
-        break;
-      } else if (status.IsColumnFamilyDropped()) {
-        status = Status::OK();
-      }
-    }
-  }
-  versions_->GetColumnFamilySet()->FreeDeadColumnFamilies();
-  return status;
-}
-
 Status DBImpl::GetLiveFiles(std::vector<std::string>& ret,
                            uint64_t* manifest_file_size,
                            bool flush_memtable) {
@@ -75,7 +31,39 @@ Status DBImpl::GetLiveFiles(std::vector<std::string>& ret,
  mutex_.Lock();

  if (flush_memtable) {
-    Status status = FlushForGetLiveFiles();
+    // flush all dirty data to disk.
+    Status status;
+    if (immutable_db_options_.atomic_flush) {
+      autovector<ColumnFamilyData*> cfds;
+      SelectColumnFamiliesForAtomicFlush(&cfds);
+      mutex_.Unlock();
+      status = AtomicFlushMemTables(cfds, FlushOptions(),
+                                    FlushReason::kGetLiveFiles);
+      if (status.IsColumnFamilyDropped()) {
+        status = Status::OK();
+      }
+      mutex_.Lock();
+    } else {
+      for (auto cfd : *versions_->GetColumnFamilySet()) {
+        if (cfd->IsDropped()) {
+          continue;
+        }
+        cfd->Ref();
+        mutex_.Unlock();
+        status = FlushMemTable(cfd, FlushOptions(), FlushReason::kGetLiveFiles);
+        TEST_SYNC_POINT("DBImpl::GetLiveFiles:1");
+        TEST_SYNC_POINT("DBImpl::GetLiveFiles:2");
+        mutex_.Lock();
+        cfd->UnrefAndTryDelete();
+        if (!status.ok() && !status.IsColumnFamilyDropped()) {
+          break;
+        } else if (status.IsColumnFamilyDropped()) {
+          status = Status::OK();
+        }
+      }
+    }
+    versions_->GetColumnFamilySet()->FreeDeadColumnFamilies();
+
    if (!status.ok()) {
      mutex_.Unlock();
      ROCKS_LOG_ERROR(immutable_db_options_.info_log, "Cannot Flush data %s\n",
@@ -166,245 +154,6 @@ Status DBImpl::GetCurrentWalFile(std::unique_ptr<LogFile>* current_log_file) {

  return wal_manager_.GetLiveWalFile(current_logfile_number, current_log_file);
 }
-
-Status DBImpl::GetLiveFilesStorageInfo(
-    const LiveFilesStorageInfoOptions& opts,
-    std::vector<LiveFileStorageInfo>* files) {
-  // To avoid returning partial results, only move to ouput on success
-  assert(files);
-  files->clear();
-  std::vector<LiveFileStorageInfo> results;
-
-  // NOTE: This implementation was largely migrated from Checkpoint.
-
-  Status s;
-  VectorLogPtr live_wal_files;
-  bool flush_memtable = true;
-  if (!immutable_db_options_.allow_2pc) {
-    if (opts.wal_size_for_flush == port::kMaxUint64) {
-      flush_memtable = false;
-    } else if (opts.wal_size_for_flush > 0) {
-      // If out standing log files are small, we skip the flush.
-      s = GetSortedWalFiles(live_wal_files);
-
-      if (!s.ok()) {
-        return s;
-      }
-
-      // Don't flush column families if total log size is smaller than
-      // log_size_for_flush. We copy the log files instead.
-      // We may be able to cover 2PC case too.
-      uint64_t total_wal_size = 0;
-      for (auto& wal : live_wal_files) {
-        total_wal_size += wal->SizeFileBytes();
-      }
-      if (total_wal_size < opts.wal_size_for_flush) {
-        flush_memtable = false;
-      }
-      live_wal_files.clear();
-    }
-  }
-
-  // This is a modified version of GetLiveFiles, to get access to more
-  // metadata.
-  mutex_.Lock();
-  if (flush_memtable) {
-    Status status = FlushForGetLiveFiles();
-    if (!status.ok()) {
-      mutex_.Unlock();
-      ROCKS_LOG_ERROR(immutable_db_options_.info_log, "Cannot Flush data %s\n",
-                      status.ToString().c_str());
-      return status;
-    }
-  }
-
-  // Make a set of all of the live table and blob files
-  for (auto cfd : *versions_->GetColumnFamilySet()) {
-    if (cfd->IsDropped()) {
-      continue;
-    }
-    VersionStorageInfo& vsi = *cfd->current()->storage_info();
-    auto& cf_paths = cfd->ioptions()->cf_paths;
-
-    auto GetDir = [&](size_t path_id) {
-      // Matching TableFileName() behavior
-      if (path_id >= cf_paths.size()) {
-        assert(false);
-        return cf_paths.back().path;
-      } else {
-        return cf_paths[path_id].path;
-      }
-    };
-
-    for (int level = 0; level < vsi.num_levels(); ++level) {
-      const auto& level_files = vsi.LevelFiles(level);
-      for (const auto& meta : level_files) {
-        assert(meta);
-
-        results.emplace_back();
-        LiveFileStorageInfo& info = results.back();
-
-        info.relative_filename = MakeTableFileName(meta->fd.GetNumber());
-        info.directory = GetDir(meta->fd.GetPathId());
-        info.file_number = meta->fd.GetNumber();
-        info.file_type = kTableFile;
-        info.size = meta->fd.GetFileSize();
-        if (opts.include_checksum_info) {
-          info.file_checksum_func_name = meta->file_checksum_func_name;
-          info.file_checksum = meta->file_checksum;
-          if (info.file_checksum_func_name.empty()) {
-            info.file_checksum_func_name = kUnknownFileChecksumFuncName;
-            info.file_checksum = kUnknownFileChecksum;
-          }
-        }
-        info.temperature = meta->temperature;
-      }
-    }
-    const auto& blob_files = vsi.GetBlobFiles();
-    for (const auto& pair : blob_files) {
-      const auto& meta = pair.second;
-      assert(meta);
-
-      results.emplace_back();
-      LiveFileStorageInfo& info = results.back();
-
-      info.relative_filename = BlobFileName(meta->GetBlobFileNumber());
-      info.directory = GetName();  // TODO?: support db_paths/cf_paths
-      info.file_number = meta->GetBlobFileNumber();
-      info.file_type = kBlobFile;
-      info.size = meta->GetBlobFileSize();
-      if (opts.include_checksum_info) {
-        info.file_checksum_func_name = meta->GetChecksumMethod();
-        info.file_checksum = meta->GetChecksumValue();
-        if (info.file_checksum_func_name.empty()) {
-          info.file_checksum_func_name = kUnknownFileChecksumFuncName;
-          info.file_checksum = kUnknownFileChecksum;
-        }
-      }
-      // TODO?: info.temperature
-    }
-  }
-
-  // Capture some final info before releasing mutex
-  const uint64_t manifest_number = versions_->manifest_file_number();
-  const uint64_t manifest_size = versions_->manifest_file_size();
-  const uint64_t options_number = versions_->options_file_number();
-  const uint64_t options_size = versions_->options_file_size_;
-  const uint64_t min_log_num = MinLogNumberToKeep();
-
-  mutex_.Unlock();
-
-  std::string manifest_fname = DescriptorFileName(manifest_number);
-  {  // MANIFEST
-    results.emplace_back();
-    LiveFileStorageInfo& info = results.back();
-
-    info.relative_filename = manifest_fname;
-    info.directory = GetName();
-    info.file_number = manifest_number;
-    info.file_type = kDescriptorFile;
-    info.size = manifest_size;
-    info.trim_to_size = true;
-    if (opts.include_checksum_info) {
-      info.file_checksum_func_name = kUnknownFileChecksumFuncName;
-      info.file_checksum = kUnknownFileChecksum;
-    }
-  }
-
-  {  // CURRENT
-    results.emplace_back();
-    LiveFileStorageInfo& info = results.back();
-
-    info.relative_filename = kCurrentFileName;
-    info.directory = GetName();
-    info.file_type = kCurrentFile;
-    // CURRENT could be replaced so we have to record the contents we want
-    // for it
-    info.replacement_contents = manifest_fname + "\n";
-    info.size = manifest_fname.size() + 1;
-    if (opts.include_checksum_info) {
-      info.file_checksum_func_name = kUnknownFileChecksumFuncName;
-      info.file_checksum = kUnknownFileChecksum;
-    }
-  }
-
-  // The OPTIONS file number is zero in read-write mode when OPTIONS file
-  // writing failed and the DB was configured with
-  // `fail_if_options_file_error == false`. In read-only mode the OPTIONS file
-  // number is zero when no OPTIONS file exist at all. In those cases we do not
-  // record any OPTIONS file in the live file list.
-  if (options_number != 0) {
-    results.emplace_back();
-    LiveFileStorageInfo& info = results.back();
-
-    info.relative_filename = OptionsFileName(options_number);
-    info.directory = GetName();
-    info.file_number = options_number;
-    info.file_type = kOptionsFile;
-    info.size = options_size;
-    if (opts.include_checksum_info) {
-      info.file_checksum_func_name = kUnknownFileChecksumFuncName;
-      info.file_checksum = kUnknownFileChecksum;
-    }
-  }
-
-  // Some legacy testing stuff  TODO: carefully clean up obsolete parts
-  TEST_SYNC_POINT("CheckpointImpl::CreateCheckpoint:FlushDone");
-
-  TEST_SYNC_POINT("CheckpointImpl::CreateCheckpoint:SavedLiveFiles1");
-  TEST_SYNC_POINT("CheckpointImpl::CreateCheckpoint:SavedLiveFiles2");
-
-  if (s.ok()) {
-    s = FlushWAL(false /* sync */);
-  }
-
-  TEST_SYNC_POINT("CheckpointImpl::CreateCustomCheckpoint:AfterGetLive1");
-  TEST_SYNC_POINT("CheckpointImpl::CreateCustomCheckpoint:AfterGetLive2");
-
-  // if we have more than one column family, we need to also get WAL files
-  if (s.ok()) {
-    s = GetSortedWalFiles(live_wal_files);
-  }
-  if (!s.ok()) {
-    return s;
-  }
-
-  size_t wal_size = live_wal_files.size();
-
-  ROCKS_LOG_INFO(immutable_db_options_.info_log,
-                 "Number of log files %" ROCKSDB_PRIszt, live_wal_files.size());
-
-  // Link WAL files. Copy exact size of last one because it is the only one
-  // that has changes after the last flush.
-  auto wal_dir = immutable_db_options_.GetWalDir();
-  for (size_t i = 0; s.ok() && i < wal_size; ++i) {
-    if ((live_wal_files[i]->Type() == kAliveLogFile) &&
-        (!flush_memtable || live_wal_files[i]->LogNumber() >= min_log_num)) {
-      results.emplace_back();
-      LiveFileStorageInfo& info = results.back();
-      auto f = live_wal_files[i]->PathName();
-      assert(!f.empty() && f[0] == '/');
-      info.relative_filename = f.substr(1);
-      info.directory = wal_dir;
-      info.file_number = live_wal_files[i]->LogNumber();
-      info.file_type = kWalFile;
-      info.size = live_wal_files[i]->SizeFileBytes();
-      // Only last should need to be trimmed
-      info.trim_to_size = (i + 1 == wal_size);
-      if (opts.include_checksum_info) {
-        info.file_checksum_func_name = kUnknownFileChecksumFuncName;
-        info.file_checksum = kUnknownFileChecksum;
-      }
-    }
-  }
-
-  if (s.ok()) {
-    // Only move output on success
-    *files = std::move(results);
-  }
-  return s;
-}
-
 }  // namespace ROCKSDB_NAMESPACE

 #endif  // ROCKSDB_LITE
@@ -1381,7 +1381,7 @@ TEST_P(DBFlushDirectIOTest, DirectIO) {
  options.disable_auto_compactions = true;
  options.max_background_flushes = 2;
  options.use_direct_io_for_flush_and_compaction = GetParam();
-  options.env = MockEnv::Create(Env::Default());
+  options.env = new MockEnv(Env::Default());
  SyncPoint::GetInstance()->SetCallBack(
      "BuildTable:create_file", [&](void* arg) {
        bool* use_direct_writes = static_cast<bool*>(arg);
@@ -2440,122 +2440,6 @@ TEST_P(DBAtomicFlushTest, RollbackAfterFailToInstallResults) {
  SyncPoint::GetInstance()->ClearAllCallBacks();
 }

-// In atomic flush, concurrent bg flush threads commit to the MANIFEST in
-// serial, in the order of their picked memtables for each column family.
-// Only when a bg flush thread finds out that its memtables are the earliest
-// unflushed ones for all the included column families will this bg flush
-// thread continue to commit to MANIFEST.
-// This unit test uses sync point to coordinate the execution of two bg threads
-// executing the same sequence of functions. The interleaving are as follows.
-// time            bg1                            bg2
-//  |   pick memtables to flush
-//  |   flush memtables cf1_m1, cf2_m1
-//  |   join MANIFEST write queue
-//  |                                     pick memtabls to flush
-//  |                                     flush memtables cf1_(m1+1)
-//  |                                     join MANIFEST write queue
-//  |                                     wait to write MANIFEST
-//  |   write MANIFEST
-//  |   IO error
-//  |                                     detect IO error and stop waiting
-//  V
-TEST_P(DBAtomicFlushTest, BgThreadNoWaitAfterManifestError) {
-  bool atomic_flush = GetParam();
-  if (!atomic_flush) {
-    return;
-  }
-  auto fault_injection_env = std::make_shared<FaultInjectionTestEnv>(env_);
-  Options options = GetDefaultOptions();
-  options.create_if_missing = true;
-  options.atomic_flush = true;
-  options.env = fault_injection_env.get();
-  // Set a larger value than default so that RocksDB can schedule concurrent
-  // background flush threads.
-  options.max_background_jobs = 8;
-  options.max_write_buffer_number = 8;
-  CreateAndReopenWithCF({"pikachu"}, options);
-
-  assert(2 == handles_.size());
-
-  WriteOptions write_opts;
-  write_opts.disableWAL = true;
-
-  ASSERT_OK(Put(0, "a", "v_0_a", write_opts));
-  ASSERT_OK(Put(1, "a", "v_1_a", write_opts));
-
-  SyncPoint::GetInstance()->DisableProcessing();
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-
-  SyncPoint::GetInstance()->LoadDependency({
-      {"BgFlushThr2:WaitToCommit", "BgFlushThr1:BeforeWriteManifest"},
-  });
-
-  std::thread::id bg_flush_thr1, bg_flush_thr2;
-  SyncPoint::GetInstance()->SetCallBack(
-      "DBImpl::BackgroundCallFlush:start", [&](void*) {
-        if (bg_flush_thr1 == std::thread::id()) {
-          bg_flush_thr1 = std::this_thread::get_id();
-        } else if (bg_flush_thr2 == std::thread::id()) {
-          bg_flush_thr2 = std::this_thread::get_id();
-        }
-      });
-
-  int called = 0;
-  SyncPoint::GetInstance()->SetCallBack(
-      "DBImpl::AtomicFlushMemTablesToOutputFiles:WaitToCommit", [&](void* arg) {
-        if (std::this_thread::get_id() == bg_flush_thr2) {
-          const auto* ptr = reinterpret_cast<std::pair<Status, bool>*>(arg);
-          assert(ptr);
-          if (0 == called) {
-            // When bg flush thread 2 reaches here for the first time.
-            ASSERT_OK(ptr->first);
-            ASSERT_TRUE(ptr->second);
-          } else if (1 == called) {
-            // When bg flush thread 2 reaches here for the second time.
-            ASSERT_TRUE(ptr->first.IsIOError());
-            ASSERT_FALSE(ptr->second);
-          }
-          ++called;
-          TEST_SYNC_POINT("BgFlushThr2:WaitToCommit");
-        }
-      });
-
-  SyncPoint::GetInstance()->SetCallBack(
-      "VersionSet::ProcessManifestWrites:BeforeWriteLastVersionEdit:0",
-      [&](void*) {
-        if (std::this_thread::get_id() == bg_flush_thr1) {
-          TEST_SYNC_POINT("BgFlushThr1:BeforeWriteManifest");
-        }
-      });
-
-  SyncPoint::GetInstance()->SetCallBack(
-      "VersionSet::LogAndApply:WriteManifest", [&](void*) {
-        if (std::this_thread::get_id() != bg_flush_thr1) {
-          return;
-        }
-        ASSERT_OK(db_->Put(write_opts, "b", "v_1_b"));
-
-        FlushOptions flush_opts;
-        flush_opts.wait = false;
-        std::vector<ColumnFamilyHandle*> cfhs(1, db_->DefaultColumnFamily());
-        ASSERT_OK(dbfull()->Flush(flush_opts, cfhs));
-      });
-
-  SyncPoint::GetInstance()->SetCallBack(
-      "VersionSet::ProcessManifestWrites:AfterSyncManifest", [&](void* arg) {
-        auto* ptr = reinterpret_cast<IOStatus*>(arg);
-        assert(ptr);
-        *ptr = IOStatus::IOError("Injected failure");
-      });
-  SyncPoint::GetInstance()->EnableProcessing();
-
-  ASSERT_TRUE(dbfull()->Flush(FlushOptions(), handles_).IsIOError());
-
-  Close();
-  SyncPoint::GetInstance()->DisableProcessing();
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-}
-
 INSTANTIATE_TEST_CASE_P(DBFlushDirectIOTest, DBFlushDirectIOTest,
                        testing::Bool());

@@ -8,7 +8,6 @@

 #include "db/db_impl/db_impl.h"
 #include "db/version_set.h"
-#include "logging/logging.h"
 #include "table/get_context.h"
 #include "util/cast_util.h"

@@ -53,7 +53,7 @@
 #include "db/version_set.h"
 #include "db/write_batch_internal.h"
 #include "db/write_callback.h"
-#include "env/unique_id_gen.h"
+#include "env/unique_id.h"
 #include "file/file_util.h"
 #include "file/filename.h"
 #include "file/random_access_file_reader.h"
@@ -92,7 +92,6 @@
 #include "table/sst_file_dumper.h"
 #include "table/table_builder.h"
 #include "table/two_level_iterator.h"
-#include "table/unique_id_impl.h"
 #include "test_util/sync_point.h"
 #include "trace_replay/trace_replay.h"
 #include "util/autovector.h"
@@ -719,11 +718,9 @@ Status DBImpl::CloseHelper() {
 Status DBImpl::CloseImpl() { return CloseHelper(); }

 DBImpl::~DBImpl() {
-  InstrumentedMutexLock closing_lock_guard(&closing_mutex_);
  if (!closed_) {
    closed_ = true;
-    closing_status_ = CloseHelper();
-    closing_status_.PermitUncheckedError();
+    CloseHelper().PermitUncheckedError();
  }
 }

@@ -3948,18 +3945,23 @@ Status DBImpl::GetDbSessionId(std::string& session_id) const {
 }

 std::string DBImpl::GenerateDbSessionId(Env*) {
-  // See SemiStructuredUniqueIdGen for its desirable properties.
-  static SemiStructuredUniqueIdGen gen;
+  // GenerateRawUniqueId() generates an identifier that has a negligible
+  // probability of being duplicated. It should have full 128 bits of entropy.
+  uint64_t a, b;
+  GenerateRawUniqueId(&a, &b);

-  uint64_t lo, hi;
-  gen.GenerateNext(&hi, &lo);
-  if (lo == 0) {
-    // Avoid emitting session ID with lo==0, so that SST unique
-    // IDs can be more easily ensured non-zero
-    gen.GenerateNext(&hi, &lo);
-    assert(lo != 0);
-  }
-  return EncodeSessionId(hi, lo);
+  // Hash and reformat that down to a more compact format, 20 characters
+  // in base-36 ([0-9A-Z]), which is ~103 bits of entropy, which is enough
+  // to expect no collisions across a billion servers each opening DBs
+  // a million times (~2^50). Benefits vs. raw unique id:
+  // * Save ~ dozen bytes per SST file
+  // * Shorter shared backup file names (some platforms have low limits)
+  // * Visually distinct from DB id format
+  std::string db_session_id(20U, '\0');
+  char* buf = &db_session_id[0];
+  PutBaseChars<36>(&buf, 10, a, /*uppercase*/ true);
+  PutBaseChars<36>(&buf, 10, b, /*uppercase*/ true);
+  return db_session_id;
 }

 void DBImpl::SetDbSessionId() {
@@ -4004,20 +4006,19 @@ Status DB::DestroyColumnFamilyHandle(ColumnFamilyHandle* column_family) {
 DB::~DB() {}

 Status DBImpl::Close() {
-  InstrumentedMutexLock closing_lock_guard(&closing_mutex_);
-  if (closed_) {
-    return closing_status_;
-  }
-  {
-    InstrumentedMutexLock l(&mutex_);
-    // If there is unreleased snapshot, fail the close call
-    if (!snapshots_.empty()) {
-      return Status::Aborted("Cannot close DB with unreleased snapshot.");
+  if (!closed_) {
+    {
+      InstrumentedMutexLock l(&mutex_);
+      // If there is unreleased snapshot, fail the close call
+      if (!snapshots_.empty()) {
+        return Status::Aborted("Cannot close DB with unreleased snapshot.");
+      }
    }
+
+    closed_ = true;
+    return CloseImpl();
  }
-  closing_status_ = CloseImpl();
-  closed_ = true;
-  return closing_status_;
+  return Status::OK();
 }

 Status DB::ListColumnFamilies(const DBOptions& db_options,
@@ -4058,10 +4059,8 @@ Status DestroyDB(const std::string& dbname, const Options& options,
          del = DestroyDB(path_to_delete, options);
        } else if (type == kTableFile || type == kWalFile ||
                   type == kBlobFile) {
-          del = DeleteDBFile(
-              &soptions, path_to_delete, dbname,
-              /*force_bg=*/false,
-              /*force_fg=*/(type == kWalFile) ? !wal_in_db_path : false);
+          del = DeleteDBFile(&soptions, path_to_delete, dbname,
+                             /*force_bg=*/false, /*force_fg=*/!wal_in_db_path);
        } else {
          del = env->DeleteFile(path_to_delete);
        }
@@ -4283,16 +4282,11 @@ Status DBImpl::RenameTempFileToOptionsFile(const std::string& file_name) {
  uint64_t options_file_number = versions_->NewFileNumber();
  std::string options_file_name =
      OptionsFileName(GetName(), options_file_number);
-  uint64_t options_file_size = 0;
-  s = GetEnv()->GetFileSize(file_name, &options_file_size);
-  if (s.ok()) {
-    // Retry if the file name happen to conflict with an existing one.
-    s = GetEnv()->RenameFile(file_name, options_file_name);
-  }
+  // Retry if the file name happen to conflict with an existing one.
+  s = GetEnv()->RenameFile(file_name, options_file_name);
  if (s.ok()) {
    InstrumentedMutexLock l(&mutex_);
    versions_->options_file_number_ = options_file_number;
-    versions_->options_file_size_ = options_file_size;
  }

  if (0 == disable_delete_obsolete_files_) {
@@ -4562,8 +4556,7 @@ Status DBImpl::IngestExternalFiles(
    SuperVersion* super_version = cfd->GetReferencedSuperVersion(this);
    Status es = ingestion_jobs[i].Prepare(
        args[i].external_files, args[i].files_checksums,
-        args[i].files_checksum_func_names, args[i].file_temperature,
-        start_file_number, super_version);
+        args[i].files_checksum_func_names, start_file_number, super_version);
    // capture first error only
    if (!es.ok() && status.ok()) {
      status = es;
@@ -4578,8 +4571,7 @@ Status DBImpl::IngestExternalFiles(
    SuperVersion* super_version = cfd->GetReferencedSuperVersion(this);
    Status es = ingestion_jobs[0].Prepare(
        args[0].external_files, args[0].files_checksums,
-        args[0].files_checksum_func_names, args[0].file_temperature,
-        next_file_number, super_version);
+        args[0].files_checksum_func_names, next_file_number, super_version);
    if (!es.ok()) {
      status = es;
    }
@@ -22,6 +22,7 @@
 #include "db/column_family.h"
 #include "db/compaction/compaction_iterator.h"
 #include "db/compaction/compaction_job.h"
+#include "db/dbformat.h"
 #include "db/error_handler.h"
 #include "db/event_helpers.h"
 #include "db/external_sst_file_ingestion_job.h"
@@ -405,10 +406,6 @@ class DBImpl : public DB {
  virtual Status GetLiveFilesChecksumInfo(
      FileChecksumList* checksum_list) override;

-  virtual Status GetLiveFilesStorageInfo(
-      const LiveFilesStorageInfoOptions& opts,
-      std::vector<LiveFileStorageInfo>* files) override;
-
  // Obtains the meta data of the specified column family of the DB.
  // TODO(yhchiang): output parameter is placed in the end in this codebase.
  virtual void GetColumnFamilyMetaData(ColumnFamilyHandle* column_family,
@@ -1235,8 +1232,6 @@ class DBImpl : public DB {
 #ifndef ROCKSDB_LITE
  void NotifyOnExternalFileIngested(
      ColumnFamilyData* cfd, const ExternalSstFileIngestionJob& ingestion_job);
-
-  Status FlushForGetLiveFiles();
 #endif  // !ROCKSDB_LITE

  void NewThreadStatusCfInfo(ColumnFamilyData* cfd) const;
@@ -2290,10 +2285,6 @@ class DBImpl : public DB {

  // Flag to check whether Close() has been called on this DB
  bool closed_;
-  // save the closing status, for re-calling the close()
-  Status closing_status_;
-  // mutex for DB::Close()
-  InstrumentedMutex closing_mutex_;

  // Conditional variable to coordinate installation of atomic flush results.
  // With atomic flush, each bg thread installs the result of flushing multiple
@@ -14,7 +14,6 @@
 #include "db/error_handler.h"
 #include "db/event_helpers.h"
 #include "file/sst_file_manager_impl.h"
-#include "logging/logging.h"
 #include "monitoring/iostats_context_imp.h"
 #include "monitoring/perf_context_imp.h"
 #include "monitoring/thread_status_updater.h"
@@ -559,15 +558,7 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
  }

  if (s.ok()) {
-    const auto wait_to_install_func =
-        [&]() -> std::pair<Status, bool /*continue to wait*/> {
-      if (!versions_->io_status().ok()) {
-        // Something went wrong elsewhere, we cannot count on waiting for our
-        // turn to write/sync to MANIFEST or CURRENT. Just return.
-        return std::make_pair(versions_->io_status(), false);
-      } else if (shutting_down_.load(std::memory_order_acquire)) {
-        return std::make_pair(Status::ShutdownInProgress(), false);
-      }
+    auto wait_to_install_func = [&]() {
      bool ready = true;
      for (size_t i = 0; i != cfds.size(); ++i) {
        const auto& mems = jobs[i]->GetMemTables();
@@ -591,40 +582,18 @@ Status DBImpl::AtomicFlushMemTablesToOutputFiles(
          break;
        }
      }
-      return std::make_pair(Status::OK(), !ready);
+      return ready;
    };

    bool resuming_from_bg_err = error_handler_.IsDBStopped();
-    while ((!resuming_from_bg_err || error_handler_.GetRecoveryError().ok())) {
-      std::pair<Status, bool> res = wait_to_install_func();
-
-      TEST_SYNC_POINT_CALLBACK(
-          "DBImpl::AtomicFlushMemTablesToOutputFiles:WaitToCommit", &res);
-
-      if (!res.first.ok()) {
-        s = res.first;
-        break;
-      } else if (!res.second) {
-        break;
-      }
+    while ((!error_handler_.IsDBStopped() ||
+            error_handler_.GetRecoveryError().ok()) &&
+           !wait_to_install_func()) {
      atomic_flush_install_cv_.Wait();
-
-      resuming_from_bg_err = error_handler_.IsDBStopped();
    }

-    if (!resuming_from_bg_err) {
-      // If not resuming from bg err, then we determine future action based on
-      // whether we hit background error.
-      if (s.ok()) {
-        s = error_handler_.GetBGError();
-      }
-    } else if (s.ok()) {
-      // If resuming from bg err, we still rely on wait_to_install_func()'s
-      // result to determine future action. If wait_to_install_func() returns
-      // non-ok already, then we should not proceed to flush result
-      // installation.
-      s = error_handler_.GetRecoveryError();
-    }
+    s = resuming_from_bg_err ? error_handler_.GetRecoveryError()
+                             : error_handler_.GetBGError();
  }

  if (s.ok()) {
@@ -2683,7 +2652,7 @@ void DBImpl::BackgroundCallFlush(Env::Priority thread_pri) {
  bool made_progress = false;
  JobContext job_context(next_job_id_.fetch_add(1), true);

-  TEST_SYNC_POINT_CALLBACK("DBImpl::BackgroundCallFlush:start", nullptr);
+  TEST_SYNC_POINT("DBImpl::BackgroundCallFlush:start");

  LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL,
                       immutable_db_options_.info_log.get());
@@ -14,7 +14,6 @@
 #include "db/db_impl/db_impl.h"
 #include "db/job_context.h"
 #include "db/version_set.h"
-#include "logging/logging.h"
 #include "rocksdb/status.h"
 #include "util/cast_util.h"

@@ -16,7 +16,6 @@
 #include "file/file_util.h"
 #include "file/filename.h"
 #include "file/sst_file_manager_impl.h"
-#include "logging/logging.h"
 #include "port/port.h"
 #include "util/autovector.h"

@@ -330,11 +329,9 @@ void DBImpl::DeleteObsoleteFileImpl(int job_id, const std::string& fname,

  Status file_deletion_status;
  if (type == kTableFile || type == kBlobFile || type == kWalFile) {
-    // Rate limit WAL deletion only if its in the DB dir
-    file_deletion_status = DeleteDBFile(
-        &immutable_db_options_, fname, path_to_sync,
-        /*force_bg=*/false,
-        /*force_fg=*/(type == kWalFile) ? !wal_in_db_path_ : false);
+    file_deletion_status =
+        DeleteDBFile(&immutable_db_options_, fname, path_to_sync,
+                     /*force_bg=*/false, /*force_fg=*/!wal_in_db_path_);
  } else {
    file_deletion_status = env_->DeleteFile(fname);
  }
@@ -13,11 +13,9 @@
 #include "db/error_handler.h"
 #include "db/periodic_work_scheduler.h"
 #include "env/composite_env_wrapper.h"
-#include "file/filename.h"
 #include "file/read_write_util.h"
 #include "file/sst_file_manager_impl.h"
 #include "file/writable_file_writer.h"
-#include "logging/logging.h"
 #include "monitoring/persistent_stats_history.h"
 #include "options/options_helper.h"
 #include "rocksdb/table.h"
@@ -678,12 +676,6 @@ Status DBImpl::Recover(
        }
      }
      versions_->options_file_number_ = options_file_number;
-      uint64_t options_file_size = 0;
-      if (options_file_number > 0) {
-        s = env_->GetFileSize(OptionsFileName(GetName(), options_file_number),
-                              &options_file_size);
-      }
-      versions_->options_file_size_ = options_file_size;
    }
  }
  return s;
@@ -10,7 +10,6 @@
 #include "db/db_impl/db_impl.h"
 #include "db/db_iter.h"
 #include "db/merge_context.h"
-#include "logging/logging.h"
 #include "monitoring/perf_context_imp.h"
 #include "util/cast_util.h"

@@ -10,7 +10,6 @@
 #include "db/arena_wrapped_db_iter.h"
 #include "db/merge_context.h"
 #include "logging/auto_roll_logger.h"
-#include "logging/logging.h"
 #include "monitoring/perf_context_imp.h"
 #include "rocksdb/configurable.h"
 #include "util/cast_util.h"
@@ -9,9 +9,7 @@

 #include <string>
 #include <vector>
-
 #include "db/db_impl/db_impl.h"
-#include "logging/logging.h"

 namespace ROCKSDB_NAMESPACE {

@@ -11,7 +11,6 @@
 #include "db/db_impl/db_impl.h"
 #include "db/error_handler.h"
 #include "db/event_helpers.h"
-#include "logging/logging.h"
 #include "monitoring/perf_context_imp.h"
 #include "options/options_helper.h"
 #include "test_util/sync_point.h"
@@ -161,7 +160,8 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
    RecordTick(stats_, WRITE_WITH_WAL);
  }

-  StopWatch write_sw(immutable_db_options_.clock, stats_, DB_WRITE);
+  StopWatch write_sw(immutable_db_options_.clock, immutable_db_options_.stats,
+                     DB_WRITE);

  write_thread_.JoinBatchGroup(&w);
  if (w.state == WriteThread::STATE_PARALLEL_MEMTABLE_WRITER) {
@@ -471,7 +471,8 @@ Status DBImpl::PipelinedWriteImpl(const WriteOptions& write_options,
                                  uint64_t* log_used, uint64_t log_ref,
                                  bool disable_memtable, uint64_t* seq_used) {
  PERF_TIMER_GUARD(write_pre_and_post_process_time);
-  StopWatch write_sw(immutable_db_options_.clock, stats_, DB_WRITE);
+  StopWatch write_sw(immutable_db_options_.clock, immutable_db_options_.stats,
+                     DB_WRITE);

  WriteContext write_context;

@@ -627,7 +628,8 @@ Status DBImpl::UnorderedWriteMemtable(const WriteOptions& write_options,
                                      SequenceNumber seq,
                                      const size_t sub_batch_cnt) {
  PERF_TIMER_GUARD(write_pre_and_post_process_time);
-  StopWatch write_sw(immutable_db_options_.clock, stats_, DB_WRITE);
+  StopWatch write_sw(immutable_db_options_.clock, immutable_db_options_.stats,
+                     DB_WRITE);

  WriteThread::Writer w(write_options, my_batch, callback, log_ref,
                        false /*disable_memtable*/);
@@ -682,7 +684,8 @@ Status DBImpl::WriteImplWALOnly(
  WriteThread::Writer w(write_options, my_batch, callback, log_ref,
                        disable_memtable, sub_batch_cnt, pre_release_callback);
  RecordTick(stats_, WRITE_WITH_WAL);
-  StopWatch write_sw(immutable_db_options_.clock, stats_, DB_WRITE);
+  StopWatch write_sw(immutable_db_options_.clock, immutable_db_options_.stats,
+                     DB_WRITE);

  write_thread->JoinBatchGroup(&w);
  assert(w.state != WriteThread::STATE_PARALLEL_MEMTABLE_WRITER);
@@ -1756,6 +1759,8 @@ void DBImpl::NotifyOnMemTableSealed(ColumnFamilyData* /*cfd*/,
 // two_write_queues_ is true (This is to simplify the reasoning.)
 Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context) {
  mutex_.AssertHeld();
+  WriteThread::Writer nonmem_w;
+  std::unique_ptr<WritableFile> lfile;
  log::Writer* new_log = nullptr;
  MemTable* new_mem = nullptr;
  IOStatus io_s;
@@ -1859,9 +1864,17 @@ Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context) {
  if (!s.ok()) {
    // how do we fail if we're not creating new log?
    assert(creating_new_log);
-    delete new_mem;
-    delete new_log;
-    context->superversion_context.new_superversion.reset();
+    if (new_mem) {
+      delete new_mem;
+    }
+    if (new_log) {
+      delete new_log;
+    }
+    SuperVersion* new_superversion =
+        context->superversion_context.new_superversion.release();
+    if (new_superversion != nullptr) {
+      delete new_superversion;
+    }
    // We may have lost data from the WritableFileBuffer in-memory buffer for
    // the current log, so treat it as a fatal error and set bg_error
    if (!io_s.ok()) {
@@ -2052,36 +2065,12 @@ Status DB::Delete(const WriteOptions& opt, ColumnFamilyHandle* column_family,

 Status DB::SingleDelete(const WriteOptions& opt,
                        ColumnFamilyHandle* column_family, const Slice& key) {
-  Status s;
-  if (opt.timestamp == nullptr) {
-    WriteBatch batch;
-    s = batch.SingleDelete(column_family, key);
-    if (!s.ok()) {
-      return s;
-    }
-    s = Write(opt, &batch);
-    return s;
-  }
-
-  const Slice* ts = opt.timestamp;
-  assert(ts != nullptr);
-  size_t ts_sz = ts->size();
-  assert(column_family->GetComparator());
-  assert(ts_sz == column_family->GetComparator()->timestamp_size());
  WriteBatch batch;
-  if (key.data() + key.size() == ts->data()) {
-    Slice key_with_ts = Slice(key.data(), key.size() + ts_sz);
-    s = batch.SingleDelete(column_family, key_with_ts);
-  } else {
-    std::array<Slice, 2> key_with_ts_slices{{key, *ts}};
-    SliceParts key_with_ts(key_with_ts_slices.data(), 2);
-    s = batch.SingleDelete(column_family, key_with_ts);
-  }
+  Status s = batch.SingleDelete(column_family, key);
  if (!s.ok()) {
    return s;
  }
-  s = Write(opt, &batch);
-  return s;
+  return Write(opt, &batch);
 }

 Status DB::DeleteRange(const WriteOptions& opt,
@@ -12,6 +12,7 @@
 #include <string>

 #include "db/db_impl/db_impl.h"
+#include "db/dbformat.h"
 #include "db/range_del_aggregator.h"
 #include "memory/arena.h"
 #include "options/cf_options.h"
@@ -19,7 +19,6 @@
 #include "rocksdb/perf_context.h"
 #include "rocksdb/perf_level.h"
 #include "rocksdb/table.h"
-#include "test_util/mock_time_env.h"
 #include "util/random.h"
 #include "util/string_util.h"

@@ -29,25 +28,6 @@ class DBPropertiesTest : public DBTestBase {
 public:
  DBPropertiesTest()
      : DBTestBase("db_properties_test", /*env_do_fsync=*/false) {}
-
-  void AssertDbStats(const std::map<std::string, std::string>& db_stats,
-                     double expected_uptime, int expected_user_bytes_written,
-                     int expected_wal_bytes_written,
-                     int expected_user_writes_by_self,
-                     int expected_user_writes_with_wal) {
-    ASSERT_EQ(std::to_string(expected_uptime), db_stats.at("db.uptime"));
-    ASSERT_EQ(std::to_string(expected_wal_bytes_written),
-              db_stats.at("db.wal_bytes_written"));
-    ASSERT_EQ("0", db_stats.at("db.wal_syncs"));
-    ASSERT_EQ(std::to_string(expected_user_bytes_written),
-              db_stats.at("db.user_bytes_written"));
-    ASSERT_EQ("0", db_stats.at("db.user_writes_by_other"));
-    ASSERT_EQ(std::to_string(expected_user_writes_by_self),
-              db_stats.at("db.user_writes_by_self"));
-    ASSERT_EQ(std::to_string(expected_user_writes_with_wal),
-              db_stats.at("db.user_writes_with_wal"));
-    ASSERT_EQ("0", db_stats.at("db.user_write_stall_micros"));
-  }
 };

 #ifndef ROCKSDB_LITE
@@ -1915,80 +1895,7 @@ TEST_F(DBPropertiesTest, BlockCacheProperties) {
  ASSERT_EQ(0, value);
 }

-TEST_F(DBPropertiesTest, GetMapPropertyDbStats) {
-  auto mock_clock = std::make_shared<MockSystemClock>(env_->GetSystemClock());
-  CompositeEnvWrapper env(env_, mock_clock);
-
-  Options opts = CurrentOptions();
-  opts.env = &env;
-  Reopen(opts);
-
-  {
-    std::map<std::string, std::string> db_stats;
-    ASSERT_TRUE(db_->GetMapProperty(DB::Properties::kDBStats, &db_stats));
-    AssertDbStats(db_stats, 0.0 /* expected_uptime */,
-                  0 /* expected_user_bytes_written */,
-                  0 /* expected_wal_bytes_written */,
-                  0 /* expected_user_writes_by_self */,
-                  0 /* expected_user_writes_with_wal */);
-  }
-
-  {
-    mock_clock->SleepForMicroseconds(1500000);
-
-    std::map<std::string, std::string> db_stats;
-    ASSERT_TRUE(db_->GetMapProperty(DB::Properties::kDBStats, &db_stats));
-    AssertDbStats(db_stats, 1.5 /* expected_uptime */,
-                  0 /* expected_user_bytes_written */,
-                  0 /* expected_wal_bytes_written */,
-                  0 /* expected_user_writes_by_self */,
-                  0 /* expected_user_writes_with_wal */);
-  }
-
-  int expected_user_bytes_written = 0;
-  {
-    // Write with WAL disabled.
-    WriteOptions write_opts;
-    write_opts.disableWAL = true;
-
-    WriteBatch batch;
-    ASSERT_OK(batch.Put("key", "val"));
-    expected_user_bytes_written += static_cast<int>(batch.GetDataSize());
-
-    ASSERT_OK(db_->Write(write_opts, &batch));
-
-    std::map<std::string, std::string> db_stats;
-    ASSERT_TRUE(db_->GetMapProperty(DB::Properties::kDBStats, &db_stats));
-    AssertDbStats(db_stats, 1.5 /* expected_uptime */,
-                  expected_user_bytes_written,
-                  0 /* expected_wal_bytes_written */,
-                  1 /* expected_user_writes_by_self */,
-                  0 /* expected_user_writes_with_wal */);
-  }
-
-  int expected_wal_bytes_written = 0;
-  {
-    // Write with WAL enabled.
-    WriteBatch batch;
-    ASSERT_OK(batch.Delete("key"));
-    expected_user_bytes_written += static_cast<int>(batch.GetDataSize());
-    expected_wal_bytes_written += static_cast<int>(batch.GetDataSize());
-
-    ASSERT_OK(db_->Write(WriteOptions(), &batch));
-
-    std::map<std::string, std::string> db_stats;
-    ASSERT_TRUE(db_->GetMapProperty(DB::Properties::kDBStats, &db_stats));
-    AssertDbStats(db_stats, 1.5 /* expected_uptime */,
-                  expected_user_bytes_written, expected_wal_bytes_written,
-                  2 /* expected_user_writes_by_self */,
-                  1 /* expected_user_writes_with_wal */);
-  }
-
-  Close();
-}
-
 #endif  // ROCKSDB_LITE
-
 }  // namespace ROCKSDB_NAMESPACE

 int main(int argc, char** argv) {
@@ -169,36 +169,17 @@ TEST_F(DBRangeDelTest, MaxCompactionBytesCutsOutputFiles) {
  opts.memtable_factory.reset(test::NewSpecialSkipListFactory(kNumPerFile));
  // Want max_compaction_bytes to trigger the end of compaction output file, not
  // target_file_size_base, so make the latter much bigger
-  //  opts.target_file_size_base = 100 * opts.max_compaction_bytes;
-  opts.target_file_size_base = 1;
+  opts.target_file_size_base = 100 * opts.max_compaction_bytes;
  DestroyAndReopen(opts);

  // snapshot protects range tombstone from dropping due to becoming obsolete.
  const Snapshot* snapshot = db_->GetSnapshot();

-  Random rnd(301);
-
-  ASSERT_OK(Put(GetNumericStr(0), rnd.RandomString(kBytesPerVal)));
-  ASSERT_OK(
-      Put(GetNumericStr(kNumPerFile - 1), rnd.RandomString(kBytesPerVal)));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put(GetNumericStr(kNumPerFile), rnd.RandomString(kBytesPerVal)));
-  ASSERT_OK(
-      Put(GetNumericStr(kNumPerFile * 2 - 1), rnd.RandomString(kBytesPerVal)));
-  ASSERT_OK(Flush());
-  MoveFilesToLevel(2);
-  ASSERT_EQ(0, NumTableFilesAtLevel(0));
-  ASSERT_EQ(NumTableFilesAtLevel(2), 2);
-
-  ASSERT_OK(db_->SetOptions(
-      db_->DefaultColumnFamily(),
-      {{"target_file_size_base", ToString(100 * opts.max_compaction_bytes)}}));
-
  // It spans the whole key-range, thus will be included in all output files
  ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(),
                             GetNumericStr(0),
                             GetNumericStr(kNumFiles * kNumPerFile - 1)));
-
+  Random rnd(301);
  for (int i = 0; i < kNumFiles; ++i) {
    std::vector<std::string> values;
    // Write 1MB (256 values, each 4K)
@@ -212,11 +193,11 @@ TEST_F(DBRangeDelTest, MaxCompactionBytesCutsOutputFiles) {
    ASSERT_EQ(i + 1, NumTableFilesAtLevel(0));
  }

-  ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr,
-                                        /*column_family=*/nullptr,
-                                        /*disallow_trivial_move=*/true));
+  ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr,
+                                        true /* disallow_trivial_move */));
  ASSERT_EQ(0, NumTableFilesAtLevel(0));
  ASSERT_GE(NumTableFilesAtLevel(1), 2);
+
  std::vector<std::vector<FileMetaData>> files;
  dbfull()->TEST_GetFilesMetaData(db_->DefaultColumnFamily(), &files);

@@ -1647,7 +1628,6 @@ TEST_F(DBRangeDelTest, OverlappedTombstones) {
  const int kNumPerFile = 4, kNumFiles = 2;
  Options options = CurrentOptions();
  options.disable_auto_compactions = true;
-  options.target_file_size_base = 9 * 1024;
  options.max_compaction_bytes = 9 * 1024;
  DestroyAndReopen(options);
  Random rnd(301);
@@ -1687,7 +1667,6 @@ TEST_F(DBRangeDelTest, OverlappedKeys) {
  const int kNumPerFile = 4, kNumFiles = 2;
  Options options = CurrentOptions();
  options.disable_auto_compactions = true;
-  options.target_file_size_base = 9 * 1024;
  options.max_compaction_bytes = 9 * 1024;
  DestroyAndReopen(options);
  Random rnd(301);
@@ -602,14 +602,7 @@ TEST_F(DBSSTTest, DBWithSstFileManagerForBlobFilesWithGC) {
  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
 }

-class DBSSTTestRateLimit : public DBSSTTest,
-                           public ::testing::WithParamInterface<bool> {
- public:
-  DBSSTTestRateLimit() : DBSSTTest() {}
-  ~DBSSTTestRateLimit() override {}
-};
-
-TEST_P(DBSSTTestRateLimit, RateLimitedDelete) {
+TEST_F(DBSSTTest, RateLimitedDelete) {
  Destroy(last_options_);
  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->LoadDependency({
      {"DBSSTTest::RateLimitedDelete:1",
@@ -647,15 +640,11 @@ TEST_P(DBSSTTestRateLimit, RateLimitedDelete) {

  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();

-  bool different_wal_dir = GetParam();
  Options options = CurrentOptions();
  SetTimeElapseOnlySleepOnReopen(&options);
  options.disable_auto_compactions = true;
  options.env = env_;
  options.statistics = CreateDBStatistics();
-  if (different_wal_dir) {
-    options.wal_dir = alternative_wal_dir_;
-  }

  int64_t rate_bytes_per_sec = 1024 * 10;  // 10 Kbs / Sec
  Status s;
@@ -667,9 +656,7 @@ TEST_P(DBSSTTestRateLimit, RateLimitedDelete) {
  sfm->delete_scheduler()->SetMaxTrashDBRatio(1.1);

  WriteOptions wo;
-  if (!different_wal_dir) {
-    wo.disableWAL = true;
-  }
+  wo.disableWAL = true;
  Reopen(options);
  // Create 4 files in L0
  for (char v = 'a'; v <= 'd'; v++) {
@@ -714,9 +701,6 @@ TEST_P(DBSSTTestRateLimit, RateLimitedDelete) {
  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
 }

-INSTANTIATE_TEST_CASE_P(RateLimitedDelete, DBSSTTestRateLimit,
-                        ::testing::Bool());
-
 TEST_F(DBSSTTest, RateLimitedWALDelete) {
  Destroy(last_options_);

@@ -45,8 +45,6 @@ void VerifyTableProperties(DB* db, uint64_t expected_entries_size) {

  ASSERT_EQ(props.size(), unique_entries.size());
  ASSERT_EQ(expected_entries_size, sum);
-
-  VerifySstUniqueIds(props);
 }
 }  // namespace

@@ -96,37 +94,6 @@ TEST_F(DBTablePropertiesTest, GetPropertiesOfAllTablesTest) {
  VerifyTableProperties(db_, 10 + 11 + 12 + 13);
 }

-TEST_F(DBTablePropertiesTest, CreateOnDeletionCollectorFactory) {
-  ConfigOptions options;
-  options.ignore_unsupported_options = false;
-
-  std::shared_ptr<TablePropertiesCollectorFactory> factory;
-  std::string id = CompactOnDeletionCollectorFactory::kClassName();
-  ASSERT_OK(
-      TablePropertiesCollectorFactory::CreateFromString(options, id, &factory));
-  auto del_factory = factory->CheckedCast<CompactOnDeletionCollectorFactory>();
-  ASSERT_NE(del_factory, nullptr);
-  ASSERT_EQ(0U, del_factory->GetWindowSize());
-  ASSERT_EQ(0U, del_factory->GetDeletionTrigger());
-  ASSERT_EQ(0.0, del_factory->GetDeletionRatio());
-  ASSERT_OK(TablePropertiesCollectorFactory::CreateFromString(
-      options, "window_size=100; deletion_trigger=90; id=" + id, &factory));
-  del_factory = factory->CheckedCast<CompactOnDeletionCollectorFactory>();
-  ASSERT_NE(del_factory, nullptr);
-  ASSERT_EQ(100U, del_factory->GetWindowSize());
-  ASSERT_EQ(90U, del_factory->GetDeletionTrigger());
-  ASSERT_EQ(0.0, del_factory->GetDeletionRatio());
-  ASSERT_OK(TablePropertiesCollectorFactory::CreateFromString(
-      options,
-      "window_size=100; deletion_trigger=90; deletion_ratio=0.5; id=" + id,
-      &factory));
-  del_factory = factory->CheckedCast<CompactOnDeletionCollectorFactory>();
-  ASSERT_NE(del_factory, nullptr);
-  ASSERT_EQ(100U, del_factory->GetWindowSize());
-  ASSERT_EQ(90U, del_factory->GetDeletionTrigger());
-  ASSERT_EQ(0.5, del_factory->GetDeletionRatio());
-}
-
 TablePropertiesCollection
 DBTablePropertiesTest::TestGetPropertiesOfTablesInRange(
    std::vector<Range> ranges, std::size_t* num_properties,
@@ -11,13 +11,11 @@
 // in Release build.
 // which is a pity, it is a good test
 #include <fcntl.h>
-
 #include <algorithm>
 #include <set>
 #include <thread>
 #include <unordered_set>
 #include <utility>
-
 #ifndef OS_WIN
 #include <unistd.h>
 #endif
@@ -54,7 +52,6 @@
 #include "rocksdb/table.h"
 #include "rocksdb/table_properties.h"
 #include "rocksdb/thread_status.h"
-#include "rocksdb/types.h"
 #include "rocksdb/utilities/checkpoint.h"
 #include "rocksdb/utilities/optimistic_transaction_db.h"
 #include "rocksdb/utilities/write_batch_with_index.h"
@@ -98,7 +95,7 @@ class DBTestWithParam
 };

 TEST_F(DBTest, MockEnvTest) {
-  std::unique_ptr<MockEnv> env{MockEnv::Create(Env::Default())};
+  std::unique_ptr<MockEnv> env{new MockEnv(Env::Default())};
  Options options;
  options.create_if_missing = true;
  options.env = env.get();
@@ -2077,13 +2074,6 @@ TEST_F(DBTest, OverlapInLevel0) {
    Flush(1);
    ASSERT_EQ("2,1,1", FilesPerLevel(1));

-    // BEGIN addition to existing test
-    // Take this opportunity to verify SST unique ids (including Plain table)
-    TablePropertiesCollection tbc;
-    ASSERT_OK(db_->GetPropertiesOfAllTables(handles_[1], &tbc));
-    VerifySstUniqueIds(tbc);
-    // END addition to existing test
-
    // Compact away the placeholder files we created initially
    dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]);
    dbfull()->TEST_CompactRange(2, nullptr, nullptr, handles_[1]);
@@ -2310,8 +2300,8 @@ TEST_F(DBTest, SnapshotFiles) {
    uint64_t manifest_number = 0;
    uint64_t manifest_size = 0;
    std::vector<std::string> files;
-    ASSERT_OK(dbfull()->DisableFileDeletions());
-    ASSERT_OK(dbfull()->GetLiveFiles(files, &manifest_size));
+    dbfull()->DisableFileDeletions();
+    dbfull()->GetLiveFiles(files, &manifest_size);

    // CURRENT, MANIFEST, OPTIONS, *.sst files (one for each CF)
    ASSERT_EQ(files.size(), 5U);
@@ -2340,17 +2330,18 @@ TEST_F(DBTest, SnapshotFiles) {
      // latest manifest file
      if (ParseFileName(files[i].substr(1), &number, &type)) {
        if (type == kDescriptorFile) {
-          ASSERT_EQ(manifest_number, 0);
-          manifest_number = number;
-          ASSERT_GE(size, manifest_size);
-          size = manifest_size;  // copy only valid MANIFEST data
+          if (number > manifest_number) {
+            manifest_number = number;
+            ASSERT_GE(size, manifest_size);
+            size = manifest_size;  // copy only valid MANIFEST data
+          }
        }
      }
      CopyFile(src, dest, size);
    }

    // release file snapshot
-    ASSERT_OK(dbfull()->EnableFileDeletions(/*force*/ false));
+    dbfull()->DisableFileDeletions();
    // overwrite one key, this key should not appear in the snapshot
    std::vector<std::string> extras;
    for (unsigned int i = 0; i < 1; i++) {
@@ -2387,8 +2378,8 @@ TEST_F(DBTest, SnapshotFiles) {
    uint64_t new_manifest_number = 0;
    uint64_t new_manifest_size = 0;
    std::vector<std::string> newfiles;
-    ASSERT_OK(dbfull()->DisableFileDeletions());
-    ASSERT_OK(dbfull()->GetLiveFiles(newfiles, &new_manifest_size));
+    dbfull()->DisableFileDeletions();
+    dbfull()->GetLiveFiles(newfiles, &new_manifest_size);

    // find the new manifest file. assert that this manifest file is
    // the same one as in the previous snapshot. But its size should be
@@ -2400,41 +2391,20 @@ TEST_F(DBTest, SnapshotFiles) {
      // latest manifest file
      if (ParseFileName(newfiles[i].substr(1), &number, &type)) {
        if (type == kDescriptorFile) {
-          ASSERT_EQ(new_manifest_number, 0);
-          uint64_t size;
-          new_manifest_number = number;
-          ASSERT_OK(env_->GetFileSize(src, &size));
-          ASSERT_GE(size, new_manifest_size);
+          if (number > new_manifest_number) {
+            uint64_t size;
+            new_manifest_number = number;
+            ASSERT_OK(env_->GetFileSize(src, &size));
+            ASSERT_GE(size, new_manifest_size);
+          }
        }
      }
    }
    ASSERT_EQ(manifest_number, new_manifest_number);
    ASSERT_GT(new_manifest_size, manifest_size);

-    // Also test GetLiveFilesStorageInfo
-    std::vector<LiveFileStorageInfo> new_infos;
-    ASSERT_OK(dbfull()->GetLiveFilesStorageInfo(LiveFilesStorageInfoOptions(),
-                                                &new_infos));
-
-    // Close DB (while deletions disabled)
-    Close();
-
-    // Validate
-    for (auto& info : new_infos) {
-      std::string path = info.directory + "/" + info.relative_filename;
-      uint64_t size;
-      ASSERT_OK(env_->GetFileSize(path, &size));
-      if (info.trim_to_size) {
-        ASSERT_LE(info.size, size);
-      } else if (!info.replacement_contents.empty()) {
-        ASSERT_EQ(info.size, info.replacement_contents.size());
-      } else {
-        ASSERT_EQ(info.size, size);
-      }
-      if (info.file_type == kDescriptorFile) {
-        ASSERT_EQ(info.file_number, manifest_number);
-      }
-    }
+    // release file snapshot
+    dbfull()->DisableFileDeletions();
  } while (ChangeCompactOptions());
 }

@@ -3149,12 +3119,6 @@ class ModelDB : public DB {
    return Status::OK();
  }

-  Status GetLiveFilesStorageInfo(
-      const LiveFilesStorageInfoOptions& /*opts*/,
-      std::vector<LiveFileStorageInfo>* /*files*/) override {
-    return Status::OK();
-  }
-
  Status GetSortedWalFiles(VectorLogPtr& /*files*/) override {
    return Status::OK();
  }
@@ -4931,7 +4895,7 @@ TEST_F(DBTest, DynamicLevelCompressionPerLevel2) {
  options.level0_stop_writes_trigger = 2;
  options.soft_pending_compaction_bytes_limit = 1024 * 1024;
  options.target_file_size_base = 20;
-  options.env = env_;
+
  options.level_compaction_dynamic_level_bytes = true;
  options.max_bytes_for_level_base = 200;
  options.max_bytes_for_level_multiplier = 8;
@@ -17,7 +17,6 @@
 #include "options/options_helper.h"
 #include "port/port.h"
 #include "port/stack_trace.h"
-#include "rocksdb/iostats_context.h"
 #include "rocksdb/persistent_cache.h"
 #include "rocksdb/trace_record.h"
 #include "rocksdb/trace_record_result.h"
@@ -37,10 +36,11 @@ class DBTest2 : public DBTestBase {
 #ifndef ROCKSDB_LITE
  uint64_t GetSstSizeHelper(Temperature temperature) {
    std::string prop;
-    EXPECT_TRUE(
+    bool s =
        dbfull()->GetProperty(DB::Properties::kLiveSstFilesSizeAtTemperature +
                                  ToString(static_cast<uint8_t>(temperature)),
-                              &prop));
+                              &prop);
+    assert(s);
    return static_cast<uint64_t>(std::atoi(prop.c_str()));
  }
 #endif  // ROCKSDB_LITE
@@ -3670,15 +3670,14 @@ TEST_F(DBTest2, IterRaceFlush1) {
    TEST_SYNC_POINT("DBTest2::IterRaceFlush:2");
  });

-  // iterator is created after the first Put(), and its snapshot sequence is
-  // assigned after second Put(), so it must see v2.
+  // iterator is created after the first Put(), so it should see either
+  // "v1" or "v2".
  {
    std::unique_ptr<Iterator> it(db_->NewIterator(ReadOptions()));
    it->Seek("foo");
    ASSERT_TRUE(it->Valid());
    ASSERT_OK(it->status());
    ASSERT_EQ("foo", it->key().ToString());
-    ASSERT_EQ("v2", it->value().ToString());
  }

  t1.join();
@@ -3701,15 +3700,14 @@ TEST_F(DBTest2, IterRaceFlush2) {
    TEST_SYNC_POINT("DBTest2::IterRaceFlush2:2");
  });

-  // iterator is created after the first Put(), and its snapshot sequence is
-  // assigned before second Put(), thus it must see v1.
+  // iterator is created after the first Put(), so it should see either
+  // "v1" or "v2".
  {
    std::unique_ptr<Iterator> it(db_->NewIterator(ReadOptions()));
    it->Seek("foo");
    ASSERT_TRUE(it->Valid());
    ASSERT_OK(it->status());
    ASSERT_EQ("foo", it->key().ToString());
-    ASSERT_EQ("v1", it->value().ToString());
  }

  t1.join();
@@ -3732,8 +3730,8 @@ TEST_F(DBTest2, IterRefreshRaceFlush) {
    TEST_SYNC_POINT("DBTest2::IterRefreshRaceFlush:2");
  });

-  // iterator is refreshed after the first Put(), and its sequence number is
-  // assigned after second Put(), thus it must see v2.
+  // iterator is created after the first Put(), so it should see either
+  // "v1" or "v2".
  {
    std::unique_ptr<Iterator> it(db_->NewIterator(ReadOptions()));
    ASSERT_OK(it->status());
@@ -3742,7 +3740,6 @@ TEST_F(DBTest2, IterRefreshRaceFlush) {
    ASSERT_TRUE(it->Valid());
    ASSERT_OK(it->status());
    ASSERT_EQ("foo", it->key().ToString());
-    ASSERT_EQ("v2", it->value().ToString());
  }

  t1.join();
@@ -5537,6 +5534,8 @@ TEST_F(DBTest2, TestCompactFiles) {
 }
 #endif  // ROCKSDB_LITE

+// TODO: figure out why this test fails in appveyor
+#ifndef OS_WIN
 TEST_F(DBTest2, MultiDBParallelOpenTest) {
  const int kNumDbs = 2;
  Options options = CurrentOptions();
@@ -5567,6 +5566,7 @@ TEST_F(DBTest2, MultiDBParallelOpenTest) {
  }

  // Verify non-empty DBs can be recovered in parallel
+  dbs.clear();
  open_threads.clear();
  for (int i = 0; i < kNumDbs; ++i) {
    open_threads.emplace_back(
@@ -5583,6 +5583,7 @@ TEST_F(DBTest2, MultiDBParallelOpenTest) {
    ASSERT_OK(DestroyDB(dbnames[i], options));
  }
 }
+#endif  // OS_WIN

 namespace {
 class DummyOldStats : public Statistics {
@@ -6518,9 +6519,6 @@ TEST_F(DBTest2, BottommostTemperature) {
  ASSERT_OK(Flush());
  ASSERT_OK(dbfull()->TEST_WaitForCompact());

-  get_iostats_context()->Reset();
-  IOStatsContext* iostats = get_iostats_context();
-
  ColumnFamilyMetaData metadata;
  db_->GetColumnFamilyMetaData(&metadata);
  ASSERT_EQ(1, metadata.file_count);
@@ -6529,31 +6527,11 @@ TEST_F(DBTest2, BottommostTemperature) {
  ASSERT_EQ(size, 0);
  size = GetSstSizeHelper(Temperature::kWarm);
  ASSERT_GT(size, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.warm_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-
-  ASSERT_EQ("bar", Get("foo"));
-
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.warm_file_read_count, 1);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_bytes_read, 0);
-  ASSERT_GT(iostats->file_io_stats_by_temperature.warm_file_bytes_read, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.cold_file_bytes_read, 0);

  // non-bottommost file still has unknown temperature
  ASSERT_OK(Put("foo", "bar"));
  ASSERT_OK(Put("bar", "bar"));
  ASSERT_OK(Flush());
-  ASSERT_EQ("bar", Get("bar"));
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.warm_file_read_count, 1);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_bytes_read, 0);
-  ASSERT_GT(iostats->file_io_stats_by_temperature.warm_file_bytes_read, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.cold_file_bytes_read, 0);
-
  db_->GetColumnFamilyMetaData(&metadata);
  ASSERT_EQ(2, metadata.file_count);
  ASSERT_EQ(Temperature::kUnknown, metadata.levels[0].files[0].temperature);
@@ -6602,8 +6580,6 @@ TEST_F(DBTest2, BottommostTemperatureUniversal) {
  ASSERT_EQ(size, 0);
  size = GetSstSizeHelper(Temperature::kHot);
  ASSERT_EQ(size, 0);
-  get_iostats_context()->Reset();
-  IOStatsContext* iostats = get_iostats_context();

  for (int i = 0; i < kTriggerNum; i++) {
    ASSERT_OK(Put("foo", "bar"));
@@ -6621,17 +6597,6 @@ TEST_F(DBTest2, BottommostTemperatureUniversal) {
  ASSERT_GT(size, 0);
  size = GetSstSizeHelper(Temperature::kWarm);
  ASSERT_EQ(size, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.warm_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ("bar", Get("foo"));
-
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.warm_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_read_count, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.hot_file_bytes_read, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.warm_file_bytes_read, 0);
-  ASSERT_EQ(iostats->file_io_stats_by_temperature.cold_file_bytes_read, 0);

  ASSERT_OK(Put("foo", "bar"));
  ASSERT_OK(Put("bar", "bar"));
@@ -13,7 +13,6 @@
 #include "env/mock_env.h"
 #include "rocksdb/convenience.h"
 #include "rocksdb/env_encryption.h"
-#include "rocksdb/unique_id.h"
 #include "rocksdb/utilities/object_registry.h"
 #include "util/random.h"

@@ -64,7 +63,7 @@ DBTestBase::DBTestBase(const std::string path, bool env_do_fsync)
  EXPECT_OK(test::CreateEnvFromSystem(config_options, &base_env, &env_guard_));
  EXPECT_NE(nullptr, base_env);
  if (getenv("MEM_ENV")) {
-    mem_env_ = MockEnv::Create(base_env, base_env->GetSystemClock());
+    mem_env_ = new MockEnv(base_env);
  }
 #ifndef ROCKSDB_LITE
  if (getenv("ENCRYPTED_ENV")) {
@@ -1655,14 +1654,4 @@ uint64_t DBTestBase::GetNumberOfSstFilesForColumnFamily(
 }
 #endif  // ROCKSDB_LITE

-void VerifySstUniqueIds(const TablePropertiesCollection& props) {
-  ASSERT_FALSE(props.empty());  // suspicious test if empty
-  std::unordered_set<std::string> seen;
-  for (auto& pair : props) {
-    std::string id;
-    ASSERT_OK(GetUniqueIdFromTableProperties(*pair.second, &id));
-    ASSERT_TRUE(seen.insert(id).second);
-  }
-}
-
 }  // namespace ROCKSDB_NAMESPACE
@@ -22,6 +22,7 @@
 #include <vector>

 #include "db/db_impl/db_impl.h"
+#include "db/dbformat.h"
 #include "file/filename.h"
 #include "rocksdb/cache.h"
 #include "rocksdb/compaction_filter.h"
@@ -1195,8 +1196,4 @@ class DBTestBase : public testing::Test {
  bool time_elapse_only_sleep_on_reopen_ = false;
 };

-// For verifying that all files generated by current version have SST
-// unique ids.
-void VerifySstUniqueIds(const TablePropertiesCollection& props);
-
 }  // namespace ROCKSDB_NAMESPACE
@@ -737,7 +737,6 @@ TEST_P(DBTestUniversalCompactionParallel, UniversalCompactionParallel) {
  Options options = CurrentOptions();
  options.compaction_style = kCompactionStyleUniversal;
  options.num_levels = num_levels_;
-  options.env = env_;
  options.write_buffer_size = 1 << 10;  // 1KB
  options.level0_file_num_compaction_trigger = 3;
  options.max_background_compactions = 3;
@@ -983,27 +983,13 @@ TEST_F(DBBasicTestWithTimestamp, SimpleForwardIterateLowerTsBound) {
  Close();
 }

-class DBBasicDeletionTestWithTimestamp
-    : public DBBasicTestWithTimestampBase,
-      public testing::WithParamInterface<enum ValueType> {
- public:
-  DBBasicDeletionTestWithTimestamp()
-      : DBBasicTestWithTimestampBase("db_basic_deletion_test_with_timestamp") {}
-};
-
-INSTANTIATE_TEST_CASE_P(
-    Timestamp, DBBasicDeletionTestWithTimestamp,
-    ::testing::Values(ValueType::kTypeSingleDeletion,
-                      ValueType::kTypeDeletionWithTimestamp));
-
-TEST_P(DBBasicDeletionTestWithTimestamp, ForwardIterateStartSeqnum) {
+TEST_F(DBBasicTestWithTimestamp, ForwardIterateStartSeqnum) {
  const int kNumKeysPerFile = 128;
  const uint64_t kMaxKey = 0xffffffffffffffff;
  const uint64_t kMinKey = kMaxKey - 1023;
  Options options = CurrentOptions();
  options.env = env_;
  options.create_if_missing = true;
-  ValueType op_type = GetParam();
  // Need to disable compaction to bottommost level when sequence number will be
  // zeroed out, causing the verification of sequence number to fail in this
  // test.
@@ -1030,11 +1016,7 @@ TEST_P(DBBasicDeletionTestWithTimestamp, ForwardIterateStartSeqnum) {
      if (k % 2) {
        s = db_->Put(write_opts, Key1(k), "value" + std::to_string(i));
      } else {
-        if (op_type == ValueType::kTypeDeletionWithTimestamp) {
-          s = db_->Delete(write_opts, Key1(k));
-        } else if (op_type == ValueType::kTypeSingleDeletion) {
-          s = db_->SingleDelete(write_opts, Key1(k));
-        }
+        s = db_->Delete(write_opts, Key1(k));
      }
      ASSERT_OK(s);
    }
@@ -1056,7 +1038,8 @@ TEST_P(DBBasicDeletionTestWithTimestamp, ForwardIterateStartSeqnum) {
    uint64_t key = kMinKey;
    for (iter->Seek(Key1(kMinKey)); iter->Valid(); iter->Next()) {
      CheckIterEntry(
-          iter.get(), Key1(key), expected_seq, (key % 2) ? kTypeValue : op_type,
+          iter.get(), Key1(key), expected_seq,
+          (key % 2) ? kTypeValue : kTypeDeletionWithTimestamp,
          (key % 2) ? "value" + std::to_string(i + 1) : std::string(),
          write_ts_list[i + 1]);
      ++key;
@@ -1079,7 +1062,7 @@ TEST_P(DBBasicDeletionTestWithTimestamp, ForwardIterateStartSeqnum) {
    SequenceNumber expected_seq = start_seqs[i] + (kMaxKey - kMinKey) + 1;
    for (it->Seek(Key1(kMinKey)); it->Valid(); it->Next()) {
      CheckIterEntry(it.get(), Key1(key), expected_seq,
-                     (key % 2) ? kTypeValue : op_type,
+                     (key % 2) ? kTypeValue : kTypeDeletionWithTimestamp,
                     "value" + std::to_string(i + 1), write_ts_list[i + 1]);
      ++key;
      --expected_seq;
@@ -2446,15 +2429,13 @@ TEST_P(DBBasicTestWithTimestampCompressionSettings, PutDeleteGet) {
  const size_t kNumL0Files =
      static_cast<size_t>(Options().level0_file_num_compaction_trigger);
  {
-    // Half of the keys will go through Deletion and remaining half with
-    // SingleDeletion. Generate enough L0 files with ts=1 to trigger compaction
-    // to L1
+    // Generate enough L0 files with ts=1 to trigger compaction to L1
    std::string ts_str = Timestamp(1, 0);
    Slice ts = ts_str;
    WriteOptions wopts;
    wopts.timestamp = &ts;
-    for (size_t i = 0; i < kNumL0Files; ++i) {
-      for (int j = 0; j < kNumKeysPerFile; ++j) {
+    for (size_t i = 0; i != kNumL0Files; ++i) {
+      for (int j = 0; j != kNumKeysPerFile; ++j) {
        ASSERT_OK(db_->Put(wopts, Key1(j), "value" + std::to_string(i)));
      }
      ASSERT_OK(db_->Flush(FlushOptions()));
@@ -2464,15 +2445,11 @@ TEST_P(DBBasicTestWithTimestampCompressionSettings, PutDeleteGet) {
    ts_str = Timestamp(3, 0);
    ts = ts_str;
    wopts.timestamp = &ts;
-    for (int i = 0; i < kNumKeysPerFile; ++i) {
+    for (int i = 0; i != kNumKeysPerFile; ++i) {
      std::string key_str = Key1(i);
      Slice key(key_str);
      if ((i % 3) == 0) {
-        if (i < kNumKeysPerFile / 2) {
-          ASSERT_OK(db_->Delete(wopts, key));
-        } else {
-          ASSERT_OK(db_->SingleDelete(wopts, key));
-        }
+        ASSERT_OK(db_->Delete(wopts, key));
      } else {
        ASSERT_OK(db_->Put(wopts, key, "new_value"));
      }
@@ -2486,11 +2463,7 @@ TEST_P(DBBasicTestWithTimestampCompressionSettings, PutDeleteGet) {
      std::string key_str = Key1(i);
      Slice key(key_str);
      if ((i % 3) == 1) {
-        if (i < kNumKeysPerFile / 2) {
-          ASSERT_OK(db_->Delete(wopts, key));
-        } else {
-          ASSERT_OK(db_->SingleDelete(wopts, key));
-        }
+        ASSERT_OK(db_->Delete(wopts, key));
      } else if ((i % 3) == 2) {
        ASSERT_OK(db_->Put(wopts, key, "new_value_2"));
      }
@@ -9,10 +9,7 @@
 #include "db/dbformat.h"

 #include <stdio.h>
-
 #include <cinttypes>
-
-#include "db/lookup_key.h"
 #include "monitoring/perf_context_imp.h"
 #include "port/port.h"
 #include "util/coding.h"
@@ -9,14 +9,19 @@

 #pragma once
 #include <stdio.h>
-
 #include <memory>
 #include <string>
 #include <utility>
-
+#include "db/lookup_key.h"
+#include "db/merge_context.h"
+#include "logging/logging.h"
+#include "monitoring/perf_context_imp.h"
 #include "rocksdb/comparator.h"
+#include "rocksdb/db.h"
+#include "rocksdb/filter_policy.h"
 #include "rocksdb/slice.h"
 #include "rocksdb/slice_transform.h"
+#include "rocksdb/table.h"
 #include "rocksdb/types.h"
 #include "util/coding.h"
 #include "util/user_comparator_wrapper.h"
@@ -187,16 +187,6 @@ class ExternalSSTFileBasicTest
  std::string sst_files_dir_;
  std::unique_ptr<FaultInjectionTestEnv> fault_injection_test_env_;
  bool random_rwfile_supported_;
-#ifndef ROCKSDB_LITE
-  uint64_t GetSstSizeHelper(Temperature temperature) {
-    std::string prop;
-    EXPECT_TRUE(
-        dbfull()->GetProperty(DB::Properties::kLiveSstFilesSizeAtTemperature +
-                                  ToString(static_cast<uint8_t>(temperature)),
-                              &prop));
-    return static_cast<uint64_t>(std::atoi(prop.c_str()));
-  }
-#endif  // ROCKSDB_LITE
 };

 TEST_F(ExternalSSTFileBasicTest, Basic) {
@@ -488,20 +478,6 @@ TEST_F(ExternalSSTFileBasicTest, IngestFileWithFileChecksum) {
    ASSERT_EQ(f.file_checksum_func_name, kUnknownFileChecksumFuncName);
  }

-  // check the temperature of the file being ingested
-  ColumnFamilyMetaData metadata;
-  db_->GetColumnFamilyMetaData(&metadata);
-  ASSERT_EQ(1, metadata.file_count);
-  ASSERT_EQ(Temperature::kUnknown, metadata.levels[6].files[0].temperature);
-  auto size = GetSstSizeHelper(Temperature::kUnknown);
-  ASSERT_GT(size, 0);
-  size = GetSstSizeHelper(Temperature::kWarm);
-  ASSERT_EQ(size, 0);
-  size = GetSstSizeHelper(Temperature::kHot);
-  ASSERT_EQ(size, 0);
-  size = GetSstSizeHelper(Temperature::kCold);
-  ASSERT_EQ(size, 0);
-
  // Reopen Db with checksum enabled
  Reopen(options);
  // Enable verify_file_checksum option
@@ -622,15 +598,6 @@ TEST_F(ExternalSSTFileBasicTest, IngestFileWithFileChecksum) {
  }
  ASSERT_OK(s) << s.ToString();
  ASSERT_OK(env_->FileExists(file6));
-  db_->GetColumnFamilyMetaData(&metadata);
-  size = GetSstSizeHelper(Temperature::kUnknown);
-  ASSERT_GT(size, 0);
-  size = GetSstSizeHelper(Temperature::kWarm);
-  ASSERT_EQ(size, 0);
-  size = GetSstSizeHelper(Temperature::kHot);
-  ASSERT_EQ(size, 0);
-  size = GetSstSizeHelper(Temperature::kCold);
-  ASSERT_EQ(size, 0);
 }

 TEST_F(ExternalSSTFileBasicTest, NoCopy) {
@@ -1699,103 +1666,6 @@ TEST_F(ExternalSSTFileBasicTest, IngestFileAfterDBPut) {
  ASSERT_EQ(Get("k"), "b");
 }

-TEST_F(ExternalSSTFileBasicTest, IngestWithTemperature) {
-  Options options = CurrentOptions();
-  const ImmutableCFOptions ioptions(options);
-  options.bottommost_temperature = Temperature::kWarm;
-  SstFileWriter sst_file_writer(EnvOptions(), options);
-  options.level0_file_num_compaction_trigger = 2;
-  Reopen(options);
-
-  auto size = GetSstSizeHelper(Temperature::kUnknown);
-  ASSERT_EQ(size, 0);
-  size = GetSstSizeHelper(Temperature::kWarm);
-  ASSERT_EQ(size, 0);
-  size = GetSstSizeHelper(Temperature::kHot);
-  ASSERT_EQ(size, 0);
-
-  // create file01.sst (1000 => 1099) and ingest it
-  std::string file1 = sst_files_dir_ + "file01.sst";
-  ASSERT_OK(sst_file_writer.Open(file1));
-  for (int k = 1000; k < 1100; k++) {
-    ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val"));
-  }
-  ExternalSstFileInfo file1_info;
-  Status s = sst_file_writer.Finish(&file1_info);
-  ASSERT_OK(s);
-  ASSERT_EQ(file1_info.file_path, file1);
-  ASSERT_EQ(file1_info.num_entries, 100);
-  ASSERT_EQ(file1_info.smallest_key, Key(1000));
-  ASSERT_EQ(file1_info.largest_key, Key(1099));
-
-  std::vector<std::string> files;
-  std::vector<std::string> files_checksums;
-  std::vector<std::string> files_checksum_func_names;
-  Temperature file_temperature = Temperature::kWarm;
-
-  files.push_back(file1);
-  IngestExternalFileOptions in_opts;
-  in_opts.move_files = false;
-  in_opts.snapshot_consistency = true;
-  in_opts.allow_global_seqno = false;
-  in_opts.allow_blocking_flush = false;
-  in_opts.write_global_seqno = true;
-  in_opts.verify_file_checksum = false;
-  IngestExternalFileArg arg;
-  arg.column_family = db_->DefaultColumnFamily();
-  arg.external_files = files;
-  arg.options = in_opts;
-  arg.files_checksums = files_checksums;
-  arg.files_checksum_func_names = files_checksum_func_names;
-  arg.file_temperature = file_temperature;
-  s = db_->IngestExternalFiles({arg});
-  ASSERT_OK(s);
-
-  // check the temperature of the file being ingested
-  ColumnFamilyMetaData metadata;
-  db_->GetColumnFamilyMetaData(&metadata);
-  ASSERT_EQ(1, metadata.file_count);
-  ASSERT_EQ(Temperature::kWarm, metadata.levels[6].files[0].temperature);
-  size = GetSstSizeHelper(Temperature::kUnknown);
-  ASSERT_EQ(size, 0);
-  size = GetSstSizeHelper(Temperature::kWarm);
-  ASSERT_GT(size, 1);
-
-  // non-bottommost file still has unknown temperature
-  ASSERT_OK(Put("foo", "bar"));
-  ASSERT_OK(Put("bar", "bar"));
-  ASSERT_OK(Flush());
-  db_->GetColumnFamilyMetaData(&metadata);
-  ASSERT_EQ(2, metadata.file_count);
-  ASSERT_EQ(Temperature::kUnknown, metadata.levels[0].files[0].temperature);
-  size = GetSstSizeHelper(Temperature::kUnknown);
-  ASSERT_GT(size, 0);
-  size = GetSstSizeHelper(Temperature::kWarm);
-  ASSERT_GT(size, 0);
-
-  // reopen and check the information is persisted
-  Reopen(options);
-  db_->GetColumnFamilyMetaData(&metadata);
-  ASSERT_EQ(2, metadata.file_count);
-  ASSERT_EQ(Temperature::kUnknown, metadata.levels[0].files[0].temperature);
-  ASSERT_EQ(Temperature::kWarm, metadata.levels[6].files[0].temperature);
-  size = GetSstSizeHelper(Temperature::kUnknown);
-  ASSERT_GT(size, 0);
-  size = GetSstSizeHelper(Temperature::kWarm);
-  ASSERT_GT(size, 0);
-
-  // check other non-exist temperatures
-  size = GetSstSizeHelper(Temperature::kHot);
-  ASSERT_EQ(size, 0);
-  size = GetSstSizeHelper(Temperature::kCold);
-  ASSERT_EQ(size, 0);
-  std::string prop;
-  ASSERT_TRUE(dbfull()->GetProperty(
-      DB::Properties::kLiveSstFilesSizeAtTemperature + std::to_string(22),
-      &prop));
-  ASSERT_EQ(std::atoi(prop.c_str()), 0);
-}
-
 INSTANTIATE_TEST_CASE_P(ExternalSSTFileBasicTest, ExternalSSTFileBasicTest,
                        testing::Values(std::make_tuple(true, true),
                                        std::make_tuple(true, false),
@@ -17,7 +17,6 @@
 #include "db/version_edit.h"
 #include "file/file_util.h"
 #include "file/random_access_file_reader.h"
-#include "logging/logging.h"
 #include "table/merging_iterator.h"
 #include "table/scoped_arena_iterator.h"
 #include "table/sst_file_writer_collectors.h"
@@ -31,8 +30,7 @@ Status ExternalSstFileIngestionJob::Prepare(
    const std::vector<std::string>& external_files_paths,
    const std::vector<std::string>& files_checksums,
    const std::vector<std::string>& files_checksum_func_names,
-    const Temperature& file_temperature, uint64_t next_file_number,
-    SuperVersion* sv) {
+    uint64_t next_file_number, SuperVersion* sv) {
  Status status;

  // Read the information of files we are ingesting
@@ -90,11 +88,6 @@ Status ExternalSstFileIngestionJob::Prepare(
    }
  }

-  // Hanlde the file temperature
-  for (size_t i = 0; i < num_files; i++) {
-    files_to_ingest_[i].file_temperature = file_temperature;
-  }
-
  if (ingestion_options_.ingest_behind && files_overlap_) {
    return Status::NotSupported("Files have overlapping ranges");
  }
@@ -435,13 +428,12 @@ Status ExternalSstFileIngestionJob::Run() {
      current_time = oldest_ancester_time =
          static_cast<uint64_t>(temp_current_time);
    }
-    FileMetaData f_metadata(
-        f.fd.GetNumber(), f.fd.GetPathId(), f.fd.GetFileSize(),
-        f.smallest_internal_key, f.largest_internal_key, f.assigned_seqno,
-        f.assigned_seqno, false, kInvalidBlobFileNumber, oldest_ancester_time,
-        current_time, f.file_checksum, f.file_checksum_func_name);
-    f_metadata.temperature = f.file_temperature;
-    edit_.AddFile(f.picked_level, f_metadata);
+
+    edit_.AddFile(f.picked_level, f.fd.GetNumber(), f.fd.GetPathId(),
+                  f.fd.GetFileSize(), f.smallest_internal_key,
+                  f.largest_internal_key, f.assigned_seqno, f.assigned_seqno,
+                  false, kInvalidBlobFileNumber, oldest_ancester_time,
+                  current_time, f.file_checksum, f.file_checksum_func_name);
  }
  return status;
 }
@@ -9,6 +9,7 @@
 #include <vector>

 #include "db/column_family.h"
+#include "db/dbformat.h"
 #include "db/internal_stats.h"
 #include "db/snapshot_impl.h"
 #include "env/file_system_tracer.h"
@@ -68,8 +69,6 @@ struct IngestedFileInfo {
  std::string file_checksum;
  // The name of checksum function that generate the checksum
  std::string file_checksum_func_name;
-  // The temperature of the file to be ingested
-  Temperature file_temperature = Temperature::kUnknown;
 };

 class ExternalSstFileIngestionJob {
@@ -101,8 +100,7 @@ class ExternalSstFileIngestionJob {
  Status Prepare(const std::vector<std::string>& external_files_paths,
                 const std::vector<std::string>& files_checksums,
                 const std::vector<std::string>& files_checksum_func_names,
-                 const Temperature& file_temperature, uint64_t next_file_number,
-                 SuperVersion* sv);
+                 uint64_t next_file_number, SuperVersion* sv);

  // Check if we need to flush the memtable before running the ingestion job
  // This will be true if the files we are ingesting are overlapping with any
@@ -1991,8 +1991,7 @@ TEST_F(ExternalSSTFileTest, CompactionDeadlock) {
    if (running_threads.load() == 0) {
      break;
    }
-    // Make sure we do a "real sleep", not a mock one.
-    SystemClock::Default()->SleepForMicroseconds(500000);
+    env_->SleepForMicroseconds(500000);
  }

  ASSERT_EQ(running_threads.load(), 0);
@@ -94,7 +94,7 @@ class FaultInjectionTest
      return false;
    } else {
      if (option_config_ == kMultiLevels) {
-        base_env_.reset(MockEnv::Create(Env::Default()));
+        base_env_.reset(new MockEnv(Env::Default()));
      }
      return true;
    }
@@ -19,6 +19,7 @@

 #include "db/blob/blob_file_completion_callback.h"
 #include "db/column_family.h"
+#include "db/dbformat.h"
 #include "db/flush_scheduler.h"
 #include "db/internal_stats.h"
 #include "db/job_context.h"
@@ -10,6 +10,7 @@
 #include <vector>
 #include <queue>

+#include "db/dbformat.h"
 #include "memory/arena.h"
 #include "rocksdb/db.h"
 #include "rocksdb/iterator.h"
@@ -10,7 +10,6 @@
 #include "db/version_edit.h"
 #include "file/file_util.h"
 #include "file/random_access_file_reader.h"
-#include "logging/logging.h"
 #include "table/merging_iterator.h"
 #include "table/scoped_arena_iterator.h"
 #include "table/sst_file_writer_collectors.h"
@@ -4,6 +4,7 @@
 #include <vector>

 #include "db/column_family.h"
+#include "db/dbformat.h"
 #include "db/external_sst_file_ingestion_job.h"
 #include "db/snapshot_impl.h"
 #include "options/db_options.h"
@@ -60,25 +60,6 @@ const std::map<LevelStatType, LevelStat> InternalStats::compaction_level_stats =
        {LevelStatType::W_BLOB_GB, LevelStat{"WblobGB", "Wblob(GB)"}},
 };

-const std::map<InternalStats::InternalDBStatsType, DBStatInfo>
-    InternalStats::db_stats_type_to_info = {
-        {InternalStats::kIntStatsWalFileBytes,
-         DBStatInfo{"db.wal_bytes_written"}},
-        {InternalStats::kIntStatsWalFileSynced, DBStatInfo{"db.wal_syncs"}},
-        {InternalStats::kIntStatsBytesWritten,
-         DBStatInfo{"db.user_bytes_written"}},
-        {InternalStats::kIntStatsNumKeysWritten,
-         DBStatInfo{"db.user_keys_written"}},
-        {InternalStats::kIntStatsWriteDoneByOther,
-         DBStatInfo{"db.user_writes_by_other"}},
-        {InternalStats::kIntStatsWriteDoneBySelf,
-         DBStatInfo{"db.user_writes_by_self"}},
-        {InternalStats::kIntStatsWriteWithWal,
-         DBStatInfo{"db.user_writes_with_wal"}},
-        {InternalStats::kIntStatsWriteStallMicros,
-         DBStatInfo{"db.user_write_stall_micros"}},
-};
-
 namespace {
 const double kMB = 1048576.0;
 const double kGB = kMB * 1024;
@@ -427,8 +408,7 @@ const std::unordered_map<std::string, DBPropertyInfo>
         {false, &InternalStats::HandleCFFileHistogram, nullptr, nullptr,
          nullptr}},
        {DB::Properties::kDBStats,
-         {false, &InternalStats::HandleDBStats, nullptr,
-          &InternalStats::HandleDBMapStats, nullptr}},
+         {false, &InternalStats::HandleDBStats, nullptr, nullptr, nullptr}},
        {DB::Properties::kBlockCacheEntryStats,
         {true, &InternalStats::HandleBlockCacheEntryStats, nullptr,
          &InternalStats::HandleBlockCacheEntryStatsMap, nullptr}},
@@ -918,12 +898,6 @@ bool InternalStats::HandleCFFileHistogram(std::string* value,
  return true;
 }

-bool InternalStats::HandleDBMapStats(
-    std::map<std::string, std::string>* db_stats, Slice /*suffix*/) {
-  DumpDBMapStats(db_stats);
-  return true;
-}
-
 bool InternalStats::HandleDBStats(std::string* value, Slice /*suffix*/) {
  DumpDBStats(value);
  return true;
@@ -1300,21 +1274,10 @@ bool InternalStats::HandleBlockCachePinnedUsage(uint64_t* value, DBImpl* /*db*/,
  return true;
 }

-void InternalStats::DumpDBMapStats(
-    std::map<std::string, std::string>* db_stats) {
-  for (int i = 0; i < static_cast<int>(kIntStatsNumMax); ++i) {
-    InternalDBStatsType type = static_cast<InternalDBStatsType>(i);
-    (*db_stats)[db_stats_type_to_info.at(type).property_name] =
-        std::to_string(GetDBStats(type));
-  }
-  double seconds_up = (clock_->NowMicros() - started_at_) / kMicrosInSec;
-  (*db_stats)["db.uptime"] = std::to_string(seconds_up);
-}
-
 void InternalStats::DumpDBStats(std::string* value) {
  char buf[1000];
  // DB-level stats, only available from default column family
-  double seconds_up = (clock_->NowMicros() - started_at_) / kMicrosInSec;
+  double seconds_up = (clock_->NowMicros() - started_at_ + 1) / kMicrosInSec;
  double interval_seconds_up = seconds_up - db_stats_snapshot_.seconds_up;
  snprintf(buf, sizeof(buf),
           "\n** DB Stats **\nUptime(secs): %.1f total, %.1f interval\n",
@@ -1351,10 +1314,8 @@ void InternalStats::DumpDBStats(std::string* value) {
           NumberToHumanString(write_other + write_self).c_str(),
           NumberToHumanString(num_keys_written).c_str(),
           NumberToHumanString(write_self).c_str(),
-           (write_other + write_self) /
-               std::max(1.0, static_cast<double>(write_self)),
-           user_bytes_written / kGB,
-           user_bytes_written / kMB / std::max(seconds_up, 0.001));
+           (write_other + write_self) / static_cast<double>(write_self + 1),
+           user_bytes_written / kGB, user_bytes_written / kMB / seconds_up);
  value->append(buf);
  // WAL
  snprintf(buf, sizeof(buf),
@@ -1362,8 +1323,8 @@ void InternalStats::DumpDBStats(std::string* value) {
           "%.2f writes per sync, written: %.2f GB, %.2f MB/s\n",
           NumberToHumanString(write_with_wal).c_str(),
           NumberToHumanString(wal_synced).c_str(),
-           write_with_wal / std::max(1.0, static_cast<double>(wal_synced)),
-           wal_bytes / kGB, wal_bytes / kMB / std::max(seconds_up, 0.001));
+           write_with_wal / static_cast<double>(wal_synced + 1),
+           wal_bytes / kGB, wal_bytes / kMB / seconds_up);
  value->append(buf);
  // Stall
  AppendHumanMicros(write_stall_micros, human_micros, kHumanMicrosLen, true);
@@ -1386,7 +1347,7 @@ void InternalStats::DumpDBStats(std::string* value) {
      NumberToHumanString(interval_num_keys_written).c_str(),
      NumberToHumanString(interval_write_self).c_str(),
      static_cast<double>(interval_write_other + interval_write_self) /
-          std::max(1.0, static_cast<double>(interval_write_self)),
+          (interval_write_self + 1),
      (user_bytes_written - db_stats_snapshot_.ingest_bytes) / kMB,
      (user_bytes_written - db_stats_snapshot_.ingest_bytes) / kMB /
          std::max(interval_seconds_up, 0.001)),
@@ -1397,15 +1358,15 @@ void InternalStats::DumpDBStats(std::string* value) {
  uint64_t interval_wal_synced = wal_synced - db_stats_snapshot_.wal_synced;
  uint64_t interval_wal_bytes = wal_bytes - db_stats_snapshot_.wal_bytes;

-  snprintf(buf, sizeof(buf),
-           "Interval WAL: %s writes, %s syncs, "
-           "%.2f writes per sync, written: %.2f GB, %.2f MB/s\n",
-           NumberToHumanString(interval_write_with_wal).c_str(),
-           NumberToHumanString(interval_wal_synced).c_str(),
-           interval_write_with_wal /
-               std::max(1.0, static_cast<double>(interval_wal_synced)),
-           interval_wal_bytes / kGB,
-           interval_wal_bytes / kMB / std::max(interval_seconds_up, 0.001));
+  snprintf(
+      buf, sizeof(buf),
+      "Interval WAL: %s writes, %s syncs, "
+      "%.2f writes per sync, written: %.2f GB, %.2f MB/s\n",
+      NumberToHumanString(interval_write_with_wal).c_str(),
+      NumberToHumanString(interval_wal_synced).c_str(),
+      interval_write_with_wal / static_cast<double>(interval_wal_synced + 1),
+      interval_wal_bytes / kGB,
+      interval_wal_bytes / kMB / std::max(interval_seconds_up, 0.001));
  value->append(buf);

  // Stall
@@ -1653,7 +1614,7 @@ void InternalStats::DumpCFStatsNoFileHistogram(std::string* value) {
  value->append(buf);

  uint64_t now_micros = clock_->NowMicros();
-  double seconds_up = (now_micros - started_at_) / kMicrosInSec;
+  double seconds_up = (now_micros - started_at_ + 1) / kMicrosInSec;
  double interval_seconds_up = seconds_up - cf_stats_snapshot_.seconds_up;
  snprintf(buf, sizeof(buf), "Uptime(secs): %.1f total, %.1f interval\n",
           seconds_up, interval_seconds_up);
@@ -1703,10 +1664,8 @@ void InternalStats::DumpCFStatsNoFileHistogram(std::string* value) {
  snprintf(buf, sizeof(buf),
           "Cumulative compaction: %.2f GB write, %.2f MB/s write, "
           "%.2f GB read, %.2f MB/s read, %.1f seconds\n",
-           compact_bytes_write / kGB,
-           compact_bytes_write / kMB / std::max(seconds_up, 0.001),
-           compact_bytes_read / kGB,
-           compact_bytes_read / kMB / std::max(seconds_up, 0.001),
+           compact_bytes_write / kGB, compact_bytes_write / kMB / seconds_up,
+           compact_bytes_read / kGB, compact_bytes_read / kMB / seconds_up,
           compact_micros / kMicrosInSec);
  value->append(buf);

@@ -96,11 +96,6 @@ struct LevelStat {
  std::string header_name;
 };

-struct DBStatInfo {
-  // This what will be property_name in the flat map returned to the user
-  std::string property_name;
-};
-
 class InternalStats {
 public:
  static const std::map<LevelStatType, LevelStat> compaction_level_stats;
@@ -135,8 +130,6 @@ class InternalStats {
    kIntStatsNumMax,
  };

-  static const std::map<InternalDBStatsType, DBStatInfo> db_stats_type_to_info;
-
  InternalStats(int num_levels, SystemClock* clock, ColumnFamilyData* cfd);

  // Per level compaction stats.  comp_stats_[level] stores the stats for
@@ -485,7 +478,6 @@ class InternalStats {
  static const std::unordered_map<std::string, DBPropertyInfo> ppt_name_to_info;

 private:
-  void DumpDBMapStats(std::map<std::string, std::string>* db_stats);
  void DumpDBStats(std::string* value);
  void DumpCFMapStats(std::map<std::string, std::string>* cf_stats);
  void DumpCFMapStats(
@@ -618,8 +610,6 @@ class InternalStats {
  bool HandleCFStats(std::string* value, Slice suffix);
  bool HandleCFStatsNoFileHistogram(std::string* value, Slice suffix);
  bool HandleCFFileHistogram(std::string* value, Slice suffix);
-  bool HandleDBMapStats(std::map<std::string, std::string>* compaction_stats,
-                        Slice suffix);
  bool HandleDBStats(std::string* value, Slice suffix);
  bool HandleSsTables(std::string* value, Slice suffix);
  bool HandleAggregatedTableProperties(std::string* value, Slice suffix);
@@ -117,7 +117,7 @@ class ProtectionInfoKVO {
  friend class ProtectionInfoKVOS<T>;
  friend class ProtectionInfoKVOC<T>;

-  explicit ProtectionInfoKVO<T>(T val) : info_(val) {
+  ProtectionInfoKVO<T>(T val) : info_(val) {
    static_assert(sizeof(ProtectionInfoKVO<T>) == sizeof(T), "");
  }

@@ -155,7 +155,7 @@ class ProtectionInfoKVOC {
 private:
  friend class ProtectionInfoKVO<T>;

-  explicit ProtectionInfoKVOC<T>(T val) : kvo_(val) {
+  ProtectionInfoKVOC<T>(T val) : kvo_(val) {
    static_assert(sizeof(ProtectionInfoKVOC<T>) == sizeof(T), "");
  }

@@ -193,7 +193,7 @@ class ProtectionInfoKVOS {
 private:
  friend class ProtectionInfoKVO<T>;

-  explicit ProtectionInfoKVOS<T>(T val) : kvo_(val) {
+  ProtectionInfoKVOS<T>(T val) : kvo_(val) {
    static_assert(sizeof(ProtectionInfoKVOS<T>) == sizeof(T), "");
  }

@@ -1000,7 +1000,6 @@ class TestFileOperationListener : public EventListener {
    file_syncs_success_.store(0);
    file_truncates_.store(0);
    file_truncates_success_.store(0);
-    file_seq_reads_.store(0);
    blob_file_reads_.store(0);
    blob_file_writes_.store(0);
    blob_file_flushes_.store(0);
@@ -1014,9 +1013,6 @@ class TestFileOperationListener : public EventListener {
    if (info.status.ok()) {
      ++file_reads_success_;
    }
-    if (info.path.find("MANIFEST") != std::string::npos) {
-      ++file_seq_reads_;
-    }
    if (EndsWith(info.path, ".blob")) {
      ++blob_file_reads_;
    }
@@ -1092,7 +1088,6 @@ class TestFileOperationListener : public EventListener {
  std::atomic<size_t> file_syncs_success_;
  std::atomic<size_t> file_truncates_;
  std::atomic<size_t> file_truncates_success_;
-  std::atomic<size_t> file_seq_reads_;
  std::atomic<size_t> blob_file_reads_;
  std::atomic<size_t> blob_file_writes_;
  std::atomic<size_t> blob_file_flushes_;
@@ -1188,30 +1183,6 @@ TEST_F(EventListenerTest, OnBlobFileOperationTest) {
  }
 }

-TEST_F(EventListenerTest, ReadManifestAndWALOnRecovery) {
-  Options options;
-  options.env = CurrentOptions().env;
-  options.create_if_missing = true;
-
-  TestFileOperationListener* listener = new TestFileOperationListener();
-  options.listeners.emplace_back(listener);
-
-  options.use_direct_io_for_flush_and_compaction = false;
-  Status s = TryReopen(options);
-  if (s.IsInvalidArgument()) {
-    options.use_direct_io_for_flush_and_compaction = false;
-  } else {
-    ASSERT_OK(s);
-  }
-  DestroyAndReopen(options);
-  ASSERT_OK(Put("foo", "aaa"));
-  Close();
-
-  size_t seq_reads = listener->file_seq_reads_.load();
-  Reopen(options);
-  ASSERT_GT(listener->file_seq_reads_.load(), seq_reads);
-}
-
 class BlobDBJobLevelEventListenerTest : public EventListener {
 public:
  explicit BlobDBJobLevelEventListenerTest(EventListenerTest* test)
@@ -1456,6 +1427,12 @@ TEST_F(EventListenerTest, BlobDBCompactFiles) {

 class BlobDBFileLevelEventListener : public EventListener {
 public:
+  BlobDBFileLevelEventListener() {
+    files_started_.store(0);
+    files_created_.store(0);
+    files_deleted_.store(0);
+  }
+
  void OnBlobFileCreationStarted(
      const BlobFileCreationBriefInfo& info) override {
    files_started_++;
@@ -1494,9 +1471,9 @@ class BlobDBFileLevelEventListener : public EventListener {
  }

 private:
-  std::atomic<uint32_t> files_started_{};
-  std::atomic<uint32_t> files_created_{};
-  std::atomic<uint32_t> files_deleted_{};
+  std::atomic<uint32_t> files_started_;
+  std::atomic<uint32_t> files_created_;
+  std::atomic<uint32_t> files_deleted_;
 };

 TEST_F(EventListenerTest, BlobDBFileTest) {
@@ -10,6 +10,7 @@
 #pragma once
 #include <string>
 #include <utility>
+#include "rocksdb/db.h"
 #include "rocksdb/slice.h"
 #include "rocksdb/types.h"

@@ -21,7 +21,6 @@
 #include "db/pinned_iterators_manager.h"
 #include "db/range_tombstone_fragmenter.h"
 #include "db/read_callback.h"
-#include "logging/logging.h"
 #include "memory/arena.h"
 #include "memory/memory_usage.h"
 #include "monitoring/perf_context_imp.h"
@@ -16,7 +16,6 @@
 #include "db/range_tombstone_fragmenter.h"
 #include "db/version_set.h"
 #include "logging/log_buffer.h"
-#include "logging/logging.h"
 #include "monitoring/thread_status_util.h"
 #include "rocksdb/db.h"
 #include "rocksdb/env.h"
@@ -12,6 +12,7 @@
 #include <string>
 #include <vector>

+#include "db/dbformat.h"
 #include "db/logs_with_prep_tracker.h"
 #include "db/memtable.h"
 #include "db/range_del_aggregator.h"
@@ -9,12 +9,12 @@
 #include <string>
 #include <vector>

+#include "db/dbformat.h"
 #include "db/merge_context.h"
 #include "db/range_del_aggregator.h"
 #include "db/snapshot_checker.h"
 #include "rocksdb/compaction_filter.h"
 #include "rocksdb/env.h"
-#include "rocksdb/merge_operator.h"
 #include "rocksdb/slice.h"
 #include "util/stop_watch.h"

@@ -26,7 +26,6 @@ class Logger;
 class MergeOperator;
 class Statistics;
 class SystemClock;
-class Version;

 class MergeHelper {
 public:
@@ -3,33 +3,30 @@
 //  COPYING file in the root directory) and Apache 2.0 License
 //  (found in the LICENSE.Apache file in the root directory).

-#include "db/merge_helper.h"
-
 #include <algorithm>
 #include <string>
 #include <vector>

-#include "db/dbformat.h"
+#include "db/merge_helper.h"
 #include "rocksdb/comparator.h"
 #include "test_util/testharness.h"
 #include "test_util/testutil.h"
 #include "util/coding.h"
-#include "util/vector_iterator.h"
 #include "utilities/merge_operators.h"

 namespace ROCKSDB_NAMESPACE {

 class MergeHelperTest : public testing::Test {
 public:
-  MergeHelperTest() : icmp_(BytewiseComparator()) { env_ = Env::Default(); }
+  MergeHelperTest() { env_ = Env::Default(); }

  ~MergeHelperTest() override = default;

  Status Run(SequenceNumber stop_before, bool at_bottom,
             SequenceNumber latest_snapshot = 0) {
-    iter_.reset(new VectorIterator(ks_, vs_, &icmp_));
+    iter_.reset(new test::VectorIterator(ks_, vs_));
    iter_->SeekToFirst();
-    merge_helper_.reset(new MergeHelper(env_, icmp_.user_comparator(),
+    merge_helper_.reset(new MergeHelper(env_, BytewiseComparator(),
                                        merge_op_.get(), filter_.get(), nullptr,
                                        false, latest_snapshot));
    return merge_helper_->MergeUntil(iter_.get(), nullptr /* range_del_agg */,
@@ -48,8 +45,7 @@ class MergeHelperTest : public testing::Test {
  }

  Env* env_;
-  InternalKeyComparator icmp_;
-  std::unique_ptr<VectorIterator> iter_;
+  std::unique_ptr<test::VectorIterator> iter_;
  std::shared_ptr<MergeOperator> merge_op_;
  std::unique_ptr<MergeHelper> merge_helper_;
  std::vector<std::string> ks_;
@@ -54,9 +54,7 @@ class ObsoleteFilesTest : public DBTestBase {
      AddKeys(numKeysPerFile, startKey);
      startKey += numKeysPerFile;
      ASSERT_OK(dbfull()->TEST_FlushMemTable());
-      ASSERT_OK(
-          dbfull()->TEST_WaitForCompact());  // wait for background flush (flush
-                                             // is also a kind of compaction).
+      ASSERT_OK(dbfull()->TEST_WaitForFlushMemTable());
    }
  }

@@ -5,9 +5,6 @@
 //
 #include "db/output_validator.h"

-#include "test_util/sync_point.h"
-#include "util/hash.h"
-
 namespace ROCKSDB_NAMESPACE {
 Status OutputValidator::Add(const Slice& key, const Slice& value) {
  if (enable_hash_) {
@@ -5,8 +5,9 @@
 //
 #pragma once
 #include "db/dbformat.h"
-#include "rocksdb/slice.h"
 #include "rocksdb/status.h"
+#include "test_util/sync_point.h"
+#include "util/hash.h"

 namespace ROCKSDB_NAMESPACE {
 // A class that validates key/value that is inserted to an SST file.
@@ -19,16 +19,15 @@ int main() {
 #include <string>
 #include <vector>

-#include "db/dbformat.h"
 #include "db/range_del_aggregator.h"
 #include "db/range_tombstone_fragmenter.h"
 #include "rocksdb/comparator.h"
 #include "rocksdb/system_clock.h"
+#include "test_util/testutil.h"
 #include "util/coding.h"
 #include "util/gflags_compat.h"
 #include "util/random.h"
 #include "util/stop_watch.h"
-#include "util/vector_iterator.h"

 using GFLAGS_NAMESPACE::ParseCommandLineFlags;

@@ -147,8 +146,8 @@ std::unique_ptr<InternalIterator> MakeRangeDelIterator(
    keys.push_back(key_and_value.first.Encode().ToString());
    values.push_back(key_and_value.second.ToString());
  }
-  return std::unique_ptr<VectorIterator>(
-      new VectorIterator(keys, values, &icmp));
+  return std::unique_ptr<test::VectorIterator>(
+      new test::VectorIterator(keys, values));
 }

 // convert long to a big-endian slice key
@@ -208,6 +207,8 @@ int main(int argc, char** argv) {
                ROCKSDB_NAMESPACE::Key(start), ROCKSDB_NAMESPACE::Key(end), j);
      }

+      auto range_del_iter =
+          ROCKSDB_NAMESPACE::MakeRangeDelIterator(persistent_range_tombstones);
      fragmented_range_tombstone_lists.emplace_back(
          new ROCKSDB_NAMESPACE::FragmentedRangeTombstoneList(
              ROCKSDB_NAMESPACE::MakeRangeDelIterator(
@@ -13,7 +13,6 @@
 #include "db/dbformat.h"
 #include "db/range_tombstone_fragmenter.h"
 #include "test_util/testutil.h"
-#include "util/vector_iterator.h"

 namespace ROCKSDB_NAMESPACE {

@@ -31,8 +30,8 @@ std::unique_ptr<InternalIterator> MakeRangeDelIter(
    keys.push_back(key_and_value.first.Encode().ToString());
    values.push_back(key_and_value.second.ToString());
  }
-  return std::unique_ptr<VectorIterator>(
-      new VectorIterator(keys, values, &bytewise_icmp));
+  return std::unique_ptr<test::VectorIterator>(
+      new test::VectorIterator(keys, values));
 }

 std::vector<std::unique_ptr<FragmentedRangeTombstoneList>>
@@ -6,10 +6,8 @@
 #include "db/range_tombstone_fragmenter.h"

 #include "db/db_test_util.h"
-#include "db/dbformat.h"
 #include "rocksdb/comparator.h"
 #include "test_util/testutil.h"
-#include "util/vector_iterator.h"

 namespace ROCKSDB_NAMESPACE {

@@ -27,8 +25,8 @@ std::unique_ptr<InternalIterator> MakeRangeDelIter(
    keys.push_back(key_and_value.first.Encode().ToString());
    values.push_back(key_and_value.second.ToString());
  }
-  return std::unique_ptr<VectorIterator>(
-      new VectorIterator(keys, values, &bytewise_icmp));
+  return std::unique_ptr<test::VectorIterator>(
+      new test::VectorIterator(keys, values));
 }

 void CheckIterPosition(const RangeTombstone& tombstone,
@@ -62,7 +62,6 @@
 #ifndef ROCKSDB_LITE

 #include <cinttypes>
-
 #include "db/builder.h"
 #include "db/db_impl/db_impl.h"
 #include "db/dbformat.h"
@@ -74,7 +73,6 @@
 #include "db/write_batch_internal.h"
 #include "file/filename.h"
 #include "file/writable_file_writer.h"
-#include "logging/logging.h"
 #include "options/cf_options.h"
 #include "rocksdb/comparator.h"
 #include "rocksdb/db.h"
@@ -103,7 +103,7 @@ Status TableCache::GetTableReader(
    std::unique_ptr<TableReader>* table_reader,
    const SliceTransform* prefix_extractor, bool skip_filters, int level,
    bool prefetch_index_and_filter_in_cache,
-    size_t max_file_size_for_l0_meta_pin, Temperature file_temperature) {
+    size_t max_file_size_for_l0_meta_pin) {
  std::string fname =
      TableFileName(ioptions_.cf_paths, fd.GetNumber(), fd.GetPathId());
  std::unique_ptr<FSRandomAccessFile> file;
@@ -132,8 +132,7 @@ Status TableCache::GetTableReader(
        new RandomAccessFileReader(
            std::move(file), fname, ioptions_.clock, io_tracer_,
            record_read_stats ? ioptions_.stats : nullptr, SST_READ_MICROS,
-            file_read_hist, ioptions_.rate_limiter.get(), ioptions_.listeners,
-            file_temperature));
+            file_read_hist, ioptions_.rate_limiter.get(), ioptions_.listeners));
    s = ioptions_.table_factory->NewTableReader(
        ro,
        TableReaderOptions(
@@ -155,13 +154,15 @@ void TableCache::EraseHandle(const FileDescriptor& fd, Cache::Handle* handle) {
  cache_->Erase(key);
 }

-Status TableCache::FindTable(
-    const ReadOptions& ro, const FileOptions& file_options,
-    const InternalKeyComparator& internal_comparator, const FileDescriptor& fd,
-    Cache::Handle** handle, const SliceTransform* prefix_extractor,
-    const bool no_io, bool record_read_stats, HistogramImpl* file_read_hist,
-    bool skip_filters, int level, bool prefetch_index_and_filter_in_cache,
-    size_t max_file_size_for_l0_meta_pin, Temperature file_temperature) {
+Status TableCache::FindTable(const ReadOptions& ro,
+                             const FileOptions& file_options,
+                             const InternalKeyComparator& internal_comparator,
+                             const FileDescriptor& fd, Cache::Handle** handle,
+                             const SliceTransform* prefix_extractor,
+                             const bool no_io, bool record_read_stats,
+                             HistogramImpl* file_read_hist, bool skip_filters,
+                             int level, bool prefetch_index_and_filter_in_cache,
+                             size_t max_file_size_for_l0_meta_pin) {
  PERF_TIMER_GUARD_WITH_CLOCK(find_table_nanos, ioptions_.clock);
  uint64_t number = fd.GetNumber();
  Slice key = GetSliceForFileNumber(&number);
@@ -185,7 +186,7 @@ Status TableCache::FindTable(
        ro, file_options, internal_comparator, fd, false /* sequential mode */,
        record_read_stats, file_read_hist, &table_reader, prefix_extractor,
        skip_filters, level, prefetch_index_and_filter_in_cache,
-        max_file_size_for_l0_meta_pin, file_temperature);
+        max_file_size_for_l0_meta_pin);
    if (!s.ok()) {
      assert(table_reader == nullptr);
      RecordTick(ioptions_.stats, NO_FILE_ERRORS);
@@ -230,7 +231,7 @@ InternalIterator* TableCache::NewIterator(
        options.read_tier == kBlockCacheTier /* no_io */,
        !for_compaction /* record_read_stats */, file_read_hist, skip_filters,
        level, true /* prefetch_index_and_filter_in_cache */,
-        max_file_size_for_l0_meta_pin, file_meta.temperature);
+        max_file_size_for_l0_meta_pin);
    if (s.ok()) {
      table_reader = GetTableReaderFromHandle(handle);
    }
@@ -430,7 +431,7 @@ Status TableCache::Get(const ReadOptions& options,
                    options.read_tier == kBlockCacheTier /* no_io */,
                    true /* record_read_stats */, file_read_hist, skip_filters,
                    level, true /* prefetch_index_and_filter_in_cache */,
-                    max_file_size_for_l0_meta_pin, file_meta.temperature);
+                    max_file_size_for_l0_meta_pin);
      if (s.ok()) {
        t = GetTableReaderFromHandle(handle);
      }
@@ -530,12 +531,10 @@ Status TableCache::MultiGet(const ReadOptions& options,
  // found in the row cache and thus the range may now be empty
  if (s.ok() && !table_range.empty()) {
    if (t == nullptr) {
-      s = FindTable(options, file_options_, internal_comparator, fd, &handle,
-                    prefix_extractor,
-                    options.read_tier == kBlockCacheTier /* no_io */,
-                    true /* record_read_stats */, file_read_hist, skip_filters,
-                    level, true /* prefetch_index_and_filter_in_cache */,
-                    0 /*max_file_size_for_l0_meta_pin*/, file_meta.temperature);
+      s = FindTable(
+          options, file_options_, internal_comparator, fd, &handle,
+          prefix_extractor, options.read_tier == kBlockCacheTier /* no_io */,
+          true /* record_read_stats */, file_read_hist, skip_filters, level);
      TEST_SYNC_POINT_CALLBACK("TableCache::MultiGet:FindTable", &s);
      if (s.ok()) {
        t = GetTableReaderFromHandle(handle);
@@ -140,8 +140,7 @@ class TableCache {
                   HistogramImpl* file_read_hist = nullptr,
                   bool skip_filters = false, int level = -1,
                   bool prefetch_index_and_filter_in_cache = true,
-                   size_t max_file_size_for_l0_meta_pin = 0,
-                   Temperature file_temperature = Temperature::kUnknown);
+                   size_t max_file_size_for_l0_meta_pin = 0);

  // Get TableReader from a cache handle.
  TableReader* GetTableReaderFromHandle(Cache::Handle* handle);
@@ -207,8 +206,7 @@ class TableCache {
                        const SliceTransform* prefix_extractor = nullptr,
                        bool skip_filters = false, int level = -1,
                        bool prefetch_index_and_filter_in_cache = true,
-                        size_t max_file_size_for_l0_meta_pin = 0,
-                        Temperature file_temperature = Temperature::kUnknown);
+                        size_t max_file_size_for_l0_meta_pin = 0);

  // Create a key prefix for looking up the row cache. The prefix is of the
  // format row_cache_id + fd_number + seq_no. Later, the user key can be
@@ -42,7 +42,7 @@ class IntTblPropCollectorFactory {
  virtual ~IntTblPropCollectorFactory() {}
  // has to be thread-safe
  virtual IntTblPropCollector* CreateIntTblPropCollector(
-      uint32_t column_family_id, int level_at_creation) = 0;
+      uint32_t column_family_id) = 0;

  // The name of the properties collector can be used for debugging purpose.
  virtual const char* Name() const = 0;
@@ -92,10 +92,9 @@ class UserKeyTablePropertiesCollectorFactory
      std::shared_ptr<TablePropertiesCollectorFactory> user_collector_factory)
      : user_collector_factory_(user_collector_factory) {}
  virtual IntTblPropCollector* CreateIntTblPropCollector(
-      uint32_t column_family_id, int level_at_creation) override {
+      uint32_t column_family_id) override {
    TablePropertiesCollectorFactory::Context context;
    context.column_family_id = column_family_id;
-    context.level_at_creation = level_at_creation;
    return new UserKeyTablePropertiesCollector(
        user_collector_factory_->CreateTablePropertiesCollector(context));
  }
@@ -40,7 +40,6 @@ class TablePropertiesTest : public testing::Test,
 namespace {
 static const uint32_t kTestColumnFamilyId = 66;
 static const std::string kTestColumnFamilyName = "test_column_fam";
-static const int kTestLevel = 1;

 void MakeBuilder(
    const Options& options, const ImmutableOptions& ioptions,
@@ -52,10 +51,11 @@ void MakeBuilder(
  std::unique_ptr<FSWritableFile> wf(new test::StringSink);
  writable->reset(
      new WritableFileWriter(std::move(wf), "" /* don't care */, EnvOptions()));
+  int unknown_level = -1;
  TableBuilderOptions tboptions(
      ioptions, moptions, internal_comparator, int_tbl_prop_collector_factories,
      options.compression, options.compression_opts, kTestColumnFamilyId,
-      kTestColumnFamilyName, kTestLevel);
+      kTestColumnFamilyName, unknown_level);
  builder->reset(NewTableBuilder(tboptions, writable->get()));
 }
 }  // namespace
@@ -199,7 +199,6 @@ class RegularKeysStartWithAFactory : public IntTblPropCollectorFactory,
  TablePropertiesCollector* CreateTablePropertiesCollector(
      TablePropertiesCollectorFactory::Context context) override {
    EXPECT_EQ(kTestColumnFamilyId, context.column_family_id);
-    EXPECT_EQ(kTestLevel, context.level_at_creation);
    if (!backward_mode_) {
      return new RegularKeysStartWithA();
    } else {
@@ -207,7 +206,7 @@ class RegularKeysStartWithAFactory : public IntTblPropCollectorFactory,
    }
  }
  IntTblPropCollector* CreateIntTblPropCollector(
-      uint32_t /*column_family_id*/, int /* level_at_creation */) override {
+      uint32_t /*column_family_id*/) override {
    return new RegularKeysStartWithAInternal();
  }
  const char* Name() const override { return "RegularKeysStartWithA"; }
@@ -63,8 +63,8 @@ Status TransactionLogIteratorImpl::OpenLogFile(
    }
  }
  if (s.ok()) {
-    file_reader->reset(new SequentialFileReader(
-        std::move(file), fname, io_tracer_, options_->listeners));
+    file_reader->reset(
+        new SequentialFileReader(std::move(file), fname, io_tracer_));
  }
  return s;
 }
Author	SHA1	Message	Date
Peter Dillinger	f55aab3d18	Fix EnvLibrados and add to CI (#9088 ) Summary: This feature was not part of any common or CI build, so no surprise it broke. Now we can at least ensure compilation. I don't know how to run the test successfully (missing config file) so it is bypassed for now. Fixes https://github.com/facebook/rocksdb/issues/9078 Pull Request resolved: https://github.com/facebook/rocksdb/pull/9088 Test Plan: CI Reviewed By: mrambacher Differential Revision: D32009467 Pulled By: pdillinger fbshipit-source-id: 3e0d1e5fde7f0ece703d48a81479e1cc7392c25c	2021-10-29 09:21:24 -07:00
Andrew Kryczka	0103296f39	update HISTORY.md and version.h for 6.25.3	2021-10-14 10:46:37 -07:00
Andrew Kryczka	fa4e0558bf	Fix sequence number bump logic in multi-CF SST ingestion (#9005 ) Summary: The code in `IngestExternalFiles()` that bumps the DB's sequence number depending on what seqnos were assigned to the files has 3 bugs: 1) There is an assertion that the sequence number is increased in all the affected column families, but this is unnecessary, it is fine if some files can stick to a lower sequence number. It is very easy to hit the assertion: it is sufficient to insert 2 files in 2 CFs, one which overlaps the CF and one that doesn't (for example the CF is empty). The line added in the `IngestFilesIntoMultipleColumnFamilies_Success` test makes the assertion fail. 2) SetLastSequence() is called with the sum of all the bumps across CFs, but we should take the maximum instead, as all CFs start with the current seqno and bump it independently. 3) The code above is accidentally under a `#ifndef NDEBUG`, so it doesn't run in optimized builds, so some files may be assigned seqnos from the future. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9005 Test Plan: Added line in `IngestFilesIntoMultipleColumnFamilies_Success` that triggers the assertion, verified that the test (and all the others) pass after the fix. Reviewed By: ajkr Differential Revision: D31597892 Pulled By: ot fbshipit-source-id: c2d3237f90290df1178736ace8653a9623f5a770	2021-10-14 10:45:19 -07:00
Giuseppe Ottaviano	3ebe8658d0	Fix race in WriteBufferManager (#9009 ) Summary: EndWriteStall has a data race: `queue_.empty()` is checked outside of the mutex, so once we enter the critical section another thread may already have cleared the list, and accessing the `front()` is undefined behavior (and causes interesting crashes under high concurrency). This PR fixes the bug, and also rewrites the logic to make it easier to reason about it. It also fixes another subtle bug: if some writers are stalled and `SetBufferSize(0)` is called, which disables the WBM, the writer are not unblocked because of an early `enabled()` check in `EndWriteStall()`. It doesn't significantly change the locking behavior, as before writers won't lock unless entering a stall condition, and `FreeMem` almost always locks if stalling is allowed, but that is inevitable with the current design. Liveness is guaranteed by the fact that if some writes are blocked, eventually all writes will be blocked due to `stall_active_`, and eventually all memory is freed. While at it, do a couple of optimizations: - In `WBMStallInterface::Signal()` signal the CV only after releasing the lock. Signaling under the lock is a common pitfall, as it causes the woken-up thread to immediately go back to sleep because the mutex is still locked by the awaker. - Move all allocations and deallocations outside of the lock. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9009 Test Plan: ``` USE_CLANG=1 make -j64 all check ``` Reviewed By: akankshamahajan15 Differential Revision: D31550668 Pulled By: ot fbshipit-source-id: 5125387c3dc7ecaaa2b8bbc736e58c4156698580	2021-10-14 10:44:46 -07:00
Andrew Kryczka	430fd40e87	update HISTORY.md and version.h for 6.25.2	2021-10-11 16:42:46 -07:00
Andrew Kryczka	2df8905531	Protect existing files in `FaultInjectionTest{Env,FS}::ReopenWritableFile()` (#8995 ) Summary: `FaultInjectionTest{Env,FS}::ReopenWritableFile()` functions were accidentally deleting WALs from previous `db_stress` runs causing verification to fail. They were operating under the assumption that `ReopenWritableFile()` would delete any existing file. It was a reasonable assumption considering the `{Env,FileSystem}::ReopenWritableFile()` documentation stated that would happen. The only problem was neither the implementations we offer nor the "real" clients in RocksDB code followed that contract. So, this PR updates the contract as well as fixing the fault injection client usage. The fault injection change exposed that `ExternalSSTFileBasicTest.SyncFailure` was relying on a fault injection `Env` dropping unsynced data written by a regular `Env`. I changed that test to make its `SstFileWriter` use fault injection `Env`, and also implemented `LinkFile()` in fault injection so the unsynced data is tracked under the new name. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8995 Test Plan: - Verified it fixes the following failure: ``` $ ./db_stress --clear_column_family_one_in=0 --column_families=1 --db=/dev/shm/rocksdb_crashtest_whitebox --delpercent=5 --expected_values_dir=/dev/shm/rocksdb_crashtest_expected --iterpercent=0 --key_len_percent_dist=1,30,69 --max_key=100000 --max_key_len=3 --nooverwritepercent=1 --ops_per_thread=1000 --prefixpercent=0 --readpercent=60 --reopen=0 --target_file_size_base=1048576 --test_batches_snapshots=0 --write_buffer_size=1048576 --writepercent=35 --value_size_mult=33 -threads=1 ... $ ./db_stress --avoid_flush_during_recovery=1 --clear_column_family_one_in=0 --column_families=1 --db=/dev/shm/rocksdb_crashtest_whitebox --delpercent=5 --destroy_db_initially=0 --expected_values_dir=/dev/shm/rocksdb_crashtest_expected --iterpercent=10 --key_len_percent_dist=1,30,69 --max_bytes_for_level_base=4194304 --max_key=100000 --max_key_len=3 --nooverwritepercent=1 --open_files=-1 --open_metadata_write_fault_one_in=8 --open_write_fault_one_in=16 --ops_per_thread=1000 --prefix_size=-1 --prefixpercent=0 --readpercent=50 --sync=1 --target_file_size_base=1048576 --test_batches_snapshots=0 --write_buffer_size=1048576 --writepercent=35 --value_size_mult=33 -threads=1 ... Verification failed for column family 0 key 000000000000001300000000000000857878787878 (1143): Value not found: NotFound: Crash-recovery verification failed :( ... ``` - `make check -j48` Reviewed By: ltamasi Differential Revision: D31495388 Pulled By: ajkr fbshipit-source-id: 7886ccb6a07cb8b78ad7b6c1c341ccf40bb68385	2021-10-11 16:39:36 -07:00
Hui Xiao	ffd4e9675e	Update HISTORY.md for #8428 (#9001 ) Summary: Context: HISTORY.md was not properly updated along with the change in https://github.com/facebook/rocksdb/pull/8428, where we introduced a change of accounting compression dictionary buffering memory and an extra condition of triggering data unbuffering. Updated HISTORY.md for https://github.com/facebook/rocksdb/pull/8428 in 6.25.0 HISTORY.md section. Updated blog post https://rocksdb.org/blog/2021/05/31/dictionary-compression.html. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9001 Reviewed By: ajkr Differential Revision: D31517836 Pulled By: hx235 fbshipit-source-id: 01f6b30de4e1ff6b315aa8221139d9b700c7c629	2021-10-11 16:38:13 -07:00
Andrew Kryczka	ab2aceb4f5	Cancel manual compactions waiting on automatic compactions to drain (#8991 ) Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8991 Test Plan: the new test hangs forever without this fix and passes with this fix. Reviewed By: hx235 Differential Revision: D31456419 Pulled By: ajkr fbshipit-source-id: a82c0e5560b6e6153089dccd8e46163c61b07bff	2021-10-11 16:36:54 -07:00
Yanqin Jin	307a65525a	Bump version	2021-09-28 13:24:51 -07:00
Yanqin Jin	9e15f7bff3	Sort per-file blob read requests by offset (#8953 ) Summary: `RandomAccessFileReader::MultiRead()` tries to merge requests in direct IO, assuming input IO requests are sorted by offsets. Add a test in direct IO mode. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8953 Test Plan: make check Reviewed By: ltamasi Differential Revision: D31183546 Pulled By: riversand963 fbshipit-source-id: 5d043ec68e2daa47a3149066150afd41ee3d73e6	2021-09-28 13:24:17 -07:00
Hui Xiao	14e3f8cd9b	Return Status::NotSupported() in RateLimiter::GetTotalPendingRequests default impl (#8950 ) Summary: Context: After more discussion, a fix in https://github.com/facebook/rocksdb/issues/8938 might turn out to be too restrictive for the case where `GetTotalPendingRequests` might be invoked on RateLimiter classes that does not support the recently added API `RateLimiter::GetTotalPendingRequests` (https://github.com/facebook/rocksdb/issues/8890) due to the `assert(false)` in https://github.com/facebook/rocksdb/issues/8938. Furthermore, sentinel value like `-1` proposed in https://github.com/facebook/rocksdb/issues/8938 is easy to be ignored and unchecked. Therefore we decided to adopt `Status::NotSupported()`, which is also a convention of adding new API to public header in RocksDB. - Changed return value type of `RateLimiter::GetTotalPendingRequests` in related declaration/definition - Passed in pointer argument to hold the output instead of returning it as before - Adapted to the changes above in calling `RateLimiter::GetTotalPendingRequests` in test - Minor improvement to `TEST_F(RateLimiterTest, GetTotalPendingRequests)`: added failure message for assertion and replaced repetitive statements with a loop Pull Request resolved: https://github.com/facebook/rocksdb/pull/8950 Reviewed By: ajkr, pdillinger Differential Revision: D31128450 Pulled By: hx235 fbshipit-source-id: 282ac9c4f3dacaa0aec6d0a993161f77ad47a040	2021-09-22 21:41:04 -07:00
sdong	0fb79b0aca	Add HISTORY.md entry to a recent bug fix. (#8948 ) Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/8948 Reviewed By: anand1976 Differential Revision: D31127368 fbshipit-source-id: a374cb0baf88c3e15cd587a8f31e8a2d84432928	2021-09-22 16:25:45 -07:00
sdong	ff033921d8	RandomAccessFileReader::MultiRead() should not return read bytes not read (#8941 ) Summary: Right now, if underlying read returns fewer bytes than asked for, RandomAccessFileReader::MultiRead() still returns those in the buffer to upper layer. This can be a surprise to upper layer. This is unlikely to cause incorrect data. To cause incorrect data, checksum checking in upper layer should pass with short reads, whose chance is low. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8941 Test Plan: Run stress tests for a while Reviewed By: anand1976 Differential Revision: D31085780 fbshipit-source-id: 999adf2d6c2712f1323d14bb68b678df59969973	2021-09-22 15:26:10 -07:00
Peter Dillinger	33f11b2625	Clean up HISTORY	2021-09-21 21:59:24 -07:00
Hui Xiao	c92f7a29aa	Make RateLimiter::GetTotalPendingRequest() non pure virtual for backward compability (#8938 ) Summary: Context/Summary: https://github.com/facebook/rocksdb/pull/8890 added a public API `RateLimiter::GetTotalPendingRequest()` but mistakenly marked it as pure virtual, forcing RateLimiter's derived classes to implement this function and breaking backward compatibility. This PR makes `RateLimiter::GetTotalPendingRequest()` as non-pure virtual method by providing a trivial implementation in rate_limiter.h Pull Request resolved: https://github.com/facebook/rocksdb/pull/8938 Test Plan: Passing existing tests Reviewed By: pdillinger Differential Revision: D31100661 Pulled By: hx235 fbshipit-source-id: 06eff1005156a6e5a881e393b2c5b2ad706897d8	2021-09-21 21:48:55 -07:00
Peter Dillinger	e74dfee7fc	Finish BackupEngine migration to IOStatus (#8940 ) Summary: Updates a few remaining functions that should have been updated from Status -> IOStatus, and adds to HISTORY for the overall change including https://github.com/facebook/rocksdb/issues/8820. This change is for inclusion in version 6.25. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8940 Test Plan: CI Reviewed By: zhichao-cao Differential Revision: D31085029 Pulled By: pdillinger fbshipit-source-id: 91557c6a39ef1d90357d4f4dcd79af0645d87c7b	2021-09-21 21:48:34 -07:00