Update HISTORY.md and version.h for 9.2.2 patch version

Summary:
A `BlockBasedTable` with `TieredSecondaryCache` containing a NVM cache inserts blocks  into the compressed cache and the corresponding compressed block into the NVM cache.  The `BlockFetcher` is used to get the uncompressed and compressed blocks by calling `ReadBlockContents()` and `GetUncompressedBlock()` respectively. If the file system supports FSBuffer (i.e returning a FS allocated buffer rather than caller provided), that buffer gets freed between the two calls. This PR fixes it by making the FSBuffer unique pointer a member rather than local variable.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12712

Test Plan:
1. Add a unit test
2. Release validation stress test

Reviewed By: jaykorean

Differential Revision: D57974026

Pulled By: anand1976

fbshipit-source-id: cfa895914e74b4f628413b40e6e39d8d8e5286bd

HISTORY.md

@@ -1,35 +1,14 @@
 # Rocksdb Change Log
 > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
 
-## 9.3.1 (05/25/2024)
+## 9.2.2 (05/31/2024)
 ### Bug Fixes
-* [internal only] Build script improvement
-
-## 9.3.0 (05/17/2024)
-### New Features
-* Optimistic transactions and pessimistic transactions with the WriteCommitted policy now support the `GetEntity` API.
-* Added new `Iterator` property, "rocksdb.iterator.is-value-pinned", for checking whether the `Slice` returned by `Iterator::value()` can be used until the `Iterator` is destroyed.
-* Optimistic transactions and WriteCommitted pessimistic transactions now support the `MultiGetEntity` API.
-* Optimistic transactions and pessimistic transactions with the WriteCommitted policy now support the `PutEntity` API. Support for read APIs and other write policies (WritePrepared, WriteUnprepared) will be added later.
-
-### Public API Changes
-* Exposed block based metadata cache options via C API
-* Exposed compaction pri via c api.
-* Add a kAdmPolicyAllowAll option to TieredAdmissionPolicy that admits all blocks evicted from the primary block cache into the compressed secondary cache.
-
-### Behavior Changes
-* CompactRange() with change_level=true on a CF with FIFO compaction will return Status::NotSupported().
-* External file ingestion with FIFO compaction will always ingest to L0.
-
-### Bug Fixes
-* Fixed a bug for databases using `DBOptions::allow_2pc == true` (all `TransactionDB`s except `OptimisticTransactionDB`) that have exactly one column family. Due to a missing WAL sync, attempting to open the DB could have returned a `Status::Corruption` with a message like "SST file is ahead of WALs".
-* Fix a bug in CreateColumnFamilyWithImport() where if multiple CFs are imported, we were not resetting files' epoch number and L0 files can have overlapping key range but the same epoch number.
-* Fixed race conditions when `ColumnFamilyOptions::inplace_update_support == true` between user overwrites and reads on the same key.
-* Fix a bug where `CompactFiles()` can compact files of range conflict with other ongoing compactions' when `preclude_last_level_data_seconds > 0` is used
-* Fixed a false positive `Status::Corruption` reported when reopening a DB that used `DBOptions::recycle_log_file_num > 0` and `DBOptions::wal_compression != kNoCompression`.
-* While WAL is locked with LockWAL(), some operations like Flush() and IngestExternalFile() are now blocked as they should have been.
 * Fixed a bug causing stale memory access when using the TieredSecondaryCache with an NVM secondary cache, and a file system that supports return an FS allocated buffer for MultiRead (FSSupportedOps::kFSBuffer is set).
 
+## 9.2.1 (05/03/2024)
+### Public API Changes
+* Add a kAdmPolicyAllowAll option to TieredAdmissionPolicy that admits all blocks evicted from the primary block cache into the compressed secondary cache.
+
 ## 9.2.0 (05/01/2024)
 ### New Features
 * Added two options `deadline` and `max_size_bytes` for CacheDumper to exit early

TARGETS

@@ -4,7 +4,6 @@
 # This file is a Facebook-specific integration for buck builds, so can
 # only be validated by Facebook employees.
 load("//rocks/buckifier:defs.bzl", "cpp_library_wrapper","rocks_cpp_library_wrapper","cpp_binary_wrapper","cpp_unittest_wrapper","fancy_bench_wrapper","add_c_test_wrapper")
-load("@fbcode_macros//build_defs:export_files.bzl", "export_file")
 
 
 cpp_library_wrapper(name="rocksdb_lib", srcs=[

buckifier/targets_cfg.py

@@ -7,7 +7,6 @@ rocksdb_target_header_template = """# This file \100generated by:
 # This file is a Facebook-specific integration for buck builds, so can
 # only be validated by Facebook employees.
 load("//rocks/buckifier:defs.bzl", "cpp_library_wrapper","rocks_cpp_library_wrapper","cpp_binary_wrapper","cpp_unittest_wrapper","fancy_bench_wrapper","add_c_test_wrapper")
-load("@fbcode_macros//build_defs:export_files.bzl", "export_file")
 
 """
 

db/c.cc

@@ -2745,24 +2745,6 @@ void rocksdb_options_set_block_based_table_factory(
   }
 }
 
-void rocksdb_block_based_options_set_top_level_index_pinning_tier(
-    rocksdb_block_based_table_options_t* options, int v) {
-  options->rep.metadata_cache_options.top_level_index_pinning =
-      static_cast<ROCKSDB_NAMESPACE::PinningTier>(v);
-}
-
-void rocksdb_block_based_options_set_partition_pinning_tier(
-    rocksdb_block_based_table_options_t* options, int v) {
-  options->rep.metadata_cache_options.partition_pinning =
-      static_cast<ROCKSDB_NAMESPACE::PinningTier>(v);
-}
-
-void rocksdb_block_based_options_set_unpartitioned_pinning_tier(
-    rocksdb_block_based_table_options_t* options, int v) {
-  options->rep.metadata_cache_options.unpartitioned_pinning =
-      static_cast<ROCKSDB_NAMESPACE::PinningTier>(v);
-}
-
 rocksdb_cuckoo_table_options_t* rocksdb_cuckoo_options_create() {
   return new rocksdb_cuckoo_table_options_t;
 }
@@ -3948,14 +3930,6 @@ void rocksdb_options_set_fifo_compaction_options(
   opt->rep.compaction_options_fifo = fifo->rep;
 }
 
-void rocksdb_options_set_compaction_pri(rocksdb_options_t* opt, int pri) {
-  opt->rep.compaction_pri = static_cast<ROCKSDB_NAMESPACE::CompactionPri>(pri);
-}
-
-int rocksdb_options_get_compaction_pri(rocksdb_options_t* opt) {
-  return opt->rep.compaction_pri;
-}
-
 char* rocksdb_options_statistics_get_string(rocksdb_options_t* opt) {
   ROCKSDB_NAMESPACE::Statistics* statistics = opt->rep.statistics.get();
   if (statistics) {

db/c_test.c

@@ -704,10 +704,6 @@ int main(int argc, char** argv) {
   rocksdb_block_based_options_set_block_cache(table_options, cache);
   rocksdb_block_based_options_set_data_block_index_type(table_options, 1);
   rocksdb_block_based_options_set_data_block_hash_ratio(table_options, 0.75);
-  rocksdb_block_based_options_set_top_level_index_pinning_tier(table_options,
-                                                               1);
-  rocksdb_block_based_options_set_partition_pinning_tier(table_options, 2);
-  rocksdb_block_based_options_set_unpartitioned_pinning_tier(table_options, 3);
   rocksdb_options_set_block_based_table_factory(options, table_options);
 
   rocksdb_options_set_compression(options, rocksdb_no_compression);
@@ -2045,6 +2041,7 @@ int main(int argc, char** argv) {
     rocksdb_options_set_advise_random_on_open(o, 1);
     CheckCondition(1 == rocksdb_options_get_advise_random_on_open(o));
 
+
     rocksdb_options_set_use_adaptive_mutex(o, 1);
     CheckCondition(1 == rocksdb_options_get_use_adaptive_mutex(o));
 
@@ -2115,9 +2112,6 @@ int main(int argc, char** argv) {
     rocksdb_options_set_compaction_style(o, 2);
     CheckCondition(2 == rocksdb_options_get_compaction_style(o));
 
-    rocksdb_options_set_compaction_pri(o, 4);
-    CheckCondition(4 == rocksdb_options_get_compaction_pri(o));
-
     rocksdb_options_set_atomic_flush(o, 1);
     CheckCondition(1 == rocksdb_options_get_atomic_flush(o));
 
@@ -2622,10 +2616,6 @@ int main(int argc, char** argv) {
     CheckCondition(1 == rocksdb_options_get_compaction_style(copy));
     CheckCondition(2 == rocksdb_options_get_compaction_style(o));
 
-    rocksdb_options_set_compaction_pri(copy, 1);
-    CheckCondition(1 == rocksdb_options_get_compaction_pri(copy));
-    CheckCondition(4 == rocksdb_options_get_compaction_pri(o));
-
     rocksdb_options_set_atomic_flush(copy, 0);
     CheckCondition(0 == rocksdb_options_get_atomic_flush(copy));
     CheckCondition(1 == rocksdb_options_get_atomic_flush(o));

db/compaction/compaction_picker.cc

@@ -340,8 +340,8 @@ Compaction* CompactionPicker::CompactFiles(
 #ifndef NDEBUG
   assert(input_files.size());
   // This compaction output should not overlap with a running compaction as
-  // `SanitizeAndConvertCompactionInputFiles` should've checked earlier and db
-  // mutex shouldn't have been released since.
+  // `SanitizeCompactionInputFiles` should've checked earlier and db mutex
+  // shouldn't have been released since.
   int start_level = Compaction::kInvalidLevel;
   for (const auto& in : input_files) {
     // input_files should already be sorted by level
@@ -1040,18 +1040,19 @@ Status CompactionPicker::SanitizeCompactionInputFilesForAllLevels(
       }
     }
   }
+  if (RangeOverlapWithCompaction(smallestkey, largestkey, output_level)) {
+    return Status::Aborted(
+        "A running compaction is writing to the same output level in an "
+        "overlapping key range");
+  }
   return Status::OK();
 }
 
-Status CompactionPicker::SanitizeAndConvertCompactionInputFiles(
+Status CompactionPicker::SanitizeCompactionInputFiles(
     std::unordered_set<uint64_t>* input_files,
-    const ColumnFamilyMetaData& cf_meta, const int output_level,
-    const VersionStorageInfo* vstorage,
-    std::vector<CompactionInputFiles>* converted_input_files) const {
+    const ColumnFamilyMetaData& cf_meta, const int output_level) const {
   assert(static_cast<int>(cf_meta.levels.size()) - 1 ==
          cf_meta.levels[cf_meta.levels.size() - 1].level);
-  assert(converted_input_files);
-
   if (output_level >= static_cast<int>(cf_meta.levels.size())) {
     return Status::InvalidArgument(
         "Output level for column family " + cf_meta.name +
@@ -1077,6 +1078,7 @@ Status CompactionPicker::SanitizeAndConvertCompactionInputFiles(
 
   Status s = SanitizeCompactionInputFilesForAllLevels(input_files, cf_meta,
                                                       output_level);
+
   if (!s.ok()) {
     return s;
   }
@@ -1117,22 +1119,6 @@ Status CompactionPicker::SanitizeAndConvertCompactionInputFiles(
     }
   }
 
-  s = GetCompactionInputsFromFileNumbers(converted_input_files, input_files,
-                                         vstorage, CompactionOptions());
-  if (!s.ok()) {
-    return s;
-  }
-  assert(converted_input_files->size() > 0);
-  if (output_level != 0 &&
-      FilesRangeOverlapWithCompaction(
-          *converted_input_files, output_level,
-          Compaction::EvaluatePenultimateLevel(
-              vstorage, ioptions_, (*converted_input_files)[0].level,
-              output_level))) {
-    return Status::Aborted(
-        "A running compaction is writing to the same output level(s) in an "
-        "overlapping key range");
-  }
   return Status::OK();
 }
 

db/compaction/compaction_picker.h

@@ -86,20 +86,16 @@ class CompactionPicker {
 
   virtual bool NeedsCompaction(const VersionStorageInfo* vstorage) const = 0;
 
-  // Sanitize the input set of compaction input files and convert it to
-  // `std::vector<CompactionInputFiles>` in the output parameter
-  // `converted_input_files`.
-  // When the input parameters do not describe a valid
-  // compaction, the function will try to fix the input_files by adding
-  // necessary files.  If it's not possible to convert an invalid input_files
-  // into a valid one by adding more files, the function will return a
-  // non-ok status with specific reason.
-  //
-  Status SanitizeAndConvertCompactionInputFiles(
-      std::unordered_set<uint64_t>* input_files,
-      const ColumnFamilyMetaData& cf_meta, const int output_level,
-      const VersionStorageInfo* vstorage,
-      std::vector<CompactionInputFiles>* converted_input_files) const;
+// Sanitize the input set of compaction input files.
+// When the input parameters do not describe a valid compaction, the
+// function will try to fix the input_files by adding necessary
+// files.  If it's not possible to conver an invalid input_files
+// into a valid one by adding more files, the function will return a
+// non-ok status with specific reason.
+//
+  Status SanitizeCompactionInputFiles(std::unordered_set<uint64_t>* input_files,
+                                      const ColumnFamilyMetaData& cf_meta,
+                                      const int output_level) const;
 
   // Free up the files that participated in a compaction
   //
@@ -113,8 +109,8 @@ class CompactionPicker {
   // object.
   //
   // Caller must provide a set of input files that has been passed through
-  // `SanitizeAndConvertCompactionInputFiles` earlier. The lock should not be
-  // released between that call and this one.
+  // `SanitizeCompactionInputFiles` earlier. The lock should not be released
+  // between that call and this one.
   Compaction* CompactFiles(const CompactionOptions& compact_options,
                            const std::vector<CompactionInputFiles>& input_files,
                            int output_level, VersionStorageInfo* vstorage,
@@ -124,7 +120,6 @@ class CompactionPicker {
 
   // Converts a set of compaction input file numbers into
   // a list of CompactionInputFiles.
-  // TODO(hx235): remove the unused paramter `compact_options`
   Status GetCompactionInputsFromFileNumbers(
       std::vector<CompactionInputFiles>* input_files,
       std::unordered_set<uint64_t>* input_set,
@@ -230,8 +225,8 @@ class CompactionPicker {
  protected:
   const ImmutableOptions& ioptions_;
 
-  // A helper function to SanitizeAndConvertCompactionInputFiles() that
-  // sanitizes "input_files" by adding necessary files.
+// A helper function to SanitizeCompactionInputFiles() that
+// sanitizes "input_files" by adding necessary files.
   virtual Status SanitizeCompactionInputFilesForAllLevels(
       std::unordered_set<uint64_t>* input_files,
       const ColumnFamilyMetaData& cf_meta, const int output_level) const;

db/db_compaction_test.cc

@@ -16,7 +16,6 @@
 #include "env/mock_env.h"
 #include "port/port.h"
 #include "port/stack_trace.h"
-#include "rocksdb/advanced_options.h"
 #include "rocksdb/concurrent_task_limiter.h"
 #include "rocksdb/experimental.h"
 #include "rocksdb/sst_file_writer.h"
@@ -6205,14 +6204,29 @@ TEST_F(DBCompactionTest, CompactionLimiter) {
               NumTableFilesAtLevel(0, 0));
   }
 
-  // Wait until all CFs are pending compaction. WaitForFlushMemtable() can
-  // return before the next compaction is scheduled, so we need to do some
-  // waiting here.
+  // All CFs are pending compaction
   unsigned int tp_len = env_->GetThreadPoolQueueLen(Env::LOW);
-  for (int i = 0; i < 10000 && tp_len < cf_count; i++) {
-    env_->SleepForMicroseconds(1000);
-    tp_len = env_->GetThreadPoolQueueLen(Env::LOW);
+  if (cf_count != tp_len) {
+    // The test is flaky and fails the assertion below.
+    // Print some debug information.
+    uint64_t num_running_flushes = 0;
+    if (db_->GetIntProperty(DB::Properties::kNumRunningFlushes,
+                            &num_running_flushes)) {
+      fprintf(stdout, "Running flushes: %" PRIu64 "\n", num_running_flushes);
+    }
+    fprintf(stdout,
+            "%zu CF in compaction queue: ", pending_compaction_cfs.size());
+    for (const auto& cf_name : pending_compaction_cfs) {
+      fprintf(stdout, "%s, ", cf_name.c_str());
+    }
+    fprintf(stdout, "\n");
+
+    // print lsm
+    for (unsigned int cf = 0; cf < cf_count; cf++) {
+      fprintf(stdout, "%s: %s\n", cf_names[cf], FilesPerLevel(cf).c_str());
+    }
   }
+
   ASSERT_EQ(cf_count, tp_len);
 
   // Unblock all compaction threads
@@ -7121,8 +7135,7 @@ class DBCompactionTestWithOngoingFileIngestionParam
       SyncPoint::GetInstance()->SetCallBack(
           "ExternalSstFileIngestionJob::Run", [&](void*) {
             SyncPoint::GetInstance()->LoadDependency(
-                {{"DBImpl::CompactFilesImpl::"
-                  "PostSanitizeAndConvertCompactionInputFiles",
+                {{"DBImpl::CompactFilesImpl::PostSanitizeCompactionInputFiles",
                   "VersionSet::LogAndApply:WriteManifest"}});
           });
     } else {
@@ -7604,72 +7617,6 @@ class DBCompactionTestL0FilesMisorderCorruption : public DBCompactionTest {
   std::shared_ptr<test::SleepingBackgroundTask> sleeping_task_;
 };
 
-TEST_F(DBCompactionTest, CompactFilesSupportKeyPlacementRangeConflict) {
-  Options options;
-  options.create_if_missing = true;
-  options.compaction_style = kCompactionStyleUniversal;
-  options.num_levels = 3;
-  DestroyAndReopen(options);
-
-  // To create LSM of below shape:
-  // L0: [k2]
-  // L1: [k3],[k4]
-  // L2: [k1,  k5]
-  ASSERT_OK(Put("k1", "v"));
-  ASSERT_OK(Put("k5", "v"));
-  ASSERT_OK(Flush());
-  CompactRangeOptions cro;
-  cro.bottommost_level_compaction = BottommostLevelCompaction::kForceOptimized;
-  ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr));
-  ASSERT_EQ("0,0,1", FilesPerLevel());
-
-  ASSERT_OK(Put("k3", "v"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k4", "v"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(experimental::PromoteL0(db_, db_->DefaultColumnFamily(), 1));
-  ASSERT_EQ("0,2,1", FilesPerLevel());
-
-  ASSERT_OK(Put("k2", "v"));
-  ASSERT_OK(Flush());
-  ASSERT_EQ("1,2,1", FilesPerLevel());
-
-  Close();
-
-  // To force below two CompactFiles() in order to coerce range conflict on L1
-  // upon (2)
-  // (1): Compact [k2] at L0 and [k3] at L1 with output to L1
-  // (2): Compact [k4] at L1 and [k1, k5] at L2 and output to L1 and L2
-  options.preclude_last_level_data_seconds = 1;
-  Reopen(options);
-
-  ColumnFamilyMetaData cf_meta_data;
-  db_->GetColumnFamilyMetaData(&cf_meta_data);
-
-  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->SetCallBack(
-      "CompactFilesImpl:0", [&](void* /*arg*/) {
-        std::vector<std::string> c2_input_files;
-        c2_input_files.push_back(cf_meta_data.levels[1].files[1].name);
-        c2_input_files.push_back(cf_meta_data.levels[2].files[0].name);
-        // To verify CompactFiles() is aborted upon range conflict instead
-        // of crashing upon internal assertion
-        Status s = db_->CompactFiles(CompactionOptions(), c2_input_files,
-                                     2 /* output_level */);
-        ASSERT_TRUE(s.IsAborted());
-      });
-  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();
-
-  std::vector<std::string> c1_input_files;
-  c1_input_files.push_back(cf_meta_data.levels[0].files[0].name);
-  c1_input_files.push_back(cf_meta_data.levels[1].files[0].name);
-  Status s = db_->CompactFiles(CompactionOptions(), c1_input_files,
-                               1 /* output_level */);
-  ASSERT_OK(s);
-
-  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
-  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
-}
-
 TEST_F(DBCompactionTestL0FilesMisorderCorruption,
        FlushAfterIntraL0LevelCompactionWithIngestedFile) {
   SetupOptions(CompactionStyle::kCompactionStyleLevel, "");

db/db_filesnapshot.cc

@@ -229,18 +229,12 @@ Status DBImpl::GetLiveFilesStorageInfo(
   // metadata.
   mutex_.Lock();
   if (flush_memtable) {
-    bool wal_locked = lock_wal_count_ > 0;
-    if (wal_locked) {
-      ROCKS_LOG_INFO(immutable_db_options_.info_log,
-                     "Can't FlushForGetLiveFiles while WAL is locked");
-    } else {
-      Status status = FlushForGetLiveFiles();
-      if (!status.ok()) {
-        mutex_.Unlock();
-        ROCKS_LOG_ERROR(immutable_db_options_.info_log,
-                        "Cannot Flush data %s\n", status.ToString().c_str());
-        return status;
-      }
+    Status status = FlushForGetLiveFiles();
+    if (!status.ok()) {
+      mutex_.Unlock();
+      ROCKS_LOG_ERROR(immutable_db_options_.info_log, "Cannot Flush data %s\n",
+                      status.ToString().c_str());
+      return status;
     }
   }
 

db/db_follower_test.cc

@@ -17,7 +17,6 @@ class DBFollowerTest : public DBTestBase {
   // Create the leader DB object
   DBFollowerTest() : DBTestBase("/db_follower_test", /*env_do_fsync*/ false) {
     follower_name_ = dbname_ + "/follower";
-    db_parent_ = dbname_;
     Close();
     Destroy(CurrentOptions());
     EXPECT_EQ(env_->CreateDirIfMissing(dbname_), Status::OK());
@@ -28,215 +27,17 @@ class DBFollowerTest : public DBTestBase {
   ~DBFollowerTest() {
     follower_.reset();
     EXPECT_EQ(DestroyDB(follower_name_, CurrentOptions()), Status::OK());
-    Destroy(CurrentOptions());
-    dbname_ = db_parent_;
   }
 
  protected:
-  class DBFollowerTestFS : public FileSystemWrapper {
-   public:
-    explicit DBFollowerTestFS(const std::shared_ptr<FileSystem>& target)
-        : FileSystemWrapper(target),
-          cv_(&mutex_),
-          barrier_(false),
-          count_(0),
-          reinit_count_(0) {}
-
-    const char* Name() const override { return "DBFollowerTestFS"; }
-
-    IOStatus NewSequentialFile(const std::string& fname,
-                               const FileOptions& file_opts,
-                               std::unique_ptr<FSSequentialFile>* result,
-                               IODebugContext* dbg = nullptr) override {
-      class DBFollowerTestSeqFile : public FSSequentialFileWrapper {
-       public:
-        DBFollowerTestSeqFile(DBFollowerTestFS* fs,
-                              std::unique_ptr<FSSequentialFile>&& file,
-                              uint64_t /*size*/)
-            : FSSequentialFileWrapper(file.get()),
-              fs_(fs),
-              file_(std::move(file)) {}
-
-        IOStatus Read(size_t n, const IOOptions& options, Slice* result,
-                      char* scratch, IODebugContext* dbg) override {
-          fs_->BarrierWait();
-          return target()->Read(n, options, result, scratch, dbg);
-        }
-
-       private:
-        DBFollowerTestFS* fs_;
-        std::unique_ptr<FSSequentialFile> file_;
-      };
-
-      std::unique_ptr<FSSequentialFile> file;
-      IOStatus s = target()->NewSequentialFile(fname, file_opts, &file, dbg);
-
-      if (s.ok() && test::GetFileType(fname) == kDescriptorFile) {
-        uint64_t size = 0;
-        EXPECT_EQ(target()->GetFileSize(fname, IOOptions(), &size, nullptr),
-                  IOStatus::OK());
-        result->reset(new DBFollowerTestSeqFile(this, std::move(file), size));
-      } else {
-        *result = std::move(file);
-      }
-      return s;
-    }
-
-    void BarrierInit(int count) {
-      MutexLock l(&mutex_);
-      barrier_ = true;
-      count_ = count;
-    }
-
-    void BarrierWait() {
-      MutexLock l(&mutex_);
-      if (!barrier_) {
-        return;
-      }
-      if (--count_ == 0) {
-        if (reinit_count_ > 0) {
-          count_ = reinit_count_;
-          reinit_count_ = 0;
-        } else {
-          barrier_ = false;
-        }
-        cv_.SignalAll();
-      } else {
-        cv_.Wait();
-      }
-    }
-
-    void BarrierWaitAndReinit(int count) {
-      MutexLock l(&mutex_);
-      if (!barrier_) {
-        return;
-      }
-      reinit_count_ = count;
-      if (--count_ == 0) {
-        if (reinit_count_ > 0) {
-          count_ = reinit_count_;
-          reinit_count_ = 0;
-        } else {
-          barrier_ = false;
-        }
-        cv_.SignalAll();
-      } else {
-        cv_.Wait();
-      }
-    }
-
-   private:
-    port::Mutex mutex_;
-    port::CondVar cv_;
-    bool barrier_;
-    int count_;
-    int reinit_count_;
-  };
-
-  class DBFollowerTestSstPartitioner : public SstPartitioner {
-   public:
-    explicit DBFollowerTestSstPartitioner(uint64_t max_keys)
-        : max_keys_(max_keys), num_keys_(0) {}
-
-    const char* Name() const override { return "DBFollowerTestSstPartitioner"; }
-
-    PartitionerResult ShouldPartition(
-        const PartitionerRequest& /*request*/) override {
-      if (++num_keys_ > max_keys_) {
-        num_keys_ = 0;
-        return PartitionerResult::kRequired;
-      } else {
-        return PartitionerResult::kNotRequired;
-      }
-    }
-
-    bool CanDoTrivialMove(const Slice& /*smallest_user_key*/,
-                          const Slice& /*largest_user_key*/) override {
-      return true;
-    }
-
-   private:
-    uint64_t max_keys_;
-    uint64_t num_keys_;
-  };
-
-  class DBFollowerTestSstPartitionerFactory : public SstPartitionerFactory {
-   public:
-    explicit DBFollowerTestSstPartitionerFactory(uint64_t max_keys)
-        : max_keys_(max_keys) {}
-
-    std::unique_ptr<SstPartitioner> CreatePartitioner(
-        const SstPartitioner::Context& /*context*/) const override {
-      std::unique_ptr<SstPartitioner> partitioner;
-      partitioner.reset(new DBFollowerTestSstPartitioner(max_keys_));
-      return partitioner;
-    }
-
-    const char* Name() const override {
-      return "DBFollowerTestSstPartitionerFactory";
-    }
-
-   private:
-    uint64_t max_keys_;
-  };
-
   Status OpenAsFollower() {
-    Options opts = CurrentOptions();
-    if (!follower_env_) {
-      follower_env_ = NewCompositeEnv(
-          std::make_shared<DBFollowerTestFS>(env_->GetFileSystem()));
-    }
-    opts.env = follower_env_.get();
-    opts.follower_refresh_catchup_period_ms = 100;
-    return DB::OpenAsFollower(opts, follower_name_, dbname_, &follower_);
+    return DB::OpenAsFollower(CurrentOptions(), follower_name_, dbname_,
+                              &follower_);
   }
-
-  std::string FollowerGet(const std::string& k) {
-    ReadOptions options;
-    options.verify_checksums = true;
-    std::string result;
-    Status s = follower()->Get(options, k, &result);
-    if (s.IsNotFound()) {
-      result = "NOT_FOUND";
-    } else if (!s.ok()) {
-      result = s.ToString();
-    }
-    return result;
-  }
-
   DB* follower() { return follower_.get(); }
-  DBFollowerTestFS* follower_fs() {
-    return static_cast<DBFollowerTestFS*>(follower_env_->GetFileSystem().get());
-  }
-
-  void CheckDirs() {
-    std::vector<std::string> db_children;
-    std::vector<std::string> follower_children;
-    EXPECT_OK(env_->GetChildren(dbname_, &db_children));
-    EXPECT_OK(env_->GetChildren(follower_name_, &follower_children));
-
-    std::set<uint64_t> db_filenums;
-    std::set<uint64_t> follower_filenums;
-    for (auto& name : db_children) {
-      if (test::GetFileType(name) != kTableFile) {
-        continue;
-      }
-      db_filenums.insert(test::GetFileNumber(name));
-    }
-    for (auto& name : follower_children) {
-      if (test::GetFileType(name) != kTableFile) {
-        continue;
-      }
-      follower_filenums.insert(test::GetFileNumber(name));
-    }
-    db_filenums.merge(follower_filenums);
-    EXPECT_EQ(follower_filenums.size(), db_filenums.size());
-  }
 
  private:
   std::string follower_name_;
-  std::string db_parent_;
-  std::unique_ptr<Env> follower_env_;
   std::unique_ptr<DB> follower_;
 };
 
@@ -250,273 +51,8 @@ TEST_F(DBFollowerTest, Basic) {
   std::string val;
   ASSERT_OK(follower()->Get(ReadOptions(), "k1", &val));
   ASSERT_EQ(val, "v1");
-  CheckDirs();
 }
 
-TEST_F(DBFollowerTest, Flush) {
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImplFollower::TryCatchupWithLeader:Begin1", "Leader::Start"},
-      {"Leader::Done", "DBImplFollower::TryCatchupWithLeader:Begin2"},
-      {"DBImplFollower::TryCatchupWithLeader:End", "Follower::WaitForCatchup"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-
-  ASSERT_OK(OpenAsFollower());
-  TEST_SYNC_POINT("Leader::Start");
-  ASSERT_OK(Put("k1", "v1"));
-  ASSERT_OK(Flush());
-  TEST_SYNC_POINT("Leader::Done");
-
-  TEST_SYNC_POINT("Follower::WaitForCatchup");
-  std::string val;
-  ASSERT_OK(follower()->Get(ReadOptions(), "k1", &val));
-  ASSERT_EQ(val, "v1");
-  CheckDirs();
-
-  SyncPoint::GetInstance()->DisableProcessing();
-}
-
-// This test creates 4 L0 files, immediately followed by a compaction to L1.
-// The follower replays the 4 flush records from the MANIFEST unsuccessfully,
-// and then successfully recovers a Version from the compaction record
-TEST_F(DBFollowerTest, RetryCatchup) {
-  Options opts = CurrentOptions();
-  opts.disable_auto_compactions = true;
-  Reopen(opts);
-
-  ASSERT_OK(OpenAsFollower());
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImplFollower::TryCatchupWithLeader:Begin1", "Leader::Start"},
-      {"DBImpl::BackgroundCompaction:Start",
-       "DBImplFollower::TryCatchupWithLeader:Begin2"},
-      {"VersionEditHandlerPointInTime::MaybeCreateVersion:Begin1",
-       "DBImpl::BackgroundCompaction:BeforeCompaction"},
-      {"DBImpl::BackgroundCallCompaction:PurgedObsoleteFiles",
-       "VersionEditHandlerPointInTime::MaybeCreateVersion:Begin2"},
-      {"DBImplFollower::TryCatchupWithLeader:End", "Follower::WaitForCatchup"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-  TEST_SYNC_POINT("Leader::Start");
-  ASSERT_OK(Put("k1", "v1"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k1", "v2"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k1", "v3"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k1", "v4"));
-  ASSERT_OK(Flush());
-
-  ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, true));
-
-  TEST_SYNC_POINT("Follower::WaitForCatchup");
-  ASSERT_EQ(FollowerGet("k1"), "v4");
-  CheckDirs();
-
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-  SyncPoint::GetInstance()->DisableProcessing();
-}
-
-// This test validates the same as the previous test, except there is a
-// MANIFEST rollover between the flushes and compaction. The follower
-// does not switch to a new MANIFEST in ReadAndApply. So it would require
-// another round of refresh before catching up.
-TEST_F(DBFollowerTest, RetryCatchupManifestRollover) {
-  Options opts = CurrentOptions();
-  opts.disable_auto_compactions = true;
-  Reopen(opts);
-
-  ASSERT_OK(Put("k1", "v1"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k1", "v2"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k1", "v3"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(OpenAsFollower());
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImplFollower::TryCatchupWithLeader:Begin1", "Leader::Start"},
-      {"Leader::Flushed", "DBImplFollower::TryCatchupWithLeader:Begin2"},
-      {"VersionEditHandlerPointInTime::MaybeCreateVersion:Begin1",
-       "Leader::Done"},
-      {"DBImpl::BackgroundCallCompaction:PurgedObsoleteFiles",
-       "VersionEditHandlerPointInTime::MaybeCreateVersion:Begin2"},
-      {"DBImplFollower::TryCatchupWithLeader:End",
-       "Follower::WaitForCatchup:1"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-  TEST_SYNC_POINT("Leader::Start");
-  ASSERT_OK(Put("k1", "v4"));
-  ASSERT_OK(Flush());
-
-  TEST_SYNC_POINT("Leader::Flushed");
-  TEST_SYNC_POINT("Leader::Done");
-  Reopen(opts);
-  ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, true));
-
-  TEST_SYNC_POINT("Follower::WaitForCatchup:1");
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImplFollower::TryCatchupWithLeader:End",
-       "Follower::WaitForCatchup:2"},
-  });
-  TEST_SYNC_POINT("Follower::WaitForCatchup:2");
-  ASSERT_EQ(FollowerGet("k1"), "v4");
-
-  SyncPoint::GetInstance()->ClearAllCallBacks();
-  SyncPoint::GetInstance()->DisableProcessing();
-}
-
-// This test creates 4 L0 files and compacts them. The follower, during catchup,
-// successfully instantiates 4 Versions corresponding to the 4 files (but
-// donesn't install them yet), followed by deleting those 4 and adding a new
-// file from compaction. The test verifies that the 4 L0 files are deleted
-// correctly by the follower.
-// We use teh Barrier* functions to ensure that the follower first sees the 4
-// L0 files and is able to link them, and then sees the compaction that
-// obsoletes those L0 files (so those L0 files are intermediates that it has
-// to explicitly delete). Suppose we don't have any barriers, its possible
-// the follower reads the L0 records and compaction records from the MANIFEST
-// in one read, which means those L0 files would have already been deleted
-// by the leader and the follower cannot link to them.
-TEST_F(DBFollowerTest, IntermediateObsoleteFiles) {
-  Options opts = CurrentOptions();
-  opts.disable_auto_compactions = true;
-  Reopen(opts);
-  ASSERT_OK(OpenAsFollower());
-
-  follower_fs()->BarrierInit(2);
-  ASSERT_OK(Put("k1", "v1"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k1", "v2"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k1", "v3"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k1", "v4"));
-  ASSERT_OK(Flush());
-  follower_fs()->BarrierWaitAndReinit(2);
-
-  ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, true));
-  follower_fs()->BarrierWait();
-
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImplFollower::TryCatchupWithLeader:End",
-       "Follower::WaitForCatchup:1"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-  TEST_SYNC_POINT("Follower::WaitForCatchup:1");
-  CheckDirs();
-  ASSERT_EQ(FollowerGet("k1"), "v4");
-}
-
-// This test verifies a scenario where the follower can recover a Version
-// partially (i.e some of the additions cannot be found), and the files
-// that are found are obsoleted by a subsequent VersionEdit.
-TEST_F(DBFollowerTest, PartialVersionRecovery) {
-  Options opts = CurrentOptions();
-  opts.disable_auto_compactions = true;
-  opts.sst_partitioner_factory =
-      std::make_shared<DBFollowerTestSstPartitionerFactory>(1);
-  Reopen(opts);
-
-  ASSERT_OK(Put("k1", "v1"));
-  ASSERT_OK(Put("k2", "v1"));
-  ASSERT_OK(Put("k3", "v1"));
-  ASSERT_OK(Flush());
-  MoveFilesToLevel(2);
-
-  ASSERT_OK(Put("k1", "v2"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k3", "v2"));
-  ASSERT_OK(Flush());
-  MoveFilesToLevel(1);
-
-  ASSERT_OK(OpenAsFollower());
-  ASSERT_OK(dbfull()->SetOptions(dbfull()->DefaultColumnFamily(),
-                                 {{"max_compaction_bytes", "1"}}));
-
-  follower_fs()->BarrierInit(2);
-  Slice key("k1");
-  ASSERT_OK(dbfull()->TEST_CompactRange(1, &key, &key, nullptr, true));
-
-  follower_fs()->BarrierWaitAndReinit(2);
-
-  // The second compaction input overlaps the previous compaction outputs
-  // by one file. This file is never added to  VersionStorageInfo since it
-  // was added and deleted before the catch up completes. We later verify that
-  // the follower correctly deleted this file.
-  key = Slice("k3");
-  ASSERT_OK(dbfull()->TEST_CompactRange(1, &key, &key, nullptr, true));
-  follower_fs()->BarrierWait();
-
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImplFollower::TryCatchupWithLeader:End",
-       "Follower::WaitForCatchup:1"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-  TEST_SYNC_POINT("Follower::WaitForCatchup:1");
-  CheckDirs();
-  ASSERT_EQ(FollowerGet("k1"), "v2");
-  ASSERT_EQ(FollowerGet("k2"), "v1");
-  ASSERT_EQ(FollowerGet("k3"), "v2");
-  SyncPoint::GetInstance()->DisableProcessing();
-}
-
-// This test verifies a scenario similar to the PartialVersionRecovery, except
-// with a MANIFEST rollover in between. When there is a rollover, the
-// follower's attempt ends without installing a new Version. The next catch up
-// attempt will recover a full Version.
-TEST_F(DBFollowerTest, PartialVersionRecoveryWithRollover) {
-  Options opts = CurrentOptions();
-  opts.disable_auto_compactions = true;
-  opts.sst_partitioner_factory =
-      std::make_shared<DBFollowerTestSstPartitionerFactory>(1);
-  Reopen(opts);
-
-  ASSERT_OK(Put("k1", "v1"));
-  ASSERT_OK(Put("k2", "v1"));
-  ASSERT_OK(Put("k3", "v1"));
-  ASSERT_OK(Flush());
-  MoveFilesToLevel(2);
-
-  ASSERT_OK(Put("k1", "v2"));
-  ASSERT_OK(Flush());
-  ASSERT_OK(Put("k3", "v2"));
-  ASSERT_OK(Flush());
-  MoveFilesToLevel(1);
-
-  opts.max_compaction_bytes = 1;
-  Reopen(opts);
-
-  ASSERT_OK(OpenAsFollower());
-
-  follower_fs()->BarrierInit(2);
-  Slice key("k1");
-  ASSERT_OK(dbfull()->TEST_CompactRange(1, &key, &key, nullptr, true));
-
-  follower_fs()->BarrierWaitAndReinit(2);
-  Reopen(opts);
-  key = Slice("k3");
-  ASSERT_OK(dbfull()->TEST_CompactRange(1, &key, &key, nullptr, true));
-  follower_fs()->BarrierWait();
-
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImplFollower::TryCatchupWithLeader:Begin1",
-       "Follower::WaitForCatchup:1"},
-      {"Follower::WaitForCatchup:2",
-       "DBImplFollower::TryCatchupWithLeader:Begin2"},
-  });
-  SyncPoint::GetInstance()->EnableProcessing();
-  TEST_SYNC_POINT("Follower::WaitForCatchup:1");
-  TEST_SYNC_POINT("Follower::WaitForCatchup:2");
-  SyncPoint::GetInstance()->LoadDependency({
-      {"DBImplFollower::TryCatchupWithLeader:End",
-       "Follower::WaitForCatchup:3"},
-  });
-  TEST_SYNC_POINT("Follower::WaitForCatchup:3");
-  CheckDirs();
-  ASSERT_EQ(FollowerGet("k1"), "v2");
-  ASSERT_EQ(FollowerGet("k2"), "v1");
-  ASSERT_EQ(FollowerGet("k3"), "v2");
-  SyncPoint::GetInstance()->DisableProcessing();
-}
 #endif
 }  // namespace ROCKSDB_NAMESPACE
 

db/db_impl/db_impl.cc

@@ -2099,7 +2099,7 @@ Status DBImpl::GetEntity(const ReadOptions& _read_options,
   if (_read_options.io_activity != Env::IOActivity::kUnknown &&
       _read_options.io_activity != Env::IOActivity::kGetEntity) {
     return Status::InvalidArgument(
-        "Can only call GetEntity with `ReadOptions::io_activity` set to "
+        "Cannot call GetEntity with `ReadOptions::io_activity` != "
         "`Env::IOActivity::kUnknown` or `Env::IOActivity::kGetEntity`");
   }
   ReadOptions read_options(_read_options);
@@ -2126,7 +2126,7 @@ Status DBImpl::GetEntity(const ReadOptions& _read_options, const Slice& key,
   if (_read_options.io_activity != Env::IOActivity::kUnknown &&
       _read_options.io_activity != Env::IOActivity::kGetEntity) {
     s = Status::InvalidArgument(
-        "Can only call GetEntity with `ReadOptions::io_activity` set to "
+        "Cannot call GetEntity with `ReadOptions::io_activity` != "
         "`Env::IOActivity::kUnknown` or `Env::IOActivity::kGetEntity`");
     for (size_t i = 0; i < num_column_families; ++i) {
       (*result)[i].SetStatus(s);
@@ -3185,55 +3185,22 @@ void DBImpl::MultiGetEntity(const ReadOptions& _read_options, size_t num_keys,
                             ColumnFamilyHandle** column_families,
                             const Slice* keys, PinnableWideColumns* results,
                             Status* statuses, bool sorted_input) {
-  assert(statuses);
-
-  if (!column_families) {
-    const Status s = Status::InvalidArgument(
-        "Cannot call MultiGetEntity without column families");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-
-    return;
-  }
-
-  if (!keys) {
-    const Status s =
-        Status::InvalidArgument("Cannot call MultiGetEntity without keys");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-
-    return;
-  }
-
-  if (!results) {
-    const Status s = Status::InvalidArgument(
-        "Cannot call MultiGetEntity without PinnableWideColumns objects");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-
-    return;
-  }
-
   if (_read_options.io_activity != Env::IOActivity::kUnknown &&
       _read_options.io_activity != Env::IOActivity::kMultiGetEntity) {
-    const Status s = Status::InvalidArgument(
-        "Can only call MultiGetEntity with `ReadOptions::io_activity` set to "
+    Status s = Status::InvalidArgument(
+        "Can only call MultiGetEntity with `ReadOptions::io_activity` is "
         "`Env::IOActivity::kUnknown` or `Env::IOActivity::kMultiGetEntity`");
     for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
+      if (statuses[i].ok()) {
+        statuses[i] = s;
+      }
     }
-
     return;
   }
-
   ReadOptions read_options(_read_options);
   if (read_options.io_activity == Env::IOActivity::kUnknown) {
     read_options.io_activity = Env::IOActivity::kMultiGetEntity;
   }
-
   MultiGetCommon(read_options, num_keys, column_families, keys,
                  /* values */ nullptr, results, /* timestamps */ nullptr,
                  statuses, sorted_input);
@@ -3243,54 +3210,22 @@ void DBImpl::MultiGetEntity(const ReadOptions& _read_options,
                             ColumnFamilyHandle* column_family, size_t num_keys,
                             const Slice* keys, PinnableWideColumns* results,
                             Status* statuses, bool sorted_input) {
-  assert(statuses);
-
-  if (!column_family) {
-    const Status s = Status::InvalidArgument(
-        "Cannot call MultiGetEntity without a column family handle");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-
-    return;
-  }
-
-  if (!keys) {
-    const Status s =
-        Status::InvalidArgument("Cannot call MultiGetEntity without keys");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-
-    return;
-  }
-
-  if (!results) {
-    const Status s = Status::InvalidArgument(
-        "Cannot call MultiGetEntity without PinnableWideColumns objects");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-
-    return;
-  }
-
   if (_read_options.io_activity != Env::IOActivity::kUnknown &&
       _read_options.io_activity != Env::IOActivity::kMultiGetEntity) {
-    const Status s = Status::InvalidArgument(
-        "Can only call MultiGetEntity with `ReadOptions::io_activity` set to "
+    Status s = Status::InvalidArgument(
+        "Can only call MultiGetEntity with `ReadOptions::io_activity` is "
         "`Env::IOActivity::kUnknown` or `Env::IOActivity::kMultiGetEntity`");
     for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
+      if (statuses[i].ok()) {
+        statuses[i] = s;
+      }
     }
     return;
   }
-
   ReadOptions read_options(_read_options);
   if (read_options.io_activity == Env::IOActivity::kUnknown) {
     read_options.io_activity = Env::IOActivity::kMultiGetEntity;
   }
-
   MultiGetCommon(read_options, column_family, num_keys, keys,
                  /* values */ nullptr, results, /* timestamps */ nullptr,
                  statuses, sorted_input);
@@ -3299,34 +3234,18 @@ void DBImpl::MultiGetEntity(const ReadOptions& _read_options,
 void DBImpl::MultiGetEntity(const ReadOptions& _read_options, size_t num_keys,
                             const Slice* keys,
                             PinnableAttributeGroups* results) {
-  assert(results);
-
-  if (!keys) {
-    const Status s =
-        Status::InvalidArgument("Cannot call MultiGetEntity without keys");
-    for (size_t i = 0; i < num_keys; ++i) {
-      for (size_t j = 0; j < results[i].size(); ++j) {
-        results[i][j].SetStatus(s);
-      }
-    }
-
-    return;
-  }
-
   if (_read_options.io_activity != Env::IOActivity::kUnknown &&
       _read_options.io_activity != Env::IOActivity::kMultiGetEntity) {
-    const Status s = Status::InvalidArgument(
-        "Can only call MultiGetEntity with `ReadOptions::io_activity` set to "
+    Status s = Status::InvalidArgument(
+        "Can only call MultiGetEntity with ReadOptions::io_activity` is "
         "`Env::IOActivity::kUnknown` or `Env::IOActivity::kMultiGetEntity`");
     for (size_t i = 0; i < num_keys; ++i) {
       for (size_t j = 0; j < results[i].size(); ++j) {
         results[i][j].SetStatus(s);
       }
     }
-
     return;
   }
-
   ReadOptions read_options(_read_options);
   if (read_options.io_activity == Env::IOActivity::kUnknown) {
     read_options.io_activity = Env::IOActivity::kMultiGetEntity;
@@ -3344,7 +3263,6 @@ void DBImpl::MultiGetEntity(const ReadOptions& _read_options, size_t num_keys,
       ++total_count;
     }
   }
-
   std::vector<Status> statuses(total_count);
   std::vector<PinnableWideColumns> columns(total_count);
   MultiGetCommon(read_options, total_count, column_families.data(),
@@ -3365,15 +3283,6 @@ void DBImpl::MultiGetEntity(const ReadOptions& _read_options, size_t num_keys,
   }
 }
 
-void DBImpl::MultiGetEntityWithCallback(
-    const ReadOptions& read_options, ColumnFamilyHandle* column_family,
-    ReadCallback* callback,
-    autovector<KeyContext*, MultiGetContext::MAX_BATCH_SIZE>* sorted_keys) {
-  assert(read_options.io_activity == Env::IOActivity::kMultiGetEntity);
-
-  MultiGetWithCallbackImpl(read_options, column_family, callback, sorted_keys);
-}
-
 Status DBImpl::WrapUpCreateColumnFamilies(
     const ReadOptions& read_options, const WriteOptions& write_options,
     const std::vector<const ColumnFamilyOptions*>& cf_options) {

db/db_impl/db_impl.h

@@ -291,11 +291,6 @@ class DBImpl : public DB {
                       const Slice* keys,
                       PinnableAttributeGroups* results) override;
 
-  void MultiGetEntityWithCallback(
-      const ReadOptions& read_options, ColumnFamilyHandle* column_family,
-      ReadCallback* callback,
-      autovector<KeyContext*, MultiGetContext::MAX_BATCH_SIZE>* sorted_keys);
-
   Status CreateColumnFamily(const ColumnFamilyOptions& cf_options,
                             const std::string& column_family,
                             ColumnFamilyHandle** handle) override {
@@ -1648,7 +1643,6 @@ class DBImpl : public DB {
   friend class ForwardIterator;
   friend struct SuperVersion;
   friend class CompactedDBImpl;
-  friend class DBImplFollower;
 #ifndef NDEBUG
   friend class DBTest_ConcurrentFlushWAL_Test;
   friend class DBTest_MixedSlowdownOptionsStop_Test;
@@ -2054,22 +2048,17 @@ class DBImpl : public DB {
       mutex_.Lock();
     }
 
-    if (immutable_db_options_.unordered_write) {
-      // Wait for the ones who already wrote to the WAL to finish their
-      // memtable write.
-      if (pending_memtable_writes_.load() != 0) {
-        // XXX: suspicious wait while holding DB mutex?
-        std::unique_lock<std::mutex> guard(switch_mutex_);
-        switch_cv_.wait(guard,
-                        [&] { return pending_memtable_writes_.load() == 0; });
-      }
-    } else {
-      // (Writes are finished before the next write group starts.)
+    if (!immutable_db_options_.unordered_write) {
+      // Then the writes are finished before the next write group starts
+      return;
     }
 
-    // Wait for any LockWAL to clear
-    while (lock_wal_count_ > 0) {
-      bg_cv_.Wait();
+    // Wait for the ones who already wrote to the WAL to finish their
+    // memtable write.
+    if (pending_memtable_writes_.load() != 0) {
+      std::unique_lock<std::mutex> guard(switch_mutex_);
+      switch_cv_.wait(guard,
+                      [&] { return pending_memtable_writes_.load() == 0; });
     }
   }
 
@@ -2998,9 +2987,9 @@ inline Status DBImpl::FailIfReadCollapsedHistory(const ColumnFamilyData* cfd,
   if (!full_history_ts_low.empty() &&
       ucmp->CompareTimestamp(ts, full_history_ts_low) < 0) {
     std::stringstream oss;
-    oss << "Read timestamp: " << ucmp->TimestampToString(ts)
+    oss << "Read timestamp: " << ts.ToString(true)
         << " is smaller than full_history_ts_low: "
-        << ucmp->TimestampToString(full_history_ts_low) << std::endl;
+        << Slice(full_history_ts_low).ToString(true) << std::endl;
     return Status::InvalidArgument(oss.str());
   }
   return Status::OK();

db/db_impl/db_impl_compaction_flush.cc

@@ -224,15 +224,9 @@ Status DBImpl::FlushMemTableToOutputFile(
   // the host crashes after flushing and before WAL is persistent, the
   // flushed SST may contain data from write batches whose updates to
   // other (unflushed) column families are missing.
-  //
-  // When 2PC is enabled, non-recent WAL(s) may be needed for crash-recovery,
-  // even when there is only one CF in the DB, for prepared transactions that
-  // had not been committed yet. Make sure we sync them to keep the persisted
-  // WAL state at least as new as the persisted SST state.
   const bool needs_to_sync_closed_wals =
       logfile_number_ > 0 &&
-      (versions_->GetColumnFamilySet()->NumberOfColumnFamilies() > 1 ||
-       allow_2pc());
+      versions_->GetColumnFamilySet()->NumberOfColumnFamilies() > 1;
 
   // If needs_to_sync_closed_wals is true, we need to record the current
   // maximum memtable ID of this column family so that a later PickMemtables()
@@ -1094,12 +1088,7 @@ Status DBImpl::IncreaseFullHistoryTsLowImpl(ColumnFamilyData* cfd,
   assert(ucmp->timestamp_size() == ts_low.size() && !ts_low.empty());
   if (!current_ts_low.empty() &&
       ucmp->CompareTimestamp(ts_low, current_ts_low) < 0) {
-    std::stringstream oss;
-    oss << "Current full_history_ts_low: "
-        << ucmp->TimestampToString(current_ts_low)
-        << " is higher than provided ts: " << ucmp->TimestampToString(ts_low)
-        << std::endl;
-    return Status::InvalidArgument(oss.str());
+    return Status::InvalidArgument("Cannot decrease full_history_ts_low");
   }
 
   Status s = versions_->LogAndApply(cfd, *cfd->GetLatestMutableCFOptions(),
@@ -1131,11 +1120,6 @@ Status DBImpl::CompactRangeInternal(const CompactRangeOptions& options,
   if (options.target_path_id >= cfd->ioptions()->cf_paths.size()) {
     return Status::InvalidArgument("Invalid target path ID");
   }
-  if (options.change_level &&
-      cfd->ioptions()->compaction_style == kCompactionStyleFIFO) {
-    return Status::NotSupported(
-        "FIFO compaction does not support change_level.");
-  }
 
   bool flush_needed = true;
 
@@ -1196,16 +1180,6 @@ Status DBImpl::CompactRangeInternal(const CompactRangeOptions& options,
                             final_output_level, options, begin, end, exclusive,
                             false /* disable_trivial_move */,
                             std::numeric_limits<uint64_t>::max(), trim_ts);
-  } else if (cfd->ioptions()->compaction_style == kCompactionStyleFIFO) {
-    // FIFOCompactionPicker::CompactRange() will ignore the input key range
-    // [begin, end] and just try to pick compaction based on the configured
-    // option `compaction_options_fifo`. So we skip checking if [begin, end]
-    // overlaps with the DB here.
-    final_output_level = 0;
-    s = RunManualCompaction(cfd, /*input_level=*/0, final_output_level, options,
-                            begin, end, exclusive,
-                            false /* disable_trivial_move */,
-                            std::numeric_limits<uint64_t>::max(), trim_ts);
   } else {
     int first_overlapped_level = kInvalidLevel;
     {
@@ -1290,7 +1264,8 @@ Status DBImpl::CompactRangeInternal(const CompactRangeOptions& options,
       CleanupSuperVersion(super_version);
     }
     if (s.ok() && first_overlapped_level != kInvalidLevel) {
-      if (cfd->ioptions()->compaction_style == kCompactionStyleUniversal) {
+      if (cfd->ioptions()->compaction_style == kCompactionStyleUniversal ||
+          cfd->ioptions()->compaction_style == kCompactionStyleFIFO) {
         assert(first_overlapped_level == 0);
         s = RunManualCompaction(
             cfd, first_overlapped_level, first_overlapped_level, options, begin,
@@ -1542,11 +1517,16 @@ Status DBImpl::CompactFilesImpl(
         std::to_string(cfd->ioptions()->num_levels - 1));
   }
 
+  Status s = cfd->compaction_picker()->SanitizeCompactionInputFiles(
+      &input_set, cf_meta, output_level);
+  TEST_SYNC_POINT("DBImpl::CompactFilesImpl::PostSanitizeCompactionInputFiles");
+  if (!s.ok()) {
+    return s;
+  }
+
   std::vector<CompactionInputFiles> input_files;
-  Status s = cfd->compaction_picker()->SanitizeAndConvertCompactionInputFiles(
-      &input_set, cf_meta, output_level, version->storage_info(), &input_files);
-  TEST_SYNC_POINT(
-      "DBImpl::CompactFilesImpl::PostSanitizeAndConvertCompactionInputFiles");
+  s = cfd->compaction_picker()->GetCompactionInputsFromFileNumbers(
+      &input_files, &input_set, version->storage_info(), compact_options);
   if (!s.ok()) {
     return s;
   }
@@ -3360,7 +3340,7 @@ void DBImpl::BackgroundCallFlush(Env::Priority thread_pri) {
       bg_cv_.SignalAll();  // In case a waiter can proceed despite the error
       mutex_.Unlock();
       ROCKS_LOG_ERROR(immutable_db_options_.info_log,
-                      "[JOB %d] Waiting after background flush error: %s, "
+                      "[JOB %d] Waiting after background flush error: %s"
                       "Accumulated background error counts: %" PRIu64,
                       job_context.job_id, s.ToString().c_str(), error_cnt);
       log_buffer.FlushBufferToLog();

db/db_impl/db_impl_follower.cc

@@ -95,28 +95,17 @@ Status DBImplFollower::TryCatchUpWithLeader() {
   assert(versions_.get() != nullptr);
   assert(manifest_reader_.get() != nullptr);
   Status s;
-
-  TEST_SYNC_POINT("DBImplFollower::TryCatchupWithLeader:Begin1");
-  TEST_SYNC_POINT("DBImplFollower::TryCatchupWithLeader:Begin2");
   // read the manifest and apply new changes to the follower instance
   std::unordered_set<ColumnFamilyData*> cfds_changed;
   JobContext job_context(0, true /*create_superversion*/);
   {
     InstrumentedMutexLock lock_guard(&mutex_);
-    std::vector<std::string> files_to_delete;
     s = static_cast_with_check<ReactiveVersionSet>(versions_.get())
             ->ReadAndApply(&mutex_, &manifest_reader_,
-                           manifest_reader_status_.get(), &cfds_changed,
-                           &files_to_delete);
-    ReleaseFileNumberFromPendingOutputs(pending_outputs_inserted_elem_);
-    pending_outputs_inserted_elem_.reset(new std::list<uint64_t>::iterator(
-        CaptureCurrentFileNumberInPendingOutputs()));
+                           manifest_reader_status_.get(), &cfds_changed);
 
     ROCKS_LOG_INFO(immutable_db_options_.info_log, "Last sequence is %" PRIu64,
                    static_cast<uint64_t>(versions_->LastSequence()));
-    ROCKS_LOG_INFO(
-        immutable_db_options_.info_log, "Next file number is %" PRIu64,
-        static_cast<uint64_t>(versions_->current_next_file_number()));
     for (ColumnFamilyData* cfd : cfds_changed) {
       if (cfd->IsDropped()) {
         ROCKS_LOG_DEBUG(immutable_db_options_.info_log, "[%s] is dropped\n",
@@ -158,33 +147,9 @@ Status DBImplFollower::TryCatchUpWithLeader() {
         sv_context.NewSuperVersion();
       }
     }
-
-    for (auto& file : files_to_delete) {
-      IOStatus io_s = fs_->DeleteFile(file, IOOptions(), nullptr);
-      if (!io_s.ok()) {
-        ROCKS_LOG_INFO(immutable_db_options_.info_log,
-                       "Cannot delete file %s: %s", file.c_str(),
-                       io_s.ToString().c_str());
-      }
-    }
   }
   job_context.Clean();
 
-  // Cleanup unused, obsolete files.
-  JobContext purge_files_job_context(0);
-  {
-    InstrumentedMutexLock lock_guard(&mutex_);
-    // Currently, follower instance does not create any database files, thus
-    // is unnecessary for the follower to force full scan.
-    FindObsoleteFiles(&purge_files_job_context, /*force=*/false);
-  }
-  if (purge_files_job_context.HaveSomethingToDelete()) {
-    PurgeObsoleteFiles(purge_files_job_context);
-  }
-  purge_files_job_context.Clean();
-
-  TEST_SYNC_POINT("DBImplFollower::TryCatchupWithLeader:End");
-
   return s;
 }
 
@@ -234,8 +199,6 @@ Status DBImplFollower::Close() {
     catch_up_thread_.reset();
   }
 
-  ReleaseFileNumberFromPendingOutputs(pending_outputs_inserted_elem_);
-
   return DBImpl::Close();
 }
 

db/db_impl/db_impl_follower.h

@@ -27,7 +27,7 @@ class DBImplFollower : public DBImplSecondary {
   bool OwnTablesAndLogs() const override {
     // TODO: Change this to true once we've properly implemented file
     // deletion for the read scaling case
-    return true;
+    return false;
   }
 
   Status Recover(const std::vector<ColumnFamilyDescriptor>& column_families,
@@ -49,6 +49,5 @@ class DBImplFollower : public DBImplSecondary {
   std::string src_path_;
   port::Mutex mu_;
   port::CondVar cv_;
-  std::unique_ptr<std::list<uint64_t>::iterator> pending_outputs_inserted_elem_;
 };
 }  // namespace ROCKSDB_NAMESPACE

db/db_impl/db_impl_secondary.cc

@@ -680,8 +680,7 @@ Status DBImplSecondary::TryCatchUpWithPrimary() {
     InstrumentedMutexLock lock_guard(&mutex_);
     s = static_cast_with_check<ReactiveVersionSet>(versions_.get())
             ->ReadAndApply(&mutex_, &manifest_reader_,
-                           manifest_reader_status_.get(), &cfds_changed,
-                           /*files_to_delete=*/nullptr);
+                           manifest_reader_status_.get(), &cfds_changed);
 
     ROCKS_LOG_INFO(immutable_db_options_.info_log, "Last sequence is %" PRIu64,
                    static_cast<uint64_t>(versions_->LastSequence()));

db/db_impl/db_impl_write.cc

@@ -221,11 +221,7 @@ Status DBImpl::WriteImpl(const WriteOptions& write_options,
     return Status::InvalidArgument(
         "`WriteOptions::protection_bytes_per_key` must be zero or eight");
   } else if (write_options.disableWAL &&
-             immutable_db_options_.recycle_log_file_num > 0 &&
-             !(two_write_queues_ && disable_memtable)) {
-    // Corruption detection in recycled WALs relies on sequential sequence
-    // numbers, but WritePreparedTxnDB uses disableWAL internally for split
-    // writes
+             immutable_db_options_.recycle_log_file_num > 0) {
     return Status::InvalidArgument(
         "WriteOptions::disableWAL option is not supported if "
         "DBOptions::recycle_log_file_num > 0");
@@ -2156,8 +2152,6 @@ void DBImpl::NotifyOnMemTableSealed(ColumnFamilyData* /*cfd*/,
 // two_write_queues_ is true (This is to simplify the reasoning.)
 Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context) {
   mutex_.AssertHeld();
-  assert(lock_wal_count_ == 0);
-
   // TODO: plumb Env::IOActivity, Env::IOPriority
   const ReadOptions read_options;
   const WriteOptions write_options;

db/db_iter.cc

@@ -109,16 +109,6 @@ Status DBIter::GetProperty(std::string prop_name, std::string* prop) {
       *prop = "Iterator is not valid.";
     }
     return Status::OK();
-  } else if (prop_name == "rocksdb.iterator.is-value-pinned") {
-    if (valid_) {
-      *prop = (pin_thru_lifetime_ && iter_.Valid() &&
-               iter_.value().data() == value_.data())
-                  ? "1"
-                  : "0";
-    } else {
-      *prop = "Iterator is not valid.";
-    }
-    return Status::OK();
   } else if (prop_name == "rocksdb.iterator.internal-key") {
     *prop = saved_key_.GetUserKey().ToString();
     return Status::OK();

db/db_iterator_test.cc

@@ -133,17 +133,11 @@ TEST_P(DBIteratorTest, IteratorProperty) {
     ASSERT_NOK(iter->GetProperty("non_existing.value", &prop_value));
     ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value));
     ASSERT_EQ("0", prop_value);
-    ASSERT_OK(
-        iter->GetProperty("rocksdb.iterator.is-value-pinned", &prop_value));
-    ASSERT_EQ("0", prop_value);
     ASSERT_OK(iter->GetProperty("rocksdb.iterator.internal-key", &prop_value));
     ASSERT_EQ("1", prop_value);
     iter->Next();
     ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value));
     ASSERT_EQ("Iterator is not valid.", prop_value);
-    ASSERT_OK(
-        iter->GetProperty("rocksdb.iterator.is-value-pinned", &prop_value));
-    ASSERT_EQ("Iterator is not valid.", prop_value);
 
     // Get internal key at which the iteration stopped (tombstone in this case).
     ASSERT_OK(iter->GetProperty("rocksdb.iterator.internal-key", &prop_value));
@@ -1686,15 +1680,12 @@ TEST_P(DBIteratorTest, PinnedDataIteratorMultipleFiles) {
   ro.pin_data = true;
   auto iter = NewIterator(ro);
 
-  std::vector<std::pair<Slice, Slice>> results;
+  std::vector<std::pair<Slice, std::string>> results;
   for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {
     std::string prop_value;
     ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value));
     ASSERT_EQ("1", prop_value);
-    ASSERT_OK(
-        iter->GetProperty("rocksdb.iterator.is-value-pinned", &prop_value));
-    ASSERT_EQ("1", prop_value);
-    results.emplace_back(iter->key(), iter->value());
+    results.emplace_back(iter->key(), iter->value().ToString());
   }
 
   ASSERT_EQ(results.size(), true_data.size());
@@ -1748,9 +1739,6 @@ TEST_P(DBIteratorTest, PinnedDataIteratorMergeOperator) {
     std::string prop_value;
     ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value));
     ASSERT_EQ("1", prop_value);
-    ASSERT_OK(
-        iter->GetProperty("rocksdb.iterator.is-value-pinned", &prop_value));
-    ASSERT_EQ("0", prop_value);
     results.emplace_back(iter->key(), iter->value().ToString());
   }
   ASSERT_OK(iter->status());
@@ -1804,15 +1792,12 @@ TEST_P(DBIteratorTest, PinnedDataIteratorReadAfterUpdate) {
     }
   }
 
-  std::vector<std::pair<Slice, Slice>> results;
+  std::vector<std::pair<Slice, std::string>> results;
   for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {
     std::string prop_value;
     ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value));
     ASSERT_EQ("1", prop_value);
-    ASSERT_OK(
-        iter->GetProperty("rocksdb.iterator.is-value-pinned", &prop_value));
-    ASSERT_EQ("1", prop_value);
-    results.emplace_back(iter->key(), iter->value());
+    results.emplace_back(iter->key(), iter->value().ToString());
   }
   ASSERT_OK(iter->status());
 

db/db_kv_checksum_test.cc

@@ -732,7 +732,7 @@ TEST_P(DbMemtableKVChecksumTest, GetWithCorruptAfterMemtableInsert) {
       });
 
   SyncPoint::GetInstance()->SetCallBack(
-      "Memtable::SaveValue:Found:entry", [&](void* entry) {
+      "Memtable::SaveValue:Begin:entry", [&](void* entry) {
         char* buf = *static_cast<char**>(entry);
         buf[corrupt_byte_offset_] += corrupt_byte_addend_;
         ++corrupt_byte_offset_;
@@ -769,7 +769,7 @@ TEST_P(DbMemtableKVChecksumTest,
       });
 
   SyncPoint::GetInstance()->SetCallBack(
-      "Memtable::SaveValue:Found:entry", [&](void* entry) {
+      "Memtable::SaveValue:Begin:entry", [&](void* entry) {
         char* buf = *static_cast<char**>(entry);
         buf[corrupt_byte_offset_] += corrupt_byte_addend_;
         ++corrupt_byte_offset_;

db/db_test.cc

@@ -3844,9 +3844,6 @@ TEST_P(DBTestWithParam, FIFOCompactionTest) {
     } else {
       CompactRangeOptions cro;
       cro.exclusive_manual_compaction = exclusive_manual_compaction_;
-      cro.change_level = true;
-      ASSERT_TRUE(db_->CompactRange(cro, nullptr, nullptr).IsNotSupported());
-      cro.change_level = false;
       ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr));
     }
     // only 5 files should survive
@@ -4677,27 +4674,24 @@ TEST_F(DBTest, DynamicMemtableOptions) {
 
 #ifdef ROCKSDB_USING_THREAD_STATUS
 namespace {
-bool VerifyOperationCount(Env* env, ThreadStatus::OperationType op_type,
+void VerifyOperationCount(Env* env, ThreadStatus::OperationType op_type,
                           int expected_count) {
   int op_count = 0;
   std::vector<ThreadStatus> thread_list;
-  EXPECT_OK(env->GetThreadList(&thread_list));
+  ASSERT_OK(env->GetThreadList(&thread_list));
   for (const auto& thread : thread_list) {
     if (thread.operation_type == op_type) {
       op_count++;
     }
   }
   if (op_count != expected_count) {
-    fprintf(stderr, "op_count: %d, expected_count %d\n", op_count,
-            expected_count);
     for (const auto& thread : thread_list) {
-      fprintf(stderr, "thread id: %" PRIu64 ", thread status: %s, cf_name %s\n",
+      fprintf(stderr, "thread id: %" PRIu64 ", thread status: %s\n",
               thread.thread_id,
-              thread.GetOperationName(thread.operation_type).c_str(),
-              thread.cf_name.c_str());
+              thread.GetOperationName(thread.operation_type).c_str());
     }
   }
-  return op_count == expected_count;
+  ASSERT_EQ(op_count, expected_count);
 }
 }  // anonymous namespace
 
@@ -4797,11 +4791,11 @@ TEST_F(DBTest, ThreadStatusFlush) {
   ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();
 
   CreateAndReopenWithCF({"pikachu"}, options);
-  ASSERT_TRUE(VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 0));
+  VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 0);
 
   ASSERT_OK(Put(1, "foo", "v1"));
   ASSERT_EQ("v1", Get(1, "foo"));
-  ASSERT_TRUE(VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 0));
+  VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 0);
 
   uint64_t num_running_flushes = 0;
   ASSERT_TRUE(db_->GetIntProperty(DB::Properties::kNumRunningFlushes,
@@ -4814,7 +4808,7 @@ TEST_F(DBTest, ThreadStatusFlush) {
   // The first sync point is to make sure there's one flush job
   // running when we perform VerifyOperationCount().
   TEST_SYNC_POINT("DBTest::ThreadStatusFlush:1");
-  ASSERT_TRUE(VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 1));
+  VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 1);
   ASSERT_TRUE(db_->GetIntProperty(DB::Properties::kNumRunningFlushes,
                                   &num_running_flushes));
   ASSERT_EQ(num_running_flushes, 1);
@@ -4825,11 +4819,17 @@ TEST_F(DBTest, ThreadStatusFlush) {
 }
 
 TEST_P(DBTestWithParam, ThreadStatusSingleCompaction) {
+  const int kTestKeySize = 16;
   const int kTestValueSize = 984;
+  const int kEntrySize = kTestKeySize + kTestValueSize;
   const int kEntriesPerBuffer = 100;
   Options options;
   options.create_if_missing = true;
+  options.write_buffer_size = kEntrySize * kEntriesPerBuffer;
   options.compaction_style = kCompactionStyleLevel;
+  options.target_file_size_base = options.write_buffer_size;
+  options.max_bytes_for_level_base = options.target_file_size_base * 2;
+  options.max_bytes_for_level_multiplier = 2;
   options.compression = kNoCompression;
   options = CurrentOptions(options);
   options.env = env_;
@@ -4864,7 +4864,7 @@ TEST_P(DBTestWithParam, ThreadStatusSingleCompaction) {
                                     &num_running_compactions));
     ASSERT_EQ(num_running_compactions, 0);
     TEST_SYNC_POINT("DBTest::ThreadStatusSingleCompaction:0");
-    ASSERT_EQ(NumTableFilesAtLevel(0),
+    ASSERT_GE(NumTableFilesAtLevel(0),
               options.level0_file_num_compaction_trigger);
 
     // This makes sure at least one compaction is running.
@@ -4872,18 +4872,10 @@ TEST_P(DBTestWithParam, ThreadStatusSingleCompaction) {
 
     if (options.enable_thread_tracking) {
       // expecting one single L0 to L1 compaction
-      // This test is flaky and fails here.
-      bool match = VerifyOperationCount(env_, ThreadStatus::OP_COMPACTION, 1);
-      if (!match) {
-        ASSERT_TRUE(db_->GetIntProperty(DB::Properties::kNumRunningCompactions,
-                                        &num_running_compactions));
-        fprintf(stderr, "running compaction: %" PRIu64 " lsm state: %s\n",
-                num_running_compactions, FilesPerLevel().c_str());
-      }
-      ASSERT_TRUE(match);
+      VerifyOperationCount(env_, ThreadStatus::OP_COMPACTION, 1);
     } else {
       // If thread tracking is not enabled, compaction count should be 0.
-      ASSERT_TRUE(VerifyOperationCount(env_, ThreadStatus::OP_COMPACTION, 0));
+      VerifyOperationCount(env_, ThreadStatus::OP_COMPACTION, 0);
     }
     ASSERT_TRUE(db_->GetIntProperty(DB::Properties::kNumRunningCompactions,
                                     &num_running_compactions));

db/db_write_test.cc

@@ -666,25 +666,11 @@ TEST_P(DBWriteTest, LockWALInEffect) {
   // try the 1st WAL created during open
   ASSERT_OK(Put("key0", "value"));
   ASSERT_NE(options.manual_wal_flush, dbfull()->WALBufferIsEmpty());
-
   ASSERT_OK(db_->LockWAL());
-
   ASSERT_TRUE(dbfull()->WALBufferIsEmpty());
-  uint64_t wal_num = dbfull()->TEST_GetCurrentLogNumber();
-  // Manual flush with wait=false should abruptly fail with TryAgain
-  FlushOptions flush_opts;
-  flush_opts.wait = false;
-  for (bool allow_write_stall : {true, false}) {
-    flush_opts.allow_write_stall = allow_write_stall;
-    ASSERT_TRUE(db_->Flush(flush_opts).IsTryAgain());
-  }
-  ASSERT_EQ(wal_num, dbfull()->TEST_GetCurrentLogNumber());
-
   ASSERT_OK(db_->UnlockWAL());
-
-  // try the 2nd wal created during SwitchWAL (not locked this time)
+  // try the 2nd wal created during SwitchWAL
   ASSERT_OK(dbfull()->TEST_SwitchWAL());
-  ASSERT_NE(wal_num, dbfull()->TEST_GetCurrentLogNumber());
   ASSERT_OK(Put("key1", "value"));
   ASSERT_NE(options.manual_wal_flush, dbfull()->WALBufferIsEmpty());
   ASSERT_OK(db_->LockWAL());
@@ -723,57 +709,21 @@ TEST_P(DBWriteTest, LockWALInEffect) {
 }
 
 TEST_P(DBWriteTest, LockWALConcurrentRecursive) {
-  // This is a micro-stress test of LockWAL and concurrency handling.
-  // It is considered the most convenient way to balance functional
-  // coverage and reproducibility (vs. the two extremes of (a) unit tests
-  // tailored to specific interleavings and (b) db_stress)
   Options options = GetOptions();
   Reopen(options);
-  ASSERT_OK(Put("k1", "k1_orig"));
+  ASSERT_OK(Put("k1", "val"));
   ASSERT_OK(db_->LockWAL());  // 0 -> 1
   auto frozen_seqno = db_->GetLatestSequenceNumber();
-
-  std::string ingest_file = dbname_ + "/external.sst";
-  {
-    SstFileWriter sst_file_writer(EnvOptions(), options);
-    ASSERT_OK(sst_file_writer.Open(ingest_file));
-    ASSERT_OK(sst_file_writer.Put("k2", "k2_val"));
-    ExternalSstFileInfo external_info;
-    ASSERT_OK(sst_file_writer.Finish(&external_info));
-  }
-  AcqRelAtomic<bool> parallel_ingest_completed{false};
-  port::Thread parallel_ingest{[&]() {
-    IngestExternalFileOptions ingest_opts;
-    ingest_opts.move_files = true;  // faster than copy
-    // Shouldn't finish until WAL unlocked
-    ASSERT_OK(db_->IngestExternalFile({ingest_file}, ingest_opts));
-    parallel_ingest_completed.Store(true);
-  }};
-
-  AcqRelAtomic<bool> flush_completed{false};
-  port::Thread parallel_flush{[&]() {
-    FlushOptions flush_opts;
-    // NB: Flush with wait=false case is tested above in LockWALInEffect
-    flush_opts.wait = true;
-    // allow_write_stall = true blocks in fewer cases
-    flush_opts.allow_write_stall = true;
-    // Shouldn't finish until WAL unlocked
-    ASSERT_OK(db_->Flush(flush_opts));
-    flush_completed.Store(true);
-  }};
-
-  AcqRelAtomic<bool> parallel_put_completed{false};
-  port::Thread parallel_put{[&]() {
-    // This can make certain failure scenarios more likely:
-    //   sleep(1);
-    // Shouldn't finish until WAL unlocked
-    ASSERT_OK(Put("k1", "k1_mod"));
-    parallel_put_completed.Store(true);
+  std::atomic<bool> t1_completed{false};
+  port::Thread t1{[&]() {
+    // Won't finish until WAL unlocked
+    ASSERT_OK(Put("k1", "val2"));
+    t1_completed = true;
   }};
 
   ASSERT_OK(db_->LockWAL());  // 1 -> 2
   // Read-only ops are OK
-  ASSERT_EQ(Get("k1"), "k1_orig");
+  ASSERT_EQ(Get("k1"), "val");
   {
     std::vector<LiveFileStorageInfo> files;
     LiveFilesStorageInfoOptions lf_opts;
@@ -782,35 +732,29 @@ TEST_P(DBWriteTest, LockWALConcurrentRecursive) {
     ASSERT_OK(db_->GetLiveFilesStorageInfo({lf_opts}, &files));
   }
 
-  port::Thread parallel_lock_wal{[&]() {
+  port::Thread t2{[&]() {
     ASSERT_OK(db_->LockWAL());  // 2 -> 3 or 1 -> 2
   }};
 
   ASSERT_OK(db_->UnlockWAL());  // 2 -> 1 or 3 -> 2
-  // Give parallel_put an extra chance to jump in case of bug
+  // Give t1 an extra chance to jump in case of bug
   std::this_thread::yield();
-  parallel_lock_wal.join();
-  ASSERT_FALSE(parallel_put_completed.Load());
-  ASSERT_FALSE(parallel_ingest_completed.Load());
-  ASSERT_FALSE(flush_completed.Load());
+  t2.join();
+  ASSERT_FALSE(t1_completed.load());
 
   // Should now have 2 outstanding LockWAL
-  ASSERT_EQ(Get("k1"), "k1_orig");
+  ASSERT_EQ(Get("k1"), "val");
 
   ASSERT_OK(db_->UnlockWAL());  // 2 -> 1
 
-  ASSERT_FALSE(parallel_put_completed.Load());
-  ASSERT_FALSE(parallel_ingest_completed.Load());
-  ASSERT_FALSE(flush_completed.Load());
-
-  ASSERT_EQ(Get("k1"), "k1_orig");
-  ASSERT_EQ(Get("k2"), "NOT_FOUND");
+  ASSERT_FALSE(t1_completed.load());
+  ASSERT_EQ(Get("k1"), "val");
   ASSERT_EQ(frozen_seqno, db_->GetLatestSequenceNumber());
 
   // Ensure final Unlock is concurrency safe and extra Unlock is safe but
   // non-OK
   std::atomic<int> unlock_ok{0};
-  port::Thread parallel_stuff{[&]() {
+  port::Thread t3{[&]() {
     if (db_->UnlockWAL().ok()) {
       unlock_ok++;
     }
@@ -823,23 +767,18 @@ TEST_P(DBWriteTest, LockWALConcurrentRecursive) {
   if (db_->UnlockWAL().ok()) {
     unlock_ok++;
   }
-  parallel_stuff.join();
+  t3.join();
 
   // There was one extra unlock, so just one non-ok
   ASSERT_EQ(unlock_ok.load(), 2);
 
   // Write can proceed
-  parallel_put.join();
-  ASSERT_TRUE(parallel_put_completed.Load());
-  ASSERT_EQ(Get("k1"), "k1_mod");
-  parallel_ingest.join();
-  ASSERT_TRUE(parallel_ingest_completed.Load());
-  ASSERT_EQ(Get("k2"), "k2_val");
-  parallel_flush.join();
-  ASSERT_TRUE(flush_completed.Load());
+  t1.join();
+  ASSERT_TRUE(t1_completed.load());
+  ASSERT_EQ(Get("k1"), "val2");
   // And new writes
-  ASSERT_OK(Put("k3", "val"));
-  ASSERT_EQ(Get("k3"), "val");
+  ASSERT_OK(Put("k2", "val"));
+  ASSERT_EQ(Get("k2"), "val");
 }
 
 TEST_P(DBWriteTest, ConcurrentlyDisabledWAL) {

db/dbformat.cc

@@ -47,8 +47,6 @@ EntryType GetEntryType(ValueType value_type) {
       return kEntryBlobIndex;
     case kTypeWideColumnEntity:
       return kEntryWideColumnEntity;
-    case kTypeValuePreferredSeqno:
-      return kEntryTimedPut;
     default:
       return kEntryOther;
   }
@@ -157,23 +155,10 @@ void ReplaceInternalKeyWithMinTimestamp(std::string* result, const Slice& key,
   result->append(key.data() + key_sz - kNumInternalBytes, kNumInternalBytes);
 }
 
-std::string ParsedInternalKey::DebugString(bool log_err_key, bool hex,
-                                           const Comparator* ucmp) const {
+std::string ParsedInternalKey::DebugString(bool log_err_key, bool hex) const {
   std::string result = "'";
-  size_t ts_sz_for_debug = ucmp == nullptr ? 0 : ucmp->timestamp_size();
   if (log_err_key) {
-    if (ts_sz_for_debug == 0) {
-      result += user_key.ToString(hex);
-    } else {
-      assert(user_key.size() >= ts_sz_for_debug);
-      Slice user_key_without_ts = user_key;
-      user_key_without_ts.remove_suffix(ts_sz_for_debug);
-      result += user_key_without_ts.ToString(hex);
-      Slice ts = Slice(user_key.data() + user_key.size() - ts_sz_for_debug,
-                       ts_sz_for_debug);
-      result += "|timestamp:";
-      result += ucmp->TimestampToString(ts);
-    }
+    result += user_key.ToString(hex);
   } else {
     result += "<redacted>";
   }
@@ -186,11 +171,11 @@ std::string ParsedInternalKey::DebugString(bool log_err_key, bool hex,
   return result;
 }
 
-std::string InternalKey::DebugString(bool hex, const Comparator* ucmp) const {
+std::string InternalKey::DebugString(bool hex) const {
   std::string result;
   ParsedInternalKey parsed;
   if (ParseInternalKey(rep_, &parsed, false /* log_err_key */).ok()) {
-    result = parsed.DebugString(true /* log_err_key */, hex, ucmp);  // TODO
+    result = parsed.DebugString(true /* log_err_key */, hex);  // TODO
   } else {
     result = "(bad)";
     result.append(EscapeString(rep_));

db/dbformat.h

@@ -148,8 +148,7 @@ struct ParsedInternalKey {
   // u contains timestamp if user timestamp feature is enabled.
   ParsedInternalKey(const Slice& u, const SequenceNumber& seq, ValueType t)
       : user_key(u), sequence(seq), type(t) {}
-  std::string DebugString(bool log_err_key, bool hex,
-                          const Comparator* ucmp = nullptr) const;
+  std::string DebugString(bool log_err_key, bool hex) const;
 
   void clear() {
     user_key.clear();
@@ -504,7 +503,7 @@ class InternalKey {
     AppendInternalKeyFooter(&rep_, s, t);
   }
 
-  std::string DebugString(bool hex, const Comparator* ucmp = nullptr) const;
+  std::string DebugString(bool hex) const;
 };
 
 inline int InternalKeyComparator::Compare(const InternalKey& a,

db/error_handler_fs_test.cc

@@ -30,14 +30,6 @@ class DBErrorHandlingFSTest : public DBTestBase {
     fault_env_.reset(new CompositeEnvWrapper(env_, fault_fs_));
   }
 
-  ~DBErrorHandlingFSTest() {
-    // Before destroying fault_env_
-    SyncPoint::GetInstance()->DisableProcessing();
-    SyncPoint::GetInstance()->LoadDependency({});
-    SyncPoint::GetInstance()->ClearAllCallBacks();
-    Close();
-  }
-
   std::string GetManifestNameFromLiveFiles() {
     std::vector<std::string> live_files;
     uint64_t manifest_size;

db/external_sst_file_ingestion_job.cc

@@ -953,15 +953,13 @@ Status ExternalSstFileIngestionJob::AssignLevelAndSeqnoForIngestedFile(
   *assigned_seqno = 0;
   auto ucmp = cfd_->user_comparator();
   const size_t ts_sz = ucmp->timestamp_size();
-  if (force_global_seqno || files_overlap_ ||
-      compaction_style == kCompactionStyleFIFO) {
+  if (force_global_seqno || files_overlap_) {
     *assigned_seqno = last_seqno + 1;
     // If files overlap, we have to ingest them at level 0.
-    if (files_overlap_ || compaction_style == kCompactionStyleFIFO) {
+    if (files_overlap_) {
       assert(ts_sz == 0);
       file_to_ingest->picked_level = 0;
-      if (ingestion_options_.fail_if_not_bottommost_level &&
-          cfd_->NumberLevels() > 1) {
+      if (ingestion_options_.fail_if_not_bottommost_level) {
         status = Status::TryAgain(
             "Files cannot be ingested to Lmax. Please make sure key range of "
             "Lmax does not overlap with files to ingest.");

db/external_sst_file_test.cc

@@ -3082,68 +3082,6 @@ TEST_P(ExternalSSTFileTest,
   delete iter;
 }
 
-TEST_F(ExternalSSTFileTest, FIFOCompaction) {
-  // FIFO always ingests SST files to L0 and assign latest sequence number.
-  Options options = CurrentOptions();
-  options.num_levels = 1;
-  options.compaction_style = kCompactionStyleFIFO;
-  options.max_open_files = -1;
-  DestroyAndReopen(options);
-  std::map<std::string, std::string> true_data;
-
-  for (int i = 0; i < 100; ++i) {
-    ASSERT_OK(Put(Key(i), Key(i) + "_val"));
-    true_data[Key(i)] = Key(i) + "_val";
-  }
-  ASSERT_OK(Flush());
-  ASSERT_EQ("1", FilesPerLevel());
-  std::vector<std::pair<std::string, std::string>> file_data;
-  for (int i = 0; i <= 20; i++) {
-    file_data.emplace_back(Key(i), Key(i) + "_ingest");
-  }
-  // Overlaps with memtable, will trigger flush
-  ASSERT_OK(GenerateAndAddExternalFile(options, file_data, -1,
-                                       /*allow_global_seqno=*/true, true, false,
-                                       false, false, &true_data));
-  ASSERT_EQ("2", FilesPerLevel());
-
-  file_data.clear();
-  for (int i = 100; i <= 120; i++) {
-    file_data.emplace_back(Key(i), Key(i) + "_ingest");
-  }
-  // global sequence number is always assigned, so this will fail
-  ASSERT_NOK(GenerateAndAddExternalFile(options, file_data, -1,
-                                        /*allow_global_seqno=*/false, true,
-                                        false, false, false, &true_data));
-  ASSERT_OK(GenerateAndAddExternalFile(options, file_data, -1,
-                                       /*allow_global_seqno=*/true, true, false,
-                                       false, false, &true_data));
-
-  // Compact to data to lower level to test multi-level FIFO later
-  options.num_levels = 7;
-  options.compaction_style = kCompactionStyleUniversal;
-  ASSERT_OK(TryReopen(options));
-  CompactRangeOptions cro;
-  cro.bottommost_level_compaction = BottommostLevelCompaction::kForceOptimized;
-  ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr));
-  ASSERT_EQ("0,0,0,0,0,0,1", FilesPerLevel());
-
-  options.num_levels = 7;
-  options.compaction_style = kCompactionStyleFIFO;
-  ASSERT_OK(TryReopen(options));
-  file_data.clear();
-  for (int i = 200; i <= 220; i++) {
-    file_data.emplace_back(Key(i), Key(i) + "_ingest");
-  }
-  // Files are ingested into L0 for multi-level FIFO
-  ASSERT_OK(GenerateAndAddExternalFile(options, file_data, -1,
-                                       /*allow_global_seqno=*/true, true, false,
-                                       false, false, &true_data));
-
-  ASSERT_EQ("1,0,0,0,0,0,1", FilesPerLevel());
-  VerifyDBFromMap(true_data);
-}
-
 class ExternalSSTFileWithTimestampTest : public ExternalSSTFileTest {
  public:
   ExternalSSTFileWithTimestampTest() = default;

db/import_column_family_job.cc

@@ -175,29 +175,22 @@ Status ImportColumnFamilyJob::Run() {
         static_cast<uint64_t>(temp_current_time);
   }
 
+  // Recover files' epoch number using dummy VersionStorageInfo
+  VersionBuilder dummy_version_builder(
+      cfd_->current()->version_set()->file_options(), cfd_->ioptions(),
+      cfd_->table_cache(), cfd_->current()->storage_info(),
+      cfd_->current()->version_set(),
+      cfd_->GetFileMetadataCacheReservationManager());
+  VersionStorageInfo dummy_vstorage(
+      &cfd_->internal_comparator(), cfd_->user_comparator(),
+      cfd_->NumberLevels(), cfd_->ioptions()->compaction_style,
+      nullptr /* src_vstorage */, cfd_->ioptions()->force_consistency_checks,
+      EpochNumberRequirement::kMightMissing, cfd_->ioptions()->clock,
+      cfd_->GetLatestMutableCFOptions()->bottommost_file_compaction_delay,
+      cfd_->current()->version_set()->offpeak_time_option());
   Status s;
-  // When importing multiple CFs, we should not reuse epoch number from ingested
-  // files. Since these epoch numbers were assigned by different CFs, there may
-  // be different files from different CFs with the same epoch number. With a
-  // subsequent intra-L0 compaction we may end up with files with overlapping
-  // key range but the same epoch number. Here we will create a dummy
-  // VersionStorageInfo per CF being imported. Each CF's files will be assigned
-  // increasing epoch numbers to avoid duplicated epoch number. This is done by
-  // only resetting epoch number of the new CF in the first call to
-  // RecoverEpochNumbers() below.
+
   for (size_t i = 0; s.ok() && i < files_to_import_.size(); ++i) {
-    VersionBuilder dummy_version_builder(
-        cfd_->current()->version_set()->file_options(), cfd_->ioptions(),
-        cfd_->table_cache(), cfd_->current()->storage_info(),
-        cfd_->current()->version_set(),
-        cfd_->GetFileMetadataCacheReservationManager());
-    VersionStorageInfo dummy_vstorage(
-        &cfd_->internal_comparator(), cfd_->user_comparator(),
-        cfd_->NumberLevels(), cfd_->ioptions()->compaction_style,
-        nullptr /* src_vstorage */, cfd_->ioptions()->force_consistency_checks,
-        EpochNumberRequirement::kMightMissing, cfd_->ioptions()->clock,
-        cfd_->GetLatestMutableCFOptions()->bottommost_file_compaction_delay,
-        cfd_->current()->version_set()->offpeak_time_option());
     for (size_t j = 0; s.ok() && j < files_to_import_[i].size(); ++j) {
       const auto& f = files_to_import_[i][j];
       const auto& file_metadata = *metadatas_[i][j];
@@ -225,39 +218,42 @@ Status ImportColumnFamilyJob::Run() {
               f.table_properties.user_defined_timestamps_persisted));
       s = dummy_version_builder.Apply(&dummy_version_edit);
     }
-    if (s.ok()) {
-      s = dummy_version_builder.SaveTo(&dummy_vstorage);
-    }
-    if (s.ok()) {
-      // force resetting epoch number for each file
-      dummy_vstorage.RecoverEpochNumbers(cfd_, /*restart_epoch=*/i == 0,
-                                         /*force=*/true);
-      edit_.SetColumnFamily(cfd_->GetID());
+  }
+
+  if (s.ok()) {
+    s = dummy_version_builder.SaveTo(&dummy_vstorage);
+  }
+  if (s.ok()) {
+    dummy_vstorage.RecoverEpochNumbers(cfd_);
+  }
+
+  // Record changes from this CF import in VersionEdit, including files with
+  // recovered epoch numbers
+  if (s.ok()) {
+    edit_.SetColumnFamily(cfd_->GetID());
 
-      for (int level = 0; level < dummy_vstorage.num_levels(); level++) {
-        for (FileMetaData* file_meta : dummy_vstorage.LevelFiles(level)) {
-          edit_.AddFile(level, *file_meta);
-          // If incoming sequence number is higher, update local sequence
-          // number.
-          if (file_meta->fd.largest_seqno > versions_->LastSequence()) {
-            versions_->SetLastAllocatedSequence(file_meta->fd.largest_seqno);
-            versions_->SetLastPublishedSequence(file_meta->fd.largest_seqno);
-            versions_->SetLastSequence(file_meta->fd.largest_seqno);
-          }
-        }
-      }
-    }
-    // Release resources occupied by the dummy VersionStorageInfo
     for (int level = 0; level < dummy_vstorage.num_levels(); level++) {
       for (FileMetaData* file_meta : dummy_vstorage.LevelFiles(level)) {
-        file_meta->refs--;
-        if (file_meta->refs <= 0) {
-          delete file_meta;
+        edit_.AddFile(level, *file_meta);
+        // If incoming sequence number is higher, update local sequence number.
+        if (file_meta->fd.largest_seqno > versions_->LastSequence()) {
+          versions_->SetLastAllocatedSequence(file_meta->fd.largest_seqno);
+          versions_->SetLastPublishedSequence(file_meta->fd.largest_seqno);
+          versions_->SetLastSequence(file_meta->fd.largest_seqno);
         }
       }
     }
   }
 
+  // Release resources occupied by the dummy VersionStorageInfo
+  for (int level = 0; level < dummy_vstorage.num_levels(); level++) {
+    for (FileMetaData* file_meta : dummy_vstorage.LevelFiles(level)) {
+      file_meta->refs--;
+      if (file_meta->refs <= 0) {
+        delete file_meta;
+      }
+    }
+  }
   return s;
 }
 

db/import_column_family_test.cc

@@ -946,108 +946,6 @@ TEST_F(ImportColumnFamilyTest, ImportMultiColumnFamilySeveralFilesWithOverlap) {
   ASSERT_OK(db_->DestroyColumnFamilyHandle(second_cfh));
 }
 
-TEST_F(ImportColumnFamilyTest, AssignEpochNumberToMultipleCF) {
-  // Test ingesting CFs where L0 files could have the same epoch number.
-  Options options = CurrentOptions();
-  options.level_compaction_dynamic_level_bytes = true;
-  options.max_background_jobs = 8;
-  env_->SetBackgroundThreads(2, Env::LOW);
-  env_->SetBackgroundThreads(0, Env::BOTTOM);
-  CreateAndReopenWithCF({"CF1", "CF2"}, options);
-
-  // CF1:
-  // L6: [0, 99], [100, 199]
-  // CF2:
-  // L6: [1000, 1099], [1100, 1199]
-  for (int i = 100; i < 200; ++i) {
-    ASSERT_OK(Put(1, Key(i), Key(i) + "_val"));
-    ASSERT_OK(Put(2, Key(1000 + i), Key(1000 + i) + "_val"));
-  }
-  ASSERT_OK(Flush(1));
-  ASSERT_OK(Flush(2));
-  for (int i = 0; i < 100; ++i) {
-    ASSERT_OK(Put(1, Key(i), Key(i) + "_val"));
-    ASSERT_OK(Put(2, Key(1000 + i), Key(1000 + i) + "_val"));
-  }
-  ASSERT_OK(Flush(1));
-  ASSERT_OK(Flush(2));
-  MoveFilesToLevel(6, 1);
-  MoveFilesToLevel(6, 2);
-
-  // CF1:
-  // level 0 epoch: 5 file num 30 smallest key000010 - key000019
-  // level 0 epoch: 4 file num 27 smallest key000000 - key000009
-  // level 0 epoch: 3 file num 23 smallest key000100 - key000199
-  // level 6 epoch: 2 file num 20 smallest key000000 - key000099
-  // level 6 epoch: 1 file num 17 smallest key000100 - key000199
-  // CF2:
-  // level 0 epoch: 5 file num 31 smallest key001010 - key001019
-  // level 0 epoch: 4 file num 28 smallest key001000 - key001009
-  // level 0 epoch: 3 file num 25 smallest key001020 - key001029
-  // level 6 epoch: 2 file num 21 smallest key001000 - key001099
-  // level 6 epoch: 1 file num 18 smallest key001100 - key001199
-  for (int i = 100; i < 200; ++i) {
-    ASSERT_OK(Put(1, Key(i), Key(i) + "_val"));
-  }
-  ASSERT_OK(Flush(1));
-  for (int i = 20; i < 30; ++i) {
-    ASSERT_OK(Put(2, Key(i + 1000), Key(i + 1000) + "_val"));
-  }
-  ASSERT_OK(Flush(2));
-
-  for (int i = 0; i < 20; ++i) {
-    ASSERT_OK(Put(1, Key(i), Key(i) + "_val"));
-    ASSERT_OK(Put(2, Key(i + 1000), Key(i + 1000) + "_val"));
-    if (i % 10 == 9) {
-      ASSERT_OK(Flush(1));
-      ASSERT_OK(Flush(2));
-    }
-  }
-  ASSERT_OK(Flush(1));
-  ASSERT_OK(Flush(2));
-
-  // Create a CF by importing these two CF1 and CF2.
-  // Then two compactions will be triggerred, one to compact from L0
-  // to L6 (files #23 and #17), and another to do intra-L0 compaction
-  // for the rest of the L0 files. Before a bug fix, we used to
-  // directly use the epoch numbers from the ingested files in the new CF.
-  // This means different files from different CFs can have the same epoch
-  // number. If the intra-L0 compaction finishes first, it can cause a
-  // corruption where two L0 files can have the same epoch number but
-  // with overlapping key range.
-  Checkpoint* checkpoint1;
-  ASSERT_OK(Checkpoint::Create(db_, &checkpoint1));
-  ASSERT_OK(checkpoint1->ExportColumnFamily(handles_[1], export_files_dir_,
-                                            &metadata_ptr_));
-  ASSERT_OK(checkpoint1->ExportColumnFamily(handles_[2], export_files_dir2_,
-                                            &metadata_ptr2_));
-  ASSERT_NE(metadata_ptr_, nullptr);
-  ASSERT_NE(metadata_ptr2_, nullptr);
-
-  std::atomic_int compaction_counter = 0;
-  SyncPoint::GetInstance()->SetCallBack(
-      "DBImpl::BackgroundCompaction:NonTrivial:BeforeRun",
-      [&compaction_counter](void*) {
-        compaction_counter++;
-        if (compaction_counter == 1) {
-          // Wait for the next compaction to finish
-          TEST_SYNC_POINT("WaitForSecondCompaction");
-        }
-      });
-  SyncPoint::GetInstance()->LoadDependency(
-      {{"DBImpl::BackgroundCompaction:AfterCompaction",
-        "WaitForSecondCompaction"}});
-  SyncPoint::GetInstance()->EnableProcessing();
-  ImportColumnFamilyOptions import_options;
-  import_options.move_files = false;
-  std::vector<const ExportImportFilesMetaData*> metadatas = {metadata_ptr_,
-                                                             metadata_ptr2_};
-  ASSERT_OK(db_->CreateColumnFamilyWithImport(options, "CF3", import_options,
-                                              metadatas, &import_cfh_));
-  WaitForCompactOptions o;
-  ASSERT_OK(db_->WaitForCompact(o));
-  delete checkpoint1;
-}
 }  // namespace ROCKSDB_NAMESPACE
 
 int main(int argc, char** argv) {

db/log_reader.cc

@@ -487,11 +487,9 @@ unsigned int Reader::ReadPhysicalRecord(Slice* result, size_t* drop_size,
          type == kRecyclableUserDefinedTimestampSizeType);
     if (is_recyclable_type) {
       header_size = kRecyclableHeaderSize;
-      if (first_record_read_ && !recycled_) {
-        // A recycled log should have started with a recycled record
-        return kBadRecord;
+      if (end_of_buffer_offset_ - buffer_.size() == 0) {
+        recycled_ = true;
       }
-      recycled_ = true;
       // We need enough for the larger header
       if (buffer_.size() < static_cast<size_t>(kRecyclableHeaderSize)) {
         int r = kEof;
@@ -869,12 +867,9 @@ bool FragmentBufferedReader::TryReadFragment(
   int header_size = kHeaderSize;
   if ((type >= kRecyclableFullType && type <= kRecyclableLastType) ||
       type == kRecyclableUserDefinedTimestampSizeType) {
-    if (first_record_read_ && !recycled_) {
-      // A recycled log should have started with a recycled record
-      *fragment_type_or_err = kBadRecord;
-      return true;
+    if (end_of_buffer_offset_ - buffer_.size() == 0) {
+      recycled_ = true;
     }
-    recycled_ = true;
     header_size = kRecyclableHeaderSize;
     while (buffer_.size() < static_cast<size_t>(kRecyclableHeaderSize)) {
       size_t old_size = buffer_.size();

db/log_test.cc

@@ -774,32 +774,6 @@ TEST_P(LogTest, RecycleWithTimestampSize) {
   ASSERT_EQ("EOF", Read());
 }
 
-// Validates that `MaybeAddUserDefinedTimestampSizeRecord`` adds padding to the
-// tail of a block and switches to a new block, if there's not enough space for
-// the record.
-TEST_P(LogTest, TimestampSizeRecordPadding) {
-  bool recyclable_log = (std::get<0>(GetParam()) != 0);
-  const size_t header_size =
-      recyclable_log ? kRecyclableHeaderSize : kHeaderSize;
-  const size_t data_len = kBlockSize - 2 * header_size;
-
-  const auto first_str = BigString("foo", data_len);
-  Write(first_str);
-
-  UnorderedMap<uint32_t, size_t> ts_sz = {
-      {2, sizeof(uint64_t)},
-  };
-  ASSERT_OK(
-      writer_->MaybeAddUserDefinedTimestampSizeRecord(WriteOptions(), ts_sz));
-  ASSERT_LT(writer_->TEST_block_offset(), kBlockSize);
-
-  const auto second_str = BigString("bar", 1000);
-  Write(second_str);
-
-  ASSERT_EQ(first_str, Read());
-  CheckRecordAndTimestampSize(second_str, ts_sz);
-}
-
 // Do NOT enable compression for this instantiation.
 INSTANTIATE_TEST_CASE_P(
     Log, LogTest,
@@ -1157,42 +1131,6 @@ TEST_P(CompressionLogTest, AlignedFragmentation) {
   ASSERT_EQ("EOF", Read());
 }
 
-TEST_P(CompressionLogTest, ChecksumMismatch) {
-  const CompressionType kCompressionType = std::get<2>(GetParam());
-  const bool kCompressionEnabled = kCompressionType != kNoCompression;
-  const bool kRecyclableLog = (std::get<0>(GetParam()) != 0);
-  if (!StreamingCompressionTypeSupported(kCompressionType)) {
-    ROCKSDB_GTEST_SKIP("Test requires support for compression type");
-    return;
-  }
-  ASSERT_OK(SetupTestEnv());
-
-  Write("foooooo");
-  int header_len;
-  if (kRecyclableLog) {
-    header_len = kRecyclableHeaderSize;
-  } else {
-    header_len = kHeaderSize;
-  }
-  int compression_record_len;
-  if (kCompressionEnabled) {
-    compression_record_len = header_len + 4;
-  } else {
-    compression_record_len = 0;
-  }
-  IncrementByte(compression_record_len + header_len /* offset */,
-                14 /* delta */);
-
-  ASSERT_EQ("EOF", Read());
-  if (!kRecyclableLog) {
-    ASSERT_GT(DroppedBytes(), 0U);
-    ASSERT_EQ("OK", MatchError("checksum mismatch"));
-  } else {
-    ASSERT_EQ(0U, DroppedBytes());
-    ASSERT_EQ("", ReportMessage());
-  }
-}
-
 INSTANTIATE_TEST_CASE_P(
     Compression, CompressionLogTest,
     ::testing::Combine(::testing::Values(0, 1), ::testing::Bool(),

db/log_writer.cc

@@ -27,8 +27,6 @@ Writer::Writer(std::unique_ptr<WritableFileWriter>&& dest, uint64_t log_number,
       block_offset_(0),
       log_number_(log_number),
       recycle_log_files_(recycle_log_files),
-      // Header size varies depending on whether we are recycling or not.
-      header_size_(recycle_log_files ? kRecyclableHeaderSize : kHeaderSize),
       manual_flush_(manual_flush),
       compression_type_(compression_type),
       compress_(nullptr) {
@@ -82,6 +80,10 @@ IOStatus Writer::AddRecord(const WriteOptions& write_options,
   const char* ptr = slice.data();
   size_t left = slice.size();
 
+  // Header size varies depending on whether we are recycling or not.
+  const int header_size =
+      recycle_log_files_ ? kRecyclableHeaderSize : kHeaderSize;
+
   // Fragment the record if necessary and emit it.  Note that if slice
   // is empty, we still want to iterate once to emit a single
   // zero-length record
@@ -100,12 +102,12 @@ IOStatus Writer::AddRecord(const WriteOptions& write_options,
     do {
       const int64_t leftover = kBlockSize - block_offset_;
       assert(leftover >= 0);
-      if (leftover < header_size_) {
+      if (leftover < header_size) {
         // Switch to a new block
         if (leftover > 0) {
           // Fill the trailer (literal below relies on kHeaderSize and
           // kRecyclableHeaderSize being <= 11)
-          assert(header_size_ <= 11);
+          assert(header_size <= 11);
           s = dest_->Append(opts,
                             Slice("\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00",
                                   static_cast<size_t>(leftover)),
@@ -118,9 +120,9 @@ IOStatus Writer::AddRecord(const WriteOptions& write_options,
       }
 
       // Invariant: we never leave < header_size bytes in a block.
-      assert(static_cast<int64_t>(kBlockSize - block_offset_) >= header_size_);
+      assert(static_cast<int64_t>(kBlockSize - block_offset_) >= header_size);
 
-      const size_t avail = kBlockSize - block_offset_ - header_size_;
+      const size_t avail = kBlockSize - block_offset_ - header_size;
 
       // Compress the record if compression is enabled.
       // Compress() is called at least once (compress_start=true) and after the
@@ -201,7 +203,8 @@ IOStatus Writer::AddCompressionTypeRecord(const WriteOptions& write_options) {
       }
     }
     // Initialize fields required for compression
-    const size_t max_output_buffer_len = kBlockSize - header_size_;
+    const size_t max_output_buffer_len =
+        kBlockSize - (recycle_log_files_ ? kRecyclableHeaderSize : kHeaderSize);
     CompressionOptions opts;
     constexpr uint32_t compression_format_version = 2;
     compress_ = StreamingCompress::Create(compression_type_, opts,
@@ -241,25 +244,6 @@ IOStatus Writer::MaybeAddUserDefinedTimestampSizeRecord(
   record.EncodeTo(&encoded);
   RecordType type = recycle_log_files_ ? kRecyclableUserDefinedTimestampSizeType
                                        : kUserDefinedTimestampSizeType;
-
-  // If there's not enough space for this record, switch to a new block.
-  const int64_t leftover = kBlockSize - block_offset_;
-  if (leftover < header_size_ + (int)encoded.size()) {
-    IOOptions opts;
-    IOStatus s = WritableFileWriter::PrepareIOOptions(write_options, opts);
-    if (!s.ok()) {
-      return s;
-    }
-
-    std::vector<char> trailer(leftover, '\x00');
-    s = dest_->Append(opts, Slice(trailer.data(), trailer.size()));
-    if (!s.ok()) {
-      return s;
-    }
-
-    block_offset_ = 0;
-  }
-
   return EmitPhysicalRecord(write_options, type, encoded.data(),
                             encoded.size());
 }

db/log_writer.h

@@ -109,14 +109,11 @@ class Writer {
 
   bool BufferIsEmpty();
 
-  size_t TEST_block_offset() const { return block_offset_; }
-
  private:
   std::unique_ptr<WritableFileWriter> dest_;
   size_t block_offset_;  // Current offset in block
   uint64_t log_number_;
   bool recycle_log_files_;
-  int header_size_;
 
   // crc32c values for all supported record types.  These are
   // pre-computed to reduce the overhead of computing the crc of the

db/memtable.cc

@@ -928,10 +928,21 @@ struct Saver {
 }  // anonymous namespace
 
 static bool SaveValue(void* arg, const char* entry) {
+  TEST_SYNC_POINT_CALLBACK("Memtable::SaveValue:Begin:entry", &entry);
   Saver* s = static_cast<Saver*>(arg);
   assert(s != nullptr);
   assert(!s->value || !s->columns);
 
+  if (s->protection_bytes_per_key > 0) {
+    *(s->status) = MemTable::VerifyEntryChecksum(
+        entry, s->protection_bytes_per_key, s->allow_data_in_errors);
+    if (!s->status->ok()) {
+      ROCKS_LOG_ERROR(s->logger, "In SaveValue: %s", s->status->getState());
+      // Memtable entry corrupted
+      return false;
+    }
+  }
+
   MergeContext* merge_context = s->merge_context;
   SequenceNumber max_covering_tombstone_seq = s->max_covering_tombstone_seq;
   const MergeOperator* merge_operator = s->merge_operator;
@@ -954,22 +965,6 @@ static bool SaveValue(void* arg, const char* entry) {
   if (user_comparator->EqualWithoutTimestamp(user_key_slice,
                                              s->key->user_key())) {
     // Correct user key
-    TEST_SYNC_POINT_CALLBACK("Memtable::SaveValue:Found:entry", &entry);
-    std::unique_ptr<ReadLock> read_lock;
-    if (s->inplace_update_support) {
-      read_lock.reset(new ReadLock(s->mem->GetLock(s->key->user_key())));
-    }
-
-    if (s->protection_bytes_per_key > 0) {
-      *(s->status) = MemTable::VerifyEntryChecksum(
-          entry, s->protection_bytes_per_key, s->allow_data_in_errors);
-      if (!s->status->ok()) {
-        ROCKS_LOG_ERROR(s->logger, "In SaveValue: %s", s->status->getState());
-        // Memtable entry corrupted
-        return false;
-      }
-    }
-
     const uint64_t tag = DecodeFixed64(key_ptr + key_length - 8);
     ValueType type;
     SequenceNumber seq;
@@ -1040,6 +1035,10 @@ static bool SaveValue(void* arg, const char* entry) {
           return false;
         }
 
+        if (s->inplace_update_support) {
+          s->mem->GetLock(s->key->user_key())->ReadLock();
+        }
+
         Slice v = GetLengthPrefixedSlice(key_ptr + key_length);
 
         *(s->status) = Status::OK();
@@ -1050,6 +1049,10 @@ static bool SaveValue(void* arg, const char* entry) {
           s->columns->SetPlainValue(v);
         }
 
+        if (s->inplace_update_support) {
+          s->mem->GetLock(s->key->user_key())->ReadUnlock();
+        }
+
         *(s->found_final_value) = true;
         *(s->is_blob_index) = true;
 
@@ -1057,6 +1060,10 @@ static bool SaveValue(void* arg, const char* entry) {
       }
       case kTypeValue:
       case kTypeValuePreferredSeqno: {
+        if (s->inplace_update_support) {
+          s->mem->GetLock(s->key->user_key())->ReadLock();
+        }
+
         Slice v = GetLengthPrefixedSlice(key_ptr + key_length);
 
         if (type == kTypeValuePreferredSeqno) {
@@ -1093,6 +1100,10 @@ static bool SaveValue(void* arg, const char* entry) {
           s->columns->SetPlainValue(v);
         }
 
+        if (s->inplace_update_support) {
+          s->mem->GetLock(s->key->user_key())->ReadUnlock();
+        }
+
         *(s->found_final_value) = true;
 
         if (s->is_blob_index != nullptr) {
@@ -1102,6 +1113,10 @@ static bool SaveValue(void* arg, const char* entry) {
         return false;
       }
       case kTypeWideColumnEntity: {
+        if (s->inplace_update_support) {
+          s->mem->GetLock(s->key->user_key())->ReadLock();
+        }
+
         Slice v = GetLengthPrefixedSlice(key_ptr + key_length);
 
         *(s->status) = Status::OK();
@@ -1143,6 +1158,10 @@ static bool SaveValue(void* arg, const char* entry) {
           *(s->status) = s->columns->SetWideColumnValue(v);
         }
 
+        if (s->inplace_update_support) {
+          s->mem->GetLock(s->key->user_key())->ReadUnlock();
+        }
+
         *(s->found_final_value) = true;
 
         if (s->is_blob_index != nullptr) {
@@ -1480,9 +1499,9 @@ Status MemTable::Update(SequenceNumber seq, ValueType value_type,
 
         // Update value, if new value size  <= previous value size
         if (new_size <= prev_size) {
-          WriteLock wl(GetLock(lkey.user_key()));
           char* p =
               EncodeVarint32(const_cast<char*>(key_ptr) + key_length, new_size);
+          WriteLock wl(GetLock(lkey.user_key()));
           memcpy(p, value.data(), value.size());
           assert((unsigned)((p + value.size()) - entry) ==
                  (unsigned)(VarintLength(key_length) + key_length +

db/version_edit_handler.cc

@@ -152,7 +152,7 @@ Status FileChecksumRetriever::ApplyVersionEdit(VersionEdit& edit,
 
 VersionEditHandler::VersionEditHandler(
     bool read_only, std::vector<ColumnFamilyDescriptor> column_families,
-    VersionSet* version_set, bool track_found_and_missing_files,
+    VersionSet* version_set, bool track_missing_files,
     bool no_error_if_files_missing, const std::shared_ptr<IOTracer>& io_tracer,
     const ReadOptions& read_options, bool skip_load_table_files,
     EpochNumberRequirement epoch_number_requirement)
@@ -160,7 +160,7 @@ VersionEditHandler::VersionEditHandler(
       read_only_(read_only),
       column_families_(std::move(column_families)),
       version_set_(version_set),
-      track_found_and_missing_files_(track_found_and_missing_files),
+      track_missing_files_(track_missing_files),
       no_error_if_files_missing_(no_error_if_files_missing),
       io_tracer_(io_tracer),
       skip_load_table_files_(skip_load_table_files),
@@ -500,8 +500,7 @@ ColumnFamilyData* VersionEditHandler::CreateCfAndInit(
   assert(builders_.find(cf_id) == builders_.end());
   builders_.emplace(cf_id,
                     VersionBuilderUPtr(new BaseReferencedVersionBuilder(cfd)));
-  if (track_found_and_missing_files_) {
-    cf_to_found_files_.emplace(cf_id, std::unordered_set<uint64_t>());
+  if (track_missing_files_) {
     cf_to_missing_files_.emplace(cf_id, std::unordered_set<uint64_t>());
     cf_to_missing_blob_files_high_.emplace(cf_id, kInvalidBlobFileNumber);
   }
@@ -514,11 +513,7 @@ ColumnFamilyData* VersionEditHandler::DestroyCfAndCleanup(
   auto builder_iter = builders_.find(cf_id);
   assert(builder_iter != builders_.end());
   builders_.erase(builder_iter);
-  if (track_found_and_missing_files_) {
-    auto found_files_iter = cf_to_found_files_.find(cf_id);
-    assert(found_files_iter != cf_to_found_files_.end());
-    cf_to_found_files_.erase(found_files_iter);
-
+  if (track_missing_files_) {
     auto missing_files_iter = cf_to_missing_files_.find(cf_id);
     assert(missing_files_iter != cf_to_missing_files_.end());
     cf_to_missing_files_.erase(missing_files_iter);
@@ -734,7 +729,7 @@ VersionEditHandlerPointInTime::VersionEditHandlerPointInTime(
     const ReadOptions& read_options,
     EpochNumberRequirement epoch_number_requirement)
     : VersionEditHandler(read_only, column_families, version_set,
-                         /*track_found_and_missing_files=*/true,
+                         /*track_missing_files=*/true,
                          /*no_error_if_files_missing=*/true, io_tracer,
                          read_options, epoch_number_requirement) {}
 
@@ -829,12 +824,6 @@ void VersionEditHandlerPointInTime::CheckIterationResult(
 
         version_set_->AppendVersion(cfd, v_iter->second);
         versions_.erase(v_iter);
-        // Let's clear found_files, since any files in that are part of the
-        // installed Version. Any files that got obsoleted would have already
-        // been moved to intermediate_files_
-        auto found_files_iter = cf_to_found_files_.find(cfd->GetID());
-        assert(found_files_iter != cf_to_found_files_.end());
-        found_files_iter->second.clear();
       }
     }
   } else {
@@ -865,16 +854,10 @@ ColumnFamilyData* VersionEditHandlerPointInTime::DestroyCfAndCleanup(
 
 Status VersionEditHandlerPointInTime::MaybeCreateVersion(
     const VersionEdit& edit, ColumnFamilyData* cfd, bool force_create_version) {
-  TEST_SYNC_POINT("VersionEditHandlerPointInTime::MaybeCreateVersion:Begin1");
-  TEST_SYNC_POINT("VersionEditHandlerPointInTime::MaybeCreateVersion:Begin2");
   assert(cfd != nullptr);
   if (!force_create_version) {
     assert(edit.GetColumnFamily() == cfd->GetID());
   }
-  auto found_files_iter = cf_to_found_files_.find(cfd->GetID());
-  assert(found_files_iter != cf_to_found_files_.end());
-  std::unordered_set<uint64_t>& found_files = found_files_iter->second;
-
   auto missing_files_iter = cf_to_missing_files_.find(cfd->GetID());
   assert(missing_files_iter != cf_to_missing_files_.end());
   std::unordered_set<uint64_t>& missing_files = missing_files_iter->second;
@@ -906,18 +889,6 @@ Status VersionEditHandlerPointInTime::MaybeCreateVersion(
     auto fiter = missing_files.find(file_num);
     if (fiter != missing_files.end()) {
       missing_files.erase(fiter);
-    } else {
-      fiter = found_files.find(file_num);
-      // Only mark new files added during this catchup attempt for deletion.
-      // These files were never installed in VersionStorageInfo.
-      // Already referenced files that are deleted by a VersionEdit will
-      // be added to the VersionStorageInfo's obsolete files when the old
-      // version is dereferenced.
-      if (fiter != found_files.end()) {
-        intermediate_files_.emplace_back(
-            MakeTableFileName(cfd->ioptions()->cf_paths[0].path, file_num));
-        found_files.erase(fiter);
-      }
     }
   }
 
@@ -933,14 +904,9 @@ Status VersionEditHandlerPointInTime::MaybeCreateVersion(
     s = VerifyFile(cfd, fpath, level, meta);
     if (s.IsPathNotFound() || s.IsNotFound() || s.IsCorruption()) {
       missing_files.insert(file_num);
-      if (s.IsCorruption()) {
-        found_files.insert(file_num);
-      }
       s = Status::OK();
     } else if (!s.ok()) {
       break;
-    } else {
-      found_files.insert(file_num);
     }
   }
 

db/version_edit_handler.h

@@ -104,7 +104,7 @@ using VersionBuilderUPtr = std::unique_ptr<BaseReferencedVersionBuilder>;
 // To use this class and its subclasses,
 // 1. Create an object of VersionEditHandler or its subclasses.
 //    VersionEditHandler handler(read_only, column_families, version_set,
-//                               track_found_and_missing_files,
+//                               track_missing_files,
 //                               no_error_if_files_missing);
 // 2. Status s = handler.Iterate(reader, &db_id);
 // 3. Check s and handle possible errors.
@@ -116,17 +116,16 @@ class VersionEditHandler : public VersionEditHandlerBase {
   explicit VersionEditHandler(
       bool read_only,
       const std::vector<ColumnFamilyDescriptor>& column_families,
-      VersionSet* version_set, bool track_found_and_missing_files,
+      VersionSet* version_set, bool track_missing_files,
       bool no_error_if_files_missing,
       const std::shared_ptr<IOTracer>& io_tracer,
       const ReadOptions& read_options,
       EpochNumberRequirement epoch_number_requirement =
           EpochNumberRequirement::kMustPresent)
-      : VersionEditHandler(read_only, column_families, version_set,
-                           track_found_and_missing_files,
-                           no_error_if_files_missing, io_tracer, read_options,
-                           /*skip_load_table_files=*/false,
-                           epoch_number_requirement) {}
+      : VersionEditHandler(
+            read_only, column_families, version_set, track_missing_files,
+            no_error_if_files_missing, io_tracer, read_options,
+            /*skip_load_table_files=*/false, epoch_number_requirement) {}
 
   ~VersionEditHandler() override {}
 
@@ -145,7 +144,7 @@ class VersionEditHandler : public VersionEditHandlerBase {
  protected:
   explicit VersionEditHandler(
       bool read_only, std::vector<ColumnFamilyDescriptor> column_families,
-      VersionSet* version_set, bool track_found_and_missing_files,
+      VersionSet* version_set, bool track_missing_files,
       bool no_error_if_files_missing,
       const std::shared_ptr<IOTracer>& io_tracer,
       const ReadOptions& read_options, bool skip_load_table_files,
@@ -196,8 +195,7 @@ class VersionEditHandler : public VersionEditHandlerBase {
   // by subsequent manifest records, Recover() will return failure status.
   std::unordered_map<uint32_t, std::string> column_families_not_found_;
   VersionEditParams version_edit_params_;
-  const bool track_found_and_missing_files_;
-  std::unordered_map<uint32_t, std::unordered_set<uint64_t>> cf_to_found_files_;
+  const bool track_missing_files_;
   std::unordered_map<uint32_t, std::unordered_set<uint64_t>>
       cf_to_missing_files_;
   std::unordered_map<uint32_t, uint64_t> cf_to_missing_blob_files_high_;
@@ -275,8 +273,6 @@ class VersionEditHandlerPointInTime : public VersionEditHandler {
 
   bool in_atomic_group_ = false;
 
-  std::vector<std::string> intermediate_files_;
-
  private:
   bool AtomicUpdateVersionsCompleted();
   bool AtomicUpdateVersionsContains(uint32_t cfid);
@@ -314,10 +310,6 @@ class ManifestTailer : public VersionEditHandlerPointInTime {
     return cfds_changed_;
   }
 
-  std::vector<std::string>& GetIntermediateFiles() {
-    return intermediate_files_;
-  }
-
  protected:
   Status Initialize() override;
 
@@ -350,7 +342,7 @@ class DumpManifestHandler : public VersionEditHandler {
                       bool json)
       : VersionEditHandler(
             /*read_only=*/true, column_families, version_set,
-            /*track_found_and_missing_files=*/false,
+            /*track_missing_files=*/false,
             /*no_error_if_files_missing=*/false, io_tracer, read_options,
             /*skip_load_table_files=*/true),
         verbose_(verbose),

db/version_set.cc

@@ -3130,10 +3130,6 @@ bool Version::MaybeInitializeFileMetaData(const ReadOptions& read_options,
   file_meta->raw_value_size = tp->raw_value_size;
   file_meta->raw_key_size = tp->raw_key_size;
   file_meta->num_range_deletions = tp->num_range_deletions;
-  // Ensure new invariants on old files
-  file_meta->num_deletions =
-      std::max(tp->num_deletions, tp->num_range_deletions);
-  file_meta->num_entries = std::max(tp->num_entries, tp->num_deletions);
   return true;
 }
 
@@ -3145,7 +3141,6 @@ void VersionStorageInfo::UpdateAccumulatedStats(FileMetaData* file_meta) {
   accumulated_file_size_ += file_meta->fd.GetFileSize();
   accumulated_raw_key_size_ += file_meta->raw_key_size;
   accumulated_raw_value_size_ += file_meta->raw_value_size;
-  assert(file_meta->num_entries >= file_meta->num_deletions);
   accumulated_num_non_deletions_ +=
       file_meta->num_entries - file_meta->num_deletions;
   accumulated_num_deletions_ += file_meta->num_deletions;
@@ -4617,27 +4612,25 @@ uint64_t VersionStorageInfo::GetMaxEpochNumberOfFiles() const {
   return max_epoch_number;
 }
 
-void VersionStorageInfo::RecoverEpochNumbers(ColumnFamilyData* cfd,
-                                             bool restart_epoch, bool force) {
-  if (restart_epoch) {
-    cfd->ResetNextEpochNumber();
+void VersionStorageInfo::RecoverEpochNumbers(ColumnFamilyData* cfd) {
+  cfd->ResetNextEpochNumber();
 
-    bool reserve_epoch_num_for_file_ingested_behind =
-        cfd->ioptions()->allow_ingest_behind;
-    if (reserve_epoch_num_for_file_ingested_behind) {
-      uint64_t reserved_epoch_number = cfd->NewEpochNumber();
-      assert(reserved_epoch_number ==
-             kReservedEpochNumberForFileIngestedBehind);
-      ROCKS_LOG_INFO(cfd->ioptions()->info_log.get(),
-                     "[%s]CF has reserved epoch number %" PRIu64
-                     " for files ingested "
-                     "behind since `Options::allow_ingest_behind` is true",
-                     cfd->GetName().c_str(), reserved_epoch_number);
-    }
+  bool reserve_epoch_num_for_file_ingested_behind =
+      cfd->ioptions()->allow_ingest_behind;
+  if (reserve_epoch_num_for_file_ingested_behind) {
+    uint64_t reserved_epoch_number = cfd->NewEpochNumber();
+    assert(reserved_epoch_number == kReservedEpochNumberForFileIngestedBehind);
+    ROCKS_LOG_INFO(cfd->ioptions()->info_log.get(),
+                   "[%s]CF has reserved epoch number %" PRIu64
+                   " for files ingested "
+                   "behind since `Options::allow_ingest_behind` is true",
+                   cfd->GetName().c_str(), reserved_epoch_number);
   }
 
-  bool missing_epoch_number = HasMissingEpochNumber();
-  if (missing_epoch_number || force) {
+  if (HasMissingEpochNumber()) {
+    assert(epoch_number_requirement_ == EpochNumberRequirement::kMightMissing);
+    assert(num_levels_ >= 1);
+
     for (int level = num_levels_ - 1; level >= 1; --level) {
       auto& files_at_level = files_[level];
       if (files_at_level.empty()) {
@@ -4648,19 +4641,17 @@ void VersionStorageInfo::RecoverEpochNumbers(ColumnFamilyData* cfd,
         f->epoch_number = next_epoch_number;
       }
     }
+
     for (auto file_meta_iter = files_[0].rbegin();
          file_meta_iter != files_[0].rend(); file_meta_iter++) {
       FileMetaData* f = *file_meta_iter;
       f->epoch_number = cfd->NewEpochNumber();
     }
-    if (missing_epoch_number) {
-      assert(epoch_number_requirement_ ==
-             EpochNumberRequirement::kMightMissing);
-      ROCKS_LOG_WARN(cfd->ioptions()->info_log.get(),
-                     "[%s]CF's epoch numbers are inferred based on seqno",
-                     cfd->GetName().c_str());
-      epoch_number_requirement_ = EpochNumberRequirement::kMustPresent;
-    }
+
+    ROCKS_LOG_WARN(cfd->ioptions()->info_log.get(),
+                   "[%s]CF's epoch numbers are inferred based on seqno",
+                   cfd->GetName().c_str());
+    epoch_number_requirement_ = EpochNumberRequirement::kMustPresent;
   } else {
     assert(epoch_number_requirement_ == EpochNumberRequirement::kMustPresent);
     cfd->SetNextEpochNumber(
@@ -6063,8 +6054,8 @@ Status VersionSet::Recover(
                        true /* checksum */, 0 /* log_number */);
     VersionEditHandler handler(
         read_only, column_families, const_cast<VersionSet*>(this),
-        /*track_found_and_missing_files=*/false, no_error_if_files_missing,
-        io_tracer_, read_options, EpochNumberRequirement::kMightMissing);
+        /*track_missing_files=*/false, no_error_if_files_missing, io_tracer_,
+        read_options, EpochNumberRequirement::kMightMissing);
     handler.Iterate(reader, &log_read_status);
     s = handler.status();
     if (s.ok()) {
@@ -7439,8 +7430,7 @@ Status ReactiveVersionSet::ReadAndApply(
     InstrumentedMutex* mu,
     std::unique_ptr<log::FragmentBufferedReader>* manifest_reader,
     Status* manifest_read_status,
-    std::unordered_set<ColumnFamilyData*>* cfds_changed,
-    std::vector<std::string>* files_to_delete) {
+    std::unordered_set<ColumnFamilyData*>* cfds_changed) {
   assert(manifest_reader != nullptr);
   assert(cfds_changed != nullptr);
   mu->AssertHeld();
@@ -7457,9 +7447,6 @@ Status ReactiveVersionSet::ReadAndApply(
   if (s.ok()) {
     *cfds_changed = std::move(manifest_tailer_->GetUpdatedColumnFamilies());
   }
-  if (files_to_delete) {
-    *files_to_delete = std::move(manifest_tailer_->GetIntermediateFiles());
-  }
 
   return s;
 }

db/version_set.h

@@ -341,15 +341,7 @@ class VersionStorageInfo {
       EpochNumberRequirement epoch_number_requirement) {
     epoch_number_requirement_ = epoch_number_requirement;
   }
-  // Ensure all files have epoch number set.
-  // If there is a file missing epoch number, all files' epoch number will be
-  // reset according to CF's epoch number. Otherwise, the CF will be updated
-  // with the max epoch number of the files.
-  //
-  // @param restart_epoch This CF's epoch number will be reset to start from 0.
-  // @param force Force resetting all files' epoch number.
-  void RecoverEpochNumbers(ColumnFamilyData* cfd, bool restart_epoch = true,
-                           bool force = false);
+  void RecoverEpochNumbers(ColumnFamilyData* cfd);
 
   class FileLocation {
    public:
@@ -1741,8 +1733,7 @@ class ReactiveVersionSet : public VersionSet {
       InstrumentedMutex* mu,
       std::unique_ptr<log::FragmentBufferedReader>* manifest_reader,
       Status* manifest_read_status,
-      std::unordered_set<ColumnFamilyData*>* cfds_changed,
-      std::vector<std::string>* files_to_delete);
+      std::unordered_set<ColumnFamilyData*>* cfds_changed);
 
   Status Recover(const std::vector<ColumnFamilyDescriptor>& column_families,
                  std::unique_ptr<log::FragmentBufferedReader>* manifest_reader,

db/version_set_test.cc

@@ -2742,8 +2742,7 @@ TEST_F(VersionSetAtomicGroupTest,
   std::unordered_set<ColumnFamilyData*> cfds_changed;
   mu.Lock();
   EXPECT_OK(reactive_versions_->ReadAndApply(
-      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed,
-      /*files_to_delete=*/nullptr));
+      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed));
   mu.Unlock();
   EXPECT_TRUE(first_in_atomic_group_);
   EXPECT_TRUE(last_in_atomic_group_);
@@ -2798,8 +2797,7 @@ TEST_F(VersionSetAtomicGroupTest,
   std::unordered_set<ColumnFamilyData*> cfds_changed;
   mu.Lock();
   EXPECT_OK(reactive_versions_->ReadAndApply(
-      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed,
-      /*files_to_delete=*/nullptr));
+      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed));
   mu.Unlock();
   // Reactive version set should be empty now.
   EXPECT_TRUE(reactive_versions_->TEST_read_edits_in_atomic_group() == 0);
@@ -2828,8 +2826,7 @@ TEST_F(VersionSetAtomicGroupTest,
   std::unordered_set<ColumnFamilyData*> cfds_changed;
   mu.Lock();
   EXPECT_OK(reactive_versions_->ReadAndApply(
-      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed,
-      /*files_to_delete=*/nullptr));
+      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed));
   mu.Unlock();
   EXPECT_TRUE(first_in_atomic_group_);
   EXPECT_FALSE(last_in_atomic_group_);
@@ -2885,8 +2882,7 @@ TEST_F(VersionSetAtomicGroupTest,
   AddNewEditsToLog(kAtomicGroupSize);
   mu.Lock();
   EXPECT_NOK(reactive_versions_->ReadAndApply(
-      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed,
-      /*files_to_delete=*/nullptr));
+      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed));
   mu.Unlock();
   EXPECT_EQ(edits_[kAtomicGroupSize / 2].DebugString(),
             corrupted_edit_.DebugString());
@@ -2936,8 +2932,7 @@ TEST_F(VersionSetAtomicGroupTest,
   AddNewEditsToLog(kAtomicGroupSize);
   mu.Lock();
   EXPECT_NOK(reactive_versions_->ReadAndApply(
-      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed,
-      /*files_to_delete=*/nullptr));
+      &mu, &manifest_reader, manifest_reader_status.get(), &cfds_changed));
   mu.Unlock();
   EXPECT_EQ(edits_[1].DebugString(),
             edit_with_incorrect_group_size_.DebugString());

db/wide/db_wide_basic_test.cc

@@ -1794,144 +1794,6 @@ TEST_F(DBWideBasicTest, PinnableWideColumnsMove) {
   test_move(/* fill_cache*/ true);
 }
 
-TEST_F(DBWideBasicTest, SanityChecks) {
-  constexpr char foo[] = "foo";
-  constexpr char bar[] = "bar";
-  constexpr size_t num_keys = 2;
-
-  {
-    constexpr ColumnFamilyHandle* column_family = nullptr;
-    PinnableWideColumns columns;
-    ASSERT_TRUE(db_->GetEntity(ReadOptions(), column_family, foo, &columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    constexpr PinnableWideColumns* columns = nullptr;
-    ASSERT_TRUE(
-        db_->GetEntity(ReadOptions(), db_->DefaultColumnFamily(), foo, columns)
-            .IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kGet;
-
-    PinnableWideColumns columns;
-    ASSERT_TRUE(
-        db_->GetEntity(read_options, db_->DefaultColumnFamily(), foo, &columns)
-            .IsInvalidArgument());
-  }
-
-  {
-    constexpr ColumnFamilyHandle* column_family = nullptr;
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    db_->MultiGetEntity(ReadOptions(), column_family, num_keys, keys.data(),
-                        results.data(), statuses.data());
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    constexpr Slice* keys = nullptr;
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    db_->MultiGetEntity(ReadOptions(), db_->DefaultColumnFamily(), num_keys,
-                        keys, results.data(), statuses.data());
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    constexpr PinnableWideColumns* results = nullptr;
-    std::array<Status, num_keys> statuses;
-
-    db_->MultiGetEntity(ReadOptions(), db_->DefaultColumnFamily(), num_keys,
-                        keys.data(), results, statuses.data());
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kMultiGet;
-
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    db_->MultiGetEntity(read_options, db_->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data());
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    constexpr ColumnFamilyHandle** column_families = nullptr;
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    db_->MultiGetEntity(ReadOptions(), num_keys, column_families, keys.data(),
-                        results.data(), statuses.data());
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    std::array<ColumnFamilyHandle*, num_keys> column_families{
-        {db_->DefaultColumnFamily(), db_->DefaultColumnFamily()}};
-    constexpr Slice* keys = nullptr;
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    db_->MultiGetEntity(ReadOptions(), num_keys, column_families.data(), keys,
-                        results.data(), statuses.data());
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    std::array<ColumnFamilyHandle*, num_keys> column_families{
-        {db_->DefaultColumnFamily(), db_->DefaultColumnFamily()}};
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    constexpr PinnableWideColumns* results = nullptr;
-    std::array<Status, num_keys> statuses;
-
-    db_->MultiGetEntity(ReadOptions(), num_keys, column_families.data(),
-                        keys.data(), results, statuses.data());
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kMultiGet;
-
-    std::array<ColumnFamilyHandle*, num_keys> column_families{
-        {db_->DefaultColumnFamily(), db_->DefaultColumnFamily()}};
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    db_->MultiGetEntity(read_options, num_keys, column_families.data(),
-                        keys.data(), results.data(), statuses.data());
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-}
-
 }  // namespace ROCKSDB_NAMESPACE
 
 int main(int argc, char** argv) {

db_stress_tool/batched_ops_stress.cc

@@ -54,12 +54,7 @@ class BatchedOpsStressTest : public StressTest {
       const std::string v = value_body + num;
       if (FLAGS_use_put_entity_one_in > 0 &&
           (value_base % FLAGS_use_put_entity_one_in) == 0) {
-        if (FLAGS_use_attribute_group) {
-          status =
-              batch.PutEntity(k, GenerateAttributeGroups({cfh}, value_base, v));
-        } else {
-          status = batch.PutEntity(cfh, k, GenerateWideColumns(value_base, v));
-        }
+        status = batch.PutEntity(cfh, k, GenerateWideColumns(value_base, v));
       } else if (FLAGS_use_timed_put_one_in > 0 &&
                  ((value_base + kLargePrimeForCommonFactorSkew) %
                   FLAGS_use_timed_put_one_in) == 0) {
@@ -305,30 +300,15 @@ class BatchedOpsStressTest : public StressTest {
 
     constexpr size_t num_keys = 10;
 
-    std::array<PinnableWideColumns, num_keys> column_results;
-    std::array<PinnableAttributeGroups, num_keys> attribute_group_results;
-
-    std::string error_msg_header = FLAGS_use_attribute_group
-                                       ? "GetEntity (AttributeGroup) error"
-                                       : "GetEntity error";
+    std::array<PinnableWideColumns, num_keys> results;
 
     for (size_t i = 0; i < num_keys; ++i) {
       const std::string key = std::to_string(i) + key_suffix;
 
-      Status s;
-      if (FLAGS_use_attribute_group) {
-        attribute_group_results[i].emplace_back(cfh);
-        s = db_->GetEntity(read_opts_copy, key, &attribute_group_results[i]);
-        if (s.ok()) {
-          s = attribute_group_results[i].back().status();
-        }
-      } else {
-        s = db_->GetEntity(read_opts_copy, cfh, key, &column_results[i]);
-      }
+      const Status s = db_->GetEntity(read_opts_copy, cfh, key, &results[i]);
 
       if (!s.ok() && !s.IsNotFound()) {
-        fprintf(stderr, "%s: %s\n", error_msg_header.c_str(),
-                s.ToString().c_str());
+        fprintf(stderr, "GetEntity error: %s\n", s.ToString().c_str());
         thread->stats.AddErrors(1);
       } else if (s.IsNotFound()) {
         thread->stats.AddGets(1, 0);
@@ -337,20 +317,14 @@ class BatchedOpsStressTest : public StressTest {
       }
     }
 
-    const WideColumns& columns_to_compare =
-        FLAGS_use_attribute_group ? attribute_group_results[0].front().columns()
-                                  : column_results[0].columns();
+    for (size_t i = 0; i < num_keys; ++i) {
+      const WideColumns& columns = results[i].columns();
 
-    for (size_t i = 1; i < num_keys; ++i) {
-      const WideColumns& columns =
-          FLAGS_use_attribute_group
-              ? attribute_group_results[i].front().columns()
-              : column_results[i].columns();
-
-      if (!CompareColumns(columns_to_compare, columns)) {
-        fprintf(stderr, "%s: inconsistent entities for key %s: %s, %s\n",
-                error_msg_header.c_str(), StringToHex(key_suffix).c_str(),
-                WideColumnsToHex(columns_to_compare).c_str(),
+      if (!CompareColumns(results[0].columns(), columns)) {
+        fprintf(stderr,
+                "GetEntity error: inconsistent entities for key %s: %s, %s\n",
+                StringToHex(key_suffix).c_str(),
+                WideColumnsToHex(results[0].columns()).c_str(),
                 WideColumnsToHex(columns).c_str());
       }
 
@@ -364,18 +338,20 @@ class BatchedOpsStressTest : public StressTest {
 
           if (value.empty() || value[value.size() - 1] != expected) {
             fprintf(stderr,
-                    "%s: incorrect column value for key "
+                    "GetEntity error: incorrect column value for key "
                     "%s, entity %s, column value %s, expected %c\n",
-                    error_msg_header.c_str(), StringToHex(key_suffix).c_str(),
+                    StringToHex(key_suffix).c_str(),
                     WideColumnsToHex(columns).c_str(),
                     value.ToString(/* hex */ true).c_str(), expected);
           }
         }
 
         if (!VerifyWideColumns(columns)) {
-          fprintf(stderr, "%s: inconsistent columns for key %s, entity %s\n",
-                  error_msg_header.c_str(), StringToHex(key_suffix).c_str(),
-                  WideColumnsToHex(columns).c_str());
+          fprintf(
+              stderr,
+              "GetEntity error: inconsistent columns for key %s, entity %s\n",
+              StringToHex(key_suffix).c_str(),
+              WideColumnsToHex(columns).c_str());
         }
       }
     }
@@ -409,140 +385,66 @@ class BatchedOpsStressTest : public StressTest {
 
       std::array<std::string, num_prefixes> keys;
       std::array<Slice, num_prefixes> key_slices;
+      std::array<PinnableWideColumns, num_prefixes> results;
+      std::array<Status, num_prefixes> statuses;
 
       for (size_t j = 0; j < num_prefixes; ++j) {
         keys[j] = std::to_string(j) + key_suffix;
         key_slices[j] = keys[j];
       }
 
-      if (FLAGS_use_attribute_group) {
-        // AttributeGroup MultiGetEntity verification
+      db_->MultiGetEntity(read_opts_copy, cfh, num_prefixes, key_slices.data(),
+                          results.data(), statuses.data());
 
-        std::vector<PinnableAttributeGroups> results;
-        results.reserve(num_prefixes);
-        for (size_t j = 0; j < num_prefixes; ++j) {
-          PinnableAttributeGroups attribute_groups;
-          attribute_groups.emplace_back(cfh);
-          results.emplace_back(std::move(attribute_groups));
+      for (size_t j = 0; j < num_prefixes; ++j) {
+        const Status& s = statuses[j];
+
+        if (!s.ok() && !s.IsNotFound()) {
+          fprintf(stderr, "MultiGetEntity error: %s\n", s.ToString().c_str());
+          thread->stats.AddErrors(1);
+        } else if (s.IsNotFound()) {
+          thread->stats.AddGets(1, 0);
+        } else {
+          thread->stats.AddGets(1, 1);
         }
-        db_->MultiGetEntity(read_opts_copy, num_prefixes, key_slices.data(),
-                            results.data());
-
-        const WideColumns& cmp_columns = results[0][0].columns();
-
-        for (size_t j = 0; j < num_prefixes; ++j) {
-          const auto& attribute_groups = results[j];
-          assert(attribute_groups.size() == 1);
-          const Status& s = attribute_groups[0].status();
-          if (!s.ok() && !s.IsNotFound()) {
-            fprintf(stderr, "MultiGetEntity (AttributeGroup) error: %s\n",
-                    s.ToString().c_str());
-            thread->stats.AddErrors(1);
-          } else if (s.IsNotFound()) {
-            thread->stats.AddGets(1, 0);
-          } else {
-            thread->stats.AddGets(1, 1);
-          }
-
-          const WideColumns& columns = results[j][0].columns();
-          if (!CompareColumns(cmp_columns, columns)) {
-            fprintf(stderr,
-                    "MultiGetEntity (AttributeGroup) error: inconsistent "
-                    "entities for key %s: %s, "
-                    "%s\n",
-                    StringToHex(key_suffix).c_str(),
-                    WideColumnsToHex(cmp_columns).c_str(),
-                    WideColumnsToHex(columns).c_str());
-          }
-          if (!columns.empty()) {
-            // The last character of each column value should be 'j' as a
-            // decimal digit
-            const char expected = static_cast<char>('0' + j);
-
-            for (const auto& column : columns) {
-              const Slice& value = column.value();
-
-              if (value.empty() || value[value.size() - 1] != expected) {
-                fprintf(stderr,
-                        "MultiGetEntity (AttributeGroup) error: incorrect "
-                        "column value for key "
-                        "%s, entity %s, column value %s, expected %c\n",
-                        StringToHex(key_suffix).c_str(),
-                        WideColumnsToHex(columns).c_str(),
-                        value.ToString(/* hex */ true).c_str(), expected);
-              }
-            }
-
-            if (!VerifyWideColumns(columns)) {
-              fprintf(stderr,
-                      "MultiGetEntity (AttributeGroup) error: inconsistent "
-                      "columns for key %s, "
-                      "entity %s\n",
-                      StringToHex(key_suffix).c_str(),
-                      WideColumnsToHex(columns).c_str());
-            }
-          }
-        }
-      } else {
-        // Non-AttributeGroup MultiGetEntity verification
-
-        std::array<PinnableWideColumns, num_prefixes> results;
-        std::array<Status, num_prefixes> statuses;
-
-        db_->MultiGetEntity(read_opts_copy, cfh, num_prefixes,
-                            key_slices.data(), results.data(), statuses.data());
 
         const WideColumns& cmp_columns = results[0].columns();
+        const WideColumns& columns = results[j].columns();
 
-        for (size_t j = 0; j < num_prefixes; ++j) {
-          const Status& s = statuses[j];
+        if (!CompareColumns(cmp_columns, columns)) {
+          fprintf(stderr,
+                  "MultiGetEntity error: inconsistent entities for key %s: %s, "
+                  "%s\n",
+                  StringToHex(key_suffix).c_str(),
+                  WideColumnsToHex(cmp_columns).c_str(),
+                  WideColumnsToHex(columns).c_str());
+        }
 
-          if (!s.ok() && !s.IsNotFound()) {
-            fprintf(stderr, "MultiGetEntity error: %s\n", s.ToString().c_str());
-            thread->stats.AddErrors(1);
-          } else if (s.IsNotFound()) {
-            thread->stats.AddGets(1, 0);
-          } else {
-            thread->stats.AddGets(1, 1);
-          }
-          const WideColumns& columns = results[j].columns();
+        if (!columns.empty()) {
+          // The last character of each column value should be 'j' as a decimal
+          // digit
+          const char expected = static_cast<char>('0' + j);
 
-          if (!CompareColumns(cmp_columns, columns)) {
-            fprintf(
-                stderr,
-                "MultiGetEntity error: inconsistent entities for key %s: %s, "
-                "%s\n",
-                StringToHex(key_suffix).c_str(),
-                WideColumnsToHex(cmp_columns).c_str(),
-                WideColumnsToHex(columns).c_str());
-          }
+          for (const auto& column : columns) {
+            const Slice& value = column.value();
 
-          if (!columns.empty()) {
-            // The last character of each column value should be 'j' as a
-            // decimal digit
-            const char expected = static_cast<char>('0' + j);
-
-            for (const auto& column : columns) {
-              const Slice& value = column.value();
-
-              if (value.empty() || value[value.size() - 1] != expected) {
-                fprintf(stderr,
-                        "MultiGetEntity error: incorrect column value for key "
-                        "%s, entity %s, column value %s, expected %c\n",
-                        StringToHex(key_suffix).c_str(),
-                        WideColumnsToHex(columns).c_str(),
-                        value.ToString(/* hex */ true).c_str(), expected);
-              }
-            }
-
-            if (!VerifyWideColumns(columns)) {
+            if (value.empty() || value[value.size() - 1] != expected) {
               fprintf(stderr,
-                      "MultiGetEntity error: inconsistent columns for key %s, "
-                      "entity %s\n",
+                      "MultiGetEntity error: incorrect column value for key "
+                      "%s, entity %s, column value %s, expected %c\n",
                       StringToHex(key_suffix).c_str(),
-                      WideColumnsToHex(columns).c_str());
+                      WideColumnsToHex(columns).c_str(),
+                      value.ToString(/* hex */ true).c_str(), expected);
             }
           }
+
+          if (!VerifyWideColumns(columns)) {
+            fprintf(stderr,
+                    "MultiGetEntity error: inconsistent columns for key %s, "
+                    "entity %s\n",
+                    StringToHex(key_suffix).c_str(),
+                    WideColumnsToHex(columns).c_str());
+          }
         }
       }
     }

db_stress_tool/cf_consistency_stress.cc

@@ -37,35 +37,25 @@ class CfConsistencyStressTest : public StressTest {
     WriteBatch batch;
 
     Status status;
-    if (FLAGS_use_attribute_group && FLAGS_use_put_entity_one_in > 0 &&
-        (value_base % FLAGS_use_put_entity_one_in) == 0) {
-      std::vector<ColumnFamilyHandle*> cfhs;
-      cfhs.reserve(rand_column_families.size());
-      for (auto cf : rand_column_families) {
-        cfhs.push_back(column_families_[cf]);
-      }
-      status = batch.PutEntity(k, GenerateAttributeGroups(cfhs, value_base, v));
-    } else {
-      for (auto cf : rand_column_families) {
-        ColumnFamilyHandle* const cfh = column_families_[cf];
-        assert(cfh);
+    for (auto cf : rand_column_families) {
+      ColumnFamilyHandle* const cfh = column_families_[cf];
+      assert(cfh);
 
-        if (FLAGS_use_put_entity_one_in > 0 &&
-            (value_base % FLAGS_use_put_entity_one_in) == 0) {
-          status = batch.PutEntity(cfh, k, GenerateWideColumns(value_base, v));
-        } else if (FLAGS_use_timed_put_one_in > 0 &&
-                   ((value_base + kLargePrimeForCommonFactorSkew) %
-                    FLAGS_use_timed_put_one_in) == 0) {
-          uint64_t write_unix_time = GetWriteUnixTime(thread);
-          status = batch.TimedPut(cfh, k, v, write_unix_time);
-        } else if (FLAGS_use_merge) {
-          status = batch.Merge(cfh, k, v);
-        } else {
-          status = batch.Put(cfh, k, v);
-        }
-        if (!status.ok()) {
-          break;
-        }
+      if (FLAGS_use_put_entity_one_in > 0 &&
+          (value_base % FLAGS_use_put_entity_one_in) == 0) {
+        status = batch.PutEntity(cfh, k, GenerateWideColumns(value_base, v));
+      } else if (FLAGS_use_timed_put_one_in > 0 &&
+                 ((value_base + kLargePrimeForCommonFactorSkew) %
+                  FLAGS_use_timed_put_one_in) == 0) {
+        uint64_t write_unix_time = GetWriteUnixTime(thread);
+        status = batch.TimedPut(cfh, k, v, write_unix_time);
+      } else if (FLAGS_use_merge) {
+        status = batch.Merge(cfh, k, v);
+      } else {
+        status = batch.Put(cfh, k, v);
+      }
+      if (!status.ok()) {
+        break;
       }
     }
 
@@ -338,122 +328,57 @@ class CfConsistencyStressTest : public StressTest {
         }
 
         if (is_consistent) {
-          if (FLAGS_use_attribute_group) {
-            PinnableAttributeGroups result;
-            result.reserve(rand_column_families.size());
-            for (size_t i = 1; i < rand_column_families.size(); ++i) {
-              assert(rand_column_families[i] >= 0);
-              assert(rand_column_families[i] <
-                     static_cast<int>(column_families_.size()));
+          for (size_t i = 1; i < rand_column_families.size(); ++i) {
+            assert(rand_column_families[i] >= 0);
+            assert(rand_column_families[i] <
+                   static_cast<int>(column_families_.size()));
 
-              result.emplace_back(column_families_[rand_column_families[i]]);
+            PinnableWideColumns result;
+            s = db_->GetEntity(read_opts_copy,
+                               column_families_[rand_column_families[i]], key,
+                               &result);
+
+            if (!s.ok() && !s.IsNotFound()) {
+              break;
             }
-            s = db_->GetEntity(read_opts_copy, key, &result);
-            if (s.ok()) {
-              for (auto& attribute_group : result) {
-                s = attribute_group.status();
-                if (!s.ok() && !s.IsNotFound()) {
-                  break;
-                }
 
-                const bool found = s.ok();
+            const bool found = s.ok();
 
-                if (!cmp_found && found) {
-                  fprintf(
-                      stderr,
-                      "Non-AttributeGroup GetEntity returns different results "
-                      "than AttributeGroup GetEntity for key %s: CF %s "
-                      "returns not found, CF %s returns entity %s \n",
+            assert(!column_family_names_.empty());
+            assert(i < column_family_names_.size());
+
+            if (!cmp_found && found) {
+              fprintf(stderr,
+                      "GetEntity returns different results for key %s: CF %s "
+                      "returns not found, CF %s returns entity %s\n",
                       StringToHex(key).c_str(), column_family_names_[0].c_str(),
-                      attribute_group.column_family()->GetName().c_str(),
-                      WideColumnsToHex(attribute_group.columns()).c_str());
-                  is_consistent = false;
-                  break;
-                }
-                if (cmp_found && !found) {
-                  fprintf(
-                      stderr,
-                      "Non-AttributeGroup GetEntity returns different results "
-                      "than AttributeGroup GetEntity for key %s: CF %s "
-                      "returns entity %s, CF %s returns not found  \n",
+                      column_family_names_[i].c_str(),
+                      WideColumnsToHex(result.columns()).c_str());
+              is_consistent = false;
+              break;
+            }
+
+            if (cmp_found && !found) {
+              fprintf(stderr,
+                      "GetEntity returns different results for key %s: CF %s "
+                      "returns entity %s, CF %s returns not found\n",
                       StringToHex(key).c_str(), column_family_names_[0].c_str(),
                       WideColumnsToHex(cmp_result.columns()).c_str(),
-                      attribute_group.column_family()->GetName().c_str());
-                  is_consistent = false;
-                  break;
-                }
-                if (found &&
-                    attribute_group.columns() != cmp_result.columns()) {
-                  fprintf(
-                      stderr,
-                      "Non-AttributeGroup GetEntity returns different results "
-                      "than AttributeGroup GetEntity for key %s: CF %s "
+                      column_family_names_[i].c_str());
+              is_consistent = false;
+              break;
+            }
+
+            if (found && result != cmp_result) {
+              fprintf(stderr,
+                      "GetEntity returns different results for key %s: CF %s "
                       "returns entity %s, CF %s returns entity %s\n",
                       StringToHex(key).c_str(), column_family_names_[0].c_str(),
                       WideColumnsToHex(cmp_result.columns()).c_str(),
-                      attribute_group.column_family()->GetName().c_str(),
-                      WideColumnsToHex(attribute_group.columns()).c_str());
-                  is_consistent = false;
-                  break;
-                }
-              }
-            }
-          } else {
-            for (size_t i = 1; i < rand_column_families.size(); ++i) {
-              assert(rand_column_families[i] >= 0);
-              assert(rand_column_families[i] <
-                     static_cast<int>(column_families_.size()));
-
-              PinnableWideColumns result;
-              s = db_->GetEntity(read_opts_copy,
-                                 column_families_[rand_column_families[i]], key,
-                                 &result);
-
-              if (!s.ok() && !s.IsNotFound()) {
-                break;
-              }
-
-              const bool found = s.ok();
-
-              assert(!column_family_names_.empty());
-              assert(i < column_family_names_.size());
-
-              if (!cmp_found && found) {
-                fprintf(stderr,
-                        "GetEntity returns different results for key %s: CF %s "
-                        "returns not found, CF %s returns entity %s\n",
-                        StringToHex(key).c_str(),
-                        column_family_names_[0].c_str(),
-                        column_family_names_[i].c_str(),
-                        WideColumnsToHex(result.columns()).c_str());
-                is_consistent = false;
-                break;
-              }
-
-              if (cmp_found && !found) {
-                fprintf(stderr,
-                        "GetEntity returns different results for key %s: CF %s "
-                        "returns entity %s, CF %s returns not found\n",
-                        StringToHex(key).c_str(),
-                        column_family_names_[0].c_str(),
-                        WideColumnsToHex(cmp_result.columns()).c_str(),
-                        column_family_names_[i].c_str());
-                is_consistent = false;
-                break;
-              }
-
-              if (found && result != cmp_result) {
-                fprintf(stderr,
-                        "GetEntity returns different results for key %s: CF %s "
-                        "returns entity %s, CF %s returns entity %s\n",
-                        StringToHex(key).c_str(),
-                        column_family_names_[0].c_str(),
-                        WideColumnsToHex(cmp_result.columns()).c_str(),
-                        column_family_names_[i].c_str(),
-                        WideColumnsToHex(result.columns()).c_str());
-                is_consistent = false;
-                break;
-              }
+                      column_family_names_[i].c_str(),
+                      WideColumnsToHex(result.columns()).c_str());
+              is_consistent = false;
+              break;
             }
           }
         }
@@ -506,219 +431,94 @@ class CfConsistencyStressTest : public StressTest {
 
     const size_t num_keys = rand_keys.size();
 
-    if (FLAGS_use_attribute_group) {
-      // AttributeGroup MultiGetEntity verification
+    for (size_t i = 0; i < num_keys; ++i) {
+      const std::string key = Key(rand_keys[i]);
 
-      std::vector<PinnableAttributeGroups> results;
-      std::vector<Slice> key_slices;
-      std::vector<std::string> key_strs;
-      results.reserve(num_keys);
-      key_slices.reserve(num_keys);
-      key_strs.reserve(num_keys);
+      std::vector<Slice> key_slices(num_cfs, key);
+      std::vector<PinnableWideColumns> results(num_cfs);
+      std::vector<Status> statuses(num_cfs);
 
-      for (size_t i = 0; i < num_keys; ++i) {
-        key_strs.emplace_back(Key(rand_keys[i]));
-        key_slices.emplace_back(key_strs.back());
-        PinnableAttributeGroups attribute_groups;
-        for (auto* cfh : cfhs) {
-          attribute_groups.emplace_back(cfh);
-        }
-        results.emplace_back(std::move(attribute_groups));
-      }
-      db_->MultiGetEntity(read_opts_copy, num_keys, key_slices.data(),
-                          results.data());
+      db_->MultiGetEntity(read_opts_copy, num_cfs, cfhs.data(),
+                          key_slices.data(), results.data(), statuses.data());
 
       bool is_consistent = true;
 
-      for (size_t i = 0; i < num_keys; ++i) {
-        const auto& result = results[i];
-        const Status& cmp_s = result[0].status();
-        const WideColumns& cmp_columns = result[0].columns();
+      for (size_t j = 0; j < num_cfs; ++j) {
+        const Status& s = statuses[j];
+        const Status& cmp_s = statuses[0];
+        const WideColumns& columns = results[j].columns();
+        const WideColumns& cmp_columns = results[0].columns();
 
-        bool has_error = false;
-
-        for (size_t j = 0; j < num_cfs; ++j) {
-          const Status& s = result[j].status();
-          const WideColumns& columns = result[j].columns();
-          if (!s.ok() && !s.IsNotFound()) {
-            fprintf(stderr, "TestMultiGetEntity (AttributeGroup) error: %s\n",
-                    s.ToString().c_str());
-            thread->stats.AddErrors(1);
-            has_error = true;
-            break;
-          }
-
-          assert(cmp_s.ok() || cmp_s.IsNotFound());
-
-          if (s.IsNotFound()) {
-            if (cmp_s.ok()) {
-              fprintf(stderr,
-                      "MultiGetEntity (AttributeGroup) returns different "
-                      "results for key %s: CF %s "
-                      "returns entity %s, CF %s returns not found\n",
-                      key_slices[i].ToString(true).c_str(),
-                      column_family_names_[0].c_str(),
-                      WideColumnsToHex(cmp_columns).c_str(),
-                      column_family_names_[j].c_str());
-              is_consistent = false;
-              break;
-            }
-
-            continue;
-          }
-
-          assert(s.ok());
-          if (cmp_s.IsNotFound()) {
-            fprintf(stderr,
-                    "MultiGetEntity (AttributeGroup) returns different results "
-                    "for key %s: CF %s "
-                    "returns not found, CF %s returns entity %s\n",
-                    key_slices[i].ToString(true).c_str(),
-                    column_family_names_[0].c_str(),
-                    column_family_names_[j].c_str(),
-                    WideColumnsToHex(columns).c_str());
-            is_consistent = false;
-            break;
-          }
-
-          if (columns != cmp_columns) {
-            fprintf(stderr,
-                    "MultiGetEntity (AttributeGroup) returns different results "
-                    "for key %s: CF %s "
-                    "returns entity %s, CF %s returns entity %s\n",
-                    key_slices[i].ToString(true).c_str(),
-                    column_family_names_[0].c_str(),
-                    WideColumnsToHex(cmp_columns).c_str(),
-                    column_family_names_[j].c_str(),
-                    WideColumnsToHex(columns).c_str());
-            is_consistent = false;
-            break;
-          }
-
-          if (!VerifyWideColumns(columns)) {
-            fprintf(stderr,
-                    "MultiGetEntity (AttributeGroup) error: inconsistent "
-                    "columns for key %s, "
-                    "entity %s\n",
-                    key_slices[i].ToString(true).c_str(),
-                    WideColumnsToHex(columns).c_str());
-            is_consistent = false;
-            break;
-          }
-        }
-        if (has_error) {
-          break;
-        } else if (!is_consistent) {
-          fprintf(stderr,
-                  "TestMultiGetEntity (AttributeGroup) error: results are not "
-                  "consistent\n");
+        if (!s.ok() && !s.IsNotFound()) {
+          fprintf(stderr, "TestMultiGetEntity error: %s\n",
+                  s.ToString().c_str());
           thread->stats.AddErrors(1);
-          // Fail fast to preserve the DB state.
-          thread->shared->SetVerificationFailure();
           break;
-        } else if (cmp_s.ok()) {
-          thread->stats.AddGets(1, 1);
-        } else if (cmp_s.IsNotFound()) {
-          thread->stats.AddGets(1, 0);
+        }
+
+        assert(cmp_s.ok() || cmp_s.IsNotFound());
+
+        if (s.IsNotFound()) {
+          if (cmp_s.ok()) {
+            fprintf(
+                stderr,
+                "MultiGetEntity returns different results for key %s: CF %s "
+                "returns entity %s, CF %s returns not found\n",
+                StringToHex(key).c_str(), column_family_names_[0].c_str(),
+                WideColumnsToHex(cmp_columns).c_str(),
+                column_family_names_[j].c_str());
+            is_consistent = false;
+            break;
+          }
+
+          continue;
+        }
+
+        assert(s.ok());
+        if (cmp_s.IsNotFound()) {
+          fprintf(stderr,
+                  "MultiGetEntity returns different results for key %s: CF %s "
+                  "returns not found, CF %s returns entity %s\n",
+                  StringToHex(key).c_str(), column_family_names_[0].c_str(),
+                  column_family_names_[j].c_str(),
+                  WideColumnsToHex(columns).c_str());
+          is_consistent = false;
+          break;
+        }
+
+        if (columns != cmp_columns) {
+          fprintf(stderr,
+                  "MultiGetEntity returns different results for key %s: CF %s "
+                  "returns entity %s, CF %s returns entity %s\n",
+                  StringToHex(key).c_str(), column_family_names_[0].c_str(),
+                  WideColumnsToHex(cmp_columns).c_str(),
+                  column_family_names_[j].c_str(),
+                  WideColumnsToHex(columns).c_str());
+          is_consistent = false;
+          break;
+        }
+
+        if (!VerifyWideColumns(columns)) {
+          fprintf(stderr,
+                  "MultiGetEntity error: inconsistent columns for key %s, "
+                  "entity %s\n",
+                  StringToHex(key).c_str(), WideColumnsToHex(columns).c_str());
+          is_consistent = false;
+          break;
         }
       }
 
-    } else {
-      // Non-AttributeGroup MultiGetEntity verification
-
-      for (size_t i = 0; i < num_keys; ++i) {
-        const std::string key = Key(rand_keys[i]);
-
-        std::vector<Slice> key_slices(num_cfs, key);
-        std::vector<PinnableWideColumns> results(num_cfs);
-        std::vector<Status> statuses(num_cfs);
-
-        db_->MultiGetEntity(read_opts_copy, num_cfs, cfhs.data(),
-                            key_slices.data(), results.data(), statuses.data());
-
-        bool is_consistent = true;
-
-        const Status& cmp_s = statuses[0];
-        const WideColumns& cmp_columns = results[0].columns();
-
-        for (size_t j = 0; j < num_cfs; ++j) {
-          const Status& s = statuses[j];
-          const WideColumns& columns = results[j].columns();
-
-          if (!s.ok() && !s.IsNotFound()) {
-            fprintf(stderr, "TestMultiGetEntity error: %s\n",
-                    s.ToString().c_str());
-            thread->stats.AddErrors(1);
-            break;
-          }
-
-          assert(cmp_s.ok() || cmp_s.IsNotFound());
-
-          if (s.IsNotFound()) {
-            if (cmp_s.ok()) {
-              fprintf(
-                  stderr,
-                  "MultiGetEntity returns different results for key %s: CF %s "
-                  "returns entity %s, CF %s returns not found\n",
-                  StringToHex(key).c_str(), column_family_names_[0].c_str(),
-                  WideColumnsToHex(cmp_columns).c_str(),
-                  column_family_names_[j].c_str());
-              is_consistent = false;
-              break;
-            }
-
-            continue;
-          }
-
-          assert(s.ok());
-          if (cmp_s.IsNotFound()) {
-            fprintf(
-                stderr,
-                "MultiGetEntity returns different results for key %s: CF %s "
-                "returns not found, CF %s returns entity %s\n",
-                StringToHex(key).c_str(), column_family_names_[0].c_str(),
-                column_family_names_[j].c_str(),
-                WideColumnsToHex(columns).c_str());
-            is_consistent = false;
-            break;
-          }
-
-          if (columns != cmp_columns) {
-            fprintf(
-                stderr,
-                "MultiGetEntity returns different results for key %s: CF %s "
-                "returns entity %s, CF %s returns entity %s\n",
-                StringToHex(key).c_str(), column_family_names_[0].c_str(),
-                WideColumnsToHex(cmp_columns).c_str(),
-                column_family_names_[j].c_str(),
-                WideColumnsToHex(columns).c_str());
-            is_consistent = false;
-            break;
-          }
-
-          if (!VerifyWideColumns(columns)) {
-            fprintf(stderr,
-                    "MultiGetEntity error: inconsistent columns for key %s, "
-                    "entity %s\n",
-                    StringToHex(key).c_str(),
-                    WideColumnsToHex(columns).c_str());
-            is_consistent = false;
-            break;
-          }
-        }
-
-        if (!is_consistent) {
-          fprintf(stderr,
-                  "TestMultiGetEntity error: results are not consistent\n");
-          thread->stats.AddErrors(1);
-          // Fail fast to preserve the DB state.
-          thread->shared->SetVerificationFailure();
-          break;
-        } else if (statuses[0].ok()) {
-          thread->stats.AddGets(1, 1);
-        } else if (statuses[0].IsNotFound()) {
-          thread->stats.AddGets(1, 0);
-        }
+      if (!is_consistent) {
+        fprintf(stderr,
+                "TestMultiGetEntity error: results are not consistent\n");
+        thread->stats.AddErrors(1);
+        // Fail fast to preserve the DB state.
+        thread->shared->SetVerificationFailure();
+        break;
+      } else if (statuses[0].ok()) {
+        thread->stats.AddGets(1, 1);
+      } else if (statuses[0].IsNotFound()) {
+        thread->stats.AddGets(1, 0);
       }
     }
   }

db_stress_tool/db_stress_common.cc

@@ -321,17 +321,6 @@ uint32_t GetValueBase(Slice s) {
   return res;
 }
 
-AttributeGroups GenerateAttributeGroups(
-    const std::vector<ColumnFamilyHandle*>& cfhs, uint32_t value_base,
-    const Slice& slice) {
-  WideColumns columns = GenerateWideColumns(value_base, slice);
-  AttributeGroups attribute_groups;
-  for (auto* cfh : cfhs) {
-    attribute_groups.emplace_back(cfh, columns);
-  }
-  return attribute_groups;
-}
-
 WideColumns GenerateWideColumns(uint32_t value_base, const Slice& slice) {
   WideColumns columns;
 

db_stress_tool/db_stress_common.h

@@ -209,11 +209,8 @@ DECLARE_int32(ingest_external_file_one_in);
 DECLARE_int32(ingest_external_file_width);
 DECLARE_int32(compact_files_one_in);
 DECLARE_int32(compact_range_one_in);
-DECLARE_int32(promote_l0_one_in);
 DECLARE_int32(mark_for_compaction_one_file_in);
 DECLARE_int32(flush_one_in);
-DECLARE_int32(key_may_exist_one_in);
-DECLARE_int32(reset_stats_one_in);
 DECLARE_int32(pause_background_one_in);
 DECLARE_int32(disable_file_deletions_one_in);
 DECLARE_int32(disable_manual_compaction_one_in);
@@ -253,7 +250,6 @@ DECLARE_string(memtablerep);
 DECLARE_int32(prefix_size);
 DECLARE_bool(use_merge);
 DECLARE_uint32(use_put_entity_one_in);
-DECLARE_bool(use_attribute_group);
 DECLARE_bool(use_full_merge_v1);
 DECLARE_int32(sync_wal_one_in);
 DECLARE_bool(avoid_unnecessary_blocking_io);
@@ -762,10 +758,6 @@ WideColumns GenerateExpectedWideColumns(uint32_t value_base,
 bool VerifyWideColumns(const Slice& value, const WideColumns& columns);
 bool VerifyWideColumns(const WideColumns& columns);
 
-AttributeGroups GenerateAttributeGroups(
-    const std::vector<ColumnFamilyHandle*>& cfhs, uint32_t value_base,
-    const Slice& slice);
-
 StressTest* CreateCfConsistencyStressTest();
 StressTest* CreateBatchedOpsStressTest();
 StressTest* CreateNonBatchedOpsStressTest();

db_stress_tool/db_stress_gflags.cc

@@ -756,10 +756,6 @@ DEFINE_int32(compact_range_one_in, 0,
              "If non-zero, then CompactRange() will be called once for every N "
              "operations on average.  0 indicates CompactRange() is disabled.");
 
-DEFINE_int32(promote_l0_one_in, 0,
-             "If non-zero, then PromoteL0() will be called once for every N "
-             "operations on average.  0 indicates PromoteL0() is disabled.");
-
 DEFINE_int32(mark_for_compaction_one_file_in, 0,
              "A `TablePropertiesCollectorFactory` will be registered, which "
              "creates a `TablePropertiesCollector` with `NeedCompact()` "
@@ -770,14 +766,6 @@ DEFINE_int32(flush_one_in, 0,
              "If non-zero, then Flush() will be called once for every N ops "
              "on average.  0 indicates calls to Flush() are disabled.");
 
-DEFINE_int32(key_may_exist_one_in, 0,
-             "If non-zero, then KeyMayExist() will be called "
-             "once for every N ops on average.  0 disables.");
-
-DEFINE_int32(reset_stats_one_in, 0,
-             "If non-zero, then ResetStats() will be called "
-             "once for every N ops on average.  0 disables.");
-
 DEFINE_int32(pause_background_one_in, 0,
              "If non-zero, then PauseBackgroundWork()+Continue will be called "
              "once for every N ops on average.  0 disables.");
@@ -952,9 +940,6 @@ DEFINE_uint32(use_put_entity_one_in, 0,
               "If greater than zero, PutEntity will be used once per every N "
               "write ops on average.");
 
-DEFINE_bool(use_attribute_group, false,
-            "If set, use the attribute_group API to put/get entities");
-
 DEFINE_bool(use_full_merge_v1, false,
             "On true, use a merge operator that implement the deprecated "
             "version of FullMerge");

db_stress_tool/db_stress_test_base.cc

@@ -505,13 +505,11 @@ Status StressTest::AssertSame(DB* db, ColumnFamilyHandle* cf,
 }
 
 void StressTest::ProcessStatus(SharedState* shared, std::string opname,
-                               const Status& s,
-                               bool ignore_injected_error) const {
+                               Status s) const {
   if (s.ok()) {
     return;
   }
-  if (!s.IsIOError() || !std::strstr(s.getState(), "injected") ||
-      !ignore_injected_error) {
+  if (!s.IsIOError() || !std::strstr(s.getState(), "injected")) {
     std::ostringstream oss;
     oss << opname << " failed: " << s.ToString();
     VerificationAbort(shared, oss.str());
@@ -958,48 +956,12 @@ void StressTest::OperateDb(ThreadState* thread) {
         if (!s.ok()) {
           fprintf(stderr, "LockWAL() failed: %s\n", s.ToString().c_str());
         } else {
-          // Verify no writes during LockWAL
           auto old_seqno = db_->GetLatestSequenceNumber();
-          // And also that WAL is not changed during LockWAL()
-          std::unique_ptr<LogFile> old_wal;
-          s = db_->GetCurrentWalFile(&old_wal);
-          if (!s.ok()) {
-            fprintf(stderr, "GetCurrentWalFile() failed: %s\n",
-                    s.ToString().c_str());
-          } else {
-            // Yield for a while
-            do {
-              std::this_thread::yield();
-            } while (thread->rand.OneIn(2));
-            // Current WAL and size should not have changed
-            std::unique_ptr<LogFile> new_wal;
-            s = db_->GetCurrentWalFile(&new_wal);
-            if (!s.ok()) {
-              fprintf(stderr, "GetCurrentWalFile() failed: %s\n",
-                      s.ToString().c_str());
-            } else {
-              if (old_wal->LogNumber() != new_wal->LogNumber()) {
-                fprintf(stderr,
-                        "Failed: WAL number changed during LockWAL(): %" PRIu64
-                        " to %" PRIu64 "\n",
-                        old_wal->LogNumber(), new_wal->LogNumber());
-              }
-              // FIXME: FaultInjectionTestFS does not report file sizes that
-              // reflect what has been flushed. Either that needs to be fixed
-              // or GetSortedWals/GetLiveWalFile need to stop relying on
-              // asking the FS for sizes.
-              if (!fault_fs_guard &&
-                  old_wal->SizeFileBytes() != new_wal->SizeFileBytes()) {
-                fprintf(stderr,
-                        "Failed: WAL %" PRIu64
-                        " size changed during LockWAL(): %" PRIu64
-                        " to %" PRIu64 "\n",
-                        old_wal->LogNumber(), old_wal->SizeFileBytes(),
-                        new_wal->SizeFileBytes());
-              }
-            }
-          }
-          // Verify no writes during LockWAL
+          // Yield for a while
+          do {
+            std::this_thread::yield();
+          } while (thread->rand.OneIn(2));
+          // Latest seqno should not have changed
           auto new_seqno = db_->GetLatestSequenceNumber();
           if (old_seqno != new_seqno) {
             fprintf(
@@ -1007,7 +969,6 @@ void StressTest::OperateDb(ThreadState* thread) {
                 "Failure: latest seqno changed from %u to %u with WAL locked\n",
                 (unsigned)old_seqno, (unsigned)new_seqno);
           }
-          // Verification done. Now unlock WAL
           s = db_->UnlockWAL();
           if (!s.ok()) {
             fprintf(stderr, "UnlockWAL() failed: %s\n", s.ToString().c_str());
@@ -1040,10 +1001,6 @@ void StressTest::OperateDb(ThreadState* thread) {
         }
       }
 
-      if (thread->rand.OneInOpt(FLAGS_promote_l0_one_in)) {
-        TestPromoteL0(thread, column_family);
-      }
-
       std::vector<int> rand_column_families =
           GenerateColumnFamilies(FLAGS_column_families, rand_column_family);
 
@@ -1084,11 +1041,6 @@ void StressTest::OperateDb(ThreadState* thread) {
         ProcessStatus(shared, "VerifyGetCurrentWalFile", status);
       }
 
-      if (thread->rand.OneInOpt(FLAGS_reset_stats_one_in)) {
-        Status status = TestResetStats();
-        ProcessStatus(shared, "ResetStats", status);
-      }
-
       if (thread->rand.OneInOpt(FLAGS_pause_background_one_in)) {
         Status status = TestPauseBackground(thread);
         ProcessStatus(shared, "Pause/ContinueBackgroundWork", status);
@@ -1183,10 +1135,6 @@ void StressTest::OperateDb(ThreadState* thread) {
         read_opts.timestamp = &read_ts;
       }
 
-      if (thread->rand.OneInOpt(FLAGS_key_may_exist_one_in)) {
-        TestKeyMayExist(thread, read_opts, rand_column_families, rand_keys);
-      }
-
       int prob_op = thread->rand.Uniform(100);
       // Reset this in case we pick something other than a read op. We don't
       // want to use a stale value when deciding at the beginning of the loop
@@ -2480,32 +2428,8 @@ void StressTest::TestCompactFiles(ThreadState* thread,
   }
 }
 
-void StressTest::TestPromoteL0(ThreadState* thread,
-                               ColumnFamilyHandle* column_family) {
-  int target_level = thread->rand.Next() % options_.num_levels;
-  Status s = db_->PromoteL0(column_family, target_level);
-  if (!s.ok()) {
-    // The second error occurs when another concurrent PromoteL0() moving the
-    // same files finishes first which is an allowed behavior
-    bool non_ok_status_allowed =
-        s.IsInvalidArgument() ||
-        (s.IsCorruption() &&
-         s.ToString().find("VersionBuilder: Cannot delete table file") !=
-             std::string::npos &&
-         s.ToString().find("since it is on level") != std::string::npos);
-    fprintf(non_ok_status_allowed ? stdout : stderr,
-            "Unable to perform PromoteL0(): %s under specified "
-            "target_level: %d.\n",
-            s.ToString().c_str(), target_level);
-    if (!non_ok_status_allowed) {
-      thread->shared->SafeTerminate();
-    }
-  }
-}
-
 Status StressTest::TestFlush(const std::vector<int>& rand_column_families) {
   FlushOptions flush_opts;
-  assert(flush_opts.wait);
   if (FLAGS_atomic_flush) {
     return db_->Flush(flush_opts, column_families_);
   }
@@ -2515,8 +2439,6 @@ Status StressTest::TestFlush(const std::vector<int>& rand_column_families) {
   return db_->Flush(flush_opts, cfhs);
 }
 
-Status StressTest::TestResetStats() { return db_->ResetStats(); }
-
 Status StressTest::TestPauseBackground(ThreadState* thread) {
   Status status = db_->PauseBackgroundWork();
   if (!status.ok()) {
@@ -2661,10 +2583,7 @@ void StressTest::TestCompactRange(ThreadState* thread, int64_t rand_key,
 
   CompactRangeOptions cro;
   cro.exclusive_manual_compaction = static_cast<bool>(thread->rand.Next() % 2);
-  if (static_cast<ROCKSDB_NAMESPACE::CompactionStyle>(FLAGS_compaction_style) !=
-      ROCKSDB_NAMESPACE::CompactionStyle::kCompactionStyleFIFO) {
-    cro.change_level = static_cast<bool>(thread->rand.Next() % 2);
-  }
+  cro.change_level = static_cast<bool>(thread->rand.Next() % 2);
   if (thread->rand.OneIn(2)) {
     cro.target_level = thread->rand.Next() % options_.num_levels;
   }
@@ -2882,8 +2801,7 @@ void StressTest::PrintEnv() const {
   fprintf(stdout, "Num times DB reopens      : %d\n", FLAGS_reopen);
   fprintf(stdout, "Batches/snapshots         : %d\n",
           FLAGS_test_batches_snapshots);
-  fprintf(stdout, "Do update in place        : %d\n",
-          FLAGS_inplace_update_support);
+  fprintf(stdout, "Do update in place        : %d\n", FLAGS_in_place_update);
   fprintf(stdout, "Num keys per lock         : %d\n",
           1 << FLAGS_log2_keys_per_lock);
   std::string compression = CompressionTypeToString(compression_type_e);
@@ -3679,6 +3597,7 @@ void InitializeOptionsFromFlags(
     options.compression_opts.checksum = true;
   }
   options.max_manifest_file_size = FLAGS_max_manifest_file_size;
+  options.inplace_update_support = FLAGS_in_place_update;
   options.max_subcompactions = static_cast<uint32_t>(FLAGS_subcompactions);
   options.allow_concurrent_memtable_write =
       FLAGS_allow_concurrent_memtable_write;

db_stress_tool/db_stress_test_base.h

@@ -89,10 +89,6 @@ class StressTest {
     return {rand_key};
   }
 
-  virtual void TestKeyMayExist(ThreadState*, const ReadOptions&,
-                               const std::vector<int>&,
-                               const std::vector<int64_t>&) {}
-
   virtual Status TestGet(ThreadState* thread, const ReadOptions& read_opts,
                          const std::vector<int>& rand_column_families,
                          const std::vector<int64_t>& rand_keys) = 0;
@@ -140,9 +136,6 @@ class StressTest {
                                 const Slice& start_key,
                                 ColumnFamilyHandle* column_family);
 
-  virtual void TestPromoteL0(ThreadState* thread,
-                             ColumnFamilyHandle* column_family);
-
   // Calculate a hash value for all keys in range [start_key, end_key]
   // at a certain snapshot.
   uint32_t GetRangeHash(ThreadState* thread, const Snapshot* snapshot,
@@ -211,8 +204,6 @@ class StressTest {
 
   Status TestFlush(const std::vector<int>& rand_column_families);
 
-  Status TestResetStats();
-
   Status TestPauseBackground(ThreadState* thread);
 
   Status TestDisableFileDeletions(ThreadState* thread);
@@ -245,8 +236,7 @@ class StressTest {
     return Status::NotSupported("TestCustomOperations() must be overridden");
   }
 
-  void ProcessStatus(SharedState* shared, std::string msg, const Status& s,
-                     bool ignore_injected_error = true) const;
+  void ProcessStatus(SharedState* shared, std::string msg, Status s) const;
 
   void VerificationAbort(SharedState* shared, std::string msg) const;
 

db_stress_tool/db_stress_tool.cc

@@ -251,14 +251,13 @@ int db_stress_tool(int argc, char** argv) {
   }
   if ((FLAGS_enable_compaction_filter || FLAGS_inplace_update_support) &&
       (FLAGS_acquire_snapshot_one_in > 0 || FLAGS_compact_range_one_in > 0 ||
-       FLAGS_iterpercent > 0 || FLAGS_prefixpercent > 0 ||
-       FLAGS_test_batches_snapshots || FLAGS_test_cf_consistency ||
-       FLAGS_check_multiget_consistency ||
+       FLAGS_iterpercent > 0 || FLAGS_test_batches_snapshots ||
+       FLAGS_test_cf_consistency || FLAGS_check_multiget_consistency ||
        FLAGS_check_multiget_entity_consistency)) {
     fprintf(
         stderr,
         "Error: acquire_snapshot_one_in, compact_range_one_in, iterpercent, "
-        "prefixpercent, test_batches_snapshots, test_cf_consistency, "
+        "test_batches_snapshots, test_cf_consistency, "
         "check_multiget_consistency, check_multiget_entity_consistency must "
         "all be 0 when using compaction filter or inplace update support\n");
     exit(1);

db_stress_tool/multi_ops_txns_stress.cc

@@ -1331,16 +1331,14 @@ uint32_t MultiOpsTxnsStressTest::GenerateNextC(ThreadState* thread) {
 }
 
 void MultiOpsTxnsStressTest::ProcessRecoveredPreparedTxnsHelper(
-    Transaction* txn, SharedState* shared) {
+    Transaction* txn, SharedState*) {
   thread_local Random rand(static_cast<uint32_t>(FLAGS_seed));
   if (rand.OneIn(2)) {
     Status s = txn->Commit();
-    ProcessStatus(shared, "ProcessRecoveredPreparedTxnsHelper", s,
-                  /*ignore_injected_error=*/false);
+    assert(s.ok());
   } else {
     Status s = txn->Rollback();
-    ProcessStatus(shared, "ProcessRecoveredPreparedTxnsHelper", s,
-                  /*ignore_injected_error=*/false);
+    assert(s.ok());
   }
 }
 
@@ -1519,15 +1517,14 @@ void MultiOpsTxnsStressTest::PreloadDb(SharedState* shared, int threads,
     WriteBatch wb;
     const auto primary_index_entry = record.EncodePrimaryIndexEntry();
     Status s = wb.Put(primary_index_entry.first, primary_index_entry.second);
-    ProcessStatus(shared, "PreloadDB", s, /*ignore_injected_error=*/false);
+    assert(s.ok());
 
     const auto secondary_index_entry = record.EncodeSecondaryIndexEntry();
     s = wb.Put(secondary_index_entry.first, secondary_index_entry.second);
-    ProcessStatus(shared, "PreloadDB", s, /*ignore_injected_error=*/false);
+    assert(s.ok());
 
     s = txn_db_->Write(wopts, &wb);
     assert(s.ok());
-    ProcessStatus(shared, "PreloadDB", s, /*ignore_injected_error=*/false);
 
     // TODO (yanqin): make the following check optional, especially when data
     // size is large.
@@ -1535,7 +1532,7 @@ void MultiOpsTxnsStressTest::PreloadDb(SharedState* shared, int threads,
     tmp_rec.SetB(record.b_value());
     s = tmp_rec.DecodeSecondaryIndexEntry(secondary_index_entry.first,
                                           secondary_index_entry.second);
-    ProcessStatus(shared, "PreloadDB", s, /*ignore_injected_error=*/false);
+    assert(s.ok());
     assert(tmp_rec == record);
 
     existing_a_uniqs[tid].insert(a);

db_stress_tool/no_batched_ops_stress.cc

@@ -462,44 +462,6 @@ class NonBatchedOpsStressTest : public StressTest {
 
   bool IsStateTracked() const override { return true; }
 
-  void TestKeyMayExist(ThreadState* thread, const ReadOptions& read_opts,
-                       const std::vector<int>& rand_column_families,
-                       const std::vector<int64_t>& rand_keys) override {
-    auto cfh = column_families_[rand_column_families[0]];
-    std::string key_str = Key(rand_keys[0]);
-    Slice key = key_str;
-    std::string ignore;
-    ReadOptions read_opts_copy = read_opts;
-
-    std::string read_ts_str;
-    Slice read_ts_slice;
-    if (FLAGS_user_timestamp_size > 0) {
-      read_ts_str = GetNowNanos();
-      read_ts_slice = read_ts_str;
-      read_opts_copy.timestamp = &read_ts_slice;
-    }
-    bool read_older_ts = MaybeUseOlderTimestampForPointLookup(
-        thread, read_ts_str, read_ts_slice, read_opts_copy);
-
-    const ExpectedValue pre_read_expected_value =
-        thread->shared->Get(rand_column_families[0], rand_keys[0]);
-    bool key_may_exist = db_->KeyMayExist(read_opts_copy, cfh, key, &ignore);
-    const ExpectedValue post_read_expected_value =
-        thread->shared->Get(rand_column_families[0], rand_keys[0]);
-
-    if (!key_may_exist && !FLAGS_skip_verifydb && !read_older_ts) {
-      if (ExpectedValueHelper::MustHaveExisted(pre_read_expected_value,
-                                               post_read_expected_value)) {
-        thread->shared->SetVerificationFailure();
-        fprintf(stderr,
-                "error : inconsistent values for key %s: expected state has "
-                "the key, TestKeyMayExist() returns false indicating the key "
-                "must not exist.\n",
-                key.ToString(true).c_str());
-      }
-    }
-  }
-
   Status TestGet(ThreadState* thread, const ReadOptions& read_opts,
                  const std::vector<int>& rand_column_families,
                  const std::vector<int64_t>& rand_keys) override {
@@ -954,8 +916,7 @@ class NonBatchedOpsStressTest : public StressTest {
 
     const std::string key = Key(rand_keys[0]);
 
-    PinnableWideColumns columns_from_db;
-    PinnableAttributeGroups attribute_groups_from_db;
+    PinnableWideColumns from_db;
 
     ReadOptions read_opts_copy = read_opts;
     std::string read_ts_str;
@@ -968,16 +929,7 @@ class NonBatchedOpsStressTest : public StressTest {
     bool read_older_ts = MaybeUseOlderTimestampForPointLookup(
         thread, read_ts_str, read_ts_slice, read_opts_copy);
 
-    Status s;
-    if (FLAGS_use_attribute_group) {
-      attribute_groups_from_db.emplace_back(cfh);
-      s = db_->GetEntity(read_opts_copy, key, &attribute_groups_from_db);
-      if (s.ok()) {
-        s = attribute_groups_from_db.back().status();
-      }
-    } else {
-      s = db_->GetEntity(read_opts_copy, cfh, key, &columns_from_db);
-    }
+    const Status s = db_->GetEntity(read_opts_copy, cfh, key, &from_db);
 
     int error_count = 0;
 
@@ -1001,13 +953,7 @@ class NonBatchedOpsStressTest : public StressTest {
       thread->stats.AddGets(1, 1);
 
       if (!FLAGS_skip_verifydb && !read_older_ts) {
-        if (FLAGS_use_attribute_group) {
-          assert(!attribute_groups_from_db.empty());
-        }
-        const WideColumns& columns =
-            FLAGS_use_attribute_group
-                ? attribute_groups_from_db.back().columns()
-                : columns_from_db.columns();
+        const WideColumns& columns = from_db.columns();
         ExpectedValue expected =
             shared->Get(rand_column_families[0], rand_keys[0]);
         if (!VerifyWideColumns(columns)) {
@@ -1077,257 +1023,130 @@ class NonBatchedOpsStressTest : public StressTest {
     std::vector<std::string> keys(num_keys);
     std::vector<Slice> key_slices(num_keys);
 
-    if (fault_fs_guard) {
-      fault_fs_guard->EnableErrorInjection();
-      SharedState::ignore_read_error = false;
-    }
-
     for (size_t i = 0; i < num_keys; ++i) {
       keys[i] = Key(rand_keys[i]);
       key_slices[i] = keys[i];
     }
 
+    std::vector<PinnableWideColumns> results(num_keys);
+    std::vector<Status> statuses(num_keys);
+
+    if (fault_fs_guard) {
+      fault_fs_guard->EnableErrorInjection();
+      SharedState::ignore_read_error = false;
+    }
+
+    db_->MultiGetEntity(read_opts_copy, cfh, num_keys, key_slices.data(),
+                        results.data(), statuses.data());
+
     int error_count = 0;
 
-    auto handle_result = [](ThreadState* _thread, const Status& s,
-                            bool is_consistent, int err_count) {
+    if (fault_fs_guard) {
+      error_count = fault_fs_guard->GetAndResetErrorCount();
+
+      if (error_count && !SharedState::ignore_read_error) {
+        int stat_nok = 0;
+        for (const auto& s : statuses) {
+          if (!s.ok() && !s.IsNotFound()) {
+            stat_nok++;
+          }
+        }
+
+        if (stat_nok < error_count) {
+          // Grab mutex so multiple threads don't try to print the
+          // stack trace at the same time
+          assert(thread->shared);
+          MutexLock l(thread->shared->GetMutex());
+
+          fprintf(stderr, "Didn't get expected error from MultiGetEntity\n");
+          fprintf(stderr, "num_keys %zu Expected %d errors, seen %d\n",
+                  num_keys, error_count, stat_nok);
+          fprintf(stderr, "Call stack that injected the fault\n");
+          fault_fs_guard->PrintFaultBacktrace();
+          std::terminate();
+        }
+      }
+
+      fault_fs_guard->DisableErrorInjection();
+    }
+
+    const bool check_get_entity =
+        !error_count && FLAGS_check_multiget_entity_consistency;
+
+    for (size_t i = 0; i < num_keys; ++i) {
+      const Status& s = statuses[i];
+
+      bool is_consistent = true;
+
+      if (s.ok() && !VerifyWideColumns(results[i].columns())) {
+        fprintf(
+            stderr,
+            "error : inconsistent columns returned by MultiGetEntity for key "
+            "%s: %s\n",
+            StringToHex(keys[i]).c_str(),
+            WideColumnsToHex(results[i].columns()).c_str());
+        is_consistent = false;
+      } else if (check_get_entity && (s.ok() || s.IsNotFound())) {
+        PinnableWideColumns cmp_result;
+        ThreadStatusUtil::SetThreadOperation(
+            ThreadStatus::OperationType::OP_GETENTITY);
+        const Status cmp_s =
+            db_->GetEntity(read_opts_copy, cfh, key_slices[i], &cmp_result);
+
+        if (!cmp_s.ok() && !cmp_s.IsNotFound()) {
+          fprintf(stderr, "GetEntity error: %s\n", cmp_s.ToString().c_str());
+          is_consistent = false;
+        } else if (cmp_s.IsNotFound()) {
+          if (s.ok()) {
+            fprintf(stderr,
+                    "Inconsistent results for key %s: MultiGetEntity returned "
+                    "ok, GetEntity returned not found\n",
+                    StringToHex(keys[i]).c_str());
+            is_consistent = false;
+          }
+        } else {
+          assert(cmp_s.ok());
+
+          if (s.IsNotFound()) {
+            fprintf(stderr,
+                    "Inconsistent results for key %s: MultiGetEntity returned "
+                    "not found, GetEntity returned ok\n",
+                    StringToHex(keys[i]).c_str());
+            is_consistent = false;
+          } else {
+            assert(s.ok());
+
+            if (results[i] != cmp_result) {
+              fprintf(
+                  stderr,
+                  "Inconsistent results for key %s: MultiGetEntity returned "
+                  "%s, GetEntity returned %s\n",
+                  StringToHex(keys[i]).c_str(),
+                  WideColumnsToHex(results[i].columns()).c_str(),
+                  WideColumnsToHex(cmp_result.columns()).c_str());
+              is_consistent = false;
+            }
+          }
+        }
+      }
+
       if (!is_consistent) {
         fprintf(stderr,
-                "TestMultiGetEntity%s error: results are not consistent\n",
-                FLAGS_use_attribute_group ? "(AttributeGroup)" : "");
-        _thread->stats.AddErrors(1);
+                "TestMultiGetEntity error: results are not consistent\n");
+        thread->stats.AddErrors(1);
         // Fail fast to preserve the DB state
-        _thread->shared->SetVerificationFailure();
+        thread->shared->SetVerificationFailure();
+        break;
       } else if (s.ok()) {
-        _thread->stats.AddGets(1, 1);
+        thread->stats.AddGets(1, 1);
       } else if (s.IsNotFound()) {
-        _thread->stats.AddGets(1, 0);
+        thread->stats.AddGets(1, 0);
       } else {
-        if (err_count == 0) {
-          fprintf(stderr, "MultiGetEntity%s error: %s\n",
-                  FLAGS_use_attribute_group ? "(AttributeGroup)" : "",
-                  s.ToString().c_str());
-          _thread->stats.AddErrors(1);
+        if (error_count == 0) {
+          fprintf(stderr, "MultiGetEntity error: %s\n", s.ToString().c_str());
+          thread->stats.AddErrors(1);
         } else {
-          _thread->stats.AddVerifiedErrors(1);
-        }
-      }
-    };
-
-    if (FLAGS_use_attribute_group) {
-      // AttributeGroup MultiGetEntity verification
-
-      std::vector<PinnableAttributeGroups> results;
-      results.reserve(num_keys);
-      for (size_t i = 0; i < num_keys; ++i) {
-        PinnableAttributeGroups attribute_groups;
-        attribute_groups.emplace_back(cfh);
-        results.emplace_back(std::move(attribute_groups));
-      }
-      db_->MultiGetEntity(read_opts_copy, num_keys, key_slices.data(),
-                          results.data());
-
-      if (fault_fs_guard) {
-        error_count = fault_fs_guard->GetAndResetErrorCount();
-
-        if (error_count && !SharedState::ignore_read_error) {
-          int stat_nok = 0;
-          for (size_t i = 0; i < num_keys; ++i) {
-            const Status& s = results[i][0].status();
-            if (!s.ok() && !s.IsNotFound()) {
-              stat_nok++;
-            }
-          }
-
-          if (stat_nok < error_count) {
-            // Grab mutex so multiple threads don't try to print the
-            // stack trace at the same time
-            assert(thread->shared);
-            MutexLock l(thread->shared->GetMutex());
-
-            fprintf(stderr,
-                    "Didn't get expected error from MultiGetEntity "
-                    "(AttributeGroup)\n");
-            fprintf(stderr, "num_keys %zu Expected %d errors, seen %d\n",
-                    num_keys, error_count, stat_nok);
-            fprintf(stderr, "Call stack that injected the fault\n");
-            fault_fs_guard->PrintFaultBacktrace();
-            std::terminate();
-          }
-        }
-        fault_fs_guard->DisableErrorInjection();
-      }
-
-      // Compare against non-attribute-group GetEntity result
-      const bool check_get_entity =
-          !error_count && FLAGS_check_multiget_entity_consistency;
-
-      for (size_t i = 0; i < num_keys; ++i) {
-        assert(results[i].size() == 1);
-        const Status& s = results[i][0].status();
-
-        bool is_consistent = true;
-
-        if (s.ok() && !VerifyWideColumns(results[i][0].columns())) {
-          fprintf(stderr,
-                  "error : inconsistent columns returned by MultiGetEntity "
-                  "(AttributeGroup) for key "
-                  "%s: %s\n",
-                  StringToHex(keys[i]).c_str(),
-                  WideColumnsToHex(results[i][0].columns()).c_str());
-          is_consistent = false;
-        } else if (check_get_entity && (s.ok() || s.IsNotFound())) {
-          PinnableWideColumns cmp_result;
-          ThreadStatusUtil::SetThreadOperation(
-              ThreadStatus::OperationType::OP_GETENTITY);
-          const Status cmp_s =
-              db_->GetEntity(read_opts_copy, cfh, key_slices[i], &cmp_result);
-
-          if (!cmp_s.ok() && !cmp_s.IsNotFound()) {
-            fprintf(stderr, "GetEntity error: %s\n", cmp_s.ToString().c_str());
-            is_consistent = false;
-          } else if (cmp_s.IsNotFound()) {
-            if (s.ok()) {
-              fprintf(stderr,
-                      "Inconsistent results for key %s: MultiGetEntity "
-                      "(AttributeGroup) returned "
-                      "ok, GetEntity returned not found\n",
-                      StringToHex(keys[i]).c_str());
-              is_consistent = false;
-            }
-          } else {
-            assert(cmp_s.ok());
-
-            if (s.IsNotFound()) {
-              fprintf(stderr,
-                      "Inconsistent results for key %s: MultiGetEntity "
-                      "(AttributeGroup) returned "
-                      "not found, GetEntity returned ok\n",
-                      StringToHex(keys[i]).c_str());
-              is_consistent = false;
-            } else {
-              assert(s.ok());
-
-              if (results[i][0].columns() != cmp_result.columns()) {
-                fprintf(stderr,
-                        "Inconsistent results for key %s: MultiGetEntity "
-                        "(AttributeGroup) returned "
-                        "%s, GetEntity returned %s\n",
-                        StringToHex(keys[i]).c_str(),
-                        WideColumnsToHex(results[i][0].columns()).c_str(),
-                        WideColumnsToHex(cmp_result.columns()).c_str());
-                is_consistent = false;
-              }
-            }
-          }
-        }
-        handle_result(thread, s, is_consistent, error_count);
-        if (!is_consistent) {
-          break;
-        }
-      }
-    } else {
-      // Non-AttributeGroup MultiGetEntity verification
-
-      std::vector<PinnableWideColumns> results(num_keys);
-      std::vector<Status> statuses(num_keys);
-
-      db_->MultiGetEntity(read_opts_copy, cfh, num_keys, key_slices.data(),
-                          results.data(), statuses.data());
-
-      if (fault_fs_guard) {
-        error_count = fault_fs_guard->GetAndResetErrorCount();
-
-        if (error_count && !SharedState::ignore_read_error) {
-          int stat_nok = 0;
-          for (const auto& s : statuses) {
-            if (!s.ok() && !s.IsNotFound()) {
-              stat_nok++;
-            }
-          }
-
-          if (stat_nok < error_count) {
-            // Grab mutex so multiple threads don't try to print the
-            // stack trace at the same time
-            assert(thread->shared);
-            MutexLock l(thread->shared->GetMutex());
-
-            fprintf(stderr, "Didn't get expected error from MultiGetEntity\n");
-            fprintf(stderr, "num_keys %zu Expected %d errors, seen %d\n",
-                    num_keys, error_count, stat_nok);
-            fprintf(stderr, "Call stack that injected the fault\n");
-            fault_fs_guard->PrintFaultBacktrace();
-            std::terminate();
-          }
-        }
-
-        fault_fs_guard->DisableErrorInjection();
-      }
-
-      const bool check_get_entity =
-          !error_count && FLAGS_check_multiget_entity_consistency;
-
-      for (size_t i = 0; i < num_keys; ++i) {
-        const Status& s = statuses[i];
-
-        bool is_consistent = true;
-
-        if (s.ok() && !VerifyWideColumns(results[i].columns())) {
-          fprintf(
-              stderr,
-              "error : inconsistent columns returned by MultiGetEntity for key "
-              "%s: %s\n",
-              StringToHex(keys[i]).c_str(),
-              WideColumnsToHex(results[i].columns()).c_str());
-          is_consistent = false;
-        } else if (check_get_entity && (s.ok() || s.IsNotFound())) {
-          PinnableWideColumns cmp_result;
-          ThreadStatusUtil::SetThreadOperation(
-              ThreadStatus::OperationType::OP_GETENTITY);
-          const Status cmp_s =
-              db_->GetEntity(read_opts_copy, cfh, key_slices[i], &cmp_result);
-
-          if (!cmp_s.ok() && !cmp_s.IsNotFound()) {
-            fprintf(stderr, "GetEntity error: %s\n", cmp_s.ToString().c_str());
-            is_consistent = false;
-          } else if (cmp_s.IsNotFound()) {
-            if (s.ok()) {
-              fprintf(
-                  stderr,
-                  "Inconsistent results for key %s: MultiGetEntity returned "
-                  "ok, GetEntity returned not found\n",
-                  StringToHex(keys[i]).c_str());
-              is_consistent = false;
-            }
-          } else {
-            assert(cmp_s.ok());
-
-            if (s.IsNotFound()) {
-              fprintf(
-                  stderr,
-                  "Inconsistent results for key %s: MultiGetEntity returned "
-                  "not found, GetEntity returned ok\n",
-                  StringToHex(keys[i]).c_str());
-              is_consistent = false;
-            } else {
-              assert(s.ok());
-
-              if (results[i] != cmp_result) {
-                fprintf(
-                    stderr,
-                    "Inconsistent results for key %s: MultiGetEntity returned "
-                    "%s, GetEntity returned %s\n",
-                    StringToHex(keys[i]).c_str(),
-                    WideColumnsToHex(results[i].columns()).c_str(),
-                    WideColumnsToHex(cmp_result.columns()).c_str());
-                is_consistent = false;
-              }
-            }
-          }
-        }
-        handle_result(thread, s, is_consistent, error_count);
-        if (!is_consistent) {
-          break;
+          thread->stats.AddVerifiedErrors(1);
         }
       }
     }
@@ -1483,13 +1302,8 @@ class NonBatchedOpsStressTest : public StressTest {
 
     if (FLAGS_use_put_entity_one_in > 0 &&
         (value_base % FLAGS_use_put_entity_one_in) == 0) {
-      if (FLAGS_use_attribute_group) {
-        s = db_->PutEntity(write_opts, k,
-                           GenerateAttributeGroups({cfh}, value_base, v));
-      } else {
-        s = db_->PutEntity(write_opts, cfh, k,
-                           GenerateWideColumns(value_base, v));
-      }
+      s = db_->PutEntity(write_opts, cfh, k,
+                         GenerateWideColumns(value_base, v));
     } else if (FLAGS_use_timed_put_one_in > 0 &&
                ((value_base + kLargePrimeForCommonFactorSkew) %
                 FLAGS_use_timed_put_one_in) == 0) {
@@ -2265,7 +2079,7 @@ class NonBatchedOpsStressTest : public StressTest {
     return Status::OK();
   }
 
-  bool VerifyOrSyncValue(int cf, int64_t key, const ReadOptions& opts,
+  bool VerifyOrSyncValue(int cf, int64_t key, const ReadOptions& /*opts*/,
                          SharedState* shared, const std::string& value_from_db,
                          std::string msg_prefix, const Status& s) const {
     if (shared->HasVerificationFailedYet()) {
@@ -2291,43 +2105,27 @@ class NonBatchedOpsStressTest : public StressTest {
         GenerateValue(expected_value.GetValueBase(), expected_value_data,
                       sizeof(expected_value_data));
 
-    std::ostringstream read_u64ts;
-    if (opts.timestamp) {
-      read_u64ts << " while read with timestamp: ";
-      uint64_t read_ts;
-      if (DecodeU64Ts(*opts.timestamp, &read_ts).ok()) {
-        read_u64ts << std::to_string(read_ts) << ", ";
-      } else {
-        read_u64ts << s.ToString()
-                   << " Encoded read timestamp: " << opts.timestamp->ToString()
-                   << ", ";
-      }
-    }
-
     // compare value_from_db with the value in the shared state
     if (s.ok()) {
       const Slice slice(value_from_db);
       const uint32_t value_base_from_db = GetValueBase(slice);
       if (ExpectedValueHelper::MustHaveNotExisted(expected_value,
                                                   expected_value)) {
-        VerificationAbort(
-            shared, msg_prefix + ": Unexpected value found" + read_u64ts.str(),
-            cf, key, value_from_db, "");
+        VerificationAbort(shared, msg_prefix + ": Unexpected value found", cf,
+                          key, value_from_db, "");
         return false;
       }
       if (!ExpectedValueHelper::InExpectedValueBaseRange(
               value_base_from_db, expected_value, expected_value)) {
-        VerificationAbort(
-            shared, msg_prefix + ": Unexpected value found" + read_u64ts.str(),
-            cf, key, value_from_db,
-            Slice(expected_value_data, expected_value_data_size));
+        VerificationAbort(shared, msg_prefix + ": Unexpected value found", cf,
+                          key, value_from_db,
+                          Slice(expected_value_data, expected_value_data_size));
         return false;
       }
       // TODO: are the length/memcmp() checks repetitive?
       if (value_from_db.length() != expected_value_data_size) {
         VerificationAbort(shared,
-                          msg_prefix + ": Length of value read is not equal" +
-                              read_u64ts.str(),
+                          msg_prefix + ": Length of value read is not equal",
                           cf, key, value_from_db,
                           Slice(expected_value_data, expected_value_data_size));
         return false;
@@ -2335,8 +2133,7 @@ class NonBatchedOpsStressTest : public StressTest {
       if (memcmp(value_from_db.data(), expected_value_data,
                  expected_value_data_size) != 0) {
         VerificationAbort(shared,
-                          msg_prefix + ": Contents of value read don't match" +
-                              read_u64ts.str(),
+                          msg_prefix + ": Contents of value read don't match",
                           cf, key, value_from_db,
                           Slice(expected_value_data, expected_value_data_size));
         return false;
@@ -2345,16 +2142,14 @@ class NonBatchedOpsStressTest : public StressTest {
       if (ExpectedValueHelper::MustHaveExisted(expected_value,
                                                expected_value)) {
         VerificationAbort(
-            shared,
-            msg_prefix + ": Value not found " + read_u64ts.str() + s.ToString(),
-            cf, key, "", Slice(expected_value_data, expected_value_data_size));
+            shared, msg_prefix + ": Value not found: " + s.ToString(), cf, key,
+            "", Slice(expected_value_data, expected_value_data_size));
         return false;
       }
     } else {
-      VerificationAbort(
-          shared,
-          msg_prefix + "Non-OK status " + read_u64ts.str() + s.ToString(), cf,
-          key, "", Slice(expected_value_data, expected_value_data_size));
+      VerificationAbort(shared, msg_prefix + "Non-OK status: " + s.ToString(),
+                        cf, key, "",
+                        Slice(expected_value_data, expected_value_data_size));
       return false;
     }
     return true;

env/io_posix.cc

@@ -1441,12 +1441,10 @@ void PosixWritableFile::SetWriteLifeTimeHint(Env::WriteLifeTimeHint hint) {
 #ifdef OS_LINUX
 // Suppress Valgrind "Unimplemented functionality" error.
 #ifndef ROCKSDB_VALGRIND_RUN
-  uint64_t fcntl_hint = hint;
-
   if (hint == write_hint_) {
     return;
   }
-  if (fcntl(fd_, F_SET_RW_HINT, &fcntl_hint) == 0) {
+  if (fcntl(fd_, F_SET_RW_HINT, &hint) == 0) {
     write_hint_ = hint;
   }
 #else

include/rocksdb/c.h

@@ -1064,21 +1064,6 @@ rocksdb_block_based_options_set_pin_top_level_index_and_filter(
     rocksdb_block_based_table_options_t*, unsigned char);
 extern ROCKSDB_LIBRARY_API void rocksdb_options_set_block_based_table_factory(
     rocksdb_options_t* opt, rocksdb_block_based_table_options_t* table_options);
-enum {
-  rocksdb_block_based_k_fallback_pinning_tier = 0,
-  rocksdb_block_based_k_none_pinning_tier = 1,
-  rocksdb_block_based_k_flush_and_similar_pinning_tier = 2,
-  rocksdb_block_based_k_all_pinning_tier = 3,
-};
-extern ROCKSDB_LIBRARY_API void
-rocksdb_block_based_options_set_top_level_index_pinning_tier(
-    rocksdb_block_based_table_options_t*, int);
-extern ROCKSDB_LIBRARY_API void
-rocksdb_block_based_options_set_partition_pinning_tier(
-    rocksdb_block_based_table_options_t*, int);
-extern ROCKSDB_LIBRARY_API void
-rocksdb_block_based_options_set_unpartitioned_pinning_tier(
-    rocksdb_block_based_table_options_t*, int);
 extern ROCKSDB_LIBRARY_API void rocksdb_options_set_write_buffer_manager(
     rocksdb_options_t* opt, rocksdb_write_buffer_manager_t* wbm);
 
@@ -1697,18 +1682,6 @@ extern ROCKSDB_LIBRARY_API void rocksdb_options_set_wal_compression(
 extern ROCKSDB_LIBRARY_API int rocksdb_options_get_wal_compression(
     rocksdb_options_t* opt);
 
-enum {
-  rocksdb_k_by_compensated_size_compaction_pri = 0,
-  rocksdb_k_oldest_largest_seq_first_compaction_pri = 1,
-  rocksdb_k_oldest_smallest_seq_first_compaction_pri = 2,
-  rocksdb_k_min_overlapping_ratio_compaction_pri = 3,
-  rocksdb_k_round_robin_compaction_pri = 4
-};
-extern ROCKSDB_LIBRARY_API void rocksdb_options_set_compaction_pri(
-    rocksdb_options_t*, int);
-extern ROCKSDB_LIBRARY_API int rocksdb_options_get_compaction_pri(
-    rocksdb_options_t*);
-
 /* RateLimiter */
 extern ROCKSDB_LIBRARY_API rocksdb_ratelimiter_t* rocksdb_ratelimiter_create(
     int64_t rate_bytes_per_sec, int64_t refill_period_us, int32_t fairness);

include/rocksdb/comparator.h

@@ -142,11 +142,6 @@ class Comparator : public Customizable, public CompareInterface {
     return Slice();
   }
 
-  // Return a human readable user-defined timestamp for debugging.
-  virtual std::string TimestampToString(const Slice& /*timestamp*/) const {
-    return "";
-  }
-
   int CompareWithoutTimestamp(const Slice& a, const Slice& b) const {
     return CompareWithoutTimestamp(a, /*a_has_ts=*/true, b, /*b_has_ts=*/true);
   }

include/rocksdb/db.h

@@ -1475,9 +1475,6 @@ class DB {
   // move the files back to the minimum level capable of holding the data set
   // or a given level (specified by non-negative options.target_level).
   //
-  // For FIFO compaction, this will trigger a compaction (if available)
-  // based on CompactionOptionsFIFO.
-  //
   // In case of user-defined timestamp, if enabled, `begin` and `end` should
   // not contain timestamp.
   virtual Status CompactRange(const CompactRangeOptions& options,
@@ -1679,8 +1676,8 @@ class DB {
   // Freezes the logical state of the DB (by stopping writes), and if WAL is
   // enabled, ensures that state has been flushed to DB files (as in
   // FlushWAL()). This can be used for taking a Checkpoint at a known DB
-  // state, though while the WAL is locked, flushes as part of CreateCheckpoint
-  // and simiar are skipped. Other operations allowed on a "read only" DB should
+  // state, though the user must use options to insure no DB flush is invoked
+  // in this frozen state. Other operations allowed on a "read only" DB should
   // work while frozen. Each LockWAL() call that returns OK must eventually be
   // followed by a corresponding call to UnlockWAL(). Where supported, non-OK
   // status is generally only possible with some kind of corruption or I/O
@@ -1861,7 +1858,6 @@ class DB {
   // supported. 4) When an ingested file contains point data and range deletion
   // for the same key, the point data currently overrides the range deletion
   // regardless which one has the higher user-defined timestamps.
-  // For FIFO compaction, SST files will always be ingested into L0.
   //
   // (1) External SST files can be created using SstFileWriter
   // (2) We will try to ingest the files to the lowest possible level
@@ -2009,8 +2005,6 @@ class DB {
     return Status::NotSupported("SuggestCompactRange() is not implemented.");
   }
 
-  // Trivially move L0 files to target level. Should not be called with another
-  // PromoteL0() concurrently
   virtual Status PromoteL0(ColumnFamilyHandle* /*column_family*/,
                            int /*target_level*/) {
     return Status::NotSupported("PromoteL0() is not implemented.");

include/rocksdb/iterator.h

@@ -78,12 +78,6 @@ class Iterator : public IteratorBase {
   //      - Iterator created with ReadOptions::pin_data = true
   //      - DB tables were created with
   //        BlockBasedTableOptions::use_delta_encoding = false.
-  // Property "rocksdb.iterator.is-value-pinned":
-  //   If returning "1", this means that the Slice returned by value() is valid
-  //   as long as the iterator is not deleted.
-  //   It is guaranteed to always return "1" if
-  //      - Iterator created with ReadOptions::pin_data = true
-  //      - The value is found in a `kTypeValue` record
   // Property "rocksdb.iterator.super-version-number":
   //   LSM version used by the iterator. The same format as DB Property
   //   kCurrentSuperVersionNumber. See its comment for more information.

include/rocksdb/types.h

@@ -67,7 +67,6 @@ enum EntryType {
   kEntryBlobIndex,
   kEntryDeleteWithTimestamp,
   kEntryWideColumnEntity,
-  kEntryTimedPut,  // That hasn't yet converted to a standard Put entry
   kEntryOther,
 };
 

include/rocksdb/utilities/ldb_cmd.h

@@ -72,7 +72,6 @@ class LDBCommand {
   static const std::string ARG_PREPOPULATE_BLOB_CACHE;
   static const std::string ARG_DECODE_BLOB_INDEX;
   static const std::string ARG_DUMP_UNCOMPRESSED_BLOBS;
-  static const std::string ARG_READ_TIMESTAMP;
 
   struct ParsedParams {
     std::string cmd;
@@ -191,9 +190,6 @@ class LDBCommand {
 
   bool create_if_missing_;
 
-  /** Encoded user provided uint64_t read timestamp. */
-  std::string read_timestamp_;
-
   /**
    * Map of options passed on the command-line.
    */
@@ -279,10 +275,6 @@ class LDBCommand {
   bool ParseBooleanOption(const std::map<std::string, std::string>& options,
                           const std::string& option, bool default_val);
 
-  /* Populate `ropts.timestamp` from command line flag --read_timestamp */
-  Status MaybePopulateReadTimestamp(ColumnFamilyHandle* cfh, ReadOptions& ropts,
-                                    Slice* read_timestamp);
-
   Options options_;
   std::vector<ColumnFamilyDescriptor> column_families_;
   ConfigOptions config_options_;

include/rocksdb/utilities/transaction.h

@@ -5,6 +5,7 @@
 
 #pragma once
 
+
 #include <limits>
 #include <string>
 #include <vector>
@@ -164,7 +165,7 @@ class Transaction {
   virtual void SetSnapshot() = 0;
 
   // Similar to SetSnapshot(), but will not change the current snapshot
-  // until Put/PutEntity/Merge/Delete/GetForUpdate/MultigetForUpdate is called.
+  // until Put/Merge/Delete/GetForUpdate/MultigetForUpdate is called.
   // By calling this function, the transaction will essentially call
   // SetSnapshot() for you right before performing the next write/GetForUpdate.
   //
@@ -267,10 +268,10 @@ class Transaction {
   // points.
   virtual void SetSavePoint() = 0;
 
-  // Undo all operations in this transaction (Put, PutEntity, Merge, Delete,
-  // PutLogData) since the most recent call to SetSavePoint() and removes the
-  // most recent SetSavePoint(). If there is no previous call to SetSavePoint(),
-  // returns Status::NotFound()
+  // Undo all operations in this transaction (Put, Merge, Delete, PutLogData)
+  // since the most recent call to SetSavePoint() and removes the most recent
+  // SetSavePoint().
+  // If there is no previous call to SetSavePoint(), returns Status::NotFound()
   virtual Status RollbackToSavePoint() = 0;
 
   // Pop the most recent save point.
@@ -317,10 +318,6 @@ class Transaction {
     return s;
   }
 
-  virtual Status GetEntity(const ReadOptions& options,
-                           ColumnFamilyHandle* column_family, const Slice& key,
-                           PinnableWideColumns* columns) = 0;
-
   virtual std::vector<Status> MultiGet(
       const ReadOptions& options,
       const std::vector<ColumnFamilyHandle*>& column_family,
@@ -357,12 +354,6 @@ class Transaction {
     }
   }
 
-  virtual void MultiGetEntity(const ReadOptions& options,
-                              ColumnFamilyHandle* column_family,
-                              size_t num_keys, const Slice* keys,
-                              PinnableWideColumns* results, Status* statuses,
-                              bool sorted_input = false) = 0;
-
   // Read this key and ensure that this transaction will only
   // be able to be committed if this key is not written outside this
   // transaction after it has first been read (or after the snapshot if a
@@ -470,9 +461,9 @@ class Transaction {
   virtual Iterator* GetIterator(const ReadOptions& read_options,
                                 ColumnFamilyHandle* column_family) = 0;
 
-  // Put, PutEntity, Merge, Delete, and SingleDelete behave similarly to the
-  // corresponding functions in WriteBatch, but will also do conflict checking
-  // on the keys being written.
+  // Put, Merge, Delete, and SingleDelete behave similarly to the corresponding
+  // functions in WriteBatch, but will also do conflict checking on the
+  // keys being written.
   //
   // assume_tracked=true expects the key be already tracked. More
   // specifically, it means the the key was previous tracked in the same
@@ -498,10 +489,6 @@ class Transaction {
                      const bool assume_tracked = false) = 0;
   virtual Status Put(const SliceParts& key, const SliceParts& value) = 0;
 
-  virtual Status PutEntity(ColumnFamilyHandle* column_family, const Slice& key,
-                           const WideColumns& columns,
-                           bool assume_tracked = false) = 0;
-
   virtual Status Merge(ColumnFamilyHandle* column_family, const Slice& key,
                        const Slice& value,
                        const bool assume_tracked = false) = 0;
@@ -541,10 +528,6 @@ class Transaction {
   virtual Status PutUntracked(const SliceParts& key,
                               const SliceParts& value) = 0;
 
-  virtual Status PutEntityUntracked(ColumnFamilyHandle* column_family,
-                                    const Slice& key,
-                                    const WideColumns& columns) = 0;
-
   virtual Status MergeUntracked(ColumnFamilyHandle* column_family,
                                 const Slice& key, const Slice& value) = 0;
   virtual Status MergeUntracked(const Slice& key, const Slice& value) = 0;
@@ -573,18 +556,18 @@ class Transaction {
   // Similar to WriteBatch::PutLogData
   virtual void PutLogData(const Slice& blob) = 0;
 
-  // By default, all Put/PutEntity/Merge/Delete operations will be indexed in
-  // the transaction so that Get/GetForUpdate/GetIterator can search for these
+  // By default, all Put/Merge/Delete operations will be indexed in the
+  // transaction so that Get/GetForUpdate/GetIterator can search for these
   // keys.
   //
   // If the caller does not want to fetch the keys about to be written,
   // they may want to avoid indexing as a performance optimization.
   // Calling DisableIndexing() will turn off indexing for all future
-  // Put/PutEntity/Merge/Delete operations until EnableIndexing() is called.
+  // Put/Merge/Delete operations until EnableIndexing() is called.
   //
-  // If a key is written (using Put/PutEntity/Merge/Delete) after
-  // DisableIndexing is called and then is fetched via
-  // Get/GetForUpdate/GetIterator, the result of the fetch is undefined.
+  // If a key is Put/Merge/Deleted after DisableIndexing is called and then
+  // is fetched via Get/GetForUpdate/GetIterator, the result of the fetch is
+  // undefined.
   virtual void DisableIndexing() = 0;
   virtual void EnableIndexing() = 0;
 
@@ -595,10 +578,9 @@ class Transaction {
   // number of keys that need to be checked for conflicts at commit time.
   virtual uint64_t GetNumKeys() const = 0;
 
-  // Returns the number of Put/PutEntity/Delete/Merge operations that have been
-  // applied to this transaction so far.
+  // Returns the number of Puts/Deletes/Merges that have been applied to this
+  // transaction so far.
   virtual uint64_t GetNumPuts() const = 0;
-  virtual uint64_t GetNumPutEntities() const = 0;
   virtual uint64_t GetNumDeletes() const = 0;
   virtual uint64_t GetNumMerges() const = 0;
 

include/rocksdb/utilities/write_batch_with_index.h

@@ -130,7 +130,7 @@ class WriteBatchWithIndex : public WriteBatchBase {
           "Cannot call this method without attribute groups");
     }
     return Status::NotSupported(
-        "PutEntity with AttributeGroups not supported by WriteBatchWithIndex");
+        "PutEntity not supported by WriteBatchWithIndex");
   }
 
   using WriteBatchBase::Merge;
@@ -284,12 +284,7 @@ class WriteBatchWithIndex : public WriteBatchBase {
   Status GetEntityFromBatchAndDB(DB* db, const ReadOptions& read_options,
                                  ColumnFamilyHandle* column_family,
                                  const Slice& key,
-                                 PinnableWideColumns* columns) {
-    constexpr ReadCallback* callback = nullptr;
-
-    return GetEntityFromBatchAndDB(db, read_options, column_family, key,
-                                   columns, callback);
-  }
+                                 PinnableWideColumns* columns);
 
   void MultiGetFromBatchAndDB(DB* db, const ReadOptions& read_options,
                               ColumnFamilyHandle* column_family,
@@ -315,13 +310,7 @@ class WriteBatchWithIndex : public WriteBatchBase {
                                     ColumnFamilyHandle* column_family,
                                     size_t num_keys, const Slice* keys,
                                     PinnableWideColumns* results,
-                                    Status* statuses, bool sorted_input) {
-    constexpr ReadCallback* callback = nullptr;
-
-    MultiGetEntityFromBatchAndDB(db, read_options, column_family, num_keys,
-                                 keys, results, statuses, sorted_input,
-                                 callback);
-  }
+                                    Status* statuses, bool sorted_input);
 
   // Records the state of the batch for future calls to RollbackToSavePoint().
   // May be called multiple times to set multiple save points.

include/rocksdb/version.h

@@ -12,8 +12,8 @@
 // NOTE: in 'main' development branch, this should be the *next*
 // minor or major version number planned for release.
 #define ROCKSDB_MAJOR 9
-#define ROCKSDB_MINOR 3
-#define ROCKSDB_PATCH 1
+#define ROCKSDB_MINOR 2
+#define ROCKSDB_PATCH 2
 
 // Do not use these. We made the mistake of declaring macros starting with
 // double underscore. Now we have to live with our choice. We'll deprecate these

table/block_based/block_based_table_iterator.cc

@@ -338,8 +338,8 @@ void BlockBasedTableIterator::InitDataBlock() {
   bool use_block_cache_for_lookup = true;
 
   if (DoesContainBlockHandles()) {
-    data_block_handle = block_handles_->front().handle_;
-    is_in_cache = block_handles_->front().is_cache_hit_;
+    data_block_handle = block_handles_.front().handle_;
+    is_in_cache = block_handles_.front().is_cache_hit_;
     use_block_cache_for_lookup = false;
   } else {
     data_block_handle = index_iter_->value().handle;
@@ -361,7 +361,7 @@ void BlockBasedTableIterator::InitDataBlock() {
       Status s;
       block_iter_.Invalidate(Status::OK());
       table_->NewDataBlockIterator<DataBlockIter>(
-          read_options_, (block_handles_->front().cachable_entry_).As<Block>(),
+          read_options_, (block_handles_.front().cachable_entry_).As<Block>(),
           &block_iter_, s);
     } else {
       auto* rep = table_->get_rep();
@@ -466,8 +466,8 @@ void BlockBasedTableIterator::AsyncInitDataBlock(bool is_first_pass) {
     bool is_in_cache = false;
 
     if (DoesContainBlockHandles()) {
-      data_block_handle = block_handles_->front().handle_;
-      is_in_cache = block_handles_->front().is_cache_hit_;
+      data_block_handle = block_handles_.front().handle_;
+      is_in_cache = block_handles_.front().is_cache_hit_;
     } else {
       data_block_handle = index_iter_->value().handle;
     }
@@ -477,7 +477,7 @@ void BlockBasedTableIterator::AsyncInitDataBlock(bool is_first_pass) {
     if (is_in_cache) {
       block_iter_.Invalidate(Status::OK());
       table_->NewDataBlockIterator<DataBlockIter>(
-          read_options_, (block_handles_->front().cachable_entry_).As<Block>(),
+          read_options_, (block_handles_.front().cachable_entry_).As<Block>(),
           &block_iter_, s);
     } else {
       table_->NewDataBlockIterator<DataBlockIter>(
@@ -524,7 +524,7 @@ bool BlockBasedTableIterator::MaterializeCurrentBlock() {
   // BlockCacheLookupForReadAheadSize is called.
   Slice first_internal_key;
   if (DoesContainBlockHandles()) {
-    first_internal_key = block_handles_->front().first_internal_key_;
+    first_internal_key = block_handles_.front().first_internal_key_;
   } else {
     first_internal_key = index_iter_->value().first_internal_key;
   }
@@ -580,7 +580,7 @@ void BlockBasedTableIterator::FindBlockForward() {
     if (DoesContainBlockHandles()) {
       // Advance and point to that next Block handle to make that block handle
       // current.
-      block_handles_->pop_front();
+      block_handles_.pop_front();
     }
 
     if (!DoesContainBlockHandles()) {
@@ -681,8 +681,7 @@ void BlockBasedTableIterator::InitializeStartAndEndOffsets(
     bool read_curr_block, bool& found_first_miss_block,
     uint64_t& start_updated_offset, uint64_t& end_updated_offset,
     size_t& prev_handles_size) {
-  assert(block_handles_ != nullptr);
-  prev_handles_size = block_handles_->size();
+  prev_handles_size = block_handles_.size();
   size_t footer = table_->get_rep()->footer.GetBlockTrailerSize();
 
   // It initialize start and end offset to begin which is covered by following
@@ -702,7 +701,7 @@ void BlockBasedTableIterator::InitializeStartAndEndOffsets(
 
       end_updated_offset = block_handle_info.handle_.offset() + footer +
                            block_handle_info.handle_.size();
-      block_handles_->emplace_back(std::move(block_handle_info));
+      block_handles_.emplace_back(std::move(block_handle_info));
 
       index_iter_->Next();
       is_index_at_curr_block_ = false;
@@ -718,17 +717,17 @@ void BlockBasedTableIterator::InitializeStartAndEndOffsets(
       // Initialize prev_handles_size to 0 as all those handles need to be read
       // again.
       prev_handles_size = 0;
-      start_updated_offset = block_handles_->front().handle_.offset();
-      end_updated_offset = block_handles_->back().handle_.offset() + footer +
-                           block_handles_->back().handle_.size();
+      start_updated_offset = block_handles_.front().handle_.offset();
+      end_updated_offset = block_handles_.back().handle_.offset() + footer +
+                           block_handles_.back().handle_.size();
     }
   } else {
     // Scenario 3 : read_curr_block is false (callback made to do additional
     //              prefetching in buffers) and the queue already has some
     //              handles from first buffer.
     if (DoesContainBlockHandles()) {
-      start_updated_offset = block_handles_->back().handle_.offset() + footer +
-                             block_handles_->back().handle_.size();
+      start_updated_offset = block_handles_.back().handle_.offset() + footer +
+                             block_handles_.back().handle_.size();
       end_updated_offset = start_updated_offset;
     } else {
       // Scenario 4 : read_curr_block is false (callback made to do additional
@@ -790,9 +789,6 @@ void BlockBasedTableIterator::BlockCacheLookupForReadAheadSize(
 
   // Initialize start and end offsets based on exisiting handles in the queue
   // and read_curr_block argument passed.
-  if (block_handles_ == nullptr) {
-    block_handles_.reset(new std::deque<BlockHandleInfo>());
-  }
   InitializeStartAndEndOffsets(read_curr_block, found_first_miss_block,
                                start_updated_offset, end_updated_offset,
                                prev_handles_size);
@@ -837,7 +833,7 @@ void BlockBasedTableIterator::BlockCacheLookupForReadAheadSize(
     }
 
     // Add the handle to the queue.
-    block_handles_->emplace_back(std::move(block_handle_info));
+    block_handles_.emplace_back(std::move(block_handle_info));
 
     // Can't figure out for current block if current block
     // is out of bound. But for next block we can find that.
@@ -855,9 +851,9 @@ void BlockBasedTableIterator::BlockCacheLookupForReadAheadSize(
   if (found_first_miss_block) {
     // Iterate cache hit block handles from the end till a Miss is there, to
     // truncate and update the end offset till that Miss.
-    auto it = block_handles_->rbegin();
+    auto it = block_handles_.rbegin();
     auto it_end =
-        block_handles_->rbegin() + (block_handles_->size() - prev_handles_size);
+        block_handles_.rbegin() + (block_handles_.size() - prev_handles_size);
 
     while (it != it_end && (*it).is_cache_hit_ &&
            start_updated_offset != (*it).handle_.offset()) {

table/block_based/block_based_table_iterator.h

@@ -338,8 +338,7 @@ class BlockBasedTableIterator : public InternalIteratorBase<Slice> {
   // different blocks when readahead_size is calculated in
   // BlockCacheLookupForReadAheadSize, to avoid index_iter_ reseek,
   // block_handles_ is used.
-  // `block_handles_` is lazily constructed to save CPU when it is unused
-  std::unique_ptr<std::deque<BlockHandleInfo>> block_handles_;
+  std::deque<BlockHandleInfo> block_handles_;
 
   // During cache lookup to find readahead size, index_iter_ is iterated and it
   // can point to a different block. is_index_at_curr_block_ keeps track of
@@ -419,11 +418,7 @@ class BlockBasedTableIterator : public InternalIteratorBase<Slice> {
                 : false);
   }
 
-  void ClearBlockHandles() {
-    if (block_handles_ != nullptr) {
-      block_handles_->clear();
-    }
-  }
+  void ClearBlockHandles() { block_handles_.clear(); }
 
   // Reset prev_block_offset_. If index_iter_ has moved ahead, it won't get
   // accurate prev_block_offset_.
@@ -431,9 +426,7 @@ class BlockBasedTableIterator : public InternalIteratorBase<Slice> {
     prev_block_offset_ = std::numeric_limits<uint64_t>::max();
   }
 
-  bool DoesContainBlockHandles() {
-    return block_handles_ != nullptr && !block_handles_->empty();
-  }
+  bool DoesContainBlockHandles() { return !block_handles_.empty(); }
 
   void InitializeStartAndEndOffsets(bool read_curr_block,
                                     bool& found_first_miss_block,

table/iterator.cc

@@ -23,10 +23,6 @@ Status Iterator::GetProperty(std::string prop_name, std::string* prop) {
     *prop = "0";
     return Status::OK();
   }
-  if (prop_name == "rocksdb.iterator.is-value-pinned") {
-    *prop = "0";
-    return Status::OK();
-  }
   return Status::InvalidArgument("Unidentified property.");
 }
 

table/sst_file_dumper.cc

@@ -521,22 +521,22 @@ Status SstFileDumper::ReadSequential(bool print_kv, uint64_t read_num_limit,
               iter->value(), oss, output_hex_);
           if (!s.ok()) {
             fprintf(stderr, "%s => error deserializing wide columns\n",
-                    ikey.DebugString(true, output_hex_, ucmp).c_str());
+                    ikey.DebugString(true, output_hex_).c_str());
             continue;
           }
           fprintf(stdout, "%s => %s\n",
-                  ikey.DebugString(true, output_hex_, ucmp).c_str(),
+                  ikey.DebugString(true, output_hex_).c_str(),
                   oss.str().c_str());
         } else if (ikey.type == kTypeValuePreferredSeqno) {
           auto [unpacked_value, preferred_seqno] =
               ParsePackedValueWithSeqno(value);
           fprintf(stdout, "%s => %s, %llu\n",
-                  ikey.DebugString(true, output_hex_, ucmp).c_str(),
+                  ikey.DebugString(true, output_hex_).c_str(),
                   unpacked_value.ToString(output_hex_).c_str(),
                   static_cast<unsigned long long>(preferred_seqno));
         } else {
           fprintf(stdout, "%s => %s\n",
-                  ikey.DebugString(true, output_hex_, ucmp).c_str(),
+                  ikey.DebugString(true, output_hex_).c_str(),
                   value.ToString(output_hex_).c_str());
         }
       } else {
@@ -545,12 +545,12 @@ Status SstFileDumper::ReadSequential(bool print_kv, uint64_t read_num_limit,
         const Status s = blob_index.DecodeFrom(value);
         if (!s.ok()) {
           fprintf(stderr, "%s => error decoding blob index\n",
-                  ikey.DebugString(true, output_hex_, ucmp).c_str());
+                  ikey.DebugString(true, output_hex_).c_str());
           continue;
         }
 
         fprintf(stdout, "%s => %s\n",
-                ikey.DebugString(true, output_hex_, ucmp).c_str(),
+                ikey.DebugString(true, output_hex_).c_str(),
                 blob_index.DebugString(output_hex_).c_str());
       }
     }

test_util/testutil.cc

@@ -565,30 +565,6 @@ void DeleteDir(Env* env, const std::string& dirname) {
   TryDeleteDir(env, dirname).PermitUncheckedError();
 }
 
-FileType GetFileType(const std::string& path) {
-  FileType type = kTempFile;
-  std::size_t found = path.find_last_of('/');
-  if (found == std::string::npos) {
-    found = 0;
-  }
-  std::string file_name = path.substr(found);
-  uint64_t number = 0;
-  ParseFileName(file_name, &number, &type);
-  return type;
-}
-
-uint64_t GetFileNumber(const std::string& path) {
-  FileType type = kTempFile;
-  std::size_t found = path.find_last_of('/');
-  if (found == std::string::npos) {
-    found = 0;
-  }
-  std::string file_name = path.substr(found);
-  uint64_t number = 0;
-  ParseFileName(file_name, &number, &type);
-  return number;
-}
-
 Status CreateEnvFromSystem(const ConfigOptions& config_options, Env** result,
                            std::shared_ptr<Env>* guard) {
   const char* env_uri = getenv("TEST_ENV_URI");

test_util/testutil.h

@@ -882,12 +882,6 @@ Status TryDeleteDir(Env* env, const std::string& dirname);
 // Delete a directory if it exists
 void DeleteDir(Env* env, const std::string& dirname);
 
-// Find the FileType from the file path
-FileType GetFileType(const std::string& path);
-
-// Get the file number given the file path
-uint64_t GetFileNumber(const std::string& path);
-
 // Creates an Env from the system environment by looking at the system
 // environment variables.
 Status CreateEnvFromSystem(const ConfigOptions& options, Env** result,

tools/check_format_compatible.sh

@@ -125,7 +125,7 @@ EOF
 
 # To check for DB forward compatibility with loading options (old version
 # reading data from new), as well as backward compatibility
-declare -a db_forward_with_options_refs=("8.6.fb" "8.7.fb" "8.8.fb" "8.9.fb" "8.10.fb" "8.11.fb" "9.0.fb" "9.1.fb" "9.2.fb")
+declare -a db_forward_with_options_refs=("8.6.fb" "8.7.fb" "8.8.fb" "8.9.fb" "8.10.fb" "8.11.fb" "9.0.fb" "9.1.fb")
 # To check for DB forward compatibility without loading options (in addition
 # to the "with loading options" set), as well as backward compatibility
 declare -a db_forward_no_options_refs=() # N/A at the moment

tools/db_crashtest.py

@@ -67,11 +67,7 @@ default_params = {
     "clear_column_family_one_in": 0,
     "compact_files_one_in":  lambda: random.choice([1000, 1000000]),
     "compact_range_one_in":  lambda: random.choice([1000, 1000000]),
-    # Disabled because of various likely related failures with
-    # "Cannot delete table file #N from level 0 since it is on level X"
-    "promote_l0_one_in": 0,
     "compaction_pri": random.randint(0, 4),
-    "key_may_exist_one_in":  lambda: random.choice([100, 100000]),
     "data_block_index_type": lambda: random.choice([0, 1]),
     "delpercent": 4,
     "delrangepercent": 1,
@@ -120,7 +116,6 @@ default_params = {
     "optimize_filters_for_memory": lambda: random.randint(0, 1),
     "partition_filters": lambda: random.randint(0, 1),
     "partition_pinning": lambda: random.randint(0, 3),
-    "reset_stats_one_in": lambda: random.choice([10000, 1000000]),
     "pause_background_one_in": lambda: random.choice([10000, 1000000]),
     "disable_file_deletions_one_in": lambda: random.choice([10000, 1000000]),
     "disable_manual_compaction_one_in": lambda: random.choice([10000, 1000000]),
@@ -152,7 +147,6 @@ default_params = {
     "use_merge": lambda: random.randint(0, 1),
     # use_put_entity_one_in has to be the same across invocations for verification to work, hence no lambda
     "use_put_entity_one_in": random.choice([0] * 7 + [1, 5, 10]),
-    "use_attribute_group": lambda: random.randint(0, 1),
     # 999 -> use Bloom API
     "bloom_before_level": lambda: random.choice([random.randint(-1, 2), random.randint(-1, 10), 0x7fffffff - 1, 0x7fffffff]),
     "value_size_mult": 32,
@@ -840,10 +834,9 @@ def finalize_and_sanitize(src_params):
         # verification.
         dest_params["acquire_snapshot_one_in"] = 0
         dest_params["compact_range_one_in"] = 0
-        # Redistribute to maintain 100% total
-        dest_params["readpercent"] += dest_params.get("iterpercent", 10) + dest_params.get("prefixpercent", 20)
+        # Give the iterator ops away to reads.
+        dest_params["readpercent"] += dest_params.get("iterpercent", 10)
         dest_params["iterpercent"] = 0
-        dest_params["prefixpercent"] = 0
         dest_params["check_multiget_consistency"] = 0
         dest_params["check_multiget_entity_consistency"] = 0
     if dest_params.get("disable_wal") == 1:
@@ -866,6 +859,7 @@ def finalize_and_sanitize(src_params):
         dest_params["use_timed_put_one_in"] = 3
     return dest_params
 
+
 def gen_cmd_params(args):
     params = {}
 
@@ -1115,13 +1109,13 @@ def whitebox_crash_main(args, unknown_args):
             }
             # Single level universal has a lot of special logic. Ensure we cover
             # it sometimes.
-            if not args.test_tiered_storage and random.randint(0, 1) == 1:
+            if random.randint(0, 1) == 1:
                 additional_opts.update(
                     {
                         "num_levels": 1,
                     }
                 )
-        elif check_mode == 2 and not args.test_tiered_storage:
+        elif check_mode == 2:
             # normal run with FIFO compaction mode
             # ops_per_thread is divided by 5 because FIFO compaction
             # style is quite a bit slower on reads with lot of files

tools/ldb_cmd.cc

@@ -107,7 +107,6 @@ const std::string LDBCommand::ARG_PREPOPULATE_BLOB_CACHE =
 const std::string LDBCommand::ARG_DECODE_BLOB_INDEX = "decode_blob_index";
 const std::string LDBCommand::ARG_DUMP_UNCOMPRESSED_BLOBS =
     "dump_uncompressed_blobs";
-const std::string LDBCommand::ARG_READ_TIMESTAMP = "read_timestamp";
 
 const char* LDBCommand::DELIM = " ==> ";
 
@@ -723,46 +722,6 @@ bool LDBCommand::ParseCompressionTypeOption(
   return false;
 }
 
-Status LDBCommand::MaybePopulateReadTimestamp(ColumnFamilyHandle* cfh,
-                                              ReadOptions& ropts,
-                                              Slice* read_timestamp) {
-  const size_t ts_sz = cfh->GetComparator()->timestamp_size();
-
-  auto iter = option_map_.find(ARG_READ_TIMESTAMP);
-  if (iter == option_map_.end()) {
-    if (ts_sz == 0) {
-      return Status::OK();
-    }
-    return Status::InvalidArgument(
-        "column family enables user-defined timestamp while --read_timestamp "
-        "is not provided.");
-  }
-  if (iter->second.empty()) {
-    if (ts_sz == 0) {
-      return Status::OK();
-    }
-    return Status::InvalidArgument(
-        "column family enables user-defined timestamp while --read_timestamp "
-        "is empty.");
-  }
-  if (ts_sz == 0) {
-    return Status::InvalidArgument(
-        "column family does not enable user-defined timestamps while "
-        "--read_timestamp is provided.");
-  }
-  uint64_t int_timestamp;
-  std::istringstream iss(iter->second);
-  if (!(iss >> int_timestamp)) {
-    return Status::InvalidArgument(
-        "column family enables user-defined timestamp while --read_timestamp "
-        "is not a valid uint64 value.");
-  }
-  EncodeU64Ts(int_timestamp, &read_timestamp_);
-  *read_timestamp = read_timestamp_;
-  ropts.timestamp = read_timestamp;
-  return Status::OK();
-}
-
 void LDBCommand::OverrideBaseOptions() {
   options_.create_if_missing = false;
 
@@ -810,10 +769,7 @@ void LDBCommand::OverrideBaseCFOptions(ColumnFamilyOptions* cf_opts) {
     }
   }
 
-  // Default comparator is BytewiseComparator, so only when it's not, it
-  // means user has a command line override.
-  if (options_.comparator != nullptr &&
-      options_.comparator != BytewiseComparator()) {
+  if (options_.comparator != nullptr) {
     cf_opts->comparator = options_.comparator;
   }
 
@@ -2890,9 +2846,9 @@ void WALDumperCommand::DoCommand() {
 GetCommand::GetCommand(const std::vector<std::string>& params,
                        const std::map<std::string, std::string>& options,
                        const std::vector<std::string>& flags)
-    : LDBCommand(options, flags, true,
-                 BuildCmdLineOptions({ARG_TTL, ARG_HEX, ARG_KEY_HEX,
-                                      ARG_VALUE_HEX, ARG_READ_TIMESTAMP})) {
+    : LDBCommand(
+          options, flags, true,
+          BuildCmdLineOptions({ARG_TTL, ARG_HEX, ARG_KEY_HEX, ARG_VALUE_HEX})) {
   if (params.size() != 1) {
     exec_state_ = LDBCommandExecuteResult::Failed(
         "<key> must be specified for the get command");
@@ -2909,7 +2865,6 @@ void GetCommand::Help(std::string& ret) {
   ret.append("  ");
   ret.append(GetCommand::Name());
   ret.append(" <key>");
-  ret.append(" [--" + ARG_READ_TIMESTAMP + "=<uint64_ts>] ");
   ret.append(" [--" + ARG_TTL + "]");
   ret.append("\n");
 }
@@ -2919,18 +2874,8 @@ void GetCommand::DoCommand() {
     assert(GetExecuteState().IsFailed());
     return;
   }
-  ReadOptions ropts;
-  Slice read_timestamp;
-  ColumnFamilyHandle* cfh = GetCfHandle();
-  Status st = MaybePopulateReadTimestamp(cfh, ropts, &read_timestamp);
-  if (!st.ok()) {
-    std::stringstream oss;
-    oss << "Get failed: " << st.ToString();
-    exec_state_ = LDBCommandExecuteResult::Failed(oss.str());
-    return;
-  }
   std::string value;
-  st = db_->Get(ropts, cfh, key_, &value);
+  Status st = db_->Get(ReadOptions(), GetCfHandle(), key_, &value);
   if (st.ok()) {
     fprintf(stdout, "%s\n",
             (is_value_hex_ ? StringToHex(value) : value).c_str());
@@ -2950,8 +2895,7 @@ MultiGetCommand::MultiGetCommand(
     const std::map<std::string, std::string>& options,
     const std::vector<std::string>& flags)
     : LDBCommand(options, flags, true,
-                 BuildCmdLineOptions({ARG_HEX, ARG_KEY_HEX, ARG_VALUE_HEX,
-                                      ARG_READ_TIMESTAMP})) {
+                 BuildCmdLineOptions({ARG_HEX, ARG_KEY_HEX, ARG_VALUE_HEX})) {
   if (params.size() < 1) {
     exec_state_ = LDBCommandExecuteResult::Failed(
         "At least one <key> must be specified for multi_get.");
@@ -2967,7 +2911,6 @@ void MultiGetCommand::Help(std::string& ret) {
   ret.append("  ");
   ret.append(MultiGetCommand::Name());
   ret.append(" <key_1> <key_2> <key_3> ...");
-  ret.append(" [--" + ARG_READ_TIMESTAMP + "=<uint64_ts>] ");
   ret.append("\n");
 }
 
@@ -2976,16 +2919,6 @@ void MultiGetCommand::DoCommand() {
     assert(GetExecuteState().IsFailed());
     return;
   }
-  ReadOptions ropts;
-  Slice read_timestamp;
-  ColumnFamilyHandle* cfh = GetCfHandle();
-  Status st = MaybePopulateReadTimestamp(cfh, ropts, &read_timestamp);
-  if (!st.ok()) {
-    std::stringstream oss;
-    oss << "MultiGet failed: " << st.ToString();
-    exec_state_ = LDBCommandExecuteResult::Failed(oss.str());
-    return;
-  }
   size_t num_keys = keys_.size();
   std::vector<Slice> key_slices;
   std::vector<PinnableSlice> values(num_keys);
@@ -2993,8 +2926,8 @@ void MultiGetCommand::DoCommand() {
   for (const std::string& key : keys_) {
     key_slices.emplace_back(key);
   }
-  db_->MultiGet(ropts, cfh, num_keys, key_slices.data(), values.data(),
-                statuses.data());
+  db_->MultiGet(ReadOptions(), GetCfHandle(), num_keys, key_slices.data(),
+                values.data(), statuses.data());
 
   bool failed = false;
   for (size_t i = 0; i < num_keys; ++i) {
@@ -3024,9 +2957,9 @@ GetEntityCommand::GetEntityCommand(
     const std::vector<std::string>& params,
     const std::map<std::string, std::string>& options,
     const std::vector<std::string>& flags)
-    : LDBCommand(options, flags, true,
-                 BuildCmdLineOptions({ARG_TTL, ARG_HEX, ARG_KEY_HEX,
-                                      ARG_VALUE_HEX, ARG_READ_TIMESTAMP})) {
+    : LDBCommand(
+          options, flags, true,
+          BuildCmdLineOptions({ARG_TTL, ARG_HEX, ARG_KEY_HEX, ARG_VALUE_HEX})) {
   if (params.size() != 1) {
     exec_state_ = LDBCommandExecuteResult::Failed(
         "<key> must be specified for the get_entity command");
@@ -3043,7 +2976,6 @@ void GetEntityCommand::Help(std::string& ret) {
   ret.append("  ");
   ret.append(GetEntityCommand::Name());
   ret.append(" <key>");
-  ret.append(" [--" + ARG_READ_TIMESTAMP + "=<uint64_ts>] ");
   ret.append(" [--" + ARG_TTL + "]");
   ret.append("\n");
 }
@@ -3053,18 +2985,9 @@ void GetEntityCommand::DoCommand() {
     assert(GetExecuteState().IsFailed());
     return;
   }
-  ReadOptions ropt;
-  Slice read_timestamp;
-  ColumnFamilyHandle* cfh = GetCfHandle();
-  Status st = MaybePopulateReadTimestamp(cfh, ropt, &read_timestamp);
-  if (!st.ok()) {
-    std::stringstream oss;
-    oss << "GetEntity failed: " << st.ToString();
-    exec_state_ = LDBCommandExecuteResult::Failed(oss.str());
-    return;
-  }
   PinnableWideColumns pinnable_wide_columns;
-  st = db_->GetEntity(ropt, cfh, key_, &pinnable_wide_columns);
+  Status st = db_->GetEntity(ReadOptions(), GetCfHandle(), key_,
+                             &pinnable_wide_columns);
   if (st.ok()) {
     std::ostringstream oss;
     WideColumnsHelper::DumpWideColumns(pinnable_wide_columns.columns(), oss,
@@ -3084,8 +3007,7 @@ MultiGetEntityCommand::MultiGetEntityCommand(
     const std::map<std::string, std::string>& options,
     const std::vector<std::string>& flags)
     : LDBCommand(options, flags, true /* is_read_only */,
-                 BuildCmdLineOptions({ARG_HEX, ARG_KEY_HEX, ARG_VALUE_HEX,
-                                      ARG_READ_TIMESTAMP})) {
+                 BuildCmdLineOptions({ARG_HEX, ARG_KEY_HEX, ARG_VALUE_HEX})) {
   if (params.size() < 1) {
     exec_state_ = LDBCommandExecuteResult::Failed(
         "At least one <key> must be specified for the multi_get_entity "
@@ -3102,7 +3024,6 @@ void MultiGetEntityCommand::Help(std::string& ret) {
   ret.append("  ");
   ret.append(MultiGetEntityCommand::Name());
   ret.append(" <key_1> <key_2> <key_3> ...");
-  ret.append(" [--" + ARG_READ_TIMESTAMP + "=<uint64_ts>] ");
   ret.append("\n");
 }
 
@@ -3112,16 +3033,6 @@ void MultiGetEntityCommand::DoCommand() {
     return;
   }
 
-  ReadOptions ropt;
-  Slice read_timestamp;
-  ColumnFamilyHandle* cfh = GetCfHandle();
-  Status st = MaybePopulateReadTimestamp(cfh, ropt, &read_timestamp);
-  if (!st.ok()) {
-    std::stringstream oss;
-    oss << "MultiGetEntity failed: " << st.ToString();
-    exec_state_ = LDBCommandExecuteResult::Failed(oss.str());
-    return;
-  }
   size_t num_keys = keys_.size();
   std::vector<Slice> key_slices;
   std::vector<PinnableWideColumns> results(num_keys);
@@ -3130,8 +3041,8 @@ void MultiGetEntityCommand::DoCommand() {
     key_slices.emplace_back(key);
   }
 
-  db_->MultiGetEntity(ropt, cfh, num_keys, key_slices.data(), results.data(),
-                      statuses.data());
+  db_->MultiGetEntity(ReadOptions(), GetCfHandle(), num_keys, key_slices.data(),
+                      results.data(), statuses.data());
 
   bool failed = false;
   for (size_t i = 0; i < num_keys; ++i) {
@@ -3289,11 +3200,11 @@ void BatchPutCommand::OverrideBaseOptions() {
 ScanCommand::ScanCommand(const std::vector<std::string>& /*params*/,
                          const std::map<std::string, std::string>& options,
                          const std::vector<std::string>& flags)
-    : LDBCommand(options, flags, true,
-                 BuildCmdLineOptions(
-                     {ARG_TTL, ARG_NO_VALUE, ARG_HEX, ARG_KEY_HEX, ARG_TO,
-                      ARG_VALUE_HEX, ARG_FROM, ARG_TIMESTAMP, ARG_MAX_KEYS,
-                      ARG_TTL_START, ARG_TTL_END, ARG_READ_TIMESTAMP})),
+    : LDBCommand(
+          options, flags, true,
+          BuildCmdLineOptions({ARG_TTL, ARG_NO_VALUE, ARG_HEX, ARG_KEY_HEX,
+                               ARG_TO, ARG_VALUE_HEX, ARG_FROM, ARG_TIMESTAMP,
+                               ARG_MAX_KEYS, ARG_TTL_START, ARG_TTL_END})),
       start_key_specified_(false),
       end_key_specified_(false),
       max_keys_scanned_(-1),
@@ -3349,7 +3260,6 @@ void ScanCommand::Help(std::string& ret) {
   ret.append(" [--" + ARG_TTL_START + "=<N>:- is inclusive]");
   ret.append(" [--" + ARG_TTL_END + "=<N>:- is exclusive]");
   ret.append(" [--" + ARG_NO_VALUE + "]");
-  ret.append(" [--" + ARG_READ_TIMESTAMP + "=<uint64_ts>] ");
   ret.append("\n");
 }
 
@@ -3361,17 +3271,8 @@ void ScanCommand::DoCommand() {
 
   int num_keys_scanned = 0;
   ReadOptions scan_read_opts;
-  ColumnFamilyHandle* cfh = GetCfHandle();
-  Slice read_timestamp;
-  Status st = MaybePopulateReadTimestamp(cfh, scan_read_opts, &read_timestamp);
-  if (!st.ok()) {
-    std::stringstream oss;
-    oss << "Scan failed: " << st.ToString();
-    exec_state_ = LDBCommandExecuteResult::Failed(oss.str());
-    return;
-  }
   scan_read_opts.total_order_seek = true;
-  Iterator* it = db_->NewIterator(scan_read_opts, cfh);
+  Iterator* it = db_->NewIterator(scan_read_opts, GetCfHandle());
   if (start_key_specified_) {
     it->Seek(start_key_);
   } else {

tools/ldb_tool.cc

@@ -85,9 +85,6 @@ void LDBCommandRunner::PrintHelp(const LDBOptions& ldb_options,
              "=<double,e.g.:0.25>\n");
   ret.append("  --" + LDBCommand::ARG_BLOB_COMPACTION_READAHEAD_SIZE +
              "=<int,e.g.:2097152>\n");
-  ret.append("  --" + LDBCommand::ARG_READ_TIMESTAMP +
-             "=<uint64_ts, e.g.:323> : read timestamp, required if column "
-             "family enables timestamp, otherwise invalid if provided.");
 
   ret.append("\n\n");
   ret.append("Data Access Commands:\n");

util/comparator.cc

@@ -276,13 +276,6 @@ class ComparatorWithU64TsImpl : public Comparator {
 
   Slice GetMinTimestamp() const override { return MinU64Ts(); }
 
-  std::string TimestampToString(const Slice& timestamp) const override {
-    assert(timestamp.size() == sizeof(uint64_t));
-    uint64_t ts = 0;
-    DecodeU64Ts(timestamp, &ts).PermitUncheckedError();
-    return std::to_string(ts);
-  }
-
   using Comparator::CompareWithoutTimestamp;
   int CompareWithoutTimestamp(const Slice& a, bool a_has_ts, const Slice& b,
                               bool b_has_ts) const override {

util/stderr_logger.cc

@@ -38,12 +38,12 @@ void StderrLogger::Logv(const char* format, va_list ap) {
 
   va_list ap_copy;
   va_copy(ap_copy, ap);
-  const size_t log_suffix_len = vsnprintf(nullptr, 0, format, ap_copy) + 1;
+  const size_t log_suffix_len = vsnprintf(nullptr, 0, format, ap_copy);
   va_end(ap_copy);
 
   // Allocate space for the context, log_prefix, and log itself
   // Extra byte for null termination
-  size_t buf_len = ctx_len + log_prefix_len + log_suffix_len;
+  size_t buf_len = ctx_len + log_prefix_len + log_suffix_len + 1;
   std::unique_ptr<char[]> buf(new char[buf_len]);
 
   // If the logger was created without a prefix, the prefix is a nullptr
@@ -55,7 +55,8 @@ void StderrLogger::Logv(const char* format, va_list ap) {
                t.tm_year + 1900, t.tm_mon + 1, t.tm_mday, t.tm_hour, t.tm_min,
                t.tm_sec, static_cast<int>(now_tv.tv_usec),
                static_cast<long long unsigned int>(thread_id), prefix);
-  vsnprintf(buf.get() + written, log_suffix_len, format, ap);
+  written += vsnprintf(buf.get() + written, log_suffix_len, format, ap);
+  buf[written] = '\0';
 
   fprintf(stderr, "%s%c", buf.get(), '\n');
 }

utilities/transactions/optimistic_transaction_test.cc

@@ -1142,12 +1142,8 @@ TEST_P(OptimisticTransactionTest, UntrackedWrites) {
   delete txn;
   txn = txn_db->BeginTransaction(write_options);
 
-  const WideColumns untracked_columns{{"hello", "world"}};
-
   ASSERT_OK(txn->Put("tracked", "1"));
   ASSERT_OK(txn->PutUntracked("untracked", "1"));
-  ASSERT_OK(txn->PutEntityUntracked(txn_db->DefaultColumnFamily(), "untracked",
-                                    untracked_columns));
   ASSERT_OK(txn->MergeUntracked("untracked", "2"));
   ASSERT_OK(txn->DeleteUntracked("untracked"));
 
@@ -1163,12 +1159,8 @@ TEST_P(OptimisticTransactionTest, UntrackedWrites) {
   delete txn;
   txn = txn_db->BeginTransaction(write_options);
 
-  const WideColumns untracked_new_columns{{"foo", "bar"}};
-
   ASSERT_OK(txn->Put("tracked", "10"));
   ASSERT_OK(txn->PutUntracked("untracked", "A"));
-  ASSERT_OK(txn->PutEntityUntracked(txn_db->DefaultColumnFamily(), "untracked",
-                                    untracked_new_columns));
 
   // Write to tracked key outside of the transaction and verify that the
   // untracked keys are not written when the commit fails.
@@ -1695,434 +1687,6 @@ TEST_P(OptimisticTransactionTest, TimestampedSnapshotSetCommitTs) {
   ASSERT_TRUE(s.IsNotSupported());
 }
 
-TEST_P(OptimisticTransactionTest, PutEntitySuccess) {
-  constexpr char foo[] = "foo";
-  const WideColumns foo_columns{
-      {kDefaultWideColumnName, "bar"}, {"col1", "val1"}, {"col2", "val2"}};
-  const WideColumns foo_new_columns{
-      {kDefaultWideColumnName, "baz"}, {"colA", "valA"}, {"colB", "valB"}};
-
-  ASSERT_OK(txn_db->PutEntity(WriteOptions(), txn_db->DefaultColumnFamily(),
-                              foo, foo_columns));
-
-  {
-    std::unique_ptr<Transaction> txn(txn_db->BeginTransaction(WriteOptions()));
-
-    ASSERT_NE(txn, nullptr);
-    ASSERT_EQ(txn->GetNumPutEntities(), 0);
-
-    {
-      PinnableWideColumns columns;
-      ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                               foo, &columns));
-      ASSERT_EQ(columns.columns(), foo_columns);
-    }
-
-    {
-      PinnableSlice value;
-      ASSERT_OK(txn->GetForUpdate(ReadOptions(), foo, &value));
-      ASSERT_EQ(value, foo_columns[0].value());
-    }
-
-    ASSERT_OK(
-        txn->PutEntity(txn_db->DefaultColumnFamily(), foo, foo_new_columns));
-
-    ASSERT_EQ(txn->GetNumPutEntities(), 1);
-
-    {
-      PinnableWideColumns columns;
-      ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                               foo, &columns));
-      ASSERT_EQ(columns.columns(), foo_new_columns);
-    }
-
-    {
-      PinnableSlice value;
-      ASSERT_OK(txn->GetForUpdate(ReadOptions(), foo, &value));
-      ASSERT_EQ(value, foo_new_columns[0].value());
-    }
-
-    ASSERT_OK(txn->Commit());
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn_db->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                                foo, &columns));
-    ASSERT_EQ(columns.columns(), foo_new_columns);
-  }
-}
-
-TEST_P(OptimisticTransactionTest, PutEntityWriteConflict) {
-  constexpr char foo[] = "foo";
-  const WideColumns foo_columns{
-      {kDefaultWideColumnName, "bar"}, {"col1", "val1"}, {"col2", "val2"}};
-  constexpr char baz[] = "baz";
-  const WideColumns baz_columns{
-      {kDefaultWideColumnName, "quux"}, {"colA", "valA"}, {"colB", "valB"}};
-
-  ASSERT_OK(txn_db->PutEntity(WriteOptions(), txn_db->DefaultColumnFamily(),
-                              foo, foo_columns));
-  ASSERT_OK(txn_db->PutEntity(WriteOptions(), txn_db->DefaultColumnFamily(),
-                              baz, baz_columns));
-
-  std::unique_ptr<Transaction> txn(txn_db->BeginTransaction(WriteOptions()));
-  ASSERT_NE(txn, nullptr);
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), foo,
-                             &columns));
-    ASSERT_EQ(columns.columns(), foo_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), baz,
-                             &columns));
-    ASSERT_EQ(columns.columns(), baz_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    txn->MultiGetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data());
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_columns);
-    ASSERT_EQ(results[1].columns(), baz_columns);
-  }
-
-  const WideColumns foo_new_columns{{kDefaultWideColumnName, "FOO"},
-                                    {"hello", "world"}};
-  const WideColumns baz_new_columns{{kDefaultWideColumnName, "BAZ"},
-                                    {"ping", "pong"}};
-
-  ASSERT_OK(
-      txn->PutEntity(txn_db->DefaultColumnFamily(), foo, foo_new_columns));
-  ASSERT_OK(
-      txn->PutEntity(txn_db->DefaultColumnFamily(), baz, baz_new_columns));
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), foo,
-                             &columns));
-    ASSERT_EQ(columns.columns(), foo_new_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), baz,
-                             &columns));
-    ASSERT_EQ(columns.columns(), baz_new_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    txn->MultiGetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data());
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_new_columns);
-    ASSERT_EQ(results[1].columns(), baz_new_columns);
-  }
-
-  // This PutEntity outside of a transaction will conflict with the previous
-  // write
-  const WideColumns foo_conflict_columns{{kDefaultWideColumnName, "X"},
-                                         {"conflicting", "write"}};
-  ASSERT_OK(txn_db->PutEntity(WriteOptions(), txn_db->DefaultColumnFamily(),
-                              foo, foo_conflict_columns));
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn_db->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                                foo, &columns));
-    ASSERT_EQ(columns.columns(), foo_conflict_columns);
-  }
-
-  ASSERT_TRUE(txn->Commit().IsBusy());  // Txn should not commit
-
-  // Verify that transaction did not write anything
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn_db->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                                foo, &columns));
-    ASSERT_EQ(columns.columns(), foo_conflict_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn_db->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                                baz, &columns));
-    ASSERT_EQ(columns.columns(), baz_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn_db->MultiGetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                           num_keys, keys.data(), results.data(),
-                           statuses.data(), sorted_input);
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_conflict_columns);
-    ASSERT_EQ(results[1].columns(), baz_columns);
-  }
-}
-
-TEST_P(OptimisticTransactionTest, PutEntityWriteConflictTxnTxn) {
-  constexpr char foo[] = "foo";
-  const WideColumns foo_columns{
-      {kDefaultWideColumnName, "bar"}, {"col1", "val1"}, {"col2", "val2"}};
-  constexpr char baz[] = "baz";
-  const WideColumns baz_columns{
-      {kDefaultWideColumnName, "quux"}, {"colA", "valA"}, {"colB", "valB"}};
-
-  ASSERT_OK(txn_db->PutEntity(WriteOptions(), txn_db->DefaultColumnFamily(),
-                              foo, foo_columns));
-  ASSERT_OK(txn_db->PutEntity(WriteOptions(), txn_db->DefaultColumnFamily(),
-                              baz, baz_columns));
-
-  std::unique_ptr<Transaction> txn(txn_db->BeginTransaction(WriteOptions()));
-  ASSERT_NE(txn, nullptr);
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), foo,
-                             &columns));
-    ASSERT_EQ(columns.columns(), foo_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), baz,
-                             &columns));
-    ASSERT_EQ(columns.columns(), baz_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    txn->MultiGetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data());
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_columns);
-    ASSERT_EQ(results[1].columns(), baz_columns);
-  }
-
-  const WideColumns foo_new_columns{{kDefaultWideColumnName, "FOO"},
-                                    {"hello", "world"}};
-  const WideColumns baz_new_columns{{kDefaultWideColumnName, "BAZ"},
-                                    {"ping", "pong"}};
-
-  ASSERT_OK(
-      txn->PutEntity(txn_db->DefaultColumnFamily(), foo, foo_new_columns));
-  ASSERT_OK(
-      txn->PutEntity(txn_db->DefaultColumnFamily(), baz, baz_new_columns));
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), foo,
-                             &columns));
-    ASSERT_EQ(columns.columns(), foo_new_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), baz,
-                             &columns));
-    ASSERT_EQ(columns.columns(), baz_new_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    txn->MultiGetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data());
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_new_columns);
-    ASSERT_EQ(results[1].columns(), baz_new_columns);
-  }
-
-  std::unique_ptr<Transaction> conflicting_txn(
-      txn_db->BeginTransaction(WriteOptions()));
-  ASSERT_NE(conflicting_txn, nullptr);
-
-  const WideColumns foo_conflict_columns{{kDefaultWideColumnName, "X"},
-                                         {"conflicting", "write"}};
-  ASSERT_OK(conflicting_txn->PutEntity(txn_db->DefaultColumnFamily(), foo,
-                                       foo_conflict_columns));
-  ASSERT_OK(conflicting_txn->Commit());
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn_db->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                                foo, &columns));
-    ASSERT_EQ(columns.columns(), foo_conflict_columns);
-  }
-
-  ASSERT_TRUE(txn->Commit().IsBusy());  // Txn should not commit
-
-  // Verify that transaction did not write anything
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn_db->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                                foo, &columns));
-    ASSERT_EQ(columns.columns(), foo_conflict_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn_db->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                                baz, &columns));
-    ASSERT_EQ(columns.columns(), baz_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn_db->MultiGetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                           num_keys, keys.data(), results.data(),
-                           statuses.data(), sorted_input);
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_conflict_columns);
-    ASSERT_EQ(results[1].columns(), baz_columns);
-  }
-}
-
-TEST_P(OptimisticTransactionTest, EntityReadSanityChecks) {
-  constexpr char foo[] = "foo";
-  constexpr char bar[] = "bar";
-  constexpr size_t num_keys = 2;
-
-  std::unique_ptr<Transaction> txn(txn_db->BeginTransaction(WriteOptions()));
-  ASSERT_NE(txn, nullptr);
-
-  {
-    constexpr ColumnFamilyHandle* column_family = nullptr;
-    PinnableWideColumns columns;
-    ASSERT_TRUE(txn->GetEntity(ReadOptions(), column_family, foo, &columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    constexpr PinnableWideColumns* columns = nullptr;
-    ASSERT_TRUE(txn->GetEntity(ReadOptions(), txn_db->DefaultColumnFamily(),
-                               foo, columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kGet;
-
-    PinnableWideColumns columns;
-    ASSERT_TRUE(txn->GetEntity(read_options, txn_db->DefaultColumnFamily(), foo,
-                               &columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    constexpr ColumnFamilyHandle* column_family = nullptr;
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn->MultiGetEntity(ReadOptions(), column_family, num_keys, keys.data(),
-                        results.data(), statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    constexpr Slice* keys = nullptr;
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn->MultiGetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), num_keys,
-                        keys, results.data(), statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    constexpr PinnableWideColumns* results = nullptr;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn->MultiGetEntity(ReadOptions(), txn_db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results, statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kMultiGet;
-
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn->MultiGetEntity(read_options, txn_db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data(),
-                        sorted_input);
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-}
-
 INSTANTIATE_TEST_CASE_P(
     InstanceOccGroup, OptimisticTransactionTest,
     testing::Values(OccValidationPolicy::kValidateSerial,

utilities/transactions/pessimistic_transaction.cc

@@ -3,6 +3,7 @@
 //  COPYING file in the root directory) and Apache 2.0 License
 //  (found in the LICENSE.Apache file in the root directory).
 
+
 #include "utilities/transactions/pessimistic_transaction.h"
 
 #include <map>
@@ -217,21 +218,6 @@ inline Status WriteCommittedTxn::GetForUpdateImpl(
                                            value, exclusive, do_validate);
 }
 
-Status WriteCommittedTxn::PutEntityImpl(ColumnFamilyHandle* column_family,
-                                        const Slice& key,
-                                        const WideColumns& columns,
-                                        bool do_validate, bool assume_tracked) {
-  return Operate(column_family, key, do_validate, assume_tracked,
-                 [column_family, &key, &columns, this]() {
-                   Status s = GetBatchForWrite()->PutEntity(column_family, key,
-                                                            columns);
-                   if (s.ok()) {
-                     ++num_put_entities_;
-                   }
-                   return s;
-                 });
-}
-
 Status WriteCommittedTxn::Put(ColumnFamilyHandle* column_family,
                               const Slice& key, const Slice& value,
                               const bool assume_tracked) {
@@ -910,11 +896,6 @@ Status PessimisticTransaction::LockBatch(WriteBatch* batch,
       RecordKey(column_family_id, key);
       return Status::OK();
     }
-    Status PutEntityCF(uint32_t column_family_id, const Slice& key,
-                       const Slice& /* unused */) override {
-      RecordKey(column_family_id, key);
-      return Status::OK();
-    }
     Status MergeCF(uint32_t column_family_id, const Slice& key,
                    const Slice& /* unused */) override {
       RecordKey(column_family_id, key);

utilities/transactions/pessimistic_transaction.h

@@ -5,6 +5,7 @@
 
 #pragma once
 
+
 #include <algorithm>
 #include <atomic>
 #include <mutex>
@@ -248,25 +249,6 @@ class WriteCommittedTxn : public PessimisticTransaction {
   Status PutUntracked(ColumnFamilyHandle* column_family, const SliceParts& key,
                       const SliceParts& value) override;
 
-  // `key` does NOT include timestamp even when it's enabled.
-  Status PutEntity(ColumnFamilyHandle* column_family, const Slice& key,
-                   const WideColumns& columns,
-                   bool assume_tracked = false) override {
-    const bool do_validate = !assume_tracked;
-
-    return PutEntityImpl(column_family, key, columns, do_validate,
-                         assume_tracked);
-  }
-
-  Status PutEntityUntracked(ColumnFamilyHandle* column_family, const Slice& key,
-                            const WideColumns& columns) override {
-    constexpr bool do_validate = false;
-    constexpr bool assume_tracked = false;
-
-    return PutEntityImpl(column_family, key, columns, do_validate,
-                         assume_tracked);
-  }
-
   using TransactionBaseImpl::Delete;
   // `key` does NOT include timestamp even when it's enabled.
   Status Delete(ColumnFamilyHandle* column_family, const Slice& key,
@@ -306,10 +288,6 @@ class WriteCommittedTxn : public PessimisticTransaction {
                           TValue* value, bool exclusive,
                           const bool do_validate);
 
-  Status PutEntityImpl(ColumnFamilyHandle* column_family, const Slice& key,
-                       const WideColumns& columns, bool do_validate,
-                       bool assume_tracked);
-
   template <typename TKey, typename TOperation>
   Status Operate(ColumnFamilyHandle* column_family, const TKey& key,
                  const bool do_validate, const bool assume_tracked,

utilities/transactions/pessimistic_transaction_db.cc

@@ -3,10 +3,10 @@
 //  COPYING file in the root directory) and Apache 2.0 License
 //  (found in the LICENSE.Apache file in the root directory).
 
+
 #include "utilities/transactions/pessimistic_transaction_db.h"
 
 #include <cinttypes>
-#include <memory>
 #include <sstream>
 #include <string>
 #include <unordered_set>
@@ -512,15 +512,15 @@ Transaction* PessimisticTransactionDB::BeginInternalTransaction(
   return txn;
 }
 
-// All user Put, PutEntity, Merge, Delete, and Write requests must be
-// intercepted to make sure that they lock all keys that they are writing to
-// avoid causing conflicts with any concurrent transactions. The easiest way to
-// do this is to wrap all write operations in a transaction.
+// All user Put, Merge, Delete, and Write requests must be intercepted to make
+// sure that they lock all keys that they are writing to avoid causing conflicts
+// with any concurrent transactions. The easiest way to do this is to wrap all
+// write operations in a transaction.
 //
-// Put(), PutEntity(), Merge(), and Delete() only lock a single key per call.
-// Write() will sort its keys before locking them.  This guarantees that
-// TransactionDB write methods cannot deadlock with each other (but still could
-// deadlock with a Transaction).
+// Put(), Merge(), and Delete() only lock a single key per call.  Write() will
+// sort its keys before locking them.  This guarantees that TransactionDB write
+// methods cannot deadlock with each other (but still could deadlock with a
+// Transaction).
 Status PessimisticTransactionDB::Put(const WriteOptions& options,
                                      ColumnFamilyHandle* column_family,
                                      const Slice& key, const Slice& val) {
@@ -545,42 +545,6 @@ Status PessimisticTransactionDB::Put(const WriteOptions& options,
   return s;
 }
 
-Status PessimisticTransactionDB::PutEntity(const WriteOptions& options,
-                                           ColumnFamilyHandle* column_family,
-                                           const Slice& key,
-                                           const WideColumns& columns) {
-  {
-    const Status s = FailIfCfEnablesTs(this, column_family);
-    if (!s.ok()) {
-      return s;
-    }
-  }
-
-  {
-    std::unique_ptr<Transaction> txn(BeginInternalTransaction(options));
-    txn->DisableIndexing();
-
-    // Since the client didn't create a transaction, they don't care about
-    // conflict checking for this write.  So we just need to do
-    // PutEntityUntracked().
-    {
-      const Status s = txn->PutEntityUntracked(column_family, key, columns);
-      if (!s.ok()) {
-        return s;
-      }
-    }
-
-    {
-      const Status s = txn->Commit();
-      if (!s.ok()) {
-        return s;
-      }
-    }
-  }
-
-  return Status::OK();
-}
-
 Status PessimisticTransactionDB::Delete(const WriteOptions& wopts,
                                         ColumnFamilyHandle* column_family,
                                         const Slice& key) {

utilities/transactions/pessimistic_transaction_db.h

@@ -50,20 +50,6 @@ class PessimisticTransactionDB : public TransactionDB {
   Status Put(const WriteOptions& options, ColumnFamilyHandle* column_family,
              const Slice& key, const Slice& val) override;
 
-  Status PutEntity(const WriteOptions& options,
-                   ColumnFamilyHandle* column_family, const Slice& key,
-                   const WideColumns& columns) override;
-  Status PutEntity(const WriteOptions& /* options */, const Slice& /* key */,
-                   const AttributeGroups& attribute_groups) override {
-    if (attribute_groups.empty()) {
-      return Status::InvalidArgument(
-          "Cannot call this method without attribute groups");
-    }
-    return Status::NotSupported(
-        "PutEntity with AttributeGroups not supported by "
-        "PessimisticTransactionDB");
-  }
-
   using StackableDB::Delete;
   Status Delete(const WriteOptions& wopts, ColumnFamilyHandle* column_family,
                 const Slice& key) override;

utilities/transactions/transaction_base.cc

@@ -3,6 +3,7 @@
 //  COPYING file in the root directory) and Apache 2.0 License
 //  (found in the LICENSE.Apache file in the root directory).
 
+
 #include "utilities/transactions/transaction_base.h"
 
 #include <cinttypes>
@@ -89,7 +90,6 @@ void TransactionBaseImpl::Clear() {
   commit_time_batch_.Clear();
   tracked_locks_->Clear();
   num_puts_ = 0;
-  num_put_entities_ = 0;
   num_deletes_ = 0;
   num_merges_ = 0;
 
@@ -177,7 +177,7 @@ void TransactionBaseImpl::SetSavePoint() {
                        autovector<TransactionBaseImpl::SavePoint>>());
   }
   save_points_->emplace(snapshot_, snapshot_needed_, snapshot_notifier_,
-                        num_puts_, num_put_entities_, num_deletes_, num_merges_,
+                        num_puts_, num_deletes_, num_merges_,
                         lock_tracker_factory_);
   write_batch_.SetSavePoint();
 }
@@ -190,7 +190,6 @@ Status TransactionBaseImpl::RollbackToSavePoint() {
     snapshot_needed_ = save_point.snapshot_needed_;
     snapshot_notifier_ = save_point.snapshot_notifier_;
     num_puts_ = save_point.num_puts_;
-    num_put_entities_ = save_point.num_put_entities_;
     num_deletes_ = save_point.num_deletes_;
     num_merges_ = save_point.num_merges_;
 
@@ -289,13 +288,6 @@ Status TransactionBaseImpl::GetImpl(const ReadOptions& read_options,
                                         pinnable_val);
 }
 
-Status TransactionBaseImpl::GetEntity(const ReadOptions& read_options,
-                                      ColumnFamilyHandle* column_family,
-                                      const Slice& key,
-                                      PinnableWideColumns* columns) {
-  return GetEntityImpl(read_options, column_family, key, columns);
-}
-
 Status TransactionBaseImpl::GetForUpdate(const ReadOptions& read_options,
                                          ColumnFamilyHandle* column_family,
                                          const Slice& key, std::string* value,
@@ -408,15 +400,6 @@ void TransactionBaseImpl::MultiGet(const ReadOptions& _read_options,
                                       sorted_input);
 }
 
-void TransactionBaseImpl::MultiGetEntity(const ReadOptions& read_options,
-                                         ColumnFamilyHandle* column_family,
-                                         size_t num_keys, const Slice* keys,
-                                         PinnableWideColumns* results,
-                                         Status* statuses, bool sorted_input) {
-  MultiGetEntityImpl(read_options, column_family, num_keys, keys, results,
-                     statuses, sorted_input);
-}
-
 std::vector<Status> TransactionBaseImpl::MultiGetForUpdate(
     const ReadOptions& read_options,
     const std::vector<ColumnFamilyHandle*>& column_family,
@@ -468,32 +451,6 @@ Iterator* TransactionBaseImpl::GetIterator(const ReadOptions& read_options,
                                           &read_options);
 }
 
-Status TransactionBaseImpl::PutEntityImpl(ColumnFamilyHandle* column_family,
-                                          const Slice& key,
-                                          const WideColumns& columns,
-                                          bool do_validate,
-                                          bool assume_tracked) {
-  {
-    constexpr bool read_only = false;
-    constexpr bool exclusive = true;
-    const Status s = TryLock(column_family, key, read_only, exclusive,
-                             do_validate, assume_tracked);
-    if (!s.ok()) {
-      return s;
-    }
-  }
-
-  {
-    const Status s = GetBatchForWrite()->PutEntity(column_family, key, columns);
-    if (!s.ok()) {
-      return s;
-    }
-  }
-
-  ++num_put_entities_;
-  return Status::OK();
-}
-
 Status TransactionBaseImpl::Put(ColumnFamilyHandle* column_family,
                                 const Slice& key, const Slice& value,
                                 const bool assume_tracked) {
@@ -721,10 +678,6 @@ uint64_t TransactionBaseImpl::GetElapsedTime() const {
 
 uint64_t TransactionBaseImpl::GetNumPuts() const { return num_puts_; }
 
-uint64_t TransactionBaseImpl::GetNumPutEntities() const {
-  return num_put_entities_;
-}
-
 uint64_t TransactionBaseImpl::GetNumDeletes() const { return num_deletes_; }
 
 uint64_t TransactionBaseImpl::GetNumMerges() const { return num_merges_; }
@@ -752,8 +705,7 @@ void TransactionBaseImpl::TrackKey(uint32_t cfh_id, const std::string& key,
   }
 }
 
-// Gets the write batch that should be used for Put/PutEntity/Merge/Delete
-// operations.
+// Gets the write batch that should be used for Put/Merge/Deletes.
 //
 // Returns either a WriteBatch or WriteBatchWithIndex depending on whether
 // DisableIndexing() has been called.
@@ -817,19 +769,6 @@ Status TransactionBaseImpl::RebuildFromWriteBatch(WriteBatch* src_batch) {
       return txn_->Put(db_->GetColumnFamilyHandle(cf), key, val);
     }
 
-    Status PutEntityCF(uint32_t cf, const Slice& key,
-                       const Slice& entity) override {
-      Slice entity_copy = entity;
-      WideColumns columns;
-      const Status s =
-          WideColumnSerialization::Deserialize(entity_copy, columns);
-      if (!s.ok()) {
-        return s;
-      }
-
-      return txn_->PutEntity(db_->GetColumnFamilyHandle(cf), key, columns);
-    }
-
     Status DeleteCF(uint32_t cf, const Slice& key) override {
       return txn_->Delete(db_->GetColumnFamilyHandle(cf), key);
     }

utilities/transactions/transaction_base.h

@@ -5,6 +5,7 @@
 
 #pragma once
 
+
 #include <stack>
 #include <string>
 #include <vector>
@@ -36,11 +37,10 @@ class TransactionBaseImpl : public Transaction {
 
   void Reinitialize(DB* db, const WriteOptions& write_options);
 
-  // Called before executing Put, PutEntity, Merge, Delete, and GetForUpdate. If
-  // TryLock returns non-OK, the Put/PutEntity/Merge/Delete/GetForUpdate will be
-  // failed. do_validate will be false if called from PutUntracked,
-  // PutEntityUntracked, DeleteUntracked, MergeUntracked, or
-  // GetForUpdate(do_validate=false)
+  // Called before executing Put, Merge, Delete, and GetForUpdate.  If TryLock
+  // returns non-OK, the Put/Merge/Delete/GetForUpdate will be failed.
+  // do_validate will be false if called from PutUntracked, DeleteUntracked,
+  // MergeUntracked, or GetForUpdate(do_validate=false)
   virtual Status TryLock(ColumnFamilyHandle* column_family, const Slice& key,
                          bool read_only, bool exclusive,
                          const bool do_validate = true,
@@ -66,10 +66,6 @@ class TransactionBaseImpl : public Transaction {
     return Get(options, db_->DefaultColumnFamily(), key, value);
   }
 
-  Status GetEntity(const ReadOptions& options,
-                   ColumnFamilyHandle* column_family, const Slice& key,
-                   PinnableWideColumns* columns) override;
-
   using Transaction::GetForUpdate;
   Status GetForUpdate(const ReadOptions& options,
                       ColumnFamilyHandle* column_family, const Slice& key,
@@ -116,11 +112,6 @@ class TransactionBaseImpl : public Transaction {
                 const Slice* keys, PinnableSlice* values, Status* statuses,
                 const bool sorted_input = false) override;
 
-  void MultiGetEntity(const ReadOptions& options,
-                      ColumnFamilyHandle* column_family, size_t num_keys,
-                      const Slice* keys, PinnableWideColumns* results,
-                      Status* statuses, bool sorted_input = false) override;
-
   using Transaction::MultiGetForUpdate;
   std::vector<Status> MultiGetForUpdate(
       const ReadOptions& options,
@@ -154,15 +145,6 @@ class TransactionBaseImpl : public Transaction {
     return Put(nullptr, key, value);
   }
 
-  Status PutEntity(ColumnFamilyHandle* column_family, const Slice& key,
-                   const WideColumns& columns,
-                   bool assume_tracked = false) override {
-    const bool do_validate = !assume_tracked;
-
-    return PutEntityImpl(column_family, key, columns, do_validate,
-                         assume_tracked);
-  }
-
   Status Merge(ColumnFamilyHandle* column_family, const Slice& key,
                const Slice& value, const bool assume_tracked = false) override;
   Status Merge(const Slice& key, const Slice& value) override {
@@ -199,15 +181,6 @@ class TransactionBaseImpl : public Transaction {
     return PutUntracked(nullptr, key, value);
   }
 
-  Status PutEntityUntracked(ColumnFamilyHandle* column_family, const Slice& key,
-                            const WideColumns& columns) override {
-    constexpr bool do_validate = false;
-    constexpr bool assume_tracked = false;
-
-    return PutEntityImpl(column_family, key, columns, do_validate,
-                         assume_tracked);
-  }
-
   Status MergeUntracked(ColumnFamilyHandle* column_family, const Slice& key,
                         const Slice& value) override;
   Status MergeUntracked(const Slice& key, const Slice& value) override {
@@ -267,8 +240,6 @@ class TransactionBaseImpl : public Transaction {
 
   uint64_t GetNumPuts() const override;
 
-  uint64_t GetNumPutEntities() const override;
-
   uint64_t GetNumDeletes() const override;
 
   uint64_t GetNumMerges() const override;
@@ -305,26 +276,6 @@ class TransactionBaseImpl : public Transaction {
   Status GetImpl(const ReadOptions& options, ColumnFamilyHandle* column_family,
                  const Slice& key, PinnableSlice* value) override;
 
-  Status GetEntityImpl(const ReadOptions& options,
-                       ColumnFamilyHandle* column_family, const Slice& key,
-                       PinnableWideColumns* columns) {
-    return write_batch_.GetEntityFromBatchAndDB(db_, options, column_family,
-                                                key, columns);
-  }
-
-  void MultiGetEntityImpl(const ReadOptions& options,
-                          ColumnFamilyHandle* column_family, size_t num_keys,
-                          const Slice* keys, PinnableWideColumns* results,
-                          Status* statuses, bool sorted_input) {
-    write_batch_.MultiGetEntityFromBatchAndDB(db_, options, column_family,
-                                              num_keys, keys, results, statuses,
-                                              sorted_input);
-  }
-
-  Status PutEntityImpl(ColumnFamilyHandle* column_family, const Slice& key,
-                       const WideColumns& columns, bool do_validate,
-                       bool assume_tracked);
-
   // Add a key to the list of tracked keys.
   //
   // seqno is the earliest seqno this key was involved with this transaction.
@@ -369,7 +320,6 @@ class TransactionBaseImpl : public Transaction {
 
   // Count of various operations pending in this transaction
   uint64_t num_puts_ = 0;
-  uint64_t num_put_entities_ = 0;
   uint64_t num_deletes_ = 0;
   uint64_t num_merges_ = 0;
 
@@ -378,7 +328,6 @@ class TransactionBaseImpl : public Transaction {
     bool snapshot_needed_ = false;
     std::shared_ptr<TransactionNotifier> snapshot_notifier_;
     uint64_t num_puts_ = 0;
-    uint64_t num_put_entities_ = 0;
     uint64_t num_deletes_ = 0;
     uint64_t num_merges_ = 0;
 
@@ -387,14 +336,12 @@ class TransactionBaseImpl : public Transaction {
 
     SavePoint(std::shared_ptr<const Snapshot> snapshot, bool snapshot_needed,
               std::shared_ptr<TransactionNotifier> snapshot_notifier,
-              uint64_t num_puts, uint64_t num_put_entities,
-              uint64_t num_deletes, uint64_t num_merges,
+              uint64_t num_puts, uint64_t num_deletes, uint64_t num_merges,
               const LockTrackerFactory& lock_tracker_factory)
         : snapshot_(snapshot),
           snapshot_needed_(snapshot_needed),
           snapshot_notifier_(snapshot_notifier),
           num_puts_(num_puts),
-          num_put_entities_(num_put_entities),
           num_deletes_(num_deletes),
           num_merges_(num_merges),
           new_locks_(lock_tracker_factory.Create()) {}
@@ -426,10 +373,10 @@ class TransactionBaseImpl : public Transaction {
   // prepare phase is not skipped.
   WriteBatch commit_time_batch_;
 
-  // If true, future Put/PutEntity/Merge/Delete operations will be indexed in
-  // the WriteBatchWithIndex. If false, future Put/PutEntity/Merge/Delete
-  // operations will be inserted directly into the underlying WriteBatch and not
-  // indexed in the WriteBatchWithIndex.
+  // If true, future Put/Merge/Deletes will be indexed in the
+  // WriteBatchWithIndex.
+  // If false, future Put/Merge/Deletes will be inserted directly into the
+  // underlying WriteBatch and not indexed in the WriteBatchWithIndex.
   bool indexing_enabled_;
 
   // SetSnapshotOnNextOperation() has been called and the snapshot has not yet

utilities/transactions/transaction_test.cc

@@ -3,10 +3,10 @@
 //  COPYING file in the root directory) and Apache 2.0 License
 //  (found in the LICENSE.Apache file in the root directory).
 
+
 #include "utilities/transactions/transaction_test.h"
 
 #include <algorithm>
-#include <array>
 #include <functional>
 #include <string>
 #include <thread>
@@ -2210,10 +2210,6 @@ TEST_P(TransactionTest, TwoPhaseLogRollingTest2) {
  * hidden behind improperly summed sequence ids
  */
 TEST_P(TransactionTest, TwoPhaseOutOfOrderDelete) {
-  // WAL recycling incompatible with disableWAL (below)
-  options.recycle_log_file_num = 0;
-  ASSERT_OK(ReOpenNoDelete());
-
   DBImpl* db_impl = static_cast_with_check<DBImpl>(db->GetRootDB());
   WriteOptions wal_on, wal_off;
   wal_on.sync = true;
@@ -3625,9 +3621,6 @@ TEST_P(TransactionTest, UntrackedWrites) {
   // Untracked writes should succeed even though key was written after snapshot
   s = txn->PutUntracked("untracked", "1");
   ASSERT_OK(s);
-  s = txn->PutEntityUntracked(db->DefaultColumnFamily(), "untracked",
-                              {{"hello", "world"}});
-  ASSERT_OK(s);
   s = txn->MergeUntracked("untracked", "2");
   ASSERT_OK(s);
   s = txn->DeleteUntracked("untracked");
@@ -3636,8 +3629,6 @@ TEST_P(TransactionTest, UntrackedWrites) {
   // Conflict
   s = txn->Put("untracked", "3");
   ASSERT_TRUE(s.IsBusy());
-  s = txn->PutEntity(db->DefaultColumnFamily(), "untracked", {{"foo", "bar"}});
-  ASSERT_TRUE(s.IsBusy());
 
   s = txn->Commit();
   ASSERT_OK(s);
@@ -6993,329 +6984,6 @@ TEST_P(TransactionTest, UnlockWALStallCleared) {
   }
 }
 
-TEST_P(TransactionTest, PutEntitySuccess) {
-  const TxnDBWritePolicy write_policy = std::get<2>(GetParam());
-  if (write_policy != TxnDBWritePolicy::WRITE_COMMITTED) {
-    ROCKSDB_GTEST_BYPASS("Test only WriteCommitted for now");
-    return;
-  }
-
-  constexpr char foo[] = "foo";
-  const WideColumns foo_columns{
-      {kDefaultWideColumnName, "bar"}, {"col1", "val1"}, {"col2", "val2"}};
-  const WideColumns foo_new_columns{
-      {kDefaultWideColumnName, "baz"}, {"colA", "valA"}, {"colB", "valB"}};
-
-  ASSERT_OK(db->PutEntity(WriteOptions(), db->DefaultColumnFamily(), foo,
-                          foo_columns));
-
-  {
-    std::unique_ptr<Transaction> txn(
-        db->BeginTransaction(WriteOptions(), TransactionOptions()));
-
-    ASSERT_NE(txn, nullptr);
-    ASSERT_NE(txn->GetID(), 0);
-    ASSERT_EQ(txn->GetNumPutEntities(), 0);
-
-    {
-      PinnableWideColumns columns;
-      ASSERT_OK(txn->GetEntity(ReadOptions(), db->DefaultColumnFamily(), foo,
-                               &columns));
-      ASSERT_EQ(columns.columns(), foo_columns);
-    }
-
-    {
-      PinnableSlice value;
-      ASSERT_OK(txn->GetForUpdate(ReadOptions(), foo, &value));
-      ASSERT_EQ(value, foo_columns[0].value());
-    }
-
-    ASSERT_OK(txn->PutEntity(db->DefaultColumnFamily(), foo, foo_new_columns));
-
-    ASSERT_EQ(txn->GetNumPutEntities(), 1);
-
-    {
-      PinnableWideColumns columns;
-      ASSERT_OK(txn->GetEntity(ReadOptions(), db->DefaultColumnFamily(), foo,
-                               &columns));
-      ASSERT_EQ(columns.columns(), foo_new_columns);
-    }
-
-    {
-      PinnableSlice value;
-      ASSERT_OK(txn->GetForUpdate(ReadOptions(), foo, &value));
-      ASSERT_EQ(value, foo_new_columns[0].value());
-    }
-
-    ASSERT_OK(txn->Commit());
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(
-        db->GetEntity(ReadOptions(), db->DefaultColumnFamily(), foo, &columns));
-    ASSERT_EQ(columns.columns(), foo_new_columns);
-  }
-}
-
-TEST_P(TransactionTest, PutEntityWriteConflict) {
-  const TxnDBWritePolicy write_policy = std::get<2>(GetParam());
-  if (write_policy != TxnDBWritePolicy::WRITE_COMMITTED) {
-    ROCKSDB_GTEST_BYPASS("Test only WriteCommitted for now");
-    return;
-  }
-
-  constexpr char foo[] = "foo";
-  const WideColumns foo_columns{
-      {kDefaultWideColumnName, "bar"}, {"col1", "val1"}, {"col2", "val2"}};
-  constexpr char baz[] = "baz";
-  const WideColumns baz_columns{
-      {kDefaultWideColumnName, "quux"}, {"colA", "valA"}, {"colB", "valB"}};
-
-  ASSERT_OK(db->PutEntity(WriteOptions(), db->DefaultColumnFamily(), foo,
-                          foo_columns));
-  ASSERT_OK(db->PutEntity(WriteOptions(), db->DefaultColumnFamily(), baz,
-                          baz_columns));
-
-  std::unique_ptr<Transaction> txn(db->BeginTransaction(WriteOptions()));
-  ASSERT_NE(txn, nullptr);
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), db->DefaultColumnFamily(), foo,
-                             &columns));
-    ASSERT_EQ(columns.columns(), foo_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), db->DefaultColumnFamily(), baz,
-                             &columns));
-    ASSERT_EQ(columns.columns(), baz_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    txn->MultiGetEntity(ReadOptions(), db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data());
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_columns);
-    ASSERT_EQ(results[1].columns(), baz_columns);
-  }
-
-  const WideColumns foo_new_columns{{kDefaultWideColumnName, "FOO"},
-                                    {"hello", "world"}};
-  const WideColumns baz_new_columns{{kDefaultWideColumnName, "BAZ"},
-                                    {"ping", "pong"}};
-
-  ASSERT_OK(txn->PutEntity(db->DefaultColumnFamily(), foo, foo_new_columns));
-  ASSERT_OK(txn->PutEntity(db->DefaultColumnFamily(), baz, baz_new_columns));
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), db->DefaultColumnFamily(), foo,
-                             &columns));
-    ASSERT_EQ(columns.columns(), foo_new_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(txn->GetEntity(ReadOptions(), db->DefaultColumnFamily(), baz,
-                             &columns));
-    ASSERT_EQ(columns.columns(), baz_new_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-
-    txn->MultiGetEntity(ReadOptions(), db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data());
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_new_columns);
-    ASSERT_EQ(results[1].columns(), baz_new_columns);
-  }
-
-  // This PutEntity outside of a transaction will conflict with the previous
-  // write
-  const WideColumns foo_conflict_columns{{kDefaultWideColumnName, "X"},
-                                         {"conflicting", "write"}};
-  ASSERT_TRUE(db->PutEntity(WriteOptions(), db->DefaultColumnFamily(), foo,
-                            foo_conflict_columns)
-                  .IsTimedOut());
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(
-        db->GetEntity(ReadOptions(), db->DefaultColumnFamily(), foo, &columns));
-    ASSERT_EQ(columns.columns(), foo_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(
-        db->GetEntity(ReadOptions(), db->DefaultColumnFamily(), baz, &columns));
-    ASSERT_EQ(columns.columns(), baz_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    db->MultiGetEntity(ReadOptions(), db->DefaultColumnFamily(), num_keys,
-                       keys.data(), results.data(), statuses.data(),
-                       sorted_input);
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_columns);
-    ASSERT_EQ(results[1].columns(), baz_columns);
-  }
-
-  ASSERT_OK(txn->Commit());
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(
-        db->GetEntity(ReadOptions(), db->DefaultColumnFamily(), foo, &columns));
-    ASSERT_EQ(columns.columns(), foo_new_columns);
-  }
-
-  {
-    PinnableWideColumns columns;
-    ASSERT_OK(
-        db->GetEntity(ReadOptions(), db->DefaultColumnFamily(), baz, &columns));
-    ASSERT_EQ(columns.columns(), baz_new_columns);
-  }
-
-  {
-    constexpr size_t num_keys = 2;
-
-    std::array<Slice, num_keys> keys{{foo, baz}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    db->MultiGetEntity(ReadOptions(), db->DefaultColumnFamily(), num_keys,
-                       keys.data(), results.data(), statuses.data(),
-                       sorted_input);
-
-    ASSERT_OK(statuses[0]);
-    ASSERT_OK(statuses[1]);
-
-    ASSERT_EQ(results[0].columns(), foo_new_columns);
-    ASSERT_EQ(results[1].columns(), baz_new_columns);
-  }
-}
-
-TEST_P(TransactionTest, EntityReadSanityChecks) {
-  constexpr char foo[] = "foo";
-  constexpr char bar[] = "bar";
-  constexpr size_t num_keys = 2;
-
-  std::unique_ptr<Transaction> txn(db->BeginTransaction(WriteOptions()));
-  ASSERT_NE(txn, nullptr);
-
-  {
-    constexpr ColumnFamilyHandle* column_family = nullptr;
-    PinnableWideColumns columns;
-    ASSERT_TRUE(txn->GetEntity(ReadOptions(), column_family, foo, &columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    constexpr PinnableWideColumns* columns = nullptr;
-    ASSERT_TRUE(
-        txn->GetEntity(ReadOptions(), db->DefaultColumnFamily(), foo, columns)
-            .IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kGet;
-
-    PinnableWideColumns columns;
-    ASSERT_TRUE(
-        txn->GetEntity(read_options, db->DefaultColumnFamily(), foo, &columns)
-            .IsInvalidArgument());
-  }
-
-  {
-    constexpr ColumnFamilyHandle* column_family = nullptr;
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn->MultiGetEntity(ReadOptions(), column_family, num_keys, keys.data(),
-                        results.data(), statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    constexpr Slice* keys = nullptr;
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn->MultiGetEntity(ReadOptions(), db->DefaultColumnFamily(), num_keys,
-                        keys, results.data(), statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    constexpr PinnableWideColumns* results = nullptr;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn->MultiGetEntity(ReadOptions(), db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results, statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kMultiGet;
-
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    txn->MultiGetEntity(read_options, db->DefaultColumnFamily(), num_keys,
-                        keys.data(), results.data(), statuses.data(),
-                        sorted_input);
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-}
-
 TEST_F(TransactionDBTest, CollapseKey) {
   ASSERT_OK(ReOpen());
   ASSERT_OK(db->Put({}, "hello", "world"));
@@ -7365,51 +7033,6 @@ TEST_F(TransactionDBTest, CollapseKey) {
   }
 }
 
-TEST_F(TransactionDBTest, FlushedLogWithPendingPrepareIsSynced) {
-  // Repro for a bug where we missed a necessary sync of the old WAL during
-  // memtable flush. It happened due to applying an optimization to skip syncing
-  // the old WAL in too many scenarios (all memtable flushes on single CF
-  // databases). That optimization is only valid when memtable flush can
-  // guarantee the old WAL will not be read by crash-recovery. When the old WAL
-  // contains a prepare record without its corresponding commit, that WAL will
-  // be read by crash-recovery and therefore it must be synced.
-  const int kStartIndex = 1000;
-  int next_index = kStartIndex;
-  ASSERT_OK(ReOpen());
-
-  ASSERT_OK(
-      db->Put(WriteOptions(), "key" + std::to_string(next_index), "value"));
-  next_index++;
-
-  Transaction* txn = db->BeginTransaction(WriteOptions(), TransactionOptions());
-  ASSERT_OK(txn->SetName("xid" + std::to_string(next_index)));
-  ASSERT_OK(
-      txn->Put(Slice("key" + std::to_string(next_index)), Slice("value")));
-  next_index++;
-  ASSERT_OK(txn->Prepare());
-
-  // Set it directly writable so new WAL containing the commit record will be
-  // recovered despite not being explicitly synced.
-  fault_fs->SetFilesystemDirectWritable(true);
-  ASSERT_OK(db->Flush(FlushOptions()));
-
-  ASSERT_OK(txn->Commit());
-  delete txn;
-
-  ASSERT_OK(
-      db->Put(WriteOptions(), "key" + std::to_string(next_index), "value"));
-  next_index++;
-
-  ASSERT_OK(ReOpenNoDelete());
-
-  for (int i = kStartIndex; i < next_index; i++) {
-    PinnableSlice value;
-    ASSERT_OK(db->Get(ReadOptions(), db->DefaultColumnFamily(),
-                      "key" + std::to_string(i), &value));
-    ASSERT_EQ("value", value);
-  }
-}
-
 }  // namespace ROCKSDB_NAMESPACE
 
 int main(int argc, char** argv) {

utilities/transactions/transaction_test.h

@@ -63,8 +63,6 @@ class TransactionTestBase : public ::testing::Test {
     options.unordered_write = write_ordering == kUnorderedWrite;
     options.level0_file_num_compaction_trigger = 2;
     options.merge_operator = MergeOperators::CreateFromStringId("stringappend");
-    // Recycling log file is generally more challenging for correctness
-    options.recycle_log_file_num = 2;
     special_env.skip_fsync_ = true;
     fault_fs.reset(new FaultInjectionTestFS(FileSystem::Default()));
     env.reset(new CompositeEnvWrapper(&special_env, fault_fs));

utilities/write_batch_with_index/write_batch_with_index.cc

@@ -823,41 +823,11 @@ void WriteBatchWithIndex::MultiGetFromBatchAndDB(
 }
 
 Status WriteBatchWithIndex::GetEntityFromBatchAndDB(
-    DB* db, const ReadOptions& _read_options, ColumnFamilyHandle* column_family,
+    DB* db, const ReadOptions& read_options, ColumnFamilyHandle* column_family,
     const Slice& key, PinnableWideColumns* columns, ReadCallback* callback) {
-  if (!db) {
-    return Status::InvalidArgument(
-        "Cannot call GetEntityFromBatchAndDB without a DB object");
-  }
-
-  if (_read_options.io_activity != Env::IOActivity::kUnknown &&
-      _read_options.io_activity != Env::IOActivity::kGetEntity) {
-    return Status::InvalidArgument(
-        "Can only call GetEntityFromBatchAndDB with `ReadOptions::io_activity` "
-        "set to `Env::IOActivity::kUnknown` or `Env::IOActivity::kGetEntity`");
-  }
-
-  ReadOptions read_options(_read_options);
-  if (read_options.io_activity == Env::IOActivity::kUnknown) {
-    read_options.io_activity = Env::IOActivity::kGetEntity;
-  }
-
-  if (!column_family) {
-    return Status::InvalidArgument(
-        "Cannot call GetEntityFromBatchAndDB without a column family handle");
-  }
-
-  const Comparator* const ucmp = rep->comparator.GetComparator(column_family);
-  size_t ts_sz = ucmp ? ucmp->timestamp_size() : 0;
-  if (ts_sz > 0 && !read_options.timestamp) {
-    return Status::InvalidArgument("Must specify timestamp");
-  }
-
-  if (!columns) {
-    return Status::InvalidArgument(
-        "Cannot call GetEntityFromBatchAndDB without a PinnableWideColumns "
-        "object");
-  }
+  assert(db);
+  assert(column_family);
+  assert(columns);
 
   columns->Reset();
 
@@ -902,78 +872,46 @@ Status WriteBatchWithIndex::GetEntityFromBatchAndDB(
   return s;
 }
 
-void WriteBatchWithIndex::MultiGetEntityFromBatchAndDB(
-    DB* db, const ReadOptions& _read_options, ColumnFamilyHandle* column_family,
-    size_t num_keys, const Slice* keys, PinnableWideColumns* results,
-    Status* statuses, bool sorted_input, ReadCallback* callback) {
-  assert(statuses);
-
+Status WriteBatchWithIndex::GetEntityFromBatchAndDB(
+    DB* db, const ReadOptions& read_options, ColumnFamilyHandle* column_family,
+    const Slice& key, PinnableWideColumns* columns) {
   if (!db) {
-    const Status s = Status::InvalidArgument(
-        "Cannot call MultiGetEntityFromBatchAndDB without a DB object");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-    return;
-  }
-
-  if (_read_options.io_activity != Env::IOActivity::kUnknown &&
-      _read_options.io_activity != Env::IOActivity::kMultiGetEntity) {
-    const Status s = Status::InvalidArgument(
-        "Can only call MultiGetEntityFromBatchAndDB with "
-        "`ReadOptions::io_activity` set to `Env::IOActivity::kUnknown` or "
-        "`Env::IOActivity::kMultiGetEntity`");
-    for (size_t i = 0; i < num_keys; ++i) {
-      if (statuses[i].ok()) {
-        statuses[i] = s;
-      }
-    }
-    return;
-  }
-
-  ReadOptions read_options(_read_options);
-  if (read_options.io_activity == Env::IOActivity::kUnknown) {
-    read_options.io_activity = Env::IOActivity::kMultiGetEntity;
+    return Status::InvalidArgument(
+        "Cannot call GetEntityFromBatchAndDB without a DB object");
   }
 
   if (!column_family) {
-    const Status s = Status::InvalidArgument(
-        "Cannot call MultiGetEntityFromBatchAndDB without a column family "
-        "handle");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-    return;
+    return Status::InvalidArgument(
+        "Cannot call GetEntityFromBatchAndDB without a column family handle");
   }
 
   const Comparator* const ucmp = rep->comparator.GetComparator(column_family);
-  const size_t ts_sz = ucmp ? ucmp->timestamp_size() : 0;
+  size_t ts_sz = ucmp ? ucmp->timestamp_size() : 0;
   if (ts_sz > 0 && !read_options.timestamp) {
-    const Status s = Status::InvalidArgument("Must specify timestamp");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-    return;
+    return Status::InvalidArgument("Must specify timestamp");
   }
 
-  if (!keys) {
-    const Status s = Status::InvalidArgument(
-        "Cannot call MultiGetEntityFromBatchAndDB without keys");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-    return;
+  if (!columns) {
+    return Status::InvalidArgument(
+        "Cannot call GetEntityFromBatchAndDB without a PinnableWideColumns "
+        "object");
   }
 
-  if (!results) {
-    const Status s = Status::InvalidArgument(
-        "Cannot call MultiGetEntityFromBatchAndDB without "
-        "PinnableWideColumns objects");
-    for (size_t i = 0; i < num_keys; ++i) {
-      statuses[i] = s;
-    }
-    return;
-  }
+  constexpr ReadCallback* callback = nullptr;
+
+  return GetEntityFromBatchAndDB(db, read_options, column_family, key, columns,
+                                 callback);
+}
+
+void WriteBatchWithIndex::MultiGetEntityFromBatchAndDB(
+    DB* db, const ReadOptions& read_options, ColumnFamilyHandle* column_family,
+    size_t num_keys, const Slice* keys, PinnableWideColumns* results,
+    Status* statuses, bool sorted_input, ReadCallback* callback) {
+  assert(db);
+  assert(column_family);
+  assert(keys);
+  assert(results);
+  assert(statuses);
 
   struct MergeTuple {
     MergeTuple(const Slice& _key, Status* _s, MergeContext&& _merge_context,
@@ -1052,8 +990,8 @@ void WriteBatchWithIndex::MultiGetEntityFromBatchAndDB(
   static_cast_with_check<DBImpl>(db->GetRootDB())
       ->PrepareMultiGetKeys(sorted_keys.size(), sorted_input, &sorted_keys);
   static_cast_with_check<DBImpl>(db->GetRootDB())
-      ->MultiGetEntityWithCallback(read_options, column_family, callback,
-                                   &sorted_keys);
+      ->MultiGetWithCallback(read_options, column_family, callback,
+                             &sorted_keys);
 
   for (const auto& merge : merges) {
     if (merge.s->ok() || merge.s->IsNotFound()) {  // DB lookup succeeded
@@ -1063,6 +1001,57 @@ void WriteBatchWithIndex::MultiGetEntityFromBatchAndDB(
   }
 }
 
+void WriteBatchWithIndex::MultiGetEntityFromBatchAndDB(
+    DB* db, const ReadOptions& read_options, ColumnFamilyHandle* column_family,
+    size_t num_keys, const Slice* keys, PinnableWideColumns* results,
+    Status* statuses, bool sorted_input) {
+  assert(statuses);
+
+  if (!db) {
+    for (size_t i = 0; i < num_keys; ++i) {
+      statuses[i] = Status::InvalidArgument(
+          "Cannot call MultiGetEntityFromBatchAndDB without a DB object");
+    }
+  }
+
+  if (!column_family) {
+    for (size_t i = 0; i < num_keys; ++i) {
+      statuses[i] = Status::InvalidArgument(
+          "Cannot call MultiGetEntityFromBatchAndDB without a column family "
+          "handle");
+    }
+  }
+
+  const Comparator* const ucmp = rep->comparator.GetComparator(column_family);
+  const size_t ts_sz = ucmp ? ucmp->timestamp_size() : 0;
+  if (ts_sz > 0 && !read_options.timestamp) {
+    for (size_t i = 0; i < num_keys; ++i) {
+      statuses[i] = Status::InvalidArgument("Must specify timestamp");
+    }
+    return;
+  }
+
+  if (!keys) {
+    for (size_t i = 0; i < num_keys; ++i) {
+      statuses[i] = Status::InvalidArgument(
+          "Cannot call MultiGetEntityFromBatchAndDB without keys");
+    }
+  }
+
+  if (!results) {
+    for (size_t i = 0; i < num_keys; ++i) {
+      statuses[i] = Status::InvalidArgument(
+          "Cannot call MultiGetEntityFromBatchAndDB without "
+          "PinnableWideColumns objects");
+    }
+  }
+
+  constexpr ReadCallback* callback = nullptr;
+
+  MultiGetEntityFromBatchAndDB(db, read_options, column_family, num_keys, keys,
+                               results, statuses, sorted_input, callback);
+}
+
 void WriteBatchWithIndex::SetSavePoint() { rep->write_batch.SetSavePoint(); }
 
 Status WriteBatchWithIndex::RollbackToSavePoint() {

utilities/write_batch_with_index/write_batch_with_index_test.cc

@@ -2973,128 +2973,6 @@ TEST_P(WriteBatchWithIndexTest, GetEntityFromBatch) {
   }
 }
 
-TEST_P(WriteBatchWithIndexTest, EntityReadSanityChecks) {
-  ASSERT_OK(OpenDB());
-
-  constexpr char foo[] = "foo";
-  constexpr char bar[] = "bar";
-  constexpr size_t num_keys = 2;
-
-  {
-    constexpr DB* db = nullptr;
-    PinnableWideColumns columns;
-    ASSERT_TRUE(batch_
-                    ->GetEntityFromBatchAndDB(db, ReadOptions(),
-                                              db_->DefaultColumnFamily(), foo,
-                                              &columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    constexpr ColumnFamilyHandle* column_family = nullptr;
-    PinnableWideColumns columns;
-    ASSERT_TRUE(batch_
-                    ->GetEntityFromBatchAndDB(db_, ReadOptions(), column_family,
-                                              foo, &columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    constexpr PinnableWideColumns* columns = nullptr;
-    ASSERT_TRUE(batch_
-                    ->GetEntityFromBatchAndDB(db_, ReadOptions(),
-                                              db_->DefaultColumnFamily(), foo,
-                                              columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kGet;
-
-    PinnableWideColumns columns;
-    ASSERT_TRUE(batch_
-                    ->GetEntityFromBatchAndDB(db_, read_options,
-                                              db_->DefaultColumnFamily(), foo,
-                                              &columns)
-                    .IsInvalidArgument());
-  }
-
-  {
-    constexpr DB* db = nullptr;
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    batch_->MultiGetEntityFromBatchAndDB(
-        db, ReadOptions(), db_->DefaultColumnFamily(), num_keys, keys.data(),
-        results.data(), statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    constexpr ColumnFamilyHandle* column_family = nullptr;
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    batch_->MultiGetEntityFromBatchAndDB(db_, ReadOptions(), column_family,
-                                         num_keys, keys.data(), results.data(),
-                                         statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    constexpr Slice* keys = nullptr;
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    batch_->MultiGetEntityFromBatchAndDB(
-        db_, ReadOptions(), db_->DefaultColumnFamily(), num_keys, keys,
-        results.data(), statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    constexpr PinnableWideColumns* results = nullptr;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    batch_->MultiGetEntityFromBatchAndDB(
-        db_, ReadOptions(), db_->DefaultColumnFamily(), num_keys, keys.data(),
-        results, statuses.data(), sorted_input);
-
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-
-  {
-    ReadOptions read_options;
-    read_options.io_activity = Env::IOActivity::kMultiGet;
-
-    std::array<Slice, num_keys> keys{{foo, bar}};
-    std::array<PinnableWideColumns, num_keys> results;
-    std::array<Status, num_keys> statuses;
-    constexpr bool sorted_input = false;
-
-    batch_->MultiGetEntityFromBatchAndDB(
-        db_, read_options, db_->DefaultColumnFamily(), num_keys, keys.data(),
-        results.data(), statuses.data(), sorted_input);
-    ASSERT_TRUE(statuses[0].IsInvalidArgument());
-    ASSERT_TRUE(statuses[1].IsInvalidArgument());
-  }
-}
-
 INSTANTIATE_TEST_CASE_P(WBWI, WriteBatchWithIndexTest, testing::Bool());
 }  // namespace ROCKSDB_NAMESPACE