Revert "Fix Windows build issue in trie_index_factory.h"

This reverts commit b7b8326c89.

CLAUDE.md

@@ -302,6 +302,7 @@ build (and where noted, run tests) across:
 | Axis | Must support |
 |------|--------------|
 | OS | Linux (x86_64 + ARM), macOS, Windows |
+| Architecture | Little-endian and big-endian targets, including Linux on s390x |
 | Compiler | GCC, Clang (libstdc++ **and** libc++), AppleClang, **MSVC (VS2022)**, MinGW (Linux cross-compile, build-only, no gflags) |
 | Build system | Make, CMake, and BUCK (internal) -- keep all in sync (see "Build system" above) |
 | Config | release (`-fno-rtti`), `ASSERT_STATUS_CHECKED`, ASAN/UBSAN/TSAN, folly, unity build, JNI/Java |
@@ -316,6 +317,12 @@ abstractions; otherwise guard with `#ifdef OS_WIN` (POSIX `<unistd.h>` ->
 Windows `<process.h>`). Because libc++ is also tested, include what you use
 rather than relying on libstdc++ transitive includes.
 
+Do not use `static_assert` to reject a supported architecture for an optional
+feature. Check an existing `port::` capability such as `port::kLittleEndian`
+and disable only the unsupported feature so the rest of RocksDB still builds.
+Use a portable representation when the feature itself must work across
+architectures.
+
 ### Unit Test
 * After all of the unit tests are added, review them and try to extract common
     reusable utility functions to reduce code duplication due to copy past between

HISTORY.md

@@ -1,6 +1,36 @@
 # Rocksdb Change Log
 > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
 
+## 11.9.1 (08/18/2026)
+### Public API Changes
+* Added an immutable `external_table_config` option that is persisted in OPTIONS files and passed to external table factories during bootstrap.
+
+## 11.9.0 (08/14/2026)
+### New Features
+* Added an integrated and improved compression auto-skip feature (new CompressionOptions, still experimental) that saves CPU by skipping compression when and where the overall ratio is inadequate. This improves on the previous auto-skip compression manager by (a) taking achieved compression ratios into account, (b) using a continuously evolving payoff estimate, and (c) combining natively with parallel compression.
+* Added the `blob_file_writable_file_max_buffer_size` mutable column family option. When set to a non-zero value, blob-file writers use it as their `WritableFileWriter` max buffer size instead of inheriting `DBOptions::writable_file_max_buffer_size`.
+* Added an experimental lazy wide-column read API, `DB::GetEntityLazy()` / `DB::MultiGetEntityLazy()` (see `include/rocksdb/lazy_wide_columns.h`). The returned `LazyWideColumns` leaves blob-backed columns as unresolved references and, like an iterator, stays valid after the call returns, so the caller resolves only the columns -- and only the byte ranges -- it needs; a byte-range read of an uncompressed blob (separate-file or embedded) reads just those bytes from storage. Adds `Env::IOActivity::kLazyResolve` and `rocksdb.blobdb.lazy.*` statistics. Requires `max_open_files == -1`.
+* Added DB property `rocksdb.num-running-remote-compactions`, reporting the number of compaction service jobs currently waiting in `CompactionService::Wait()`.
+* Added DB property `rocksdb.num-unscheduled-compactions`, reporting compactions waiting in the DB's internal compaction queue but not yet assigned to a background job.
+* Added `DB::AbortCompactions()` and `DB::ResumeCompactions()` for per-column-family compaction abort and resume, with the same active-abort semantics as the DB-wide `AbortAllCompactions()`/`ResumeAllCompactions()` but scoped to a single column family. Calls are reference-counted independently per column family and compose with the DB-wide abort.
+
+### Public API Changes
+* Add IndexFactory compatibility names for the existing user-defined index API.
+* Added the experimental `CoroDB` capability, exposed by `DB::GetCoroDB()`, with lazy native coroutine variants of `Get` and `MultiGet`. The existing `DB::GetAsync()` and `DB::MultiGetAsync()` APIs dispatch through it.
+* Added `rocksdb_batched_multi_get_pinned_cf()` to return batch-owned pinned MultiGet values and per-key errors without allocating one wrapper per found value.
+* Add `DBOptions::read_io_executor_threads` to increase the maximum thread count of the shared filesystem read I/O executor used for asynchronous reads when opening a DB. Opening a DB never reduces the shared executor thread count. Change the experimental `FSRandomAccessFile::SubmitReadAsync()` API to return whether the asynchronous path was used, and add the `FILE_SUBMIT_ASYNC_READ_FALLBACK` ticker to count calls that fall back to synchronous reads.
+* `ReadOptions::table_filter` is now a *pointer* to a caller-owned `std::function` instead of a `std::function` value, to improve efficiency in copying and destroying `ReadOptions` objects (which RocksDB does internally). This is a breaking change for users of this relatively obscure option, who will now need to declare a `std::function` that outlives the read. As a side effect of `ReadOptions` being trivially copyable, new but legitimate compiler warnings may surface.
+* Removed the experimental `CreateAutoSkipCompressionManager()`. Auto-skip compression is now a first-class feature of the block-based table builder, controlled by the new experimental `CompressionOptions::auto_skip`.
+* Added `CoroStackableDB` for implementing wrappers over native coroutine reads and used it to add coroutine `Get` and `MultiGet` support to `DBWithTTL` and WritePrepared `TransactionDB`. A stackable wrapper advertises coroutine support only when its wrapped DB supports it; TTL value decoding and transaction visibility semantics are preserved.
+* Added a `kSstFileWriter` value to `TableFileCreationReason`, which `SstFileWriter` now reports instead of the catch-all `kMisc`. This, along with new `db_name` and `is_remote_compaction` fields in `FilterBuildingContext`, improves the contextual information available to custom filter and compression strategies.
+* Renamed `TransactionLogIterator` to `WalIterator` (and `include/rocksdb/transaction_log.h` to `include/rocksdb/wal_iterator.h`), because it iterates the write-ahead log and is unrelated to `TransactionDB`. The old names still work. `DB::GetUpdatesSince()` is unchanged. In Java, `WalIterator` is the new name and `TransactionLogIterator` remains as a subclass. Also documented existing behavior that was previously unstated: only writes that reached the WAL are visible (so `WriteOptions::disableWAL` and `IngestExternalFile()` leave holes that end iteration), what each iterator state means, and that `GetUpdatesSince()` silently starts later when the requested sequence number is unavailable.
+
+### Bug Fixes
+* Fixed a bug where `BlobFileBuilder` passed an SST path to `OnBlobFileCompleted()` instead of the completed blob path during BlobDB garbage-collection compactions, causing incorrect `SstFileManager` accounting and compaction failures on filesystems where the SST was not yet pathname-visible.
+
+### Performance Improvements
+* When `DBOptions::avoid_unnecessary_blocking_io` is true, obsolete `OPTIONS-*` files found during DB open are deleted by background purge instead of synchronously on the opening thread.
+
 ## 11.8.0 (07/28/2026)
 ### Public API Changes
 * Add callback-based asynchronous read APIs, `DB::GetAsync()` and `DB::MultiGetAsync()`. The idea is that when IO is required, RocksDB can suspend its internal coroutine read path, allowing the read executor thread to do other work. When the IO is complete, RocksDB invokes the user callback. This requires filesystem support for full performance benefits. A new filesystem API `FSRandomAccessFile::SubmitReadAsync()` is introduced for this. Unlike `ReadAsync`, the filesystem is responsible for eventually calling the callback. `FileSystem::GetReadExecutor()` returns the IO executor whose EventBases run coroutine read processing. In the Posix filesystem, an IO-uring based event loop is used to complete the IO.

include/rocksdb/external_table.h

@@ -270,6 +270,23 @@ class ExternalTableFactory : public Customizable {
   virtual ExternalTableBuilder* NewTableBuilder(
       const ExternalTableBuilderOptions& builder_options,
       const std::string& file_path, FSWritableFile* file) const = 0;
+
+  // Configures this factory using an implementation-defined string. RocksDB
+  // calls this method while preparing the table factory, before it is used to
+  // create table readers or builders. The configuration is immutable after
+  // the DB is opened and is persisted in the RocksDB OPTIONS file.
+  //
+  // Implementations must copy any needed data from config and should make this
+  // method idempotent because RocksDB may prepare an object more than once.
+  // The configuration must not contain secrets because OPTIONS files are
+  // stored as plaintext.
+  virtual Status Configure(const std::string& config) {
+    if (config.empty()) {
+      return Status::OK();
+    }
+    return Status::NotSupported(
+        "External table factory does not support configuration");
+  }
 };
 
 // Allocate a TableFactory that wraps around an ExternalTableFactory. Use this

include/rocksdb/version.h

@@ -13,7 +13,7 @@
 // minor or major version number planned for release.
 #define ROCKSDB_MAJOR 11
 #define ROCKSDB_MINOR 9
-#define ROCKSDB_PATCH 0
+#define ROCKSDB_PATCH 1
 
 // Make it easy to do conditional compilation based on version checks, i.e.
 // #if ROCKSDB_VERSION_GE(4, 5, 6)

options/cf_options.cc

@@ -134,6 +134,22 @@ static Status ParseCompressionOptions(const std::string& value,
   return Status::OK();
 }
 
+static Status CloneTableFactoryForOptionsUpdate(
+    TableFactory* table_factory,
+    std::shared_ptr<TableFactory>* cloned_factory) {
+  assert(table_factory != nullptr);
+  assert(cloned_factory != nullptr);
+
+  std::unique_ptr<TableFactory> clone = table_factory->Clone();
+  if (clone == nullptr) {
+    return Status::InvalidArgument(
+        "Table factory does not support option updates: " +
+        std::string(table_factory->Name()));
+  }
+  *cloned_factory = std::move(clone);
+  return Status::OK();
+}
+
 static Status TableFactoryParseFn(const ConfigOptions& opts,
                                   const std::string& name,
                                   const std::string& value, void* addr) {
@@ -170,7 +186,11 @@ static Status TableFactoryParseFn(const ConfigOptions& opts,
     if (table_factory->get() != nullptr) {
       std::string factory_name = table_factory->get()->Name();
       if (factory_name == TableFactory::kBlockBasedTableName()) {
-        new_factory = table_factory->get()->Clone();
+        s = CloneTableFactoryForOptionsUpdate(table_factory->get(),
+                                              &new_factory);
+        if (!s.ok()) {
+          return s;
+        }
       } else {
         s = Status::InvalidArgument("Cannot modify " + factory_name + " as " +
                                     name);
@@ -185,7 +205,11 @@ static Status TableFactoryParseFn(const ConfigOptions& opts,
     if (table_factory->get() != nullptr) {
       std::string factory_name = table_factory->get()->Name();
       if (factory_name == TableFactory::kPlainTableName()) {
-        new_factory = table_factory->get()->Clone();
+        s = CloneTableFactoryForOptionsUpdate(table_factory->get(),
+                                              &new_factory);
+        if (!s.ok()) {
+          return s;
+        }
       } else {
         s = Status::InvalidArgument("Cannot modify " + factory_name + " as " +
                                     name);
@@ -204,7 +228,10 @@ static Status TableFactoryParseFn(const ConfigOptions& opts,
       s = TableFactory::CreateFromString(opts, value, &new_factory);
     }
   } else if (table_factory->get() != nullptr) {
-    new_factory = table_factory->get()->Clone();
+    s = CloneTableFactoryForOptionsUpdate(table_factory->get(), &new_factory);
+    if (!s.ok()) {
+      return s;
+    }
     // Presumably passing a value for a specific field of the table factory
     s = new_factory->ConfigureOption(opts, name, value);
   } else {

table/block_based/block_based_table_builder.cc

@@ -364,6 +364,7 @@ struct BlockBasedTableBuilder::ParallelCompressionRep {
     std::string uncompressed;
     GrowableBuffer compressed;
     CompressionType compression_type = kNoCompression;
+    uint32_t contents_checksum = 0;
     // Set by the emit thread: this data block's compression was auto-skipped,
     // so no worker should compress it -- StateTransition auto-completes it
     // straight to the write stage. Set on every emit (true=skip,
@@ -2124,6 +2125,7 @@ void BlockBasedTableBuilder::EmitBlockForParallel(
                                       block_rep->prepared_index_entry.get());
   block_rep->compressed.Reset();
   block_rep->compression_type = kNoCompression;
+  block_rep->contents_checksum = 0;
   // Emit thread makes the auto-skip decision here so skipped blocks are never
   // dispatched to a worker for (no-op) compression; StateTransition
   // auto-completes them straight to the write stage.
@@ -2148,7 +2150,7 @@ void BlockBasedTableBuilder::EmitBlockForParallel(
       Status s = CompressAndVerifyBlock(
           block_rep->uncompressed, /*is_data_block=*/true,
           r->data_block_working_area, &block_rep->compressed,
-          &block_rep->compression_type);
+          &block_rep->compression_type, &block_rep->contents_checksum);
       if (UNLIKELY(!s.ok())) {
         r->SetStatus(s);
         pc_rep.SetAbort(pc_rep.emit_thread_state);
@@ -2208,6 +2210,7 @@ void BlockBasedTableBuilder::WriteBlock(const Slice& uncompressed_block_data,
   // Single-threaded context only
   assert(!r->IsParallelCompressionActive());
   CompressionType type = kNoCompression;
+  uint32_t contents_checksum = 0;
   bool is_data_block = block_type == BlockType::kData;
   // NOTE: only index and data blocks are currently compressed
   assert(is_data_block || block_type == BlockType::kIndex);
@@ -2219,11 +2222,11 @@ void BlockBasedTableBuilder::WriteBlock(const Slice& uncompressed_block_data,
       r->AutoSkipDecideBypassEligible()) {
     r->AutoSkipRecordBypass(uncompressed_block_data.size());
   } else {
-    Status compress_status =
-        CompressAndVerifyBlock(uncompressed_block_data, is_data_block,
-                               is_data_block ? r->data_block_working_area
-                                             : r->index_block_working_area,
-                               &r->single_threaded_compressed_output, &type);
+    Status compress_status = CompressAndVerifyBlock(
+        uncompressed_block_data, is_data_block,
+        is_data_block ? r->data_block_working_area
+                      : r->index_block_working_area,
+        &r->single_threaded_compressed_output, &type, &contents_checksum);
     r->SetStatus(compress_status);
     if (UNLIKELY(!ok())) {
       return;
@@ -2235,7 +2238,8 @@ void BlockBasedTableBuilder::WriteBlock(const Slice& uncompressed_block_data,
   WriteMaybeCompressedBlock(
       type == kNoCompression ? uncompressed_block_data
                              : Slice(r->single_threaded_compressed_output),
-      type, handle, block_type, &uncompressed_block_data, skip_delta_encoding);
+      type, handle, block_type, &uncompressed_block_data, skip_delta_encoding,
+      type == kNoCompression ? nullptr : &contents_checksum);
   r->single_threaded_compressed_output.Reset();
   if (is_data_block) {
     r->props.data_size = r->get_offset();
@@ -2274,7 +2278,8 @@ void BlockBasedTableBuilder::BGWorker(WorkingAreaPair& working_area) {
     auto compress_fn = [this, block_rep, &ios, &working_area]() {
       ios = status_to_io_status(CompressAndVerifyBlock(
           block_rep->uncompressed, /*is_data_block=*/true, working_area,
-          &block_rep->compressed, &block_rep->compression_type));
+          &block_rep->compressed, &block_rep->compression_type,
+          &block_rep->contents_checksum));
     };
     auto write_fn = [this, block_rep, &ios]() {
       Slice compressed = block_rep->compressed;
@@ -2284,7 +2289,10 @@ void BlockBasedTableBuilder::BGWorker(WorkingAreaPair& working_area) {
           block_rep->compression_type == kNoCompression ? uncompressed
                                                         : compressed,
           block_rep->compression_type, &rep_->pending_handle, BlockType::kData,
-          &uncompressed, &skip_delta_encoding);
+          &uncompressed, &skip_delta_encoding,
+          block_rep->compression_type == kNoCompression
+              ? nullptr
+              : &block_rep->contents_checksum);
       if (LIKELY(ios.ok())) {
         rep_->props.data_size = rep_->get_offset();
         rep_->props.uncompressed_data_size += block_rep->uncompressed.size();
@@ -2332,9 +2340,10 @@ void BlockBasedTableBuilder::BGWorker(WorkingAreaPair& working_area) {
 Status BlockBasedTableBuilder::CompressAndVerifyBlock(
     const Slice& uncompressed_block_data, bool is_data_block,
     WorkingAreaPair& working_area, GrowableBuffer* compressed_output,
-    CompressionType* result_compression_type) {
+    CompressionType* result_compression_type, uint32_t* result_checksum) {
   Rep* r = rep_.get();
   Status status;
+  *result_checksum = 0;
 
   UnownedPtr<Compressor> compressor = nullptr;
   Decompressor* verify_decomp = nullptr;
@@ -2380,6 +2389,16 @@ Status BlockBasedTableBuilder::CompressAndVerifyBlock(
           "BlockBasedTableBuilder::CompressAndVerifyBlock:TamperWithResultType",
           &type);
 
+      if (type != kNoCompression) {
+        *result_checksum = ComputeBuiltinChecksumWithLastByte(
+            r->table_options.checksum, compressed_output->data(),
+            compressed_output->size(), /*last_byte*/ type);
+        TEST_SYNC_POINT_CALLBACK(
+            "BlockBasedTableBuilder::CompressAndVerifyBlock:"
+            "TamperWithCompressedDataBeforeVerify",
+            compressed_output);
+      }
+
       // Some of the compression algorithms are known to be unreliable. If
       // the verify_compression flag is set then try to de-compress the
       // compressed data and compare to the input.
@@ -2485,19 +2504,19 @@ Status BlockBasedTableBuilder::CompressAndVerifyBlock(
 void BlockBasedTableBuilder::WriteMaybeCompressedBlock(
     const Slice& block_contents, CompressionType comp_type, BlockHandle* handle,
     BlockType block_type, const Slice* uncompressed_block_data,
-    bool* skip_delta_encoding) {
+    bool* skip_delta_encoding, const uint32_t* precomputed_checksum) {
   // Must have pre-checked status in single-threaded context
   assert(status().ok());
   assert(io_status().ok());
   rep_->SetIOStatus(WriteMaybeCompressedBlockImpl(
       block_contents, comp_type, handle, block_type, uncompressed_block_data,
-      skip_delta_encoding));
+      skip_delta_encoding, precomputed_checksum));
 }
 
 IOStatus BlockBasedTableBuilder::WriteMaybeCompressedBlockImpl(
     const Slice& block_contents, CompressionType comp_type, BlockHandle* handle,
     BlockType block_type, const Slice* uncompressed_block_data,
-    bool* skip_delta_encoding) {
+    bool* skip_delta_encoding, const uint32_t* precomputed_checksum) {
   // File format contains a sequence of blocks where each block has:
   //    block_data: uint8[n]
   //    compression_type: uint8
@@ -2567,6 +2586,18 @@ IOStatus BlockBasedTableBuilder::WriteMaybeCompressedBlockImpl(
     uncompressed_block_data = &block_contents;
     assert(comp_type == kNoCompression);
   }
+  assert(precomputed_checksum == nullptr || comp_type != kNoCompression);
+
+  r->compression_types_used.Add(comp_type);
+  std::array<char, kBlockTrailerSize> trailer;
+  trailer[0] = comp_type;
+  uint32_t checksum =
+      precomputed_checksum != nullptr
+          ? *precomputed_checksum
+          : ComputeBuiltinChecksumWithLastByte(
+                r->table_options.checksum, block_contents.data(),
+                block_contents.size(), /*last_byte*/ comp_type);
+  checksum += ChecksumModifierForContext(r->base_context_checksum, offset);
 
   // TODO: consider a variant of this function that puts the trailer after
   // block_contents (if it comes from a std::string) so we only need one
@@ -2578,14 +2609,6 @@ IOStatus BlockBasedTableBuilder::WriteMaybeCompressedBlockImpl(
     }
   }
 
-  r->compression_types_used.Add(comp_type);
-  std::array<char, kBlockTrailerSize> trailer;
-  trailer[0] = comp_type;
-  uint32_t checksum = ComputeBuiltinChecksumWithLastByte(
-      r->table_options.checksum, block_contents.data(), block_contents.size(),
-      /*last_byte*/ comp_type);
-  checksum += ChecksumModifierForContext(r->base_context_checksum, offset);
-
   if (block_type == BlockType::kFilter) {
     io_s = status_to_io_status(
         r->filter_builder->MaybePostVerifyFilter(block_contents));

table/block_based/block_based_table_builder.h

@@ -154,14 +154,16 @@ class BlockBasedTableBuilder : public TableBuilder {
   void WriteBlock(const Slice& block_contents, BlockHandle* handle,
                   BlockType block_type, bool* skip_delta_encoding = nullptr);
   // Directly write data to the file.
-  void WriteMaybeCompressedBlock(const Slice& block_contents, CompressionType,
-                                 BlockHandle* handle, BlockType block_type,
-                                 const Slice* uncompressed_block_data = nullptr,
-                                 bool* skip_delta_encoding = nullptr);
+  void WriteMaybeCompressedBlock(
+      const Slice& block_contents, CompressionType, BlockHandle* handle,
+      BlockType block_type, const Slice* uncompressed_block_data = nullptr,
+      bool* skip_delta_encoding = nullptr,
+      const uint32_t* precomputed_checksum = nullptr);
   IOStatus WriteMaybeCompressedBlockImpl(
       const Slice& block_contents, CompressionType, BlockHandle* handle,
       BlockType block_type, const Slice* uncompressed_block_data = nullptr,
-      bool* skip_delta_encoding = nullptr);
+      bool* skip_delta_encoding = nullptr,
+      const uint32_t* precomputed_checksum = nullptr);
 
   void SetupCacheKeyPrefix(const TableBuilderOptions& tbo);
 
@@ -240,7 +242,8 @@ class BlockBasedTableBuilder : public TableBuilder {
                                 bool is_data_block,
                                 WorkingAreaPair& working_area,
                                 GrowableBuffer* compressed_output,
-                                CompressionType* result_compression_type);
+                                CompressionType* result_compression_type,
+                                uint32_t* result_checksum);
 
   // If configured, start worker threads for parallel compression
   void MaybeStartParallelCompression();

table/external_table.cc

@@ -6,18 +6,22 @@
 #include "rocksdb/external_table.h"
 
 #include <chrono>
+#include <cstddef>
+#include <unordered_map>
 
 #include "db/dbformat.h"
 #include "logging/logging.h"
 #include "rocksdb/listener.h"
 #include "rocksdb/statistics.h"
 #include "rocksdb/table.h"
+#include "rocksdb/utilities/options_type.h"
 #include "table/block_based/block.h"
 #include "table/get_context.h"
 #include "table/internal_iterator.h"
 #include "table/meta_blocks.h"
 #include "table/table_builder.h"
 #include "table/table_reader.h"
+#include "util/string_util.h"
 
 namespace ROCKSDB_NAMESPACE {
 
@@ -640,14 +644,46 @@ class ExternalTableBuilderAdapter : public TableBuilder {
       table_properties_collectors_;
 };
 
+struct ExternalTableFactoryAdapterOptions {
+  static const char* kName() { return "ExternalTableFactoryAdapterOptions"; }
+
+  std::string config;
+};
+
+static const std::unordered_map<std::string, OptionTypeInfo>&
+GetExternalTableFactoryAdapterOptionsTypeInfo() {
+  static const std::unordered_map<std::string, OptionTypeInfo> type_info = {
+      {"external_table_config",
+       OptionTypeInfo(offsetof(ExternalTableFactoryAdapterOptions, config),
+                      OptionType::kString)
+           .SetSerializeFunc([](const ConfigOptions&, const std::string&,
+                                const void* addr, std::string* value) {
+             const std::string* config = static_cast<const std::string*>(addr);
+             *value = "{" + EscapeOptionString(*config) + "}";
+             return Status::OK();
+           })}};
+  return type_info;
+}
+
 class ExternalTableFactoryAdapter : public TableFactory {
  public:
   explicit ExternalTableFactoryAdapter(
       std::shared_ptr<ExternalTableFactory> inner)
-      : inner_(std::move(inner)) {}
+      : inner_(std::move(inner)) {
+    RegisterOptions(&options_,
+                    &GetExternalTableFactoryAdapterOptionsTypeInfo());
+  }
 
   const char* Name() const override { return inner_->Name(); }
 
+  Status PrepareOptions(const ConfigOptions& config_options) override {
+    Status status = TableFactory::PrepareOptions(config_options);
+    if (status.ok()) {
+      status = inner_->Configure(options_.config);
+    }
+    return status;
+  }
+
   using TableFactory::NewTableReader;
   Status NewTableReader(
       const ReadOptions& ro, const TableReaderOptions& topts,
@@ -746,6 +782,7 @@ class ExternalTableFactoryAdapter : public TableFactory {
   };
 
   std::shared_ptr<ExternalTableFactory> inner_;
+  ExternalTableFactoryAdapterOptions options_;
 };
 
 }  // namespace

table/table_test.cc

@@ -33,6 +33,7 @@
 #include "monitoring/statistics_impl.h"
 #include "options/cf_options.h"
 #include "options/options_helper.h"
+#include "options/options_parser.h"
 #include "port/port.h"
 #include "port/stack_trace.h"
 #include "rocksdb/cache.h"
@@ -7277,6 +7278,72 @@ class ExternalTableTest : public DBTestBase {
     bool read_via_options_fs_;
   };
 
+  class ConfigurableDummyExternalTableFactory
+      : public DummyExternalTableFactory {
+   public:
+    ConfigurableDummyExternalTableFactory()
+        : DummyExternalTableFactory(/*support_property_block=*/false) {}
+
+    static const char* kClassName() {
+      return "ConfigurableDummyExternalTableFactory";
+    }
+
+    static const char* kValidConfig() { return "mode=fast;limit=7"; }
+
+    static const char* kAlternateValidConfig() {
+      return "mode=compact;limit=11";
+    }
+
+    static std::string ValidFactoryConfig() {
+      return "{id=" + std::string(kClassName()) + ";external_table_config={" +
+             kValidConfig() + "}}";
+    }
+
+    static std::string SerializedValidConfig() {
+      return SerializeConfig(kValidConfig());
+    }
+
+    static std::string SerializedAlternateValidConfig() {
+      return SerializeConfig(kAlternateValidConfig());
+    }
+
+    const char* Name() const override { return kClassName(); }
+
+    Status Configure(const std::string& config) override {
+      if (config.empty() || config == kValidConfig() ||
+          config == kAlternateValidConfig()) {
+        config_ = config;
+        return Status::OK();
+      }
+      return Status::InvalidArgument("Invalid dummy external table config");
+    }
+
+    const std::string& config() const { return config_; }
+
+   private:
+    static std::string SerializeConfig(const std::string& config) {
+      return "{" + config + "}";
+    }
+
+    std::string config_;
+  };
+
+  static void RegisterConfigurableDummyExternalTableFactory() {
+    static FactoryFunc<TableFactory> registration =
+        ObjectLibrary::Default()->AddFactory<TableFactory>(
+            ConfigurableDummyExternalTableFactory::kClassName(),
+            [](const std::string& /*uri*/, std::unique_ptr<TableFactory>* guard,
+               std::string* /*errmsg*/) {
+              std::shared_ptr<ExternalTableFactory> inner =
+                  std::make_shared<ConfigurableDummyExternalTableFactory>();
+              std::unique_ptr<TableFactory> factory =
+                  NewExternalTableFactory(std::move(inner));
+              guard->reset(factory.release());
+              return guard->get();
+            });
+    (void)registration;
+  }
+
   class CountingFileReadListener : public EventListener {
    public:
     bool ShouldBeNotifiedOnFileIO() override { return true; }
@@ -7329,6 +7396,157 @@ class ExternalTableTest : public DBTestBase {
   };
 };
 
+TEST_F(ExternalTableTest, BootstrapConfig) {
+  RegisterConfigurableDummyExternalTableFactory();
+
+  ConfigOptions config_options;
+  std::shared_ptr<TableFactory> table_factory;
+  ASSERT_OK(TableFactory::CreateFromString(
+      config_options,
+      ConfigurableDummyExternalTableFactory::ValidFactoryConfig(),
+      &table_factory));
+  ASSERT_NE(table_factory, nullptr);
+
+  std::string serialized_config;
+  ASSERT_OK(table_factory->GetOption(config_options, "external_table_config",
+                                     &serialized_config));
+  ASSERT_EQ(serialized_config,
+            ConfigurableDummyExternalTableFactory::SerializedValidConfig());
+
+  std::shared_ptr<TableFactory> invalid_factory;
+  const std::string invalid_config =
+      "{id=" +
+      std::string(ConfigurableDummyExternalTableFactory::kClassName()) +
+      ";external_table_config={mode=invalid}}";
+  ASSERT_NOK(TableFactory::CreateFromString(config_options, invalid_config,
+                                            &invalid_factory));
+}
+
+TEST_F(ExternalTableTest, BootstrapConfigOptionsFileRoundTrip) {
+  RegisterConfigurableDummyExternalTableFactory();
+
+  DBOptions db_options;
+  db_options.env = env_;
+  ConfigOptions config_options(db_options);
+  std::shared_ptr<TableFactory> table_factory;
+  ASSERT_OK(TableFactory::CreateFromString(
+      config_options,
+      ConfigurableDummyExternalTableFactory::ValidFactoryConfig(),
+      &table_factory));
+
+  ColumnFamilyOptions cf_options;
+  cf_options.table_factory = table_factory;
+  const std::vector<std::string> cf_names{"default"};
+  const std::vector<ColumnFamilyOptions> all_cf_options{cf_options};
+  const std::string options_file =
+      test::PerThreadDBPath("external_table_config_options");
+  std::shared_ptr<FileSystem> fs = db_options.env->GetFileSystem();
+  ASSERT_OK(PersistRocksDBOptions(WriteOptions(), config_options, db_options,
+                                  cf_names, all_cf_options, options_file,
+                                  fs.get()));
+
+  std::string contents;
+  ASSERT_OK(ReadFileToString(env_, options_file, &contents));
+  ASSERT_NE(contents.find("external_table_config={mode=fast;limit=7}"),
+            std::string::npos);
+
+  RocksDBOptionsParser parser;
+  ASSERT_OK(parser.Parse(config_options, options_file, fs.get()));
+  ASSERT_EQ(parser.cf_opts()->size(), 1U);
+  ASSERT_NE(parser.cf_opts()->front().table_factory, nullptr);
+  std::string loaded_config;
+  ASSERT_OK(parser.cf_opts()->front().table_factory->GetOption(
+      config_options, "external_table_config", &loaded_config));
+  ASSERT_EQ(loaded_config,
+            ConfigurableDummyExternalTableFactory::SerializedValidConfig());
+
+  ASSERT_OK(env_->DeleteFile(options_file));
+}
+
+TEST_F(ExternalTableTest, BootstrapConfigIsImmutable) {
+  RegisterConfigurableDummyExternalTableFactory();
+
+  Options options = GetDefaultOptions();
+  ConfigOptions config_options(options);
+  ASSERT_OK(TableFactory::CreateFromString(
+      config_options,
+      ConfigurableDummyExternalTableFactory::ValidFactoryConfig(),
+      &options.table_factory));
+  options.create_if_missing = true;
+
+  const std::string dbname =
+      test::PerThreadDBPath("external_table_config_immutable");
+  ASSERT_OK(DestroyDB(dbname, options));
+  std::unique_ptr<DB> db;
+  ASSERT_OK(DB::Open(options, dbname, &db));
+  Status update_status =
+      db->SetOptions({{"table_factory.external_table_config", "{mode=slow}"}});
+  ASSERT_TRUE(update_status.IsInvalidArgument()) << update_status.ToString();
+
+  Options current_options = db->GetOptions();
+  std::string current_config;
+  ASSERT_OK(current_options.table_factory->GetOption(
+      config_options, "external_table_config", &current_config));
+  ASSERT_EQ(current_config,
+            ConfigurableDummyExternalTableFactory::SerializedValidConfig());
+
+  ASSERT_OK(db->Close());
+  db.reset();
+  ASSERT_OK(DestroyDB(dbname, options));
+}
+
+TEST_F(ExternalTableTest, BootstrapConfigCanChangeBetweenDBOpens) {
+  Options options = GetDefaultOptions();
+  options.create_if_missing = true;
+
+  std::shared_ptr<ConfigurableDummyExternalTableFactory> first_factory =
+      std::make_shared<ConfigurableDummyExternalTableFactory>();
+  options.table_factory = NewExternalTableFactory(first_factory);
+  ConfigOptions config_options(options);
+  ASSERT_OK(options.table_factory->ConfigureFromString(
+      config_options,
+      "external_table_config=" +
+          ConfigurableDummyExternalTableFactory::SerializedValidConfig()));
+  ASSERT_EQ(first_factory->config(),
+            ConfigurableDummyExternalTableFactory::kValidConfig());
+
+  const std::string dbname =
+      test::PerThreadDBPath("external_table_config_reopen");
+  ASSERT_OK(DestroyDB(dbname, options));
+  std::unique_ptr<DB> db;
+  ASSERT_OK(DB::Open(options, dbname, &db));
+  ASSERT_EQ(first_factory->config(),
+            ConfigurableDummyExternalTableFactory::kValidConfig());
+  ASSERT_OK(db->Close());
+  db.reset();
+
+  std::shared_ptr<ConfigurableDummyExternalTableFactory> second_factory =
+      std::make_shared<ConfigurableDummyExternalTableFactory>();
+  options.table_factory = NewExternalTableFactory(second_factory);
+  ConfigOptions reopen_config_options(options);
+  ASSERT_OK(options.table_factory->ConfigureFromString(
+      reopen_config_options, "external_table_config=" +
+                                 ConfigurableDummyExternalTableFactory::
+                                     SerializedAlternateValidConfig()));
+  ASSERT_EQ(second_factory->config(),
+            ConfigurableDummyExternalTableFactory::kAlternateValidConfig());
+
+  ASSERT_OK(DB::Open(options, dbname, &db));
+  ASSERT_EQ(second_factory->config(),
+            ConfigurableDummyExternalTableFactory::kAlternateValidConfig());
+
+  std::string reopened_config;
+  ASSERT_OK(db->GetOptions().table_factory->GetOption(
+      reopen_config_options, "external_table_config", &reopened_config));
+  ASSERT_EQ(
+      reopened_config,
+      ConfigurableDummyExternalTableFactory::SerializedAlternateValidConfig());
+
+  ASSERT_OK(db->Close());
+  db.reset();
+  ASSERT_OK(DestroyDB(dbname, options));
+}
+
 TEST_F(ExternalTableTest, BasicTest) {
   std::shared_ptr<ExternalTableFactory> factory =
       std::make_shared<DummyExternalTableFactory>(

unreleased_history/bug_fixes/big_endian_fault_injection_build.md

@@ -0,0 +1 @@
+Fixed RocksDB builds on big-endian architectures such as s390x by disabling the optional persistent fault-injection log, whose binary format is little-endian-only.

unreleased_history/bug_fixes/blob_compaction_wrong_output_path.md

@@ -1 +0,0 @@
-Fixed a bug where `BlobFileBuilder` passed an SST path to `OnBlobFileCompleted()` instead of the completed blob path during BlobDB garbage-collection compactions, causing incorrect `SstFileManager` accounting and compaction failures on filesystems where the SST was not yet pathname-visible.

unreleased_history/new_features/auto_skip_compression.md

@@ -1 +0,0 @@
-Added an integrated and improved compression auto-skip feature (new CompressionOptions, still experimental) that saves CPU by skipping compression when and where the overall ratio is inadequate. This improves on the previous auto-skip compression manager by (a) taking achieved compression ratios into account, (b) using a continuously evolving payoff estimate, and (c) combining natively with parallel compression.

unreleased_history/new_features/blob_file_writable_file_max_buffer_size.md

@@ -1 +0,0 @@
-Added the `blob_file_writable_file_max_buffer_size` mutable column family option. When set to a non-zero value, blob-file writers use it as their `WritableFileWriter` max buffer size instead of inheriting `DBOptions::writable_file_max_buffer_size`.

unreleased_history/new_features/lazy_wide_columns.md

@@ -1 +0,0 @@
-Added an experimental lazy wide-column read API, `DB::GetEntityLazy()` / `DB::MultiGetEntityLazy()` (see `include/rocksdb/lazy_wide_columns.h`). The returned `LazyWideColumns` leaves blob-backed columns as unresolved references and, like an iterator, stays valid after the call returns, so the caller resolves only the columns -- and only the byte ranges -- it needs; a byte-range read of an uncompressed blob (separate-file or embedded) reads just those bytes from storage. Adds `Env::IOActivity::kLazyResolve` and `rocksdb.blobdb.lazy.*` statistics. Requires `max_open_files == -1`.

unreleased_history/new_features/num_running_remote_compactions_property.md

@@ -1 +0,0 @@
-Added DB property `rocksdb.num-running-remote-compactions`, reporting the number of compaction service jobs currently waiting in `CompactionService::Wait()`.

unreleased_history/new_features/num_unscheduled_compactions_property.md

@@ -1 +0,0 @@
-Added DB property `rocksdb.num-unscheduled-compactions`, reporting compactions waiting in the DB's internal compaction queue but not yet assigned to a background job.

unreleased_history/new_features/per_cf_compaction_abort.md

@@ -1 +0,0 @@
-Added `DB::AbortCompactions()` and `DB::ResumeCompactions()` for per-column-family compaction abort and resume, with the same active-abort semantics as the DB-wide `AbortAllCompactions()`/`ResumeAllCompactions()` but scoped to a single column family. Calls are reference-counted independently per column family and compose with the DB-wide abort.

unreleased_history/performance_improvements/delete-obsolete-options-files-background-on-open.md

@@ -1 +0,0 @@
-When `DBOptions::avoid_unnecessary_blocking_io` is true, obsolete `OPTIONS-*` files found during DB open are deleted by background purge instead of synchronously on the opening thread.

unreleased_history/public_api_changes/index_factory_aliases.md

@@ -1 +0,0 @@
-Add IndexFactory compatibility names for the existing user-defined index API.

unreleased_history/public_api_changes/native_coroutine_reads.md

@@ -1 +0,0 @@
-Added the experimental `CoroDB` capability, exposed by `DB::GetCoroDB()`, with lazy native coroutine variants of `Get` and `MultiGet`. The existing `DB::GetAsync()` and `DB::MultiGetAsync()` APIs dispatch through it.

unreleased_history/public_api_changes/pinnable_multi_get_batch.md

@@ -1 +0,0 @@
-Added `rocksdb_batched_multi_get_pinned_cf()` to return batch-owned pinned MultiGet values and per-key errors without allocating one wrapper per found value.

unreleased_history/public_api_changes/read_io_executor_threads.md

@@ -1 +0,0 @@
-Add `DBOptions::read_io_executor_threads` to increase the maximum thread count of the shared filesystem read I/O executor used for asynchronous reads when opening a DB. Opening a DB never reduces the shared executor thread count. Change the experimental `FSRandomAccessFile::SubmitReadAsync()` API to return whether the asynchronous path was used, and add the `FILE_SUBMIT_ASYNC_READ_FALLBACK` ticker to count calls that fall back to synchronous reads.

unreleased_history/public_api_changes/read_options_table_filter_pointer.md

@@ -1 +0,0 @@
-`ReadOptions::table_filter` is now a *pointer* to a caller-owned `std::function` instead of a `std::function` value, to improve efficiency in copying and destroying `ReadOptions` objects (which RocksDB does internally). This is a breaking change for users of this relatively obscure option, who will now need to declare a `std::function` that outlives the read. As a side effect of `ReadOptions` being trivially copyable, new but legitimate compiler warnings may surface.

unreleased_history/public_api_changes/remove_auto_skip_compression_manager.md

@@ -1 +0,0 @@
-Removed the experimental `CreateAutoSkipCompressionManager()`. Auto-skip compression is now a first-class feature of the block-based table builder, controlled by the new experimental `CompressionOptions::auto_skip`.

unreleased_history/public_api_changes/stackable_coroutine_reads.md

@@ -1 +0,0 @@
-Added `CoroStackableDB` for implementing wrappers over native coroutine reads and used it to add coroutine `Get` and `MultiGet` support to `DBWithTTL` and WritePrepared `TransactionDB`. A stackable wrapper advertises coroutine support only when its wrapped DB supports it; TTL value decoding and transaction visibility semantics are preserved.

unreleased_history/public_api_changes/table_file_creation_reason_sst_file_writer.md

@@ -1 +0,0 @@
-Added a `kSstFileWriter` value to `TableFileCreationReason`, which `SstFileWriter` now reports instead of the catch-all `kMisc`. This, along with new `db_name` and `is_remote_compaction` fields in `FilterBuildingContext`, improves the contextual information available to custom filter and compression strategies.

unreleased_history/public_api_changes/wal_iterator_rename.md

@@ -1 +0,0 @@
-Renamed `TransactionLogIterator` to `WalIterator` (and `include/rocksdb/transaction_log.h` to `include/rocksdb/wal_iterator.h`), because it iterates the write-ahead log and is unrelated to `TransactionDB`. The old names still work. `DB::GetUpdatesSince()` is unchanged. In Java, `WalIterator` is the new name and `TransactionLogIterator` remains as a subclass. Also documented existing behavior that was previously unstated: only writes that reached the WAL are visible (so `WriteOptions::disableWAL` and `IngestExternalFile()` leave holes that end iteration), what each iterator state means, and that `GetUpdatesSince()` silently starts later when the requested sequence number is unavailable.

util/compression_test.cc

@@ -16,6 +16,7 @@
 #include "table/block_based/block_builder.h"
 #include "table/block_based/data_block_footer.h"
 #include "test_util/testutil.h"
+#include "util/aligned_buffer.h"
 #include "util/auto_tune_compressor.h"
 #include "util/coding.h"
 #include "util/random.h"
@@ -931,6 +932,74 @@ TEST_P(CompressionFailuresTest, CompressionFailures) {
   }
 }
 
+TEST_F(DBCompressionTest, VerifyCompressionChecksFinalCompressedBlockContents) {
+  CompressionType compression_type = kNoCompression;
+  for (CompressionType supported : GetSupportedCompressions()) {
+    if (supported != kNoCompression) {
+      compression_type = supported;
+      break;
+    }
+  }
+  if (compression_type == kNoCompression) {
+    return;
+  }
+
+  struct SyncPointCleanup {
+    ~SyncPointCleanup() {
+      SyncPoint::GetInstance()->DisableProcessing();
+      SyncPoint::GetInstance()->ClearAllCallBacks();
+    }
+  } sync_point_cleanup;
+
+  for (uint32_t parallel_threads : {1, 4}) {
+    SCOPED_TRACE("parallel_threads=" + std::to_string(parallel_threads));
+
+    Options options = CurrentOptions();
+    options.compression = compression_type;
+    options.compression_opts.max_compressed_bytes_per_kb = 1024;
+    options.compression_opts.parallel_threads = parallel_threads;
+
+    BlockBasedTableOptions table_options;
+    table_options.block_size = 512;
+    table_options.verify_compression = true;
+    options.table_factory.reset(NewBlockBasedTableFactory(table_options));
+
+    DestroyAndReopen(options);
+
+    SyncPoint::GetInstance()->DisableProcessing();
+    SyncPoint::GetInstance()->ClearAllCallBacks();
+
+    std::atomic<int> tamper_count{0};
+    SyncPoint::GetInstance()->SetCallBack(
+        "BlockBasedTableBuilder::CompressAndVerifyBlock:"
+        "TamperWithCompressedDataBeforeVerify",
+        [&](void* arg) {
+          auto* output = static_cast<GrowableBuffer*>(arg);
+          ASSERT_FALSE(output->empty());
+          output->data()[output->size() - 1]++;
+          tamper_count++;
+        });
+    SyncPoint::GetInstance()->EnableProcessing();
+
+    Random rnd(405);
+    constexpr int kValUnitSize = 16;
+    constexpr int kValSize = 256;
+    for (int i = 0; i < 5; i++) {
+      std::string value_unit = rnd.RandomString(kValUnitSize);
+      std::string value;
+      for (int j = 0; j < kValSize; j += kValUnitSize) {
+        value += value_unit;
+      }
+      ASSERT_OK(Put(Key(i), value));
+    }
+
+    Status s = Flush();
+
+    ASSERT_GT(tamper_count.load(), 0);
+    ASSERT_TRUE(s.IsCorruption()) << s.ToString();
+  }
+}
+
 TEST_F(DBCompressionTest, CompressionOptions) {
   if (!Zlib_Supported() || !Snappy_Supported()) {
     return;

utilities/fault_injection_fs.cc

@@ -26,6 +26,7 @@
 #include "env/io_posix.h"
 #include "monitoring/thread_status_util.h"
 #include "port/lang.h"
+#include "port/port.h"
 #include "port/stack_trace.h"
 #include "rocksdb/env.h"
 #include "rocksdb/io_status.h"
@@ -79,7 +80,9 @@ uint8_t DetailPayloadSize(const InjectedErrorLog::TaggedEntryDetail& detail) {
 
 int OpenInjectedErrorLogFile(const std::string& path) {
 #ifndef OS_WIN
-  if (path.empty()) {
+  // Entry is written in native layout, while the decoder expects little-endian
+  // fields. Keep the optional log disabled on unsupported architectures.
+  if (!port::kLittleEndian || path.empty()) {
     return -1;
   }
   int flags = O_WRONLY | O_CREAT | O_TRUNC;

utilities/fault_injection_fs.h

@@ -44,6 +44,8 @@ namespace ROCKSDB_NAMESPACE {
 // A binary log that records injected errors directly to a file.
 // Thread-safe for concurrent writes. The log file can be flushed on exit or
 // from a signal handler so records survive clean exits and crash paths.
+// Disabled on big-endian platforms because the decoder expects little-endian
+// fields in the native Entry layout.
 class InjectedErrorLog {
  public:
   static constexpr size_t kMaxOpNameLen = 32;
@@ -86,12 +88,13 @@ class InjectedErrorLog {
     char op_name[kMaxOpNameLen];
     char file_name[kMaxFileNameLen];
     char status_message[kMaxStatusMessageLen];
+    // Explicit tail padding so the size does not depend on the alignment of
+    // uint64_t (4 bytes on i386, 8 bytes on most other targets).
+    uint8_t padding[4];
   };
 
   static_assert(sizeof(Entry) == 256,
                 "Injected error log entry size must stay stable");
-  static_assert(port::kLittleEndian,
-                "Injected error log requires little-endian platforms");
 
   explicit InjectedErrorLog(const std::string& path);
   ~InjectedErrorLog();

utilities/fault_injection_fs_test.cc

@@ -10,6 +10,7 @@
 #include <thread>
 #include <vector>
 
+#include "port/port.h"
 #include "test_util/testharness.h"
 
 namespace ROCKSDB_NAMESPACE {
@@ -79,6 +80,11 @@ class CloseCountingWritableFile : public FSWritableFileOwnerWrapper {
 // the expected fixed-width fields. The test writes one entry, flushes the
 // file, and decodes the raw bytes back into an entry struct.
 TEST_F(InjectedErrorLogTest, BasicRecordAndFlush) {
+  if (!port::kLittleEndian) {
+    ROCKSDB_GTEST_BYPASS(
+        "Persistent injected error log requires little-endian");
+    return;
+  }
   std::string path = test::PerThreadDBPath("injected_error_log_basic.bin");
   {
     InjectedErrorLog log(path);
@@ -114,6 +120,11 @@ TEST_F(InjectedErrorLogTest, BasicRecordAndFlush) {
 // to the old in-memory ring size. The test writes more than 1,000 entries and
 // checks that both the first and last ones are still present in the file.
 TEST_F(InjectedErrorLogTest, DirectLogKeepsAllEntries) {
+  if (!port::kLittleEndian) {
+    ROCKSDB_GTEST_BYPASS(
+        "Persistent injected error log requires little-endian");
+    return;
+  }
   std::string path =
       test::PerThreadDBPath("injected_error_log_all_entries.bin");
   {
@@ -144,6 +155,11 @@ TEST_F(InjectedErrorLogTest, DirectLogKeepsAllEntries) {
 // shared file. The test has several threads emit records concurrently and then
 // checks the raw file size matches the total number of writes.
 TEST_F(InjectedErrorLogTest, ConcurrentRecord) {
+  if (!port::kLittleEndian) {
+    ROCKSDB_GTEST_BYPASS(
+        "Persistent injected error log requires little-endian");
+    return;
+  }
   std::string path = test::PerThreadDBPath("injected_error_log_concurrent.bin");
   constexpr int kNumThreads = 4;
   constexpr int kRecordsPerThread = 200;
@@ -180,6 +196,11 @@ TEST_F(InjectedErrorLogTest, ConcurrentRecord) {
 // in the fixed-width record. The test logs a path longer than the file-name
 // field and checks the stored sample matches the expected tail bytes.
 TEST_F(InjectedErrorLogTest, LongFileNameKeepsSuffix) {
+  if (!port::kLittleEndian) {
+    ROCKSDB_GTEST_BYPASS(
+        "Persistent injected error log requires little-endian");
+    return;
+  }
   std::string path =
       test::PerThreadDBPath("injected_error_log_suffix_truncation.bin");
   const std::string long_file_name =