Update version and HISTORY.md for 11.3.2 release

Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14881

Read-only DB instances can share a directory with a live writer, so their view of live files is only a snapshot from the time they opened. After read-only DB open began setting `opened_successfully_` for its intended open-success semantics, that state had an unintended side effect: the common close path treated a successful read-only open like a read-write open and ran obsolete-file cleanup. If the writer created or made files live after the read-only handle opened, that cleanup could use the stale read-only live set and delete files still needed by the writer.

This change records whether a `DBImpl` was opened read-only and skips close-time obsolete-file cleanup for read-only DBs instead of simply keeping `opened_successfully_=false`, which could be confusing and easily mistaken in the future again.

For completeness I also updated other callsites of `opened_successfully_`, but those should not be real bugs as ReadOnly DBs do not run flushes/compactions or write to WALs.

Reviewed By: xingbowang

Differential Revision: D109622445

fbshipit-source-id: d7be4b20fce86ccb218a63ac6f5b707b316aac79

CLAUDE.md

@@ -217,6 +217,41 @@ The following patterns emerged as frequent sources of review feedback:
 * Use `make dbg` command to build all of the unit test in debug mode.
 * For -j in make command, use the number of CPU cores to decide it.
 
+### When to run `make clean` (avoid mixing build modes)
+
+The Makefile does **not** track build mode, so object files from a prior
+build are silently reused even when compiled with different flags, leading
+to confusing linker errors, sanitizer false negatives, ODR violations, or
+"phantom" bugs.
+
+Run `make clean` before switching any of these:
+* **`ASSERT_STATUS_CHECKED=1` ↔ unset** — changes the `Status` class layout (ABI break).
+* **Sanitizer builds** — toggling any of `COMPILE_WITH_ASAN=1`,
+    `COMPILE_WITH_UBSAN=1`, `COMPILE_WITH_TSAN=1` on/off.
+* `DEBUG_LEVEL=0` (release) ↔ `DEBUG_LEVEL=1` (debug, default for `make dbg`).
+* Different compilers, `OPT` levels, or other flags affecting codegen/ABI.
+
+**Notable exception:** `DEBUG_LEVEL=2` can be safely mixed with
+`DEBUG_LEVEL=1` — rebuild a subset of files with `DEBUG_LEVEL=2` to get
+extra/more accurate runtime checks for those files without a full clean.
+
+When in doubt, `make clean` is cheap insurance compared to chasing a
+phantom bug.
+
+### Source checks
+* Run `make check-sources` before committing. This catches non-ASCII
+    characters in source files and other source-level issues that CI will
+    reject. In particular, **do not use Unicode characters** (em dashes,
+    smart quotes, etc.) in comments or strings -- use ASCII equivalents
+    (`--` instead of em dash, `'` instead of smart quote, etc.).
+
+### RTTI and dynamic_cast
+* Production code and `db_stress` must build in **release mode
+    (`-fno-rtti`)**. Do not use `dynamic_cast` anywhere except unit tests.
+    Use `static_cast_with_check` from `util/cast_util.h` (validates with
+    `dynamic_cast` in debug builds, plain `static_cast` in release).
+* Unit tests (`*_test.cc`) are built in debug mode with RTTI enabled.
+
 ### Unit Test
 * After all of the unit tests are added, review them and try to extract common
     reusable utility functions to reduce code duplication due to copy past between

HISTORY.md

@@ -1,6 +1,35 @@
 # Rocksdb Change Log
 > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
 
+## 11.3.2 (06/24/2026)
+### Bug Fixes
+* Fixed a bug where closing a read-only DB instance could delete live SST files created by a concurrent read-write DB sharing the same directory.
+
+## 11.3.1 (06/04/2026)
+* Don't call `EventListener::OnCompactionPreCommit` during DB shutdown, like other compaction callbacks
+
+## 11.3.0 (05/15/2026)
+### New Features
+* Add experimental DB option `async_wal_precreate` to precreate the next WAL file in a background thread and reduce foreground WAL rotation latency. The option is sanitized to false when WAL recycling is enabled.
+* Added a new `EventListener::OnCompactionPreCommit` callback that fires after a compaction job finishes but before its input files are released (i.e. while `FileMetaData::being_compacted` is still true). Listeners that maintain bookkeeping of which files are currently being compacted can clean up such state in this new callback to avoid races with concurrent compaction picking, where another thread might pick up the same files for a new compaction immediately after `being_compacted` is flipped back to false but before `OnCompactionCompleted` fires. The default implementation is a no-op so this is not a breaking change.
+* Add mutable DBOption `optimize_manifest_for_recovery` (default false). When enabled, RocksDB can reduce recovery work after a clean shutdown, which may lower DB::Open latency on warm reopens.
+* Added public utility APIs `ParseCompressionNameForDisplay()` to convert `TableProperties::compression_name` into a human-readable compression name for both legacy and format_version 7+ SST metadata, including custom `CompressionManager`-provided display names for custom compression types.
+* Add `reuse_manifest_on_open` DBOption (default false). When enabled, DB::Open reuses the existing MANIFEST file for append instead of creating a fresh one, avoiding the cost of serializing the entire database state into a new MANIFEST on the first post-open write. To prevent this feature from interfering with manifest file size auto-tuning, an extra forward-compatible field is now always added to the MANIFEST (to track the last "compacted" size).
+
+### Behavior Changes
+* Read-only open with `error_if_wal_file_exists=true` now tolerates empty WAL files so empty precreated WALs do not prevent inspection.
+* WriteCommitted TransactionDB now matches WritePrepared and WriteUnprepared compaction filtering in both single- and two-write-queue modes: a compaction filter's FilterMergeOperand will not be invoked on merge operands at or below the latest published sequence number.
+
+### Bug Fixes
+* Fixed blob-backed wide-column merge reads to preserve correct status
+propagation and resolution across memtable, read-only, and secondary DB
+paths.
+* Fixed a bug where `DB::GetCreationTimeOfOldestFile()` could return inaccurate results instead of the real creation time when called shortly after opening a legacy DB (one whose manifest lacks `file_creation_time`) with `open_files_async = true`. The API now waits for background SST file loading to complete only when needed; modern DBs are unaffected.
+* Fixed merge reads against wide-column/blob-backed base values to preserve precise failure statuses, including `GetMergeOperands()` and direct-write memtable reads.
+* Fix bug in range tombstone synthesis that covers live keys added during an IngestExternalFile
+* Reject the empty string as a column family name in `DB::CreateColumnFamily` / `DB::CreateColumnFamilies`. Previously such calls returned OK and a usable handle, but the column family was not persisted in the manifest, so any data written to it was silently lost on DB reopen.
+* Fixed a bug where a WriteCommitted TransactionDB using commit-bypass WBWI ingestion could drop an entry that is still visible at the published sequence boundary.
+
 ## 11.2.0 (04/18/2026)
 ### New Features
 * Added experimental `DBOptions::fast_sst_open` option. When enabled, RocksDB retrieves opaque file system metadata for SST files after flush, compaction, and external file ingestion, persists it in the MANIFEST, and passes it back to the file system on subsequent file opens to accelerate DB open time.

Makefile

@@ -327,6 +327,9 @@ missing_make_config_paths := $(shell				\
 $(foreach path, $(missing_make_config_paths), \
 	$(warning Warning: $(path) does not exist))
 
+# This (the first rule) must depend on "all".
+default: all
+
 ifeq ($(PLATFORM), OS_AIX)
 # no debug info
 else ifneq ($(PLATFORM), IOS)
@@ -485,9 +488,6 @@ ifdef ROCKSDB_MODIFY_NPHASH
   PLATFORM_CXXFLAGS += -DROCKSDB_MODIFY_NPHASH=1
 endif
 
-# This (the first rule) must depend on "all".
-default: all
-
 WARNING_FLAGS = -W -Wextra -Wall -Wsign-compare -Wshadow \
   -Wunused-parameter
 
@@ -990,7 +990,7 @@ check_0:
 		        awk -v s=$(CI_SHARD_INDEX) -v n=$(CI_TOTAL_SHARDS) '(NR-1)%n==s'; \
 		      else cat; fi; \
 		fi; \
-		find t -name 'run-*' -print; \
+		$(FIND) t -name 'run-*' -print; \
 	} \
 	  | $(prioritize_long_running_tests)				\
 	  | grep -E '$(tests-regexp)'					\
@@ -1012,7 +1012,7 @@ valgrind_check_0:
 	  '  run "make watch-log" in a separate window' '';		\
 	{								\
 	  printf './%s\n' $(filter-out $(PARALLEL_TEST) %skiplist_test options_settable_test, $(TESTS));		\
-	  find t -name 'run-*' -print; \
+	  $(FIND) t -name 'run-*' -print; \
 	}								\
 	  | $(prioritize_long_running_tests)				\
 	  | grep -E '$(tests-regexp)'					\
@@ -1230,11 +1230,17 @@ clean-not-downloaded: clean-ext-libraries-bin clean-rocks clean-not-downloaded-r
 
 clean-rocks:
 # Not practical to exactly match all versions/variants in naming (e.g. debug or not)
-	rm -f ${LIBNAME}*.so* ${LIBNAME}*.a
-	rm -f $(BENCHMARKS) $(TOOLS) $(TESTS) $(PARALLEL_TEST) $(MICROBENCHS)
-	rm -rf $(CLEAN_FILES) ios-x86 ios-arm scan_build_report
-	$(FIND) . -name "*.[oda]" -exec rm -f {} \;
-	$(FIND) . -type f \( -name "*.gcda" -o -name "*.gcno" \) -exec rm -f {} \;
+	@echo Removing link targets
+	@rm -f ${LIBNAME}*.so* ${LIBNAME}*.a $(BENCHMARKS) $(TOOLS) $(TESTS) $(PARALLEL_TEST) $(MICROBENCHS)
+	@echo Finding and cleaning other files
+	@rm -rf $(CLEAN_FILES) ios-x86 ios-arm scan_build_report
+	@if $(FIND) Makefile -regextype awk &> /dev/null; then \
+		$(FIND) . -regextype awk \
+			-regex '.*/([.].*|third-party)' -prune -o \
+			-type f -regex '.*[.]([oda]|gcda|gcno)' -exec rm -f {} \; ; \
+	else \
+		$(FIND) . -name "*.[oda]" -exec rm -f {} \; ; \
+	fi
 
 clean-rocksjava: clean-rocks
 	rm -rf jl jls
@@ -1247,7 +1253,7 @@ clean-ext-libraries-all:
 	rm -rf bzip2* snappy* zlib* lz4* zstd*
 
 clean-ext-libraries-bin:
-	find . -maxdepth 1 -type d \( -name bzip2\* -or -name snappy\* -or -name zlib\* -or -name lz4\* -or -name zstd\* \) -prune -exec rm -rf {} \;
+	$(FIND) . -maxdepth 1 -type d \( -name bzip2\* -or -name snappy\* -or -name zlib\* -or -name lz4\* -or -name zstd\* \) -prune -exec rm -rf {} \;
 
 tags:
 	ctags -R .

build_tools/getdeps_fallback_mirror.py

@@ -1,3 +1,7 @@
+#  Copyright (c) Meta Platforms, Inc. and affiliates.
+#  This source code is licensed under both the GPLv2 (found in the COPYING file in the root directory)
+#  and the Apache 2.0 License (found in the LICENSE.Apache file in the root directory).
+
 #!/usr/bin/env python3
 """
 Pre-download packages with unreliable mirrors using fallback mirrors.

db/c.cc

@@ -5286,6 +5286,15 @@ size_t rocksdb_options_get_recycle_log_file_num(rocksdb_options_t* opt) {
   return opt->rep.recycle_log_file_num;
 }
 
+void rocksdb_options_set_async_wal_precreate(rocksdb_options_t* opt,
+                                             unsigned char v) {
+  opt->rep.async_wal_precreate = v;
+}
+
+unsigned char rocksdb_options_get_async_wal_precreate(rocksdb_options_t* opt) {
+  return opt->rep.async_wal_precreate;
+}
+
 void rocksdb_options_set_soft_pending_compaction_bytes_limit(
     rocksdb_options_t* opt, size_t v) {
   opt->rep.soft_pending_compaction_bytes_limit = v;

db/c_test.c

@@ -2869,6 +2869,9 @@ int main(int argc, char** argv) {
     rocksdb_options_set_track_and_verify_wals_in_manifest(o, 42);
     CheckCondition(1 ==
                    rocksdb_options_get_track_and_verify_wals_in_manifest(o));
+    CheckCondition(0 == rocksdb_options_get_async_wal_precreate(o));
+    rocksdb_options_set_async_wal_precreate(o, 1);
+    CheckCondition(1 == rocksdb_options_get_async_wal_precreate(o));
 
     /* Blob Options */
     rocksdb_options_set_enable_blob_files(o, 1);

db/compaction/compaction.h

@@ -454,6 +454,18 @@ class Compaction {
     return notify_on_compaction_completion_;
   }
 
+  // Called by DBImpl::NotifyOnCompactionPreCommit to ensure the
+  // OnCompactionPreCommit listener callback fires at most once per
+  // compaction lifecycle, even though the notify helper is invoked at multiple
+  // potential ReleaseCompactionFiles() sites for safety.
+  void SetNotifyOnCompactionPreCommitCalled() {
+    notify_on_compaction_pre_commit_called_ = true;
+  }
+
+  bool WasNotifyOnCompactionPreCommitCalled() const {
+    return notify_on_compaction_pre_commit_called_;
+  }
+
   static constexpr int kInvalidLevel = -1;
 
   // Evaluate proximal output level. If the compaction supports
@@ -626,6 +638,12 @@ class Compaction {
   // begin.
   bool notify_on_compaction_completion_;
 
+  // Tracks whether OnCompactionPreCommit has already fired for this
+  // compaction. The notify helper is called from multiple potential
+  // ReleaseCompactionFiles() call sites; this flag guarantees that listeners
+  // observe the callback exactly once per compaction.
+  bool notify_on_compaction_pre_commit_called_ = false;
+
   // Enable/disable GC collection for blobs during compaction.
   bool enable_blob_garbage_collection_;
 

db/db_etc3_test.cc

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License
@@ -145,6 +150,61 @@ TEST_F(DBEtc3Test, AutoTuneManifestSize) {
   AddCfFn();
   ASSERT_LT(prev_manifest_num, cur_manifest_num);
 
+  // ---- Verify persisted compacted manifest size survives close/reopen ----
+  // Close with CFs still live. Reopen with reuse_manifest_on_open so the
+  // manifest is NOT rewritten from scratch. The persisted compacted size
+  // should be loaded and used for auto-tuning.
+  // At this point we have 7 CF handles plus default.
+  ASSERT_EQ(handles.size(), 7U);
+
+  // Collect CF names for reopen, then release handles (Close needs this)
+  std::vector<std::string> cf_names = {"default"};
+  for (auto* h : handles) {
+    cf_names.push_back(h->GetName());
+  }
+  for (auto* h : handles) {
+    ASSERT_OK(db_->DestroyColumnFamilyHandle(h));
+  }
+  handles.clear();
+
+  Close();
+  // Use a large max_manifest_file_size so the reused manifest (which is
+  // already ~10KB) does NOT trigger rotation on the first few writes.
+  // Auto-tuning with the persisted compacted size (~5KB) at 200% amp
+  // gives a tuned threshold of ~15KB. Without persistence, the threshold
+  // would be max(max_manifest_file_size, 0 * anything) =
+  // max_manifest_file_size.
+  //
+  // We set max_manifest_file_size to two values to distinguish:
+  // - 3000: if persisted compacted size is NOT loaded, tuned = 3000,
+  //   and the first AddCf will rotate (manifest is already ~10KB > 3000)
+  // - With persisted compacted size loaded, tuned = max(3000, 5000*3) = 15000,
+  //   so no rotation until we exceed 15KB
+  options.max_manifest_file_size = 3000;
+  options.max_manifest_space_amp_pct = 200;
+  options.reuse_manifest_on_open = true;
+  uint64_t manifest_num_before_reopen = cur_manifest_num;
+  ReopenWithColumnFamilies(cf_names, options);
+  cur_manifest_num = dbfull()->TEST_Current_Manifest_FileNo();
+
+  // With persistence: the manifest file number should NOT have changed
+  // during reopen, because the persisted compacted size keeps the tuned
+  // threshold high enough. Without persistence, last_compacted = 0, so
+  // tuned = max_manifest_file_size = 3000, and the first LogAndApply
+  // during Open rotates the manifest because it's already ~10KB > 3000.
+  ASSERT_EQ(manifest_num_before_reopen, cur_manifest_num);
+
+  // Adding CFs should still not trigger rotation because the tuned
+  // threshold (~15KB) exceeds the current manifest size (~10KB + adds).
+  for (int i = 1; i <= 4; ++i) {
+    AddCfFn();
+    ASSERT_EQ(prev_manifest_num, cur_manifest_num);
+  }
+  for (int i = 1; i <= 4; ++i) {
+    AddCfFn();
+    ASSERT_EQ(prev_manifest_num, cur_manifest_num);
+  }
+
   // Wrap up
   while (!handles.empty()) {
     DropCfFn();

db/db_impl/db_impl.cc

@@ -175,6 +175,7 @@ DBImpl::DBImpl(const DBOptions& options, const std::string& dbname,
                const bool seq_per_batch, const bool batch_per_txn,
                bool read_only)
     : dbname_(dbname),
+      read_only_(read_only),
       own_info_log_(options.info_log == nullptr),
       initial_db_options_(SanitizeOptions(dbname, options, read_only,
                                           &init_logger_creation_s_)),
@@ -653,7 +654,8 @@ void DBImpl::UnregisterBlobDirectWriteColumnFamily() {
 Status DBImpl::MaybeWriteWalMarkersToManifestOnClose() {
   mutex_.AssertHeld();
   if (!mutable_db_options_.optimize_manifest_for_recovery ||
-      !opened_successfully_ || versions_ == nullptr || logs_.empty()) {
+      !opened_successfully_ || read_only_ || versions_ == nullptr ||
+      logs_.empty()) {
     return Status::OK();
   }
 
@@ -789,12 +791,27 @@ Status DBImpl::CloseHelper() {
          bg_flush_scheduled_ || bg_purge_scheduled_ ||
          bg_pressure_callback_in_progress_ ||
          bg_async_file_open_state_ == AsyncFileOpenState::kScheduled ||
+         async_wal_precreate_state_ == AsyncWALPrecreateState::kScheduled ||
          pending_purge_obsolete_files_ ||
          error_handler_.IsRecoveryInProgress()) {
     TEST_SYNC_POINT("DBImpl::~DBImpl:WaitJob");
     bg_cv_.Wait();
   }
 
+  // Release any opened-but-unpublished WAL writer after the in-flight worker
+  // has published its result. Clear the DB-owned async slot while holding
+  // mutex_, but destroy the detached writer after dropping mutex_ because
+  // log::Writer / WritableFileWriter destruction can flush and close the file.
+  // The file itself can be left behind as an empty future WAL; recovery already
+  // tolerates it and marks its file number used if observed.
+  UnpublishedWAL unused_async_wal = std::move(async_wal_precreate_wal_);
+  async_wal_precreate_state_ = AsyncWALPrecreateState::kNotScheduled;
+  if (unused_async_wal.writer) {
+    mutex_.Unlock();
+    unused_async_wal.Reset();
+    mutex_.Lock();
+  }
+
   // Ensure subclasses don't forget to schedule async file opening
   assert(!immutable_db_options_.open_files_async || !opened_successfully_ ||
          bg_async_file_open_state_ != AsyncFileOpenState::kNotScheduled);
@@ -840,7 +857,7 @@ Status DBImpl::CloseHelper() {
   // manifest file), it is not able to identify live files correctly. As a
   // result, all "live" files can get deleted by accident. However, corrupted
   // manifest is recoverable by RepairDB().
-  if (opened_successfully_) {
+  if (opened_successfully_ && !read_only_) {
     JobContext job_context(next_job_id_.fetch_add(1));
     FindObsoleteFiles(&job_context, true);
 

db/db_impl/db_impl.h

@@ -15,6 +15,8 @@
 #include <limits>
 #include <list>
 #include <map>
+#include <memory>
+#include <optional>
 #include <set>
 #include <string>
 #include <unordered_map>
@@ -1372,6 +1374,7 @@ class DBImpl : public DB {
 
  protected:
   const std::string dbname_;
+  const bool read_only_;
   // TODO(peterd): unify with VersionSet::db_id_
   std::string db_id_;
   // db_session_id_ is an identifier that gets reset
@@ -1527,6 +1530,15 @@ class DBImpl : public DB {
                                    const Status& st,
                                    const CompactionJobStats& job_stats,
                                    int job_id);
+  // Fires the OnCompactionPreCommit listener callback. Mirrors
+  // NotifyOnCompactionCompleted but is invoked before ReleaseCompactionFiles()
+  // (i.e. while input files still have being_compacted == true). Idempotent:
+  // safe to call from multiple potential release sites; only fires once per
+  // compaction, and only if NotifyOnCompactionBegin previously fired.
+  void NotifyOnCompactionPreCommit(ColumnFamilyData* cfd, Compaction* c,
+                                   const Status& st,
+                                   const CompactionJobStats& job_stats,
+                                   int job_id);
   void NotifyOnMemTableSealed(ColumnFamilyData* cfd,
                               const MemTableInfo& mem_table_info);
 
@@ -2574,10 +2586,15 @@ class DBImpl : public DB {
   // Helper function to perform trivial move by updating manifest metadata
   // without rewriting data files. This is called when IsTrivialMove() is true.
   // REQUIRES: mutex held
-  // Returns: Status of the trivial move operation
-  Status PerformTrivialMove(Compaction& c, LogBuffer* log_buffer,
-                            bool& compaction_released, size_t& moved_files,
-                            size_t& moved_bytes);
+  // Populates the Compaction's edit with DeleteFile/AddFile entries for
+  // a trivial move (no data rewriting). Does not commit the edit.
+  void PrepareTrivialMoveEdit(Compaction& c, LogBuffer* log_buffer,
+                              size_t& moved_files, size_t& moved_bytes);
+
+  // REQUIRES: mutex held, PrepareTrivialMoveEdit already called
+  // Commits the trivial move by calling LogAndApply on the prepared edit.
+  // Returns: Status of the manifest write
+  Status CommitTrivialMove(Compaction& c, bool& compaction_released);
 
   // REQUIRES: mutex unlocked
   void TrackOrUntrackFiles(const std::vector<std::string>& existing_data_files,
@@ -2810,7 +2827,68 @@ class DBImpl : public DB {
   size_t GetWalPreallocateBlockSize(uint64_t write_buffer_size) const;
   Env::WriteLifeTimeHint CalculateWALWriteHint() { return Env::WLTH_SHORT; }
 
-  IOStatus CreateWAL(const WriteOptions& write_options, uint64_t log_file_num,
+  // Returns true when async WAL precreation is enabled and compatible with the
+  // active WAL strategy. WAL recycling already avoids file creation latency, so
+  // precreation is disabled when recycle_log_file_num is non-zero.
+  bool AsyncWALPrecreateEnabled() const;
+
+  // A WAL file that has a reserved file number and may have an opened writer,
+  // but has not been added to DBImpl's in-memory logical WAL tracking lists
+  // (logs_ and alive_wal_files_).
+  struct UnpublishedWAL {
+    uint64_t log_number = 0;
+    std::unique_ptr<log::Writer> writer;
+
+    UnpublishedWAL() = default;
+    UnpublishedWAL(const UnpublishedWAL&) = delete;
+    UnpublishedWAL& operator=(const UnpublishedWAL&) = delete;
+
+    UnpublishedWAL(UnpublishedWAL&& other) noexcept {
+      *this = std::move(other);
+    }
+    UnpublishedWAL& operator=(UnpublishedWAL&& other) noexcept {
+      if (this != &other) {
+        log_number = other.log_number;
+        writer = std::move(other.writer);
+        other.Reset();
+      }
+      return *this;
+    }
+
+    void Reset() {
+      log_number = 0;
+      writer.reset();
+    }
+  };
+
+  // Reserves the next WAL file number and schedules a HIGH-priority background
+  // task to precreate that WAL file. A precreated WAL is not a logical WAL
+  // until a foreground WAL rotation consumes it.
+  void MaybeScheduleAsyncWALPrecreate(size_t preallocate_block_size);
+
+  // Background task for opening the reserved future WAL and publishing the
+  // result under mutex_.
+  static void BGWorkAsyncWALPrecreate(void* arg);
+
+  // Waits for an in-flight async WAL precreation and returns a prepared WAL if
+  // one is available. If precreation failed, returns an empty WAL and lets the
+  // foreground rotation create the WAL synchronously. Caller must hold mutex_.
+  UnpublishedWAL WaitForAsyncWALPrecreate();
+
+  // Opens and preallocates a WAL writer without writing logical WAL records.
+  // Used by async WAL precreation and by synchronous WAL creation.
+  IOStatus CreateWALWriter(const DBOptions& db_options, uint64_t log_file_num,
+                           uint64_t recycle_log_number,
+                           size_t preallocate_block_size,
+                           UnpublishedWAL* new_wal);
+
+  // Starts an opened WAL file by writing the initial records required before it
+  // can be installed as the current WAL for foreground writes.
+  IOStatus StartWALFile(const WriteOptions& write_options,
+                        const PredecessorWALInfo& predecessor_wal_info,
+                        log::Writer* new_log);
+  IOStatus CreateWAL(const DBOptions& db_options,
+                     const WriteOptions& write_options, uint64_t log_file_num,
                      uint64_t recycle_log_number, size_t preallocate_block_size,
                      const PredecessorWALInfo& predecessor_wal_info,
                      log::Writer** new_log);
@@ -3306,6 +3384,28 @@ class DBImpl : public DB {
   AsyncFileOpenState bg_async_file_open_state_ =
       AsyncFileOpenState::kNotScheduled;
 
+  // State machine for the single async WAL precreation slot protected by
+  // mutex_. Background precreation failure returns to kNotScheduled; foreground
+  // rotation handles it the same as no prepared WAL and creates one
+  // synchronously. kScheduled owns a reserved file number; kReady owns an
+  // opened writer that has not been started or added to logical WAL tracking.
+  enum class AsyncWALPrecreateState : uint8_t {
+    kNotScheduled = 0,  // No WAL precreate work is in-flight or ready.
+    kScheduled,         // Background task owns creation of the reserved WAL.
+    kReady,             // Reserved WAL writer is open but not logically live.
+  };
+
+  // Protected by mutex_. Tracks at most one background precreated WAL. A
+  // precreated WAL is only reserved empty storage until SwitchMemtable()
+  // consumes it and installs it in DBImpl's in-memory logical WAL tracking
+  // lists (logs_ and alive_wal_files_).
+  AsyncWALPrecreateState async_wal_precreate_state_ =
+      AsyncWALPrecreateState::kNotScheduled;
+
+  // Reserved in-flight/ready precreated WAL. The writer is populated only while
+  // state is kReady.
+  UnpublishedWAL async_wal_precreate_wal_;
+
   std::deque<ManualCompactionState*> manual_compaction_dequeue_;
 
   // shall we disable deletion of obsolete files

db/db_impl/db_impl_compaction_flush.cc

@@ -1581,9 +1581,8 @@ Status DBImpl::CompactFiles(const CompactionOptions& compact_options,
   return s;
 }
 
-Status DBImpl::PerformTrivialMove(Compaction& c, LogBuffer* log_buffer,
-                                  bool& compaction_released,
-                                  size_t& moved_files, size_t& moved_bytes) {
+void DBImpl::PrepareTrivialMoveEdit(Compaction& c, LogBuffer* log_buffer,
+                                    size_t& moved_files, size_t& moved_bytes) {
   mutex_.AssertHeld();
 
   ROCKS_LOG_BUFFER(log_buffer, "[%s] Moving %d files to level-%d\n",
@@ -1616,6 +1615,10 @@ Status DBImpl::PerformTrivialMove(Compaction& c, LogBuffer* log_buffer,
     }
     moved_files += c.num_input_files(l);
   }
+}
+
+Status DBImpl::CommitTrivialMove(Compaction& c, bool& compaction_released) {
+  mutex_.AssertHeld();
 
   // Install the new version
   const ReadOptions read_options(Env::IOActivity::kCompaction);
@@ -1740,8 +1743,8 @@ Status DBImpl::CompactFilesImpl(
     bool compaction_released = false;
     size_t moved_files = 0;
     size_t moved_bytes = 0;
-    Status status = PerformTrivialMove(
-        *c.get(), log_buffer, compaction_released, moved_files, moved_bytes);
+    PrepareTrivialMoveEdit(*c.get(), log_buffer, moved_files, moved_bytes);
+    Status status = CommitTrivialMove(*c.get(), compaction_released);
 
     if (status.ok()) {
       InstallSuperVersionAndScheduleWork(
@@ -2009,6 +2012,46 @@ void DBImpl::NotifyOnCompactionCompleted(
   // flush process.
 }
 
+void DBImpl::NotifyOnCompactionPreCommit(
+    ColumnFamilyData* cfd, Compaction* c, const Status& st,
+    const CompactionJobStats& compaction_job_stats, const int job_id) {
+  if (immutable_db_options_.listeners.size() == 0U) {
+    return;
+  }
+  mutex_.AssertHeld();
+  if (shutting_down_.load(std::memory_order_acquire)) {
+    return;
+  }
+
+  // Only fire if OnCompactionBegin has fired for this compaction.
+  if (c->ShouldNotifyOnCompactionCompleted() == false) {
+    return;
+  }
+  // Idempotency: ensure listeners observe at most one
+  // OnCompactionPreCommit per compaction lifecycle, even though the
+  // notify helper is invoked at multiple potential ReleaseCompactionFiles()
+  // sites for safety.
+  if (c->WasNotifyOnCompactionPreCommitCalled()) {
+    return;
+  }
+  c->SetNotifyOnCompactionPreCommitCalled();
+
+  int num_l0_files = cfd->current()->storage_info()->NumLevelFiles(0);
+  // release lock while notifying events
+  mutex_.Unlock();
+  TEST_SYNC_POINT("DBImpl::NotifyOnCompactionPreCommit::UnlockMutex");
+  {
+    CompactionJobInfo info{};
+    info.num_l0_files = num_l0_files;
+    BuildCompactionJobInfo(cfd, c, st, compaction_job_stats, job_id, &info);
+    for (const auto& listener : immutable_db_options_.listeners) {
+      listener->OnCompactionPreCommit(this, info);
+    }
+    info.status.PermitUncheckedError();
+  }
+  mutex_.Lock();
+}
+
 // REQUIREMENT: block all background work by calling PauseBackgroundWork()
 // before calling this function
 // TODO (hx235): Replace Status::NotSupported() with Status::Aborted() for
@@ -3180,7 +3223,7 @@ void DBImpl::ResumeAllCompactions() {
 void DBImpl::MaybeScheduleFlushOrCompaction() {
   mutex_.AssertHeld();
   TEST_SYNC_POINT("DBImpl::MaybeScheduleFlushOrCompaction:Start");
-  if (!opened_successfully_) {
+  if (!opened_successfully_ || read_only_) {
     // Compaction may introduce data race to DB open
     return;
   }
@@ -4255,6 +4298,8 @@ Status DBImpl::BackgroundCompaction(bool* made_progress,
     for (const auto& f : *c->inputs(0)) {
       c->edit()->DeleteFile(c->level(), f->fd.GetNumber());
     }
+    NotifyOnCompactionPreCommit(c->column_family_data(), c.get(), status,
+                                compaction_job_stats, job_context->job_id);
     status = versions_->LogAndApply(
         c->column_family_data(), read_options, write_options, c->edit(),
         &mutex_, directories_.GetDbDir(),
@@ -4473,6 +4518,8 @@ Status DBImpl::BackgroundCompaction(bool* made_progress,
         ++out_file_metadata_it;
       }
 
+      NotifyOnCompactionPreCommit(c->column_family_data(), c.get(), status,
+                                  compaction_job_stats, job_context->job_id);
       status = versions_->LogAndApply(
           c->column_family_data(), read_options, write_options, c->edit(),
           &mutex_, directories_.GetDbDir(),
@@ -4549,8 +4596,12 @@ Status DBImpl::BackgroundCompaction(bool* made_progress,
     // Perform the trivial move
     size_t moved_files = 0;
     size_t moved_bytes = 0;
-    status = PerformTrivialMove(*c.get(), log_buffer, compaction_released,
-                                moved_files, moved_bytes);
+    // Populate the edit first so that CompactionJobInfo has output file
+    // info when OnCompactionPreCommit fires.
+    PrepareTrivialMoveEdit(*c.get(), log_buffer, moved_files, moved_bytes);
+    NotifyOnCompactionPreCommit(c->column_family_data(), c.get(), status,
+                                compaction_job_stats, job_context->job_id);
+    status = CommitTrivialMove(*c.get(), compaction_released);
     io_s = versions_->io_status();
     InstallSuperVersionAndScheduleWork(
         c->column_family_data(), job_context->superversion_contexts.data());
@@ -4673,6 +4724,10 @@ Status DBImpl::BackgroundCompaction(bool* made_progress,
       ReleaseOptionsFileNumber(min_options_file_number_elem);
     }
 
+    // Fire OnCompactionPreCommit before compaction_job.Install runs
+    // ReleaseCompactionFiles in its manifest write callback.
+    NotifyOnCompactionPreCommit(c->column_family_data(), c.get(), status,
+                                compaction_job_stats, job_context->job_id);
     status = compaction_job.Install(&compaction_released);
     io_s = compaction_job.io_status();
     if (status.ok()) {
@@ -4692,6 +4747,14 @@ Status DBImpl::BackgroundCompaction(bool* made_progress,
 
   if (c != nullptr) {
     if (!compaction_released) {
+      // Safety net: if any of the paths above did not fire the
+      // OnCompactionPreCommit notification (e.g. status was not OK and
+      // an early-out skipped the LogAndApply / Install), still fire the
+      // notify here while files are about to be released. The notify is
+      // idempotent and is a no-op if it has already fired (or if Begin never
+      // fired).
+      NotifyOnCompactionPreCommit(c->column_family_data(), c.get(), status,
+                                  compaction_job_stats, job_context->job_id);
       c->ReleaseCompactionFiles(status);
     } else {
 #ifndef NDEBUG

db/db_impl/db_impl_open.cc

@@ -21,6 +21,7 @@
 #include "file/writable_file_writer.h"
 #include "logging/logging.h"
 #include "monitoring/persistent_stats_history.h"
+#include "monitoring/statistics_impl.h"
 #include "monitoring/thread_status_util.h"
 #include "options/options_helper.h"
 #include "rocksdb/options.h"
@@ -122,6 +123,16 @@ DBOptions SanitizeOptions(const std::string& dbname, const DBOptions& src,
     result.recycle_log_file_num = 0;
   }
 
+  if (result.async_wal_precreate && result.recycle_log_file_num != 0) {
+    // Async WAL precreation reserves a future log number, while WAL recycling
+    // chooses from old WAL files. Keep the recycling behavior and disable only
+    // the async optimization.
+    result.async_wal_precreate = false;
+    ROCKS_LOG_WARN(result.info_log,
+                   "async_wal_precreate is disabled since "
+                   "recycle_log_file_num is non-zero");
+  }
+
   if (result.db_paths.size() == 0) {
     result.db_paths.emplace_back(dbname, std::numeric_limits<uint64_t>::max());
   } else if (result.wal_dir.empty()) {
@@ -799,9 +810,19 @@ Status DBImpl::Recover(
 
     if (!wal_files.empty()) {
       if (error_if_wal_file_exists) {
-        return Status::Corruption(
-            "The db was opened in readonly mode with error_if_wal_file_exists"
-            "flag but a WAL file already exists");
+        for (const auto& wal_file : wal_files) {
+          uint64_t bytes;
+          s = env_->GetFileSize(wal_file.second, &bytes);
+          if (!s.ok()) {
+            return s;
+          }
+          if (bytes > 0) {
+            return Status::Corruption(
+                "The db was opened in readonly mode with "
+                "error_if_wal_file_exists flag but a non-empty WAL file "
+                "already exists");
+          }
+        }
       } else if (error_if_data_exists_in_wals) {
         for (auto& wal_file : wal_files) {
           uint64_t bytes;
@@ -2377,16 +2398,17 @@ Status DB::OpenAndTrimHistory(
   return s;
 }
 
-IOStatus DBImpl::CreateWAL(const WriteOptions& write_options,
-                           uint64_t log_file_num, uint64_t recycle_log_number,
-                           size_t preallocate_block_size,
-                           const PredecessorWALInfo& predecessor_wal_info,
-                           log::Writer** new_log) {
+IOStatus DBImpl::CreateWALWriter(const DBOptions& db_options,
+                                 uint64_t log_file_num,
+                                 uint64_t recycle_log_number,
+                                 size_t preallocate_block_size,
+                                 UnpublishedWAL* new_wal) {
+  assert(new_wal);
+  assert(new_wal->log_number == 0);
+  assert(new_wal->writer == nullptr);
   IOStatus io_s;
   std::unique_ptr<FSWritableFile> lfile;
 
-  DBOptions db_options =
-      BuildDBOptions(immutable_db_options_, mutable_db_options_);
   FileOptions opt_file_options =
       fs_->OptimizeForLogWrite(file_options_, db_options);
   opt_file_options.write_hint = CalculateWALWriteHint();
@@ -2424,21 +2446,162 @@ IOStatus DBImpl::CreateWAL(const WriteOptions& write_options,
         Histograms::HISTOGRAM_ENUM_MAX /* hist_type */, listeners, nullptr,
         tmp_set.Contains(FileType::kWalFile),
         tmp_set.Contains(FileType::kWalFile)));
-    *new_log = new log::Writer(std::move(file_writer), log_file_num,
-                               immutable_db_options_.recycle_log_file_num > 0,
-                               immutable_db_options_.manual_wal_flush,
-                               immutable_db_options_.wal_compression,
-                               immutable_db_options_.track_and_verify_wals);
-    io_s = (*new_log)->AddCompressionTypeRecord(write_options);
-    if (io_s.ok()) {
-      io_s = (*new_log)->MaybeAddPredecessorWALInfo(write_options,
-                                                    predecessor_wal_info);
-    }
+    new_wal->log_number = log_file_num;
+    new_wal->writer = std::make_unique<log::Writer>(
+        std::move(file_writer), log_file_num,
+        immutable_db_options_.recycle_log_file_num > 0,
+        immutable_db_options_.manual_wal_flush,
+        immutable_db_options_.wal_compression,
+        immutable_db_options_.track_and_verify_wals);
   }
 
   return io_s;
 }
 
+IOStatus DBImpl::StartWALFile(const WriteOptions& write_options,
+                              const PredecessorWALInfo& predecessor_wal_info,
+                              log::Writer* new_log) {
+  assert(new_log);
+  IOStatus io_s = new_log->AddCompressionTypeRecord(write_options);
+  TEST_SYNC_POINT_CALLBACK("DBImpl::StartWALFile:AfterCompressionTypeRecord",
+                           &io_s);
+  if (io_s.ok()) {
+    io_s = new_log->MaybeAddPredecessorWALInfo(write_options,
+                                               predecessor_wal_info);
+  }
+  return io_s;
+}
+
+IOStatus DBImpl::CreateWAL(const DBOptions& db_options,
+                           const WriteOptions& write_options,
+                           uint64_t log_file_num, uint64_t recycle_log_number,
+                           size_t preallocate_block_size,
+                           const PredecessorWALInfo& predecessor_wal_info,
+                           log::Writer** new_log) {
+  assert(new_log);
+  assert(*new_log == nullptr);
+  UnpublishedWAL wal;
+  IOStatus io_s = CreateWALWriter(db_options, log_file_num, recycle_log_number,
+                                  preallocate_block_size, &wal);
+  if (io_s.ok()) {
+    io_s = StartWALFile(write_options, predecessor_wal_info, wal.writer.get());
+  }
+  if (io_s.ok()) {
+    *new_log = wal.writer.release();
+  }
+  return io_s;
+}
+
+bool DBImpl::AsyncWALPrecreateEnabled() const {
+  return immutable_db_options_.async_wal_precreate &&
+         immutable_db_options_.recycle_log_file_num == 0;
+}
+
+struct AsyncWALPrecreateContext {
+  DBImpl* db = nullptr;
+  DBOptions db_options;
+  uint64_t log_number = 0;
+  size_t preallocate_block_size = 0;
+};
+
+void DBImpl::MaybeScheduleAsyncWALPrecreate(size_t preallocate_block_size) {
+  mutex_.AssertHeld();
+
+  if (!AsyncWALPrecreateEnabled() ||
+      shutting_down_.load(std::memory_order_acquire) ||
+      error_handler_.IsBGWorkStopped() ||
+      async_wal_precreate_state_ != AsyncWALPrecreateState::kNotScheduled) {
+    return;
+  }
+
+  async_wal_precreate_wal_.Reset();
+  async_wal_precreate_wal_.log_number = versions_->NewFileNumber();
+  async_wal_precreate_state_ = AsyncWALPrecreateState::kScheduled;
+
+  auto* ctx = new AsyncWALPrecreateContext();
+  ctx->db = this;
+  ctx->db_options = BuildDBOptions(immutable_db_options_, mutable_db_options_);
+  ctx->log_number = async_wal_precreate_wal_.log_number;
+  ctx->preallocate_block_size = preallocate_block_size;
+  env_->Schedule(&DBImpl::BGWorkAsyncWALPrecreate, ctx, Env::Priority::HIGH,
+                 nullptr);
+}
+
+DBImpl::UnpublishedWAL DBImpl::WaitForAsyncWALPrecreate() {
+  mutex_.AssertHeld();
+  UnpublishedWAL result;
+
+  if (!AsyncWALPrecreateEnabled()) {
+    return result;
+  }
+
+  TEST_SYNC_POINT("DBImpl::WaitForAsyncWALPrecreate:Begin");
+  bool waited = false;
+  uint64_t wait_start_micros = 0;
+  while (async_wal_precreate_state_ == AsyncWALPrecreateState::kScheduled) {
+    TEST_SYNC_POINT("DBImpl::WaitForAsyncWALPrecreate:BeforeWait");
+    if (!waited) {
+      waited = true;
+      wait_start_micros = immutable_db_options_.clock->NowMicros();
+      RecordTick(stats_, WAL_PRECREATE_WAITED);
+    }
+    bg_cv_.Wait();
+  }
+  if (waited) {
+    RecordTick(stats_, WAL_PRECREATE_WAIT_MICROS,
+               immutable_db_options_.clock->NowMicros() - wait_start_micros);
+  }
+
+  if (async_wal_precreate_state_ == AsyncWALPrecreateState::kReady) {
+    result = std::move(async_wal_precreate_wal_);
+    async_wal_precreate_state_ = AsyncWALPrecreateState::kNotScheduled;
+    RecordTick(stats_, WAL_PRECREATE_HIT);
+    return result;
+  }
+
+  assert(async_wal_precreate_state_ == AsyncWALPrecreateState::kNotScheduled);
+  RecordTick(stats_, WAL_PRECREATE_MISS);
+  return result;
+}
+
+void DBImpl::BGWorkAsyncWALPrecreate(void* arg) {
+  TEST_SYNC_POINT("DBImpl::BGWorkAsyncWALPrecreate:Start");
+  std::unique_ptr<AsyncWALPrecreateContext> ctx(
+      static_cast<AsyncWALPrecreateContext*>(arg));
+  DBImpl* db = ctx->db;
+
+  UnpublishedWAL new_wal;
+  IOStatus io_s = db->CreateWALWriter(ctx->db_options, ctx->log_number,
+                                      0 /* recycle_log_number */,
+                                      ctx->preallocate_block_size, &new_wal);
+  if (!io_s.ok()) {
+    RecordTick(db->stats_, WAL_PRECREATE_FAILED);
+    ROCKS_LOG_WARN(db->immutable_db_options_.info_log,
+                   "Async WAL precreate failed for #%" PRIu64 ": %s",
+                   ctx->log_number, io_s.ToString().c_str());
+  }
+
+  TEST_SYNC_POINT("DBImpl::BGWorkAsyncWALPrecreate:BeforePublish");
+  TEST_SYNC_POINT("DBImpl::BGWorkAsyncWALPrecreate:Publish");
+  {
+    InstrumentedMutexLock l(&db->mutex_);
+    if (db->async_wal_precreate_state_ == AsyncWALPrecreateState::kScheduled &&
+        db->async_wal_precreate_wal_.log_number == ctx->log_number) {
+      if (io_s.ok()) {
+        db->async_wal_precreate_wal_ = std::move(new_wal);
+        db->async_wal_precreate_state_ = AsyncWALPrecreateState::kReady;
+      } else {
+        // Async WAL precreation is best-effort. Clear the slot so foreground
+        // rotation falls back to normal synchronous WAL creation.
+        db->async_wal_precreate_wal_.Reset();
+        db->async_wal_precreate_state_ = AsyncWALPrecreateState::kNotScheduled;
+      }
+    }
+    db->bg_cv_.SignalAll();
+  }
+  TEST_SYNC_POINT("DBImpl::BGWorkAsyncWALPrecreate:Done");
+}
+
 void DBImpl::TrackExistingDataFiles(
     const std::vector<std::string>& existing_data_files) {
   TrackOrUntrackFiles(existing_data_files, /*track=*/true);
@@ -2534,8 +2697,10 @@ Status DBImpl::Open(const DBOptions& db_options, const std::string& dbname,
     // created during DB open with predecessor WALs from previous DB session due
     // to `avoid_flush_during_recovery == true`. This can protect the last WAL
     // recovered.
-    s = impl->CreateWAL(write_options, new_log_number, 0 /*recycle_log_number*/,
-                        preallocate_block_size,
+    const DBOptions db_options_snapshot =
+        BuildDBOptions(impl->immutable_db_options_, impl->mutable_db_options_);
+    s = impl->CreateWAL(db_options_snapshot, write_options, new_log_number,
+                        0 /*recycle_log_number*/, preallocate_block_size,
                         PredecessorWALInfo() /* predecessor_wal_info */,
                         &new_log);
     if (s.ok()) {
@@ -2740,6 +2905,8 @@ Status DBImpl::Open(const DBOptions& db_options, const std::string& dbname,
     if (impl->immutable_db_options_.open_files_async) {
       impl->ScheduleAsyncFileOpening();
     }
+    impl->MaybeScheduleAsyncWALPrecreate(
+        impl->GetWalPreallocateBlockSize(max_write_buffer_size));
     impl->mutex_.Unlock();
   }
 

db/db_impl/db_impl_write.cc

@@ -16,6 +16,7 @@
 #include "db/error_handler.h"
 #include "db/event_helpers.h"
 #include "db/wide/wide_columns_helper.h"
+#include "file/filename.h"
 #include "logging/logging.h"
 #include "memtable/wbwi_memtable.h"
 #include "monitoring/perf_context_imp.h"
@@ -3107,6 +3108,7 @@ Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context,
   const WriteOptions write_options;
 
   log::Writer* new_log = nullptr;
+  UnpublishedWAL prepared_wal;
   MemTable* new_mem = nullptr;
   IOStatus io_s;
 
@@ -3134,8 +3136,16 @@ Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context,
       !wal_recycle_files_.empty() && IsFileDeletionsEnabled()) {
     recycle_log_number = wal_recycle_files_.front();
   }
-  uint64_t new_log_number =
-      creating_new_log ? versions_->NewFileNumber() : cur_wal_number_;
+  uint64_t new_log_number = cur_wal_number_;
+  if (creating_new_log) {
+    prepared_wal = WaitForAsyncWALPrecreate();
+    if (prepared_wal.writer) {
+      new_log_number = prepared_wal.log_number;
+    } else {
+      assert(prepared_wal.log_number == 0);
+      new_log_number = versions_->NewFileNumber();
+    }
+  }
   // For use outside of holding DB mutex
   const MutableCFOptions mutable_cf_options_copy =
       cfd->GetLatestMutableCFOptions();
@@ -3156,6 +3166,8 @@ Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context,
   // Log this later after lock release. It may be outdated, e.g., if background
   // flush happens before logging, but that should be ok.
   int num_imm_unflushed = cfd->imm()->NumNotFlushed();
+  const DBOptions db_options_snapshot =
+      BuildDBOptions(immutable_db_options_, mutable_db_options_);
   const auto preallocate_block_size =
       GetWalPreallocateBlockSize(mutable_cf_options_copy.write_buffer_size);
   mutex_.Unlock();
@@ -3171,13 +3183,35 @@ Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context,
     wal_write_mutex_.Unlock();
     // TODO: Write buffer size passed in should be max of all CF's instead
     // of mutable_cf_options.write_buffer_size.
-    io_s = CreateWAL(write_options, new_log_number, recycle_log_number,
-                     preallocate_block_size, info, &new_log);
+    if (prepared_wal.writer) {
+      io_s = StartWALFile(write_options, info, prepared_wal.writer.get());
+      if (io_s.ok()) {
+        new_log = prepared_wal.writer.release();
+      }
+    } else {
+      io_s =
+          CreateWAL(db_options_snapshot, write_options, new_log_number,
+                    recycle_log_number, preallocate_block_size, info, &new_log);
+    }
     TEST_SYNC_POINT_CALLBACK("DBImpl::SwitchMemtable:AfterCreateWAL", &io_s);
     if (s.ok()) {
       s = io_s;
     }
   }
+  if (!s.ok() && prepared_wal.writer) {
+    const uint64_t prepared_wal_number = prepared_wal.log_number;
+    const std::string prepared_wal_fname =
+        LogFileName(immutable_db_options_.GetWalDir(), prepared_wal_number);
+    prepared_wal.Reset();
+    Status delete_s = env_->DeleteFile(prepared_wal_fname);
+    if (!delete_s.ok() && !delete_s.IsNotFound()) {
+      ROCKS_LOG_WARN(immutable_db_options_.info_log,
+                     "Failed to delete unstarted async precreated WAL #%" PRIu64
+                     ": %s",
+                     prepared_wal_number, delete_s.ToString().c_str());
+    }
+    delete_s.PermitUncheckedError();
+  }
   if (s.ok()) {
     // FIXME: from the comment for GetEarliestSequenceNumber(), any key with
     //  seqno >= earliest_seqno should be in this or later memtable. This means
@@ -3359,6 +3393,7 @@ Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context,
   new_mem->Ref();
   cfd->SetMemtable(new_mem);
   InstallSuperVersionAndScheduleWork(cfd, &context->superversion_context);
+  MaybeScheduleAsyncWALPrecreate(preallocate_block_size);
 
   // Notify client that memtable is sealed, now that we have successfully
   // installed a new memtable

db/db_options_test.cc

@@ -102,6 +102,40 @@ TEST_F(DBOptionsTest, ImmutableTrackAndVerifyWalsInManifest) {
   ASSERT_FALSE(s.ok());
 }
 
+TEST_F(DBOptionsTest, ImmutableAsyncWalPrecreate) {
+  Options options;
+  options.env = env_;
+  options.async_wal_precreate = true;
+
+  ImmutableDBOptions db_options(options);
+  ASSERT_TRUE(db_options.async_wal_precreate);
+
+  Reopen(options);
+  ASSERT_TRUE(dbfull()->GetDBOptions().async_wal_precreate);
+
+  Status s = dbfull()->SetDBOptions({{"async_wal_precreate", "false"}});
+  ASSERT_FALSE(s.ok());
+}
+
+TEST_F(DBOptionsTest, SanitizeAsyncWalPrecreateWithWalRecycle) {
+  // This checks the option-level contract for the incompatible WAL paths:
+  // SanitizeOptions() preserves WAL recycling and disables only the
+  // opportunistic async precreation path, so the immutable options reflect
+  // effective behavior.
+  Options options;
+  options.env = env_;
+  options.async_wal_precreate = true;
+  options.recycle_log_file_num = 1;
+
+  Options sanitized_options = SanitizeOptions(dbname_, options);
+  ASSERT_FALSE(sanitized_options.async_wal_precreate);
+  ASSERT_EQ(1U, sanitized_options.recycle_log_file_num);
+
+  options.recycle_log_file_num = 0;
+  sanitized_options = SanitizeOptions(dbname_, options);
+  ASSERT_TRUE(sanitized_options.async_wal_precreate);
+}
+
 TEST_F(DBOptionsTest, ImmutableVerifySstUniqueIdInManifest) {
   Options options;
   options.env = env_;

db/db_test.cc

@@ -8071,6 +8071,41 @@ INSTANTIATE_TEST_CASE_P(OpenFilesAsync, OpenFilesAsyncTest,
                                            ::testing::Values(-1, 10),
                                            ::testing::Bool()));
 
+TEST_F(DBTest, ReadOnlyCloseDoesNotDeleteWriterFiles) {
+  Options options = CurrentOptions();
+  options.create_if_missing = true;
+  options.disable_auto_compactions = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("before", "value"));
+  ASSERT_OK(Flush());
+
+  std::unique_ptr<DB> read_only_db;
+  ASSERT_OK(DB::OpenForReadOnly(options, dbname_, &read_only_db));
+
+  ASSERT_OK(Put("after", "value"));
+  ASSERT_OK(Flush());
+
+  std::vector<LiveFileMetaData> live_files;
+  db_->GetLiveFilesMetaData(&live_files);
+  ASSERT_GE(live_files.size(), 2);
+
+  std::string newest_sst_path;
+  uint64_t newest_file_number = 0;
+  for (const auto& live_file : live_files) {
+    if (live_file.file_number > newest_file_number) {
+      newest_file_number = live_file.file_number;
+      newest_sst_path = live_file.directory + "/" + live_file.relative_filename;
+    }
+  }
+  ASSERT_FALSE(newest_sst_path.empty());
+  ASSERT_OK(env_->FileExists(newest_sst_path));
+
+  read_only_db.reset();
+
+  ASSERT_OK(env_->FileExists(newest_sst_path));
+}
+
 // Test mix of races with async file open, reads, compactions
 TEST_P(OpenFilesAsyncTest, ConcurrentFileAccess) {
   Options options = CurrentOptions();

db/db_wal_test.cc

@@ -7,6 +7,8 @@
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file. See the AUTHORS file for names of contributors.
 
+#include <algorithm>
+
 #include "db/db_test_util.h"
 #include "db/db_with_timestamp_test_util.h"
 #include "options/options_helper.h"
@@ -19,12 +21,62 @@
 #include "utilities/fault_injection_env.h"
 #include "utilities/fault_injection_fs.h"
 
+#if defined(OS_LINUX)
+#include <sys/statfs.h>
+#endif
+
+#if defined(OS_LINUX) && !defined(BTRFS_SUPER_MAGIC)
+#define BTRFS_SUPER_MAGIC 0x9123683E
+#endif
+
+#if defined(OS_LINUX) && !defined(TMPFS_MAGIC)
+#define TMPFS_MAGIC 0x01021994
+#endif
+
+#if defined(OS_LINUX) && !defined(OVERLAYFS_SUPER_MAGIC)
+#define OVERLAYFS_SUPER_MAGIC 0x794c7630
+#endif
+
+#if defined(OS_LINUX) && !defined(ZFS_SUPER_MAGIC)
+#define ZFS_SUPER_MAGIC 0x2fc12fc1
+#endif
+
 namespace ROCKSDB_NAMESPACE {
 class DBWALTestBase : public DBTestBase {
  protected:
   explicit DBWALTestBase(const std::string& dir_name)
       : DBTestBase(dir_name, /*env_do_fsync=*/true) {}
 
+  std::vector<uint64_t> ListWalNumbers() {
+    std::vector<uint64_t> wal_numbers;
+    std::vector<std::string> files;
+    EXPECT_OK(env_->GetChildren(dbname_, &files));
+    for (const auto& file : files) {
+      uint64_t number = 0;
+      FileType type = kWalFile;
+      if (ParseFileName(file, &number, &type) && type == kWalFile) {
+        wal_numbers.push_back(number);
+      }
+    }
+    std::sort(wal_numbers.begin(), wal_numbers.end());
+    return wal_numbers;
+  }
+
+  void CreateEmptyWal(uint64_t wal_number) {
+    std::unique_ptr<WritableFile> file;
+    ASSERT_OK(env_->NewWritableFile(LogFileName(dbname_, wal_number), &file,
+                                    EnvOptions()));
+    ASSERT_OK(file->Close());
+  }
+
+  void CreateWalWithContents(uint64_t wal_number, const Slice& contents) {
+    std::unique_ptr<WritableFile> file;
+    ASSERT_OK(env_->NewWritableFile(LogFileName(dbname_, wal_number), &file,
+                                    EnvOptions()));
+    ASSERT_OK(file->Append(contents));
+    ASSERT_OK(file->Close());
+  }
+
 #if defined(ROCKSDB_PLATFORM_POSIX)
  public:
 #if defined(ROCKSDB_FALLOCATE_PRESENT)
@@ -57,6 +109,31 @@ class DBWALTestBase : public DBTestBase {
     assert(err == 0);
     return sbuf.st_blocks * 512;
   }
+
+#if defined(ROCKSDB_FALLOCATE_PRESENT)
+  bool ShouldSkipAllocationCheck(const std::string& file_name) {
+    (void)file_name;
+#if defined(OS_LINUX)
+    struct statfs fs_stat;
+    if (statfs(file_name.c_str(), &fs_stat) == 0) {
+      if (fs_stat.f_type ==
+          static_cast<decltype(fs_stat.f_type)>(BTRFS_SUPER_MAGIC)) {
+        return true;
+      } else if (fs_stat.f_type ==
+                 static_cast<decltype(fs_stat.f_type)>(ZFS_SUPER_MAGIC)) {
+        return true;
+      } else if (fs_stat.f_type ==
+                 static_cast<decltype(fs_stat.f_type)>(TMPFS_MAGIC)) {
+        return true;
+      } else if (fs_stat.f_type ==
+                 static_cast<decltype(fs_stat.f_type)>(OVERLAYFS_SUPER_MAGIC)) {
+        return true;
+      }
+    }
+#endif
+    return false;
+  }
+#endif  // ROCKSDB_FALLOCATE_PRESENT
 #endif  // ROCKSDB_PLATFORM_POSIX
 };
 
@@ -2752,16 +2829,35 @@ TEST_F(DBWALTest, TruncateLastLogAfterRecoverWithoutFlush) {
   auto& file_before = log_files_before[0];
   ASSERT_LT(file_before->SizeFileBytes(), 1 * kKB);
   // The log file has preallocated space.
-  ASSERT_GE(GetAllocatedFileSize(dbname_ + file_before->PathName()),
-            preallocated_size);
+  {
+    std::string fname = dbname_ + file_before->PathName();
+    uint64_t allocated = GetAllocatedFileSize(fname);
+    if (ShouldSkipAllocationCheck(fname)) {
+      fprintf(stderr,
+              "Skipping preallocation check on this filesystem for %s\n",
+              fname.c_str());
+    } else if (allocated < preallocated_size) {
+      fprintf(stderr,
+              "Warning: allocated size (%lu) less than preallocated size (%lu) "
+              "for %s, skipping check. This may indicate the filesystem does "
+              "not support preallocation or does not report it.\n",
+              allocated, preallocated_size, fname.c_str());
+    } else {
+      ASSERT_GE(allocated, preallocated_size);
+    }
+  }
   Reopen(options);
   VectorLogPtr log_files_after;
   ASSERT_OK(dbfull()->GetSortedWalFiles(log_files_after));
   ASSERT_EQ(1, log_files_after.size());
   ASSERT_LT(log_files_after[0]->SizeFileBytes(), 1 * kKB);
   // The preallocated space should be truncated.
-  ASSERT_LT(GetAllocatedFileSize(dbname_ + file_before->PathName()),
-            preallocated_size);
+  {
+    std::string fname = dbname_ + file_before->PathName();
+    if (!ShouldSkipAllocationCheck(fname)) {
+      ASSERT_LT(GetAllocatedFileSize(fname), preallocated_size);
+    }
+  }
 }
 // Tests that we will truncate the preallocated space of the last log from
 // previous.
@@ -2788,8 +2884,23 @@ TEST_F(DBWALTest, TruncateLastLogAfterRecoverWithFlush) {
   ASSERT_EQ(1, log_files_before.size());
   auto& file_before = log_files_before[0];
   ASSERT_LT(file_before->SizeFileBytes(), 1 * kKB);
-  ASSERT_GE(GetAllocatedFileSize(dbname_ + file_before->PathName()),
-            preallocated_size);
+  {
+    std::string fname = dbname_ + file_before->PathName();
+    uint64_t allocated = GetAllocatedFileSize(fname);
+    if (ShouldSkipAllocationCheck(fname)) {
+      fprintf(stderr,
+              "Skipping preallocation check on this filesystem for %s\n",
+              fname.c_str());
+    } else if (allocated < preallocated_size) {
+      fprintf(stderr,
+              "Warning: allocated size (%lu) less than preallocated size (%lu) "
+              "for %s, skipping check. This may indicate the filesystem does "
+              "not support preallocation or does not report it.\n",
+              allocated, preallocated_size, fname.c_str());
+    } else {
+      ASSERT_GE(allocated, preallocated_size);
+    }
+  }
   // The log file has preallocated space.
   Close();
 
@@ -2807,8 +2918,12 @@ TEST_F(DBWALTest, TruncateLastLogAfterRecoverWithFlush) {
   // if  the process is in a crash loop, the log file may not get
   // deleted and thte preallocated space will keep accumulating. So we need
   // to ensure it gets trtuncated.
-  EXPECT_LT(GetAllocatedFileSize(dbname_ + file_before->PathName()),
-            preallocated_size);
+  {
+    std::string fname = dbname_ + file_before->PathName();
+    if (!ShouldSkipAllocationCheck(fname)) {
+      EXPECT_LT(GetAllocatedFileSize(fname), preallocated_size);
+    }
+  }
   TEST_SYNC_POINT(
       "DBWALTest::TruncateLastLogAfterRecoverWithFlush:AfterTruncate");
   reopen_thread.join();
@@ -2863,14 +2978,31 @@ TEST_F(DBWALTest, TruncateLastLogAfterRecoverWALEmpty) {
   log_file->PrepareWrite(0, 4096);
   log_file.reset();
 
-  ASSERT_GE(GetAllocatedFileSize(last_log), preallocated_size);
+  {
+    uint64_t allocated = GetAllocatedFileSize(last_log);
+    if (ShouldSkipAllocationCheck(last_log)) {
+      fprintf(stderr,
+              "Skipping preallocation check on this filesystem for %s\n",
+              last_log.c_str());
+    } else if (allocated < preallocated_size) {
+      fprintf(stderr,
+              "Warning: allocated size (%lu) less than preallocated size (%lu) "
+              "for %s, skipping check. This may indicate the filesystem does "
+              "not support preallocation or does not report it.\n",
+              allocated, preallocated_size, last_log.c_str());
+    } else {
+      ASSERT_GE(allocated, preallocated_size);
+    }
+  }
 
   port::Thread reopen_thread([&]() { Reopen(options); });
 
   TEST_SYNC_POINT(
       "DBWALTest::TruncateLastLogAfterRecoverWithFlush:AfterRecover");
   // The preallocated space should be truncated.
-  EXPECT_LT(GetAllocatedFileSize(last_log), preallocated_size);
+  if (!ShouldSkipAllocationCheck(last_log)) {
+    EXPECT_LT(GetAllocatedFileSize(last_log), preallocated_size);
+  }
   TEST_SYNC_POINT(
       "DBWALTest::TruncateLastLogAfterRecoverWithFlush:AfterTruncate");
   reopen_thread.join();
@@ -2912,8 +3044,23 @@ TEST_F(DBWALTest, ReadOnlyRecoveryNoTruncate) {
   auto& file_before = log_files_before[0];
   ASSERT_LT(file_before->SizeFileBytes(), 1 * kKB);
   // The log file has preallocated space.
-  auto db_size = GetAllocatedFileSize(dbname_ + file_before->PathName());
-  ASSERT_GE(db_size, preallocated_size);
+  std::string fname = dbname_ + file_before->PathName();
+  auto db_size = GetAllocatedFileSize(fname);
+  {
+    if (ShouldSkipAllocationCheck(fname)) {
+      fprintf(stderr,
+              "Skipping preallocation check on this filesystem for %s\n",
+              fname.c_str());
+    } else if (db_size < preallocated_size) {
+      fprintf(stderr,
+              "Warning: allocated size (%lu) less than preallocated size (%lu) "
+              "for %s, skipping check. This may indicate the filesystem does "
+              "not support preallocation or does not report it.\n",
+              db_size, preallocated_size, fname.c_str());
+    } else {
+      ASSERT_GE(db_size, preallocated_size);
+    }
+  }
   Close();
 
   // enable truncate and open DB as readonly, the file should not be truncated
@@ -2927,8 +3074,12 @@ TEST_F(DBWALTest, ReadOnlyRecoveryNoTruncate) {
   ASSERT_EQ(log_files_after[0]->PathName(), file_before->PathName());
   // The preallocated space should NOT be truncated.
   // the DB size is almost the same.
-  ASSERT_NEAR(GetAllocatedFileSize(dbname_ + file_before->PathName()), db_size,
-              db_size / 100);
+  {
+    std::string fname2 = dbname_ + file_before->PathName();
+    if (!ShouldSkipAllocationCheck(fname2)) {
+      ASSERT_NEAR(GetAllocatedFileSize(fname2), db_size, db_size / 100);
+    }
+  }
   SyncPoint::GetInstance()->DisableProcessing();
   SyncPoint::GetInstance()->ClearAllCallBacks();
 }
@@ -3071,6 +3222,284 @@ TEST_F(DBWALTest, EmptyWalReopenTest) {
   }
 }
 
+TEST_F(DBWALTest, AsyncWalPrecreateWaitsForPendingFile) {
+  Options options = CurrentOptions();
+  options.env = env_;
+  options.statistics = CreateDBStatistics();
+  options.async_wal_precreate = true;
+  options.recycle_log_file_num = 0;
+  options.write_buffer_size = 1024;
+  options.max_write_buffer_number = 4;
+
+  // Goal: cover the race where foreground WAL rotation reaches the switch
+  // before the background precreate publishes its result. The sync points hold
+  // the async worker after it has created the file but before it marks it
+  // ready, then verify the foreground writer waits instead of allocating a
+  // higher WAL number.
+  SyncPoint::GetInstance()->LoadDependency({
+      {"DBImpl::BGWorkAsyncWALPrecreate:BeforePublish",
+       "DBWALTest::AsyncWalPrecreateWaitsForPendingFile:WorkerBlocked"},
+      {"DBWALTest::AsyncWalPrecreateWaitsForPendingFile:AllowPublish",
+       "DBImpl::BGWorkAsyncWALPrecreate:Publish"},
+      {"DBImpl::WaitForAsyncWALPrecreate:BeforeWait",
+       "DBWALTest::AsyncWalPrecreateWaitsForPendingFile:ForegroundWaiting"},
+  });
+  SyncPoint::GetInstance()->EnableProcessing();
+  Defer cleanup_sync_point([] {
+    SyncPoint::GetInstance()->DisableProcessing();
+    SyncPoint::GetInstance()->ClearAllCallBacks();
+  });
+
+  Reopen(options);
+  TEST_SYNC_POINT(
+      "DBWALTest::AsyncWalPrecreateWaitsForPendingFile:WorkerBlocked");
+
+  ASSERT_OK(Put("fill", "value"));
+  Status writer_status;
+  port::Thread writer(
+      [&]() { writer_status = dbfull()->TEST_SwitchMemtable(); });
+
+  TEST_SYNC_POINT(
+      "DBWALTest::AsyncWalPrecreateWaitsForPendingFile:ForegroundWaiting");
+  TEST_SYNC_POINT(
+      "DBWALTest::AsyncWalPrecreateWaitsForPendingFile:AllowPublish");
+  writer.join();
+  ASSERT_OK(writer_status);
+  ASSERT_OK(Put("trigger", "value"));
+  ASSERT_EQ("value", Get("trigger"));
+  ASSERT_EQ(1, options.statistics->getTickerCount(WAL_PRECREATE_HIT));
+  ASSERT_EQ(1, options.statistics->getTickerCount(WAL_PRECREATE_WAITED));
+  ASSERT_EQ(0, options.statistics->getTickerCount(WAL_PRECREATE_MISS));
+}
+
+TEST_F(DBWALTest, AsyncWalPrecreateConsumesReadyFile) {
+  Options options = CurrentOptions();
+  options.env = env_;
+  options.statistics = CreateDBStatistics();
+  options.async_wal_precreate = true;
+  options.recycle_log_file_num = 0;
+  options.write_buffer_size = 1024;
+  options.max_write_buffer_number = 4;
+
+  // Goal: exercise the no-wait consumption path. The background task publishes
+  // a prepared future WAL before the foreground memtable switch, and the switch
+  // must install exactly that reserved WAL number instead of allocating
+  // another.
+  SyncPoint::GetInstance()->LoadDependency({
+      {"DBImpl::BGWorkAsyncWALPrecreate:Done",
+       "DBWALTest::AsyncWalPrecreateConsumesReadyFile:PrecreateDone"},
+  });
+  SyncPoint::GetInstance()->EnableProcessing();
+  Defer cleanup_sync_point([] {
+    SyncPoint::GetInstance()->DisableProcessing();
+    SyncPoint::GetInstance()->ClearAllCallBacks();
+  });
+
+  Reopen(options);
+  TEST_SYNC_POINT(
+      "DBWALTest::AsyncWalPrecreateConsumesReadyFile:PrecreateDone");
+
+  const std::vector<uint64_t> wal_numbers = ListWalNumbers();
+  ASSERT_GE(wal_numbers.size(), 2);
+  const uint64_t prepared_wal_number = wal_numbers.back();
+  ASSERT_NE(prepared_wal_number, dbfull()->TEST_GetCurrentLogNumber());
+
+  ASSERT_OK(Put("fill", "value"));
+  ASSERT_OK(dbfull()->TEST_SwitchMemtable());
+  ASSERT_EQ(prepared_wal_number, dbfull()->TEST_GetCurrentLogNumber());
+  ASSERT_EQ(1, options.statistics->getTickerCount(WAL_PRECREATE_HIT));
+  ASSERT_EQ(0, options.statistics->getTickerCount(WAL_PRECREATE_MISS));
+  ASSERT_EQ(0, options.statistics->getTickerCount(WAL_PRECREATE_WAITED));
+}
+
+TEST_F(DBWALTest, AsyncWalPrecreateRecoveryToleratesActualFutureWal) {
+  Options options = CurrentOptions();
+  options.env = env_;
+  options.async_wal_precreate = true;
+  options.recycle_log_file_num = 0;
+  options.track_and_verify_wals = true;
+  options.track_and_verify_wals_in_manifest = true;
+  options.avoid_flush_during_shutdown = true;
+  options.avoid_flush_during_recovery = true;
+
+  // Goal: cover the actual async precreation path, not a hand-forged empty WAL.
+  // The background task opens the future WAL, clean close releases the
+  // unpublished writer and leaves the empty future WAL behind, and recovery
+  // must tolerate that file while replaying the preceding WAL with real user
+  // data.
+  SyncPoint::GetInstance()->LoadDependency({
+      {"DBImpl::BGWorkAsyncWALPrecreate:Done",
+       "DBWALTest::AsyncWalPrecreateRecoveryToleratesActualFutureWal:"
+       "PrecreateDone"},
+  });
+  SyncPoint::GetInstance()->EnableProcessing();
+  Defer cleanup_sync_point([] {
+    SyncPoint::GetInstance()->DisableProcessing();
+    SyncPoint::GetInstance()->ClearAllCallBacks();
+  });
+
+  DestroyAndReopen(options);
+  TEST_SYNC_POINT(
+      "DBWALTest::AsyncWalPrecreateRecoveryToleratesActualFutureWal:"
+      "PrecreateDone");
+
+  const std::vector<uint64_t> wal_numbers = ListWalNumbers();
+  ASSERT_GE(wal_numbers.size(), 2);
+  const uint64_t future_wal_number = wal_numbers.back();
+  uint64_t future_wal_size = 0;
+  ASSERT_OK(env_->GetFileSize(LogFileName(dbname_, future_wal_number),
+                              &future_wal_size));
+  ASSERT_EQ(0, future_wal_size);
+
+  ASSERT_OK(Put("key", "value"));
+  Close();
+
+  ASSERT_OK(TryReopen(options));
+  ASSERT_EQ("value", Get("key"));
+  ASSERT_OK(Put("after-recovery", "value2"));
+  ASSERT_EQ("value2", Get("after-recovery"));
+}
+
+TEST_F(DBWALTest, AsyncWalPrecreateDeletesFileOnStartFailure) {
+  Options options = CurrentOptions();
+  options.env = env_;
+  options.statistics = CreateDBStatistics();
+  options.async_wal_precreate = true;
+  options.recycle_log_file_num = 0;
+  options.write_buffer_size = 1024;
+  options.max_write_buffer_number = 4;
+
+  // Goal: force failure after consuming a prepared WAL but before it becomes a
+  // logical WAL. The failed, unpublished file must be deleted and its writer
+  // destroyed outside mutex_.
+  SyncPoint::GetInstance()->LoadDependency({
+      {"DBImpl::BGWorkAsyncWALPrecreate:Done",
+       "DBWALTest::AsyncWalPrecreateDeletesFileOnStartFailure:PrecreateDone"},
+  });
+  SyncPoint::GetInstance()->EnableProcessing();
+  Defer cleanup_sync_point([] {
+    SyncPoint::GetInstance()->DisableProcessing();
+    SyncPoint::GetInstance()->ClearAllCallBacks();
+  });
+
+  Reopen(options);
+  TEST_SYNC_POINT(
+      "DBWALTest::AsyncWalPrecreateDeletesFileOnStartFailure:PrecreateDone");
+  SyncPoint::GetInstance()->SetCallBack(
+      "DBImpl::StartWALFile:AfterCompressionTypeRecord", [](void* arg) {
+        *static_cast<IOStatus*>(arg) =
+            IOStatus::IOError("injected async WAL start failure");
+      });
+  const std::vector<uint64_t> wal_numbers = ListWalNumbers();
+  ASSERT_GE(wal_numbers.size(), 2);
+  const uint64_t prepared_wal_number = wal_numbers.back();
+
+  ASSERT_OK(Put("fill", "value"));
+  ASSERT_NOK(dbfull()->TEST_SwitchMemtable());
+  ASSERT_TRUE(
+      env_->FileExists(LogFileName(dbname_, prepared_wal_number)).IsNotFound());
+  ASSERT_EQ(1, options.statistics->getTickerCount(WAL_PRECREATE_HIT));
+}
+
+TEST_F(DBWALTest, AsyncWalPrecreateFailureFallsBackToSyncCreation) {
+  Options options = CurrentOptions();
+  options.env = env_;
+  options.statistics = CreateDBStatistics();
+  options.async_wal_precreate = true;
+  options.recycle_log_file_num = 0;
+  options.write_buffer_size = 1024;
+  options.max_write_buffer_number = 4;
+
+  // Goal: fail the background precreate file open after normal DB open has
+  // created the current WAL. The foreground switch should observe no prepared
+  // WAL, create one synchronously, and expose both failure and miss counters.
+  SyncPoint::GetInstance()->LoadDependency({
+      {"DBImpl::BGWorkAsyncWALPrecreate:Done",
+       "DBWALTest::AsyncWalPrecreateFailureFallsBackToSyncCreation:"
+       "PrecreateDone"},
+  });
+  SyncPoint::GetInstance()->SetCallBack(
+      "DBImpl::Open:AfterDeleteFiles",
+      [&](void*) { env_->non_writable_count_ = 1; });
+  SyncPoint::GetInstance()->EnableProcessing();
+  Defer cleanup_sync_point([&] {
+    env_->non_writable_count_ = 0;
+    SyncPoint::GetInstance()->DisableProcessing();
+    SyncPoint::GetInstance()->ClearAllCallBacks();
+  });
+
+  Reopen(options);
+  TEST_SYNC_POINT(
+      "DBWALTest::AsyncWalPrecreateFailureFallsBackToSyncCreation:"
+      "PrecreateDone");
+  ASSERT_EQ(1, options.statistics->getTickerCount(WAL_PRECREATE_FAILED));
+
+  ASSERT_OK(Put("fill", "value"));
+  ASSERT_OK(dbfull()->TEST_SwitchMemtable());
+  ASSERT_EQ(1, options.statistics->getTickerCount(WAL_PRECREATE_MISS));
+}
+
+TEST_F(DBWALTest, ReadOnlyErrorIfWalFileExistsIgnoresEmptyWals) {
+  Options options = CurrentOptions();
+  options.env = env_;
+  DestroyAndReopen(options);
+  Close();
+
+  // Goal: preserve error_if_wal_file_exists for WALs containing data while
+  // allowing empty future WALs left by async precreation. This creates empty
+  // WAL files directly so the test isolates the read-only open contract.
+  const std::vector<uint64_t> existing_wal_numbers = ListWalNumbers();
+  const uint64_t empty_wal_number =
+      existing_wal_numbers.empty() ? 1 : existing_wal_numbers.back() + 1;
+  CreateEmptyWal(empty_wal_number);
+
+  DBOptions db_options(options);
+  std::vector<ColumnFamilyDescriptor> column_families;
+  column_families.emplace_back(kDefaultColumnFamilyName,
+                               ColumnFamilyOptions(options));
+  std::vector<ColumnFamilyHandle*> handles;
+  std::unique_ptr<DB> read_only_db;
+  ASSERT_OK(DB::OpenForReadOnly(db_options, dbname_, column_families, &handles,
+                                &read_only_db,
+                                true /* error_if_wal_file_exists */));
+  for (ColumnFamilyHandle* handle : handles) {
+    ASSERT_OK(read_only_db->DestroyColumnFamilyHandle(handle));
+  }
+  read_only_db.reset();
+  handles.clear();
+
+  CreateWalWithContents(empty_wal_number + 1, Slice("not empty"));
+  ASSERT_NOK(DB::OpenForReadOnly(db_options, dbname_, column_families, &handles,
+                                 &read_only_db,
+                                 true /* error_if_wal_file_exists */));
+}
+
+TEST_F(DBWALTest, RecoveryToleratesEmptyFutureWal) {
+  Options options = CurrentOptions();
+  options.env = env_;
+  options.track_and_verify_wals = true;
+  options.track_and_verify_wals_in_manifest = true;
+  options.avoid_flush_during_recovery = true;
+  DestroyAndReopen(options);
+
+  ASSERT_OK(Put("key", "value"));
+  Close();
+
+  const std::vector<uint64_t> wal_numbers = ListWalNumbers();
+  ASSERT_FALSE(wal_numbers.empty());
+  const uint64_t future_wal_number = wal_numbers.back() + 1;
+  CreateEmptyWal(future_wal_number);
+
+  // Goal: model a crash after async WAL precreation but before consumption.
+  // Recovery should process the zero-byte future WAL, mark its file number
+  // used, and keep replaying the real preceding WAL without reporting a WAL
+  // chain or MANIFEST-tracking error.
+  ASSERT_OK(TryReopen(options));
+  ASSERT_EQ("value", Get("key"));
+  ASSERT_OK(Put("after-recovery", "value2"));
+  ASSERT_EQ("value2", Get("after-recovery"));
+}
+
 TEST_F(DBWALTest, RecoveryFlushSwitchWALOnEmptyMemtable) {
   Options options = CurrentOptions();
   auto fault_fs = std::make_shared<FaultInjectionTestFS>(FileSystem::Default());

db/listener_test.cc

@@ -201,6 +201,121 @@ TEST_F(EventListenerTest, OnSingleDBCompactionTest) {
   }
 }
 
+// Listener that asserts OnCompactionPreCommit fires strictly between
+// OnCompactionBegin and OnCompactionCompleted, and that input files'
+// being_compacted flag is still true at that point.
+class TestCompactionPreCommitListener : public EventListener {
+ public:
+  explicit TestCompactionPreCommitListener(EventListenerTest* test)
+      : test_(test) {}
+
+  void OnCompactionBegin(DB* /*db*/, const CompactionJobInfo& ci) override {
+    std::lock_guard<std::mutex> lock(mutex_);
+    ++begin_count_;
+    last_begin_job_id_ = ci.job_id;
+    EXPECT_EQ(begin_count_, pre_commit_count_ + 1);
+    EXPECT_EQ(begin_count_, completed_count_ + 1);
+  }
+
+  void OnCompactionPreCommit(DB* db, const CompactionJobInfo& ci) override {
+    std::lock_guard<std::mutex> lock(mutex_);
+    ++pre_commit_count_;
+    // Must fire after Begin and before Completed for this compaction.
+    EXPECT_EQ(pre_commit_count_, begin_count_);
+    EXPECT_EQ(pre_commit_count_, completed_count_ + 1);
+    EXPECT_EQ(ci.job_id, last_begin_job_id_);
+    EXPECT_GT(ci.input_files.size(), 0U);
+
+    // Verify input files are still marked being_compacted.
+    std::vector<std::vector<FileMetaData>> files_by_level;
+    test_->dbfull()->TEST_GetFilesMetaData(test_->handles_[ci.cf_id],
+                                           &files_by_level);
+    EXPECT_EQ(test_->db_.get(), db);
+    for (const auto& info : ci.input_file_infos) {
+      bool found = false;
+      for (const auto& level_files : files_by_level) {
+        for (const auto& meta : level_files) {
+          if (meta.fd.GetNumber() == info.file_number) {
+            found = true;
+            EXPECT_TRUE(meta.being_compacted)
+                << "input file " << info.file_number
+                << " should still be being_compacted in "
+                   "OnCompactionPreCommit";
+          }
+        }
+      }
+      EXPECT_TRUE(found) << "input file " << info.file_number
+                         << " not found in DB";
+    }
+  }
+
+  void OnCompactionCompleted(DB* /*db*/, const CompactionJobInfo& ci) override {
+    std::lock_guard<std::mutex> lock(mutex_);
+    ++completed_count_;
+    // Must fire after the matching PreCommit.
+    EXPECT_EQ(completed_count_, pre_commit_count_);
+    EXPECT_EQ(completed_count_, begin_count_);
+    EXPECT_EQ(ci.job_id, last_begin_job_id_);
+  }
+
+  size_t BeginCount() {
+    std::lock_guard<std::mutex> lock(mutex_);
+    return begin_count_;
+  }
+  size_t PreCommitCount() {
+    std::lock_guard<std::mutex> lock(mutex_);
+    return pre_commit_count_;
+  }
+  size_t CompletedCount() {
+    std::lock_guard<std::mutex> lock(mutex_);
+    return completed_count_;
+  }
+
+ private:
+  EventListenerTest* test_;
+  std::mutex mutex_;
+  size_t begin_count_ = 0;
+  size_t pre_commit_count_ = 0;
+  size_t completed_count_ = 0;
+  int last_begin_job_id_ = -1;
+};
+
+TEST_F(EventListenerTest, OnCompactionPreCommitOrdering) {
+  Options options = CurrentOptions();
+  options.create_if_missing = true;
+  options.compaction_style = kCompactionStyleLevel;
+  options.compression = kNoCompression;
+  options.level0_file_num_compaction_trigger = 4;
+  auto* listener = new TestCompactionPreCommitListener(this);
+  options.listeners.emplace_back(listener);
+
+  // Verify sync-point ordering: Begin -> PreCommit -> Completed.
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->LoadDependency(
+      {{"DBImpl::NotifyOnCompactionBegin::UnlockMutex",
+        "DBImpl::NotifyOnCompactionPreCommit::UnlockMutex"},
+       {"DBImpl::NotifyOnCompactionPreCommit::UnlockMutex",
+        "DBImpl::NotifyOnCompactionCompleted::UnlockMutex"}});
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->EnableProcessing();
+
+  CreateAndReopenWithCF({"pikachu"}, options);
+
+  Random rnd(301);
+  for (int i = 0; i < 4; i++) {
+    for (int j = 0; j < 10; j++) {
+      ASSERT_OK(Put(1, rnd.RandomString(10), rnd.RandomString(10)));
+    }
+    ASSERT_OK(Flush(1));
+  }
+  ASSERT_OK(dbfull()->TEST_WaitForCompact());
+
+  EXPECT_GT(listener->BeginCount(), 0U);
+  EXPECT_EQ(listener->PreCommitCount(), listener->BeginCount());
+  EXPECT_EQ(listener->CompletedCount(), listener->BeginCount());
+
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->DisableProcessing();
+  ROCKSDB_NAMESPACE::SyncPoint::GetInstance()->ClearAllCallBacks();
+}
+
 // This simple Listener can only handle one flush at a time.
 class TestFlushListener : public EventListener {
  public:

db/version_edit.cc

@@ -240,6 +240,13 @@ bool VersionEdit::EncodeTo(std::string* dst,
     PutLengthPrefixedSlice(dst, progress_data);
   }
 
+  if (has_last_compacted_manifest_file_size_) {
+    PutVarint32(dst, kLastCompactedManifestFileSize);
+    std::string varint_size;
+    PutVarint64(&varint_size, last_compacted_manifest_file_size_);
+    PutLengthPrefixedSlice(dst, Slice(varint_size));
+  }
+
   return true;
 }
 
@@ -890,6 +897,17 @@ Status VersionEdit::DecodeFrom(const Slice& src) {
         break;
       }
 
+      case kLastCompactedManifestFileSize: {
+        Slice encoded;
+        if (GetLengthPrefixedSlice(&input, &encoded) &&
+            GetVarint64(&encoded, &last_compacted_manifest_file_size_)) {
+          has_last_compacted_manifest_file_size_ = true;
+        } else {
+          msg = "last compacted manifest file size";
+        }
+        break;
+      }
+
       default:
         if (tag & kTagSafeIgnoreMask) {
           // Tag from future which can be safely ignored.
@@ -1064,6 +1082,10 @@ std::string VersionEdit::DebugString(bool hex_key) const {
     r.append("\n SubcompactionProgress: ");
     r.append(subcompaction_progress_.ToString());
   }
+  if (has_last_compacted_manifest_file_size_) {
+    r.append("\n  LastCompactedManifestFileSize: ");
+    AppendNumberTo(&r, last_compacted_manifest_file_size_);
+  }
   r.append("\n}\n");
   return r;
 }
@@ -1220,6 +1242,10 @@ std::string VersionEdit::DebugJSON(int edit_num, bool hex_key) const {
     jw << "SubcompactionProgress" << subcompaction_progress_.ToString();
   }
 
+  if (has_last_compacted_manifest_file_size_) {
+    jw << "LastCompactedManifestFileSize" << last_compacted_manifest_file_size_;
+  }
+
   jw.EndObject();
 
   return jw.Get();

db/version_edit.h

@@ -74,6 +74,7 @@ enum Tag : uint32_t {
   kWalDeletion2,
   kPersistUserDefinedTimestamps,
   kSubcompactionProgress,
+  kLastCompactedManifestFileSize,
 };
 
 enum SubcompactionProgressPerLevelCustomTag : uint32_t {
@@ -1016,6 +1017,17 @@ class VersionEdit {
     subcompaction_progress_.Clear();
   }
 
+  void SetLastCompactedManifestFileSize(uint64_t size) {
+    has_last_compacted_manifest_file_size_ = true;
+    last_compacted_manifest_file_size_ = size;
+  }
+  bool HasLastCompactedManifestFileSize() const {
+    return has_last_compacted_manifest_file_size_;
+  }
+  uint64_t GetLastCompactedManifestFileSize() const {
+    return last_compacted_manifest_file_size_;
+  }
+
   // Recovery-time per-column-family edits only need to be written when they
   // advance the CF's log number or carry some other manifest state.
   bool ShouldEmitPerColumnFamilyRecoveryEdit(uint64_t current_log_number) const;
@@ -1111,6 +1123,9 @@ class VersionEdit {
   bool has_subcompaction_progress_ = false;
   SubcompactionProgress subcompaction_progress_;
 
+  bool has_last_compacted_manifest_file_size_ = false;
+  uint64_t last_compacted_manifest_file_size_ = 0;
+
   // Newly created table files and blob files are eligible for deletion if they
   // are not registered as live files after the background jobs creating them
   // have finished. In case committing the VersionEdit containing such changes

db/version_edit_handler.cc

@@ -642,6 +642,11 @@ Status VersionEditHandler::ExtractInfoFromVersionEdit(ColumnFamilyData* cfd,
              version_edit_params_.GetLastSequence() <= edit.GetLastSequence());
       version_edit_params_.SetLastSequence(edit.GetLastSequence());
     }
+    if (edit.HasLastCompactedManifestFileSize()) {
+      version_set_->last_compacted_manifest_file_size_ =
+          edit.GetLastCompactedManifestFileSize();
+      version_set_->TuneMaxManifestFileSize();
+    }
     if (!version_edit_params_.HasPrevLogNumber()) {
       version_edit_params_.SetPrevLogNumber(0);
     }

db/version_set.cc

@@ -7496,6 +7496,28 @@ Status VersionSet::WriteCurrentStateToManifest(
       }
     }
   }
+
+  // Record the approximate compacted manifest size so it's available at
+  // recovery time for TuneMaxManifestFileSize(). This record must come last
+  // in WriteCurrentStateToManifest for an accurate size estimate.
+  {
+    // Include a rough estimate of this record's own size (~15 bytes for the
+    // VersionEdit payload + log record header).
+    constexpr uint64_t kEstimatedRecordOverhead = 15;
+    VersionEdit edit;
+    edit.SetLastCompactedManifestFileSize(log->file()->GetFileSize() +
+                                          kEstimatedRecordOverhead);
+    std::string record;
+    if (!edit.EncodeTo(&record)) {
+      return Status::Corruption("Unable to Encode VersionEdit:" +
+                                edit.DebugString(true));
+    }
+    io_s = log->AddRecord(write_options, record);
+    if (!io_s.ok()) {
+      return io_s;
+    }
+  }
+
   return Status::OK();
 }
 

db_stress_tool/db_stress_common.h

@@ -114,6 +114,7 @@ DECLARE_double(memtable_prefix_bloom_size_ratio);
 DECLARE_bool(memtable_whole_key_filtering);
 DECLARE_int32(open_files);
 DECLARE_bool(open_files_async);
+DECLARE_bool(async_wal_precreate);
 DECLARE_uint64(compressed_secondary_cache_size);
 DECLARE_int32(compressed_secondary_cache_numshardbits);
 DECLARE_int32(secondary_cache_update_interval);
@@ -472,6 +473,8 @@ DECLARE_int32(compaction_on_deletion_window_size);
 DECLARE_double(compaction_on_deletion_ratio);
 DECLARE_double(read_triggered_compaction_threshold);
 
+DECLARE_string(listener_uri);
+
 constexpr long KB = 1024;
 constexpr int kRandomValueMaxFactor = 3;
 constexpr int kValueMaxLen = 100;

db_stress_tool/db_stress_compaction_service.cc

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License

db_stress_tool/db_stress_gflags.cc

@@ -206,6 +206,10 @@ DEFINE_int32(open_files, ROCKSDB_NAMESPACE::Options().max_open_files,
 DEFINE_bool(open_files_async, ROCKSDB_NAMESPACE::Options().open_files_async,
             "Options.open_files_async");
 
+DEFINE_bool(async_wal_precreate,
+            ROCKSDB_NAMESPACE::Options().async_wal_precreate,
+            "Options.async_wal_precreate");
+
 DEFINE_uint64(compressed_secondary_cache_size, 0,
               "Number of bytes to use as a cache of compressed data."
               " 0 means use default settings.");
@@ -1664,6 +1668,10 @@ DEFINE_double(read_triggered_compaction_threshold,
               ROCKSDB_NAMESPACE::Options().read_triggered_compaction_threshold,
               "Sets CF option read_triggered_compaction_threshold.");
 
+DEFINE_string(listener_uri, "",
+              "URI for an additional EventListener to attach (e.g. "
+              "auto_tuner://). Empty means none.");
+
 DEFINE_bool(
     auto_refresh_iterator_with_snapshot,
     ROCKSDB_NAMESPACE::ReadOptions().auto_refresh_iterator_with_snapshot,

db_stress_tool/db_stress_listener.h

@@ -6,7 +6,9 @@
 #ifdef GFLAGS
 #pragma once
 
+#include <cinttypes>
 #include <mutex>
+#include <unordered_map>
 #include <unordered_set>
 
 #include "db_stress_tool/db_stress_compaction_service.h"
@@ -19,6 +21,7 @@
 #include "rocksdb/listener.h"
 #include "rocksdb/table_properties.h"
 #include "rocksdb/unique_id.h"
+#include "util/atomic.h"
 #include "util/gflags_compat.h"
 #include "util/random.h"
 #include "utilities/fault_injection_fs.h"
@@ -69,6 +72,13 @@ class DbStressListener : public EventListener {
   static const char* kClassName() { return "DBStressListener"; }
 
   ~DbStressListener() override { assert(num_pending_file_creations_ == 0); }
+
+  // Signal that the DB is about to shut down. Must be called before DB::Close()
+  // or CancelAllBackgroundWork(). Why? Listener notifications, especially for
+  // compaction, have different (mostly relaxed) contracts during shutdown, and
+  // not all callbacks take a DB object. Ideally, public API improvements would
+  // make this unnecessary in the future.
+  void NotifyShuttingDown() { shutting_down_.Store(true); }
   void OnFlushCompleted(DB* /*db*/, const FlushJobInfo& info) override {
     assert(IsValidColumnFamilyName(info.cf_name));
     VerifyFilePath(info.file_path);
@@ -118,11 +128,83 @@ class DbStressListener : public EventListener {
     }
   }
 
-  void OnTableFileDeleted(const TableFileDeletionInfo& /*info*/) override {
+  void OnTableFileDeleted(const TableFileDeletionInfo& info) override {
+    // A file must not be deleted while it is still between
+    // OnCompactionBegin and OnCompactionPreCommit (i.e. actively being
+    // compacted with being_compacted == true). (Perhaps more realistically,
+    // this is checking for failure to call OnCompactionPreCommit.)
+    //
+    // During shutdown, compaction notifications are skipped so tracking
+    // state may be stale -- skip the check.
+    if (!shutting_down_.Load()) {
+      std::lock_guard<std::mutex> lock(compacting_files_mu_);
+      uint64_t file_number = FileNumberFromPath(info.file_path);
+      if (file_number != 0 &&
+          compacting_files_.find(file_number) != compacting_files_.end()) {
+        fprintf(stderr,
+                "OnTableFileDeleted for file tracked as being compacted "
+                "(between Begin and PreCommit): file_number=%" PRIu64 "\n",
+                file_number);
+        fflush(stderr);
+        std::abort();
+      }
+    }
     RandomSleep();
   }
 
-  void OnCompactionBegin(DB* /*db*/, const CompactionJobInfo& /*ci*/) override {
+  void OnCompactionBegin(DB* /*db*/, const CompactionJobInfo& ci) override {
+    // Sanity check inspired by a Meta-internal check: the same input file must
+    // not be in two concurrent compactions. Inserting input file numbers into a
+    // shared set on Begin and removing them on PreCommit (rather than on
+    // Completed) is what exercises the new OnCompactionPreCommit
+    // callback. Removing on Completed would race with another picker that
+    // grabs the same file as soon as ReleaseCompactionFiles flips
+    // FileMetaData::being_compacted back to false; PreCommit runs
+    // while being_compacted is still true and so closes that race.
+    {
+      std::lock_guard<std::mutex> lock(compacting_files_mu_);
+      for (const auto& info : ci.input_file_infos) {
+        auto [_, inserted] = compacting_files_.insert(info.file_number);
+        if (!inserted) {
+          fprintf(stderr,
+                  "Concurrent compaction of SST file detected: cf=%s "
+                  "file_number=%" PRIu64 "\n",
+                  ci.cf_name.c_str(), info.file_number);
+          fflush(stderr);
+          std::abort();
+        }
+      }
+    }
+    RandomSleep();
+  }
+
+  void OnCompactionPreCommit(DB* /*db*/, const CompactionJobInfo& ci) override {
+    // Pair with OnCompactionBegin's bookkeeping: move files from
+    // compacting_files_ and record the job for Completed verification.
+    {
+      std::lock_guard<std::mutex> lock(compacting_files_mu_);
+      std::unordered_set<uint64_t> job_files;
+      for (const auto& info : ci.input_file_infos) {
+        size_t erased = compacting_files_.erase(info.file_number);
+        if (erased != 1) {
+          fprintf(stderr,
+                  "OnCompactionPreCommit for file not in tracking set: "
+                  "cf=%s file_number=%" PRIu64 "\n",
+                  ci.cf_name.c_str(), info.file_number);
+          fflush(stderr);
+          std::abort();
+        }
+        job_files.insert(info.file_number);
+      }
+      auto [_, inserted] =
+          precommitted_jobs_.emplace(ci.job_id, std::move(job_files));
+      if (!inserted) {
+        fprintf(stderr, "OnCompactionPreCommit: duplicate job_id %d\n",
+                ci.job_id);
+        fflush(stderr);
+        std::abort();
+      }
+    }
     RandomSleep();
   }
 
@@ -135,6 +217,34 @@ class DbStressListener : public EventListener {
     for (const auto& file_path : ci.output_files) {
       VerifyFilePath(file_path);
     }
+
+    // Verify that OnCompactionPreCommit fired before OnCompactionCompleted
+    // for the same job, with matching input files.
+    {
+      std::lock_guard<std::mutex> lock(compacting_files_mu_);
+      auto it = precommitted_jobs_.find(ci.job_id);
+      if (it == precommitted_jobs_.end()) {
+        fprintf(stderr,
+                "OnCompactionCompleted without prior OnCompactionPreCommit: "
+                "cf=%s job_id=%d\n",
+                ci.cf_name.c_str(), ci.job_id);
+        fflush(stderr);
+        std::abort();
+      }
+      // Verify input file sets match between PreCommit and Completed.
+      for (const auto& info : ci.input_file_infos) {
+        if (it->second.find(info.file_number) == it->second.end()) {
+          fprintf(stderr,
+                  "OnCompactionCompleted: input file %" PRIu64
+                  " not in PreCommit set for job_id=%d\n",
+                  info.file_number, ci.job_id);
+          fflush(stderr);
+          std::abort();
+        }
+      }
+      precommitted_jobs_.erase(it);
+    }
+
     // pretending doing some work here
     RandomSleep();
   }
@@ -377,6 +487,39 @@ class DbStressListener : public EventListener {
   SharedState* shared_;
   mutable std::mutex bg_pressure_mu_;
   BackgroundJobPressure last_bg_pressure_;
+  // Files (by file_number) currently in flight from OnCompactionBegin to
+  // OnCompactionPreCommit. Used to detect concurrent compaction of
+  // the same SST file -- the bug fixed by the OnCompactionPreCommit
+  // callback. Protected by compacting_files_mu_.
+  std::mutex compacting_files_mu_;
+  std::unordered_set<uint64_t> compacting_files_;
+  // Set before DB close to suppress false positives from stale tracking.
+  Atomic<bool> shutting_down_{false};
+  // Jobs that have passed OnCompactionPreCommit but not yet
+  // OnCompactionCompleted. Maps job_id -> input file numbers.
+  // Used to verify Begin -> PreCommit -> Completed ordering per job.
+  // Protected by compacting_files_mu_.
+  std::unordered_map<int, std::unordered_set<uint64_t>> precommitted_jobs_;
+
+  // Extract file number from a file path like "/path/to/000123.sst".
+  // Returns 0 if the path cannot be parsed.
+  static uint64_t FileNumberFromPath(const std::string& file_path) {
+    size_t pos = file_path.find_last_of('/');
+    // Avoid copying file_path when no '/' separator is found
+    std::string file_name_buf;
+    if (pos != std::string::npos) {
+      file_name_buf = file_path.substr(pos + 1);
+    }
+    const std::string& file_name =
+        (pos == std::string::npos) ? file_path : file_name_buf;
+    uint64_t file_number = 0;
+    FileType file_type;
+    if (ParseFileName(file_name, &file_number, &file_type) &&
+        file_type == kTableFile) {
+      return file_number;
+    }
+    return 0;
+  }
 };
 }  // namespace ROCKSDB_NAMESPACE
 #endif  // GFLAGS

db_stress_tool/db_stress_test_base.cc

@@ -95,6 +95,9 @@ StressTest::StressTest()
 }
 
 void StressTest::CleanUp() {
+  // Notify listener before DB close so it can tolerate stale tracking
+  // from skipped notifications during shutdown.
+  NotifyListenerShuttingDown();
   CleanUpColumnFamilies();
   if (db_) {
     db_->Close();
@@ -105,6 +108,15 @@ void StressTest::CleanUp() {
   secondary_db_.reset();
 }
 
+void StressTest::NotifyListenerShuttingDown() {
+  for (auto& listener : options_.listeners) {
+    if (strcmp(listener->Name(), DbStressListener::kClassName()) == 0) {
+      static_cast_with_check<DbStressListener>(listener.get())
+          ->NotifyShuttingDown();
+    }
+  }
+}
+
 void StressTest::CleanUpColumnFamilies() {
   for (auto cf : column_families_) {
     delete cf;
@@ -3962,6 +3974,19 @@ void StressTest::Open(SharedState* shared, bool reopen) {
         FLAGS_db, options_.db_paths, cf_descriptors, shared));
     RegisterAdditionalListeners();
 
+    if (!FLAGS_listener_uri.empty()) {
+      std::shared_ptr<EventListener> listener;
+      Status listener_status =
+          ObjectRegistry::Default()->NewSharedObject<EventListener>(
+              FLAGS_listener_uri, &listener);
+      if (!listener_status.ok()) {
+        fprintf(stderr, "Failed to create listener from URI '%s': %s\n",
+                FLAGS_listener_uri.c_str(), listener_status.ToString().c_str());
+        exit(1);
+      }
+      options_.listeners.emplace_back(std::move(listener));
+    }
+
     // If this is for DB reopen,  error injection may have been enabled.
     // Disable it here in case there is no open fault injection.
     if (fault_fs_guard) {
@@ -4229,6 +4254,10 @@ void StressTest::Open(SharedState* shared, bool reopen) {
 }
 
 void StressTest::Reopen(ThreadState* thread) {
+  // Notify listener before DB close so it can tolerate stale tracking
+  // from skipped notifications during shutdown.
+  NotifyListenerShuttingDown();
+
   // BG jobs in WritePrepared must be canceled first because i) they can access
   // the db via a callbac ii) they hold on to a snapshot and the upcoming
   // ::Close would complain about it.
@@ -4589,6 +4618,7 @@ void InitializeOptionsFromFlags(
   }
   options.max_open_files = FLAGS_open_files;
   options.open_files_async = FLAGS_open_files_async;
+  options.async_wal_precreate = FLAGS_async_wal_precreate;
   if (FLAGS_open_files_async && !FLAGS_skip_stats_update_on_db_open) {
     FLAGS_skip_stats_update_on_db_open = true;
     fprintf(stderr,

db_stress_tool/db_stress_test_base.h

@@ -71,6 +71,9 @@ class StressTest {
   Options GetOptions(int cf_id);
   void CleanUp();
 
+ private:
+  void NotifyListenerShuttingDown();
+
  protected:
   static int GetMinInjectedErrorCount(int error_count_1, int error_count_2) {
     if (error_count_1 > 0 && error_count_2 > 0) {

docs/components/write_flow/03_wal.md

@@ -94,11 +94,23 @@ Step 3 - **Synced**: `fsync()` is called when `WriteOptions::sync` is true, or w
 
 Step 4 - **Obsolete**: After all memtables referencing this WAL are flushed, the WAL becomes obsolete and is either recycled or deleted.
 
+## Async WAL Precreation
+
+When `DBOptions::async_wal_precreate` is enabled, RocksDB keeps at most one future WAL file precreated in the background. RocksDB reserves a file number before scheduling the background task; the task opens the file/writer, but deliberately leaves the file as reserved empty storage:
+
+- It does not write compression metadata.
+- It does not write predecessor WAL info.
+- It does not add the file to `logs_`, `alive_wal_files_`, or MANIFEST WAL tracking.
+
+The file becomes a logical WAL only when foreground `SwitchMemtable()` consumes it. At that point RocksDB writes the normal WAL metadata, flushes the previous WAL writer buffer, and installs the new WAL in the same order as synchronous WAL creation. If foreground rotation reaches the switch while the background task is still running, the writer waits for the reserved file number instead of allocating a higher numbered WAL. This avoids a late lower-numbered WAL appearing after a newer WAL has become live. If background precreation fails, the background task logs the failure and foreground rotation falls back to normal synchronous WAL creation.
+
+Unused precreated WALs are safe across crashes and clean shutdowns because they are zero-record future WAL files. Recovery processes WAL files by log number, marks every observed WAL number as used, and treats an empty future WAL as EOF. On clean close, RocksDB releases the unpublished WAL writer but does not need to delete the empty file. The option is sanitized to false when `recycle_log_file_num > 0`.
+
 ## WAL Tracking in MANIFEST
 
 `WalSet` (see `db/wal_edit.h`, tracked in `VersionSet`) manages WAL metadata in the MANIFEST:
 
-- `WalAddition` records are written when a WAL is created or when a closed/inactive WAL's synced size is finalized
+- `WalAddition` records are written when a closed/inactive WAL's synced size is finalized
 - `WalDeletion` records are written when a WAL becomes obsolete after flush
 - Live-WAL syncs (via `DB::SyncWAL()` or `WriteOptions::sync`) are intentionally not tracked in MANIFEST
 

docs/components/write_flow/04_memtable_insert.md

@@ -103,7 +103,7 @@ When a flush is triggered, `DBImpl::SwitchMemtable()` (see `db/db_impl/db_impl_w
 
 Step 1 - Writes any recoverable state to the current memtable.
 
-Step 2 - Creates a new WAL file (unless the current WAL is empty) with the next file number from `VersionSet::NewFileNumber()`.
+Step 2 - Creates a new WAL file (unless the current WAL is empty) with the next file number from `VersionSet::NewFileNumber()`. If `DBOptions::async_wal_precreate` has prepared a future WAL, this step consumes that reserved file number and finalizes the WAL metadata instead of creating the file synchronously.
 
 Step 3 - Constructs a new `MemTable` with `earliest_seq` set to the current last sequence.
 

env/env_test.cc

@@ -38,6 +38,18 @@
 #define BTRFS_SUPER_MAGIC 0x9123683E
 #endif
 
+#if !defined(TMPFS_MAGIC)
+#define TMPFS_MAGIC 0x01021994
+#endif
+
+#if !defined(OVERLAYFS_SUPER_MAGIC)
+#define OVERLAYFS_SUPER_MAGIC 0x794c7630
+#endif
+
+#if !defined(ZFS_SUPER_MAGIC)
+#define ZFS_SUPER_MAGIC 0x2fc12fc1
+#endif
+
 #ifdef ROCKSDB_FALLOCATE_PRESENT
 #include <cerrno>
 #endif
@@ -1384,16 +1396,34 @@ TEST_P(EnvPosixTestWithParam, AllocateTest) {
     struct stat f_stat;
     ASSERT_EQ(stat(fname.c_str(), &f_stat), 0);
     ASSERT_EQ((unsigned int)kDataSize, f_stat.st_size);
-    // btrfs accepts fallocate but uses copy-on-write, so preallocated extents
-    // are not reflected in st_blocks. Skip block-count verification there.
+    // btrfs (and other CoW filesystems) accept fallocate but use
+    // copy-on-write, so preallocated extents are not reliably reflected in
+    // st_blocks (especially under load). Skip block-count verification on
+    // those filesystems.
+    // Also skip on tmpfs and overlayfs, which may not report preallocated
+    // blocks in st_blocks reliably.
     bool skip_block_checks = false;
 #ifdef OS_LINUX
     struct statfs fs_stat;
-    if (statfs(fname.c_str(), &fs_stat) == 0 &&
-        fs_stat.f_type ==
-            static_cast<decltype(fs_stat.f_type)>(BTRFS_SUPER_MAGIC)) {
-      fprintf(stderr, "Skipping preallocation block count checks on btrfs\n");
-      skip_block_checks = true;
+    if (statfs(fname.c_str(), &fs_stat) == 0) {
+      if (fs_stat.f_type ==
+          static_cast<decltype(fs_stat.f_type)>(BTRFS_SUPER_MAGIC)) {
+        fprintf(stderr, "Skipping preallocation block count checks on btrfs\n");
+        skip_block_checks = true;
+      } else if (fs_stat.f_type ==
+                 static_cast<decltype(fs_stat.f_type)>(ZFS_SUPER_MAGIC)) {
+        fprintf(stderr, "Skipping preallocation block count checks on zfs\n");
+        skip_block_checks = true;
+      } else if (fs_stat.f_type ==
+                 static_cast<decltype(fs_stat.f_type)>(TMPFS_MAGIC)) {
+        fprintf(stderr, "Skipping preallocation block count checks on tmpfs\n");
+        skip_block_checks = true;
+      } else if (fs_stat.f_type ==
+                 static_cast<decltype(fs_stat.f_type)>(OVERLAYFS_SUPER_MAGIC)) {
+        fprintf(stderr,
+                "Skipping preallocation block count checks on overlayfs\n");
+        skip_block_checks = true;
+      }
     }
 #endif
     if (!skip_block_checks) {
@@ -1403,8 +1433,20 @@ TEST_P(EnvPosixTestWithParam, AllocateTest) {
       // expect.
       // It looks like some FS give us more blocks that we asked for. That's
       // fine. It might be worth investigating further.
-      ASSERT_LE((unsigned int)(kPreallocateSize / kBlockSize),
-                f_stat.st_blocks);
+      if ((unsigned int)(kPreallocateSize / kBlockSize) > f_stat.st_blocks) {
+        // Preallocation may not be supported or reflected in st_blocks on this
+        // filesystem. Print a warning and skip the check rather than failing.
+        fprintf(stderr,
+                "Warning: preallocated blocks (%u) less than expected (%u), "
+                "skipping block count check. This may indicate the filesystem "
+                "does not support preallocation or does not report it in "
+                "st_blocks.\n",
+                (unsigned int)f_stat.st_blocks,
+                (unsigned int)(kPreallocateSize / kBlockSize));
+      } else {
+        ASSERT_LE((unsigned int)(kPreallocateSize / kBlockSize),
+                  f_stat.st_blocks);
+      }
     }
 
     // close the file, should deallocate the blocks

include/rocksdb/blob_file_partition_strategy.h

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 // Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License

include/rocksdb/c.h

@@ -205,6 +205,8 @@ extern ROCKSDB_LIBRARY_API rocksdb_t* rocksdb_open(
 extern ROCKSDB_LIBRARY_API rocksdb_t* rocksdb_open_with_ttl(
     const rocksdb_options_t* options, const char* name, int ttl, char** errptr);
 
+/* If error_if_wal_file_exists is non-zero, returns an error when a non-empty
+ * WAL file exists. Empty WAL files are tolerated. */
 extern ROCKSDB_LIBRARY_API rocksdb_t* rocksdb_open_for_read_only(
     const rocksdb_options_t* options, const char* name,
     unsigned char error_if_wal_file_exists, char** errptr);
@@ -450,6 +452,8 @@ extern ROCKSDB_LIBRARY_API rocksdb_t* rocksdb_open_column_families_with_ttl(
     rocksdb_column_family_handle_t** column_family_handles, const int* ttls,
     char** errptr);
 
+/* If error_if_wal_file_exists is non-zero, returns an error when a non-empty
+ * WAL file exists. Empty WAL files are tolerated. */
 extern ROCKSDB_LIBRARY_API rocksdb_t*
 rocksdb_open_for_read_only_column_families(
     const rocksdb_options_t* options, const char* name, int num_column_families,
@@ -1818,6 +1822,11 @@ extern ROCKSDB_LIBRARY_API void rocksdb_options_set_recycle_log_file_num(
     rocksdb_options_t*, size_t);
 extern ROCKSDB_LIBRARY_API size_t
 rocksdb_options_get_recycle_log_file_num(rocksdb_options_t*);
+// Set/get DBOptions::async_wal_precreate.
+extern ROCKSDB_LIBRARY_API void rocksdb_options_set_async_wal_precreate(
+    rocksdb_options_t*, unsigned char);
+extern ROCKSDB_LIBRARY_API unsigned char
+rocksdb_options_get_async_wal_precreate(rocksdb_options_t*);
 extern ROCKSDB_LIBRARY_API void
 rocksdb_options_set_soft_pending_compaction_bytes_limit(rocksdb_options_t* opt,
                                                         size_t v);

include/rocksdb/db.h

@@ -172,6 +172,10 @@ class DB {
 
   // Open the database for read only.
   //
+  // If error_if_wal_file_exists is true, returns an error when a non-empty
+  // WAL file exists. Empty WAL files can be left behind by features such as
+  // async WAL precreation and are tolerated.
+  //
   static Status OpenForReadOnly(const Options& options, const std::string& name,
                                 std::unique_ptr<DB>* dbptr,
                                 bool error_if_wal_file_exists = false);
@@ -183,6 +187,10 @@ class DB {
   // to specify default column family. The default column family name is
   // 'default' and it's stored in ROCKSDB_NAMESPACE::kDefaultColumnFamilyName
   //
+  // If error_if_wal_file_exists is true, returns an error when a non-empty
+  // WAL file exists. Empty WAL files can be left behind by features such as
+  // async WAL precreation and are tolerated.
+  //
   static Status OpenForReadOnly(
       const DBOptions& db_options, const std::string& name,
       const std::vector<ColumnFamilyDescriptor>& column_families,

include/rocksdb/listener.h

@@ -674,16 +674,69 @@ class EventListener : public Customizable {
   // Note that the this function must be implemented in a way such that
   // it should not run for an extended period of time before the function
   // returns.  Otherwise, RocksDB may be blocked.
+  //
+  // WART: this callback is skipped during DB shutdown, so a compaction
+  // that commits during shutdown may never be observed by listeners.
   virtual void OnCompactionBegin(DB* /*db*/, const CompactionJobInfo& /*ci*/) {}
 
+  // A callback function for RocksDB which will be called when a registered
+  // RocksDB has finished writing a compaction job's output files, but
+  // *before* the manifest write commits the new Version (i.e. before
+  // input files are released and output files become visible to readers
+  // and the compaction picker). At this point the input files in
+  // `ci.input_files` still have `FileMetaData::being_compacted == true`,
+  // so no other thread can pick them up for a new compaction.
+  //
+  // This callback fires strictly between `OnCompactionBegin` and
+  // `OnCompactionCompleted` for the same compaction. It is the right
+  // place for listeners to clean up their own bookkeeping of which input
+  // files they think are being compacted: doing it here, rather than in
+  // `OnCompactionCompleted`, avoids a race where `OnCompactionBegin` for
+  // the same file fires before `OnCompactionCompleted` for the previous
+  // compaction. The default implementation is a no-op.
+  //
+  // Unlike `OnCompactionCompleted`, `ci.status` does not reflect the
+  // final outcome: the manifest commit has not yet occurred, so it
+  // does not capture commit failures. Other `CompactionJobInfo` quirks
+  // that also apply to `OnCompactionCompleted`: for trivial moves and
+  // FIFO deletion-only compactions, `ci.stats` is minimal (no
+  // `CompactionJob` runs), and FIFO deletions have no output files.
+  //
+  // As with `OnCompactionCompleted`, the DB mutex is released for the
+  // duration of this callback, but other manifest writers may be waiting,
+  // so cheap implementations are strongly preferred.
+  //
+  // WART: this callback is skipped during DB shutdown, so a compaction
+  // that commits during shutdown may never be observed by listeners.
+  //
+  // @param db a pointer to the rocksdb instance which just compacted a file.
+  // @param ci a reference to a CompactionJobInfo struct. 'ci' is released
+  //  after this function is returned, and must be copied if it is needed
+  //  outside of this function.
+  virtual void OnCompactionPreCommit(DB* /*db*/,
+                                     const CompactionJobInfo& /*ci*/) {}
+
   // A callback function for RocksDB which will be called whenever
   // a registered RocksDB compacts a file. The default implementation
   // is a no-op.
   //
+  // At the time this callback fires, the manifest commit has applied:
+  // output files in `ci.output_files` are live in the DB (visible to
+  // readers and the compaction picker, and possibly already picked up
+  // by another compaction), and input files in `ci.input_files` have
+  // been removed from the live Version with `being_compacted == false`
+  // (their on-disk files may still exist if held by snapshots / iterators
+  // / pending physical deletion; see `OnTableFileDeleted`). To observe
+  // state *before* input files are released, use
+  // `OnCompactionPreCommit` instead.
+  //
   // Note that this function must be implemented in a way such that
   // it should not run for an extended period of time before the function
   // returns. Otherwise, RocksDB may be blocked.
   //
+  // WART: this callback is skipped during DB shutdown, so a compaction
+  // that commits during shutdown may never be observed by listeners.
+  //
   // @param db a pointer to the rocksdb instance which just compacted
   //   a file.
   // @param ci a reference to a CompactionJobInfo struct. 'ci' is released

include/rocksdb/options.h

@@ -975,6 +975,17 @@ struct DBOptions {
   // Default: 0
   size_t recycle_log_file_num = 0;
 
+  // EXPERIMENTAL: If true, RocksDB asynchronously precreates the next WAL file
+  // so foreground memtable switching can usually avoid the filesystem latency
+  // of creating a new WAL. The precreated file is only reserved empty storage;
+  // it does not become a logical WAL and is not added to WAL tracking until it
+  // is consumed by a foreground WAL rotation.
+  //
+  // The option is sanitized to false when recycle_log_file_num is non-zero.
+  //
+  // Default: false
+  bool async_wal_precreate = false;
+
   // The manifest file is rolled over on reaching this limit AND the
   // space amp limit described in max_manifest_space_amp_pct. More trade-off
   // details there.

include/rocksdb/perf_context.h

@@ -106,7 +106,7 @@ struct PerfContextBase {
   uint64_t compressed_sec_cache_compressed_bytes;
 
   uint64_t block_checksum_time;    // total nanos spent on block checksum
-  uint64_t block_decompress_time;   // total nanos spent on block decompression
+  uint64_t block_decompress_time;  // total nanos spent on block decompression
   uint64_t block_decompress_count;  // total number of block decompressions
 
   uint64_t get_read_bytes;       // bytes for vals returned by Get

include/rocksdb/statistics.h

@@ -223,6 +223,16 @@ enum Tickers : uint32_t {
   GET_UPDATES_SINCE_CALLS,
   WAL_FILE_SYNCED,  // Number of times WAL sync is done
   WAL_FILE_BYTES,   // Number of bytes written to WAL
+  // Number of WAL rotations that consumed an async precreated WAL.
+  WAL_PRECREATE_HIT,
+  // Number of WAL rotations that found no async precreated WAL to consume.
+  WAL_PRECREATE_MISS,
+  // Number of WAL rotations that waited for an in-flight WAL precreation.
+  WAL_PRECREATE_WAITED,
+  // Total foreground wait time for in-flight WAL precreation.
+  WAL_PRECREATE_WAIT_MICROS,
+  // Number of async WAL precreation attempts that failed.
+  WAL_PRECREATE_FAILED,
 
   // Writes can be processed by requesting thread or by the thread at the
   // head of the writers queue.

include/rocksdb/utilities/sorted_run_builder.h

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License

include/rocksdb/version.h

@@ -13,7 +13,7 @@
 // minor or major version number planned for release.
 #define ROCKSDB_MAJOR 11
 #define ROCKSDB_MINOR 3
-#define ROCKSDB_PATCH 0
+#define ROCKSDB_PATCH 2
 
 // Make it easy to do conditional compilation based on version checks, i.e.
 // #if ROCKSDB_VERSION_GE(4, 5, 6)

java/Makefile

@@ -332,10 +332,7 @@ endif
 clean: clean-not-downloaded clean-downloaded
 
 clean-not-downloaded:
-	$(AM_V_at)rm -rf $(NATIVE_INCLUDE)
-	$(AM_V_at)rm -rf $(OUTPUT)
-	$(AM_V_at)rm -rf $(BENCHMARK_OUTPUT)
-	$(AM_V_at)rm -rf $(SAMPLES_OUTPUT)
+	$(AM_V_at)rm -rf $(NATIVE_INCLUDE) $(OUTPUT) $(BENCHMARK_OUTPUT) $(SAMPLES_OUTPUT)
 
 clean-downloaded:
 	$(AM_V_at)rm -rf $(JAVA_TEST_LIBDIR)

java/rocksjni/portal.h

@@ -5011,6 +5011,16 @@ class TickerTypeJni {
         return 0x53;
       case ROCKSDB_NAMESPACE::Tickers::WAL_FILE_BYTES:
         return 0x54;
+      case ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_HIT:
+        return -0x6C;
+      case ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_MISS:
+        return -0x6D;
+      case ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_WAITED:
+        return -0x6E;
+      case ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_WAIT_MICROS:
+        return -0x6F;
+      case ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_FAILED:
+        return -0x70;
       case ROCKSDB_NAMESPACE::Tickers::WRITE_DONE_BY_SELF:
         return 0x55;
       case ROCKSDB_NAMESPACE::Tickers::WRITE_DONE_BY_OTHER:
@@ -5509,6 +5519,16 @@ class TickerTypeJni {
         return ROCKSDB_NAMESPACE::Tickers::WAL_FILE_SYNCED;
       case 0x54:
         return ROCKSDB_NAMESPACE::Tickers::WAL_FILE_BYTES;
+      case -0x6C:
+        return ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_HIT;
+      case -0x6D:
+        return ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_MISS;
+      case -0x6E:
+        return ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_WAITED;
+      case -0x6F:
+        return ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_WAIT_MICROS;
+      case -0x70:
+        return ROCKSDB_NAMESPACE::Tickers::WAL_PRECREATE_FAILED;
       case 0x55:
         return ROCKSDB_NAMESPACE::Tickers::WRITE_DONE_BY_SELF;
       case 0x56:

java/src/main/java/org/rocksdb/RocksDB.java

@@ -404,7 +404,7 @@ public class RocksDB extends RocksObject {
    * @param options {@link Options} instance.
    * @param path the path to the RocksDB.
    * @param errorIfWalFileExists true to raise an error when opening the db
-   *            if a Write Ahead Log file exists, false otherwise.
+   *            if a non-empty Write Ahead Log file exists, false otherwise.
    * @return a {@link RocksDB} instance on success, null if the specified
    *     {@link RocksDB} can not be opened.
    *
@@ -493,7 +493,7 @@ public class RocksDB extends RocksObject {
    * @param columnFamilyHandles will be filled with ColumnFamilyHandle instances
    *     on open.
    * @param errorIfWalFileExists true to raise an error when opening the db
-   *            if a Write Ahead Log file exists, false otherwise.
+   *            if a non-empty Write Ahead Log file exists, false otherwise.
    * @return a {@link RocksDB} instance on success, null if the specified
    *     {@link RocksDB} can not be opened.
    *

java/src/main/java/org/rocksdb/TickerType.java

@@ -394,6 +394,31 @@ public enum TickerType {
      */
     WAL_FILE_BYTES((byte) 0x54),
 
+    /**
+     * Number of WAL rotations that consumed an async precreated WAL.
+     */
+    WAL_PRECREATE_HIT((byte) -0x6C),
+
+    /**
+     * Number of WAL rotations that found no async precreated WAL to consume.
+     */
+    WAL_PRECREATE_MISS((byte) -0x6D),
+
+    /**
+     * Number of WAL rotations that waited for an in-flight WAL precreation.
+     */
+    WAL_PRECREATE_WAITED((byte) -0x6E),
+
+    /**
+     * Total foreground wait time for in-flight WAL precreation.
+     */
+    WAL_PRECREATE_WAIT_MICROS((byte) -0x6F),
+
+    /**
+     * Number of async WAL precreation attempts that failed.
+     */
+    WAL_PRECREATE_FAILED((byte) -0x70),
+
     /**
      * Writes can be processed by requesting thread or by the thread at the
      * head of the writers queue.

java/src/test/java/org/rocksdb/ReadOnlyTest.java

@@ -6,6 +6,8 @@ package org.rocksdb;
 
 import static org.assertj.core.api.Assertions.assertThat;
 
+import java.io.IOException;
+import java.nio.file.Files;
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.List;
@@ -200,11 +202,12 @@ public class ReadOnlyTest {
   }
 
   @Test(expected = RocksDBException.class)
-  public void errorIfWalFileExists() throws RocksDBException {
+  public void errorIfWalFileExists() throws RocksDBException, IOException {
     try (final Options options = new Options().setCreateIfMissing(true);
          final RocksDB ignored = RocksDB.open(options, dbFolder.getRoot().getAbsolutePath())) {
       // no-op
     }
+    Files.write(dbFolder.getRoot().toPath().resolve("999999.log"), new byte[] {1});
 
     try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions()) {
       final List<ColumnFamilyDescriptor> cfDescriptors = Collections.singletonList(

java/src/test/java/org/rocksdb/RocksDBTest.java

@@ -1631,7 +1631,8 @@ public class RocksDBTest {
       try (final RocksDB db = RocksDB.open(options, dbPath)) {
         final RocksDB.LiveFiles livefiles = db.getLiveFiles(true);
         assertThat(livefiles).isNotNull();
-        assertThat(livefiles.manifestFileSize).isEqualTo(116);
+        assertThat(livefiles.manifestFileSize).isGreaterThanOrEqualTo(100L);
+        assertThat(livefiles.manifestFileSize).isLessThanOrEqualTo(300L);
         assertThat(livefiles.files.size()).isEqualTo(3);
         assertThat(livefiles.files.get(0)).isEqualTo("/CURRENT");
         assertThat(livefiles.files.get(1)).isEqualTo("/MANIFEST-000005");

monitoring/statistics.cc

@@ -122,6 +122,11 @@ const std::vector<std::pair<Tickers, std::string>> TickersNameMap = {
     {GET_UPDATES_SINCE_CALLS, "rocksdb.getupdatessince.calls"},
     {WAL_FILE_SYNCED, "rocksdb.wal.synced"},
     {WAL_FILE_BYTES, "rocksdb.wal.bytes"},
+    {WAL_PRECREATE_HIT, "rocksdb.wal.precreate.hit"},
+    {WAL_PRECREATE_MISS, "rocksdb.wal.precreate.miss"},
+    {WAL_PRECREATE_WAITED, "rocksdb.wal.precreate.waited"},
+    {WAL_PRECREATE_WAIT_MICROS, "rocksdb.wal.precreate.wait.micros"},
+    {WAL_PRECREATE_FAILED, "rocksdb.wal.precreate.failed"},
     {WRITE_DONE_BY_SELF, "rocksdb.write.self"},
     {WRITE_DONE_BY_OTHER, "rocksdb.write.other"},
     {WRITE_WITH_WAL, "rocksdb.write.wal"},

options/db_options.cc

@@ -315,6 +315,10 @@ static std::unordered_map<std::string, OptionTypeInfo>
          {offsetof(struct ImmutableDBOptions, recycle_log_file_num),
           OptionType::kSizeT, OptionVerificationType::kNormal,
           OptionTypeFlags::kNone}},
+        {"async_wal_precreate",
+         {offsetof(struct ImmutableDBOptions, async_wal_precreate),
+          OptionType::kBoolean, OptionVerificationType::kNormal,
+          OptionTypeFlags::kNone}},
         {"log_file_time_to_roll",
          {offsetof(struct ImmutableDBOptions, log_file_time_to_roll),
           OptionType::kSizeT, OptionVerificationType::kNormal,
@@ -770,6 +774,7 @@ ImmutableDBOptions::ImmutableDBOptions(const DBOptions& options)
       log_file_time_to_roll(options.log_file_time_to_roll),
       keep_log_file_num(options.keep_log_file_num),
       recycle_log_file_num(options.recycle_log_file_num),
+      async_wal_precreate(options.async_wal_precreate),
       table_cache_numshardbits(options.table_cache_numshardbits),
       WAL_ttl_seconds(options.WAL_ttl_seconds),
       WAL_size_limit_MB(options.WAL_size_limit_MB),
@@ -893,6 +898,8 @@ void ImmutableDBOptions::Dump(Logger* log) const {
   ROCKS_LOG_HEADER(
       log, "                   Options.recycle_log_file_num: %" ROCKSDB_PRIszt,
       recycle_log_file_num);
+  ROCKS_LOG_HEADER(log, "                    Options.async_wal_precreate: %d",
+                   async_wal_precreate);
   ROCKS_LOG_HEADER(log, "                        Options.allow_fallocate: %d",
                    allow_fallocate);
   ROCKS_LOG_HEADER(log, "                       Options.allow_mmap_reads: %d",

options/db_options.h

@@ -48,6 +48,8 @@ struct ImmutableDBOptions {
   size_t log_file_time_to_roll;
   size_t keep_log_file_num;
   size_t recycle_log_file_num;
+  // Immutable copy of DBOptions::async_wal_precreate.
+  bool async_wal_precreate;
   int table_cache_numshardbits;
   uint64_t WAL_ttl_seconds;
   uint64_t WAL_size_limit_MB;

options/options_helper.cc

@@ -101,6 +101,7 @@ void BuildDBOptions(const ImmutableDBOptions& immutable_db_options,
   options.log_file_time_to_roll = immutable_db_options.log_file_time_to_roll;
   options.keep_log_file_num = immutable_db_options.keep_log_file_num;
   options.recycle_log_file_num = immutable_db_options.recycle_log_file_num;
+  options.async_wal_precreate = immutable_db_options.async_wal_precreate;
   options.max_manifest_file_size = mutable_db_options.max_manifest_file_size;
   options.max_manifest_space_amp_pct =
       mutable_db_options.max_manifest_space_amp_pct;

options/options_settable_test.cc

@@ -431,6 +431,7 @@ TEST_F(OptionsSettableTest, DBOptionsAllFieldsSettable) {
       "strict_bytes_per_sync=true;"
       "enable_thread_tracking=false;"
       "recycle_log_file_num=0;"
+      "async_wal_precreate=true;"
       "create_missing_column_families=true;"
       "log_file_time_to_roll=3097;"
       "max_background_flushes=35;"

options/options_test.cc

@@ -148,6 +148,7 @@ TEST_F(OptionsTest, GetOptionsFromMapTest) {
       {"track_and_verify_wals_in_manifest", "true"},
       {"track_and_verify_wals", "true"},
       {"verify_sst_unique_id_in_manifest", "true"},
+      {"async_wal_precreate", "true"},
       {"fast_sst_open", "true"},
       {"max_open_files", "32"},
       {"max_total_wal_size", "33"},
@@ -332,6 +333,7 @@ TEST_F(OptionsTest, GetOptionsFromMapTest) {
   ASSERT_EQ(new_db_opt.track_and_verify_wals_in_manifest, true);
   ASSERT_EQ(new_db_opt.track_and_verify_wals, true);
   ASSERT_EQ(new_db_opt.verify_sst_unique_id_in_manifest, true);
+  ASSERT_EQ(new_db_opt.async_wal_precreate, true);
   ASSERT_EQ(new_db_opt.fast_sst_open, true);
   ASSERT_EQ(new_db_opt.max_open_files, 32);
   ASSERT_EQ(new_db_opt.max_total_wal_size, static_cast<uint64_t>(33));
@@ -906,6 +908,7 @@ TEST_F(OptionsTest, OldInterfaceTest) {
       {"track_and_verify_wals_in_manifest", "true"},
       {"track_and_verify_wals", "true"},
       {"verify_sst_unique_id_in_manifest", "true"},
+      {"async_wal_precreate", "true"},
       {"fast_sst_open", "true"},
       {"max_open_files", "32"},
       {"daily_offpeak_time_utc", "06:30-23:30"},
@@ -923,6 +926,7 @@ TEST_F(OptionsTest, OldInterfaceTest) {
   ASSERT_EQ(new_db_opt.track_and_verify_wals_in_manifest, true);
   ASSERT_EQ(new_db_opt.track_and_verify_wals, true);
   ASSERT_EQ(new_db_opt.verify_sst_unique_id_in_manifest, true);
+  ASSERT_EQ(new_db_opt.async_wal_precreate, true);
   ASSERT_EQ(new_db_opt.fast_sst_open, true);
   ASSERT_EQ(new_db_opt.max_open_files, 32);
   db_options_map["unknown_option"] = "1";
@@ -2489,6 +2493,7 @@ TEST_F(OptionsOldApiTest, GetOptionsFromMapTest) {
       {"track_and_verify_wals_in_manifest", "true"},
       {"track_and_verify_wals", "true"},
       {"verify_sst_unique_id_in_manifest", "true"},
+      {"async_wal_precreate", "true"},
       {"fast_sst_open", "true"},
       {"max_open_files", "32"},
       {"max_total_wal_size", "33"},
@@ -2678,6 +2683,7 @@ TEST_F(OptionsOldApiTest, GetOptionsFromMapTest) {
   ASSERT_EQ(new_db_opt.paranoid_checks, true);
   ASSERT_EQ(new_db_opt.track_and_verify_wals_in_manifest, true);
   ASSERT_EQ(new_db_opt.track_and_verify_wals, true);
+  ASSERT_EQ(new_db_opt.async_wal_precreate, true);
   ASSERT_EQ(new_db_opt.max_open_files, 32);
   ASSERT_EQ(new_db_opt.max_total_wal_size, static_cast<uint64_t>(33));
   ASSERT_EQ(new_db_opt.use_fsync, true);

table/block_based/block_util.h

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License

test_util/testutil.cc

@@ -315,6 +315,7 @@ void RandomInitDBOptions(DBOptions* db_opt, Random* rnd) {
   db_opt->use_adaptive_mutex = rnd->Uniform(2);
   db_opt->use_fsync = rnd->Uniform(2);
   db_opt->recycle_log_file_num = rnd->Uniform(2);
+  db_opt->async_wal_precreate = rnd->Uniform(2);
   db_opt->avoid_flush_during_recovery = rnd->Uniform(2);
   db_opt->avoid_flush_during_shutdown = rnd->Uniform(2);
   db_opt->enforce_single_del_contracts = rnd->Uniform(2);

tools/db_bench_tool.cc

@@ -1916,6 +1916,8 @@ DEFINE_bool(track_and_verify_wals_in_manifest, false,
 
 DEFINE_bool(track_and_verify_wals, false, "See Options.track_and_verify_wals");
 
+DEFINE_bool(async_wal_precreate, false, "See Options.async_wal_precreate");
+
 DEFINE_int32(same_value_percentage, 0,
              "Percentage of time value will be same i.e good for compression "
              "of the block");
@@ -5063,6 +5065,7 @@ class Benchmark {
     options.track_and_verify_wals_in_manifest =
         FLAGS_track_and_verify_wals_in_manifest;
     options.track_and_verify_wals = FLAGS_track_and_verify_wals;
+    options.async_wal_precreate = FLAGS_async_wal_precreate;
 
     // Integrated BlobDB
     options.enable_blob_files = FLAGS_enable_blob_files;

tools/db_crashtest.py

@@ -228,6 +228,7 @@ default_params = {
     "nooverwritepercent": 1,
     "open_files": lambda: random.choice([-1, -1, 100, 500000]),
     "open_files_async": lambda: random.choice([0, 1]),
+    "async_wal_precreate": lambda: random.choice([0, 1]),
     "optimize_filters_for_memory": lambda: random.randint(0, 1),
     "partition_filters": lambda: random.randint(0, 1),
     "partition_pinning": lambda: random.randint(0, 3),

tools/run_clang_tidy.py

@@ -1,3 +1,7 @@
+#  Copyright (c) Meta Platforms, Inc. and affiliates.
+#  This source code is licensed under both the GPLv2 (found in the COPYING file in the root directory)
+#  and the Apache 2.0 License (found in the LICENSE.Apache file in the root directory).
+
 #!/usr/bin/env python3
 """
 Run clang-tidy on locally changed code and filter results to changed lines.

unreleased_history/behavior_changes/write_committed_merge_operand_filtering

@@ -1 +0,0 @@
-WriteCommitted TransactionDB now matches WritePrepared and WriteUnprepared compaction filtering in both single- and two-write-queue modes: a compaction filter's FilterMergeOperand will not be invoked on merge operands at or below the latest published sequence number.

unreleased_history/bug_fixes/blob_backed_wide_column_merge_reads.md

@@ -1,3 +0,0 @@
-Fixed blob-backed wide-column merge reads to preserve correct status
-propagation and resolution across memtable, read-only, and secondary DB
-paths.

unreleased_history/bug_fixes/get_creation_time_of_oldest_file_async_open

@@ -1 +0,0 @@
-Fixed a bug where `DB::GetCreationTimeOfOldestFile()` could return inaccurate results instead of the real creation time when called shortly after opening a legacy DB (one whose manifest lacks `file_creation_time`) with `open_files_async = true`. The API now waits for background SST file loading to complete only when needed; modern DBs are unaffected.

unreleased_history/bug_fixes/merge_reads_preserve_precise_statuses.md

@@ -1 +0,0 @@
-Fixed merge reads against wide-column/blob-backed base values to preserve precise failure statuses, including `GetMergeOperands()` and direct-write memtable reads.

unreleased_history/bug_fixes/range_tombstone_synthesis

@@ -1 +0,0 @@
-Fix bug in range tombstone synthesis that covers live keys added during an IngestExternalFile

unreleased_history/bug_fixes/reject_empty_cf_name.md

@@ -1 +0,0 @@
-Reject the empty string as a column family name in `DB::CreateColumnFamily` / `DB::CreateColumnFamilies`. Previously such calls returned OK and a usable handle, but the column family was not persisted in the manifest, so any data written to it was silently lost on DB reopen.

unreleased_history/bug_fixes/write_committed_wbwi_ingestion_missing_entry

@@ -1 +0,0 @@
-Fixed a bug where a WriteCommitted TransactionDB using commit-bypass WBWI ingestion could drop an entry that is still visible at the published sequence boundary.

unreleased_history/new_features/optimize_manifest_for_recovery.md

@@ -1 +0,0 @@
-Add mutable DBOption `optimize_manifest_for_recovery` (default false). When enabled, RocksDB can reduce recovery work after a clean shutdown, which may lower DB::Open latency on warm reopens.

unreleased_history/new_features/parse_compression_name_for_display.md

@@ -1 +0,0 @@
-Added public utility APIs `ParseCompressionNameForDisplay()` to convert `TableProperties::compression_name` into a human-readable compression name for both legacy and format_version 7+ SST metadata, including custom `CompressionManager`-provided display names for custom compression types.

unreleased_history/new_features/reuse_manifest_on_open.md

@@ -1 +0,0 @@
-Add `reuse_manifest_on_open` DBOption (default false). When enabled, DB::Open reuses the existing MANIFEST file for append instead of creating a fresh one, avoiding the cost of serializing the entire database state into a new MANIFEST on the first post-open write.

util/interval_test.cc

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License

utilities/fault_injection_fs_test.cc

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License

utilities/sorted_run_builder/sorted_run_builder.cc

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License

utilities/sorted_run_builder/sorted_run_builder_test.cc

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License

utilities/transactions/write_prepared_transaction_seqno_test.cc

@@ -1,3 +1,8 @@
+//  Copyright (c) Meta Platforms, Inc. and affiliates.
+//  This source code is licensed under both the GPLv2 (found in the
+//  COPYING file in the root directory) and Apache 2.0 License
+//  (found in the LICENSE.Apache file in the root directory).
+
 //  Copyright (c) 2011-present, Facebook, Inc.  All rights reserved.
 //  This source code is licensed under both the GPLv2 (found in the
 //  COPYING file in the root directory) and Apache 2.0 License