mirror of
https://github.com/facebook/rocksdb.git
synced 2026-11-06 02:52:35 -05:00
main
1055
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Peter Dillinger
|
9d3733d9d8 |
Accurate SST file times, plus DB ownership metadata for SST files (#15583)
Summary: Ideally, each age-based feature acts on the time that matches its purpose, which requires tracking three different times for each SST file: * How old its data is (when its keys were written, as bounded by the oldest ancestor time and the `newest_key_time` table property), so that TTL expires data based on its actual age. * When the file's contents were built, i.e. when its data last went through a flush or compaction (`file_creation_time`). Periodic compaction (commonly used so that all data regularly goes through a compaction filter) and `GetCreationTimeOfOldestFile()` should act on this. * When this DB began holding the file. `bottommost_file_compaction_delay` should act on this: it limits how soon a recently written bottommost file is rewritten alone, after snapshot releases, to zero its sequence numbers or drop data no longer protected, so that a series of snapshot releases does not rewrite the same data repeatedly. External storage accounting (how long each file occupies storage) also needs this time. Knowing how the DB got each file (built it, or ingested or imported it) is also useful, e.g. for storage accounting. Previous behavior: A file's age for periodic compaction comes from its MANIFEST `file_creation_time`, then (only if the table reader is pinned) the `file_creation_time` table property, then the oldest ancestor time, then the file's mtime (unchanged by this commit). However: * Files written by `SstFileWriter` had table property `file_creation_time` = 0 (unknown). * Files flushed during WAL recovery had `file_creation_time` = 0 in both the table properties and the MANIFEST. * `RepairDB` restored each file's oldest ancestor time from its table properties but not its `file_creation_time`, so every repaired file lost its MANIFEST creation time. * `IngestExternalFile()` and `CreateColumnFamilyWithImport()` set both `file_creation_time` and the oldest ancestor time to the ingest/import time, ignoring times carried by the file's table properties or the export metadata. Data written or last compacted long before ingestion was therefore not due for periodic compaction (or TTL) until a full period after ingestion. Also, `Checkpoint::ExportColumnFamily()` left the time fields of its export metadata unset. * A FIFO temperature change done as a trivial copy (with `allow_trivial_copy_when_change_temperature`) recorded the copy time as the new file's MANIFEST `file_creation_time`, although its contents and table properties are the original's. * FIFO TTL compaction picks expired files oldest first, judging each by the `newest_key_time` table property or else the oldest ancestor time table property. `SstFileWriter` files have neither, so an ingested `SstFileWriter` file stopped the scan: neither it nor any newer file was ever deleted by TTL. * Nothing recorded when the DB began holding a file, or whether it was ingested, imported, or built by this DB. (For ingested and imported files, `file_creation_time` happened to be the time they were added, which `bottommost_file_compaction_delay` used.) New behavior, for when contents were built: * The block-based table builder now records the `file_creation_time` property itself, reading the clock when it finishes the file, so that the time is no earlier than when any entry was added. So every block-based SST file carries one (unless reading the clock fails), including `SstFileWriter` files, whose entries can be added over a long time. (For flush and compaction outputs, the MANIFEST value is still read when the output file is started, so it can be slightly earlier than the property.) The WAL recovery flush now also sets the MANIFEST value, and `RepairDB` restores it from the table property. * Ingestion and import keep the creation and oldest ancestor times carried by the file when known: for ingestion, from its table properties; for import, from the export metadata, or else the file's table properties. A carried time later than the current time suggests an unreliable clock, so it is logged as a WARNING and replaced with the current time; an unknown time becomes the current time. When no oldest ancestor time is carried (true of all `SstFileWriter` files), it falls back to the resolved creation time, an upper bound on when the data was written, so TTL never fires early. `ExportColumnFamily()` now fills in both time fields. * A FIFO temperature-change copy keeps the original's `file_creation_time`. * FIFO TTL compaction falls back to `file_creation_time` when a file has neither of the table properties above. Every key in a file was written no later than the file's contents were built, so this never deletes early. * New option `reject_too_old_file` (default false) in `IngestExternalFileOptions` and `ImportColumnFamilyOptions` fails the whole ingestion/import with `InvalidArgument`, before any file is linked or copied, if any file is already too old for the column family to keep without recompaction or deletion: for leveled and universal compaction, if its known creation time is older than the effective `periodic_compaction_seconds`; for FIFO compaction, if its newest data (the `newest_key_time` table property when known, else its creation time) is older than `ttl`. Such data should already have been recompacted (e.g. through a compaction filter) or deleted. Leveled `ttl` is not considered because it does not limit how long data can be kept, only how long it stays above the last level. Files without the needed time are not rejected. C and Java bindings included. New behavior, for when the DB began holding a file: * New `SstFileMetaData::db_ownership_start_time` (also in `LiveFileMetaData` and export metadata), backed by a new MANIFEST field: the time this DB began holding the physical file (file number). That is when the DB started writing it (flush or compaction output, local or remote), when ingestion linked or copied it into the DB (in the prepare phase, for two-phase ingestion), when import did, or when a FIFO temperature-change copy wrote it. Moves between levels preserve it. A long compaction's outputs start when they are created, not when the compaction finishes. * New `SstFileMetaData::db_ownership_info`, decoded by the new public struct `DbOwnershipInfo`, backed by another MANIFEST field: how the DB got the contents (flush, compaction, ingestion, or import) plus sticky flags for things that have happened to the file without its contents being rewritten (moved between levels; copied to a new temperature; temperature updated from the file system by `UpdateManifestForFilesState()`; recorded by `RepairDB`). * `bottommost_file_compaction_delay` is now measured from `db_ownership_start_time` (falling back to `file_creation_time` when unknown). For files built by the DB that is the same as before; ingested files again get the delay from when they were ingested, rather than skipping it because of old contents (which only matters for ingested DB-generated files, since normally ingested `SstFileWriter` files have sequence number 0 and are never marked). Neither new field is a table property: they describe one DB's relationship to a file, and the same file contents can be added to several DBs. Times not recorded, for lack of a consumer: when the DB first held the contents across temperature copies, when a file became visible (MANIFEST commit time), and when a file arrived at its current level. Implementation notes: * `TableBuilderOptions::file_creation_time` (and its positional constructor parameter) is removed, since the builder now reads the clock itself. All callers passing arguments past that position were updated. * `ResolveExternalFileTimes()` and `CheckExternalFileNotTooOld()` are shared by ingestion and import. * The new MANIFEST fields use new safe-to-ignore `NewFile` tags, written only when known. Their packed bit layout is defined next to the tags; unknown bits are ignored on read and preserved when a file's record is carried over. * New `VersionEdit::AddExistingFile()` re-adds a file under the same file number with all of its persisted metadata. It replaces field-by-field copies into the positional `AddFile()` at the five places that move files (trivial move, `CompactRange()` with `change_level`, `PromoteL0()`, the level migration at open, and `UpdateManifestForFilesState()`) and in the MANIFEST snapshot writer, so MANIFEST-only fields, including future ones, carry over. The WAL recovery flush, `RepairDB`, and import now use the `FileMetaData` overload of `AddFile()`; as a side effect, recovery flush outputs now also record min/max timestamps and file open metadata in the MANIFEST, as regular flush outputs do. Compatibility and limitations: * Behavior change: ingesting or importing old data can make it due for periodic compaction or TTL soon afterward rather than a full period later, so bulk-ingesting old SSTs can cause a burst of compactions. Users who must not publish such data can set `reject_too_old_file`. * Files already in a DB keep their MANIFEST entries; the new fields are unknown (0) for them. For files ingested, imported, or temperature-copied by older versions, `file_creation_time` was the ingestion, import, or copy time, so the `bottommost_file_compaction_delay` fallback is accurate for them. Files from older `SstFileWriter` versions carry no creation time and still get the ingestion time. * Older RocksDB versions ignore the new MANIFEST fields and drop them when rewriting the MANIFEST. A DB opened from a checkpoint or backup keeps the values recorded by the DB it was copied from. `RepairDB` loses them for files that existed before the repair. * Plain and cuckoo table builders still record no time properties. * The times recorded for remote compaction outputs come from the remote worker's clock. * Possible follow-ups: letting `SstFileWriter` callers supply an authentic oldest ancestor time; and avoiding bottommost-files compactions that rewrite an ingested `SstFileWriter` file only to zero a global sequence number that exists only in the MANIFEST (found by code reading, not yet confirmed by a test). Pull Request resolved: https://github.com/meta/rocksdb/pull/15583 Test Plan: New tests, using mock clocks where time matters: * `DBTablePropertiesTest.FileCreationTimeAlwaysSet`: `SstFileWriter` file plus ingestion, WAL recovery flush, and `RepairDB` all record a creation time, checked in both table properties and the raw MANIFEST `FileMetaData` (`GetLiveFilesMetaData()` would mask a missing MANIFEST value by falling back on pinned table properties). * `ExternalSSTFileTimeTest.*` (external_sst_file_test): `KeepRecordedTimes` covers recorded, unknown, and future creation times; `KeepDBGeneratedFileTimes` checks that `allow_db_generated_files` keeps both times from the source DB; `PeriodicCompactionFromWriteTime` checks that periodic compaction triggers relative to when an `SstFileWriter` file was written, not when it was ingested; `RejectTooOldFile` covers leveled, universal, and FIFO: a batch containing a too-old file is rejected with the DB unchanged, a file just under the limit and a file with unknown creation time are accepted, and the option off accepts the too-old file; `RejectFIFOFileWithOldNewestKey` checks that for FIFO, a recently built file whose newest key time is older than `ttl` is rejected; `CreationTimeIsWhenFileIsFinished` checks that an `SstFileWriter` file's creation time is when it was finished, not opened; `FIFOTtlDeletesIngestedFile` checks that an ingested `SstFileWriter` file and a newer flushed file expire by TTL; `RecordsDbOwnership` checks the ownership start time and origin for one-phase ingestion, two-phase ingestion (from the prepare phase, not the commit), and ingestion of a DB-generated file; `BottommostCompactionDelayFromIngestion` checks that an ingested DB-generated file with old contents is not compacted as a bottommost file until the delay has passed since ingestion. * `ImportColumnFamilyTest.ImportKeepsExportedFileTimes` and `ImportFileTimesAndRejectTooOldFile`: same coverage for export/import, plus falling back on the file's table properties when the import metadata has no time, and ownership metadata in the export and after import. * `DBOwnershipTest.*` (db_etc3_test): ownership start time and origin of flush, WAL recovery flush, and compaction outputs (including a compaction delayed after creating its output), and in public metadata; preservation, with the "moved" flag, through `PromoteL0()`, trivial move, `CompactRange()` with `change_level`, the level migration at open, and reopens (MANIFEST replay and snapshot); FIFO temperature-change copy (same creation time, new ownership start time, "temperature copied" flag); `UpdateManifestForFilesState()`; and `RepairDB` (unknown ownership for pre-existing tables, flush origin for tables built from WALs). * `CompactionServiceTest.BasicCompactions`: remote compaction outputs record ownership like local ones. * `VersionEditTest.*`: MANIFEST encoding of the new fields (round trip, preserved reserved bits, written only when known, safe to ignore, debug output), the persisted `DbOwnershipInfo` bit layout, `AddExistingFile()`, and the fallback for an unknown ownership start time. * Java getter/setter tests for the new option. The JNI code was compiled against `javac`-generated headers, but the Java tests were not run locally. `FIFOTtlDeletesIngestedFile` was verified to fail before the FIFO TTL fix. `BottommostCompactionDelayFromIngestion`, `FIFOTemperatureChangeCopy`, `MovesPreserveOwnership`, `TemperatureUpdatedFromFileSystem`, and `AddExistingFile` were verified to fail with, respectively, the delay measured from `file_creation_time`, the copy time used as creation time, and `AddExistingFile()` dropping the ownership start time. `CreationTimeIsWhenFileIsFinished`, `RejectFIFOFileWithOldNewestKey`, and `ImportFileTimesAndRejectTooOldFile` were verified to fail with, respectively, the creation time read when the builder is created, FIFO rejection based only on creation time, and import ignoring the file's table properties. The creation-time tests were verified to fail with the new logic stubbed out to the old behavior before rebasing onto current main, and `ImportKeepsExportedFileTimes` also fails with only the `ExportColumnFamily()` fix reverted. Adjusted `ExternalSSTFileBasicTest.AtomicReplaceRangeAvoidsRedundantTimestampScans`: it models legacy files with no recorded creation time and relied on `SstFileWriter` files having a zero `file_creation_time` property (pinned readers fall back on it). It now writes those files with a writer clock that reads 0. Reviewed By: xingbowang Differential Revision: D129668538 Pulled By: pdillinger fbshipit-source-id: 1e8ad8e9738d7b0427c408d693bc22c46deca913 |
||
 Michael C Huang
|
a51c5e21ec |
Repair missing blob references during manual compaction (#15538)
Summary: Pull Request resolved: https://github.com/meta/rocksdb/pull/15538 What changed? - Adds `CompactRangeOptions::skip_missing_blob_files`, default `false`, for manual repair when referenced blob files are absent from the compaction's input Version. https://pxl.cl/j8V47 - Rewrites selected SSTs. Lost scalar values become deletions. Wide entities keep surviving columns. - Manual `CompactRange` only. Runs locally; compaction services are bypassed. FIFO is unsupported. Why? Best efforts recovery can leave dangling blob references in SSTs. These can cause later compactions to fail. Offline repair prepares the DB for normal reopening. Reviewed By: xingbowang Differential Revision: D128836898 fbshipit-source-id: 3acdf3b2deb3193dd181e748522d85e8efd64d97 |
||
 Steph Pontikes
|
d0e6a4ca27 |
Add a dedicated READ Env thread pool (#15576)
Summary: Pull Request resolved: https://github.com/meta/rocksdb/pull/15576 Add `Env::Priority::READ` as a schedulable pool and keep `USER` as a metrics-only priority category. Propagate the new pool through POSIX, Windows, thread-status, and Java/JNI APIs. Reviewed By: xingbowang Differential Revision: D129424000 fbshipit-source-id: 9dcf7c25ac6be6ec8931072dba2ad0bab8146300 |
||
 Chaitanya Datye
|
75e965d2a6 |
Validate table factory options when creating a column family (#15514)
Summary: Pull Request resolved: https://github.com/meta/rocksdb/pull/15514 This makes `CreateColumnFamily()` and `CreateColumnFamilies()` run the same table factory option validation that `DB::Open()` runs, and makes the external table adapter forward that validation to the factory it wraps. `DB::Open()` validates each column family twice: `ColumnFamilyData::ValidateOptions()` checks the plain column family options, and `ValidateOptions()` in `options_helper` walks the configurable objects, which is where `TableFactory::ValidateOptions()` runs. `CreateColumnFamiliesImpl()` only ran the first. A column family whose options the table factory rejects, for example a block-based table with a hash index and no prefix extractor, was therefore created and persisted, and the next `DB::Open()` failed on it. Validating at creation turns that into an `InvalidArgument` from `CreateColumnFamily()`. The external table adapter returned by `NewExternalTableFactory()` did not override `ValidateOptions()`, and `Configurable::ValidateOptions()` only checks registered options, not `Inner()`, so an external table factory had no way to reject a column family it cannot serve. This matters for ingestion-only factories whose builders fail every write: in a column family without `disallow_memtable_writes`, one stray `Put` makes the next flush fail with a background error that stops writes to the whole DB. Such a factory can now override `ValidateOptions()` to require `disallow_memtable_writes`, and with the first change the check covers column families created at runtime as well as those opened with the DB. This is a behavior change for callers that create column families with options the table factory rejects. Those calls used to succeed and leave a DB that failed to reopen; they now fail up front. A release note is included. Changes: 1. `CreateColumnFamiliesImpl()` calls `ValidateOptions(db_options, cf_options)` after `ColumnFamilyData::ValidateOptions()`. 2. `ExternalTableFactoryAdapter::ValidateOptions()` validates its own options and then calls the wrapped factory's `ValidateOptions()`. 3. `ColumnFamilyTest.CreateRejectsOptionsThatOpenRejects` checks that a block-based hash index without a prefix extractor is rejected at creation and that the DB still reopens. 4. RocksJava's `OptionsUtilTest` created a column family whose block-based options `DB::Open()` rejects (`cache_index_and_filter_blocks` and `pin_l0_filter_and_index_blocks_in_cache` with `no_block_cache`, and `block_align` with compression and a 35 KiB block size); it only round-trips the options file, so it never noticed. It now uses a block cache, a 32 KiB block size and no compression for that column family, keeping every other non-default table option it verifies. 5. `ExternalTableTest.WrappedFactoryValidatesColumnFamilyOptions` checks that an external factory's `ValidateOptions()` is reached directly, through `CreateColumnFamily()` and through `DB::Open()`. Reviewed By: xingbowang Differential Revision: D128151632 fbshipit-source-id: 348e7820aa3995ddfa798b02bb38dddb8528e76c |
||
|
Peter Dillinger
|
0adcea5ea4 |
Update for GitHub repository move to meta/rocksdb (#15508)
Summary: The RocksDB GitHub repository moved from github.com/facebook/rocksdb to github.com/meta/rocksdb. Nearly all GitHub Actions jobs are gated on `github.repository_owner == 'facebook'` (to keep them from running on forks), so after the move those jobs were skipped on new PRs and pushes, and in the nightly and weekly schedules. This change updates the repository for the new location: * Fix: All `repository_owner` checks in the workflows now compare against 'meta', so the PR, nightly, weekly, and clang-tidy jobs run again. * Change: CI container images now come from ghcr.io/meta/rocksdb_ubuntu, and the Dockerfile build/push instructions match. * Change: Workflow names facebook/rocksdb/* are now meta/rocksdb/*. The Claude review workflow triggers on the pr-jobs workflow by name, so its `workflow_run` trigger is updated to match. (Required status checks are keyed by job name, not workflow name.) * Change: Every github.com/facebook/rocksdb URL in the repo now uses github.com/meta/rocksdb. This covers URLs used by CI and build scripts (third-party-deps release downloads, including DEPS_URL for Java builds; git remotes in setup-upstream and check_format_compatible.sh), the docs site config (`ghrepo`) and navigation, Maven SCM metadata, and links in code comments, docs, HISTORY.md, and blog posts. Historical links are included because GitHub's redirect from the old URL is not dependable for every link: some old issue URLs return 404 while the same issue under meta/rocksdb works. * Change: build_tools/format-diff.sh finds the upstream remote by matching either meta/rocksdb.git or facebook/rocksdb.git, because existing clones still use the old remote URL (which GitHub redirects for git operations). * Remove: The stale CircleCI status badge from README.md. CircleCI is no longer used. * Add: A final step in nightly's build-format-compatible job that fails if https://github.com/facebook/rocksdb stops redirecting to https://github.com/meta/rocksdb (or the old name is taken by another repo). Existing releases (e.g. DEPS_URL in java/Makefile as of v11.12), existing clones, and external links still depend on that redirect, so this alerts the RocksDB team if it breaks. The step runs even when the compatibility test fails. Not addressed here: runner labels (16-core-ubuntu, etc.) and secrets are configured per GitHub org, outside the repo. Any that are missing in the meta org need to be set up there. Pull Request resolved: https://github.com/meta/rocksdb/pull/15508 Test Plan: CI on this PR exercises the main fix: jobs should run instead of being skipped. Manual checks from an unrestricted network: * https://github.com/facebook/rocksdb returns a single 301 to https://github.com/meta/rocksdb, then 200, and `git ls-remote` via the old URL works. Old pull, wiki, blob, and release-download URLs redirect, but old issue URLs for https://github.com/meta/rocksdb/issues/9275, https://github.com/meta/rocksdb/issues/10080, and https://github.com/meta/rocksdb/issues/13293 returned 404 (https://github.com/meta/rocksdb/issues/12038 redirected) even though all four exist under meta/rocksdb. * A CI log showed ghcr.io/meta/rocksdb_ubuntu:22.1 pulling successfully. The new nightly step won't run before merge. The exact curl command it uses printed the expected `200 https://github.com/meta/rocksdb` when run manually. Separately, its script was extracted from the parsed workflow YAML and run against a stub curl: it passes for a redirect to meta, fails for a 404 and for a 200 without redirect, and reports a network failure distinctly. After merge, a manual workflow_dispatch of the nightly workflow can confirm it in CI. Reviewed By: archang19 Differential Revision: D128195050 Pulled By: pdillinger fbshipit-source-id: 3d42aa271b706eec2ef6c1586643056084fc866f |
||
 Bikash Chandra
|
83747eb79f |
Support BlobDB in remote compaction (#15496)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15496 Remote compaction previously transferred only SST inputs and output metadata. A worker could not reliably read blob-backed values for a compaction filter, output SST metadata did not preserve references to existing blob files, and blob garbage collection could derive a different cutoff from the worker view. This change adds BlobDB support to the existing remote-compaction path. Existing `enable_blob_files` and `blob_file_starting_level` settings determine whether eligible output may create blobs; compactions whose input SSTs reference blobs transport those dependencies regardless of new-blob eligibility. Each subcompaction transports the live blob-file suffix referenced by overlapping input SSTs. For blob-eligible output, the primary atomically reserves a half-open blob-file-number range. The worker uses native `CompactionJob`, `BlobFileBuilder`, and `BlobGarbageMeter` machinery and returns `BlobFileAddition`, `BlobFileGarbage`, and SST dependency metadata. When blob GC is enabled, the primary computes the cutoff from its authoritative input Version and serializes it in `CompactionServiceInput`; the worker consumes that cutoff rather than recomputing from its local view. Workers advertise result-side BlobDB and GC-cutoff protocol markers. The primary declines old or non-capable results before installation. It validates reserved additions, transported-input garbage, output dependencies, cross-subcompaction aggregate garbage, staged file existence, optional content checksums, and each staged blob footer count plus the overflow-safe header+record-bytes+footer physical size. Metadata or structural corruption declines the result and reruns locally, while genuine validation I/O errors remain `kFailure`. Blob files move before SSTs and register through `BlobFileCompletionCallback`; rollback physically deletes each installed blob before removing its `SstFileManager` entry, and a cleanup error remains a failure with the tracking entry preserved. Resumable workers checkpoint blob additions, cumulative garbage, and the next reserved number. Checkpoints cut SST and blob outputs before preparing the next key. `CompactionIterator` records when processing has already consumed an earlier input entry of the current user key, and progress is not checkpointed at that unsafe boundary. Resume therefore does not seek back over already-counted input, fixing both the baseline record-count mismatch and blob-garbage double counting. At safe boundaries, the raw pre-filter inflow for the current entry is subtracted from the blob-garbage checkpoint; per-entry inflow metering exists only for resumable jobs. `REMOTE_COMPACT_INPUT_BLOB_FILE_COUNT` counts each distinct transported blob file once per compaction job. `REMOTE_COMPACT_BLOB_RESULT_DECLINED_COUNT` counts primary-side blob-result declines only; service-side `kUseLocal` and pre-dispatch reservation fallback do not increment it. Both tickers are exposed through C++ only; JNI and Java bindings are intentionally omitted because the Java ticker jbyte space is exhausted. Compatibility and failure behavior: New fields deserialize with safe defaults. A new primary declines an old worker when blob inputs, blob-eligible output, or a primary-provided GC cutoff requires the new protocol. A new worker receiving an old input without the cutoff field disables blob GC for that job, avoiding a worker-local cutoff. Primary-side declines rerun locally, so `CompactRange` and `CompactFiles` succeed without a background error. The `CompactionService` contract is unchanged: a service returning `kFailure` preserves the worker status, including `Status::Incomplete`; mapping a worker decline to `kUseLocal` remains the integrator's choice. ZippyDB production additionally maps service failures to local fallback through `HandleFailure(FALLBACK_LOCAL)`. Reviewed By: jaykorean Differential Revision: D122097088 fbshipit-source-id: 7790768fe8a25dd2451a5d69d809d356899cabab |
||
|
Peter Dillinger
|
539532377d |
Remove Solaris support (#15449)
Summary: Solaris support was contributed by outside users starting with |
||
 Xingbo Wang
|
baa4d373a6 |
Commit authoritative OPTIONS file numbers in MANIFEST (#15305)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15305 Why the change is required: Column-family topology is durable in MANIFEST, while its operational configuration snapshot is stored in a separate OPTIONS file. CreateColumnFamily and DropColumnFamily previously committed MANIFEST first and wrote OPTIONS afterward. An OPTIONS I/O failure could therefore make the API report failure after the topology change was already durable. Persisted-options readers also selected the highest file number and could consume an orphan left by a failed publication. Previous behavior: RocksDB had no durable correspondence between a MANIFEST state and the OPTIONS snapshot describing it. LoadLatestOptions and GetLatestOptionsFileName treated the highest-numbered OPTIONS file as authoritative. Column-family creation and deletion returned the later OPTIONS persistence error even when their MANIFEST edit had already succeeded. Multi-column-family operations wrote a complete OPTIONS snapshot after every successful member, producing quadratic OPTIONS bytes. Ordinary DB::Open consumes caller-supplied DB and column-family options rather than loading OPTIONS. This change preserves that contract. New behavior: Add the default-disabled experimental DB option track_options_file_number_in_manifest. Its value is serialized into each OPTIONS file so the selection rule is self-describing and participates in the normal DB-options compatibility checks. When enabled, RocksDB first durably creates and renames OPTIONS-N and then commits N with one safely ignorable MANIFEST field. A normal OPTIONS rotation uses one MANIFEST commit. Column-family creation and deletion prewrite the proposed snapshot and commit its number together with a topology edit in the same VersionEdit. A failure before the OPTIONS file is durable changes no topology. A failure after the file is durable but before MANIFEST commit leaves an orphan that binding-aware readers ignore, while the previous committed snapshot remains authoritative. Each multi-CF create or drop writes one proposed snapshot, independent of the batch size. Create commits the new number with the first successful add; drop commits it with the last successful drop. Existing partial-success semantics can therefore leave a committed snapshot with extra CFs, but never one missing a live MANIFEST CF. Persisted-options APIs filter this safe superset against recovered MANIFEST membership. A snapshot missing a live CF is corruption. DB::Open with create_missing_column_families uses the same batched path. If the newest OPTIONS file has tracking disabled, it retains the legacy highest-numbered-file behavior, including after an old or disabled writer has run. If it has tracking enabled, readers use the recovered MANIFEST commit and reject newer tracked orphans. A missing usable commit falls back to the highest tracked snapshot for RepairDB and legacy-rewrite recovery. DB open logs that fallback through its configured logger; logger-less utility APIs remain silent. Writable open removes rejected tracked orphans when local obsolete-file deletion is enabled. Read-only open never mutates the DB directory. A DB with no recovered commit and tracking disabled takes the legacy fast path without parsing the newest OPTIONS file. When inspection is required, RocksDB reads only the tracking marker. Marker-read failures used for selection do not fail DB open; a recovered commit remains authoritative and ambiguous newer candidates are conservatively retained. The first topology change after enabling tracking first binds the current topology, so failure of the proposed change cannot make its orphan authoritative. MANIFEST failure during an ordinary tracked OPTIONS rotation also enters the existing manifest-write background-error path. Implementation: - Add the safely ignorable CommittedOptionsFileNumber VersionEdit field and carry it through recovery, MANIFEST snapshots, rollover, and subset checkpoints. - Persist the tracking marker through C++, C, and Java options, generated round-trip coverage, compatibility checking, and db_stress configuration. - Centralize binding-aware selection for DB open and public persisted-options APIs. Reuse RecoveryContext's directory listing, use a narrow marker scan, and recover live CF names while replaying MANIFEST. - Filter manifest-selected safe-superset snapshots to live MANIFEST CFs in LoadLatestOptions and compatibility verification without discarding their option maps. - Prevalidate and sanitize CF batches outside the DB mutex, write one complete snapshot while options_mutex_ serializes topology changes, and hold the DB mutex only for the existing individual MANIFEST commits. Batch missing-CF creation during DB open follows the same locking boundary. - Keep the committed snapshot live in cleanup and live-file APIs, and make subset checkpoints write an OPTIONS snapshot and filtered MANIFEST with identical membership in tracked and legacy modes. - Preserve the first partial multi-CF drop error while consuming legacy OPTIONS persistence Status, and sync the checkpoint MANIFEST directory after replacement. - Pass checkpoint helper callbacks by reference and group VersionEdit storage by alignment so the new paths add neither callback copies nor avoidable object padding. Safety boundaries and unsupported cases: - Tracking defaults to false. Disabled writers and never-tracked databases retain legacy write ordering and highest-numbered selection. - Older binaries safely ignore the new MANIFEST tag and continue choosing the highest OPTIONS number. Downgrade is readable but not crash-atomic: an old binary can consume a tracked orphan. That is the same class of OPTIONS/MANIFEST inconsistency as the previous behavior. - A rollback binary must know the serialized option name before the feature is enabled. Exact compatibility checks intentionally report a mismatch when the caller's tracking setting differs from the persisted snapshot. - A newer untracked OPTIONS file deliberately supersedes an older committed number, allowing an old or disabled writer to become authoritative without rewriting MANIFEST. - Reactive secondary and follower instances do not enforce or repair the binding. Ordinary DB::Open still uses caller-supplied options. - Writable open does not delete rejected tracked or ambiguous files when a compaction service owns OPTIONS-file retention. Read-only open performs no repair writes. - Direct consumers that scan OPTIONS filenames themselves are unchanged; the binding is enforced by RocksDB's persisted-options APIs. - If the committed snapshot is missing and only tracked candidates remain, selection intentionally falls back to the highest candidate so RepairDB output remains usable. Validation coverage and known limitations: Focused coverage exercises encoding compatibility, tracked and legacy selection, the never-tracked open fast path, marker-read failures, exact tracking-marker compatibility, activation and disable/re-enable transitions, batched create/drop and create-missing behavior, partial-batch safe supersets, OPTIONS/MANIFEST failure injection, background-error propagation, orphan cleanup and retention, read-only non-mutation, subset-checkpoint behavior, stale OPTIONS recovery, RepairDB option sanitization, and transient read-corruption reconstruction. The assertion-enabled full executable suite passes. The repository check-format and check-sources wrappers cannot run because they invoke Git inside this Sapling checkout; fbsource lint is used instead. Cross-platform CI is not part of local validation. Reviewed By: pdillinger Differential Revision: D123104845 fbshipit-source-id: 77d94c989243c3f3f9fedfa81fa364c166250303 |
||
 Ranjan Banerjee
|
0561153b73 |
Add configurable largest-memtable flush policies to WriteBufferManager (#15047)
Summary: Motivation A WriteBufferManager can be shared by many RocksDB instances and column families. Under the existing policy, the DB whose writer first observes memory pressure flushes its own oldest mutable memtable. That DB might own only a small fraction of the reclaimable memory, so workloads with many DBs or CFs can generate many small flushes while the largest memory consumer remains untouched. Scanning every registered DB while holding process-wide or DB mutexes would improve selection accuracy but would serialize unrelated write paths and is not viable for deployments with thousands of DBs. Previous behavior kFlushOldest was the only behavior: the write-group leader that noticed WBM pressure synchronously selected the oldest eligible mutable memtable in its own DB. The WBM had no per-DB view of reclaimable memory and could not direct reclamation to another DB sharing the same budget. New behavior This change adds configurable WriteBufferFlushPolicy modes: - kFlushOldest preserves the existing behavior and remains the default. - kFlushLargest selects the largest eligible flushable column family in the writing DB. - kFlushLargestAcrossDBs selects the DB advertising the most reclaimable memory across all DBs sharing the WBM, then flushes its largest eligible CF. Atomic-flush DBs advertise and flush the aggregate eligible memory across their CFs. High-level implementation Each writable DB registers a lightweight FlushInitiator with the shared WBM. Per-DB accounting publishes two approximate values using atomics: total mutable memory and the largest flushable CF contribution, including eligible immutable memtables waiting for flush. Allocator reporting is batched, so the allocation hot path avoids registry scans and DB mutexes. A dedicated WBM coordinator thread maintains an approximate ranked snapshot of registered DBs. Refreshes are driven by accounting changes, threshold crossings, and periodic pressure checks. Ranking and O(number of DBs) work happen on that coordinator thread; writers only update counters and signal the coordinator. Registration lifetime is protected independently so DB close can safely drain an in-flight handoff. Threading and flush handoff The coordinator reserves and pins one ranked candidate at a time, without holding a registry lock while invoking the DB. The target DB uses TryLock on its DB mutex and rejects the handoff if it is closing, paused, stopped, stalled, already handling a WBM job, or no longer has flushable memory. Rejected candidates do not block consideration of the next DB. An accepted handoff queues one LOW-priority WBM job. The LOW worker joins the target DB write thread and revalidates eligibility before switching the memtable. The resulting ordinary flush is then scheduled through RocksDB's existing flush machinery on the HIGH-priority flush pool. Keeping the write-thread join off the requesting writer avoids making one DB's write path wait on another DB's mutex or write queue. Only one manager-wide remote handoff is active at a time. Completion or cancellation clears the handoff and requests a rerank. If memory reaches the hard limit while remote coordination is delayed, a rate-limited local fallback lets a writer seal a local memtable to preserve forward progress. Write stalls, shutdown, WaitForCompact, policy changes, and DB deregistration explicitly drain or reject outstanding handoffs. Compatibility The legacy constructor and kFlushOldest behavior remain unchanged unless a new policy is explicitly selected. There are no on-disk format changes. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15047 Reviewed By: xingbowang Differential Revision: D112396106 fbshipit-source-id: 3049adc7034dd4e00d74d7d86e1d19004292d299 |
||
|
Peter Dillinger
|
c83a84e98c |
Rename TableProperties::creation_time to oldest_ancestor_time (#15304)
Summary: The `creation_time` table property has long stored the file's oldest ancestor time (the oldest key time among the file's ancestors, falling back to flush/compaction time), not the file's creation time, which is `file_creation_time`. A TODO in table_properties.h acknowledged the misleading name. This change starts the API migration: `TableProperties::oldest_ancestor_time` is the new name, and `creation_time` remains as a deprecated alias sharing storage via an anonymous union, so existing external reads and writes continue to compile and behave identically. An anonymous union is used rather than a `uint64_t&` alias member. A reference member would delete TableProperties copy/move assignment, make the implicit copy constructor bind the copy's alias to the source object (dangling), permit writes through a const TableProperties, add 8 bytes, and break the uint64_t-range walk in TEST_SetRandomTableProperties. The union leaves size, layout, copying, constness, and offsetof unchanged. Reading the other union member is formally UB in ISO C++, but both members are uint64_t and this is supported in practice by GCC, Clang, and MSVC. All in-repo uses of the old name now use `oldest_ancestor_time`. Names that are externally observable or persisted are intentionally unchanged: * On-disk property name `rocksdb.creation.time` (`TablePropertiesNames::kCreationTime`) * The `"creation_time"` key in TableProperties serialization (used by remote compaction) * The event-log JSON key and `TableProperties::ToString()` label * Java `TableProperties.getCreationTime()` and C `rocksdb_table_properties_creation_time()` C API: the auto-binding generator (tools/c_api_gen/auto_simple_bindings.py) only considered direct FieldDecls and failed on the anonymous union's implicit unnamed field. It now treats anonymous union members as direct fields. Regenerating adds a new C getter `rocksdb_table_properties_oldest_ancestor_time()` alongside the existing `rocksdb_table_properties_creation_time()`. Follow-up: remove the deprecated `creation_time` alias in a future release. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15304 Test Plan: New unit test DBTablePropertiesTest.CreationTimeAliasesOldestAncestorTime verifies the two names share an address and that a write through the old name is visible through the new one. options_settable_test and compaction_service_test cover TableProperties serialization round trips through the unchanged "creation_time" key, now mapped to the renamed field. Reviewed By: jaykorean Differential Revision: D123079083 Pulled By: pdillinger fbshipit-source-id: 18b07d91f303a8f6de58da0c730f80d8c371505c |
||
 Josh Kang
|
c0512c4c34 |
Remove deprecated write_global_seqno ingestion support (#15287)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15287 `IngestExternalFileOptions::write_global_seqno` has been deprecated since RocksDB 8.0 and will soon be removed from the API. It requires ingestion to mutate SST files, sync them, regenerate checksums, and expose file-offset plumbing solely for that mutation. Remove the option and its C, Java, and `ldb` APIs. Assigned sequence numbers now live only in the MANIFEST, leaving newly ingested SST bytes and whole-file checksums unchanged. ## Backward Compatibility New readers continue to support existing SSTs containing a nonzero global sequence number written by legacy ingestion. Because that old write did not update the properties-block checksum, checksum verification retries against a temporary copy with only the global-seqno value restored to its original zero placeholder. The parsed sequence number and file contents remain unchanged. This fallback only accepts mismatches explained by the legacy rewrite. Other checksum failures continue through normal corruption handling, and `verify_checksums=false` continues to bypass verification. Reviewed By: xingbowang Differential Revision: D122405856 fbshipit-source-id: 934e9bc755d58ea8d84e1c085e9769015bd8dfd7 |
||
|
Xingbo Wang
|
4c2a977aa5 |
Track newer snapshot versions in RocksDB metadata reads (#15086)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15086 Why: An application built on top of RocksDB could use cache population reads through explicit snapshots and needs to know when the returned value has already been superseded, without a second point read. Previous behavior: Get and MultiGet returned the snapshot-visible value and optional timestamp but could not report a newer point update or covering range deletion. New behavior: experimental C++, C, and Java metadata-read APIs report newer committed puts, deletes, merges, and covering range tombstones; ordinary primary-DB writes completed before the read are guaranteed to be reported, racing writes remain best-effort, and true has no false positives. Implementation: metadata reads widen lookup behind read-callback hooks while keeping metadata-only state off passive common object layouts; per-key state stays in KeyContext/GetContext, ordinary MultiGet row-cache replay semantics are preserved, BlobDB/TTL use shared output helpers, dead single-CF plumbing and an invalid cross-key stress assertion are removed, and the range-tombstone ordering invariant is documented without changing behavior. Safety and unsupported cases: tracking is opt-in and requires an explicit snapshot; persisted-tier reads, custom transactional visibility callbacks, secondary DBs, and unsupported subclasses return NotSupported, while read-only/compacted DBs return false because their in-process views are static. Tests and limitations: 1,265 Buck tests plus focused metadata, 100x coerced-context-switch row-cache, write-committed/write-prepared crash tests, builds, and lint passed. Current-parent release db_bench found no material passive regression: readrandom +0.65%, MultiGet batch 4 -1.40%, and batch 32 +0.48%. Reviewed By: pdillinger Differential Revision: D104182397 fbshipit-source-id: 36f7be1cd50fe2383689adaae31a6bb7d7e462dc |
||
|
Peter Dillinger
|
8f492b2f47 |
Store common user-key prefix once per block (format_version 8) (#15117)
Summary:
Building on D116105558, this change finishes and publishes `format_version = 8` as
production-ready.
The headline feature is a tweaked block encoding that stores the common user-key
prefix shared by all keys in a block **once**, in the block's leading bytes
`[0, restarts[0])`, instead of repeating it at every restart point. This shrinks
prefix-heavy blocks (and their block-cache footprint) and, for bytewise-ordered
keys, speeds up `Seek` by comparing prefix-stripped suffixes. It is applied to
**data blocks** and **every index tier** (leaf/partition index, the
partitioned-index top-level, and the partitioned-filter top-level).
Motivation: MyRocks/ZippyDB/timeseries-style keys carry a long shared prefix
(table/index id, partition, etc.). Repeating it at each restart wastes space and
cache, and every Seek re-compares those bytes. Profiles show index-block seek CPU
is comparable to data-block seek CPU (partitioned index does two index seeks per
data seek), so index blocks are a first-class target, not just data blocks.
### Format (self-describing, no new footer bit)
- The common prefix occupies the block's leading bytes; a non-zero `restarts[0]`
(the offset of the first entry, i.e. the prefix length) self-signals its
presence. `restarts[0]` is 0 in every block that does not use the feature (all
versions, all block types), so a single fv8 reader transparently handles any
mix of stripped and unstripped blocks -- no footer bit is consumed.
- Previous versions would misinterpret this encoding by quietly ignoring the
common prefix, which (along with the lack of extensibility in data-block
metadata) is why it is gated behind format_version 8.
- Only restart-point entries change on disk; non-restart entries are byte
identical to fv7, so the writer produces the exact same bytes regardless of
implementation.
### Public option
New `BlockBasedTableOptions::optimize_key_common_prefix`
(`enum class OptimizeKeyCommonPrefix`), default `kIfFastSeek`:
- `kDisabled` -- never use it.
- `kIfFastSeek` (default) -- use it only where it also enables the Seek speedup:
format_version >= 8 with a built-in (reverse-)bytewise comparator.
- `kEnabled` -- use it for all comparators at fv8; (reverse-)bytewise get space +
fast seek, other comparators get the space savings only (the reader
reconstructs full keys and compares with the real comparator).
Expressed as a criteria so it need not be toggled per CF based on the comparator
(which can vary by CF). It additionally requires `use_delta_encoding` (the
feature builds on delta encoding) and `ts_sz == 0`; when `use_delta_encoding` is
false -- the setting that also preserves zero-copy key pinning -- it is a no-op.
Default format_version is unchanged (still 7); the feature is opt-in via fv8.
### Scope: every index tier
The reader handles all index-like blocks uniformly (a non-zero `restarts[0]`
self-signals a stripped block regardless of block type), so the writer strips the
common prefix from every tier:
- **Data blocks** and **leaf / partition index blocks**: enabled.
- **Partitioned-index top-level block**: enabled. Under the default
`index_shortening = kShortenSeparatorsAndSuccessor` the file's last separator
would otherwise be a short successor sharing no leading bytes, collapsing the
single-block top-level's common prefix to 0. When common-prefix is on the
builder suppresses that one successor (keeps the full last key -- i.e.
`kShortenSeparators` behavior for that one entry), so the top-level strips
uniformly under every shortening mode; the full last key is also a tighter file
upper bound, so it is not worse.
- **Partitioned-filter top-level index**: enabled (same uniform treatment). Its
space/CPU effect is small at default partitioning (the top-level filter index
is tiny) but grows with many small partitions -- a filter miss short-circuits
at, and is dominated by, the top-level filter-index seek (see Performance).
Included for a uniform on-disk surface across tiers.
Bloom filter *contents* still hash full user keys, so prefix/whole-key filtering
is unaffected.
### Implementation
- Writer (`block_builder.{h,cc}`): keys are encoded incrementally in the fv7
layout while the running common prefix is tracked cheaply (reusing the delta
`shared`); at `Finish()` only the restart-point entries are rewritten in place
with the prefix stripped, and the prefix is written once at the block start.
The rewrite is generalized to the value-delta (fv4) index layout used by index
blocks (no `value_size` header; bulk-copy the inline BlockHandle and
non-restart entries verbatim).
- Reader (`block.{h,cc}`, `dbformat.h`): `Block` decodes the prefix from
`restarts[0]`; `BlockIter::ParseNextKey` prepends it on a restart point
(`shared == 0`). The suffix-seek helpers (strip target once, compare shortened
suffixes) live in the templated `BlockIter` base and are shared by data and
index iterators (binary search, templated on `DecodeKey`/`DecodeKeyV4`); for
non-bytewise comparators (data) and the hash `PrefixSeek` refine step, the
reader reconstructs the probed restart key and compares full keys.
- Gating (`block_based_table_builder.cc`, `index_builder.{h,cc}`,
`partitioned_filter_block.{h,cc}`): the criteria is threaded to the data-block
builder and to every index tier -- leaf/partition index, the partitioned-index
top-level (`use_common_prefix_top`), and the partitioned-filter top-level --
together with the last-separator successor suppression
(`ShortenedIndexBuilder::GetSeparatorWithSeq`) that lets the top-level strip
under the default shortening mode. The index base gate also requires
`super_block_alignment_size == 0` (block alignment can trigger
`skip_delta_encoding`, producing non-restart `shared == 0` entries the reader
could not distinguish from stripped restart keys).
- Memory: the per-block/per-iterator prefix is stored as a `uint32_t` length
(the prefix is always at offset 0) rather than a full `Slice`, and `Block` /
`BlockBuilder` members were repacked so the feature adds little or no struct
size.
- Java: a new `OptimizeKeyCommonPrefix` enum and a `BlockBasedTableConfig`
getter/setter, plumbed through JNI in both directions (the
`newTableFactoryHandle` writer and the options-reconstruction constructor).
### Whole-file (table-level) prefix: investigated and rejected
A natural extension is to store the whole-file common prefix once per *file*
(when an `SstPartitioner` guarantees every key in the file shares a fixed prefix)
instead of once per *block* -- e.g. the MyRocks index/table id, or a future
128-bit global id. It was prototyped end-to-end (partitioner-provided prefix
captured at file open, stored in a `user_key_common_prefix` table property, and
stripped from every data/index block; the reader re-prepends it) and then
dropped. The `user_key_common_prefix` table property reserved in the
previous diff is dropped in this one. The rationale, in case it is revisited:
- Marginal benefit over per-block. Per-block already strips the full shared
prefix from every key; the only thing whole-file removes on top is the single
per-block copy of that prefix, i.e. `~= prefix_len / block_size` of the
uncompressed (block-cache) footprint, with no meaningful read-CPU change
(per-block already delivers strip-once + suffix compares). Measured at
prefix_len=12, 4 KB blocks this was ~-0.3% incremental space; at the 8-16 KB
data blocks typical of the space-constrained target (MyRocks/UDB) it is
~0.1-0.2% even with a 16-byte global id -- a fraction of a percent, seen only
by partitioner users. Small block-size workloads are becoming more
niche, and even where they exist, they are also CPU-sensitive (see next
paragraph).
- Real, if small, CPU tax on the *per-block* path. Reconstructing a key from two
non-contiguous parts (file prefix + block residual) cost ~0.5-1.0% additional
retired instructions on the per-block read path (deterministic across gcc and
clang), wall-clock-neutral in cache-resident tests but present even when no
whole-file prefix is configured. Attempts to remove it did not pan out: a
compile-time specialization of the reconstruction *increased* instructions
(+2.9%) by defeating inlining; a per-block combined-prefix buffer trades the
CPU cost for a memory cost on every (default) block; and a per-iterator buffer
was the original larger regression. Only invasive iterator-level
monomorphization would truly zero it, which is not justified on this hot path.
- Decision. A fraction-of-a-percent, partitioner-only space win does not justify
a permanent per-block-read-path CPU tax plus additional on-disk format surface
(a table property that changes whole-file interpretation). Dropped. The generic
`SstPartitioner::ShouldPartitionByPrefix` hook is kept (cheap, and useful
independently), and the reserved `user_key_common_prefix` table property is
removed since no writer will emit it.
### Performance (db_bench, release/native, cache-warm, best-of-3, pinned)
Three arms: (A) fv7 on the base/main branch, (B) fv7 with this change (should be
neutral / byte-identical writing), (C) fv8 with this change and
`optimize_key_common_prefix=enabled` (the optimization is active). Partitioned
index + filters; `compression_type=none`, so SST bytes are the uncompressed
block-cache footprint. `int` = db_bench default keys (variation-early, near worst
case); `pfxN` = an N-byte shared prefix + varying suffix (MyRocks-like).
Setup: `num=3,000,000`, `reads=1,500,000`, single thread,
`metadata_block_size=4096`, cache holds the working set, pinned to one core.
| shape (key/val) | arm | SST bytes | readrandom us/op | seekrandom us/op |
|---|---|---:|---:|---:|
| int (32/8) | A fv7 main | 138,983,658 | 3.484 | 4.995 |
| | B fv7 ours | 138,983,146 | 3.518 | 5.024 |
| | C fv8 ours | 137,681,420 | 3.490 | 4.986 |
| pfx24 (32/8) | A fv7 main | 71,697,531 | 3.417 | 5.561 |
| | B fv7 ours | 71,698,043 | 3.448 | 5.579 |
| | C fv8 ours | 65,578,236 | 3.284 | 5.386 |
| pfx56 (64/16) | A fv7 main | 102,942,266 | 3.624 | 5.830 |
| | B fv7 ours | 102,942,266 | 3.678 | 5.835 |
| | C fv8 ours | 90,554,642 | 3.444 | 5.626 |
fv8 vs fv7-main (C vs A) -- space / readrandom / seekrandom:
- int (32/8): **-0.9%** / ~neutral / ~neutral
- pfx24 (32/8): **-8.5%** / **-3.9%** / **-3.1%**
- pfx56 (64/16): **-12.0%** / **-5.0%** / **-3.5%**
fv7-ours vs fv7-main (B vs A): SST bytes byte-identical (within 512 B; the fv8
paths are gated off), read/seek us/op within run-to-run noise (~1%, best-of-3 on
a shared host). Net: fv8 is a space **and** read/seek-CPU win on prefix-heavy
data and neutral on variation-early (`int`) data, so `kIfFastSeek` is safe on by
default at fv8.
### Space by tier (compression=none, num=3M, mdb=4096; fv7 -> fv8)
Where the win comes from -- the index tiers strip the most:
| shape | data | leaf/partition index | top-level index | filter |
|---|---:|---:|---:|---:|
| int | -0.8% | -29% | -45% | -0.2% |
| pfx24 | -8.3% | -69% | -89% | -0.8% |
| pfx56 | -11.1% | -83% | -96% | -1.6% |
Data bytes dominate the absolute total, but every index tier is stripped:
leaf/partition index -29% to -83% and the partitioned-index top-level -45% to
-96%. Filter *blocks* barely move (-0.2% to -1.6%) because filter bits hash full
user keys; only the small top-level filter index is stripped.
### Point-miss lookups (readmissing)
A partitioned-filter miss short-circuits at the filter (no data-block read), so
it is dominated by the top-level filter-index seek plus the partition bloom. At a
typical `metadata_block_size=4096` (few partitions, tiny top-level filter index)
fv8 is neutral on misses (int +0.9%, pfx24 -1.0%, pfx56 -0.7% us/op -- all
noise). The top-level-filter stripping pays only with many small partitions: at
`metadata_block_size=256` it is pfx24 1.166->1.138 (**-2.4%**) and pfx56
1.273->1.188 (**-6.7%**) us/op.
### Backward / forward compatibility
- Default `format_version` remains 7; the feature is inert unless a CF opts into
fv8. Older RocksDB rejects fv8 files at footer decode (fail-safe).
- Within fv8, the stripped-block set a writer can emit is fixed and every fv8
reader can read it (the `restarts[0]` signal plus uniform reader handling), so
the on-disk surface is self-consistent across builds/downgrade.
### Follow-ups (not in this change)
- Suffix-aware interpolation / hash index seek (currently correct via the
reconstruction fallback, not perf-tuned).
Pull Request resolved: https://github.com/facebook/rocksdb/pull/15117
Test Plan:
- New focused unit tests (`table_test`), run at the full `format_version` matrix
including the now-supported fv8:
`GeneralTableTest.OptimizeKeyCommonPrefix` (data-block space+read+Seek
correctness: existing and missing keys straddling the prefix, before/after-all,
SeekForPrev), `OptimizeKeyCommonPrefixNonBytewise` (reconstruction path), and
`IndexBlockCommonPrefixEquivalence` across `{kBinarySearch,
kBinarySearchWithFirstKey, kHashSearch, kTwoLevelIndexSearch}` x
`{partitioned filters}` x `{bytewise, reverse-bytewise}` (so both top-level
tiers -- partitioned-index and partitioned-filter -- are exercised), asserting
identical reads and a shrunken fv8 index.
- The randomized `ParameterizedHarnessTest` now runs at fv8 (reverse-bytewise,
restart intervals `{1,16,1024}`, special/empty keys fuzzed against the feature).
- `db_crashtest.py` randomizes `format_version` including 8 and
`optimize_key_common_prefix` in {0,1,2}.
- Removed the reserved-property carve-out: dropped the `ReservedUserKeyCommonPrefix`
reader test and the C API getter for the property; C API regenerated via
`tools/c_api_gen/` (`make regen-c-api`), and the intentional public-C-API
removal is recorded in `tools/c_api_gen/api_compatibility_allowlist.json`.
- Java: `BlockBasedTableConfigTest.optimizeKeyCommonPrefix` (default
`kIfFastSeek`, plus set/get of every value).
- `table_test` `FooterTests` updated for the now-published fv8 footer, whose
metaindex-gap encoding stores the metaindex block size in 16 bits (the test
caps that size for fv >= 8).
- Manual verification: the db_bench runs in Performance above (three-arm
space + read/seek CPU, per-tier byte breakdown via `sst_dump`, and
`readmissing`); and `readrandom`/`seekrandom` with
`--block_protection_bytes_per_key=8` find 100% of keys at both fv7 and fv8
(reconstructed keys pass per-key checksum).
Reviewed By: xingbowang
Differential Revision: D116365016
Pulled By: pdillinger
fbshipit-source-id: 66f4b5c60ea4fe0ff48d34c98d191ba8ebdf0665
|
||
|
Peter Dillinger
|
7217bf34a7 |
Add SST build context: is_remote_compaction, db_name, kSstFileWriter (#15085)
Summary: Custom filter and compression strategies decide what to build per SST from a FilterBuildingContext (shared by FilterPolicy::GetBuilderWithContext and CompressionManager::GetCompressorForSST). This adds more context so those strategies can adapt to where and how a file is being produced. FilterBuildingContext gains two fields: * is_remote_compaction: true only when the file is produced by an offloaded CompactionService worker. Detected with a new virtual CompactionJob::IsRemoteCompaction() (default false) overridden by CompactionServiceCompactionJob, mirroring the existing GetTableFileName hook. Local/fallback compactions and flushes are false. * db_name: a non-owning Slice naming the DB, populated for flush, compaction, recovery, and repair, and empty only where there is no DB, as with SstFileWriter. Under remote compaction it is the DB as opened by the worker, which can differ from the primary's. Both are threaded through TableBuilderOptions and populated in BlockBasedTableBuilder::Rep, whose "is this file in the LSM" test becomes a switch with no default case, so that a future TableFileCreationReason has to be classified there. Separately, TableFileCreationReason is now backed by uint8_t and gains a kSstFileWriter value, which SstFileWriter reports instead of the catch-all kMisc. It is appended after kMisc so existing enumerator values are unchanged; those values are public API, surfaced to EventListener and mirrored by the Java binding (updated here). Note that SstFileWriter neither notifies listeners nor consults CompactionFilterFactory, so for now the new value is only observable through FilterBuildingContext. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15085 Test Plan: * New CompactionServiceTest.IsRemoteCompactionInCompressorContext installs a custom CompressionManager and asserts GetCompressorForSST sees is_remote_compaction == true for the offloaded worker's output files and false for the primary's flushes in the same run, so the flag is shown to be per-file rather than global. IsRemoteCompactionFalseForLocalCompaction covers the same manager with no compaction_service configured. * DBBloomFilterTest.ContextCustomFilterPolicy now reports db_name as a token (expected / none / UNEXPECTED:<value>), so the value rather than just its presence is checked for flush and compaction, and expects kSstFileWriter instead of kMisc for the external SstFileWriter file. * bloom_test's "Like SST file writer" case uses kSstFileWriter so that it keeps modelling SstFileWriter. * Java EventListenerTest references TableFileCreationReason.MISC.getValue() instead of a hardcoded enum byte, so it stays correct across enum changes. Reviewed By: jaykorean Differential Revision: D115603616 Pulled By: pdillinger fbshipit-source-id: ad73fbc391a5377ace406725238ae6eede46c8d9 |
||
|
Peter Dillinger
|
af0eae4de3 |
Rename TransactionLogIterator to WalIterator and document its behavior (#15056)
Summary:
`TransactionLogIterator` iterates the write-ahead log. It does not return transactions, does not require `TransactionDB`, and in fact returns `Status::NotSupported()` for two of the three `TransactionDB` write policies (WritePrepared and WriteUnprepared). The name predates `TransactionDB` and comes from "transaction log" being an old synonym for WAL. Everything else in this header was already renamed away from that terminology -- `LogFile` -> `WalFile`, `VectorLogPtr` -> `VectorWalPtr` -- and the C API has long used `rocksdb_wal_iterator_t` and `rocksdb_wal_iter_*`. This finishes the job.
The public class is now `WalIterator` in `include/rocksdb/wal_iterator.h`. `include/rocksdb/transaction_log.h` remains as a forwarding header and `TransactionLogIterator` remains as an alias, both marked deprecated by comment only (not `[[deprecated]]`) so existing users compiling with -Werror are unaffected, matching how `LogFile` and `VectorLogPtr` were retired. `DB::GetUpdatesSince()` keeps its name.
More importantly, the documentation was incomplete in ways that could cost a user data, so this states plainly what the API has always done:
- It only sees the WAL. Writes made with `WriteOptions::disableWAL`, and sequence numbers consumed by `IngestExternalFile()`, advance the DB sequence number without producing a WAL record. They leave permanent holes that end iteration. This was not documented anywhere, so it was unreasonable to expect a user to know those features silently break WAL tailing.
- The three iterator states, in particular that `!Valid()` with an OK `status()` means caught up rather than finished, and that calling `Next()` again is the intended way to tail a DB. The old comment on `Next()` said "REQUIRES: Valid() to be true", which forbids the primary use case.
- `Status::TryAgain`, which a caller must handle but which appeared nowhere in the public headers.
- That `GetUpdatesSince(seq)` silently starts later when `seq` is unavailable. Combined with the above this is the sharp edge: recovering from a spent iterator means calling `GetUpdatesSince()` again, so the recovery step is exactly where data can be skipped unnoticed. A consumer applying these batches to a copy of the DB diverges permanently and undetectably if it misses one, so the header now says such consumers must verify continuity themselves using `BatchResult::sequence` and `WriteBatch::Count()`.
Almost all of this diff is mechanical: renaming the type and its files, updating `#include` paths, rewriting comments, and updating build lists (`src.mk`, `CMakeLists.txt`, `BUCK`, `java/Makefile`, `java/CMakeLists.txt`). `db/c.cc` is regenerated from `tools/c_api_gen/c_base.cc`. Reviewers should focus on the four places that are not:
1. `WalIteratorImpl::UpdateCurrentWriteBatch()` in `db/wal_iterator_impl.cc` (renamed from `db/transaction_log_impl.cc`) drops dead code. On a sequence discontinuity it set `current_status_` to `NotFound` and then called `SeekToStartSequence()`, which returns immediately when `current_status_` is not OK. That guard and the early return in `Next()` were both added by
|
||
|
Peter Dillinger
|
b7b6d7c8c0 |
Mirror CI third-party build deps on a GitHub release instead of S3 (#15041)
Summary: CI/build downloaded pinned third-party binaries (compression source tarballs, Java test jars, and clang-format-diff.py) from an S3 bucket. Move the mirror to assets on an unlisted "third-party-deps" GitHub release, so the files live alongside the code with no separate bucket to maintain and no per-repo bandwidth quota (unlike Git LFS). Recent releases on the RocksDB release page are not polluted (https://github.com/facebook/rocksdb/releases/, far down the list probably from tagging a very old release). The compression download variables are refactored from *_DOWNLOAD_BASE (a directory URL) to *_DOWNLOAD_PREFIX (directory + "<project>-", or "...archive/v"), so each release asset can carry a project-name prefix in the otherwise flat release namespace while the upstream defaults still resolve to byte-identical URLs. The Java DEPS_URL defaults (Makefile and CMake) and the clang-format-diff.py download are repointed at the release. A comment at each dependency site links to the release edit page for adding/updating assets. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15041 Test Plan: CI and... Downloaded all 11 assets from the release and confirmed their SHA256 sums match the files previously served from S3 (tarball/jar sums are also enforced by the existing Makefile checksum checks). Verified the refactored Makefile upstream defaults expand to the same URLs as before, and that the CI *_DOWNLOAD_PREFIX overrides plus DEPS_URL resolve to the uploaded asset names. Covered by the normal PR/nightly/weekly jobs that build compression from source and run the Java tests (Make and CMake). Reviewed By: xingbowang Differential Revision: D114443886 Pulled By: pdillinger fbshipit-source-id: 8c59435e7064f0d9931cba103c8120fe113b0d6d |
||
|
Peter Dillinger
|
afc922bf5f |
Make ReadOptions::table_filter a caller-owned pointer for cheaper copies (#15005)
Summary: ReadOptions is copied frequently on read paths, and its only non-trivially-copyable member, table_filter, is a std::function that is almost always empty yet still gets deep-copied on every ReadOptions copy (and can heap-allocate). This changes table_filter to a pointer to a caller-owned std::function instead of storing one by value: when non-nullptr, the caller is responsible for keeping the pointed-to function alive for the duration of the read (like the other pointer fields such as iterate_upper_bound). With this change every ReadOptions member is trivially copyable, so copying ReadOptions no longer touches a std::function at all. That trivial-copyability is locked in with a static_assert in options_test.cc (kept out of the public header). This is inspired by D113109635, which targeted the same ReadOptions-copy hotspot by relocating table_filter to the end of the struct so the trivial members memcpy in one contiguous block. This is intended to be a superior solution: rather than only defragmenting the copy, it removes the std::function copy entirely. This is a breaking API change, tolerated because table_filter is a relatively obscure option. A nullptr, or a pointer to an empty std::function, means no filtering (preserving the previous empty-function behavior). A shared internal helper HasTableFilter(const ReadOptions&) in options_helper.h centralizes the nullptr + empty-function check used on the read paths. The C API and Java bindings now own the std::function and hand ReadOptions a pointer to it; their public signatures are unchanged, and the C API rocksdb_readoptions_t clears the owned function when the callback is null. Making ReadOptions trivially copyable also let -Werror=unused-but-set-variable fire on a pre-existing dead local in db_bench's MultiScan: a ReadOptions that set async_io/readahead_size but was never passed to NewMultiScan (the warning had been suppressed while ReadOptions had a non-trivial member). That local was in fact inert for multiscan -- BlockBasedTableIterator::Prepare overrides ReadOptions::async_io with MultiScanArgs::use_async_io for the coalesced prefetch, so multiscan async I/O is driven by --multiscan_use_async_io, not ReadOptions::async_io. Removed the dead local; MultiScan now uses read_options_ directly, consistent with MultiScanRandom. Pull Request resolved: https://github.com/facebook/rocksdb/pull/15005 Test Plan: Existing table_filter coverage was adapted to the new pointer API and still exercises the behavior end to end: db_iterator_test.TableFilter (callback invoked once per table; returning false hides a table) and its range-conversion rejection case; db_bloom_filter_test SstQueryFilter/FixedWidthSegments range queries; and trie_index_db_test's range-query iterator (which returns iterators outliving the filter, exercising the caller-owned lifetime). db/c_test covers the C API set/clear/has_table_filter path. options_test asserts std::is_trivially_copyable_v<ReadOptions>. Manually verified the db_bench MultiScan async behavior with a ~200MB DB and --use_direct_reads: with ReadOptions::async_io held true, async reads tracked only --multiscan_use_async_io (rocksdb.read.async.micros count 329511 with use_async_io=true vs 0 with false), confirming ReadOptions::async_io does not drive multiscan async and the removed local had no effect. Reviewed By: anand1976 Differential Revision: D113613525 Pulled By: pdillinger fbshipit-source-id: 38d73b607500a5c7ca2b89dd8f4187ef43940aa4 |
||
|
Josh Kang
|
c64f682946 |
Fix IoUringBackend crash in ReadIOExecutor worker threads (#15020)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15020 Each read executor worker creates its own `IoUringBackend`, and Folly's I/O thread pool creates its worker threads and EventBases lazily. A successful calling-thread probe or `DB::Open()` therefore does not validate every worker. A later read can start a new worker; previously, an `IoUringBackend` initialization exception on that worker could escape startup and terminate the process. The previous 512-submit setup also allowed Folly's capacity retry to reduce the completion queue to 512 while retaining a 1024-entry submission queue, which the kernel rejects with `EINVAL`. Use a 256-submit limit so the 512-entry retry remains valid. Configure the shared read executor during `DB::Open()` through `DBOptions::read_io_executor_threads`, defaulting to one thread. The backend factory catches per-worker io_uring initialization failures and uses Folly's default backend. Each worker's thread-local availability marker starts false and is set true only after `IoUringBackend` construction succeeds. When construction fails, the marker remains false and `SubmitReadAsync()` performs the read synchronously. The executor's maximum thread capacity can be raised or lowered; `db_stress` requests eight threads for async DB APIs. To expose this degradation, `FSRandomAccessFile::SubmitReadAsync()` now reports whether the asynchronous path was used. `RandomAccessFileReader` increments the `FILE_SUBMIT_ASYNC_READ_FALLBACK` ticker for every submission that falls back to synchronous I/O, making the fallback visible through the DB's `Statistics`. Reviewed By: xingbowang Differential Revision: D114031018 fbshipit-source-id: 1b7f6c1cea693e5dd743ea1ada0fca1cf757b3f9 |
||
|
Xingbo Wang
|
03c3c090fb |
Measure IODispatcher async read latency (#15001)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/15001 Add IODispatcher-specific histograms for async read observed completion latency, `FileSystem::Poll` wait time, and submit-to-consumer-poll prefetch lead time. Record submit timestamps on async read state, report completion latency from the callback, and report poll wait/prefetch lead when the block consumer polls the handle. Keep Java histogram byte mappings stable by assigning the new histogram IDs after the existing range. Add a controlled async filesystem test that verifies one deferred async read records all three histograms. Address Infer feedback by iterating `async_state->block_indices` directly when removing async IO map entries. Keep `PollAndProcessAsyncIO()` holding an owning `shared_ptr<AsyncIOState>` while erasing those entries so the coalesced request state cannot be destroyed mid-loop. Format `util/io_dispatcher_imp.cc` and `util/io_dispatcher_test.cc` with clang-format 21.1.2 to fix the format signal. Reviewed By: pdillinger, anand1976 Differential Revision: D113549401 fbshipit-source-id: dafc272c9252e1baccf404f681d33f76890ce0b2 |
||
|
Peter Dillinger
|
bb77d17072 |
Make Flush() optionally wait for OnFlushCompleted (FlushOptions::listener_wait) (#14992)
Summary: DB::Flush() with wait=true only waits until the flush result is committed (memtables removed from the immutable list). The EventListener::OnFlushCompleted callbacks run afterward, on the background flush thread, with the DB mutex released, and the waiter is only woken by a shared condition variable that many unrelated events also signal. As a result a Flush(wait=true) caller can return before -- or in the middle of -- its OnFlushCompleted callbacks. This surfaced as a flaky failure in the c_test table_filter_listener_metadata case, which asserts listener_state.flush_count == 3 immediately after Flush() returns (https://github.com/facebook/rocksdb/actions/runs/29976431449). It is also long-standing behavior that other tests have worked around with TEST_WaitForBackgroundWork(). This adds an opt-in FlushOptions::listener_wait (default false). When set together with wait=true, Flush() does not return until the OnFlushCompleted callbacks for the flushed memtables have finished. Default behavior is unchanged. Implementation: MemTableList gains a per-column-family num_pending_flush_notifications_ counter (guarded by the DB mutex). The background flush thread reserves a pending notification at flush-commit time -- before the mutex is released to run the callbacks -- and releases it after they return, in both the single-CF (FlushMemTableToOutputFile) and atomic (AtomicFlushMemTablesToOutputFiles) paths. WaitForFlushMemTables treats a CF as finished only when its normal predicate holds and the counter is zero. Because the memtables are removed and the counter incremented under one continuous mutex hold (and, for atomic flush, all CFs are reserved before the per-CF notify loop starts releasing the mutex), a listener_wait waiter can never observe the flush as finished before the callbacks complete. The counter is per-CF rather than per-flush, so a listener_wait waiter can wait somewhat longer than strictly necessary: under rapid flushing it may block until an unrelated, later flush's OnFlushCompleted on the same CF also drains (e.g. a waiter satisfied by GetEarliestMemTableID() advancing still waits for the whole CF's pending notifications). This can only over-wait, never return early, and is tolerated in favor of implementation simplicity; a per-memtable-id watermark could tighten it if it ever matters. To keep the counter balanced against future edits, the release uses an RAII Defer (runs on every exit path) and ~MemTableList asserts the counter has drained to zero; SubPendingFlushNotifications asserts against underflow. A reservation is taken only when a callback will actually fire (committed flush info non-empty, not a mempurge, CF not dropped, listeners registered). Also adds the C API rocksdb_flushoptions_set/get_listener_wait (regenerated via tools/c_api_gen) and a DBImpl::TEST_FlushMemTableWithListenerWait() helper. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14992 Test Plan: New db_flush_test cases ListenerWaitWaitsForOnFlushCompleted and ListenerWaitAtomicFlushWaitsForAllOnFlushCompleted use a listener that blocks inside OnFlushCompleted plus spurious bg_cv_ wakeups to deterministically verify that Flush(wait=true, listener_wait=true) does not return until the callback (all CFs, for atomic flush) has finished; the atomic case guards the per-CF mutex-release gap in the notify loop. Converted existing racy/worked-around cases to the new option: c_test table_filter_listener_metadata, listener_test OnSingleDBFlushTest and MultiDBMultiListeners, and db_flush_test FireOnFlushCompletedAfterCommittedResult (the last previously relied on TEST_WaitForBackgroundWork()). The auto-generated C API roundtrip test covers the new flushoptions field. db_stress StressTest::TestFlush now sets listener_wait on a hard-coded 1 in 4 of its manual flushes (both the atomic and non-atomic branches), so the crash test exercises the new path with the DbStressListener registered. Reviewed By: jaykorean Differential Revision: D113425407 Pulled By: pdillinger fbshipit-source-id: 6fd0f5bc93fabaccd8010c2fead1d93c4c44ccfb |
||
 Andrew Chang
|
0745290ca6 |
Add OpenAndCompact DB open latency histogram (#14986)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14986 Why: We want to measure the DB open time during offloaded compactions. What: Add a histogram for the DB open time in `DB::OpenAndCompact()`. Reviewed By: anand1976 Differential Revision: D113136652 fbshipit-source-id: f8b2d74bcda3506f7b9ac80fa3e50a172deafc5e |
||
 Xingbo Wang
|
de7b8e8f7d |
Add RocksDB flush reason and atomic flush request metrics (#14898)
Summary:
This PR adds flush observability for debugging write stalls and flush bottlenecks.
Changes include:
- Add flush reason tickers for physical flush jobs:
- rocksdb.flush.reason.write_buffer_full
- rocksdb.flush.reason.write_buffer_manager
- rocksdb.flush.reason.memtable_max_range_deletions
- Add flush memtable size histograms:
- rocksdb.flush.memtable.memory.bytes
- rocksdb.flush.memtable.total.data.size.bytes
- rocksdb.flush.write_buffer_full.memtable.memory.bytes
- rocksdb.flush.write_buffer_manager.memtable.memory.bytes
- Add FlushReason::kMemtableMaxRangeDeletions and propagate it from memtables when memtable_max_range_deletions triggers a flush.
- Preserve atomic flush as a single request-level reason. For atomic flush, the reason is derived only from CFs that scheduled the flush, not every CF selected into the atomic flush cut.
- Add atomic flush request-level counters:
- rocksdb.atomic_flush.request.reason.write_buffer_full
- rocksdb.atomic_flush.request.reason.write_buffer_manager
- rocksdb.atomic_flush.request.reason.memtable_max_range_deletions
- rocksdb.atomic_flush.request.reason.other
- Keep existing per-CF flush job reason counters unchanged. Under atomic flush, rocksdb.flush.reason.* still counts physical CF flush jobs, while rocksdb.atomic_flush.request.reason.* counts one logical atomic flush request.
- Wire the new flush reason, ticker, and histogram types through Java and JNI bindings.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14898
Test Plan: CI
Reviewed By: anand1976
Differential Revision: D110214337
Pulled By: xingbowang
fbshipit-source-id: 4a445c4175d2e09ff5edbe66fde81fa00bb9e8da
|
||
|
Xingbo Wang
|
75c1ffd4e3 |
Add RocksDB rate limiter I/O usage metrics (#14884)
Summary: Add rate limiter statistics that make it easier to diagnose whether RocksDB background IO is being throttled by the GenericRateLimiter. New ticker counters: - rocksdb.rate.limiter.bytes.read/write: bytes granted by the limiter, split by operation type. - rocksdb.rate.limiter.requests.read/write: requests granted by the limiter, split by operation type. - rocksdb.rate.limiter.delayed.requests.read/write: requests that actually had to wait for a future refill because available tokens were exhausted. - rocksdb.rate.limiter.total.wait.micros.read/write: cumulative wait time for delayed requests. New histograms: - rocksdb.rate.limiter.wait.micros.read/write: per-request wait latency for delayed requests, enabling percentile debugging. GenericRateLimiter now overrides the OpType-aware Request() path so these stats preserve the configured limiter mode. The legacy three-argument Request() path is treated as write IO for compatibility. Existing NUMBER_RATE_LIMITER_DRAINS is left intact, but the new delayed request counters are the clearer signal for out-of-token throttling because one delayed request can observe multiple drain intervals. Also update the Java ticker and histogram mappings so Java consumers can access the new statistics. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14884 Test Plan: - make format-auto - make -j16 rate_limiter_test statistics_test - ./rate_limiter_test - ./statistics_test - make check-sources Reviewed By: anand1976 Differential Revision: D109694030 Pulled By: xingbowang fbshipit-source-id: e92756788ba629f25655958c7beb91099184cda4 |
||
|
Josh Kang
|
1ede57e5b0 |
Add file ingestion histograms (#14836)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14836 Adds a public `Statistics` histogram, `INGEST_EXTERNAL_FILE_TIME` (`"rocksdb.ingest.external.file.micros"`), recording the end-to-end latency in microseconds of each `IngestExternalFile(s)` call. Ingestion timing was previously only available through per-thread `perf_context` counters, which require setting a `PerfLevel` and are not aggregated, so there was no process-wide latency distribution (p50/p99/max) for dashboards. It is recorded with an RAII `StopWatch` at the top of `DBImpl::IngestExternalFiles` -- one sample per call (not per column family), covering all return paths. It is null-safe and self-gating on the stats level, so there is no cost when statistics are off, and ingestion is not a hot path. Java bindings are kept in sync per the `statistics.h` requirement; the C API needs no change. Reviewed By: xingbowang Differential Revision: D107721260 fbshipit-source-id: 0705f9e0f7392329a7bcbdbd9f3afd34594d20bb |
||
|
Peter Dillinger
|
8053b9414f |
Change default compression from Snappy to LZ4 (#14818)
Summary: The historical default block compression `kSnappyCompression` dates to when Snappy was the obvious fast/cheap choice. On modern server CPUs LZ4 matches or beats Snappy on compression ratio while decompressing substantially cheaper, so it is a better default. This changes the default `ColumnFamilyOptions::compression` to `kLZ4Compression`, with a runtime fallback of LZ4 -> Snappy -> none depending on what is compiled into the binary (new `GetDefaultCompressionType()` in util/compression.h). Only column families that do not explicitly set `compression` are affected (including compaction output when `CompactionOptions::compression == kDisableCompressionOption`), and only newly written SST files; existing data is read as before. Doc comments, the Java bindings, and the sorted_run_builder example are updated to describe the new default. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14818 Test Plan: Adjusted two unit tests that implicitly depended on the old Snappy default to pin `compression = kSnappyCompression`: db_iterator_test's ReadAhead (readahead byte thresholds assume Snappy-sized files) and compaction_service_test's CustomFileChecksum (LSM shape after auto-compaction determined whether a manual CompactRange had work to do). This change is primarily validated by performance testing. Added a `compressreject` db_bench benchmark (output buffer sized just below the predicted compressed size) to measure the cost of attempting then declining compression, alongside `compress`/`uncompress`. Both db_bench and sst_dump compress/decompress a modest block at a time, as in a real workload. sst_dump on SST files from production workloads (4 files, 16KB blocks, single thread) on recent server-class AMD, Intel, and ARM CPUs, LZ4 vs Snappy: - Compression ratio: comparable; LZ4 is slightly smaller on the more compressible files (up to ~8%) and within ~2% on the rest. - Compression (write) CPU: a wash, within ~2% either direction. - Decompression (read) CPU: the clear win -- Snappy costs ~1.2x-1.5x as much as LZ4, i.e. LZ4 saves ~25-30% read CPU, consistently across AMD, Intel, and ARM. db_bench synthetic workload (100-byte values), at 1 / 12 / 160 threads, LZ4 vs Snappy: - Compression throughput: LZ4 ~10-30% higher. - Decompression throughput: LZ4 much higher, and the advantage grows with core count -- from ~+12% at 12 threads to ~+45-50% at 160 threads, i.e. better multi-core scaling. - Rejection (insufficient ratio) path: comparable; LZ4 ~15-18% faster on compressible-but-rejected blocks and Snappy within ~10% on barely-compressible blocks. No meaningful regression, confirming incompressible data is still efficiently detected and stored raw. Reviewed By: xingbowang Differential Revision: D107536490 Pulled By: pdillinger fbshipit-source-id: f8abaee630d782674778338148e36ed0c84e3661 |
||
|
Michael Huang
|
9270dc8149 |
Add blob_cache_read_byte perf context counter (#14792)
Summary: Added blob_cache_read_byte in the rocksdb::PerContextBase to expose the blob cache read bytes when blob cache is enabled. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14792 Reviewed By: xingbowang Differential Revision: D106528572 Pulled By: mikechuangmeta fbshipit-source-id: 555b2f01785bb819e62ed834ee45f0436dfb2875 |
||
|
Peter Dillinger
|
e82af29adb |
Makefile fix and speed up 'clean' (#14767)
Summary: * Fix Makefile default target (was ordered after a folly target) * Improved the speed of `make clean` by using just one `find` and by pruning "hidden" .* and third-party directories that should not be modified anyway. * Reduce excessive output from `make clean` Pull Request resolved: https://github.com/facebook/rocksdb/pull/14767 Test Plan: manual Reviewed By: mszeszko-meta Differential Revision: D105972958 Pulled By: pdillinger fbshipit-source-id: 2c0f6097c74c3129b815450f23c19ef07bfbe656 |
||
|
Xingbo Wang
|
21723bbbef |
Add async WAL precreation
Summary: - Add experimental immutable `DBOptions::async_wal_precreate` to reserve and open one future WAL on a background HIGH-priority task, with sanitization that disables the optimization when WAL recycling is configured. - Split WAL creation into open/preallocate and start phases so `SwitchMemtable()` can consume a prepared WAL after writing normal WAL metadata, wait for in-flight precreation, fall back to synchronous creation, and delete an unstarted prepared WAL on start failure. - Keep WAL numbering, close, recovery, and read-only open safe for empty future WAL files left by async precreation; `error_if_wal_file_exists=true` now rejects non-empty WALs while tolerating empty WALs. - Add public option plumbing for the C API, options parsing/stringification, random option testing, `db_bench`, `db_stress`, and crash-test configuration. - Add WAL precreate statistics counters plus Java `TickerType`/JNI mappings, and update C++, C, and Java read-only-open documentation for the empty-WAL behavior. - Add focused WAL/option/C/Java tests for async precreate ready/wait/failure/recovery paths, read-only WAL detection, option sanitization, and API plumbing, plus write-flow docs and unreleased history entries for the new feature and behavior change. PR https://github.com/facebook/rocksdb/pull/14738 Reviewed By: pdillinger Differential Revision: D105020559 fbshipit-source-id: 5059b424702e021abb8de65ceeb6d3b975280ffc |
||
|
Peter Dillinger
|
87c554b492 |
Persist compacted manifest size for auto-tuning across DB::Open (#14725)
Summary: last_compacted_manifest_file_size_ drives TuneMaxManifestFileSize() to compute the manifest rotation threshold, but it started at 0 on every DB::Open and was only populated after the first manifest rotation. This is really only a problem with reuse_manifest_on_open, because no fresh manifest is created on open. Add a new forward-compatible (safe-to-ignore) MANIFEST tag kLastCompactedManifestFileSize that records the approximate compacted manifest size at the end of WriteCurrentStateToManifest. During recovery, the value is loaded and used to immediately tune the rotation threshold. The record includes a rough estimate of its own overhead (~15 bytes) and must be the last record written by WriteCurrentStateToManifest for accurate estimation. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14725 Test Plan: Extended AutoTuneManifestSize in db_etc3_test to close and reopen with reuse_manifest_on_open after establishing a known auto-tuning state. Verifies that the manifest file number is preserved (no spurious rotation) and that subsequent CF additions don't trigger rotation -- proving the persisted compacted size keeps the tuned threshold correct. Verified the test fails when the recovery loading is disabled. Relax a fragile Java test that was dependent on the exact size of the manifest file. SHORT_TEST=1 ./tools/check_format_compatible.sh Reviewed By: anand1976 Differential Revision: D104464522 Pulled By: pdillinger fbshipit-source-id: 4f5d22d2e149bd40a523ee11780e5e3344803c19 |
||
 Danny Chen
|
2e7cf42cda |
Add MANIFEST_VALIDATION_FAILURE_COUNT statistic (#14657)
Summary: CONTEXT: The manifest validation on close feature (verify_manifest_content_on_close) detects corruption but does not increment any statistics counter, making it harder to monitor in production. WHAT: Add a new ticker MANIFEST_VALIDATION_FAILURE_COUNT that is incremented each time content validation detects manifest corruption during DB::Close(). The counter fires per corruption detection, so it can increment up to 2 times per close (once on initial check, once after rewrite attempt). Updated all existing manifest validation tests to verify the counter value. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14657 Test Plan: - All 7 manifest validation tests pass with new stat assertions - 5x repeat with COERCE_CONTEXT_SWITCH=1 shows no flakiness - Full version_set_test suite (212 tests) passes Reviewed By: anand1976 Differential Revision: D102404260 Pulled By: dannyhchen fbshipit-source-id: 21a0aa1ad8de12a935caf5642e41ccf2a47b46d9 |
||
|
Josh Kang
|
3f51c0a185 |
Convert sequential single deletes into range tombstones (#14448)
Summary:
Add a read-path optimization that converts contiguous point tombstones into range tombstones during forward/reverse iteration. When a configurable threshold of consecutive point deletions (kTypeDeletion, kTypeDeletionWithTimestamp, kTypeSingleDeletion — with no live keys between them) is detected, a range tombstone covering `[first_tombstone_key, next_live_key)` is inserted into the active mutable memtable. This benefits future iterators by enabling efficient skipping via range tombstone fragmentation.
If there is a memtable switch during the read iteration, then the range deletion entry is discarded.
The inserted range tombstones are logically redundant (they don't delete anything that isn't already deleted by point tombstones), skip WAL (they're a derived optimization regenerated by future reads on crash), and use the max tombstone sequence number so they don't interfere with newer writes.
## Key changes
- **New option `min_tombstones_for_range_conversion`** (`AdvancedColumnFamilyOptions`): Threshold of contiguous point tombstones before converting to a range tombstone. Default 0 (disabled). Dynamically changeable via `SetOptions()`.
- **`DBIter` tracking logic** (`db/db_iter.cc`): Tracks contiguous tombstones during `FindNextUserEntryInternal()` (forward) and `PrevInternal()` (reverse). When a live key terminates a run that meets the threshold, `MaybeInsertRangeTombstone()` inserts `[first_tombstone, live_key)` into the active memtable.
- **`FindValueForCurrentKey` `found_visible` output** (`db/db_iter.cc`): Distinguishes "key deleted at this snapshot" from "no visible entries" so reverse tracking doesn't treat post-snapshot keys as tombstones.
- **`IterKey::Swap()`** (`db/dbformat.h`): Efficiently tracks reverse tombstone run end keys without extra allocations.
- **`MemTable::AddLogicallyRedundantRangeTombstone()`** (`db/memtable.cc`): Concurrent-safe range tombstone insertion into the active memtable. Range tombstone skiplist always uses concurrent inserts.
- **`ConstructFragmentedRangeTombstones` race fix** (`db/db_impl/db_impl_write.cc`): Moved after `MarkImmutable()` to prevent lost entries.
- **`MarkImmutable` ordering fix** (`db/memtable_list.cc`): Called before `current_->Add()` to close a race window.
- **Prefix filter awareness** (`db/db_iter.cc`): Tombstone tracking scoped to the seek prefix when prefix filtering is active. See dedicated section below.
- **Transaction awareness** (`db/db_iter.cc`): Tombstones with `seq > snapshot` excluded from tracking. `min_uncommitted` guard uses `insert_seq` (which may be bumped to `earliest_seq`) instead of `range_tomb_max_seq_`. See dedicated section below.
- **Duplicate range check**: Skips insertion if the memtable already covers `[start, end)`.
- **New statistics**: `READ_PATH_RANGE_TOMBSTONES_INSERTED` and `READ_PATH_RANGE_TOMBSTONES_DISCARDED`.
- **Memtable MultiGet batch lookup** (`memtable/inlineskiplist.h`, `db/memtable.cc`): `InlineSkipList::MultiGet()` with cached search path ("finger") for sorted key lookups.
- **New option `memtable_batch_lookup_optimization`** (`AdvancedColumnFamilyOptions`): Enables batch lookup for memtable MultiGet. Default false. Immutable.
## Deciding Range Tombstone Seqno
- The range tombstone is inserted with `insert_seq = max(range_tomb_max_seq_, earliest_seq)` where `range_tomb_max_seq_` is the maximum sequence number across all point tombstones in the contiguous run, and `earliest_seq` is the memtable's earliest sequence number. This preserves the memtable's `earliest_seqno_` invariant.
- If the iterator's snapshot sequence (`sequence_`) predates the memtable's `earliest_seq`, insertion is skipped entirely to avoid unintentionally covering entries between `sequence_` and `earliest_seq`.
## ConstructFragmentedRangeTombstones Race Fix
- `MarkImmutable()` and `ConstructFragmentedRangeTombstones()` are now called before `mutex_.Lock()` in `SwitchMemtable`, keeping this work outside the DB mutex. `MarkImmutable()` blocks concurrent `AddLogicallyRedundantRangeTombstone()` calls via `immutable_mutex_`, ensuring no range tombstones are inserted after the fragmented list is built. `MarkImmutable()` is idempotent, so `MemTableList::Add()` calling it again inside the mutex is harmless.
## Prefix Filter Safety
- When prefix filtering is active, the BBTI bloom filter may reject SST files outside the seek prefix, but the memtable (no bloom filter) returns keys across prefix boundaries. Tombstone tracking is scoped to the seek prefix so that converted range tombstones cannot cover live keys hidden in filtered files.
- `total_order_seek=true` disables prefix filtering — all files are visible, so tombstones safely span prefix boundaries.
- **Behavior change**: Seeking to an out-of-domain key with `total_order_seek=false` now treats it as total-order (prefix_ not set). When `prefix_same_as_start=true`, iterating past an out-of-domain key cleanly invalidates the iterator instead of calling `Transform()` on it (which was UB in release builds with `FixedPrefixTransform`). This is a requirement because an incorrect iterator scan could lead to a range tombstone covering a live key.
## Transaction Support
- Tombstones written by the transaction's own uncommitted writes (sequence > snapshot) are now excluded from contiguous tombstone tracking entirely at the tracking site in `FindNextUserEntryInternal()` and `PrevInternal()`. Previously, tracking relied on a `min_uncommitted` check at insertion time, but this was insufficient — a transaction's own Delete with `seq > snapshot` could extend a run of committed tombstones, and the resulting range tombstone would cover data visible to other snapshots.
- The fix skips any tombstone with `ikey_.sequence > sequence_` during tracking. If a transaction-owned tombstone appears mid-run, it flushes the accumulated committed run first, then resets tracking. This ensures only tombstones visible to the current snapshot are ever converted.
- Both WritePrepared and WriteUnprepared transactions are supported with dedicated test coverage:
- **WritePrepared**: When tombstones are committed before `Prepare()`, their seqnos are below `min_uncommitted` and insertion proceeds safely. When `Prepare()` happens first, tombstone seqnos exceed `min_uncommitted` and insertion is blocked.
- **WriteUnprepared**: Multiple unprepared batches with different seqno ranges are handled correctly. Own transaction Deletes that extend a committed tombstone run block insertion of the entire run. After rollback, data correctness is verified.
## UDT support
- When user-defined timestamps (UDT) are enabled, keys include an 8-byte timestamp suffix. The comparator, Put/Delete APIs, and ReadOptions all require timestamps.
- Forward exhaustion with UDT: `iterate_upper_bound_` is a plain user key without a timestamp suffix. It is padded with min timestamp via `AppendKeyWithMinTimestamp()` so it sorts after all entries with this user key, preserving the exclusive bound semantics.
- Reverse exhaustion with UDT: The end key comes from either the previous live key (which already has a proper timestamp suffix) or the seek target set by `SetSavedKeyToSeekForPrevTarget()` (which appends a timestamp via `SetInternalKey(..., timestamp_ub_)` and `UpdateInternalKey(..., ts)`). In both cases, the end key is already properly timestamped, so no additional padding is needed unlike forward exhaustion.
- Contiguous tombstone detection works correctly with UDT because the underlying `kTypeDeletionWithTimestamp` entries are tracked the same way as `kTypeDeletion`.
## Concurrent Iterators
- Should concurrent iterators happen to read the range at the same time, both will produce the same range and seqno entry. Only one will be accepted by the skip list and the others will be rejected. Future iterators will read the range and not even attempt to insert the range.
- There is nothing preventing similar ranges from being inserted however. Two iterators can produce overlapping ranges, but this protection would be complicated to implement and there is no evidence that it is a likely scenario yet.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/14448
Test Plan:
- **Unit tests** (`db/db_iterator_test.cc`): `ReadPathRangeTombstoneTest` parameterized by forward/reverse with cases for basic insertion, non-contiguous (below threshold), memtable switch, exhausted iterator with/without bounds, direction change, mixed Delete/SingleDelete, single-delete-only runs, snapshot predating memtable, block cache tier incomplete, skip when covered by existing range, UDT basic scan, UDT exhaustion, prefix filter cross-prefix scan (`PrefixFilterCrossPrefixScanCoversLiveKey` with default/total_order_seek/prefix_same_as_start variants), stale ikey from forward-then-reverse scan (`StaleIkeyFromForwardThenReverse`), and reseek stale ikey (`ReseekStaleIkey`).
- **Concurrency test** (`db/db_test2.cc`): `DBTestConcurrentRangeTombstoneConversions` parameterized by `(allow_concurrent_memtable_write, min_tombstones_for_range_conversion)` with mixed writers, deleters, range deleters, and concurrent forward/reverse readers.
- **Transaction tests** (`utilities/transactions/write_prepared_transaction_test.cc`, `write_unprepared_transaction_test.cc`): Tests for WritePrepared (insertion allowed when tombstones committed before prepare, blocked when after; seqno bump shadowing prepared writes `RangeTombstoneSeqnoBumpShadowsPreparedWrite`) and WriteUnprepared (multiple batches, extended visibility with CalcMaxVisibleSeq, own deletions with rollback).
- **IterKey::Swap tests** (`db/dbformat_test.cc`): `IterKeySwapTest` parameterized over `(key_len, copy, use_secondary)` × 2 covering all inline/heap/pinned/secondary combinations.
- **InlineSkipList MultiGet tests** (`memtable/inlineskiplist_test.cc`): Basic, exact matches, empty, single key, randomized validation against `std::set::lower_bound`, duplicate keys with callback walk, and concurrent MultiGet with read-after-write consistency.
- **Memtable MultiGet tests** (`db/db_basic_test.cc`): Batch lookup, overwrite, flush, merge, disabled by default, paranoid checks, and snapshot tests.
- **Stress test coverage**: `min_tombstones_for_range_conversion` and `memtable_batch_lookup_optimization` options added to `db_crashtest.py` and `db_stress` flags.
- `make check` passes all tests.
### Benchmark results
Tombstones scattered randomly in clusters via `seek_nexts_to_delete` for realistic workloads.
**DB Setup A (scattered deletes, 100 per seek)**:
```
# Step 1: Create and compact
./db_bench --benchmarks=fillseq,compact --seed=1 --compression_type=none --num=1000000 --db=<DB>
# Step 2: Scatter tombstones (5000 seeks × 100 deletes ≈ 500k tombstones)
./db_bench --benchmarks=seekrandom,flush --seed=1 --compression_type=none --num=2000 \
--seek_nexts=0 --seek_nexts_to_delete=100 --use_existing_db=1 --threads=1 --db=<DB>
```
**DB Setup A2 (scattered deletes, 8 per seek)**:
```
# Step 1: Create and compact
./db_bench --benchmarks=fillseq,compact --seed=1 --compression_type=none --num=1000000 --db=<DB>
# Step 2: Scatter tombstones (5000 seeks × 8 deletes ≈ 40k tombstones)
./db_bench --benchmarks=seekrandom,flush --seed=1 --compression_type=none --num=2000 \
--seek_nexts=0 --seek_nexts_to_delete=8 --use_existing_db=1 --threads=1 --db=<DB>
```
**DB Setup B (no deletes)**:
```
./db_bench --benchmarks=fillseq,compact --seed=1 --compression_type=none --num=1000000 \
[--key_size=100] --db=<DB>
```
**Read workload** (same for all):
```
./db_bench --benchmarks=seekrandom --seek_nexts=100 --threads=8 \
--reverse_iterator={true,false} --seed=1 --use_existing_db=1 \
--compression_type=none --num=1000000 --duration=10 \
--disable_auto_compactions \
[--key_size=100] [--min_tombstones_for_range_conversion=X] --db=<DB_COPY>
```
Each workload averaged over 3 runs.
**Table 1: seekrandom forward, scattered deletes (2000 seeks × 100 deletes/seek)**
| Variant | avg ops/s | % vs main |
|---------|-----------|-----------|
| main | 2,895 | - |
| threshold=0 | 2,869 | -0.9% |
| threshold=8 | 287,334 | +9,824% |
**Table 2: seekrandom reverse, scattered deletes (2000 seeks × 100 deletes/seek)**
| Variant | avg ops/s | % vs main |
|---------|-----------|-----------|
| main | 544 | - |
| threshold=0 | 548 | +0.7% |
| threshold=8 | 206,491 | +37,860% |
**Table 3: seekrandom forward, scattered deletes (2000 seeks × 8 deletes/seek)**
| Variant | avg ops/s | % vs main |
|---------|-----------|-----------|
| main | 194,049 | - |
| threshold=0 | 195,703 | +0.9% |
| threshold=8 | 310,740 | +60.1% |
**Table 4: seekrandom reverse, scattered deletes (2000 seeks × 8 deletes/seek)**
| Variant | avg ops/s | % vs main |
|---------|-----------|-----------|
| main | 63,854 | - |
| threshold=0 | 69,266 | +8.5% |
| threshold=8 | 218,101 | +241.6% |
**Table 5: seekrandom forward, no deletes (regression check)**
| Variant | key=16B avg ops/s | % vs main | key=100B avg ops/s | % vs main |
|---------|-------------------|-----------|---------------------|-----------|
| main | 330,901 | - | 236,048 | - |
| threshold=0 | 328,398 | -0.8% | 238,055 | +0.9% |
| threshold=8 | 332,539 | +0.5% | 233,776 | -1.0% |
**Table 6: seekrandom reverse, no deletes (regression check)**
| Variant | key=16B avg ops/s | % vs main | key=100B avg ops/s | % vs main |
|---------|-------------------|-----------|---------------------|-----------|
| main | 261,445 | - | 192,177 | - |
| threshold=0 | 265,020 | +1.4% | 191,616 | -0.3% |
| threshold=8 | 250,881 | -4.0% | 189,239 | -1.5% |
Reviewed By: xingbowang
Differential Revision: D96203950
Pulled By: joshkang97
fbshipit-source-id: 06ba66ebde3c355f04671d1e681f1b1586e8751d
|
||
|
Josh Kang
|
5db0603613 |
Read-triggered compactions (#14426)
Summary: Add read-triggered compaction, a new feature that reduces read amplification by compacting SST files that receive high read traffic. When an SST file's read frequency (`num_reads_sampled / file_size`) exceeds a configurable threshold, it is marked for compaction to a lower level. The feature introduces two new options: a CF option `read_triggered_compaction_threshold` (default 0, disabled) and a DB option `max_periodic_compaction_trigger_seconds` (default 43200s) that controls how often the background thread re-evaluates compaction scores on quiet databases. Both options are dynamically changeable. Lowering `max_periodic_compaction_trigger_seconds` does add some overhead, but generally is minimal, so running this every couple of minutes in a production environment seems fairly reasonable. ## Key changes - **New CF option `read_triggered_compaction_threshold`** (`advanced_options.h`): When positive, files with `reads_per_byte > threshold` are marked for compaction. Files at the last non-empty level are skipped (bottommost compaction handles those separately). Marked files are sorted by hotness (reads_per_byte descending). - **New DB option `max_periodic_compaction_trigger_seconds`** (`options.h`): Replaces the hardcoded 12-hour ceiling in `ComputeTriggerCompactionPeriod()`. Essential for read-triggered compaction on quiet DBs since there are no writes to trigger score re-evaluation. - **Leveled compaction picker** (`compaction_picker_level.cc`): Adds read-triggered as the lowest-priority compaction reason in `SetupInitialFiles()`, using the existing `PickFileToCompact` helper. - **Universal compaction picker** (`compaction_picker_universal.cc`): Adds `PickReadTriggeredCompaction` as lowest priority. Refactors shared "find output level + compute overlapping inputs + create Compaction" logic from both `PickDeleteTriggeredCompaction` and `PickReadTriggeredCompaction` into `BuildCompactionToNextLevel`, handling both single-level and multi-level universal cases. - **Periodic trigger integration** (`db_impl.cc`): `TriggerPeriodicCompaction` now also fires for CFs with `read_triggered_compaction_threshold > 0`, even without time-based compaction configured. - **Stress test & db_bench support**: Both `db_stress` and `db_bench` support the new options. `db_crashtest.py` randomly enables read-triggered compaction and sets a short periodic trigger interval when enabled. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14426 Test Plan: **Unit tests**: - `compaction_picker_test` — 7 new tests: `ReadTriggeredCompactionDisabled`, `ReadTriggeredCompactionBelowThreshold`, `ReadTriggeredCompactionAboveThreshold`, `NeedsCompactionReadTriggered`, `ReadTriggeredPicksFile`, `UniversalReadTriggeredCompaction`, `ReadTriggeredSkipsLastLevel`, `UniversalReadTriggeredNoPickWhenNotMarked` - `db_compaction_test` — `ReadTriggeredCompaction` integration test verifying end-to-end behavior with sync points - Stress test coverage **Stress test**: ``` make V=1 -j "CRASH_TEST_EXT_ARGS=--duration=600 --max_key=2500000 --max_compaction_trigger_wakeup_seconds=10 --read_triggered_compaction_threshold=0.0001 --interval=600" blackbox_crash_test ``` - confirmed read triggered compactions from LOGS **Benchmark** (`db_bench`): Setup: 5M keys (100B values, 16B keys), leveled compaction, 5 levels, 4MB target file size. DB fully compacted, then 2M overlapping keys written without compaction to create L0/L1 overlap (82 files, ~294MB). LSM shape change during readrandom with read-triggered compaction: ``` BEFORE: L0=9 files (15MB), L1=4 (16MB), L2=20 (69MB), L3=49 (194MB) — 82 files, 294MB AFTER: L3=66 files (223MB) ``` | Benchmark | Config | avg ops/s | % change | |-----------|--------|-----------|----------| | readrandom (8 threads, 5M reads) | baseline (threshold=0) | 1,086,965 | — | | readrandom (8 threads, 5M reads) | threshold=0.000001, trigger=5s | 1,453,697 | **+33.7%** | Reviewed By: xingbowang Differential Revision: D97838716 Pulled By: joshkang97 fbshipit-source-id: a21fcb270c7fadd4f78d98b9c821982f220dd3f0 |
||
|
Xingbo Wang
|
1a4b1e42cc |
Add include_blob_files option to GetApproximateSizes (#14501)
Summary: **Summary:** Add a new boolean flag `include_blob_files` (default: `false`) to `SizeApproximationOptions` and a corresponding `INCLUDE_BLOB_FILES` enum value to `SizeApproximationFlags`. When set to `true`, the returned size includes an approximation of blob file data in the queried key range. **Algorithm:** The blob file size contribution is prorated using the SST size ratio: ``` blob_size_in_range ≈ total_blob_size * (sst_size_in_range / total_sst_size) ``` The blob-to-SST ratio (`total_blob_size / total_sst_size`) is computed once before the per-range loop, so iterating levels and blob files only happens once per `GetApproximateSizes` call regardless of how many ranges are queried. The per-range SST size (`ApproximateSize`) is computed once and shared between `include_files` and `include_blob_files`. **Limitations:** - Assumes blob data is distributed proportionally to SST data across the key space. May be inaccurate if blob value sizes vary significantly across different key ranges (e.g., one range has large blobs while another has small ones). - If there are no SST files (all data in memtables), the blob size contribution will be 0 even if blob files exist on disk. **Changes:** - `include/rocksdb/options.h`: New `include_blob_files` field in `SizeApproximationOptions`; updated doc comments for `include_memtables`/`include_files` - `include/rocksdb/db.h`: New `INCLUDE_BLOB_FILES` in `SizeApproximationFlags` enum, updated flags-to-options mapping - `include/rocksdb/c.h`: New `rocksdb_size_approximation_flags_include_blob_files` C API enum value - `java/`: Added `INCLUDE_BLOB_FILES` to `SizeApproximationFlag.java` and JNI flag mapping in `rocksjni.cc` - `db/db_impl/db_impl.cc`: Blob-to-SST ratio computed once before loop, SST range size computed once per range and shared - `db_stress_tool/db_stress_test_base.cc`: Randomized `include_blob_files` in stress test Pull Request resolved: https://github.com/facebook/rocksdb/pull/14501 Test Plan: - New `DBBlobBasicTest.GetApproximateSizesIncludingBlobFiles` — verifies: - Size with blobs > without (full range) - Non-overlapping range returns 0 - Partial range returns proportionally less than full range - `SizeApproximationFlags` API works - Multi-range query: two sub-ranges sum approximately to the full-range result - Stress test now exercises the new option randomly Reviewed By: hx235 Differential Revision: D97984211 Pulled By: xingbowang fbshipit-source-id: e9127eac3308687fd4f0b17a771fd61fba6a8380 |
||
|
Josh Kang
|
3ad23b2d94 |
Support automated interpolation search (#14383)
Summary: Add automatic per-block interpolation search selection (`kAuto` mode) for index blocks. During SST construction, each index block's key distribution is analyzed using the coefficient of variation (CV) of gaps between restart-point keys. Blocks with uniformly distributed keys are flagged via a new bit in the data block footer, and at read time, `kAuto` resolves to interpolation search for uniform blocks and binary search otherwise. ## Key changes - **New `BlockSearchType::kAuto` enum value**: Resolves per-block at read time to either `kInterpolation` or `kBinary` based on the block's uniformity flag. Falls back to `kBinary` on older versions that don't recognize it. - **Write-path uniformity analysis**: `BlockBuilder::ScanForUniformity()` uses Welford's online algorithm to incrementally compute the CV of key gaps at restart points. The result is stored in a new bit (bit 30) of the data block footer's packed restart count. - **New table option `uniform_cv_threshold`** (default: -1 `disabled`): Controls how strict the uniformity check is. Set to negative to disable. Exposed in C++, Java (JNI), and `db_bench`. - **Code reorganization**: Block entry decode helpers (`DecodeEntry`, `DecodeKey`, `DecodeKeyV4`, `ReadBe64FromKey`) moved from `block.cc` to a new shared header `block_util.h` so they can be reused by `BlockBuilder` on the write path. - **New histogram `BLOCK_KEY_DISTRIBUTION_CV`**: Records the CV (scaled by 10000) of each index block's key distribution for observability. - **Java bindings**: `IndexSearchType.kAuto`, `uniformCvThreshold` getter/setter, JNI portal constructor signature updated, and `HistogramType.BLOCK_KEY_DISTRIBUTION_CV` added. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14383 Test Plan: - `IndexBlockTest.IndexValueEncodingTest` parameterized to include `kAuto` search type alongside `kBinary` and `kInterpolation`, verifying correct seek/iteration behavior across all combinations of key distributions, restart intervals, and key lengths. - Uniformity detection validated: blocks with uniform key distribution correctly set `is_uniform = true`, blocks with clustered/non-uniform keys set `is_uniform = false`. - Stress test coverage - Updated check_format_compatible to also include a "uniform" dataset. By default using uniform_cv_threshold=-1 does not result in an incompatibility issues. When manually changing the threshold (e.g. `uniform_cv_threshold=1000`), I see `bad block contents`, which is expected ## Benchmark readrandom with `fillrandom,compact -seed=1 --statistics`: | Benchmark | Branch | Params | avg ops/s | % change vs main | CV P50 | |-----------|--------|--------|-----------|------------------|--------| | readrandom | main | `binary_search, shortening=1` | 335,791 | baseline | N/A | | readrandom | feature | `binary_search, shortening=1` (default) | 335,749 | -0.0% | 1,500 | | readrandom | feature | `auto_search, shortening=1` (kAuto) | 366,832 | **+9.2%** | 1,500 | | readrandom | feature | `interpolation_search, shortening=1` | 366,598 | **+9.2%** | 1,500 | | readrandom | feature | `auto_search, shortening=2` (kAuto) | 344,631 | **+2.6%** | 1,030,000 | | readrandom | feature | `interpolation_search, shortening=2` | 201,178 | **-40.1%** | 1,030,000 | As seen with shortening=2, a non-uniform distribution produces a high CV, which does not use interpolation search. ## Write benchmark There is a write overhead which scans each restart entry for a block upon Finish. In practice this is very low because currently it is only applied to index blocks. See cpu profile (https://fburl.com/strobelight/io5hwj9h) here of `-benchmarks=fillseq,compact -compression_type=none -disable_wal=1`. Only 0.08% attributed to `ScanForUniformity`. Reviewed By: pdillinger Differential Revision: D94738890 Pulled By: joshkang97 fbshipit-source-id: 9661ac593c5fef89d49f3a8a027f1338a0c96766 |
||
|
Josh Kang
|
f25fb41da6 |
Add option to validate sst files in the background on DB open (#14322)
Summary: Add `open_files_async` option for faster DB startup. When enabled, SST file opening and validation is deferred to a background thread after `DB::Open` returns, reducing startup latency for databases with many SST files. WAL recovery remains synchronous. To support this, `FindTable` is extended with a pinning mechanism that stores the cache handle directly on `FileMetaData` via a new `PinnedTableReader` class, and sets the table reader atomically so subsequent reads skip cache lookups. `FileDescriptor::table_reader` is replaced with `PinnedTableReader pinned_reader` which wraps a `std::atomic<TableReader*>` with acquire/release ordering to safely handle concurrent access between the background opener and read threads. Should validations fail, the background opener sets a `kAsyncFileOpen` background error. Future read requests will look up the table reader again via the cache, and if any validations fail there it will get propagated to the user (existing behavior when `max_open_files > 0`). This feature is most useful when `max_open_files=-1`, because otherwise file opening is already capped at 16 files and DB open should be fast. ## Restrictions - This feature also is incompatible with fifo compaction because fifo compaction requires reading table properties under DB mutex. When table reader is unpinned, this may cause a DB hang. - This feature is also incompatible with `skip_stats_update_on_db_open=false` because it will result in even longer DB open ## Key changes - New `open_files_async` DB option with C, Java, and `db_bench` bindings - `BGWorkAsyncFileOpen` background worker that opens all SST files post-`DB::Open`, with shutdown awareness via `shutting_down_` flag - New `PinnedTableReader` class in `version_edit.h` — thread-safe wrapper holding `std::atomic<TableReader*>` and `Cache::Handle*` with proper acquire/release ordering. Replaces the old `FileDescriptor::table_reader` raw pointer and `FileMetaData::table_reader_handle` - Extract `LoadTableHandlersHelper` into `db/version_util.cc` — shared between `VersionBuilder::LoadTableHandlers` (for version edits during recovery) and `BGWorkAsyncFileOpen` (for base storage post-open) - `FindTable` extended with `pin_table_handle` and `out_table_reader` params — when pinning is enabled, the table reader is stored on `FileMetaData` so Get/MultiGet/Iterator skip redundant cache lookups. `FindTable` now performs the pinned-reader fast-path check internally instead of requiring callers to check `fd.table_reader` beforehand - Note: pinning is explicit (not default) because some callers create temporary `FileMetaData`s that would need to properly clean up table handles - `CompactedDBImpl` updated to use `FindTable` + pinning instead of raw `fd.table_reader` access for Get/MultiGet - New `kAsyncFileOpen` background error reason in `listener.h` and `error_handler.cc` - Add a check in ~DBImpl to ensure async file open task has not been forgotten to be scheduled in (future) subclasses of DBImpl. Certain subclasses that never use it will need to explicitly mark it. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14322 Test Plan: - `OpenFilesAsyncTest` parameterized over `num_flushes` (1, 20), `ReadType` (Get, MultiGet, Iterator), `max_open_files` (-1, 10), and `read_only` (true, false) - **ConcurrentFileAccess**: concurrent reads and compactions race with async opener - **AfterRead**: reads happen before async opener, verifying lazy open and that the opener sees already-pinned readers - **BeforeRead**: async opener completes first, verifying reads use pre-loaded table readers - **Shutdown**: DB closes before async opener starts, verifying clean cancellation with 0 file opens - **Error**: corrupted SST files, verifying `kAsyncFileOpen` background error is set and reads return corruption - **DropColumnFamily**: CF dropped before async opener runs, verifying the opener gracefully skips dropped CFs - Added to crash test ### Benchmark To simulate a high-latency remote filesystem, I set up a virtual filesystem with dm-delay using 10ms reads, 0 ms writes. ``` # Generate a DB with many L0 files TEST_TMPDIR=/data/users/jkangs/dm-delay-test/mnt ./db_bench -benchmarks=fillseq -disable_auto_compactions=true -write_buffer_size=1000 -num=1000000 ``` ``` ./db_bench -use_existing_db=true -db=/data/users/jkangs/dm-delay-test/mnt/dbbench -benchmarks=readrandom -reads=1 -report_open_timing=true -open_files_async=true -use_direct_reads -file_opening_threads=1 -skip_stats_update_on_db_open OpenDb: 25.1419 milliseconds ``` ``` ./db_bench -use_existing_db=true -db=/data/users/jkangs/dm-delay-test/mnt/dbbench -benchmarks=readrandom -reads=1 -report_open_timing=true -open_files_async=false -use_direct_reads -file_opening_threads=1 -skip_stats_update_on_db_open OpenDb: 23109.4 milliseconds ``` ### No read regressions On main branch ``` ./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30 readrandom : 4.827 micros/op 1657100 ops/sec 30.005 seconds 49720992 operations; 183.3 MB/s (6198999 of 6198999 found) ``` On this branch ``` ./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30 readrandom : 4.863 micros/op 1644808 ops/sec 30.007 seconds 49354992 operations; 182.0 MB/s (6099999 of 6099999 found) ./db_bench -use_existing_db=true -db=/dev/shm/dbbench -benchmarks=readrandom -seed=1 -threads=8 -duration=30 -open_files_async=true readrandom : 4.803 micros/op 1665392 ops/sec 30.004 seconds 49968992 operations; 184.2 MB/s (6222999 of 6222999 found) ``` Reviewed By: pdillinger, xingbowang Differential Revision: D93538033 Pulled By: joshkang97 fbshipit-source-id: 32ac70c112cd733b7c1e1c1e2e7ce6422318a5ae |
||
|
Josh Kang
|
901c88e37b |
Separate keys and values in data blocks (#14287)
Summary: Introduce new table option with separated key-value storage in data blocks. This PR implements a new SST block format where keys and values are stored in separate sections within data blocks, rather than interleaved. Keys are stored first, followed by all values in a contiguous section. The motivation is better cpu cache hit rate during seeks and potentially better compression. The additional storage cost is a varint per restart point, and 4 bytes additional block footer. For a data block with a restart interval of 16, it is approximately 1 bit of overhead per entry. But compression actually performs better, resulting in ~3% storage savings from benchmark. For now I've opted to not separate kvs in non-data blocks since restart interval for those blocks is typically 1, and values are typically small and probably better inlined. ### New block layout ``` +------------------+ | Keys Section | <- Key entries with delta encoding +------------------+ | Values Section | <- (new) Values stored contiguously +------------------+ | Restart Array | <- Fixed32 offsets to restart points +------------------+ | Values Offset | <- (new) 4 bytes: offset to values section | Footer | <- 4 bytes: packed index_type + num_restarts +------------------+ ``` ### Entry Format **At restart points** ``` +--------------+------------------+----------------+-----------------+-----------+ | shared (v32) | non_shared (v32) | value_sz (v32) | value_off (v32) | key_delta | +--------------+------------------+----------------+-----------------+-----------+ ``` **At non-restart points** ``` +--------------+------------------+----------------+-----------+ | shared (v32) | non_shared (v32) | value_sz (v32) | key_delta | +--------------+------------------+----------------+-----------+ ``` - `value_offset` is only stored at restart points to save space - For non-restart entries, value offset is computed as: `prev_value_offset + prev_value_size` ### Forward Compatibility - We make use of reserved block footer bits to mark if a block has separated kv format. Should an older version read this, it will assume a very large block restart interval and result in a corruption error. ### Key Changes - **BlockBuilder**: Accumulates values in a separate buffer; value offsets are stored only at restart points (other entries derive offset from previous value's position). There is an additional memcpy cost to place the value data after the key data. - **Block iteration**: Iteration now needs to know if we are at a restart point. This will rely on `cur_entry_idx_`, which was previously only used for per-kv checksum purposes. In this new format, we also need to know the block_restart_interval, which was previously also only calculated for per-kv checksums. - **Table properties**: Store `data_block_restart_interval`, `index_block_restart_interval`, and `separate_key_value_in_data_block` in table properties Pull Request resolved: https://github.com/facebook/rocksdb/pull/14287 Test Plan: - Extended block_test, table_test, compaction_test to contain new separated_kv param - Added new parameter to crash test --- ## Benchmark ### Varying Value Size **Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --value_size=<X> --benchmarks=fillrandom,compact` **Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq` | Value Size | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | | |----------:|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:| | | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | | 1 | 253,280 | 264,977 | 0.516 | 0.497 (**-3.7%**) | 596,328 | 586,625 (**-1.6%**) | 5,904,681 | 6,069,581 (**+2.8%**) | 5.1 | 4.2 (**-18.6%**) | | 16 | 235,757 | 242,367 | 0.572 | 0.533 (**-6.8%**) | 570,751 | 572,815 (**+0.4%**) | 5,371,138 | 5,604,908 (**+4.4%**) | 14.7 | 13.4 (**-8.5%**) | | 100 | 264,299 | 265,427 | 0.461 | 0.454 (**-1.5%**) | 323,696 | 332,790 (**+2.8%**) | 4,239,725 | 4,232,416 (**-0.2%**) | 38.8 | 37.6 (**-3.2%**) | | 1,000 | 238,992 | 242,764 | 2.349 | 2.329 (**-0.9%**) | 244,608 | 261,403 (**+6.9%**) | 1,285,394 | 1,265,868 (**-1.5%**) | 342.1 | 342.0 (**-0.0%**) | ### Varying Block Restart Interval **Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --block_restart_interval=<X> --benchmarks=fillrandom,compact` **Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq` | BRI | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | | |----:|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:| | | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | | 1 | 251,654 | 263,707 | 0.453 | 0.485 (**+7.1%**) | 334,653 | 328,708 (**-1.8%**) | 4,194,342 | 3,954,291 (**-5.7%**) | 40.6 | 42.4 (**+4.5%**) | | 4 | 253,797 | 252,394 | 0.476 | 0.476 (**+0.0%**) | 332,719 | 341,676 (**+2.7%**) | 4,135,691 | 4,051,151 (**-2.0%**) | 39.2 | 39.3 (**+0.3%**) | | 8 | 260,143 | 262,273 | 0.496 | 0.460 (**-7.3%**) | 330,859 | 337,567 (**+2.0%**) | 4,144,081 | 4,187,389 (**+1.0%**) | 38.9 | 38.1 (**-2.1%**) | | 16 | 252,875 | 263,176 | 0.464 | 0.455 (**-1.9%**) | 323,783 | 335,418 (**+3.6%**) | 4,127,310 | 4,217,028 (**+2.2%**) | 38.8 | 37.6 (**-3.2%**) | | 32 | 260,224 | 269,422 | 0.464 | 0.451 (**-2.8%**) | 304,001 | 314,989 (**+3.6%**) | 4,310,162 | 4,247,248 (**-1.5%**) | 38.8 | 37.3 (**-3.8%**) | ### Varying Compression **Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --compression_type=<X> [--compression_level=<N>] --benchmarks=fillrandom,compact` **Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq` | Compression | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | | |------------|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:| | | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | | None | 252,494 | 260,552 | 0.413 | 0.419 (**+1.5%**) | 356,290 | 371,535 (**+4.3%**) | 4,479,261 | 4,507,133 (**+0.6%**) | 73.5 | 73.6 (**+0.2%**) | | LZ4 | 246,010 | 256,360 | 0.477 | 0.455 (**-4.6%**) | 342,497 | 345,882 (**+1.0%**) | 4,400,570 | 4,268,102 (**-3.0%**) | 38.3 | 37.6 (**-2.0%**) | | ZSTD (L3) | 254,748 | 258,556 | 1.067 | 1.055 (**-1.1%**) | 176,724 | 177,566 (**+0.5%**) | 2,736,841 | 2,717,739 (**-0.7%**) | 32.9 | 31.3 (**-4.7%**) | | ZSTD (L6) | 256,459 | 259,388 | 1.556 | 1.462 (**-6.0%**) | 177,390 | 176,691 (**-0.4%**) | 2,754,336 | 2,688,682 (**-2.4%**) | 32.8 | 31.1 (**-5.1%**) | ### Varying Block Size **Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --block_size=<X> --benchmarks=fillrandom,compact` **Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq` | Block Size | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | | |-----------:|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:| | | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | | 4 KB | 263,203 | 260,362 | 0.469 | 0.461 (**-1.7%**) | 324,178 | 332,249 (**+2.5%**) | 4,231,537 | 4,217,763 (**-0.3%**) | 38.8 | 37.6 (**-3.1%**) | | 16 KB | 252,742 | 263,161 | 0.426 | 0.428 (**+0.5%**) | 227,805 | 222,873 (**-2.2%**) | 5,146,997 | 5,081,080 (**-1.3%**) | 38.1 | 36.7 (**-3.6%**) | | 64 KB | 257,490 | 260,225 | 0.423 | 0.414 (**-2.1%**) | 86,807 | 91,586 (**+5.5%**) | 5,380,403 | 5,372,372 (**-0.1%**) | 36.3 | 35.0 (**-3.5%**) | ### Varying Key Size **Write:** `db_bench --num=1000000 --separate_key_value_in_data_block=<bool> --min_key_size=10 --max_key_size=100 --benchmarks=fillrandom,compact` **Read:** `db_bench --db=$DB --use_existing_db --benchmarks=readrandom,readseq` | Key Size | FillRandom (ops/s) | | Compact (s) | | ReadRandom (ops/s) | | ReadSeq (ops/s) | | SST Size (MB) | | |---------:|-------------------:|-----:|----------:|-----:|-------------------:|-----:|----------------:|-----:|-------------:|-----:| | | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | baseline | sep_kv | | 10–100 | 243,740 | 255,183 | 0.618 | 0.622 (**+0.6%**) | 284,304 | 307,569 (**+8.2%**) | 3,738,921 | 3,686,676 (**-1.4%**) | 41.5 | 41.2 (**-0.8%**) | ### CPU Profile Notes - No compression: DataBlock::SeekForGet uses less cpu (13.2% vs 13.9%) - https://fburl.com/strobelight/6mwwebft with separated KV - https://fburl.com/strobelight/m9m798ka without - ZSTD compression: rocksdb::DecompressSerializedBlock uses more CPU (45.8% vs 44.9%), while DataBlock::SeekForGet uses less cpu (5.09% vs 6.52%) - https://fburl.com/strobelight/3x5nw1k4 with separated KV - https://fburl.com/strobelight/e7809046 without --- Reviewed By: xingbowang, pdillinger Differential Revision: D92103024 Pulled By: joshkang97 fbshipit-source-id: 47cfeb656ff3c20d34975f0b6c4c0462935a83dc |
||
 Hui Xiao
|
29819f37e1 |
Remove deprecated ReadOptions::managed, `ColumnFamilyOptions::snap_refresh_nanos (#14350)
Summary: **Context/Summary:** Remove deprecated, unused APIs and options: - ReadOptions::managed: This option was not used anymore. The functionality it controlled has been removed long ago. - ColumnFamilyOptions::snap_refresh_nanos: Deprecated and unused option. Corresponding C API (rocksdb_readoptions_set_managed) and Java API (ReadOptions.managed/setManaged) are also removed. All related checks an db_impl and db_impl_secondary iterators are cleaned up. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14350 Test Plan: make check Reviewed By: pdillinger Differential Revision: D93812438 Pulled By: hx235 fbshipit-source-id: e4a9d21c65f83294b6d0878286ba14024f049bac |
||
 xingbowang
|
3556c22059 |
Remove deprecated option skip_checking_sst_file_sizes_on_db_open (#14346)
Summary: Remove deprecated option skip_checking_sst_file_sizes_on_db_open Pull Request resolved: https://github.com/facebook/rocksdb/pull/14346 Test Plan: Unit test Reviewed By: hx235 Differential Revision: D93602683 Pulled By: xingbowang fbshipit-source-id: f576825cb107bb0aeb14f4ff29fef0df269b8728 |
||
|
Peter Dillinger
|
d3817f058d |
Remove deprecated DB::Open raw pointer variants (and more) (#14335)
Summary: and remove deprecated DB::MaxMemCompactionLevel(). In the process of pushing through a relatively clean refactoring of uses of the old functions, some other minor public APIs are also migrated from raw DB pointers to unique_ptr. Claude did pretty much all the work, but requiring dozens of prompts to actually push through relatively clean phase out of raw DB pointers from what needed to be touched, and leaving that code in better shape. (Hundreds of `DB*` still remain all over the place even outside C and Java bindings.) Pull Request resolved: https://github.com/facebook/rocksdb/pull/14335 Test Plan: existing tests; no functional changes intended Reviewed By: xingbowang, mszeszko-meta Differential Revision: D93523820 Pulled By: pdillinger fbshipit-source-id: e4ca22ad81cd2cfe91122d7507d7ca34fe03d043 |
||
|
Xingbo Wang
|
b040ab83e1 |
Add a new picking algorithm in fifo compaction (#14326)
Summary: Add a new kv ratio based compaction picking algorithm in fifo compaction Pull Request resolved: https://github.com/facebook/rocksdb/pull/14326 Test Plan: Unit test Reviewed By: pdillinger Differential Revision: D93257941 Pulled By: xingbowang fbshipit-source-id: fd2d0e1356c7b54682a1197475a1bd26cb45c9d4 |
||
|
Josh Kang
|
9f47518676 |
Add interpolation search as an alternative to binary search (#14247)
Summary: Interpolation search is an alternative algorithm to binary search, which performs better on uniformly distributed keys. Instead of binary search always computing the mid point of the left and right boundaries, interpolation search "interpolates" the mid point based on the distance to the target. Fortunately, we can re-use existing block format to support interpolation search. For a given block, we compute the shared_prefix length of the first and last key. Interpolation search is usually done with numerical target values, so for a variable binary length key, we calculate the "value" as the first 8 non-shared bytes. This also means interpolation search would only really be effective for bytewise comparator (guarded via options validations). #### Fallback to binary search - if the the val(left_key) == val(right_key) then we fallback to classic binary search (to avoid divide by 0) - interpolation search is significantly more computationally expensive than binary search, so when the search distance is small, we also fallback to binary search. - if interpolation search does not make significant progress (i.e. reduces search space by more than half each iteration), we can assume data is non-uniform and fallback. Interpolation search also performs best when there is minimal shortening, especially shortening of the last block, as it can heavily skew the distribution of the actual keys. Note that each search algorithm is guaranteed to make progress because at each iteration the search space is guaranteed to be reduce by at least 1. For now this change only applies to index block seeks, as data block seeks and other blocks do not have as many entries and would not require significant number of search rounds, but it could be easily extended to include that support. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14247 Test Plan: Updated unit tests and crash test with new search option ### Benchmark The default benchmark sets up keys in generally uniform distribution, so it was a good way to test performance improvements. Setup: `./db_bench -benchmarks=fillseq,compact -index_shortening_mode=1` #### Before this change ``` ./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 readrandom : 2.899 micros/op 344973 ops/sec 2.899 seconds 1000000 operations; 38.2 MB/s (1000000 of 1000000 found) ``` #### After this change Notice how key comparison counts are the same between the two. ``` ./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 -index_search_type=binary_search readrandom : 2.881 micros/op 347128 ops/sec 2.881 seconds 1000000 operations; 38.4 MB/s (1000000 of 1000000 found) ``` ``` ./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 -index_search_type=interpolation_search readrandom : 2.609 micros/op 383209 ops/sec 2.610 seconds 1000000 operations; 42.4 MB/s (1000000 of 1000000 found) ``` With a non-uniform distribution, `i.e. index_shortening_mode=2` ``` ./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 -index_search_type=binary_search readrandom : 2.958 micros/op 338075 ops/sec 2.958 seconds 1000000 operations; 37.4 MB/s (1000000 of 1000000 found) ``` ``` ./db_bench -use_existing_db=true -benchmarks=readrandom -seed=1 -index_search_type=interpolation_search readrandom : 5.502 micros/op 181750 ops/sec 5.502 seconds 1000000 operations; 20.1 MB/s (1000000 of 1000000 found) ``` Reviewed By: pdillinger Differential Revision: D91063163 Pulled By: joshkang97 fbshipit-source-id: 151d6aa76f8713740b714de6e406aff40d28ccbc |
||
|
Peter Dillinger
|
871f79d6ef |
Reformat source files (#14331)
Summary: probably something changed, maybe https://github.com/facebook/rocksdb/issues/14311 Full command: ``` git ls-files '*.cc' '*.h' | grep -v '^third-party/' | grep -v 'range_tree' | xargs clang-format -i ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/14331 Test Plan: CI Reviewed By: mszeszko-meta Differential Revision: D93246992 Pulled By: pdillinger fbshipit-source-id: 6bc5b97978fef8aee52823dadb6daa4bea57343d |
||
|
Xingbo Wang
|
656b734a5f |
Support abort background compaction jobs. (#14227)
Summary: This adds a new public API to allow applications to abort all running compactions and prevent new ones from starting. Unlike DisableManualCompaction() which only pauses manual compactions and waits for them to finish naturally, AbortAllCompactions() actively signals running compactions (both automatic and manual) to terminate early and waits for them to complete before returning. The abort signal is checked periodically during compaction (every 100 keys), so ongoing compactions abort quickly. Any output files from aborted compactions are automatically cleaned up to prevent partial results from being installed. This is useful for scenarios where applications need to quickly stop all compaction activity, such as during graceful shutdown or when performing maintenance operations. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14227 Test Plan: - Unit tests in db_compaction_abort_test.cc cover various abort scenarios including: abort before/during compaction, abort with multiple subcompactions, nested abort/resume calls, abort with CompactFiles API, abort across multiple column families, and timing guarantees - Updated compaction_job_test.cc to include the new parameter Reviewed By: anand1976 Differential Revision: D91480994 Pulled By: xingbowang fbshipit-source-id: 36837971d8a540cd34d3ec28a78bc94b582625b0 |
||
 Maciej Szeszko
|
6b5ccbbec6 |
Remove inline values support (#14270)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14270 Legacy BlobDB's inline values feature (storing small values directly in the LSM tree via `min_blob_size` threshold) is unused in production - all deployments use `min_blob_size = 0`. This removes the functionality entirely. Changes: - Remove `min_blob_size` from `BlobDBOptions` - Remove `IsInlined()` check from compaction filter (dead code path) - Remove inline-related statistics (`BLOB_DB_WRITE_INLINED*`) - Remove `InlineSmallValues` test - Update stale comments referencing inlined data Reviewed By: xingbowang Differential Revision: D91088985 fbshipit-source-id: ec67848ece1a7dc071ca8e8a17faebb435394733 |
||
 Anand Ananthabhotla
|
b89d290c20 |
Add MultiScan statistics (#14248)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/14248 ### Overview This diff introduces the addition of multi-scan statistics to RocksDB, enhancing the database's ability to monitor and analyze performance during multi-scan operations. ### Key Changes #### Implemented Multi-Scan Statistics The following statistics were implemented to provide deeper insights into multi-scan operations: - **MULTISCAN_PREPARE_MICROS**: Measures the time (in microseconds) spent preparing for multi-scan operations. - **MULTISCAN_BLOCKS_PER_PREPARE**: Tracks the number of blocks processed per multi-scan prepare operation. - **Wasted Prefetch Blocks Count**: Counts the number of prefetched blocks that were not used (i.e., wasted) if the iterator is abandoned before accessing them. - **MULTISCAN_TOTAL_BLOCKS_SCANNED**: Tracks the total number of blocks scanned during all multi-scan operations. - **MULTISCAN_TOTAL_KEYS_SCANNED**: Measures the total number of keys scanned across all multi-scan operations. - **MULTISCAN_TOTAL_MICROS**: Captures the total time (in microseconds) spent in multi-scan operations. - **MULTISCAN_PREFETCHED_BLOCKS**: Counts the number of blocks that were prefetched during multi-scan operations. - **MULTISCAN_USED_PREFETCH_BLOCKS**: Tracks the number of prefetched blocks that were actually used during multi-scan operations. ### Impact This diff provides more fine-grained statistics for multi-scan operations, allowing developers and users to better understand and optimize the performance of their RocksDB instances. Reviewed By: krhancoc Differential Revision: D91053297 fbshipit-source-id: 7158741b9f026c0b5ce8ba1264dbd137e7fe985d |
||
|
Peter Dillinger
|
a6af317476 |
Use format_version=7 by default, fix perf bug (#14239)
Summary: Since it's been > 6 months and we have production uses, migrate to fv=7 by default. One unit test needed an update for the change to table properties with fv=7. On making this change, PresetCompressionDictTest tests detected extra memory usage by decompressing LZ4 with dictionary compression. This turned out to be a bug in `std::find` usage that led to using the ZSTD-optimized decompressor (with digested dictionary usage) in cases where it is not needed. I've fixed the bug and improved the unit tests that found the bug. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14239 Test Plan: existing tests, including format compatible CI job (updated, and run locally with SHORT_TEST=1) Reviewed By: hx235 Differential Revision: D90728697 Pulled By: pdillinger fbshipit-source-id: 8f1a0e9ca59a88c18eaa4cdfdea00309175ce30a |
||
 Siying Dong
|
c757f5b4e3 |
Java's Get() to directly return for NotFound (#14095)
Summary: Right now, in Java's Get() calls, the way Get() is treated is inefficient. Status.NotFound is turned into an exception in the JNI layer, and is caught in the same function to turn into not found return. This causes significant overhead in the scenario where most of the queries ending up with not found. For example, in Spark's deduplication query, this exception creation overhead is higher than Get() itself. With the proposed change, if return status is NotFound, we directly return, rather than going through the exception path Pull Request resolved: https://github.com/facebook/rocksdb/pull/14095 Test Plan: Existing tests should cover all Get() cases, and they are passing. Reviewed By: jaykorean Differential Revision: D86797594 Pulled By: cbi42 fbshipit-source-id: 1202d24e46a2358976bb7c8ff38a2fd4783d0f99 |
||
|
Hui Xiao
|
8edb99f904 |
Statistics for successfully resumed compaction output bytes (#14054)
Summary: Context/Summary: as titled Pull Request resolved: https://github.com/facebook/rocksdb/pull/14054 Test Plan: new UT, manually checking Reviewed By: jaykorean Differential Revision: D84828431 Pulled By: hx235 fbshipit-source-id: 56e1a9159f7597a10d6c549657d8b22788aa0599 |
||
|
Xingbo Wang
|
742741b175 |
Support Super Block Alignment (#13909)
Summary: Pad block based table based on super block alignment Pull Request resolved: https://github.com/facebook/rocksdb/pull/13909 Test Plan: Unit Test No impact on perf observed due to change in the inner loop of flush. upstream/main branch 202.15 MB/s ``` for i in `seq 1 10`; do ./db_bench --benchmarks=fillseq -num=10000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=1000 -fifo_compaction_allow_compaction=0 -disable_wal -write_buffer_size=12000000 -format_version=7 >> /tmp/x1 2>&1; grep fillseq /tmp/x1 | grep -Po "\d+\.\d+ MB/s" | grep -Po "\d+\.\d+" | awk '{sum+=$1} END {print sum/NR}' ``` After the change without super block alignment 203.44 MB/s ``` for i in `seq 1 10`; do ./db_bench --benchmarks=fillseq -num=10000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=1000 -fifo_compaction_allow_compaction=0 -disable_wal -write_buffer_size=12000000 -format_version=7 >> /tmp/x1 2>&1 ``` After the change with super block alignment 204.47 MB/s ``` for i in `seq 1 10`; do ./db_bench --benchmarks=fillseq -num=10000000 -compaction_style=2 -fifo_compaction_max_table_files_size_mb=1000 -fifo_compaction_allow_compaction=0 -disable_wal -write_buffer_size=12000000 -format_version=7 --super_block_alignment_size=131072 --super_block_alignment_max_padding_size=4096 >> /tmp/x1 2>&1; ``` Reviewed By: pdillinger Differential Revision: D83068913 Pulled By: xingbowang fbshipit-source-id: eecd65088ab3e9dbc7902aab8c2580f1bc8575df |
||
|
Peter Dillinger
|
f5fb597bac |
Resolve missing/inconsistent tickers in Java (#14012)
Summary: Pretty self-explanatory from the changes, including re-arranging the "COOL" entries for easier tracking of which values are used. I'm not touching the TICKER_ENUM_MAX issue because IIRC we've gotten in trouble in the past for changing any Java ticker values. Pull Request resolved: https://github.com/facebook/rocksdb/pull/14012 Test Plan: CI, sufficient prompts to get AI to discover the known issues relayed by hx235, to help ensure we found any other outstanding issues. Reviewed By: hx235 Differential Revision: D83497503 Pulled By: pdillinger fbshipit-source-id: ec0bd7e28188e0430fb03fc5bd79c2ed7b28f3ad |