Compare commits

...
7 Commits
Author SHA1 Message Date
anand1976 26432d79f7 Merge pull request #15360 from evolvedbinary/hotfix/11.7.fb/windows-compilation
Fix for Windows build issue
2026-10-15 10:46:45 -07:00
anand76 557960bd9f Revert "Fix Windows build issue in trie_index_factory.h"
This reverts commit 96012d9954.
2026-10-15 10:38:31 -07:00
Anand Ananthabhotla 6720218931 Make InjectedErrorLog::Entry 256 bytes on 32-bit targets (#15388)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/15388

`InjectedErrorLog::Entry` in `utilities/fault_injection_fs.h` is written to the injected error log in its native layout, and a `static_assert` requires `sizeof(Entry) == 256` so the record size stays stable. That assert fails on i386 builds.

Root cause: the fields add up to only 252 bytes. On targets where `alignof(uint64_t)` is 8 (x86_64, aarch64, armv7), the compiler adds 4 bytes of implicit tail padding, so the size is 256. On i386, `uint64_t` members are 4-byte aligned inside a struct, so there is no tail padding and the size is 252.

Fix: add an explicit trailing `uint8_t padding[4]` field so the size no longer depends on `uint64_t` alignment.

Compatibility: on 64-bit targets the new field takes the place of the old implicit tail padding, so every field offset and the 256-byte record layout are unchanged. `Record()` value-initializes the entry (`Entry entry{}`), so the padding bytes are always written as zero. No decoder in the repo depends on the padding bytes.

___

landed-with-radar-review

Differential Revision: D124873369

fbshipit-source-id: f5c6f43ca85013b0e769853ad13b8d4f3f89b958
2026-10-13 10:54:47 -07:00
xingbowang 7ced0aaac9 Fix big-endian builds for the fault-injection filesystem (#15297)
Summary:
The persistent injected-error log writes `InjectedErrorLog::Entry` objects using their native in-memory layout, while the log format's consumer expects little-endian multi-byte fields. The existing `static_assert(port::kLittleEndian)` protected that format by rejecting every big-endian build that included the fault-injection filesystem, even though the log is an optional diagnostic.

This change keeps RocksDB and the fault-injection filesystem available on supported big-endian platforms while disabling only the incompatible persistent log. Log-file creation now returns an inactive descriptor when `port::kLittleEndian` is false, which makes `Record()` and `Flush()` remain safe no-ops. The fixed 256-byte entry-size assertion remains in place, and format-dependent tests are bypassed on big-endian targets.

The change also adds a release note and documents big-endian support plus the preferred capability-check pattern for optional architecture-specific features.

## Compatibility and limitations

- Little-endian behavior and the on-disk log layout are unchanged.
- On big-endian platforms, configuring an injected-error log path does not create or populate a log file; the rest of fault injection remains available.
- Windows already leaves this POSIX-backed diagnostic disabled.
- This does not introduce a portable log encoding. Supporting persistent injected-error logs on big-endian platforms would require a format change.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/15297

Test Plan:
- `build_tools/rockstest.sh fault_injection_fs_test '--gtest_filter=InjectedErrorLogTest.*'` (little-endian macOS): all 5 tests passed.
- `make check-sources`
- `git diff --check upstream/main...2026_10_01_endian_fix`

Native big-endian build and test execution were not available locally; that path relies on the compile-time `port::kLittleEndian` capability and should be covered by big-endian CI.

Reviewed By: jaykorean

Differential Revision: D122880760

Pulled By: xingbowang

fbshipit-source-id: 59dae9661a47608aa12f41078ad1b3ef0fa7231e
2026-10-13 10:54:38 -07:00
anand76 96012d9954 Fix Windows build issue in trie_index_factory.h 2026-10-13 10:54:31 -07:00
Adam Retter 626d471544 Fix for Windows build issue (see: https://github.com/facebook/rocksdb/issues/14977) 2026-10-08 11:10:05 +02:00
anand1976 1129d3510f Update HISTORY.md for 11.7.0 release 2026-07-16 13:27:49 -07:00
11 changed files with 49 additions and 9 deletions

No files matched your search

+7
View File
@@ -298,6 +298,7 @@ build (and where noted, run tests) across:
| Axis | Must support |
|------|--------------|
| OS | Linux (x86_64 + ARM), macOS, Windows |
| Architecture | Little-endian and big-endian targets, including Linux on s390x |
| Compiler | GCC, Clang (libstdc++ **and** libc++), AppleClang, **MSVC (VS2022)**, MinGW (Linux cross-compile, build-only, no gflags) |
| Build system | Make, CMake, and BUCK (internal) -- keep all in sync (see "Build system" above) |
| Config | release (`-fno-rtti`), `ASSERT_STATUS_CHECKED`, ASAN/UBSAN/TSAN, folly, unity build, JNI/Java |
@@ -312,6 +313,12 @@ abstractions; otherwise guard with `#ifdef OS_WIN` (POSIX `<unistd.h>` ->
Windows `<process.h>`). Because libc++ is also tested, include what you use
rather than relying on libstdc++ transitive includes.
Do not use `static_assert` to reject a supported architecture for an optional
feature. Check an existing `port::` capability such as `port::kLittleEndian`
and disable only the unsupported feature so the rest of RocksDB still builds.
Use a portable representation when the feature itself must work across
architectures.
### Unit Test
* After all of the unit tests are added, review them and try to extract common
reusable utility functions to reduce code duplication due to copy past between
+11
View File
@@ -1,6 +1,17 @@
# Rocksdb Change Log
> NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
## 11.7.0 (07/16/2026)
### Public API Changes
* Added `RangeLockManagerHandle::SetIsKilledCallback()`, a `std::function<bool()>` predicate polled during range-lock waits. When it returns true, a blocked `GetRangeLock()` wait returns promptly with `Status::Aborted()` instead of waiting for the lock timeout. Range-lock users that install no callback keep the original wait-until-grant-or-timeout behavior.
* Added `MultiScanArgs::reverse` to support reverse MultiScan reads over bounded scan ranges.
### Behavior Changes
* When using `LockWAL()`, live-file capture APIs such as checkpoint and backup now flush WAL-disabled unpersisted data; they can block until `UnlockWAL()` or return `Aborted` when called from the same thread that holds `LockWAL()` to avoid deadlock. `UnlockWAL()` must now also be called from the same thread that called `LockWAL()`.
### Performance Improvements
* Fixed slow DB open with `best_efforts_recovery`, and slow catch-up of secondary and follower (read-only) instances, in which time and memory grew quadratically with the number of files in the DB (so the slowdown was worst for large DBs with long history).
## 11.6.0 (07/02/2026)
### New Features
* Added EXPERIMENTAL embedded blob SST support through `SstFileWriter::OpenWithEmbeddedBlobs()`, storing eligible large values as same-file blob records in block-based SST files and resolving them transparently for reads. This niche feature currently supports uncompressed embedded blobs only; compression options are placeholders and compression support is deferred to follow-up work.
@@ -1 +0,0 @@
When using `LockWAL()`, live-file capture APIs such as checkpoint and backup now flush WAL-disabled unpersisted data; they can block until `UnlockWAL()` or return `Aborted` when called from the same thread that holds `LockWAL()` to avoid deadlock. `UnlockWAL()` must now also be called from the same thread that called `LockWAL()`.
@@ -0,0 +1 @@
Fixed RocksDB builds on big-endian architectures such as s390x by disabling the optional persistent fault-injection log, whose binary format is little-endian-only.
@@ -1 +0,0 @@
Fixed slow DB open with `best_efforts_recovery`, and slow catch-up of secondary and follower (read-only) instances, in which time and memory grew quadratically with the number of files in the DB (so the slowdown was worst for large DBs with long history).
@@ -1 +0,0 @@
Added `RangeLockManagerHandle::SetIsKilledCallback()`, a `std::function<bool()>` predicate polled during range-lock waits. When it returns true, a blocked `GetRangeLock()` wait returns promptly with `Status::Aborted()` instead of waiting for the lock timeout. Range-lock users that install no callback keep the original wait-until-grant-or-timeout behavior.
@@ -1 +0,0 @@
Added `MultiScanArgs::reverse` to support reverse MultiScan reads over bounded scan ranges.
+4 -1
View File
@@ -26,6 +26,7 @@
#include "env/io_posix.h"
#include "monitoring/thread_status_util.h"
#include "port/lang.h"
#include "port/port.h"
#include "port/stack_trace.h"
#include "rocksdb/env.h"
#include "rocksdb/io_status.h"
@@ -79,7 +80,9 @@ uint8_t DetailPayloadSize(const InjectedErrorLog::TaggedEntryDetail& detail) {
int OpenInjectedErrorLogFile(const std::string& path) {
#ifndef OS_WIN
if (path.empty()) {
// Entry is written in native layout, while the decoder expects little-endian
// fields. Keep the optional log disabled on unsupported architectures.
if (!port::kLittleEndian || path.empty()) {
return -1;
}
int flags = O_WRONLY | O_CREAT | O_TRUNC;
+5 -2
View File
@@ -44,6 +44,8 @@ namespace ROCKSDB_NAMESPACE {
// A binary log that records injected errors directly to a file.
// Thread-safe for concurrent writes. The log file can be flushed on exit or
// from a signal handler so records survive clean exits and crash paths.
// Disabled on big-endian platforms because the decoder expects little-endian
// fields in the native Entry layout.
class InjectedErrorLog {
public:
static constexpr size_t kMaxOpNameLen = 32;
@@ -86,12 +88,13 @@ class InjectedErrorLog {
char op_name[kMaxOpNameLen];
char file_name[kMaxFileNameLen];
char status_message[kMaxStatusMessageLen];
// Explicit tail padding so the size does not depend on the alignment of
// uint64_t (4 bytes on i386, 8 bytes on most other targets).
uint8_t padding[4];
};
static_assert(sizeof(Entry) == 256,
"Injected error log entry size must stay stable");
static_assert(port::kLittleEndian,
"Injected error log requires little-endian platforms");
explicit InjectedErrorLog(const std::string& path);
~InjectedErrorLog();
+21
View File
@@ -10,6 +10,7 @@
#include <thread>
#include <vector>
#include "port/port.h"
#include "test_util/testharness.h"
namespace ROCKSDB_NAMESPACE {
@@ -79,6 +80,11 @@ class CloseCountingWritableFile : public FSWritableFileOwnerWrapper {
// the expected fixed-width fields. The test writes one entry, flushes the
// file, and decodes the raw bytes back into an entry struct.
TEST_F(InjectedErrorLogTest, BasicRecordAndFlush) {
if (!port::kLittleEndian) {
ROCKSDB_GTEST_BYPASS(
"Persistent injected error log requires little-endian");
return;
}
std::string path = test::PerThreadDBPath("injected_error_log_basic.bin");
{
InjectedErrorLog log(path);
@@ -114,6 +120,11 @@ TEST_F(InjectedErrorLogTest, BasicRecordAndFlush) {
// to the old in-memory ring size. The test writes more than 1,000 entries and
// checks that both the first and last ones are still present in the file.
TEST_F(InjectedErrorLogTest, DirectLogKeepsAllEntries) {
if (!port::kLittleEndian) {
ROCKSDB_GTEST_BYPASS(
"Persistent injected error log requires little-endian");
return;
}
std::string path =
test::PerThreadDBPath("injected_error_log_all_entries.bin");
{
@@ -144,6 +155,11 @@ TEST_F(InjectedErrorLogTest, DirectLogKeepsAllEntries) {
// shared file. The test has several threads emit records concurrently and then
// checks the raw file size matches the total number of writes.
TEST_F(InjectedErrorLogTest, ConcurrentRecord) {
if (!port::kLittleEndian) {
ROCKSDB_GTEST_BYPASS(
"Persistent injected error log requires little-endian");
return;
}
std::string path = test::PerThreadDBPath("injected_error_log_concurrent.bin");
constexpr int kNumThreads = 4;
constexpr int kRecordsPerThread = 200;
@@ -180,6 +196,11 @@ TEST_F(InjectedErrorLogTest, ConcurrentRecord) {
// in the fixed-width record. The test logs a path longer than the file-name
// field and checks the stored sample matches the expected tail bytes.
TEST_F(InjectedErrorLogTest, LongFileNameKeepsSuffix) {
if (!port::kLittleEndian) {
ROCKSDB_GTEST_BYPASS(
"Persistent injected error log requires little-endian");
return;
}
std::string path =
test::PerThreadDBPath("injected_error_log_suffix_truncation.bin");
const std::string long_file_name =
@@ -298,12 +298,10 @@ class TrieIndexFactory : public UserDefinedIndexFactory {
// builds) to surface programming errors immediately.
UserDefinedIndexBuilder* NewBuilder() const override {
abort();
return nullptr;
}
std::unique_ptr<UserDefinedIndexReader> NewReader(
Slice& /*index_block*/) const override {
abort();
return nullptr;
}
// New API with comparator.