mirror of
https://github.com/facebook/rocksdb.git
synced 2026-11-07 03:22:34 -05:00
Jay Huh
15951afbb5
Summary: Pull Request resolved: https://github.com/meta/rocksdb/pull/15563 Why: Tuning RocksDB options for a workload is manual. LLMs can now reason about RocksDB's stats and workload well enough to propose option changes, and RocksDB already has the signals (DB properties, statistics, the query tracer) and can change mutable options online. This adds the loop that connects them: an on-demand tuning cycle inside RocksDB that measures the DB, asks an advisor for changes, applies them, watches the result and keeps or reverts them, with every step pluggable and logged. What (EXPERIMENTAL; nothing runs unless the application calls `StartLiveTuneCycle()`): - One cycle: baseline metrics from the DB's own properties -> a short query trace (`DB::StartTrace()`) and its analysis -> a recommendation -> checks (allowed options, bounds, `max_changes`, `max_step_ratio`) -> `SetOptions()` / `SetDBOptions()` (`skip_apply` stops before this) -> each following window compared with the baseline and decided on -> kept or reverted. - API (`include/rocksdb/live_tune.h`): `StartLiveTuneCycle()`, `GetLiveTuneStatus()`, `StopLiveTuneCycle()`, `GetLiveTuneAppliedOptions()`, and `LiveTuneOptions`: the intent (a goal in plain words, allowed options with bounds, optional constraints), window and limit settings, `skip_apply`, `allow_compaction`. - `GetLiveTuneAppliedOptions()` returns the options live tuning has changed and not reverted, with their values: the running cycle's changes (from just before they are applied until they are reverted) and the changes kept by cycles since the DB was opened. An application that also sets options, e.g. by re-applying its own configuration periodically, uses it to leave those options alone instead of undoing the cycle's change. - Changes that take effect only on SST files written after them (e.g. `block_size`): the advisor says so with `LiveTuneRecommendation::requires_compaction`. With `allow_compaction` the cycle compacts the column family (`CompactRange()`) after applying them and before watching; without it, such a recommendation is rejected. Table-property metrics (`avg_data_block_bytes`, `index_bytes`, `filter_bytes`) show their effect. - With a `Statistics` object, the metrics also come from the DB's tickers: `db_keys_read` (Get and MultiGet keys), `db_iterator_seeks`, `db_iterator_steps`, `db_block_cache_data_hits` and `db_block_cache_data_misses`, with derived `sst_file_reads_per_key` (the column family's SST file reads per key read, sought or stepped) and `db_block_cache_data_hit_ratio`. Read efficiency can then be judged per key rather than on raw counts that follow traffic, with no help from the application. Tickers cover every column family, and every DB sharing the `Statistics` object, so their metrics carry a `db_` prefix; the public header lists which metrics are per column family and which are for the whole DB. - Pluggable: - `LiveTuneAdvisor` (required): recommends changes, e.g. by asking an LLM, with the objectives and constraints to judge them by. - `LiveTuneTraceAnalyzer`: summarizes the trace; the built-in one uses RocksDB's `TraceAnalyzer` (parent change). - `LiveTuneVerdict`: `GetWindowDecision()` after every watch window: keep watching, keep the changes or revert them. By default a broken constraint reverts, and a change is kept once a one-sided 95% Wilson lower bound of winning windows reaches `min_confidence`. An override can call the default and add its own checks, e.g. revert a change the default would keep. - `LiveTuneMetricsProvider`: the application's own metrics. - `LiveTuneEventListener`: every step as a JSON event, and `OnCycleDone()` with the final result, e.g. to roll a kept change out to other DBs. - Every event also goes to the info LOG as `LIVE_TUNE {json}`. Safety and limits: - Only allowed options within bounds change, through `SetOptions()` / `SetDBOptions()`; they are reverted on a broken constraint, a revert decision, `StopLiveTuneCycle()` or DB close. A change that mixes column family and DB options is checked before anything is set, and if `SetDBOptions()` still fails after `SetOptions()` succeeded, the column family options are restored, so nothing stays half applied. Because `SetOptions()` and `SetDBOptions()` can fail with the new values already in effect (for example when the OPTIONS file write fails), the values are read back after an error to decide what is in effect. A revert restores the values read right before apply, after the advisor call, so it doesn't undo an option the application changed during the trace or the advisor call; if a change is still in effect after a revert, the outcome ends with "; revert failed: <status>" and `GetLiveTuneAppliedOptions()` still lists it. One cycle per DB at a time. Any DB opened read-only (read-only, compacted, secondary or follower) returns NotSupported. - Lifecycle: an event listener may stop the cycle from its callbacks (the stop is only requested), and starting the next cycle from a callback returns Busy; start it from another thread, e.g. once the "done" event is seen. Starting a new cycle joins the previous cycle's thread without holding the live tune mutex. The advisor's `CancelRecommendation()` is called only while its `GetRecommendation()` is in progress. - Option values are read and written with RocksDB's own options serde: single options by name through `CFOptionsAsConfigurable()` / `DBOptionsAsConfigurable()` and `Configurable::GetOption()`, including dotted fields of a struct or of the table factory (`compaction_options_universal.size_ratio`, `block_based_table_factory.block_size`), which `SetOptions()` also takes directly, keeping the block cache and filter. Recommended values are rewritten the way RocksDB writes them (e.g. `32K` as `32768`) by parsing them as their option, so checks and read-back comparisons are type-aware. They are parsed into a copy of the table factory and its filter policy, because the table factory parser sets a mutable field under it (a ribbon filter's `filter_policy.bloom_before_level`) on the existing object, so checking a recommendation never changes the DB; a value the option doesn't accept is rejected ("rejected: invalid value: ...") before anything changes, and like any rejected recommendation it is logged with its changes and recorded in the history the next cycle's advisor gets. Numbers are parsed with `strtod`, like RocksDB's own parsing of double options. - `max_step_ratio` bounds every numeric change; a move to or from 0 or a negative value (often "disabled" or "no limit" in RocksDB) is accepted only within explicit intent bounds. - If the current values of the allowed options cannot be read (an option fails to serialize), the cycle ends as "rejected: cannot read the current option values: <status>" before anything changes, rather than recommending from incomplete values. If applying fails, the previous values are restored the same way as a revert, read back, and the outcome adds "; revert failed: <status>" only for a change still in effect. An allowed option that neither the column family nor the DB options have makes `StartLiveTuneCycle()` return InvalidArgument, before any baseline. - Objectives and constraints must be on measured metrics: a recommendation with changes that names an unmeasured one is rejected (an answer with no changes is "no change" whatever objectives it lists), and an enforced constraint (the intent's, or `default_constraints` when the intent has none) on an unmeasured metric ends the cycle right after the baseline ("rejected: intent constraint metric <metric> is not measured", or "default constraint metric"), before any trace or advisor call, instead of never being checked. - Each window is compared with a baseline taken before the change (no control group). If the application already has a trace running (`DB::StartTrace()`), the cycle leaves it alone and goes on without a trace (the "traced" event has status Busy); if the application starts one while the cycle traces, the cycle's trace ends and the application's keeps running. This uses two internal `DBImpl` methods, `StartTraceIfNone()` and `EndTraceIfCurrent()`; `DB::StartTrace()` itself still replaces a running trace. A trace file left by a cycle that ended with the process (e.g. a crash) is deleted when the next cycle on the DB starts. SST read metrics come from the file read histograms read without a lock, so they are approximate. Table-property metrics come from `GetPropertiesOfAllTables()`, which reads the files' properties without holding the DB mutex. Not yet supported in db_bench, db_stress, Java or the C API. Reviewed By: xingbowang Differential Revision: D128710591 fbshipit-source-id: 06b7af65841593870e9c2325c0f305bca94c7787
806 lines
52 KiB
Makefile
806 lines
52 KiB
Makefile
# These are the sources from which librocksdb.a is built:
|
|
LIB_SOURCES = \
|
|
cache/cache.cc \
|
|
cache/cache_entry_roles.cc \
|
|
cache/cache_key.cc \
|
|
cache/cache_helpers.cc \
|
|
cache/cache_reservation_manager.cc \
|
|
cache/charged_cache.cc \
|
|
cache/clock_cache.cc \
|
|
cache/lru_cache.cc \
|
|
cache/compressed_secondary_cache.cc \
|
|
cache/secondary_cache.cc \
|
|
cache/secondary_cache_adapter.cc \
|
|
cache/sharded_cache.cc \
|
|
cache/tiered_secondary_cache.cc \
|
|
db/arena_wrapped_db_iter.cc \
|
|
db/attribute_group_iterator_impl.cc \
|
|
db/blob/blob_contents.cc \
|
|
db/blob/blob_fetcher.cc \
|
|
db/blob/blob_file_addition.cc \
|
|
db/blob/blob_file_builder.cc \
|
|
db/blob/blob_file_cache.cc \
|
|
db/blob/blob_file_partition_manager.cc \
|
|
db/blob/blob_file_garbage.cc \
|
|
db/blob/blob_file_meta.cc \
|
|
db/blob/blob_file_reader.cc \
|
|
db/blob/blob_garbage_meter.cc \
|
|
db/blob/blob_gen2_format.cc \
|
|
db/blob/blob_log_format.cc \
|
|
db/blob/blob_log_sequential_reader.cc \
|
|
db/blob/blob_log_writer.cc \
|
|
db/blob/blob_source.cc \
|
|
db/blob/blob_write_batch_transformer.cc \
|
|
db/blob/prefetch_buffer_collection.cc \
|
|
db/builder.cc \
|
|
db/c.cc \
|
|
db/coalescing_iterator.cc \
|
|
db/column_family.cc \
|
|
db/compaction/compaction.cc \
|
|
db/compaction/compaction_iterator.cc \
|
|
db/compaction/compaction_job.cc \
|
|
db/compaction/compaction_picker.cc \
|
|
db/compaction/compaction_picker_fifo.cc \
|
|
db/compaction/compaction_picker_level.cc \
|
|
db/compaction/compaction_picker_universal.cc \
|
|
db/compaction/compaction_service_job.cc \
|
|
db/compaction/compaction_state.cc \
|
|
db/compaction/compaction_outputs.cc \
|
|
db/compaction/sst_partitioner.cc \
|
|
db/compaction/subcompaction_state.cc \
|
|
db/convenience.cc \
|
|
db/coro_db.cc \
|
|
db/db_filesnapshot.cc \
|
|
db/db_impl/compacted_db_impl.cc \
|
|
db/db_impl/db_impl.cc \
|
|
db/db_impl/db_impl_compaction_flush.cc \
|
|
db/db_impl/db_impl_debug.cc \
|
|
db/db_impl/db_impl_experimental.cc \
|
|
db/db_impl/db_impl_files.cc \
|
|
db/db_impl/db_impl_follower.cc \
|
|
db/db_impl/db_impl_open.cc \
|
|
db/db_impl/db_impl_readonly.cc \
|
|
db/db_impl/db_impl_secondary.cc \
|
|
db/db_impl/db_impl_write.cc \
|
|
db/db_info_dumper.cc \
|
|
db/db_iter.cc \
|
|
db/dbformat.cc \
|
|
db/error_handler.cc \
|
|
db/event_helpers.cc \
|
|
db/experimental.cc \
|
|
db/external_sst_file_ingestion_job.cc \
|
|
db/file_indexer.cc \
|
|
db/flush_job.cc \
|
|
db/flush_scheduler.cc \
|
|
db/forward_iterator.cc \
|
|
db/import_column_family_job.cc \
|
|
db/internal_stats.cc \
|
|
db/logs_with_prep_tracker.cc \
|
|
db/log_reader.cc \
|
|
db/log_writer.cc \
|
|
db/malloc_stats.cc \
|
|
db/manifest_ops.cc \
|
|
db/memtable.cc \
|
|
db/memtable_list.cc \
|
|
db/merge_helper.cc \
|
|
db/merge_operator.cc \
|
|
db/multi_scan.cc \
|
|
db/output_validator.cc \
|
|
db/periodic_compaction_phaser.cc \
|
|
db/periodic_task_scheduler.cc \
|
|
db/range_del_aggregator.cc \
|
|
db/range_tombstone_fragmenter.cc \
|
|
db/repair.cc \
|
|
db/seqno_to_time_mapping.cc \
|
|
db/snapshot_impl.cc \
|
|
db/table_cache.cc \
|
|
db/table_properties_collector.cc \
|
|
db/trim_history_scheduler.cc \
|
|
db/version_builder.cc \
|
|
db/version_edit.cc \
|
|
db/version_edit_handler.cc \
|
|
db/version_set.cc \
|
|
db/version_util.cc \
|
|
db/wal_edit.cc \
|
|
db/wal_iterator_impl.cc \
|
|
db/wal_manager.cc \
|
|
db/wide/lazy_wide_columns.cc \
|
|
db/wide/read_path_blob_resolver.cc \
|
|
db/wide/wide_column_serialization.cc \
|
|
db/wide/wide_columns.cc \
|
|
db/wide/wide_columns_helper.cc \
|
|
db/write_batch.cc \
|
|
db/write_batch_base.cc \
|
|
db/write_controller.cc \
|
|
db/write_stall_stats.cc \
|
|
db/write_thread.cc \
|
|
env/composite_env.cc \
|
|
env/env.cc \
|
|
env/env_chroot.cc \
|
|
env/env_encryption.cc \
|
|
env/env_posix.cc \
|
|
env/file_system.cc \
|
|
env/fs_on_demand.cc \
|
|
env/fs_posix.cc \
|
|
env/fs_remap.cc \
|
|
env/file_system_tracer.cc \
|
|
env/io_posix.cc \
|
|
env/mock_env.cc \
|
|
env/unique_id_gen.cc \
|
|
file/delete_scheduler.cc \
|
|
file/file_prefetch_buffer.cc \
|
|
file/file_util.cc \
|
|
file/filename.cc \
|
|
file/line_file_reader.cc \
|
|
file/random_access_file_reader.cc \
|
|
file/read_write_util.cc \
|
|
file/readahead_raf.cc \
|
|
file/sequence_file_reader.cc \
|
|
file/sst_file_manager_impl.cc \
|
|
file/writable_file_writer.cc \
|
|
logging/auto_roll_logger.cc \
|
|
logging/event_logger.cc \
|
|
logging/log_buffer.cc \
|
|
memory/arena.cc \
|
|
memory/concurrent_arena.cc \
|
|
memory/jemalloc_nodump_allocator.cc \
|
|
memory/memkind_kmem_allocator.cc \
|
|
memory/memory_allocator.cc \
|
|
memtable/alloc_tracker.cc \
|
|
memtable/hash_linklist_rep.cc \
|
|
memtable/hash_skiplist_rep.cc \
|
|
memtable/skiplistrep.cc \
|
|
memtable/vectorrep.cc \
|
|
memtable/wbwi_memtable.cc \
|
|
memtable/write_buffer_manager.cc \
|
|
monitoring/histogram.cc \
|
|
monitoring/histogram_windowing.cc \
|
|
monitoring/in_memory_stats_history.cc \
|
|
monitoring/instrumented_mutex.cc \
|
|
monitoring/iostats_context.cc \
|
|
monitoring/perf_context.cc \
|
|
monitoring/perf_level.cc \
|
|
monitoring/persistent_stats_history.cc \
|
|
monitoring/statistics.cc \
|
|
monitoring/thread_status_impl.cc \
|
|
monitoring/thread_status_updater.cc \
|
|
monitoring/thread_status_updater_debug.cc \
|
|
monitoring/thread_status_util.cc \
|
|
monitoring/thread_status_util_debug.cc \
|
|
options/cf_options.cc \
|
|
options/configurable.cc \
|
|
options/customizable.cc \
|
|
options/db_options.cc \
|
|
options/offpeak_time_info.cc \
|
|
options/options.cc \
|
|
options/options_helper.cc \
|
|
options/options_parser.cc \
|
|
port/mmap.cc \
|
|
port/port_posix.cc \
|
|
port/win/env_default.cc \
|
|
port/win/env_win.cc \
|
|
port/win/io_win.cc \
|
|
port/win/port_win.cc \
|
|
port/win/win_logger.cc \
|
|
port/win/win_thread.cc \
|
|
port/stack_trace.cc \
|
|
table/adaptive/adaptive_table_factory.cc \
|
|
table/block_based/binary_search_index_reader.cc \
|
|
table/block_based/block.cc \
|
|
table/block_based/block_based_table_builder.cc \
|
|
table/block_based/block_based_table_factory.cc \
|
|
table/block_based/block_based_table_iterator.cc \
|
|
table/block_based/multi_scan_index_iterator.cc \
|
|
table/block_based/block_based_table_reader.cc \
|
|
table/block_based/block_builder.cc \
|
|
table/block_based/block_cache.cc \
|
|
table/block_based/block_prefetcher.cc \
|
|
table/block_based/block_prefix_index.cc \
|
|
table/block_based/builtin_index_factory.cc \
|
|
table/block_based/data_block_hash_index.cc \
|
|
table/block_based/data_block_footer.cc \
|
|
table/block_based/filter_block_reader_common.cc \
|
|
table/block_based/filter_policy.cc \
|
|
table/block_based/flush_block_policy.cc \
|
|
table/block_based/full_filter_block.cc \
|
|
table/block_based/hash_index_reader.cc \
|
|
table/block_based/index_builder.cc \
|
|
table/block_based/index_reader_common.cc \
|
|
table/block_based/parsed_full_filter_block.cc \
|
|
table/block_based/partitioned_filter_block.cc \
|
|
table/block_based/partitioned_index_iterator.cc \
|
|
table/block_based/partitioned_index_reader.cc \
|
|
table/block_based/reader_common.cc \
|
|
table/block_based/uncompression_dict_reader.cc \
|
|
table/block_fetcher.cc \
|
|
table/cuckoo/cuckoo_table_builder.cc \
|
|
table/cuckoo/cuckoo_table_factory.cc \
|
|
table/cuckoo/cuckoo_table_reader.cc \
|
|
table/external_table.cc \
|
|
table/format.cc \
|
|
table/get_context.cc \
|
|
table/iterator.cc \
|
|
table/merging_iterator.cc \
|
|
table/compaction_merging_iterator.cc \
|
|
table/meta_blocks.cc \
|
|
table/persistent_cache_helper.cc \
|
|
table/plain/plain_table_bloom.cc \
|
|
table/plain/plain_table_builder.cc \
|
|
table/plain/plain_table_factory.cc \
|
|
table/plain/plain_table_index.cc \
|
|
table/plain/plain_table_key_coding.cc \
|
|
table/plain/plain_table_reader.cc \
|
|
table/range_del_block.cc \
|
|
table/sst_file_dumper.cc \
|
|
table/sst_file_reader.cc \
|
|
table/sst_file_writer.cc \
|
|
table/table_factory.cc \
|
|
table/table_properties.cc \
|
|
table/two_level_iterator.cc \
|
|
table/unique_id.cc \
|
|
test_util/sync_point.cc \
|
|
test_util/sync_point_impl.cc \
|
|
test_util/transaction_test_util.cc \
|
|
tools/dump/db_dump_tool.cc \
|
|
trace_replay/trace_record_handler.cc \
|
|
trace_replay/trace_analyzer.cc \
|
|
trace_replay/trace_record_result.cc \
|
|
trace_replay/trace_record.cc \
|
|
trace_replay/trace_replay.cc \
|
|
trace_replay/block_cache_tracer.cc \
|
|
trace_replay/io_tracer.cc \
|
|
util/async_file_reader.cc \
|
|
util/auto_tune_compressor.cc \
|
|
util/build_version.cc \
|
|
util/cleanable.cc \
|
|
util/coding.cc \
|
|
util/compaction_job_stats_impl.cc \
|
|
util/comparator.cc \
|
|
util/compression.cc \
|
|
util/compression_context_cache.cc \
|
|
util/concurrent_task_limiter_impl.cc \
|
|
util/coro_stats_util.cc \
|
|
util/crc32c.cc \
|
|
util/crc32c_arm64.cc \
|
|
util/data_structure.cc \
|
|
util/dynamic_bloom.cc \
|
|
util/hash.cc \
|
|
util/murmurhash.cc \
|
|
util/random.cc \
|
|
util/rate_limiter.cc \
|
|
util/ribbon_config.cc \
|
|
util/slice.cc \
|
|
util/file_checksum_helper.cc \
|
|
util/simple_mixed_compressor.cc \
|
|
util/status.cc \
|
|
util/stderr_logger.cc \
|
|
util/string_util.cc \
|
|
util/thread_local.cc \
|
|
util/threadpool_imp.cc \
|
|
util/io_dispatcher_imp.cc \
|
|
util/io_prefetch_scheduler.cc \
|
|
util/udt_util.cc \
|
|
util/write_batch_util.cc \
|
|
util/xxhash.cc \
|
|
utilities/agg_merge/agg_merge.cc \
|
|
utilities/backup/backup_engine.cc \
|
|
utilities/blob_db/blob_compaction_filter.cc \
|
|
utilities/blob_db/blob_db.cc \
|
|
utilities/blob_db/blob_db_impl.cc \
|
|
utilities/blob_db/blob_db_impl_filesnapshot.cc \
|
|
utilities/blob_db/blob_file.cc \
|
|
utilities/cache_dump_load.cc \
|
|
utilities/cache_dump_load_impl.cc \
|
|
utilities/cassandra/cassandra_compaction_filter.cc \
|
|
utilities/cassandra/format.cc \
|
|
utilities/cassandra/merge_operator.cc \
|
|
utilities/checkpoint/checkpoint_impl.cc \
|
|
utilities/live_tune/live_tune_options_helper.cc \
|
|
utilities/live_tune/live_tuner.cc \
|
|
utilities/copy_engine/copy_engine.cc \
|
|
utilities/compaction_filters.cc \
|
|
utilities/db_split_merge.cc \
|
|
utilities/sorted_run_builder/sorted_run_builder.cc \
|
|
utilities/compaction_filters/remove_emptyvalue_compactionfilter.cc \
|
|
utilities/convenience/info_log_finder.cc \
|
|
utilities/counted_fs.cc \
|
|
utilities/debug.cc \
|
|
utilities/env_mirror.cc \
|
|
utilities/env_timed.cc \
|
|
utilities/fault_injection_env.cc \
|
|
utilities/fault_injection_fs.cc \
|
|
utilities/fault_injection_secondary_cache.cc \
|
|
utilities/leveldb_options/leveldb_options.cc \
|
|
utilities/memory/memory_util.cc \
|
|
utilities/merge_operators.cc \
|
|
utilities/merge_operators/max.cc \
|
|
utilities/merge_operators/put.cc \
|
|
utilities/merge_operators/sortlist.cc \
|
|
utilities/merge_operators/string_append/stringappend.cc \
|
|
utilities/merge_operators/string_append/stringappend2.cc \
|
|
utilities/merge_operators/uint64add.cc \
|
|
utilities/merge_operators/bytesxor.cc \
|
|
utilities/object_registry.cc \
|
|
utilities/option_change_migration/option_change_migration.cc \
|
|
utilities/options/options_util.cc \
|
|
utilities/persistent_cache/block_cache_tier.cc \
|
|
utilities/persistent_cache/block_cache_tier_file.cc \
|
|
utilities/persistent_cache/block_cache_tier_metadata.cc \
|
|
utilities/persistent_cache/persistent_cache_tier.cc \
|
|
utilities/persistent_cache/volatile_tier_impl.cc \
|
|
utilities/secondary_index/secondary_index_iterator.cc \
|
|
utilities/secondary_index/simple_secondary_index.cc \
|
|
utilities/simulator_cache/cache_simulator.cc \
|
|
utilities/simulator_cache/sim_cache.cc \
|
|
utilities/table_properties_collectors/compact_for_tiering_collector.cc \
|
|
utilities/table_properties_collectors/compact_on_deletion_collector.cc \
|
|
utilities/trace/file_trace_reader_writer.cc \
|
|
utilities/trace/replayer_impl.cc \
|
|
utilities/transactions/lock/lock_manager.cc \
|
|
utilities/transactions/lock/point/point_lock_tracker.cc \
|
|
utilities/transactions/lock/point/point_lock_manager.cc \
|
|
utilities/transactions/optimistic_transaction.cc \
|
|
utilities/transactions/optimistic_transaction_db_impl.cc \
|
|
utilities/transactions/pessimistic_transaction.cc \
|
|
utilities/transactions/pessimistic_transaction_db.cc \
|
|
utilities/transactions/snapshot_checker.cc \
|
|
utilities/transactions/transaction_base.cc \
|
|
utilities/transactions/transaction_db_mutex_impl.cc \
|
|
utilities/transactions/transaction_util.cc \
|
|
utilities/transactions/write_prepared_txn.cc \
|
|
utilities/transactions/write_prepared_txn_db.cc \
|
|
utilities/transactions/write_unprepared_txn.cc \
|
|
utilities/transactions/write_unprepared_txn_db.cc \
|
|
utilities/ttl/db_ttl_impl.cc \
|
|
utilities/trie_index/bitvector.cc \
|
|
utilities/trie_index/louds_trie.cc \
|
|
utilities/trie_index/trie_index_factory.cc \
|
|
utilities/types_util.cc \
|
|
utilities/wal_filter.cc \
|
|
utilities/write_batch_with_index/write_batch_with_index.cc \
|
|
utilities/write_batch_with_index/write_batch_with_index_internal.cc \
|
|
|
|
ifeq (,$(shell $(CXX) -fsyntax-only -maltivec -xc /dev/null 2>&1))
|
|
LIB_SOURCES_ASM =\
|
|
util/crc32c_ppc_asm.S
|
|
LIB_SOURCES_C = \
|
|
util/crc32c_ppc.c
|
|
else
|
|
LIB_SOURCES_ASM =
|
|
LIB_SOURCES_C =
|
|
endif
|
|
|
|
WITH_FAISS_LIB_SOURCES = \
|
|
utilities/secondary_index/faiss_ivf_index.cc \
|
|
|
|
RANGE_TREE_SOURCES =\
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/concurrent_tree.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/keyrange.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/lock_request.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/locktree.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/manager.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/range_buffer.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/treenode.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/txnid_set.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/locktree/wfg.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/standalone_port.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/util/dbt.cc \
|
|
utilities/transactions/lock/range/range_tree/lib/util/memarena.cc \
|
|
utilities/transactions/lock/range/range_tree/range_tree_lock_manager.cc \
|
|
utilities/transactions/lock/range/range_tree/range_tree_lock_tracker.cc
|
|
|
|
TOOL_LIB_SOURCES = \
|
|
db_stress_tool/db_stress_compression_manager.cc \
|
|
tools/io_tracer_parser_tool.cc \
|
|
tools/ldb_cmd.cc \
|
|
tools/ldb_tool.cc \
|
|
tools/sst_dump_tool.cc \
|
|
utilities/blob_db/blob_dump_tool.cc \
|
|
|
|
ANALYZER_LIB_SOURCES = \
|
|
tools/block_cache_analyzer/block_cache_trace_analyzer.cc \
|
|
tools/trace_analyzer_tool.cc \
|
|
|
|
MOCK_LIB_SOURCES = \
|
|
table/mock_table.cc \
|
|
|
|
BENCH_LIB_SOURCES = \
|
|
tools/db_bench_tool.cc \
|
|
tools/tool_hooks.cc \
|
|
tools/simulated_hybrid_file_system.cc \
|
|
|
|
CACHE_BENCH_LIB_SOURCES = \
|
|
cache/cache_bench_tool.cc \
|
|
|
|
POINT_LOCK_BENCH_LIB_SOURCES = \
|
|
utilities/transactions/lock/point/point_lock_bench_tool.cc \
|
|
|
|
STRESS_LIB_SOURCES = \
|
|
db_stress_tool/batched_ops_stress.cc \
|
|
db_stress_tool/cf_consistency_stress.cc \
|
|
db_stress_tool/db_stress_common.cc \
|
|
db_stress_tool/db_stress_compaction_service.cc \
|
|
db_stress_tool/db_stress_compression_manager.cc \
|
|
db_stress_tool/db_stress_driver.cc \
|
|
db_stress_tool/db_stress_filters.cc \
|
|
db_stress_tool/db_stress_gflags.cc \
|
|
db_stress_tool/db_stress_listener.cc \
|
|
db_stress_tool/db_stress_shared_state.cc \
|
|
db_stress_tool/db_stress_test_base.cc \
|
|
db_stress_tool/db_stress_tool.cc \
|
|
db_stress_tool/db_stress_wide_merge_operator.cc \
|
|
db_stress_tool/expected_state.cc \
|
|
db_stress_tool/expected_value.cc \
|
|
db_stress_tool/no_batched_ops_stress.cc \
|
|
db_stress_tool/multi_ops_txns_stress.cc \
|
|
test_util/simple_external_table_factory.cc \
|
|
|
|
TEST_LIB_SOURCES = \
|
|
db/db_test_util.cc \
|
|
db/db_with_timestamp_test_util.cc \
|
|
test_util/mock_time_env.cc \
|
|
test_util/simple_external_table_factory.cc \
|
|
test_util/secondary_cache_test_util.cc \
|
|
test_util/testharness.cc \
|
|
test_util/testutil.cc \
|
|
utilities/agg_merge/test_agg_merge.cc \
|
|
utilities/cassandra/test_utils.cc \
|
|
|
|
FOLLY_SOURCES = \
|
|
$(FOLLY_DIR)/folly/container/detail/F14Table.cpp \
|
|
$(FOLLY_DIR)/folly/detail/Futex.cpp \
|
|
$(FOLLY_DIR)/folly/lang/Exception.cpp \
|
|
$(FOLLY_DIR)/folly/lang/SafeAssert.cpp \
|
|
$(FOLLY_DIR)/folly/lang/ToAscii.cpp \
|
|
$(FOLLY_DIR)/folly/ScopeGuard.cpp \
|
|
$(FOLLY_DIR)/folly/synchronization/AtomicNotification.cpp \
|
|
$(FOLLY_DIR)/folly/synchronization/DistributedMutex.cpp \
|
|
$(FOLLY_DIR)/folly/synchronization/ParkingLot.cpp \
|
|
|
|
TOOLS_MAIN_SOURCES = \
|
|
db_stress_tool/db_stress.cc \
|
|
tools/blob_dump.cc \
|
|
tools/block_cache_analyzer/block_cache_trace_analyzer_tool.cc \
|
|
tools/db_split_merge.cc \
|
|
tools/db_repl_stress.cc \
|
|
tools/db_sanity_test.cc \
|
|
tools/ldb.cc \
|
|
tools/io_tracer_parser.cc \
|
|
tools/sst_dump.cc \
|
|
tools/write_stress.cc \
|
|
tools/dump/rocksdb_dump.cc \
|
|
tools/dump/rocksdb_undump.cc \
|
|
tools/trace_analyzer.cc \
|
|
tools/io_tracer_parser_tool.cc \
|
|
|
|
BENCH_MAIN_SOURCES = \
|
|
cache/cache_bench.cc \
|
|
db/range_del_aggregator_bench.cc \
|
|
memtable/memtablerep_bench.cc \
|
|
table/table_reader_bench.cc \
|
|
tools/db_bench.cc \
|
|
util/filter_bench.cc \
|
|
utilities/persistent_cache/persistent_cache_bench.cc \
|
|
|
|
TEST_MAIN_SOURCES = \
|
|
cache/cache_test.cc \
|
|
cache/cache_reservation_manager_test.cc \
|
|
cache/compressed_secondary_cache_test.cc \
|
|
cache/lru_cache_test.cc \
|
|
cache/tiered_secondary_cache_test.cc \
|
|
db/blob/blob_counting_iterator_test.cc \
|
|
db/blob/blob_file_addition_test.cc \
|
|
db/blob/blob_file_builder_test.cc \
|
|
db/blob/blob_file_cache_test.cc \
|
|
db/blob/blob_file_garbage_test.cc \
|
|
db/blob/blob_file_reader_test.cc \
|
|
db/blob/blob_garbage_meter_test.cc \
|
|
db/blob/blob_source_test.cc \
|
|
db/blob/db_blob_basic_test.cc \
|
|
db/blob/db_blob_direct_write_test.cc \
|
|
db/blob/db_blob_compaction_test.cc \
|
|
db/blob/db_blob_corruption_test.cc \
|
|
db/blob/db_blob_index_test.cc \
|
|
db/column_family_test.cc \
|
|
db/compact_files_test.cc \
|
|
db/compaction/clipping_iterator_test.cc \
|
|
db/compaction/compaction_iterator_test.cc \
|
|
db/compaction/compaction_job_test.cc \
|
|
db/compaction/compaction_job_stats_test.cc \
|
|
db/compaction/compaction_picker_test.cc \
|
|
db/compaction/compaction_service_test.cc \
|
|
db/compaction/tiered_compaction_test.cc \
|
|
db/comparator_db_test.cc \
|
|
db/corruption_test.cc \
|
|
db/cuckoo_table_db_test.cc \
|
|
db/db_basic_test.cc \
|
|
db/db_block_cache_test.cc \
|
|
db/db_bloom_filter_test.cc \
|
|
db/db_compaction_abort_test.cc \
|
|
db/db_compaction_filter_test.cc \
|
|
db/db_compaction_test.cc \
|
|
db/db_clip_test.cc \
|
|
utilities/db_split_merge_test.cc \
|
|
db/db_dynamic_level_test.cc \
|
|
db/db_encryption_test.cc \
|
|
db/db_etc2_test.cc \
|
|
db/db_etc3_test.cc \
|
|
db/db_flush_test.cc \
|
|
db/db_follower_test.cc \
|
|
db/db_readonly_with_timestamp_test.cc \
|
|
db/db_with_timestamp_basic_test.cc \
|
|
db/import_column_family_test.cc \
|
|
db/db_inplace_update_test.cc \
|
|
db/db_io_failure_test.cc \
|
|
db/db_iter_test.cc \
|
|
db/db_iter_stress_test.cc \
|
|
db/db_iterator_test.cc \
|
|
db/db_kv_checksum_test.cc \
|
|
db/db_log_iter_test.cc \
|
|
db/db_memtable_test.cc \
|
|
db/db_merge_operator_test.cc \
|
|
db/db_merge_operand_test.cc \
|
|
db/db_open_with_config_test.cc \
|
|
db/db_options_test.cc \
|
|
db/db_properties_test.cc \
|
|
db/db_range_del_test.cc \
|
|
db/db_rate_limiter_test.cc \
|
|
db/db_secondary_test.cc \
|
|
db/db_sst_test.cc \
|
|
db/db_statistics_test.cc \
|
|
db/db_table_properties_test.cc \
|
|
db/db_tailing_iter_test.cc \
|
|
db/db_test.cc \
|
|
db/db_logical_block_size_cache_test.cc \
|
|
db/db_universal_compaction_test.cc \
|
|
db/db_wal_test.cc \
|
|
db/db_with_timestamp_compaction_test.cc \
|
|
db/db_write_buffer_manager_test.cc \
|
|
db/db_write_test.cc \
|
|
db/dbformat_test.cc \
|
|
db/deletefile_test.cc \
|
|
db/error_handler_fs_test.cc \
|
|
db/external_sst_file_basic_test.cc \
|
|
db/external_sst_file_test.cc \
|
|
db/fault_injection_test.cc \
|
|
db/file_indexer_test.cc \
|
|
db/filename_test.cc \
|
|
db/flush_job_test.cc \
|
|
db/listener_test.cc \
|
|
db/log_test.cc \
|
|
db/manual_compaction_test.cc \
|
|
db/memtable_list_test.cc \
|
|
db/merge_helper_test.cc \
|
|
db/merge_test.cc \
|
|
db/multi_cf_iterator_test.cc \
|
|
db/obsolete_files_test.cc \
|
|
db/options_file_test.cc \
|
|
db/perf_context_test.cc \
|
|
db/periodic_task_scheduler_test.cc \
|
|
db/plain_table_db_test.cc \
|
|
db/prefix_test.cc \
|
|
db/repair_test.cc \
|
|
db/range_del_aggregator_test.cc \
|
|
db/range_tombstone_fragmenter_test.cc \
|
|
db/seqno_time_test.cc \
|
|
db/table_properties_collector_test.cc \
|
|
db/version_builder_test.cc \
|
|
db/version_edit_test.cc \
|
|
db/version_set_test.cc \
|
|
db/wal_manager_test.cc \
|
|
db/wide/db_lazy_entity_test.cc \
|
|
db/wide/db_wide_basic_test.cc \
|
|
db/wide/db_wide_blob_direct_write_test.cc \
|
|
db/wide/wide_column_serialization_test.cc \
|
|
db/wide/wide_columns_helper_test.cc \
|
|
db/write_batch_test.cc \
|
|
db/write_callback_test.cc \
|
|
db/write_controller_test.cc \
|
|
env/env_basic_test.cc \
|
|
env/env_test.cc \
|
|
env/io_posix_test.cc \
|
|
env/mock_env_test.cc \
|
|
file/delete_scheduler_test.cc \
|
|
file/prefetch_test.cc \
|
|
file/random_access_file_reader_test.cc \
|
|
logging/auto_roll_logger_test.cc \
|
|
logging/env_logger_test.cc \
|
|
logging/event_logger_test.cc \
|
|
memory/arena_test.cc \
|
|
memory/memory_allocator_test.cc \
|
|
memtable/inlineskiplist_test.cc \
|
|
memtable/skiplist_test.cc \
|
|
memtable/write_buffer_manager_test.cc \
|
|
monitoring/histogram_test.cc \
|
|
monitoring/iostats_context_test.cc \
|
|
monitoring/statistics_test.cc \
|
|
monitoring/stats_history_test.cc \
|
|
options/configurable_test.cc \
|
|
options/customizable_test.cc \
|
|
options/options_settable_test.cc \
|
|
options/options_test.cc \
|
|
table/block_based/block_based_table_reader_test.cc \
|
|
table/block_based/block_test.cc \
|
|
table/block_based/builtin_index_factory_test.cc \
|
|
table/block_based/data_block_hash_index_test.cc \
|
|
table/block_based/full_filter_block_test.cc \
|
|
table/block_based/partitioned_filter_block_test.cc \
|
|
table/cleanable_test.cc \
|
|
table/cuckoo/cuckoo_table_builder_test.cc \
|
|
table/cuckoo/cuckoo_table_reader_test.cc \
|
|
table/merger_test.cc \
|
|
table/sst_file_reader_test.cc \
|
|
table/table_test.cc \
|
|
table/block_fetcher_test.cc \
|
|
test_util/testutil_test.cc \
|
|
util/compression_test.cc \
|
|
tools/block_cache_analyzer/block_cache_trace_analyzer_test.cc \
|
|
tools/io_tracer_parser_test.cc \
|
|
tools/ldb_cmd_test.cc \
|
|
tools/reduce_levels_test.cc \
|
|
tools/sst_dump_test.cc \
|
|
tools/trace_analyzer_test.cc \
|
|
trace_replay/block_cache_tracer_test.cc \
|
|
trace_replay/io_tracer_test.cc \
|
|
util/autovector_test.cc \
|
|
util/bloom_test.cc \
|
|
util/coding_test.cc \
|
|
util/crc32c_test.cc \
|
|
util/defer_test.cc \
|
|
util/dynamic_bloom_test.cc \
|
|
util/filelock_test.cc \
|
|
util/file_reader_writer_test.cc \
|
|
util/hash_test.cc \
|
|
util/heap_test.cc \
|
|
util/interval_test.cc \
|
|
util/io_dispatcher_test.cc \
|
|
util/random_test.cc \
|
|
util/rate_limiter_test.cc \
|
|
util/repeatable_thread_test.cc \
|
|
util/ribbon_test.cc \
|
|
util/slice_test.cc \
|
|
util/slice_transform_test.cc \
|
|
util/timer_queue_test.cc \
|
|
util/timer_test.cc \
|
|
util/thread_list_test.cc \
|
|
util/thread_local_test.cc \
|
|
util/udt_util_test.cc \
|
|
util/work_queue_test.cc \
|
|
utilities/agg_merge/agg_merge_test.cc \
|
|
utilities/backup/backup_engine_test.cc \
|
|
utilities/blob_db/blob_db_test.cc \
|
|
utilities/cassandra/cassandra_format_test.cc \
|
|
utilities/cassandra/cassandra_functional_test.cc \
|
|
utilities/cassandra/cassandra_row_merge_test.cc \
|
|
utilities/cassandra/cassandra_serialize_test.cc \
|
|
utilities/checkpoint/checkpoint_test.cc \
|
|
utilities/live_tune/live_tuner_test.cc \
|
|
utilities/copy_engine/copy_engine_test.cc \
|
|
utilities/sorted_run_builder/sorted_run_builder_test.cc \
|
|
utilities/env_timed_test.cc \
|
|
utilities/fault_injection_fs_test.cc \
|
|
utilities/memory/memory_test.cc \
|
|
utilities/merge_operators/string_append/stringappend_test.cc \
|
|
utilities/object_registry_test.cc \
|
|
utilities/option_change_migration/option_change_migration_test.cc \
|
|
utilities/options/options_util_test.cc \
|
|
utilities/persistent_cache/hash_table_test.cc \
|
|
utilities/persistent_cache/persistent_cache_test.cc \
|
|
utilities/simulator_cache/cache_simulator_test.cc \
|
|
utilities/simulator_cache/sim_cache_test.cc \
|
|
utilities/table_properties_collectors/compact_for_tiering_collector_test.cc \
|
|
utilities/table_properties_collectors/compact_on_deletion_collector_test.cc \
|
|
utilities/transactions/optimistic_transaction_test.cc \
|
|
utilities/transactions/lock/range/range_locking_test.cc \
|
|
utilities/transactions/transaction_test.cc \
|
|
utilities/transactions/lock/point/point_lock_manager_test.cc \
|
|
utilities/transactions/lock/point/point_lock_manager_stress_test.cc \
|
|
utilities/transactions/write_prepared_transaction_test.cc \
|
|
utilities/transactions/write_prepared_transaction_seqno_test.cc \
|
|
utilities/transactions/write_unprepared_transaction_test.cc \
|
|
utilities/transactions/write_committed_transaction_ts_test.cc \
|
|
utilities/transactions/timestamped_snapshot_test.cc \
|
|
utilities/ttl/ttl_test.cc \
|
|
utilities/trie_index/trie_index_db_test.cc \
|
|
utilities/trie_index/trie_index_test.cc \
|
|
utilities/types_util_test.cc \
|
|
utilities/util_merge_operators_test.cc \
|
|
utilities/write_batch_with_index/write_batch_with_index_test.cc \
|
|
|
|
TEST_MAIN_SOURCES_C = \
|
|
db/c_test.c \
|
|
|
|
WITH_FAISS_TEST_MAIN_SOURCES = \
|
|
utilities/secondary_index/faiss_ivf_index_test.cc \
|
|
|
|
MICROBENCH_SOURCES = \
|
|
microbench/ribbon_bench.cc \
|
|
microbench/db_basic_bench.cc \
|
|
|
|
JNI_NATIVE_SOURCES = \
|
|
java/rocksjni/backupenginejni.cc \
|
|
java/rocksjni/backup_engine_options.cc \
|
|
java/rocksjni/checkpoint.cc \
|
|
java/rocksjni/clock_cache.cc \
|
|
java/rocksjni/cache.cc \
|
|
java/rocksjni/columnfamilyhandle.cc \
|
|
java/rocksjni/compact_range_options.cc \
|
|
java/rocksjni/compaction_filter.cc \
|
|
java/rocksjni/compaction_filter_factory.cc \
|
|
java/rocksjni/compaction_filter_factory_jnicallback.cc \
|
|
java/rocksjni/compaction_job_info.cc \
|
|
java/rocksjni/compaction_job_stats.cc \
|
|
java/rocksjni/compaction_options.cc \
|
|
java/rocksjni/compaction_options_fifo.cc \
|
|
java/rocksjni/compaction_options_universal.cc \
|
|
java/rocksjni/comparator.cc \
|
|
java/rocksjni/comparatorjnicallback.cc \
|
|
java/rocksjni/compression_options.cc \
|
|
java/rocksjni/concurrent_task_limiter.cc \
|
|
java/rocksjni/config_options.cc \
|
|
java/rocksjni/export_import_files_metadatajni.cc \
|
|
java/rocksjni/env.cc \
|
|
java/rocksjni/env_options.cc \
|
|
java/rocksjni/event_listener.cc \
|
|
java/rocksjni/event_listener_jnicallback.cc \
|
|
java/rocksjni/import_column_family_options.cc \
|
|
java/rocksjni/ingest_external_file_options.cc \
|
|
java/rocksjni/filter.cc \
|
|
java/rocksjni/hyper_clock_cache.cc \
|
|
java/rocksjni/iterator.cc \
|
|
java/rocksjni/jni_perf_context.cc \
|
|
java/rocksjni/jni_multiget_helpers.cc \
|
|
java/rocksjni/jnicallback.cc \
|
|
java/rocksjni/loggerjnicallback.cc \
|
|
java/rocksjni/lru_cache.cc \
|
|
java/rocksjni/memtablejni.cc \
|
|
java/rocksjni/memory_util.cc \
|
|
java/rocksjni/merge_operator.cc \
|
|
java/rocksjni/native_comparator_wrapper_test.cc \
|
|
java/rocksjni/optimistic_transaction_db.cc \
|
|
java/rocksjni/optimistic_transaction_options.cc \
|
|
java/rocksjni/options.cc \
|
|
java/rocksjni/options_util.cc \
|
|
java/rocksjni/persistent_cache.cc \
|
|
java/rocksjni/ratelimiterjni.cc \
|
|
java/rocksjni/remove_emptyvalue_compactionfilterjni.cc \
|
|
java/rocksjni/cassandra_compactionfilterjni.cc \
|
|
java/rocksjni/cassandra_value_operator.cc \
|
|
java/rocksjni/restorejni.cc \
|
|
java/rocksjni/rocks_callback_object.cc \
|
|
java/rocksjni/rocksjni.cc \
|
|
java/rocksjni/rocksdb_exception_test.cc \
|
|
java/rocksjni/slice.cc \
|
|
java/rocksjni/snapshot.cc \
|
|
java/rocksjni/sst_file_manager.cc \
|
|
java/rocksjni/sst_file_writerjni.cc \
|
|
java/rocksjni/sst_file_readerjni.cc \
|
|
java/rocksjni/sst_file_reader_iterator.cc \
|
|
java/rocksjni/sst_partitioner.cc \
|
|
java/rocksjni/statistics.cc \
|
|
java/rocksjni/statisticsjni.cc \
|
|
java/rocksjni/stderr_logger.cc \
|
|
java/rocksjni/table.cc \
|
|
java/rocksjni/table_filter.cc \
|
|
java/rocksjni/table_filter_jnicallback.cc \
|
|
java/rocksjni/table_properties_collector_factory.cc \
|
|
java/rocksjni/thread_status.cc \
|
|
java/rocksjni/trace_writer.cc \
|
|
java/rocksjni/trace_writer_jnicallback.cc \
|
|
java/rocksjni/transaction.cc \
|
|
java/rocksjni/transaction_db.cc \
|
|
java/rocksjni/transaction_options.cc \
|
|
java/rocksjni/transaction_db_options.cc \
|
|
java/rocksjni/transaction_notifier.cc \
|
|
java/rocksjni/transaction_notifier_jnicallback.cc \
|
|
java/rocksjni/ttl.cc \
|
|
java/rocksjni/testable_event_listener.cc \
|
|
java/rocksjni/wal_filter.cc \
|
|
java/rocksjni/wal_filter_jnicallback.cc \
|
|
java/rocksjni/wal_iterator.cc \
|
|
java/rocksjni/write_batch.cc \
|
|
java/rocksjni/writebatchhandlerjnicallback.cc \
|
|
java/rocksjni/write_batch_test.cc \
|
|
java/rocksjni/write_batch_with_index.cc \
|
|
java/rocksjni/write_buffer_manager.cc
|