mirror of
https://github.com/FerretDB/documentdb.git
synced 2026-11-07 03:22:46 -05:00
Deepak Bisht
b1471da24d
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) BugFix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes `documentdb.forceBitmapScanForLookup` now controls if we need force bitmap index scan paths for lookup JOIN queries ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) - $lookup JOIN condition pushed to index more reliably and avoids bitmap heap scan conversion. ### Description Fix index pushdown for $lookup with more data. Problem description: 1- Today when the lookup index scan paths are converted to bitmap heap scan path and while doing so we loose the parameterization info already build by PG planner, and the converted path considers all rows to be returned, this impacts index selectivity as the path becomes expensive. Solution: - Fixed the bitmap heap scan path by adjusting the estimated rows returned from the parameterized info. - Skip force conversion of index path to bitmap heap scan path for $lookup JOIN cases behind a GUC which should be default behavior going forward ---- #### AI description (iteration 1) #### PR Classification Bug fix addressing incorrect $lookup index pushdown and lookup join behavior. #### PR Summary This PR refines the planner and execution paths for $lookup queries by improving bitmap scan conversion, selectivity estimation, and associated configuration while updating regression tests to validate the new behavior. - `pgmongo/src/test/docdb_compat/expected/bson_aggregation_tests_lookup_inner_join.out` and `oss/internal/pg_documentdb_distributed/src/test/regress/sql/bson_aggregation_tests_lookup_inner_join.sql`: Updated test cases and expected outputs to exercise various lookup join scenarios and index usage modes. - `oss/pg_documentdb/src/planner/documents_planner.c`: Introduced the helper `IsBitmapHeapConversionSupported` to centralize conditions for pushing down bitmap heap conversion, replacing the older merge query check. - `oss/pg_documentdb/src/customscan/custom_scan_continuation.c`: Improved handling of parameterization in index pushdown paths for lookup queries to restore accurate row estimates. - `oss/pg_documentdb/src/opclass/index_support.c` and `oss/pg_documentdb/src/query/bson_dollar_selectivity.c`: Modified lookup filter support functions to enforce low selectivity for lookup-modified index quals. - `oss/pg_documentdb/src/configs/feature_flag_configs.c` and `oss/pg_documentdb/include/query/bson_dollar_selectivity.h`: Added and centralized the `forceBitmapScanForLookup` flag along with selectivity constants for consistent behavior. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4522655