mirror of
https://github.com/FerretDB/documentdb.git
synced 2026-11-07 03:22:46 -05:00
main
36
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vinod Sridharan
|
84b999ae88 |
Merged PR 1815453: [Composite] Support optimized elemMatch pushdown
### Does this PR have any customer impact?
Yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
### Description
Improve pushdown of $elemMatch for composite indexes.
When there's an elemMatch of the form `{ "a": { "$elemMatch": { "$gt": 10, "$lt": 15 }}`
This can scan the index only from 10 -> 15
However given the multi-key index requirement we were doing 2 scans:
- One from 10 to infinity and the other from -infinity to 15.
Optimize this path for $elemMatch.
----
#### AI description (iteration 1)
#### PR Classification
This PR introduces a new feature that optimizes the pushdown of `$elemMatch` queries for composite indexes.
#### PR Summary
The pull request implements optimized handling of `$elemMatch` predicates on composite indexes by converting them into specialized index pushdown expressions and updating the corresponding test expectations.
- **`oss/pg_documentdb/src/opclass/bson_gin_composite_core.c` & `index_support.c`**: Added logic to detect `$elemMatch` in range queries and convert them into optimized pushdown expressions (via functions like `GetElemMatchIndexPushdownOperator`).
- **Configuration Changes**: Introduced a new GUC (`documentdb.useNewElemMatchIndexOperatorOnPushdown`) in `pg_documentdb/src/configs/feature_flag_configs.c` to control the new behavior.
- **Test Updates**: Modified expected explain outputs in both `/oss/internal/pg_documentdb_distributed/src/test/regress/` and `/pgmongo/src/test/docdb_compat/expected/` to reflect the revised index condition format using `elemMatchIndexOp`.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
|
||
 Deepak Bisht
|
0f390b2cbb |
Merged PR 1798239: Fix Segmentation faults in ordered aggregate
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) BugFix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description - For aggregates such as `$last` or `$first` where the total number of aggregate values is 1 and existing aggregate state is more than the space required we try to use the same memory space while also updating it on the go. This will overwrite some later referred variable from the state. - The changes also fixes a memory leak in ordered aggregates where the new transition state was allocate in top level aggregate context and PG makes a copy and discards the original value (without freeing it), now we allocate the new transition state only in the CurrentContext which is the per tuple execution context So always create a new memory space for transition in ordered aggregates ---- #### AI description (iteration 1) #### PR Classification This pull request fixes a bug causing segmentation faults in ordered aggregate operations involving the `$last` operator used with the `$setWindowFields` stage. #### PR Summary This update resolves memory issues by removing unsafe byte array reuse in the state serialization for ordered aggregates and enhances regression tests to verify correct behavior. - `oss/pg_documentdb/src/aggregation/bson_sorted_accumulator.c`: Modified `SerializeOrderState` to always allocate new memory and removed the old byte array reuse logic. - `oss/pg_documentdb/include/aggregation/bson_sorted_accumulator.h`: Updated the function signature of `SerializeOrderState` to reflect the new implementation. - Test files in `oss/internal/pg_documentdb_distributed/src/test/regress` and `pgmongo/src/test/docdb_compat`: Added new tests for `$last` to check ordered aggregate functionality with `$setWindowFields`. - Documentation files (`docs/release_notes/pgmongo_1.108.md` and `oss/CHANGELOG.md`): Updated release notes and changelog to document the segmentation fault fix. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4687821 Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
 Jinrui Zhang
|
3ccd656663 |
Merged PR 1777067: [Refector] Update to match coding standards - part 0
### Description 1. Fixing issues identified by similarity checker 2. Fix the corresponding sql unit tests 3. Fix JSTests failure due to above changes 4. Fix corresponding C# test failure ---- #### AI description (iteration 1) #### PR Classification This pull request is a code cleanup that standardizes and improves error message wording across the codebase. #### PR Summary The changes update error messages throughout the repository to provide clearer, more consistent, and more detailed feedback when operations fail. - Updates in geospatial operator files (e.g. in `$geoNear`, `$near`, `$nearSphere`) replace informal phrasing with precise messages. - Numerous error messages in aggregation and projection test files have been reworded for consistent phrasing <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 Siwen Tang
|
be11798bef |
Merged PR 1778507: [Refector] Update to match coding standards - part 5
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description draft update error msg ---- #### AI description (iteration 1) #### PR Classification This is a code cleanup and bug‑fix pull request that standardizes and clarifies error messages and log details across multiple modules. #### PR Summary The changes improve the consistency, clarity, and accuracy of user‑facing messages and logging throughout the code base. Key changes include: - In files under directories such as `oss/internal/pg_documentdb_distributed/src/test/regress/expected/` and `pgmongo/src/test/docdb_compat/expected/`, error messages for geospatial query operators (e.g. geoNear, $center, $within) and aggregation stage operators have been rewritten for clarity (e.g. replacing vague “Too many geoNear expressions” with “Excessive number of geoNear query expressions” and clarifying numeric parameter validations). - Several operators’ messages (e.g. `$cond`, `$not`, `$abs`, `$floor`, `$divide`, `$log`, etc.) have been updated to state exactly how many arguments are required, using <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 Duan Pei
|
6a129514a9 |
Merged PR 1777887: [Refector] Update to match coding standards - part 1
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description update errmsg ---- #### AI description (iteration 1) #### PR Classification This pull request focuses on updating and standardizing error messages across various operators, aggregation stages, and test expectation files. #### PR Summary The changes reword error messages throughout the codebase and test outputs to provide clearer, more consistent, and more informative feedback for users. Key updates include: - **`/oss/internal/pg_documentdb_distributed/src/test/regress/expected/*.out`** – Revised expected error outputs for aggregation, update, and geospatial tests to use clearer phrasing (e.g., for `$merge`, `$pop`, `$indexOfArray`, and `$geoNear`). - **Core operator files (e.g., `bson_aggregation_output_pipeline.c`, `bson_expression_arithmetic_operators.c`, `bson_expression_conditional_operators.c`)** – Updated error message texts to explicitly state operator requirements and types. - **Geospatial and vector modules (e.g., files under `bson_geojson_utils.c`, `bson_geospatial_*`, and vector search/index configuration files)** – Standardized validations and error reporting for GeoJSON, CRS, and vector search parameters. - **Additional modules (e.g., collection view creation and user management files)** – Improved error handling messages to better guide users on input misconfigurations. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Vinod Sridharan
|
10b395a75b |
Merged PR 1773514: [Orderby] Improve order by pushdown behavior & selectivity
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Pushing down order by and filters to a composite index should prefer the smallest index with the highest number of order by columns matching. To do this we need to participate in the core add_path set up as opposed to after. Consequently, we mutate the indexPath when its created in create_index_path as opposed to modifying it in the relpathlist_hook. We also reorder the indexes by paths so that for filters, we prefer indexes with fewer paths over many that can match the same paths. ---- #### AI description (iteration 1) #### PR Classification This pull request refines the ORDER BY pushdown mechanism in index scans by adding new processing logic and improving index selectivity while introducing a feature flag for the legacy behavior. #### PR Summary The changes enhance query performance by introducing distinct pushdown paths for ORDER BY clauses, refining composite index handling, and updating test expectations to reflect the new "orderByScan" operator. The following key modifications were made: - **`oss/pg_documentdb/src/opclass/index_support.c`**: Added new functions (e.g., `ConsiderIndexOrderByPushdownNew`, `ProcessOrderByStatements`) to split new versus legacy ORDER BY pushdown logic. - **Test Files** (under `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/` and `/pgmongo/src/test/docdb_compat/expected/`): Updated expected query plans to use the new "orderByScan" predicate instead of "fullScan". - **Index and Planner Modules** (e.g., `oss/pg_documentdb/src/index_am/rum.c` and `planner/documents_planner.c`): Enhanced composite index ranking to factor in the number of index paths. - **Configuration Files** (e.g., `oss/pg_documentdb/src/configs/feature_flag_configs.c`): Introduced the `EnableIndexOrderbyPushdownLegacy` flag for toggling the legacy behavior. - **Supporting Modules** (in aggregation and query operators): Adjusted functions in files like `bson_query_common.c` and related headers to handle new order-by scan directions consistently. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Vinod Sridharan
|
623ef26f85 |
Merged PR 1742378: [Hints] Add support for index hints in DocumentDB
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Index hints are a mongo feature that allows customers to specify which index they would like to target for a query. Typically this is used in diagnostic scenarios when testing query performance. With the new composite index op class coming in, we’re introducing side by side support for composite which makes it so we would want customers to be able to test which index is better for planning. In this spirit, index hints is introduced which follows the same infra as the force pushdown for geonear and text and adds a new option for index hints. As part of this the code is also refactored to cleanup some of the logic in the relpathlist and fix correctness issues around text index force pushdown (which was needed to handle index hints) ---- #### AI description (iteration 1) #### PR Classification This pull request introduces a new feature to support index hints in DocumentDB. #### PR Summary The changes integrate index hint support into query planning and execution, allowing users to force a specific index during query processing. Key updates include: - **`src/opclass/index_support.c`**: Added new functions and a force index operator type (`ForceIndexOpType_IndexHint`) for filtering available indexes and generating custom operator expressions based on index hints. - **`src/planner/documents_planner.c`**: Modified base table query planning logic to detect index hints and adjust index pushdown, including updates to path and restriction handling. - New UDFs (e.g., `bson_dollar_index_hint`) and configuration variables (`EnableIndexHintSupport`, `UseLegacyForcePushdownBehavior`) were added to support the hint mechanism. - Test files and SQL scripts (e.g., in `/oss/internal/pg_documentdb_distributed/src/test/regress/`) were introduced or updated to validate index hint pushdown behavior. - Documentation and release notes (e.g., `docs/pgmongo/design_docs/data_plane/indexing/index_hints.md`, `pgmongo_1.106.md`, and `CHANGELOG.md`) were updated to describe the index hint feature. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 Cornelius Boateng
|
f983efd1da |
Merged PR 1683401: [engine][let] Support for update
### Does this PR have any customer impact? No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No
### Are you introducing any new config? If yes, do you have tests with and without them being set? No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
### Description
Support let and `$$NOW` variables with update queries.
The values of the variables can be accessed in the query `q` with `$expr` and 'u' with an aggregation pipeline.
Example:
```
{
update: <collection>,
updates: [
{
q: {$expr: {$eq: ["$_id", "$$varRef"]}},
u: [{$set: {"a": "$$NOW"}}],
multi: false, // updateOne
}
],
let: {"varRef": 1},
...
}
```
We parse the let value (together with the time system variables - $$NOW) and propagate it through the aggregation and variable contexts.
To enable feature, turn on the `EnableVariablesSupportForWriteCommands` GUC.
----
#### AI description (iteration 1)
#### PR Classification
This PR implements a new feature by adding support for the "let" variable specification in update commands.
#### PR Summary
The changes integrate "let" support into update operations, allowing dynamic variable specifications to be applied during update candidate selection, serialization, and execution. This enhancement improves the flexibility and expressiveness of update queries in the engine.
- In `src/commands/update.c`, the code now parses and applies a new `variableSpec` (under the "let" field) when building update queries and managing SQL arguments.
- In `include/infrastructure/documentdb_plan_cache.h`, new plan cache macros are added to account for update candidates using let and collation.
- In `include/commands/update.h`, the `UpdateOneParams` structure is updated to include the `variableSpec` parameter.
- In `src/commands/delete.c`, references to the unsupported "let" field in delete commands are removed.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Related work items: #3423115, #3613875
|
||
|
Duan Pei
|
c550bfba32 |
Merged PR 1688875: [Fix] LastN and BottomN Accumulator
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Issue1: Our lastN has some errors. When taking the last N elements, the original order is destroyed. For example: for[31,33,99,1], if want to get last3, it should be [33,99,1], our current lastN will get [1,99,33]: left is native mongo, right one is our pgmongo  Fix: When there is no sort, the result is reversed in the transit function. When there is sort, the result is reversed in the final function. Issue2: BottomN also has similar errors as mentioned above. left is native mongo, right one is our pgmongo  #### AI description (iteration 1) #### PR Classification Bug fix addressing incorrect accumulation logic for the LastN accumulator. #### PR Summary This PR fixes issues in the LastN accumulator by streamlining memory copy operations and adding robust error checking for BSON data integrity, while also correcting a misconfigured boolean flag in related functions. - `oss/pg_documentdb/src/aggregation/bson_sorted_accumulator.c`: Simplified the copy routine by replacing conditional branches with a unified memmove and added error checks to detect and report insufficient or corrupted BSON entries. - `oss/pg_documentdb/src/aggregation/bson_first_last.c`: Updated the `isLast` flag from true to false in both transition and combine functions to ensure correct accumulator behavior. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Cornelius Boateng
|
bece6f35dc |
Merged PR 1715993: [engine][collation] Support for delete
### Does this PR have any customer impact? No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No
### Are you introducing any new config? If yes, do you have tests with and without them being set? No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
### Description
Support collation with delete queries.
Collation is applicable to all filter and sort operations.
```
{
delete: <collection>,
deletes: [
{
q : <query>,
limit : <integer>,
collation: <document>,
...
},
...
],
...
}
```
**Sharded collections**
- `deleteMany (limit = 0)`: If the shard key value is collation-sensitive , we distribute the query (by omitting filtering by shard_key_value filter).
- `deleteOne (limit = 1)`: If the shard key value is collation-sensitive, we check for an `_id` filter in the query. If there is an `_id` filter, we use that to find a shard key value for the actual delete operation. (we consider the collation sensitivity of the `_id` filter value also). If there is no `_id` filter value, we fail.
This is because our deleteOne on sharded collections currently targets only 1 shard.
**GUC**
To enable feature, turn on the EnableCollationForWriteCommands GUC.
----
#### AI description (iteration 1)
#### PR Classification
New feature: This pull request implements collation support for delete commands.
#### PR Summary
This pull request introduces case- and locale-aware deletions by adding a new collation parameter to delete operations. It updates the delete command logic, SQL generation, tests, and feature flags to enable collation with write commands.
- Updates in test files (e.g. in `oss/internal/pg_documentdb_distributed/src/test/regress/expected/bson_dollar_ops_collation_tests_runtime.out`, `/pgmongo/src/test/docdb_compat/expected/bson_dollar_ops_collation_tests_runtime.out`, and SQL tests) add scenarios for collation-aware delete operations.
- Changes in `oss/pg_documentdb/src/commands/delete.c` and related files incorporate a new collation string parameter, modify query generation, and adjust _id filtering based on collation sensitivity.
- Modifications in configuration (in `oss/pg_documentdb/src/configs/feature_flag_configs.c`) add the `EnableCollationForWriteCommands` flag to control collation support on write commands.
- Auxiliary updates in files such as `commands/commands_common.c`, `commands/update.c`, and aggregation modules extend collation checks to sorting and query matching.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Related work items: #3685457, #3922008, #4006283
|
||
 Zinan Ren
|
b11753f8d2 |
Merged PR 1712315: [oss][test cleanup] Vector search related cleanup
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) No ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Cleanup vector tests |
||
 Parag Jain
|
2dc64d2081 |
Merged PR 1711000: [oss] Cleanup as per new change guideline, part -1
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description 3 c files ---- #### AI description (iteration 1) #### PR Classification This pull request focuses on code cleanup and documentation updates. #### PR Summary The changes standardize and simplify documentation comments and examples across several operator implementations, removing obsolete references and unnecessary details. - `oss/pg_documentdb/src/aggregation/bson_aggregation_output_pipeline.c`: Updated doc examples for `$merge` and `$out` to use generic placeholders and removed temporary test-related comments. - `oss/pg_documentdb/src/query/bson_dollar_operators.c`: Cleaned up comments for multiple BSON dollar operators by eliminating "Mongo's" references and redundant test notes. - `oss/pg_documentdb_core/src/types/pcre_regex.c`: Refined regex compile option comments and removed legacy test reference lines. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Vinod Sridharan
|
ccc471e169 |
Merged PR 1703619: [Cursors] Improve stability of file based cursors
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description ---- #### AI description (iteration 1) #### PR Classification Enhancement and bug fix to improve the stability and management of file-based cursors. #### PR Summary This pull request refactors and bolsters file-based cursor management by introducing shared memory support, improved cleanup, error handling, and configuration changes. It also adds new API endpoints and logging for better cursor lifecycle control. - **`cursor_store.c` and related files:** Added shared memory functions (e.g., `FileCursorShmemSize`, `InitializeFileCursorShmem`), cursor count regulation (with `IncrementCursorCount`/`DecrementCursorCount`), and cleanup routines to ensure stable cursor file operations. - **Test and expected output files:** Updated regression tests and expected outputs to cover scenarios like duplicate cursor creation, cursor deletion limits, and error reporting when the file cursor capacity is reached. - **API enhancements:** Introduced new UDFs (e.g., `command_delete_cursors`) and tenant client methods for killing and deleting cursors, improving session management. - **Configuration updates:** Renamed and added GUC parameters (e.g., `MaxAllowedCursorIntermediateFileSizeMB`, `MaxCursorFileCount`, and cursor expiry settings) to control file-based cursor behavior. - **Logging and metrics improvements:** Enhanced logging in `mdsd_logger.c` to report current cursor counts and related metrics for operational insight. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Vinod Sridharan
|
c6bc2bcb73 |
Merged PR 1694458: [POC|Order by] Order by Index & Pushdown improvements
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description More general improvements to order by and composite: Create Indexes 1) Support creating a composite index side by side with a regular index 2) Block creation of descending or wildcard and unsupported set ups for composite indexes Planner: 1) Support ordering by the Nth path of an index if the prefixes are equality. 2) Validate that if it's not the first column, prefixes are provided 3) Support incremental sort with index pushdown. ---- #### AI description (iteration 1) #### PR Classification This pull request delivers a new feature and API enhancement to enable order-by index pushdown and refined composite index support. #### PR Summary The update streamlines composite index processing by introducing stricter equivalency checks, new UDF functions for full-scan order-by pushdown, and enhanced validations during index creation while updating explain outputs and tests. - `src/metadata/index.c` refines index options equivalency logic to correctly compare composite term settings. - `src/opclass/bson_gin_composite_entrypoint.c` and `src/index_am/rum.c` add functions to support order-by pushdown and manage duplicate entries efficiently. - `sql/udfs/query/bson_dollar_comparison--0.105-0.sql` (and related operator files) introduce new UDFs including `bson_dollar_fullscan` to enable full scan operations. - `src/commands/create_indexes.c` enforces new composite index validations (restricting use to ascending, non-wildcard regular indexes) with precise error reporting. - Test scripts and explain plan outputs in the `regress` and `pgmongo` directories are updated to confirm the new composite order-by behavior. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 Shuai Tian
|
17d4306a28 |
Merged PR 1422337: [pgmongo] Aggregate stage $bucketAuto implementation
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description This pull request adds documentation for the `$bucketAuto` aggregation stage in PostgreSQL, detailing its design, usage, and implementation. - Adds `bucketAuto.md` with details on the `$bucketAuto` aggregation stage - code implementation - support all granularity types: POWERSOF2, 1-2-5, Renard numbers, E series Related work items: #3234185 |
||
 Santiago Fernandez Madero
|
0d616cdcde |
Merged PR 1679451: Make first page cursor batch size configurable via GUC
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Make first page cursor batch size configurable via GUC ---- #### AI description (iteration 1) #### PR Classification This pull request implements a new feature by introducing a configurable GUC for the first page cursor batch size. #### PR Summary The changes enable users to control the default batch size for the first page of cursors through a custom GUC, ensuring it respects underlying limitations. - `src/configs/system_configs.c`: Added a custom integer variable (`defaultCursorFirstPageBatchSize`) with a default value and proper bounds via the GUC interface. - `src/commands/aggregation_commands.c`: Updated the first page query data generation to use the new configurable variable instead of a hardcoded value. - Test files under `/oss/pg_documentdb/src/test/regress/`: Extended both SQL and expected output tests to verify the behavior of the GUC under various configurations. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4225119 |
||
|
Vinod Sridharan
|
8261e772f4 |
Merged PR 1672946: [Update] Make update parsing cheaper
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Currently for Update/Delete, we currently parse the update spec, query spec, etc into pgbsons on startup. Then reuse it per commands. This has a few issues - for large document requests, this causes us to duplicate memory and persist it for the duration of the update and also transcode all up front - this is worse for multi-node when all we need to do is build yet another update spec and push it to the worker. Instead, we now only cache pointers to the spec document; then when we need it we convert to pgbson only if necessary. A consequence of this is that on a single node update, if we're doing a simple updateOne on _id then we would never serialize the updateSpec at all (for instance). We also more eagerly free memory on the intermediate update specs. ---- #### AI description (iteration 1) #### PR Classification Performance optimization for update parsing by refactoring BSON value handling. #### PR Summary This PR streamlines update parsing by replacing heavy BSON initialization routines with lightweight wrappers and converting update-related types from `pgbson*` to `bson_value_t*`, which reduces overhead and improves coordinator/worker compatibility. - `src/commands/update.c`: Replaced multiple `PgbsonInitFromDocumentBsonValue` and similar calls with efficient wrappers like `CreateBsonValueFrom` and updated serialization (including back-compatible array filter and update document formats). - `include/commands/update.h`: Updated update structure fields to use `bson_value_t*` instead of `pgbson*`, aligning interfaces with the new parsing approach. - `src/update/bson_update.c` and `bson_update_operators_workflow.c`: Refactored update type determination and replace document logic to use new BSON value iterators and helper functions. - Other modules (e.g. `commands/find_and_modify.c`, `commands/delete.c`, query operators, sharding) were updated to leverage the lighter-weight BSON representation and its associated helper routines. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Deepak Bisht
|
8b7b93ced7 |
Merged PR 1578129: [Pending release cut 1.102]- Fix projection with bson_dollar_merge_documents for multiple scrnarios
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) BugFix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) Bugfix to override nested arrays in dotted path in `bson_dollar_merge_documents` UDF for few cases ### Description bson_dollar_merge_documents function today recurses into nested arrays always and project in each element of array but that is not correct with mongo semantics. This is only applicable if the projectetion is a dotted path. The function is used in 7 different stages/features today: 1- list_collection: no dotted paths for projection, so this is okay to follow any projection. 2- graphlookup: Mongo does a shallow projection in this case. 3- lookup: Mongo does a shallow projection in this case. 4- inverseMatch: okay as no dotted path projection 5- count: okay as no dotted path projection 6- group : okay as no dotted paths are allowed 7- setWindowFields: only this stage does a full projection even inside nested arrays ---- #### AI description (iteration 1) #### PR Classification Bug fix #### PR Summary This pull request addresses multiple scenarios for fixing projection with `bson_dollar_merge_documents`. - Added new functions in `oss/pg_documentdb/sql/udfs/projection/bson_projection--0.102-0.sql` to handle various BSON document operations. - Updated `bson_dollar_merge_documents` function to include an `overrideArray` parameter. - Modified test expectations in `oss/internal/pg_documentdb_distributed/src/test/regress/expected/bson_aggregation_pipeline_tests_graphlookup.out` to reflect changes in the `bson_dollar_merge_documents` function. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Vinod Sridharan
|
1153e6b6ac |
Merged PR 1661728: [Performance] Read & Query Perf Wave 2
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description 1) In the Gateway, when preparing queries, embed the commandKey and userOid. That way a given prepared query is unique for a given user and command. This can be used for a future optimization to cache ACL check results 2) Reduce serialization costs from GW -> BE: Don't send the database name in the query. This is already in the request as a '$db' field. Support suppressing this (add a GUC to rollback just in case) 3) In the backend start parsing the $db if it's not sent by the GW 4) From the commandFuncs -> Query we were going through a layer of fmgr. We don't need this. Instead call directly into the functions in C to avoid the fmgr overhead. Additionally for GetMore this means we don't need to repack the heap tuple. As part of this we discovered that registering functions in pg_dist_object can regress perf for dispatch functions. Add a test to validate which ones of these are public. ---- #### AI description (iteration 1) #### PR Classification Performance improvement. #### PR Summary This pull request focuses on enhancing the performance of read and query operations by optimizing function calls and parameter handling. - Refactored `ExecuteQueryCore` in `command_funcs.c` to improve query execution by directly calling specific functions and adding ACL checks. - Changed function signatures in `bson_aggregation_pipeline.h` to use `text` instead of `Datum` for database parameters, improving type consistency. - Added `aggregation_commands.h` to export new aggregation command functions. - Updated `PostgresClient.cs` to conditionally include database names in query parameters, enhancing query flexibility. - Introduced new configuration option `SendDatabaseNameInParams` in `PostgresTenantConfiguration.cs` to control database name parameterization. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Cornelius Boateng
|
f6d20fbf8a |
Merged PR 1646920: [engine][collation] Support for Arrays and Documents
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Support collation with comparison of `array` and `document` values. Also adds support for `$in` on nested arrays and documents. ---- #### AI description (iteration 1) #### PR Classification New feature: Adds support for collation-aware hashing of arrays and documents in BSON. #### PR Summary This pull request introduces collation-aware hashing for BSON arrays and documents, enhancing the handling of collation in hash sets. It refactors existing hash functions to accommodate collation strings and recursive hashing of nested BSON structures. - `hashset_utils.c`: Implements collation-aware hashing functions for BSON values, including recursive handling of arrays and documents. - `bson_expression_set_operators.c`: Replaces existing hash set creation with collation-aware hash set creation. - `bson_dollar_operators.c`: Updates logic to use collation-aware hash sets for `$in` queries, removing unsupported scenarios. - `hashset_utils.h`: Adds a new struct `BsonValueHashEntry` to store BSON values with collation strings. - `collation.h`: Moves `IsBsonTypeCollationAware` function to handle collation applicability checks. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4111861, #4115239 |
||
|
Cornelius Boateng
|
8ea9769598 |
Merged PR 1639388: [engine][collation] Unsupported Aggregation Stages
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Aggregation stages which do not support collation yet: `bucket, group, setWindowFields, fill, geoNear, sortByCount` If collation is provided for an aggregation whose pipeline has any of these stages, the aggregation fails with `FEATURE_NOT_SUPPORTED` ---- #### AI description (iteration 1) #### PR Classification Code enhancement to handle unsupported aggregation stages with collation. #### PR Summary This pull request adds error handling for unsupported aggregation stages when collation is applied. - `bson_aggregation_pipeline.c`, `bson_aggregation_nested_pipeline.c`, `bson_aggregation_window_operators.c`: Added checks and error messages for unsupported collation in stages like `$bucket`, `$replaceWith`, `$geoNear`, `$fill`, `$group`, `$documents`, `$setWindowFields`, and `$sortByCount`. - `bson_dollar_ops_collation_tests_runtime.sql`: Added test cases for unsupported aggregation stages with collation. - `bson_dollar_ops_collation_tests_runtime.out`: Updated expected output to include error messages for unsupported collation in various stages. - `bson_query.h`: Refactored `IsBsonTypeCollationAware` function for better readability. - `bson_dollar_operators.c`: Simplified collation applicability check using `IsBsonTypeCollationAware`. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4022806 |
||
|
Cornelius Boateng
|
4da81c67e6 |
Merged PR 1588632: [engine][collation] Support for Aggregation, Find, Query Match on Sharded Collections
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Support collation for aggregation and query match queries on sharded collections. If the `shard key filter` is collation-aware, we make the query distributed and push it to all shards. ---- #### AI description (iteration 1) #### PR Classification New feature: Collation support for aggregation and query match on sharded collections. #### PR Summary This pull request adds collation support for aggregation and query match operations on sharded collections. - `bson_dollar_ops_collation_tests_runtime.sql`: Added tests for collation-aware shard key queries. - `sharding.c`: Introduced logic to track collation-aware shard key fields and updated related functions. - `query_operator.c`: Modified to include shard key filters only if they are not collation-sensitive. - `bson_aggregation_pipeline.c`: Added collation-aware checks for shard key filters in aggregation pipelines. - `bson_query.h`: Added utility function `IsBsonTypeCollationAware` to check if BSON types are collation-aware. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #3915171, #3922023 |
||
|
Vinod Sridharan
|
5ad889feeb |
Merged PR 1629302: [Composite POC] Fix up operators and type bracketing
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description 1) Fix up the remaining operators - implementations are sub-optimal 2) Add type bracketing options for all types and use them in operators 3) Add support for truncation of terms TODO: - Support recheck on the index for operators than just bounds - Support N boundaries for index terms. ---- #### AI description (iteration 1) #### PR Classification Bug fix #### PR Summary This pull request addresses issues with operators and type bracketing in BSON composite indexes. - Added `bson_gin_index_types_core.c` to manage BSON type bounds for indexes. - Updated `bson_gin_composite_entrypoint.c` to fix operator handling and apply type bracketing for various BSON index strategies. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Cornelius Boateng
|
90c92c44ec |
Merged PR 1591616: [engine][collation] Support for project
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description more ---- #### AI description (iteration 1) #### PR Classification New feature #### PR Summary This pull request adds support for collation in BSON projection functions. - Added new SQL functions in `sql/udfs/projection/bson_projection--0.102-0.sql` to handle collation. - Updated `src/aggregation/bson_project.c` to include collation string handling in BSON projection. - Modified `src/aggregation/bson_aggregation_pipeline.c` to support collation in projection stages. - Enhanced `src/query/bson_dollar_operators.c` to populate collation string in query state. - Added collation-related tests in `sql/bson_dollar_ops_collation_tests_runtime.sql`. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #3918941 |
||
|
Deepak Bisht
|
83e22d92f3 |
Merged PR 1573978: Improve Let with Lookup
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) Improve Perf in $lookup with let stage with nested join pipeline ### Description Today the pipeline `$match` stage is not inlined with lookup right collection query in case it has `$expr` and let expression. Move this to right query if `$match` is the first non-inlined stage. This improves the JOIN criteria and fetches lesser documents from right collection. We have seen cx cases where very small set of docucments can breach the 100 MB document limit otherwise. ---- #### AI description (iteration 1) #### PR Classification Code optimization and cleanup. #### PR Summary This pull request improves the handling of `$let` with `$lookup` in the aggregation pipeline by simplifying and optimizing the code. - Removed the `QueryDocumentHasExpr` function and simplified the `CanInlineLookupStageMatch` function in `bson_aggregation_pipeline.c`. - Enhanced the `OptimizeLookup` function to build and use `$let` expressions more efficiently in `bson_aggregation_nested_pipeline.c`. - Simplified the `ProcessLookupCoreWithLet` function to directly use variables from the left query in `bson_aggregation_nested_pipeline.c`. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Cornelius Boateng
|
b0289c3bf3 |
Merged PR 1518085: [engine] $$NOW in Aggregate and Find with Cursor Support
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description Store the value of $$NOW in the variable spec as `now`. Pass it through the ParseAggregationContext to be utilized by operators that implement the new aggregation framework. Persist value in the cursor's continuation document to be used for get_more queries. Enable/Disable with `EnableNowSystemVariable` GUC. ---- #### AI description (iteration 1) #### PR Classification New feature #### PR Summary This pull request introduces the `NOW_SYS_VARIABLE` to the BSON aggregation pipeline, allowing the use of the current timestamp in aggregation operations. - Modified `bson_aggregation_pipeline_tests_add_to_set_group.out` to include `NOW_SYS_VARIABLE` in the BSON expressions for various aggregation operations. Related work items: #3613264, #3613268 |
||
|
Deepak Bisht
|
6f293a05f2 |
Merged PR 1553295: Lookup and unwind optimization Part 2
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) NA ### Description - support preserveNullAndEmptyArrays of unwind for lookup optimization with LEFT JOIN and COALESCE expression while merging so that we don't end up with NULL values for rows that don't have matching rows ---- #### AI description (iteration 1) #### PR Classification Code optimization #### PR Summary This pull request optimizes the lookup and unwind stages in the aggregation pipeline by allowing certain lookups to be inlined with unwind stages. - `bson_aggregation_nested_pipeline.c`: Added logic to check if lookup stage can be inlined with unwind stage, including handling of various unwind options. - `bson_aggregation_stage_lookup_tests.out`: Added new test cases to verify the inlining behavior of lookup and unwind stages. |
||
 Xing Fan
|
722ef60784 |
Merged PR 1479858: $percentile & $median for $group accumulator
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description draft ---- #### AI description (iteration 1) #### PR Classification New feature #### PR Summary This pull request introduces new aggregation functions and support for t-digest in the project. - Added `bson_tdigest.c` to implement t-digest helper functions. - Created new SQL aggregate functions in `group_aggregates--1.23-0.sql` and `group_aggregates_support--1.23-0.sql`. - Updated `bson_aggregation_pipeline.c` to handle the new `$percentile` accumulator. - Modified `metadata_cache.c` and `metadata_cache.h` to include OID for the new `BsonPercentileAggregateFunction`. - Updated migration scripts `pg_helio_api--1.22-0--1.23-0.sql` and `pgmongo--1.22-0--1.23-0.sql` to include new SQL files. |
||
|
Shuai Tian
|
e3dc6c889e |
Merged PR 1404676: stage implementation: redact
### Does this PR have any customer impact? yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes, UDF ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description Implement for stage redact Related work items: #1798586 |
||
|
Deepak Bisht
|
7a674be896 |
Merged PR 1542122: [Part 1] Improve $lookup and $unwind aggregation stage combination when the lookup options are simple
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) Aggregation performance pipeline optimization for $lookup and $unwind stage combination on same fields. ### Description 1. Optimize $lookup and $unwind stage combination on same field and lookup options are simple `localField` and `foreignField` 2. Delete dead code for $lookup ---- #### AI description (iteration 1) #### PR Classification Code cleanup and optimization. #### PR Summary This pull request improves the combination of `$lookup` and `$unwind` aggregation stages by removing redundant and complex code, simplifying the logic, and optimizing performance. - Removed the `ProcessLookupCore` function and its associated logic from `bson_aggregation_nested_pipeline.c`. - Removed the `UnwindWalkerContext` struct and related functions for optimizing `$unwind` with `ARRAY_AGG` from `bson_aggregation_nested_pipeline.c`. |
||
|
Vinod Sridharan
|
02b0c3af75 |
Merged PR 1547234: [Infra] Command timeout improvements: Move command timeouts for common commands to the backend
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description The Gateway does a convoluted processing for maxTimeMS including having a second connection pool and creating transaction wrappers. Instead, this is simpler if this is moved into the backend and let the backend set custom timeouts directly. This part just codes up the backend piece and puts it behind a GUC. The Gateway changes will be made separately. ---- #### AI description (iteration 1) #### PR Classification Code improvement #### PR Summary This pull request improves command timeout handling by moving common command timeouts to the backend. - Added new test files `isolation-command_timeout.out` and `isolation-command_timeout.spec` to validate command timeout scenarios. - Modified `bson_aggregation_pipeline.c` to remove the `TryHandleSimplifyAggregationRequest` function. - Updated function signatures in `bson_aggregation_pipeline.h` to include `setStatementTimeout` parameter. - Adjusted `documents_planner.c` to handle the new `setStatementTimeout` parameter in various query generation functions. |
||
|
Vinod Sridharan
|
0cc46eca57 |
Merged PR 1542496: [Perf] Custom planner for point read
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description Improve performance for a point read set up for unsharded collections. In the scenario where we have a point read in a local unsharded collection with colocation, we can bypass a lot of the planner and go directly to the PK index. Use this to simplify the planner. With this change, point reads on YCSB are able to hit ~16k on Marlin and ~17.5-18k on flex ---- #### AI description (iteration 1) #### PR Classification New feature #### PR Summary Introduces a custom planner for point read operations to optimize query performance. - Added `documents_custom_planner.c` to implement the custom point read planner. - Updated `aggregation_commands.c` to handle the new point read cursor type. - Modified `planner.c` to integrate the custom point read planner and expand aggregation functions. |
||
|
Deepak Bisht
|
c136836096 |
Merged PR 1537692: [Performance] Push Aggregates directly to shard for unsharded tables in single node
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) - Push Aggregation queries directly to shard if the collection is unsharded in a single node and pipeline stages refer only 1 collection ### Description Push Aggregation queries directly to shard if the collection is unsharded in a single node and pipeline stages refer only 1 collection ---- #### AI description (iteration 1) #### PR Classification Performance improvement #### PR Summary This pull request optimizes the aggregation pipeline by pushing aggregates directly to shards for unsharded tables in a single node setup. - Added tests in `bson_aggregation_pipeline_tests.out` to verify the new optimization for unsharded collections. - Updated `bson_aggregation_pipeline_tests.out` in both `pg_helio_distributed` and `helio_compat` to include new test cases and EXPLAIN plans demonstrating the performance improvements. |
||
|
Zinan Ren
|
8e7bd960bf |
Merged PR 1540262: [OSS] Remove helio references from sources of documentdb_core
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Refactoring ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description Remove helio from file name Remove helio references from source content except schema reference Start local documentdb server with the documentdb extension Regression tests using the documentdb extension ---- #### AI description (iteration 1) #### PR Classification Code cleanup #### PR Summary This pull request removes references to "helio" from the `documentdb_core` source code. - `scripts/utils.sh`: Renamed variables from `preloadExtensions` to `preloadLibraries`. - `pg_documentdb_core/include/utils/error_utils.h`: Updated function and macro names to replace "Helio" with "DocumentDB". - `pg_documentdb_core/include/io/helio_bson_core.h` and `pg_documentdb_core/include/query/helio_bson_compare.h`: Moved to `pg_documentdb_core/include/io/bson_core.h` and `pg_documentdb_core/include/query/bson_compare.h`. - `pg_documentdb_core/src/collation/collation.c` and `pg_documentdb_core/src/io/pgbson.c`: Replaced error codes and messages to use "DocumentDB" instead of "Helio". - `scripts/validate_helio_extension_state.sh`: Added exceptions for certain files to skip validation for "helio" references. |
||
|
Shuai Tian
|
19f20b4efb |
Merged PR 1540255: [OSS] remove dependencies on helio errors
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description [OSS] remove dependencies on helio errors |
||
|
Vinod Sridharan
|
3f9a507562 |
Merged PR 1539950: [OSS] Move helio_api and helio_distributed sources under documentdb
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Refactor ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description Move source files under new directory. ---- #### AI description (iteration 1) #### PR Classification Code reorganization #### PR Summary This pull request moves the `helio_api` and `helio_distributed` sources under the `documentdb` directory, updating references and initialization logic accordingly. - Added `pg_documentdb.c` for initializing the `pg_documentdb` extension. - Added `documentdb_distributed.c` for initializing the `pg_documentdb_distributed` extension. - Updated `validate_helio_extension_state.sh` to include new checks for `documentdb`. - Moved `helio_api_extension_version.c` to `documentdb_extension_version.c` without code changes. - Updated various `Makefile` configurations to reflect the new directory structure and dependencies. |
