Commit Graph
911 Commits
Author SHA1 Message Date
Jinrui Zhang a04ec8ea00 Merged PR 1713380: [oss][test cleanup] test check for regex related case
### Description
[oss][test cleanup] test check for regex related case

----
#### AI description  (iteration 1)
#### PR Classification
This pull request performs test cleanup by standardizing regex test cases.

#### PR Summary
The PR refactors multiple regex test files to improve consistency by renaming JSON keys and updating test string values and patterns.
- Files like `oss/internal/pg_documentdb_distributed/src/test/regress/expected/regex_options_index.out` and `.../sql/regex_options_core.sql` now use the key `msg` with updated regex patterns in place of the old key.
- Files such as `oss/internal/pg_documentdb_distributed/src/test/regress/expected/regex.out` (and the corresponding `/pgmongo` versions) change the key from `sku` to `b` and adjust description texts and Unicode values.
- SQL scripts in `.../sql/regex.sql` are updated accordingly to reflect these key renames and ensure accurate index creation and count validations.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Vinod Sridharan c214197c0e Merged PR 1723877: [Feature Flag Cleanup] Clean up Feature Flags + Obsolete Version checks
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Now that 105 is done, delete any version checks < 102 and remove obsolete feature flags

----
#### AI description  (iteration 1)
#### PR Classification
This PR performs code cleanup by removing obsolete feature flag checks and legacy version conditionals.

#### PR Summary
The changes simplify function implementations and configuration management by eliminating outdated feature flags and redundant version checks.
- **`/oss/pg_documentdb/src/metadata/metadata_cache.c` and aggregation files**: Removed conditional branches based on cluster versions (e.g., always using 5 parameters) to simplify BSON merge and projection functions.
- **`/oss/pg_documentdb/src/configs/feature_flag_configs.c` and related config files**: Cleaned up obsolete feature flag variables and custom variable definitions (such as those for index term truncation, index operator bounds, and RUM fast path).
- **Test scripts and expected output files (under `/oss/internal/pg_documentdb_distributed` and `/pgmongo`)**: Updated query plan expectations and removed toggles for index term truncation, reflecting the new default behavior.
- **Other code areas (e.g., in `query_operator.c`, `opclass/gin_index_term.c`, and `oid_cache.c`)**: Eliminated outdated version checks and conditional logic, resulting in cleaner and more consistent code paths.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Deepak Pemmaraju 5b892aec1a Merged PR 1722262: Fixing error when roles are not passed in an array
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix to enforce that the "roles" field is passed as an array.

#### PR Summary
This pull request adds a validation to ensure the "roles" field is an array during user creation, throwing a specific error if it is not, and updates the corresponding test cases to verify this behavior.
- `oss/pg_documentdb/src/commands/users.c`: Added a check to throw an error when the "roles" field is not an array.
- `oss/pg_documentdb/src/test/regress/sql/user_crud_commands.sql` and `/expected/user_crud_commands.out`: Updated tests to expect the error message on invalid "roles" input.
- `pgmongo/src/test/regress/sql/user_crud_commands.sql` and `/expected/user_crud_commands.out`: Applied similar test updates to enforce the array requirement for the "roles" field.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Ritvik Jayaswal 87d61f6cc6 Merged PR 1665978: Added RequestContext to pass through requests
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

We discussed the option to:

have a RequestContext object to passed down to each function and sub functions which is
RequestContext
{
    header (this is the mongo wire protocol header containing length, request id, response to, op code
    activity_id (a guid we generate upon receiving the request, used for logging and eventing)
    connection_context (a back reference to the owner)
    request_info (info about request including a tracker)
}

Right now, we have connection context and header sent down to the api, otherwise we'll only have 1 param
also right now, we stick activity id in the header which sort of belongs to Mongo.

----
#### AI description  (iteration 1)
#### PR Classification
This PR implements a refactoring change by introducing a new RequestContext to streamline request handling and parameter propagation.

#### PR Summary
The update adds a RequestContext that encapsulates the header, a generated activity_id, and the connection_context, and integrates it across key processing flows to improve request tracking.
- `src/context/request.rs`: New file that defines RequestContext along with its constructor and atomic connection_id generation.
- `src/auth.rs`: Updates the authentication process to accept a header and create a RequestContext for tracking requests.
- `src/processor/process.rs`: Refactors process_request to receive a RequestContext and extract the connection_context from it.
- `src/lib.rs` and `src/context/mod.rs`: Adjusts module exports and function calls to integrate RequestContext into the overall processing pipeline.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #4034215, #4086147
2025-07-09 18:56:11 +08:00
Vinod Sridharan 497ccc1e24 Merged PR 1722322: [RUM] Fix rum crash during data split Page
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Fix rum crash

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix that resolves crashes in rum index operations.

#### PR Summary
This pull request fixes a crash by enhancing error handling and implementing recovery logic when page splits occur during index scans. The changes improve robustness and stability during query operations.
- `oss/pg_documentdb/src/commands/commands_common.c`: Expanded error handling to detect both "Lost Path" and "invalid offset on rumpage" errors, updating error codes and messages before throwing errors.
- `pgmongo_rum/src/rumutil.c`: Added two custom boolean configuration variables to control refinding leaf pages and throwing errors on invalid data pages.
- `pgmongo_rum/src/rumget.c`: Inserted a loop in `entryGetNextItem` to refind the correct leaf page when a page is split.
- `pgmongo_rum/src/rum.h`: Integrated a check for invalid offsets with error logging and defined new macros and extern declarations for the additional configuration options.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Vinod Sridharan a6aeba7be3 Merged PR 1722022: [OSS] Introduce documentdb_rum under the internal folder
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Introduce documentdb_rum which is a fork of the OSS Rum index that will be used for advanced documentdb scenarios.

This change:
1) introduces the access method and forks the code (as-is) makes it build in the repo.
2) Removes any tsvector/tsquery, arrayutils/rum btree  logic that was there before
3) Deletes custom logic that was built for PG9.6 -> pg14
4) Trims the SQL file to only have hte Access method.

This change does not wire up the RUM index with documentdb or provide the modifications needed for the documentdb scenarios (yet). This will be done in subsequent changes.
2025-07-09 18:56:11 +08:00
Juan Carlos Aguilera Mendez 5428e5e4ea Merged PR 1720111: Move error to the correct file
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Move Location13103 error to documentdb_codes.txt

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix: Moving an error code definition to the correct file.

#### PR Summary
This pull request corrects the placement of the "Location13103" error by removing it from one file and adding it into the proper location.
- `pg_documentdb_gw/include/gateway_errors.txt`: Removed the "Location13103" error entry.
- `pg_documentdb_core/include/utils/documentdb_codes.txt`: Added the "Location13103" error entry.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
William Wang bd1cb2d295 Merged PR 1700856: [OSS] [Engine] Adding Support for Disabling Native Authentication
### Does this PR have any customer impact?
Yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
GUC
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
GUC
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
1.05
### Description
This PR adds a GUC, pgmongo.EnableNativeAuthentication, and code that will block customer requests if 1.) they are native, and 2.) native authentication is disabled.

----
#### AI description  (iteration 1)
#### PR Classification
New feature implementation to add a configurable toggle for native authentication.

#### PR Summary
This pull request introduces support for disabling native authentication by adding a new GUC variable and enforcing checks in user management commands.
- `/pgmongo/src/pgmongo.c`: Defines the custom GUC `pgmongo.enableNativeAuthentication` and initializes the `EnableNativeAuthentication` boolean with its default value.
- `/oss/pg_documentdb/src/commands/users.c`: Integrates checks in user creation, update, drop, and retrieval functions to error out when native authentication is disabled for internal users.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #3978592, #4311471
2025-07-09 18:56:11 +08:00
William Wang 7f8c4a7e9e Merged PR 1708526: Adding Support for Connection Status Command in Rust Gateway
### Does this PR have any customer impact?
Yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
None
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
105
### Description
We developed connection_status for the backend, and are adding it as a feature in our Rust GW.

----
#### AI description  (iteration 1)
#### PR Classification
New feature implementation adding connection status support to the Rust Gateway.

#### PR Summary
This pull request enables detailed connection status queries by introducing a new execution path and updating processing logic, query catalogs, and test coverage.
- `tests/operation_coverage/connection_status_tests.rs`: Added comprehensive integration tests for connection status with different user roles and privilege settings.
- `src/postgres/data_client.rs`: Introduced the async `execute_connection_status` method to handle connection status queries.
- `src/processor/process.rs` and `src/processor/users.rs`: Refactored connection status request handling to delegate to a new async function.
- `src/postgres/query_catalog.rs` and `gateway/native/src/query_catalog.rs`: Updated the query catalog to include the connection status SQL query.
- `gateway/native/tests/common/mod.rs`: Added the `get_client_with_credentials` helper to support credential-based testing.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #4344671
2025-07-09 18:56:11 +08:00
Deepak Bisht 19c65dd504 Merged PR 1712186: [SQL] update coments for all SQL upgrade files
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Comments

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

All the previous version SQL changes should be merged into this PR so that we can update the hashes in this PR

----
#### AI description  (iteration 1)
#### PR Classification
This PR is a documentation update that cleans up inline SQL comments to align with the latest contributing guidelines.

#### PR Summary
The changes refine the SQL file comments for geospatial operator classes and functions by removing outdated references to Mongo indexing and validations.
- `oss/pg_documentdb/sql/operators/bson_gist_geospatial_op_classes--0.16-0.sql`: Updated comments by replacing "mongo 2d index options" and "mongo 2dsphjere index options" with simplified descriptions.
- `oss/pg_documentdb/sql/udfs/geospatial/bson_gist_extensibility_functions--0.16-0.sql` and `oss/pg_documentdb/sql/udfs/geospatial/bson_gist_extensibility_functions--latest.sql`: Revised function comments to remove "mongo db validations" for a more general description of geometry validations.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #4354111
2025-07-09 18:56:11 +08:00
Yang Qiao dbd318023b Merged PR 1713343: [oss][test cleanup] cleanup of or9
### Does this PR have any customer impact?
NO
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

clean up
2025-07-09 18:56:11 +08:00
Parag Jain d1e2446c40 Merged PR 1707681: [perf] Insert custom planner
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Perf
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Build Custom Plan for single insert for shard table so we skip PG planning phase and directly execute query insert query.

this change improves 500 TPS majored with pgbench and we are not matching native PG for single node , insert one cases
1. created a local PG table same as our data table (even for that I haven't created btree index so we expect operation to be faster)  ran the Pgbench using INSERT INTO command
2. ran the pgbench with our insert() udf
![image (2).png](https://msdata.visualstudio.com/ba574a88-a171-48e0-8fcb-5fef6d23739c/_apis/git/repositories/501a5e7e-ffdc-4d1e-a5e6-910f73c6eb9a/pullRequests/1707681/attachments/image%20%282%29.png)
2025-07-09 18:56:11 +08:00
Andrew Khoma 59e8a0140d Merged PR 1718849: [Gateway][Rust] remove unused Rust deps
### Does this PR have any customer impact?
no
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
cleanup
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
no
### Are you introducing any new config? If yes, do you have tests with and without them being set?
no
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
no
### Description
remove unused Rust deps
2025-07-09 18:56:11 +08:00
CosmosDB Build Service (msdata) 7c62c43e51 Merged PR 1720001: Update extension version for new release 1.106
This PR updates the extension version for the new release 0.106.

Changes included:
- Control files updated to new version
- SQL files generated for version upgrades
- CHANGELOG.md updated with new version entry
2025-07-09 18:56:11 +08:00
Juan Carlos Aguilera Mendez 02bfca5839 Merged PR 1713127: Provide the correct codeName on error and codegen for error codes and mapping
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Added code generation to create all error codes off of shared csv definition files as well as the means to map an i64 to an error code. Also, if we have a known error code, fetch the proper error string, if exists, or None

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix to correctly convert error codes to text strings.

#### PR Summary
This pull request fixes the misassignment of error code and message, ensuring that error codes are properly converted to their text representation.
- `src/responses/error.rs`: Reordered parameters in the `CommandError::new` call to swap the error code and message.
- `src/responses/pg.rs`: Updated error handling to use `ErrorCode::from_i64` for converting error codes to text and added a dependency on `num_traits::FromPrimitive`.
- `src/error.rs`: Added the `IndexNotFound` error variant with corresponding mapping in the `FromPrimitive` implementation.
- `tests/test_search_tests.rs`: Introduced a test to validate that a text query fails with the expected "text index required for $text query" error message.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #4293767
v0.105-0
2025-07-09 18:56:11 +08:00
Duan Pei d74e1c6580 Merged PR 1713341: [oss][test cleanup]graphlookup stage
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

graphlookup stage .sql .out
2025-07-09 18:56:11 +08:00
Shuai Tian e8b8f070c0 Merged PR 1713374: [oss][test cleanup] update tests for inverseMatch, lookup, setWindowFields
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Test cleanup
2025-07-09 18:56:11 +08:00
Vinod Sridharan 528b744a4f Merged PR 1717610: [Selectivity] Make Function Selectivity Match Opr Selectivity for Index
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
For Operator Selectivity, we have a feature flag and function that does better operator selecitivty for operator expressions. However, when doing planning, the query operator generates mostly FuncExprs for index operators. This means the selectivity logic for funcitons defaults to 0.3333 even if opr selectivity can do better.

For Functions, when opr selectivity is enabled, use the same logic to determine selectivity enabling better index pushdowns.

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements a new feature to align function selectivity with operator selectivity for index operations in the DocumentDB Postgres extension.

#### PR Summary
The PR refactors selectivity calculations to ensure consistency between function and operator estimations, updates associated test cases, and introduces new configuration flags. Key changes include:
- **`pg_documentdb/src/opclass/index_support.c`**: Revised the `dollar_support` function to route selectivity requests correctly using a new pointer variable and support for selectivity mode.
- **`pg_documentdb/src/query/bson_dollar_selectivity.c` and `include/query/bson_dollar_selectivity.h`**: Updated the selectivity computation routines to accept a default selectivity parameter and adjust calculations based on the new operator selectivity mode.
- **`configs/feature_flag_configs.c` and `system_configs.c`**: Added new configuration flags (`DisableDollarSupportFuncSelectivity` and `EnableNewOperatorSelectivityMode`) and adjusted default values to control the new selectivity behavior.
- **Test files in `/oss/internal/pg_documentdb_distributed/src/test/regress/` and `/pgmongo/src/test/docdb_compat/expected/`**: Extended and modified SQL regression tests to verify expected index query plans and selectivity estimates.
- **Configuration scripts (`/scripts/utils.sh` and related test configs)**: Made minor adjustments such as reducing the index build schedule interval to better support the new functionality.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Jinrui Zhang 8aed4f088f Merged PR 1712457: [oss][test cleanup] finish test cleanup for insert_basic_types
[oss][test cleanup] finish test cleanup for insert_basic_types

----
#### AI description  (iteration 1)
#### PR Classification
Test cleanup.

#### PR Summary
This pull request finalizes the test cleanup for basic type inserts by updating the expected outputs to reflect new test data.
- In `pgmongo/src/test/docdb_compat/expected/bson_insert_basic_types.out`, the expected string, binary, and array values are updated (e.g., the string now reads "Today is a very good day and I am happy." instead of "A quick brown fox jumps over the lazy dog.", and the binary/base64 payload and array object contents are revised).
- In `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/bson_insert_basic_types.out` and `/oss/internal/pg_documentdb_distributed/src/test/regress/sql/bson_insert_basic_types.sql`, the expected outputs in the insert queries and BSON hex representations are modified to match the new test data.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Andrew Khoma dcc902c5d7 Merged PR 1714932: [Gateway][Rust] Enable the retry logic for objects that depend on postgres availbility
### Does this PR have any customer impact?
no
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
bugfix
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
no
### Are you introducing any new config? If yes, do you have tests with and without them being set?
no
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
- Enable the retry logic on startup for DynamicConfiguration and ConnectionPool
- Cleanup the ConnectionPool creation
- separate the startup functions from the lib.rs functions in a separate startup.rs file
- cleanup some test configs

Related work items: #4307229
2025-07-09 18:56:11 +08:00
Zinan Ren 8fc773b01b Merged PR 1713435: [oss] Test cleanup to follow code guidelines
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Cleanup tests for index creation
2025-07-09 18:56:11 +08:00
Vinod Sridharan 4eb4dcfbcc Merged PR 1716351: [$lookup] Re-enable lookup left join and make inner join behind a GUC
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix/Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Re-enable lookup left join and make inner join behind a GUC

----
#### AI description  (iteration 1)
#### PR Classification
This PR re-enables lookup left join functionality while making inner join behavior optional via a new configuration flag.

#### PR Summary
The pull request refactors lookup join processing so that by default the system uses left join (adding a null check filter) while inner join is enabled only when the new GUC is turned on. Key changes include:
- **`pg_documentdb/src/aggregation/bson_aggregation_nested_pipeline.c`**: Updated join logic to use LEFT JOIN LATERAL with an added null filter and replaced the deprecated `useInnerJoin` flag with `preserveNullAndEmptyArrays`.
- **`pg_documentdb/src/configs/feature_flag_configs.c`**: Introduced the new boolean GUC `enableLookupInnerJoin` (with a default of false) to control join type.
- **Test Files (e.g., expected output and SQL tests under `pg_documentdb` and `pgmongo` directories)**: Modified expected query plans and test scripts to validate the updated lookup join behavior.
- **`pg_documentdb/src/customscan/custom_explain_scan.c`**: Propagated parameter information to custom explain scan outputs.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Vinod Sridharan 91e673939e Merged PR 1715796: [OSS] Fix cursor cleanup cron job
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

The cron job for cursor cleanup on disk was broken since it was declared as a function but the call was done as a procedure. Fix the job command.

----
#### AI description  (iteration 1)
#### PR Classification
This pull request fixes a bug related to the cursor cleanup cron job.

#### PR Summary
The changes address incorrect scheduling and command syntax for cursor cleanup tasks in different modules.
- `/pgmongo/sql/pgmongo--1.104-1--1.105-0.sql`: Added a plpgsql block to conditionally schedule the `helio_cursor_cleanup_task` if it does not exist.
- `/oss/pg_documentdb/sql/documentdb--0.104-0--0.104-1.sql`: Updated the cron job syntax for `documentdb_cursor_cleanup_task` to call the correct cleanup function.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Jayanta Mondal faf4341b19 Merged PR 1711910: [oss] Code cleanup and documentation updates
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Code cleanup and documentation updates according to the latest contribution guidelines
2025-07-09 18:56:11 +08:00
Vinod Sridharan 4c190c5f02 Merged PR 1713276: [POC: Order by] Support order by descending pushdown to the index if available
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
1) Support order by descending as a feature under a feature flag if the underlying index can support it
2) Fix up memory performance issues causing OOMs with composite index with large numbers of bounds
3) On doing this noticed that row_get_bson was busted for int arrays, fix this behavior to not crash
2025-07-09 18:56:11 +08:00
Xing Fan eaf56fcfd4 Merged PR 1713368: [oss] [test clean] Cleanup get indexes related tests
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

update test cases for get indexes

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a test cleanup that revises expected outputs and SQL scripts for get indexes functionality.

#### PR Summary
The changes update several test files to reflect new naming conventions for databases, collections, and index definitions.
- `oss/internal/pg_documentdb_distributed`: Updated expected output files (`bson_get_indexes_a.out`, `bson_get_indexes_b.out`) and SQL scripts (`bson_get_indexes_a.sql`, `bson_get_indexes_b.sql`), replacing old identifiers (e.g., `msdb`, `employee`, `Compound_test`, etc.) with new ones (e.g., `orderdb`, `order_data`, `compound_orders`) and adjusting index names and fields accordingly.
- `pgmongo`: Modified corresponding expected output files to conform to the new naming and index configurations.
- Revised error messages and test cases for non-existent collections/DBs, as well as sparse and unique index test cases.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Xing Fan aa7b0f3154 Merged PR 1713395: [oss][code cleanup] Cleanup comments in several files
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

cleanup statics file

----
#### AI description  (iteration 1)
#### PR Classification
This pull request performs a code cleanup by standardizing and simplifying comments in the statistics operator functions.

#### PR Summary
The PR updates in-code comments to remove redundant references and streamline wording regarding null and numeric return behaviors in statistical functions.
- Updated `oss/pg_documentdb/src/aggregation/bson_aggregation_statistics.c` to replace mentions of "Mongo returns" with "Returns" across multiple functions, clarifying behavior for empty sets, single numeric values, and non-numeric inputs.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Jia Hu aad3233bab Merged PR 1713334: [oss][test cleanup] rewrite sql test for text search
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

text search

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a cleanup refactoring that rewrites the SQL tests for text search functionality.

#### PR Summary
The changes update the SQL tests to align with new collection names and field conventions while revising search queries and TSQuery generation.
- `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/bson_dollar_ops_text_search_tests.out`: Updated expected outputs by replacing the old "text_search" references with "bson_dollar_ops_text_search" and adjusted field names.
- `/oss/internal/pg_documentdb_distributed/src/test/regress/sql/bson_dollar_ops_text_search_tests.sql`: Rewritten insert statements, index definitions, and query tests with new field names, search terms, and projections.
- Revised TSQuery generation tests to use updated search phrases and parameters, ensuring alignment with the current text search design.
- Minor formatting and comment cleanup to reflect the updated test conventions.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Zinan Ren b11753f8d2 Merged PR 1712315: [oss][test cleanup] Vector search related cleanup
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
No
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Cleanup vector tests
2025-07-09 18:56:11 +08:00
Santiago Fernandez Madero c0b63a1fd2 Merged PR 1713045: [oss] Cleanup to conform with contributing guidelines
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

[oss] Cleanup to conform with contributing guidelines

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a code cleanup that standardizes naming conventions to conform with the contributing guidelines.
2025-07-09 18:56:11 +08:00
Vinod Sridharan 1c433c472b Merged PR 1714381: [Order by + Test] Fix up more correctness of order by + Testing
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
1) There were order by tests not running but had the sql files in there. This was catching contracts related to arrays with null values where 2 cases were ordered later than they should be. This fixes that. As part of that we noticed that this breaks when there's paths that have array indexes (e.g. a.b.0). This also fixes that contract.
2) Add checks to say all tests must run or exist in a schedule file and fix the output
3) There's flakeyness in currentop due to tracking the version refresh query - skip the version refresh query in currentop.

----
#### AI description  (iteration 1)
#### PR Classification
This pull request fixes correctness issues in composite order-by functionality and improves associated testing.

#### PR Summary
The changes refine the handling of intermediate arrays in order-by operations by updating traversal and comparison logic, while enhancing the test suite and validation scripts for composite order-by scenarios.
- `pg_documentdb/src/query/bson_dollar_operators.c` and related files were modified to correctly process nested arrays and compare composite order-by values.
- Test files in `pgmongo/src/test/regress/expected/` and `pgmongo/src/test/regress/sql/` were updated to add new order-by cases and remove obsolete tests.
- Validation scripts (e.g. in `oss/internal/pg_documentdb_distributed/src/test/regress/validate_test_output.sh` and `oss/pg_documentdb/src/test/regress/validate_test_output.sh`) were enhanced for more reliable schedule checks and collection ID extraction.
- Several BSON traversal and update operator functions were updated to include an `isArrayIndexSearch` parameter for finer control during array element processing.
- `pg_documentdb/src/commands/current_op.c` and `pg_documentdb/src/utils/version_utils.c` were updated to skip version refresh queries appropriately.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Santiago Fernandez Madero 5d7eeeac53 Merged PR 1713311: [oss] Cleanup to follow documentdb coding guidelines
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

[oss] Cleanup to follow documentdb coding guidelines

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a code cleanup aimed at aligning the codebase with DocumentDB coding guidelines.
2025-07-09 18:56:11 +08:00
Vinod Sridharan 8a7e2fd58b Merged PR 1713818: [PoC] Fix up correctness of order by scans with truncation
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Fix up correctness with truncation - truncation required checks on recheckOrderby (separate from recheckItem for filters).

We had 2 flags - recheckItem and recheckItemOrderby. The problem is truncation doesn't check the root truncation term with order by scans - so we need to handle this by reusing recheck on order by (which checks truncation) with recheck on filter.

----
#### AI description  (iteration 1)
#### PR Classification
Bug fix addressing the correctness of order by scans with truncation in RUM.

#### PR Summary
This pull request fixes issues in the RUM ordering logic by renaming and repositioning key recheck variables and reorganizing scan state structures, while also extending the test suite for forced order by scans.
- `pgmongo/src/test/regress/expected/internal_bson_composite_order_by_index_tests.out`: Added new test cases for forced order by scans and recheck on runtime with long keys.
- `pgmongo_rum/src/rum.h`: Reorganized struct fields by moving the `orderStack` and reintroducing `isMatchMinimalTuple` and bitmap scan fields, along with new GUC parameters for Pgmongo.
- `pgmongo_rum/src/rumget.c`: Replaced the variable `recheckCurItemOrderBy` with `recheckCurItem` to correctly control recheck behavior during ordered scans.
- `pgmongo/src/test/regress/sql/internal_bson_composite_order_by_index_tests.sql`: Revised SQL tests to reflect the changes in ordering and recheck logic.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Deepak Bisht 42942c236e Merged PR 1713941: [AZL3] Fix the Segmentize test to handle errors generically
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Testing

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
NA

### Description

- Handle test errors generically to support AZL3

----
#### AI description  (iteration 1)
#### PR Classification
This pull request fixes a failing test by updating error handling in the Segmentize test.

#### PR Summary
The pull request updates the expected outputs in several geospatial tests to handle error messages more generically by checking for both "Self-intersection at" and "geometry is invalid" cases, returning a unified error message. This change ensures consistency across tests without relying on specific numeric details.
- `oss/internal/pg_documentdb_distributed/src/test/regress/expected/geospatial_extract_2d_geometries.out`: Modified error handling to output a generic error for both self-intersection and invalid geometry cases.
- `pgmongo/src/test/docdb_compat/expected/geospatial_extract_2d_geometries.out`: Applied similar generic error message updates for consistency in the compatibility tests.
- `oss/internal/pg_documentdb_distributed/src/test/regress/sql/geospatial_extract_2d_geometries.sql`: Updated SQL error handling to reflect the unified error message.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Vinod Sridharan f42e4ce7ee Merged PR 1713840: Add sql command to currentOp in tests
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Testing

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
CurrentOp tests have flakeyness due to some background process. Log the SQL query for tests for currentOp so that we can track the source of flakeyness

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements a new feature that adds SQL command details to the currentOp output for tests.

#### PR Summary
The changes update the expected test outputs and the server logic to include a new "sql" field in currentOp responses, which displays the actual SQL command executed for each operation.
- **`pgmongo/src/test/regress/expected/isolation-currentop.out`**: Updated expected outputs to include the "sql" field for various operations.
- **`oss/pg_documentdb/src/commands/current_op.c`**: Modified to append the SQL command in the output when the feature flag is enabled.
- **`oss/pg_documentdb/src/configs/testing_configs.c`**: Added a new configuration variable (`CurrentOpAddSqlCommand`) to control SQL command inclusion.
- **`pgmongo/src/test/regress/isolation_postgresql.conf`**: Enabled the SQL command logging feature in the test configuration.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Jinrui Zhang a745009b72 Merged PR 1712417: [oss][test cleanup] finish sql test renaming for negation related sql
[oss][test cleanup] finish sql test renaming for negation related sql

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a test cleanup focused on standardizing the naming of negation-related SQL tests.

#### PR Summary
The changes update numerous test files and SQL scripts by replacing all references to the collection name `negation_tests` (and related variants) with `simple_negation_tests` to ensure naming consistency.
- Files in the `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/` and `/pgmongo/src/test/docdb_compat/expected/` folders now reference the new collection name throughout queries and expected outputs.
- SQL scripts for both core and explain tests have been updated to use `simple_negation_tests` in insert, search, sharding, and index creation commands.
- Related runtime and core SQL test files have been modified, ensuring that DROP, CREATE, and indexing operations are aligned with the new naming.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Deepak Bisht 49f5969149 Merged PR 1712527: [C] Aligning C files to contributing guidelines
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactor

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

- Change .c files to align with contributing guidelines

----
#### AI description  (iteration 1)
#### PR Classification
Code cleanup and refactoring to align naming conventions with contributing guidelines.

#### PR Summary
This pull request standardizes naming and comments across several C source files by replacing outdated "mongo" terminology with "documentdb" or "collection" as appropriate. The changes improve consistency and clarity in code and documentation.
- **`oss/pg_documentdb/src/aggregation/bson_aggregation_window_operators.c`**: Renamed enums, macros, and validation checks from FRAMEOPTION_MONGO_* to FRAMEOPTION_DOCUMENTDB_* and updated related comments.
- **`oss/pg_documentdb/src/aggregation/index_stats.c`**: Updated variable names and output messages to use “collection” terminology and reflect wire compatible formats.
- **`oss/pg_documentdb/src/aggregation/bson_densify.c`**: Replaced isMongoDataTable with isCollectionDataTable and revised related comments and function calls.
- **Other files (`bson_tree.c`, `bson_unwind.c`, and testing_configs.c)**: Modified comments and error messages to remove Mongo-specific references, ensuring consistency with the new naming guidelines.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Vinod Sridharan 166eaf2980 Merged PR 1713498: [Lookup Crash] Fix crash with $lookup/$graphLookup
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Lookup/$graphLookup pushdown to index was broken. An earlier attempt to fix it removed inner joins as a way since it reproed this. However, this could still repro in other cases, and was recently found in graphLookup too. The real root cause seems to be due to the removal of paramPath when converting index scan to bitmap scan- fix this.

----
#### AI description  (iteration 1)
#### PR Classification
This PR fixes a bug causing crashes in $lookup/$graphLookup operations by switching to inner join semantics when null preservation is not required.

#### PR Summary
The changes refactor the lookup join processing to eliminate the crash by introducing a new boolean flag (`useInnerJoin`) in the lookup context and adjusting the join type accordingly, while also updating the tests to validate the new behavior.
- **`src/aggregation/bson_aggregation_nested_pipeline.c`**: Removed redundant null filtering logic and added the `useInnerJoin` flag to control join behavior.
- **`src/aggregation/bson_aggregation_nested_pipeline.c`**: Updated the `MakeLookupJoinRte` function to set the join type to INNER JOIN if `useInnerJoin` is true.
- **Test files in `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/` and `/pgmongo/src/test/docdb_compat/expected/`**: Added new lookup inner join test cases and updated expected output files.
- **`src/customscan/custom_scan_continuation.c`**: Adjusted planner path handling to correctly propagate parameter path info in lookup-related queries.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Jinrui Zhang 197f24b4ea Merged PR 1712204: [oss] [test cleanup] finish checking for fill linearfill locf in .c .sql .out
### Description
Code clean up. Finish checking for fill linearfill locf in .c .sql .out

----
#### AI description  (iteration 1)
#### PR Classification
Code cleanup improving inline comment clarity in a core aggregation function.

#### PR Summary
This pull request refines the inline comments in the BSON aggregation window function to remove redundant references and clarify expected numeric behaviors.
- `oss/pg_documentdb/src/aggregation/bson_aggregation_window_functions.c`: Updated comments in the `bson_linear_fill` function to remove "As Mongo does," and clearly describe that the filled value should be numeric or nullish, as well as outlining type expectations for return values.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Siwen Tang 8f25bf6e45 Merged PR 1712410: [oss][test cleanup] update bson_query_operator_array related tests
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Update bson_query_operator_array related tests
----
#### AI description  (iteration 1)
#### PR Classification
Test cleanup and refactoring.

#### PR Summary
This pull request refactors BSON query operator array tests by updating collection, index, and field naming conventions. Key changes include:
- Files in `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/` and similar output files now replace `"comp_arrdb"/"composite_array_ops"` with `"array_query_db"/"array_operator_tests"` and change the field from `"a"` to `"value"`.
- SQL test scripts (e.g. in `/oss/internal/pg_documentdb_distributed/src/test/regress/sql/`) have been updated to reflect the new collection and field names.
- Updates in the compatibility tests under `/pgmongo/src/test/docdb_compat/` mirror the schema changes.

<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Jinrui Zhang 5beece377f Merged PR 1712330: [oss][test cleanup] Finish cleanning up of densify .c .sql
finish cleanning up of densify .c .sql

----
#### AI description  (iteration 1)
#### PR Classification
Code cleanup and refactoring.

#### PR Summary
This PR cleans up naming and comments in the densify module to enhance clarity and consistency by replacing Mongo-specific terminology with generic collection terminology.
- In `oss/pg_documentdb/src/aggregation/bson_densify.c`, renamed the variable from `isMongoDataTable` to `isCollectionDataTable` and updated its usage.
- In the same file, updated comments to remove outdated Mongo references and clarify functionality.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Shuai Tian 7d43550312 Merged PR 1712338: [oss][test cleanup] update tests for stages bucket, bucketAuto, group with push, addToSet, facet
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
update tests
2025-07-09 18:56:11 +08:00
Jia Hu cfe3ef6395 Merged PR 1713394: [oss][code cleanup] cleanup for bson_expression_type_operators.c, comments rewrite
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

comments

----
#### AI description  (iteration 1)
#### PR Classification
This pull request performs code cleanup by revising comment texts.

#### PR Summary
The changes update comments in `bson_expression_type_operators.c` to replace references to "native mongo" with "protocol," ensuring the comments accurately reflect protocol behavior.
- `bson_expression_type_operators.c`: Updated comments in ValidateStringIsNotHexBase, ProcessDollarType, and ProcessDollarToDate to clarify protocol-specific behavior regarding hexadecimal validation, missing field paths, and unsupported int32 to date conversions.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Yang Qiao a0786173cc Merged PR 1712344: [oss][script cleanup] OSS scripts cleanup
### Does this PR have any customer impact?
NO

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Cleanup

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
NO

### Are you introducing any new config? If yes, do you have tests with and without them being set?
NO

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
cleanup OSS scripts
2025-07-09 18:56:11 +08:00
Xing Fan dee469b791 Merged PR 1713345: [oss] [test clean] Cleanup decimal128 tests
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

update decimal128 tests

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a test cleanup that updates decimal128 expected outputs and SQL test queries.

#### PR Summary
The changes standardize decimal128 representations by updating numerical values and error messages in test output files and SQL queries. This improves consistency with protocol‐defined behavior for Decimal128 arithmetic operations.

- `oss/internal/pg_documentdb_distributed/src/test/regress/expected/bson_decimal128.out`: Updated expected numeric outputs for various Decimal128 operations.
- `pgmongo/src/test/docdb_compat/expected/bson_decimal128.out`: Revised expected Decimal128 values to align with new protocol behavior.
- `oss/internal/pg_documentdb_distributed/src/test/regress/sql/bson_decimal128.sql`: Modified SQL test cases and inserted data values for Decimal128, including minor comment cleanups.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Jia Hu 8270a276d4 Merged PR 1712201: [oss] [cleanup] json schema related, including sql test and code comments
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
2025-07-09 18:56:11 +08:00
Duan Pei 1986039242 Merged PR 1712345: [oss][test cleanup]expMovingAvg maxN minN topN bottomN
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

expMovingAvg maxN minN topN bottomN .sql and .out
2025-07-09 18:56:11 +08:00
Xing Fan 6ea6a54460 Merged PR 1712487: [oss] [test clean] Cleanup $regex related
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

related test cases

----
#### AI description  (iteration 1)
#### PR Classification
This PR is a test cleanup focused on updating regex-related tests.

#### PR Summary
The changes refactor the test cases by replacing document fields "sku" and "description" with "address" and "comment," and by updating the corresponding regex query filters and expected output files.
- `oss/internal/pg_documentdb_distributed/src/test/regress/expected/bson_dollar_ops_basic_text_ops_tests_index.out` and `.../tests_runtime.out`: Modified inserted document fields and regex query conditions.
- `pgmongo/src/test/docdb_compat/expected/bson_dollar_ops_basic_text_ops_tests_index.out` and `.../tests_runtime.out`: Applied consistent field and query updates across compatibility tests.
- `oss/internal/pg_documentdb_distributed/src/test/regress/sql/bson_dollar_ops_basic_text_ops_tests_core.sql`: Updated document insert statements and query filters to use the new field names.
- `oss/internal/pg_documentdb_distributed/src/test/regress/expected/bson_dollar_ops_basic_text_ops_tests_explain_index.out` and `.../tests_explain_runtime.out`: Adjusted minor formatting in the QUERY PLAN output.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Parag Jain a516aa9117 Merged PR 1712434: [oss] Cleanup as per new change guideline, part -3
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Cleanup as per new change guideline, part -3

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a code cleanup that standardizes and clarifies inline comments across multiple command files.

#### PR Summary
The changes update comment wording and remove redundant explanations to align with new guidelines, improving overall code clarity and consistency. Key changes include:
- Updates in `coll_mod.c` to clarify lock acquisition and index TTL constraints.
- Revised comments in `coll_stats.c` and `db_stats.c` to standardize responses and scale conversion details.
- Simplified comment phrasing in `current_op.c` and `delete.c` by removing unnecessary Mongo-specific references.
- Cleaned up prevalidation explanations in `drop_indexes.c` and support option comments in `commands_common.c`.
- Removed redundant commentary in `find_and_modify.c` regarding field casing.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00
Parag Jain 57a1be6f40 Merged PR 1712473: [oss] Cleanup as per new change guideline, part -4
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
[oss] Cleanup as per new change guideline

----
#### AI description  (iteration 1)
#### PR Classification
This pull request is a test cleanup update designed to standardize test collection names and expected outputs in both aggregation pipeline and set operator tests.

#### PR Summary
The changes update and rename test data to comply with new guidelines. Key modifications include:
- Updated expected output files for BSON aggregation “merge objects” tests in `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/` and `/pgmongo/src/test/docdb_compat/expected/` by renaming collections (e.g. from `objColl1` to `mergeObjectsGroupColl`) and adjusting document content.
- Modified SQL test files in `/oss/pg_documentdb/src/test/regress/sql/` and `/oss/internal/pg_documentdb_distributed/src/test/regress/sql/` to reflect new collection names and query parameters.
- Revised BSON aggregation set operators tests to update date values (switching from 2019 timestamps to 2013 values) and associated expected outputs.
- Updated drop and shard collection commands accordingly to maintain consistency across tests.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-09 18:56:11 +08:00