mirror of
https://github.com/FerretDB/documentdb.git
synced 2026-11-07 03:22:46 -05:00
main
390
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vinod Sridharan
|
84b999ae88 |
Merged PR 1815453: [Composite] Support optimized elemMatch pushdown
### Does this PR have any customer impact?
Yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
### Description
Improve pushdown of $elemMatch for composite indexes.
When there's an elemMatch of the form `{ "a": { "$elemMatch": { "$gt": 10, "$lt": 15 }}`
This can scan the index only from 10 -> 15
However given the multi-key index requirement we were doing 2 scans:
- One from 10 to infinity and the other from -infinity to 15.
Optimize this path for $elemMatch.
----
#### AI description (iteration 1)
#### PR Classification
This PR introduces a new feature that optimizes the pushdown of `$elemMatch` queries for composite indexes.
#### PR Summary
The pull request implements optimized handling of `$elemMatch` predicates on composite indexes by converting them into specialized index pushdown expressions and updating the corresponding test expectations.
- **`oss/pg_documentdb/src/opclass/bson_gin_composite_core.c` & `index_support.c`**: Added logic to detect `$elemMatch` in range queries and convert them into optimized pushdown expressions (via functions like `GetElemMatchIndexPushdownOperator`).
- **Configuration Changes**: Introduced a new GUC (`documentdb.useNewElemMatchIndexOperatorOnPushdown`) in `pg_documentdb/src/configs/feature_flag_configs.c` to control the new behavior.
- **Test Updates**: Modified expected explain outputs in both `/oss/internal/pg_documentdb_distributed/src/test/regress/` and `/pgmongo/src/test/docdb_compat/expected/` to reflect the revised index condition format using `elemMatchIndexOp`.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
|
||
|
Vinod Sridharan
|
8514e44ac2 |
Merged PR 1817415: Fix up planning cost estimate for order by
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Order by expressions have a filter for full scan that makes it so it can push down to the index. However the default selectivity of 1% kicks in at the time. Instead ensure that It has a selectivity of 100% Also fix list collections with distribution failing on views ---- #### AI description (iteration 1) #### PR Classification Bug fix addressing errors in BSON range operator selectivity and index pushdown behavior. #### PR Summary This PR fixes issues with full scan detection and selectivity handling in BSON range queries while enhancing index pushdown and metadata listing tests. The changes improve the correctness of query planning and operator behavior in DocumentDB integrations. - **`src/query/bson_dollar_selectivity.c` & `src/opclass/index_support.c`**: Added functions to detect full scans and correctly set selectivity (e.g., returning 1.0) for `$range` queries. - **Test Files (`bson_composite_order_by_index_tests.out`, `list_metadata_cursor_tests.out`, and SQL scripts)**: Extended tests for composite order-by indexes and distributed metadata, ensuring proper index selection and query plans. - **Configuration and Startup Scripts (`feature_flag_configs.c`, `start_oss_server.sh`)**: Introduced the `setSelectivityForFullScan` flag and updated extended RUM extension settings. - **SQL Migration Scripts**: Altered operator restrict definitions to use updated selectivity functions for consistent operator handling. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
b38e58c455 |
Merged PR 1818535: Move the PG18 build container to the latest image
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) DevOps ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Move the PG18 build container to the latest image ---- #### AI description (iteration 1) #### PR Classification Configuration update for build container images. #### PR Summary This pull request updates the PG18 build container image to use the latest normalized tag, ensuring consistency with the PG17 configuration. - Updated ``/.pipelines/templates/common_variables.yml`` by replacing the hard-coded tag for PG18 with ``variables.NormalizedContainerTag``. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
 Santiago Fernandez Madero
|
6f19dbec81 |
Merged PR 1817238: TTL fix descending and continous purging
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Continous TTL was not taking into consideration the max continous timeout but instead the single ttl task so it was always capped at 20 seconds. TTL descending needed to use the order by function and fullrange so that it goes through the order by push down logic rather than PGs logic. Also, add an entry to the log to indicate if it is descending or not. ---- #### AI description (iteration 1) #### PR Classification This PR fixes TTL purging issues by correcting the descending sort behavior and ensuring continuous purging. #### PR Summary The changes update TTL deletion logic to use descending order when enabled and adjust associated logging and configuration options. This enhances purging consistency and aligns test expectations. - `pgmongo/src/test/regress/expected/pgmongo_internal_ttl_tests.out` and related test outputs now include the `use_desc_sort` flag in log messages. - `oss/pg_documentdb_distributed/src/test/regress/expected/commands_create_ttl_indexes.out` and SQL test files have been modified to set descending sort and disable sequential scans. - `oss/pg_documentdb/src/ttl/ttl_index.c` has been updated to conditionally append descending order queries using index orderby pushdown. - Configuration files such as `oss/pg_documentdb/src/configs/background_job_configs.c` now define and enable TTL descending sort options (e.g., `EnableTTLDescSort` and `EnableIndexOrderbyPushdown`). <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
f0d4514dfc |
Merged PR 1814310: [DocumentDB] Support running backend tests with extended_rum
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Testing ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Currently documentdb tests only run with default rum and non-composite op-class. While with PG18 they run with ex_rum as a binary they still don't leverage the extended_rum index am and op-class. Add an option to add backend tests that test the combo until we make this the default path. ---- #### AI description (iteration 1) #### PR Classification This pull request introduces a new feature by adding support for running backend tests that leverage extended_rum in DocumentDB. #### PR Summary The changes integrate an extended_rum test suite into the DocumentDB backend testing infrastructure, ensuring that extended index handlers are properly configured and validated. - **`src/test/extended_rum_tests`**: Added new SQL test files, expected output files, scheduling files, a Makefile, a shell script (`mutate_schedule.sh`), a custom `postgresql.conf`, and a `.gitignore` to support and isolate extended_rum tests. - **`src/test/regress/validate_test_output.sh`**: Modified to correctly resolve directories for validating test outputs. - **Top-level and project Makefiles**: Updated (`Makefile`, `oss/pg_documentdb/Makefile`, and `src/test/Makefile`) with new targets to run extended_rum tests alongside the existing regression tests. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
2108af32e6 |
Merged PR 1812986: [Hidden Index] Support Hidden Indexes with Distribution
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Add support for hidden indexes with distributed tables. - Add function that supports per node distribution with shards: Select all shards for the table and pick one shard per node. - Add a planner worker function that executes on every shard - On the shard skip execution if it's not the selected shard - Run the local function on the worker function. - Plumb in hooks to execute the index functions remotely. ---- #### AI description (iteration 1) #### PR Classification This pull request implements a new feature to support hidden indexes in a distributed environment. #### PR Summary The changes integrate hidden index support into distributed index update operations by introducing new per-node functions, updating API hooks, and modifying planner and UDF logic for proper shard routing and logging. - **`node_distribution_operations.c`**: New file that adds functions (e.g., `UpdateDistributedPostgresIndex` and its worker) to execute index update commands per node in a distributed setting. - **`diagnostic_commands_common.c`**: Introduces `command_node_worker` to ensure that update commands target the appropriate shard based on table identity. - **`coll_mod.c` and `coll_mod.h`**: Updated to use `UpdatePostgresIndexWithOverride` and correctly incorporate hidden index modifications with distributed operations. - **API hooks and metadata files** (e.g., `api_hooks.c`, `api_hooks_def.h`, `api_hooks.h`, and `metadata_cache.c`): Enhanced to support new function pointers and caching mechanisms for hidden index updates via the new command worker. - **SQL UDF scripts**: New UDF definitions for `command_node_worker` and `update_postgres_index_worker` enable proper registration and execution of hidden index operations in the distributed framework. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
4a38f05454 |
Merged PR 1813645: Fix up currentOp, index stats logging for Parallel Builds & a few index fixes
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature/Bugfix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description - Add tracking of parallel workers in currentOp. - Ensure index build progress works for blocking index builds - Fix crash in parallel rum index builds - Allow RUM parallel index build path even for serial build (so we can unify to one codepath) - Turn on amcanparalllelbuild for PG >= 17 based on the GUC ---- #### AI description (iteration 1) #### PR Classification Bug fix addressing logging inaccuracies for parallel build operations. #### PR Summary This pull request improves logging for current operations and index build progress by incorporating additional metadata and refining SQL queries to better handle parallel workers. - In `oss/pg_documentdb/src/commands/current_op.c`, new attributes (backend type and leader PID) are extracted from query results and appended to BSON output for accurate identification of parallel workers. - In `oss/pg_documentdb/src/commands/current_op.c`, progress logging is enhanced by including a "concurrent" flag and updating phase-to-message mappings for index build initialization. - In `pgmongo/src/infrastructure/index_stats_logging.c`, the index progress query now uses left joins with COALESCE to reliably capture index and table names, improving stats logging for parallel builds. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
f2d0656e1c |
Merged PR 1810273: [RUM] Lost Path Fixes
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Bugfixes ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description 1) Fix bugs around post-recovery cleanup and incomplete splits in some cases causing Lost path 2) The retry around the failure was also not working due to case differences in errors 3) Ensure tracking vs fixing are tracked by different GUCs. ---- #### AI description (iteration 1) #### PR Classification Bug fix addressing lost index paths in RUM indexes by ensuring incomplete page splits are properly handled. #### PR Summary This pull request fixes lost path errors in RUM indexes by enhancing the insertion and page split logic to detect, complete, and clear incomplete splits. - **`pgmongo_rum/src/rumbtree.c`**: Introduced new functions (e.g., `rumPlaceToPage`, `rumFinishSplit`, `rumFinishOldSplit`) to complete pending page splits during insertions and update parent links appropriately. - **`pgmongo_rum/src/rum.h`, `rumdatapage.c`, and `rumentrypage.c`**: Updated to track and clear the `RUM_INCOMPLETE_SPLIT` flag and to integrate new callbacks for incomplete split scenarios. - **`pgmongo_rum/src/rumutil.c` and related configuration files**: Added custom GUC variables (`track_incomplete_split`, `fix_incomplete_split`, `enable_inject_page_split_incomplete`) to control the behavior of incomplete split handling. - **Test files in `/pgmongo/src/test/regress/`**: Added new regression tests to validate the correct handling of incomplete splits and the associated lost path conditions. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Santiago Fernandez Madero
|
9ac8147799 |
Merged PR 1786779: [background unique build 1/3] Create index as unique support
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description This new option for create indexes creates an index as if it was a UNIQUE composite term index, but without actually creating the constraint on the table, so it is a regular index. This is the first phase of 3 needed. We introduce a three-step workflow that lets customers build the index structure in the background (non-blocking), then progressively enable uniqueness enforcement: 1. buildAsUnique (createIndex) — build the index as if unique but without registering any table constraint (non-blocking) 2. prepareUnique (collMod) — register the uniqueness constraint in the system catalogs but mark it unvalidated so it starts enforcing uniqueness for new writes only 3. convertUnique (collMod) — validate/enforce uniqueness for existing rows (walk the table or short-circuit if an existing validated unique index already proves uniqueness) ## Goals - Allow background construction of unique-capable indexes without blocking writes - Provide a fast metadata-only step to enable uniqueness for new writes - Provide a validation step that enforces uniqueness for pre-existing rows only when it is safe - Preserve data integrity and PostgreSQL metadata correctness - Expose these steps through familiar MongoDB commands (`createIndex` and `collMod`) - Migrating non ordered unique indexes to ordered indexes without blocking production writes. ## High-level Workflow Step 1 — buildAsUnique (createIndex): - Customer requests an index with storageEngine.buildAsUnique=true. - The system creates a regular index physically (non-blocking) but records metadata indicating the index is intended to become unique Step 2 — prepareUnique (collMod): - Customer requests convert-to-unique preparation via `collMod` (prepareUnique) - The system registers a constraint in the PostgreSQL catalogs but does not validate it against existing data; it enforces uniqueness on new writes going forward - This is a fast, mostly metadata-only operation (convalidated = false) Step 3 — convertUnique (collMod): - Customer requests final conversion via `collMod` (convertUnique) - The system validates uniqueness for existing rows by walking the table (or aborts if duplicates are found), or it can skip the full walk when an existing validated unique index already proves uniqueness ---- #### AI description (iteration 1) #### PR Classification This pull request implements a new feature that adds support for the "prepareForUnique" option in index creation. #### PR Summary The changes extend the index creation mechanism to allow a non-table unique index to be prepared via the "prepareForUnique" flag, with appropriate error handling and ... Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
c3c0ffc450 |
Merged PR 1812268: [Sort] Add reverse order to 106.1 Schema
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Move order by support into 106.1 schema as a hotfix option ---- #### AI description (iteration 1) #### PR Classification This pull request implements a new feature by adding reverse order support for the 106.1 schema. #### PR Summary The changes refine reverse sorting by updating conditional checks in the C code and aligning SQL migration scripts with the new version. - `oss/pg_documentdb/src/commands/current_op.c`: Added a null check for `detail` before allocating and returning the index specification, ensuring proper handling. - `oss/pg_documentdb/src/opclass/index_support.c`: Modified reverse order conditions to include the 106.1 patch check along with the existing 107.0 version check. - `pgmongo/sql/`: Moved the SQL migration file (renamed from `pgmongo--1.106-0--1.107-0.sql` to `pgmongo--1.106-0--1.106-1.sql`) and updated the corresponding sha1 checksum files. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
 Cornelius Boateng
|
abafc44b4f |
Merged PR 1804989: [update] Internal Funcs Should Have No Defaults Arg Values
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Remove default arg value from the new update function. Function has not yet been used in C code. ---- #### AI description (iteration 1) #### PR Classification This pull request is an API change that revises internal UDF functions for update operations by removing default argument values. #### PR Summary The changes introduce a new SQL file for bson update functions and update existing function signatures to eliminate default values—resulting in stricter parameter handling and an added parameter for V2 functions. - `oss/pg_documentdb/sql/udfs/commands_crud/bson_update_document--0.108-0.sql`: Added a new file defining the update functions. - `oss/pg_documentdb/sql/udfs/commands_crud/bson_update_document--latest.sql`: Modified the `__API_SCHEMA_INTERNAL_V2__.update_bson_document` function by removing default values from `arrayFilters` and `variableSpec` and adding the `collationString text` parameter. - `pgmongo/sql/pgmongo--1.107-0--1.108-0.sql` and `oss/pg_documentdb/sql/documentdb--0.107-0--0.108-0.sql`: Updated include directives to load the new update document SQL file. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
 Deepak Bisht
|
1a695bdf9f |
Merged PR 1811981: [Observability] Add feature counters for all query operators
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Observability ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Add feature counters for all public query operators ---- #### AI description (iteration 1) #### PR Classification This PR implements a new feature by adding comprehensive feature counters for all query operators to enhance observability. #### PR Summary The changes extend the observability framework by introducing new feature counter definitions, mappings, and usage in query operator implementations. - Updated `oss/pg_documentdb/include/utils/feature_counter.h` and `oss/pg_documentdb/src/infrastructure/feature_counter.c` to add new query operator feature enum values and corresponding string mappings with an increased `MAX_FEATURE_COUNT`. - Modified `oss/pg_documentdb/src/planner/mongo_query_operator.c` to assign the new feature counters to each query operator. - Revised test expected outputs in `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/feature_counters.out` and `/pgmongo/src/test/docdb_compat/expected/feature_counters.out` to reflect the updated counters. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Deepak Bisht
|
9cec4df623 |
Merged PR 1807532: Crash Dump fix while building aggregate queries with $group aggregates inside $lookup
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) BugFix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description - Today before building the complete pipeline query of lookup we initialize the varlevelsup of `Var` to be set to INT_MAX but if there is an aggregate such as $sum it then tries to get the parseState of those many levels of parent and hence crashes out. The fix is we earlier set varlevelsup to int max so that we can uniquely identify these Vars after building the query for pipeline but instead of using varlevelsup now we use location field to uniquely identify these Vars and adjust the levelsup later. ---- #### AI description (iteration 1) #### PR Classification This pull request implements a bug fix to resolve a crash when building aggregate queries that include $group aggregates within $lookup. #### PR Summary The changes introduce a new test case to validate the correct query plan for lookup queries with aggregates and adjust internal variable handling in the aggregation pipeline builder to prevent crash dumps. - **`bson_aggregation_nested_pipeline.c`**: Updated variable resolution by setting `varlevelsup` to 0 and adjusting `location` values to ensure proper handling in nested pipeline contexts. - **Test files (`bson_aggregation_stage_lookup_pg17_tests.out`, `bson_aggregation_stage_lookup_tests.out`, and related SQL tests)**: Added new expected outputs for lookup queries incorporating aggregates to verify the fix across different test suites. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4114368, #4696693 Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
ef8f7ae41c |
Merged PR 1809984: Add version logging to the backend
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) DevOps ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Add Version Logging in the backend metrics collection. This makes it so we're not reliant on the gateway to log version information to metrics. ---- #### AI description (iteration 1) #### PR Classification This pull request introduces a new feature to log backend version metrics. #### PR Summary The changes add functionality to periodically log version information from various components, ensuring updated logging when version changes or timer intervals trigger a new record. - `pgmongo/src/infrastructure/mdsd_logger.c`: Added new functions (`append_version_metric_data` and `LogVersionMetricData`) to construct and send version metrics, and integrated these into the main logging workflow. - `pgmongo/src/pgmongo_extension_version.c`: Introduced `PgmongoGetExtendedVersionString` and updated the version-returning function to use this extended string. - `oss/pg_documentdb/src/utils/version_utils.c`: Added `GetCurrentShortVersionStringForLogging` to format a concise version string for backend logging. - `pgmongo/src/metadata/oid_cache.c`: Added the `CitusVersionFunctionOid` function and updated the OidCache structure to support version logging. - Header updates in `oss/pg_documentdb/include/utils/version_utils_private.h` and `pgmongo/include/metadata/oid_cache.h` to declare the new version utility and Oid functions. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
f46a64e6dd |
Merged PR 1809207: [RUM] Port more changes from internal -> documentdb_rum
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Refactoring ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Reduce deltas between private and documentdb_rum - port more changes between internal and documentdb_rum. ---- #### AI description (iteration 1) #### PR Classification This pull request ports internal changes to the documentdb_rum extension, introducing new unredacted logging hook integration and enhanced scan logic. #### PR Summary The changes integrate a new hook setter for unredacted logging, add support for pre-consistent scanning using function pointers, and update TS function exports and minor comments for clarity. - **`pgmongo_rum/src/rumutil.c` & `pgmongo_rum/src/rum.h`**: Removed the legacy unredacted logging function and added a new exported hook setter (`SetRumUnredactedLogEmitHook`) along with macros for unredacted logging. - **`oss/internal/pg_documentdb_extended_rum/src/rumget.c` & related files**: Enhanced scan routines by incorporating pre-consistent function pointers and bounds checking to determine the appropriate scan type. - **`oss/pg_documentdb/src/index_am/rum.c`**: Modified the initialization routine to load and invoke the new unredacted logging hook from the external library. - **TS utility files (`rum_ts_utils.c` and others)**: Added PGDLLEXPORT declarations for TS functions and made minor comment updates for improved clarity. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
a4f7922959 |
Merged PR 1807745: [PG18] More changes to build & test with PG18
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Enable more builds and tests with PG18 Enable backend tests for documentdb with pg18 ---- #### AI description (iteration 7) #### PR Classification This pull request implements PG18 support by updating APIs, tests, and build pipelines. #### PR Summary The changes introduce PG18‑specific logic across regression tests, build scripts, and core GIN indexing code to enhance compatibility and performance with PostgreSQL 18. - **`oss/pg_documentdb/src/test/regress/expected/`**: New PG18 output files (e.g. `collection_management_pg18.out` and inverse match explain tests) were added while obsolete PG17/PG15 files were removed or moved. - **`/.pipelines/templates/extension_build_flex_core.yml`**: Build pipelines were updated with new parameters (`buildExtensionPackage`, `buildDiskANN`) and conditional steps to support PG18 builds. - **`helio_gin` sources (e.g. `ginget.c`, `ginutil.c`, `ginordered.c`)**: Core GIN code was refactored using `#if PG_VERSION_NUM >= 180000` conditionals to utilize new PG18 functions and adapt to API changes. - **Test scheduling and mutation scripts**: Updated to include PG18 tokens and adjust SQL file references for proper test execution on PostgreSQL 18. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
 Andrew Khoma
|
bd432acad2 |
Merged PR 1807721: Use the correct GUC
### Does this PR have any customer impact? no ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) bugfix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) no ### Are you introducing any new config? If yes, do you have tests with and without them being set? no ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) * Use the appropriate GUC for the user_crud_commands.sql ### Description change `documentdb_api.enableUsersInfoPrivileges` to `documentdb.enableUsersInfoPrivileges` in user_crud_commands.sql Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
 Parag Jain
|
d24041a82d |
Merged PR 1788960: [perf] minor performance improvement and cleanup in delete and update code path
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description 1. replaced GetObjectIdFilterFromQueryDocument with GetObjectIdFilterFromQueryDocumentValue We have already seen this in updateOne path in cpu and fixed it , missed it updatemany and delete so added in this PR 2. In delete path at once place we were calling GetObjectIdFilterFromQueryDocument early where it not used so pushed it down Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
95d7a4b6a6 |
Merged PR 1806432: [OSS] Remove explicit RUM dependency on the documentdb extension
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Refactoring ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Make DocumentDB not explicitly depend on the RUM extension (it's done implicitly) on the binary. Instead pull in the SQL dependency into documentdb and reference the C functions internally. This allows us to swap between rum.so and extended_rum. For Pg18 (where there isn't yet official RUM support) default to using extended_rum for the .so file and remove all dependencies on the "rum.so" ---- #### AI description (iteration 2) #### PR Classification This PR is an API and dependency update that removes the explicit dependency on the external RUM extension by integrating its functionality internally. #### PR Summary The update integrates the RUM text-search capabilities directly into DocumentDB and cleans up the external linkage. - `oss/internal/pg_documentdb_extended_rum/src/rum_ts_utils.c` is newly added with core text-search functions. - `oss/pg_documentdb/src/index_am/rum.c` is modified to load internal wrappers that call the new RUM functions. - SQL UDF adapter files and operator class definitions are updated to reference the internal functions instead of public RUM functions. - Build scripts, metadata, and control files are adjusted to remove the explicit dependency on the external RUM extension. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
4bcd774983 |
Merged PR 1804966: [Update performance] Improve update performance in a few codepaths
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description 1) Use prepared statements with UpdateAllMatchingDocuments 2) Don't pass shard_key_value by default if using the `@@` operator 3) Do something similar for SELECT update candidate. ---- #### AI description (iteration 1) #### PR Classification Performance optimization and bug fix for update query execution. #### PR Summary This PR enhances update performance by refining shard key filter handling and optimizing SPI plan selection, while also introducing new query plan cache keys and a configuration flag for legacy behavior. - `oss/pg_documentdb/src/commands/update.c`: Refactors update candidate selection by conditionally adding shard key filters, adjusting argument indexing, and using the full collection object instead of separate identifiers. - `oss/pg_documentdb/include/infrastructure/documentdb_plan_cache.h`: Introduces new constants for query plan caching to support optimized update queries. - `oss/pg_documentdb/src/configs/feature_flag_configs.c`: Adds the `UseLegacyShardKeyFilterOnUpdate` flag to offer backward compatibility with the previous shard key filtering behavior. - Test files in the distributed module: Update expected SQL query strings to match the revamped query construction logic. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
a0c2c59c97 |
Merged PR 1804958: [Code Cleanup] Remove composite feature flags that are default
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Refactoring ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Remove feature flags that are default on with the v106: - EnableNewCompositeIndexOpclass - EnableDescendingCompositeIndex - EnableCompositeUniqueIndex ---- #### AI description (iteration 1) #### PR Classification Code cleanup aimed at removing redundant composite feature flags that are now always default. #### PR Summary This PR simplifies the codebase by eliminating conditional logic, configuration variables, and test settings related to composite feature flags such as descending composite index, composite unique index, and the new composite index opclass. The changes remove obsolete definitions and related conditional checks while updating tests and config files accordingly. - In `configs/feature_flag_configs.c`, removed custom variable definitions and configuration settings for composite flags and introduced new default flags like `EnableIndexOnlyScan` and `EnableRangeOptimizationForComposite`. - In files such as `commands/create_indexes.c`, `index_am/rum.c`, and others, removed conditional logic and external declarations dependent on composite feature flags. - Updated numerous test scripts and configuration files to remove legacy settings (e.g., removing `enableNewCompositeIndexOpClass` and `enableDescendingCompositeIndex`) and adjust expected outputs. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
6f03ee4762 |
Merged PR 1803707: [Perf | Persistent cursors] Improve performance in simple persistent cursors
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Perf ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Minor perf fixes for persistent cursor - Don't generate cursorId in persistent cursor creation - Don't copy the persistent cursor plan on generation - Don't eagerly create a hold store unless the cursor is not terminated. ---- #### AI description (iteration 1) #### PR Classification Performance improvement for persistent cursors through enhanced cursor ID generation and optimized portal hold management. #### PR Summary This PR refines persistent cursor handling by introducing conditional logic based on a new feature flag, resulting in faster and more efficient cursor operations. - `/oss/pg_documentdb/src/commands/aggregation_commands.c`: Adjusted cursor ID generation and continuation document creation with support for delayed portal hold via a new conditional path. - `/oss/pg_documentdb/src/commands/cursors.c`: Optimized portal management by selectively holding portals and modifying cursor options according to the delayed hold flag. - `/oss/pg_documentdb/src/configs/feature_flag_configs.c`: Introduced the `EnableDelayedHoldPortal` flag to control and tune portal hold behavior. - `/scripts/build_dev_flex_container.sh`: Updated container cleanup logic to respect the SKIP_CONTAINER_CLEANUP environment variable. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
080ea66e2b |
Merged PR 1803246: [Hidden Index] Support for hidden indexes in DocumentDB
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Add support for hidden indexes with collMod This will mark the index as not ready in the postgres catalog. Handles distributed setups as well. However, this is currently restricted to single node (sharded/unsharded). A subsequent change can help multi-node set ups. ---- #### AI description (iteration 1) #### PR Classification This pull request implements a new feature by adding support for hidden indexes in DocumentDB. #### PR Summary The pull request extends the collMod command and associated index management logic to recognize and update a new “hidden” flag in index options, while enforcing constraints (e.g. disallowing hidden modifications for primary or unique indexes) in both single-node and distributed environments. - **`coll_mod.c`**: Introduces helper functions (e.g. GetHiddenFlagFromOptions, UpdateHiddenInIndexOptions, UpdatePostgresIndex) and SPI query updates to process the hidden flag properly. - **Test Files**: Adds and updates multiple test cases (e.g. in `commands_collmod_tests_hidden.out` and SQL test scripts) to verify hidden index creation, query planning, and error handling. - **Distributed Hooks and API**: Updates in files like `distributed_hooks.c` and `api_hooks.c` (with corresponding header changes) enable proper handling of hidden index updates across shard indexes in distributed environments. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
 William Wang
|
190c5f7710 |
Merged PR 1750825: [Full RBAC] Adding Support for RolesInfo Command in Engine (2/2, RolesInfo Impl.)
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) 1.108 ### Description We are adding support for the RolesInfo command in Engine ---- #### AI description (iteration 1) #### PR Classification New feature enhancement: introducing the `rolesInfo` command to retrieve detailed role information including inherited roles and privileges. #### PR Summary This pull request adds comprehensive support for the `rolesInfo` command in the engine, implementing new parsing, processing, and BSON output functions while updating tests and auxiliary query strings. - `oss/pg_documentdb/src/commands/roles.c`: Added functions (e.g., `ParseRolesInfoSpec`, `WriteRoleDocumentWithInheritedRoles`, `WriteInheritedRolesToArray`) to parse rolesInfo parameters and construct role documents with inherited roles and optional privileges. - `oss/pg_documentdb/src/test/regress/expected/role_crud_commands.out` and `pgmongo/src/test/regress/expected/role_crud_commands.out`: Expanded test suites with numerous rolesInfo scenarios, error conditions, and inheritance verification tests. - `oss/pg_documentdb/include/roles.h`: Introduced new types and function prototypes to support rolesInfo and role CRUD operations. - `pgmongo/src/pgmongo.c`: Updated query string definitions to handle role name substitutions for OSS compatibility. - `oss/pg_documentdb/src/commands/users.c`: Modified role checking and privilege extraction utilities to align with the new rolesInfo functionality. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4552244 Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Deepak Bisht
|
0f390b2cbb |
Merged PR 1798239: Fix Segmentation faults in ordered aggregate
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) BugFix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description - For aggregates such as `$last` or `$first` where the total number of aggregate values is 1 and existing aggregate state is more than the space required we try to use the same memory space while also updating it on the go. This will overwrite some later referred variable from the state. - The changes also fixes a memory leak in ordered aggregates where the new transition state was allocate in top level aggregate context and PG makes a copy and discards the original value (without freeing it), now we allocate the new transition state only in the CurrentContext which is the per tuple execution context So always create a new memory space for transition in ordered aggregates ---- #### AI description (iteration 1) #### PR Classification This pull request fixes a bug causing segmentation faults in ordered aggregate operations involving the `$last` operator used with the `$setWindowFields` stage. #### PR Summary This update resolves memory issues by removing unsafe byte array reuse in the state serialization for ordered aggregates and enhances regression tests to verify correct behavior. - `oss/pg_documentdb/src/aggregation/bson_sorted_accumulator.c`: Modified `SerializeOrderState` to always allocate new memory and removed the old byte array reuse logic. - `oss/pg_documentdb/include/aggregation/bson_sorted_accumulator.h`: Updated the function signature of `SerializeOrderState` to reflect the new implementation. - Test files in `oss/internal/pg_documentdb_distributed/src/test/regress` and `pgmongo/src/test/docdb_compat`: Added new tests for `$last` to check ordered aggregate functionality with `$setWindowFields`. - Documentation files (`docs/release_notes/pgmongo_1.108.md` and `oss/CHANGELOG.md`): Updated release notes and changelog to document the segmentation fault fix. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4687821 Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
32b11ab18f |
Merged PR 1799781: [Reindex] Support internally handling Reindex with background index processor
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Add infrastructure to handle REINDEX CONCURRENTLY using hte background index queue. This doesn't expose any new public API at this time, and is meant as a first pass to enable administrators to enqueue reindex concurrently in the background. Progress can be tracked by currentOp ---- #### AI description (iteration 2) #### PR Classification This PR introduces a new feature that adds asynchronous reindex operations using the background index processor. #### PR Summary This update extends the background index processor to support reindexing by implementing new C functions and updating index queue management, SQL UDF definitions, and tests. Key changes include: - `src/commands/create_indexes_background.c`: Added `command_reindex_index_background` and `command_reindex_index_background_internal` functions and refactored common command core logic. - `src/metadata/index.c`: Modified index queue functions and metadata handling to support the new REINDEX command type. - `sql/udfs/index_mgmt/create_index_background--0.108-0.sql`: Added new SQL UDFs for reindexing indexes. - Test files under `src/test/regress` and `/pgmongo/src/test/regress`: Updated to validate the background reindexing functionality. - Minor updates in logging and distributed hook functions to accommodate reindex operations in the background processor. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
c02bc5ac46 |
Merged PR 1798510: [DocumentDB] Add basic validation for pg18
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Refactor ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Add runs for documentdb_core tests, and documentdb check minimal to validate extension startup with pg18. Fix the issue with pgvector upgrade. ---- #### AI description (iteration 3) #### PR Classification This pull request implements a new feature by adding basic validation and compatibility enhancements for pg18 support in DocumentDB. #### PR Summary The changes update test validations, pipeline configurations, and code conventions to ensure proper pg18 behavior and compatibility. Highlights include: - **`pgmongo/src/test/regress/sql/extension_upgrade_vector.sql` & expected output**: Replace `\dx vector` commands with SQL queries that verify the pgvector extension version. - **`oss/internal/pg_documentdb_extended_rum/src/rumutil.c` and `pgmongo_rum/src/rumutil.c`**: Refactor rum options parsing by introducing a constant (`offsetIfDefault`) instead of hardcoded `-1` values. - **Pipelines (`.pipelines/templates/oss_documentdb_build.yml`, `.pipelines/templates/common_variables.yml`, `.pipelines/pgmongo_backend_test.yml`)**: Add a new parameter (`runMinimalTests`) and update container image references to support minimal OSS backend tests for pg18. - **`pgmongo/src/test/regress/Makefile`**: Modify dummy pgvector version file generation to dynamically select the latest available version. - **`oss/pg_documentdb/src/background_worker/background_worker.c`**: Remove the `WL_POSTMASTER_DEATH` flag from WaitLatch calls to refine background worker dependency handling. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
fc4fc65237 |
Merged PR 1797043: [DocumentDB] Part 1: Enable builds of DocumentDB for pg18
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Refactor ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? no ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Support building documentdb for pg18 This sets up DocumentDB for PG18 builds. Note that execution and tests are not validated yet (just compilation). Also bumps up extension dependencies to support the builds of pg18. ---- #### AI description (iteration 7) #### PR Classification This pull request adds PG18 support by implementing the necessary compatibility and feature changes in DocumentDB. #### PR Summary The update enables DocumentDB builds for PG18 by adapting ICU collation handling, updating version and pipeline configurations, and incorporating compatibility macros and API adjustments across the codebase. Key changes include: - **`oss/pg_documentdb_core/src/collation/collation.c`**: Introduced new ICU converter functions and updated collation conversion logic. - **`oss/scripts/setup_versions.sh` and pipelines**: Adjusted dependency versions and container build configurations to support PG18. - **`oss/pg_documentdb_core/include/utils/documentdb_pg_compatibility.h`**: Added a new compatibility header with macros to bridge PG18 API differences. - **Extended RUM modules (`oss/internal/pg_documentdb_extended_rum` and `pgmongo_rum`)**: Updated type definitions (e.g., replacing `PostingItem` with `RumPostingItem`) and switched from `GinStatsData` to `RumStatsData`. - **Various DocumentDB source files**: Modified executor calls, background worker logic, and tuple attribute access to comply with PG18’s API changes. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
 Jia Hu
|
2a8a93311a |
Merged PR 1554321: [json schema] support keyword required
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md)) ### Description support keyword `required` in `$jsonSchema` ---- #### AI description (iteration 1) #### PR Classification New feature #### PR Summary This pull request adds support for the `required` keyword in JSON schema validation. - Added `ParseRequired` function to handle the `required` keyword in `bson_json_schema_tree.c`. - Updated `BuildSchemaTreeCoreOnNode` to call `ParseRequired` when the `required` keyword is encountered. - Modified `ValidateBsonValueObject` in `bson_json_schema_validator.c` to validate required fields. - Added test cases for the `required` keyword in multiple test files (`bson_dollar_ops_json_schema_query_tests.out`, `bson_dollar_ops_json_schema_build_tree_tests.out`). - Updated `ValidationsObject` structure in `bson_json_schema_tree.h` to include a `required` field. Related work items: #4638803 Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Vinod Sridharan
|
8a54698ec0 |
Merged PR 1797301: Fix up param usage paths for point reads
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Bugfix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Fix up param usage paths for point reads ---- #### AI description (iteration 1) #### PR Classification Bug fix: This change corrects parameter usage paths in point read operations. #### PR Summary This PR refines how outer relation IDs are computed for primary key lookups by replacing outdated outer_relids with clause_relids and merging lateral and clause members. - In `oss/pg_documentdb/src/opclass/index_support.c`, the code now uses `bms_copy` and `bms_add_members` for combining IDs, removes redundant union logic, and under PG versions below 16, ensures empty ID sets are set to NULL. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
Cornelius Boateng
|
ce0cb575be |
Merged PR 1794050: [engine][collation] Support for $sortArray
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Support for the `$sortArray` aggregation operator. ---- #### AI description (iteration 1) #### PR Classification This pull request implements collation support for the $sortArray operator as a new feature. #### PR Summary This change integrates collation awareness into the $sortArray operator so that array sorting properly respects collation rules. The modifications update both the underlying sorting logic and the tests to reflect these new capabilities. - `pg_documentdb/src/operators/bson_expression_array_operators.c`: Updated parsing of the $sortArray operator to copy the collation string into the sort context. - `pg_documentdb/src/utils/sort_utils.c` and `pg_documentdb/include/utils/sort_utils.h`: Switched to collation-aware comparison functions by replacing calls to CompareBsonValueAndType with CompareBsonValueAndTypeWithCollation. - Test expectation files under `/oss/internal/pg_documentdb_distributed/` and `/pgmongo/` were updated to include new $sortArray outputs with different collation settings. - Integration CSV files were modified to mark the previously failing `$sortArray` tests as passing. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Signed-off-by: Shuai Tian <shuaitian@microsoft.com> |
||
|
William Wang
|
5a37471d7d |
Merged PR 1784924: Utility File and Query Refactoring for Role/User CRUD APIs
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description ---- #### AI description (iteration 1) #### PR Classification This PR refactors and centralizes role/user CRUD utility functions. #### PR Summary The changes migrate and consolidate the logic for user and role management into a new utility module and header file, streamlining the codebase and removing redundant implementations from command files. - `src/utils/role_utils.c` adds core functions for password hashing, privilege consolidation, and user/role document writing. - `include/utils/role_utils.h` provides declarations for the newly centralized role/user utilities. - `src/commands/users.c` and `src/commands/roles.c` are cleaned up by removing duplicated role/user functionality now handled in the new module. - `pgmongo.c` and related metadata files are updated to use the new role replacement variable and adjust related constants. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4639026, #4639027 |
||
|
Parag Jain
|
5676ead634 |
Merged PR 1788923: [Cleanup] code cleanup as now empty data table function migrated to 3 arguments
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description [Cleanup] code cleanup as now empty data table function migrated to 3 arguments ---- #### AI description (iteration 1) #### PR Classification This pull request performs code cleanup by removing obsolete logic around the empty data table function and standardizing its use with three arguments. #### PR Summary The changes simplify the code in `oss/pg_documentdb/src/aggregation/bson_aggregation_pipeline.c` by inlining the call to `BsonEmptyDataTableFunctionId()`, removing dynamic retrieval of the function's result type, and directly setting the column count to 3. - `oss/pg_documentdb/src/aggregation/bson_aggregation_pipeline.c`: Inlined the function call to `BsonEmptyDataTableFunctionId()`, eliminated the temporary variable and associated dynamic result type logic, and removed the conditional check for a fourth column. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Deepak Bisht
|
2570b9c884 |
Merged PR 1788297: [OSS] Fix concurrent upsert bug, reapply the update spec on conflict while instead of replacing with new document
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Bugfix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) BugFix, data loss in concurrent upserts. ### Description On conflict for upserts we were replacing the document with newer document which is not the right behavior, we need to apply the second update on top of the first document for upserts. ---- #### AI description (iteration 1) #### PR Classification Bug fix: Resolve a concurrent upsert issue by reapplying the update spec on conflict instead of replacing the document. #### PR Summary This pull request corrects the upsert behavior by modifying the queries and function interfaces to reapply the update specification. It also adds new tests to validate the corrected concurrent upsert logic. - `oss/pg_documentdb/src/commands/insert.c` and `oss/pg_documentdb/include/commands/insert.h`: Modified the upsert SQL query to use `bson_update_document` and updated the function signature to require an additional update document parameter. - `oss/pg_documentdb/src/commands/update.c`: Updated calls to the upsert logic to pass the update spec, ensuring the update is reapplied on conflict. - `oss/pg_documentdb/src/update/bson_update.c`: Added logic to build update metadata when none is provided. - `pgmongo/src/test/regress/specs/isolation-concurrent-upserts.spec` and the corresponding expected output file: Introduced new tests and updated the test schedule to verify the concurrent upsert behavior. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4644372 |
||
|
Deepak Bisht
|
2044bce7a2 |
Merged PR 1786924: Fetch the TTL index entries with desc sort order behind a GUC
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Refactoring ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes, `documentdb.enableTTLDescSort` fetches the TTL documents on descending sort order ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description - Fetch the TTL index entries with desc sort order behind a GUC - Adds an ORDER BY expression in the SELECT query for potential documents to be deleted by TTL ---- #### AI description (iteration 1) #### PR Classification This pull request introduces a new feature that enables fetching TTL index entries in descending sort order controlled by a new GUC. #### PR Summary The changes add support for descending TTL sort ordering and update related regression tests to verify the behavior. - `oss/pg_documentdb/src/ttl/ttl_index.c`: Adds logic to append a descending ORDER BY clause when `EnableTTLDescSort` is true and the index is ordered. - `oss/pg_documentdb/src/configs/testing_configs.c`: Defines and registers the new GUC variable for enabling TTL descending sort. - Test output files in both `oss/internal/pg_documentdb_distributed/src/test/regress/expected/commands_create_ttl_indexes.out` and `/pgmongo/src/test/docdb_compat/expected/commands_create_ttl_indexes.out`: Updated to include commands and expected outputs for the descending TTL ordering. - `oss/internal/pg_documentdb_distributed/src/test/regress/sql/commands_create_ttl_indexes.sql`: Modifies SQL regression tests to incorporate the descending TTL ordering scenario. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4644360 |
||
|
Cornelius Boateng
|
7492f14676 |
Merged PR 1754062: [engine] Enable EnableNowSystemVariable & EnableVariablesSupportForWriteCommands GUCs
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Automatically support `$$NOW` for find and aggregate queries and all variables for delete, update, findAndModify queries. ---- #### AI description (iteration 1) #### PR Classification This pull request deprecates and removes the `EnableNowSystemVariable` GUC to simplify time variable handling in the engine. #### PR Summary The changes eliminate conditionals based on `EnableNowSystemVariable` and update related function signatures to always process time system variables uniformly. - **`oss/pg_documentdb/src/operators/bson_expression.c`**: Removed the `isWriteCommand` parameter and flag-based checks, so that the current time value is always appended without referencing `EnableNowSystemVariable`. - **`oss/pg_documentdb/src/commands/*` and `oss/pg_documentdb/src/aggregation/*`**: Replaced conditional logic dependent on `EnableNowSystemVariable` with direct checks for valid time system variable values. - **`oss/pg_documentdb/src/configs/feature_flag_configs.c` and header files**: Deleted the definition, default value, and extern declarations for `EnableNowSystemVariable`. - **Configuration scripts and test files**: Removed the now redundant `enableNowSystemVariable` settings from various configuration and test files. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4287488 |
||
|
Cornelius Boateng
|
d90748a718 |
Merged PR 1784757: [fix] Fix Crash Due to NULL variableSpec and Valid Collation
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Bugfix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Internal functions (`*WithLetAndCollation`) that accept both variableSpec and collationString must pass a `null const` value in place of variableSpec if the variableSpec is NULL, with a valid collation string. ---- #### AI description (iteration 1) #### PR Classification Bug fix to prevent a crash when an empty variableSpec is provided together with a valid collation. #### PR Summary This pull request fixes a crash by properly handling NULL/empty variableSpec in conjunction with collation in both the core aggregation functions and nested pipelines. Highly important changes include: - **`pg_documentdb/src/aggregation/bson_aggregation_pipeline.c`**: Updated the logic to create and pass collation and variableSpec constants safely, avoiding null dereferences. - **`pg_documentdb/src/aggregation/bson_aggregation_nested_pipeline.c`** and **`pg_documentdb/src/query/query_operator.c`**: Adjusted handling of variableSpec and collation parameters in nested pipeline and query operator functions. - **Test files (e.g., in `/oss/internal/pg_documentdb_distributed/src/test/regress/expected/bson_dollar_ops_collation_tests_runtime.out` and `/pgmongo/src/test/docdb_compat/expected/bson_dollar_ops_collation_tests_runtime.out`)**: Added session settings and BEGIN/ROLLBACK blocks to disable auto $$NOW generation and to validate collation-dependent queries. - **Documentation and changelog**: Updated release notes and the CHANGELOG to document the bug fix regarding the empty variableSpec with collation. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
William Wang
|
26ac331188 |
Merged PR 1758648: Adding Support for DropRole Command in Engine
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) 1.107 ### Description This PR adds the dropRole command to Engine. ---- #### AI description (iteration 1) #### PR Classification New feature: implementing support for the dropRole command. #### PR Summary This pull request adds the dropRole command to the engine, enabling deletion of custom roles with proper validations and dependency checks. It includes comprehensive tests and updates to documentation and feature tracking. - `oss/pg_documentdb/src/commands/roles.c`: Added functions to parse dropRole specifications and execute the drop operation with dependency and error handling. - `oss/pg_documentdb/include/roles.h` and feature counter files: Updated interfaces and feature mappings to expose dropRole support. - Test files under `oss/pg_documentdb/src/test/regress` and `pgmongo/src/test/regress`: Introduced extensive regression tests for dropRole, covering both success scenarios and various error cases. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4333807 |
||
 Jinrui Zhang
|
3ccd656663 |
Merged PR 1777067: [Refector] Update to match coding standards - part 0
### Description 1. Fixing issues identified by similarity checker 2. Fix the corresponding sql unit tests 3. Fix JSTests failure due to above changes 4. Fix corresponding C# test failure ---- #### AI description (iteration 1) #### PR Classification This pull request is a code cleanup that standardizes and improves error message wording across the codebase. #### PR Summary The changes update error messages throughout the repository to provide clearer, more consistent, and more detailed feedback when operations fail. - Updates in geospatial operator files (e.g. in `$geoNear`, `$near`, `$nearSphere`) replace informal phrasing with precise messages. - Numerous error messages in aggregation and projection test files have been reworded for consistent phrasing <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 Siwen Tang
|
be11798bef |
Merged PR 1778507: [Refector] Update to match coding standards - part 5
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description draft update error msg ---- #### AI description (iteration 1) #### PR Classification This is a code cleanup and bug‑fix pull request that standardizes and clarifies error messages and log details across multiple modules. #### PR Summary The changes improve the consistency, clarity, and accuracy of user‑facing messages and logging throughout the code base. Key changes include: - In files under directories such as `oss/internal/pg_documentdb_distributed/src/test/regress/expected/` and `pgmongo/src/test/docdb_compat/expected/`, error messages for geospatial query operators (e.g. geoNear, $center, $within) and aggregation stage operators have been rewritten for clarity (e.g. replacing vague “Too many geoNear expressions” with “Excessive number of geoNear query expressions” and clarifying numeric parameter validations). - Several operators’ messages (e.g. `$cond`, `$not`, `$abs`, `$floor`, `$divide`, `$log`, etc.) have been updated to state exactly how many arguments are required, using <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 Xing Fan
|
3b6061e6ea |
Merged PR 1777085: [Refector] Update to match coding standards - part 6
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description draft ---- #### AI description (iteration 1) #### PR Classification Bug fix: This PR revises error messages and inline comments to improve clarity and consistency across the code and test outputs. #### PR Summary The changes refactor and update a large number of error messages across various modules—ranging from aggregation and update operators to geospatial, schema validation, and command handling—to provide more precise diagnostic feedback. - **Tests (e.g. in `tests/expected/... <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 Duan Pei
|
6a129514a9 |
Merged PR 1777887: [Refector] Update to match coding standards - part 1
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description update errmsg ---- #### AI description (iteration 1) #### PR Classification This pull request focuses on updating and standardizing error messages across various operators, aggregation stages, and test expectation files. #### PR Summary The changes reword error messages throughout the codebase and test outputs to provide clearer, more consistent, and more informative feedback for users. Key updates include: - **`/oss/internal/pg_documentdb_distributed/src/test/regress/expected/*.out`** – Revised expected error outputs for aggregation, update, and geospatial tests to use clearer phrasing (e.g., for `$merge`, `$pop`, `$indexOfArray`, and `$geoNear`). - **Core operator files (e.g., `bson_aggregation_output_pipeline.c`, `bson_expression_arithmetic_operators.c`, `bson_expression_conditional_operators.c`)** – Updated error message texts to explicitly state operator requirements and types. - **Geospatial and vector modules (e.g., files under `bson_geojson_utils.c`, `bson_geospatial_*`, and vector search/index configuration files)** – Standardized validations and error reporting for GeoJSON, CRS, and vector search parameters. - **Additional modules (e.g., collection view creation and user management files)** – Improved error handling messages to better guide users on input misconfigurations. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Jia Hu
|
7db51ff43b |
Merged PR 1777927: [Refector] Update to match coding standards - part 2
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) similarity issue ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description ---- #### AI description (iteration 1) #### PR Classification This pull request is a code cleanup aimed at standardizing and clarifying error and diagnostic messages throughout the codebase. #### PR Summary The changes update error messages and logging across many modules (aggregation, query, update, index creation, collation, etc.) to use clearer, more precise, and consistent language. - **Aggregation Operators:** Many files in the aggregation code (e.g. in `bson_aggregation_*` and related operator implementations) now report errors such as division by zero, invalid inputs, and argument count in a formal and uniform manner. - **Index & Projection Validation <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 Yang Qiao
|
7f0c061b83 |
Merged PR 1778033: [Refactor] Update to match coding standards - part 4
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Testing ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Update to match coding standards ---- |
||
|
Yang Qiao
|
f9cbecf598 |
Merged PR 1777990: [Refector] Update to match coding standards - part 3
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Testing ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description Fix issues identified by similarity checker ---- #### AI description (iteration 1) #### PR Classification This pull request is a bug fix and code cleanup that standardizes and clarifies error messages across aggregation, query, and index creation operators. #### PR Summary The changes update error text throughout the codebase to be more explicit and consistent regarding type validations and operator requirements. Notable updates include: - Updates to error messages in aggregation operator files (e.g. in `bson_expression_bitwise_operators.c`, `bson_expression_conditional_operators.c`, and similar files) to clearly specify expected input types and constant requirements. - Rewording of messages for operators such as `$regexFindAll`, `$merge`, `$switch`, `$ifNull`, and trigonometric functions to use precise language (e.g. “requires” or “must be provided” instead of “only supports”). - Improved clarity in messages for view cycle detection (e.g. in `create_indexes.c` and view creation tests) and hashed index validations. - Consistent rephrasing for messages in densify and percentile operators to better indicate numeric or date constraints and limits. - Refinements in error reporting for operators in type conversion and array handling (e.g. in `$arrayElemAt`, `$slice`, `$getField`, and `$unsetField`), ensuring that expected value types (such as string, numeric, or object) are explicitly stated. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
Parag Jain
|
fa57f40720 |
Merged PR 1779662: [Observability] advanced feature usage for write commands
### Does this PR have any customer impact? ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) ### Are you introducing any new config? If yes, do you have tests with and without them being set? ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description [Insert] Feature Usage Insert ---- #### AI description (iteration 1) #### PR Classification New feature addition that implements feature usage tracking for insert and update commands. #### PR Summary This pull request adds inline functions to report feature usage based on batch sizes for insert and update operations and updates the associated feature enum definitions. - `/oss/pg_documentdb/src/commands/insert.c`: Added `ReportInsertFeatureUsage` to report usage metrics based on batch size and integrated its call during batch insertion processing. - `/oss/pg_documentdb/src/commands/update.c`: Introduced `ReportUpdateFeatureUsage` with similar threshold-based reporting, integrated into the batch update process. - `/oss/pg_documentdb/include/utils/feature_counter.h`: Extended the feature counter enums with new constants for detailed insert and update reporting, and increased `MAX_FEATURE_COUNT`. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
|
William Wang
|
665098c897 |
Merged PR 1761487: Adding Support for RolesInfo Command in Engine (1/2, RolesInfo Parsing)
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) 1.107 ### Description We are implementing the RolesInfo command in Engine with three separate PRs. The first one will include the parsing of the input, the second one will be the core implementation of the command, and the third one will be the implementation of showBuiltInRoles. ---- #### AI description (iteration 1) #### PR Classification This pull request implements a new feature by adding support for the rolesInfo command in the engine. #### PR Summary This update introduces functions to parse and handle the rolesInfo command, supporting various input formats and validations while updating error messages and tests. - **`roles.c`**: Added new parsing functions (`ParseRolesInfoSpec`, `ParseSingleRoleFromArray`, and `ParseRoleDocument`) and integrated distributed query logic in `roles_info`. - **`command_roles_info` & `create_role`**: Updated command wrappers and role validation to use the new logic and the `IS_SUPPORTED_BUILTIN_ROLE` macro. - **`roles.h`**: Introduced the `RolesInfoSpec` structure and defined the `IS_SUPPORTED_BUILTIN_ROLE` macro. - **Test Files**: Extended regression tests and expected outputs in both DocumentDB and Helio API test suites to cover multiple rolesInfo scenarios and error cases. - **`users.c` & Feature Counters**: Removed the redundant built-in role support function and added a new feature counter for rolesInfo on built-in roles. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4333805 |
||
|
Deepak Bisht
|
e8c5686c8c |
Merged PR 1775218: [Debugging] Add PG query sourcetext behind a config
### Does this PR have any customer impact? No ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Debugging ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? Yes. `documentdb.enableDebugQueryText` to enable query text for debugging ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description This PR adds a test configuration `documentdb.enableDebugQueryText` that starts deparsing the query tree and set it as source text either in QueryDesc if run via executor or source text in the PlannedStmp if run via Portal. This will enhance our ability to have deeper insights on the query that results in an unexpected behavior or dump. NOTE: This should be used **only and only** for debugging purposes, as it starts deparsing all the queries which will take a performance hit. ---- #### AI description (iteration 1) #### PR Classification This pull request implements a new debugging feature to conditionally capture and display PG query source text during query execution. #### PR Summary The changes update multiple query execution functions to extract the query definition via pg_get_querydef when the EnableQueryText flag is enabled, and pass this source text for debugging purposes. - `oss/pg_documentdb/src/commands/cursors.c`: Modified functions to conditionally retrieve the query source text and pass it to executors and portal definitions, including updating the DrainStatementViaExecutor signature. - `oss/pg_documentdb/src/configs/system_configs.c`: Added a new custom Boolean configuration variable (EnableQueryText) with a default value and appropriate documentation. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> Related work items: #4578637 |
||
|
Vinod Sridharan
|
c7613604d7 |
Merged PR 1777606: [Bugfix] Fixes for point read planner and lookup
### Does this PR have any customer impact? Yes ### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Bugfix ### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) No ### Are you introducing any new config? If yes, do you have tests with and without them being set? No ### ChangeLog (Refer [Template](../oss/CHANGELOG.md)) ### Description For $lookup we were ignoring the joinquals for point read pushdown - this happened to work before if the planner chose a point read plan before. But now that we're more explicit in looking for this we need to handle this explicitly. Also ensure point read plan generation considers outer_relids (this wasn't done before). ---- #### AI description (iteration 1) #### PR Classification Bugfix: This PR corrects issues in the point read planner and lookup functionality. #### PR Summary The changes fix bugs related to index selection and query planning for multi-layer lookups by improving how join and restriction information is handled, and by extending test coverage for point read scenarios. - **`oss/pg_documentdb/src/opclass/index_support.c`**: Updated handling of outerRelids and join restrictions to ensure proper index path identification. - **`oss/pg_documentdb/src/planner/documents_planner.c` and `oss/pg_documentdb/include/opclass/bson_index_support.h`**: Modified calls to pass join info into restriction path processing for accurate planning. - Test files under **`oss/internal/pg_documentdb_distributed`** and **`pgmongo`**: Added new multi-layer lookup queries and expected outputs to verify that lookups consistently pick the _id index. <!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot --> |
||
 CosmosDB Build Service (msdata)
|
72fb39d9a9 |
Merged PR 1775074: [tagv.1.108] Update extension version for new release 1.108
This PR updates the extension version for the new release 1.108. Changes included: - Control files updated to version 1.107-0 - SQL files generated for version upgrades - Release notes file created for version 1.108 - TOC.yml updated with new release notes entry - CHANGELOG.md updated with new version entry |