Commit Graph
54 Commits
Author SHA1 Message Date
Vinod Sridharan 2d8ec7f2db Merged PR 1441131: [Helio compat] Making more compat tests pass
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Test
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

Making more compat tests pass
2024-08-28 16:38:56 +03:00
Xing Fan ⛳ ce4a709922 Merged PR 1412546: Window operators: $covariancePop & $covarianceSamp
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
UDF

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Implemented covariancePop & covarianceSamp for setWindowFields

----
#### AI description  (iteration 1)
#### PR Classification
Documentation

#### PR Summary
This pull request adds design documentation for the `$covariancePop` and `$covarianceSamp` aggregation operators.
- Added `covariancePop.md` detailing the specification, behavior, examples, implementation plan, and test plan for the `$covariancePop` operator.
- Added `covarianceSamp.md` detailing the specification, behavior, examples, implementation plan, and test plan for the `$covarianceSamp` operator.
2024-08-26 13:12:13 +03:00
Vinod Sridharan 76eddc46c8 Merged PR 1437491: [OSS} Port the first 4 tests to distributed
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Port 4 tests to helio_distributed and clean up the code to make them pass.

----
#### AI description  (iteration 1)
#### PR Classification
Porting initial tests to a distributed environment.

#### PR Summary
This pull request ports the first four tests to a distributed environment, ensuring that the query evaluation functions work correctly in this setup.
- Added distributed test cases for various BSON expression evaluations in `src/test/regress/expected/bson_expr_eval_tests.out`.
2024-08-26 13:12:13 +03:00
Deepak Pemmaraju 009b69d31e Merged PR 1400474: Adding code for CreateUser, GetUsers, UpdateUser and DropUser
### Does this PR have any customer impact?
It exposes a new API customers can use

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Adding code for CreateUser, UpdateUser and DropUser

Related work items: #3264072, #3278392, #3292323, #3292361, #3292364, #3315917
2024-08-26 13:12:13 +03:00
Vinod Sridharan f51b8b04ec Merged PR 1433853: [OSS] Prep for porting backend distributed tests
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactor

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Prep for porting backend distributed tests

----
#### AI description  (iteration 1)
#### PR Classification
Code cleanup and preparation for porting backend distributed tests.

#### PR Summary
This pull request removes obsolete code and prepares the codebase for porting backend distributed tests.
- Deleted `cluster_operations.c` and `version_utils.c` from `pgmongo/src`.
- Added new functions and triggers in `helio_distributed_test_helpers.sql` and `helio_distributed_test_helpers.out` for testing.
- Updated `cluster_operations.c` in `pg_helio_distributed` to include new cluster management functions.
- Added new SQL files for cluster operations and version utilities in `pg_helio_distributed`.
- Modified `pgmongo.c` to set up global variables for distributed functions and index build queue table creation.
2024-08-26 13:12:13 +03:00
Parag Jain 90c74bcb94 Merged PR 1412316: [Feature] Merge Index Push down for join condition Part-3
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

----
#### AI description  (iteration 1)
#### PR Classification
New feature: Adds support for index pushdown in join conditions for the `$merge` stage in aggregation pipelines.

#### PR Summary
This pull request introduces index pushdown capabilities for join conditions in the `$merge` stage, enhancing query performance.
- `src/opclass/index_support.c`: Added support functions for index pushdown in `$merge` join filters.
- `sql/udfs/aggregation/bson_merge_functions--1.20-0.sql`: Created new SQL functions to support index pushdown and handle merge operations.
- `src/aggregation/bson_aggregation_output_pipeline.c`: Implemented functions to create join expressions and extract merge filters.
- `Tests.Postgres.CSharp/Integration/Aggregation/MergeTests.cs`: Added integration tests to verify index pushdown functionality.
- `src/metadata/metadata_cache.c`: Added function OIDs for new merge-related functions.
2024-08-16 18:24:51 +03:00
Manish Gaur 2fff0dca08 Merged PR 1424268: Version update to start pgmongo 1.20 release
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

Version update to start pgmongo 1.20 release

----
#### AI description  (iteration 1)
#### PR Classification
Version update

#### PR Summary
This pull request updates the version numbers and adds release notes for the pgmongo 1.20 release.
- Added release notes in `docs/release_notes/pgmongo_1.21.md`
- Updated version numbers in `pg_helio_api.control`, `pg_helio_core.control`, and `pgmongo.control`
- Added SQL migration scripts for version updates in `pg_helio_api`, `pg_helio_core`, and `pgmongo`
- Updated table of contents in `docs/release_notes/toc.yml`
- Updated SHA1 checksums for SQL scripts in `pgmongo/sql/sha1`
2024-08-16 18:24:51 +03:00
Deepak Bisht b08ac5b39e Merged PR 1402307: Support $avg and $push in $setWindowFields stage with inverse transition
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
- Add support for $avg, $push in $setWindowFields stage.
- Add inverse transition functions for $sum, $avg, $push

### Description
Add the inverse transition function for $sum, $avg and $push for optimal window aggregation.

Also for `$push` it is not currently usable in windowing because of its specific state requirement, need to add another state for window operation for `$push`
2024-08-16 18:24:51 +03:00
Xiaochun Liang 622d392bf2 Merged PR 1376029: stdDevPop and stdDevSamp accumulator for group by aggregation
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Implementation of $stdDevPop and $stdDevSamp accumulators.

Reference:
[stdDevPop](https://www.mongodb.com/docs/manual/reference/operator/aggregation/stdDevPop/)
[stdDevSamp](https://www.mongodb.com/docs/manual/reference/operator/aggregation/stdDevSamp/)

Related work items: #3277074
2024-08-16 18:24:51 +03:00
Vinod Sridharan f64558301b Merged PR 1411883: [Lookup with Let] Fix up some JStests for lookup with Let and misc lookup fixes
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes and Yes
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
- Fix up bugs with $lookup with Let by rewriting lookup to be let friendly
- Get through jstests having lookup with let
-  Optimize Lookup where hte left and right both are on _id to support joins directly on object_id
- Allow the lookup pipeline to be split as "inlined" and "non-inlined" and allow pushing up the inlined pipeline
- Fix bugs with facet/graphlookup arising from Lookup with let tests
- Allow inlining the $unwind path after a $lookup if the $unwind matches a $lookup as (behind a feature flag)
2024-07-31 17:09:22 +03:00
Manish Gaur 65380e5677 Merged PR 1404339: pgmongo version update for 1.19 release - Round 2
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

pgmongo version update for 1.19 release - Round 2
2024-07-31 17:09:22 +03:00
Vinod Sridharan c4ed06beeb Merged PR 1402545: [Let] Support let in all remaining stages except $lookup
### Does this PR have any customer impact?
Not yet

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Features

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Support remaining aggregation stages except $lookup and $mergeObjects on sorted with $group.
2024-07-31 17:09:22 +03:00
Vinod Sridharan 3cbc0e0037 Merged PR 1400815: [Let support] Add support for let in more stages
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Features
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
n/a
### Description
Add support for let in
- $sortByCount
- $group (partial)
- $addFields
- $project
- $facet
- $unionWith
- $graphLookup
- $project
2024-07-31 17:09:22 +03:00
Deepak Bisht 91b9946227 Merged PR 1383926: setWindowFields aggregation stage
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
- Support for `$setWindowFields` aggregation pipeline stage

### Description
This PR adds the support for `$setWindowFields` aggregation stage using the Postgres `Window aggregation` operators.

For the first PR it only adds support with `$sum` aggregation operator.
Next set of PRs will individually support all the other window aggregation operators supported by `$setWindowFields` stage.

[TODO] Currently $sum is used as is (the way $group uses it), there is way to optimize this using the `Inverse Transition` functions for Window aggregation which modifies the aggregated state in a way to accomodate rows leaving the group in a moving window, which will be done later (this is done to make review on smaller set of files)

Related work items: #2272584
2024-07-11 17:56:53 +03:00
Vinod Sridharan 65a0cf4a46 Merged PR 1399519: Let support: Part 1: Add support for let in find/aggregate
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Add support for `let` in `find` and `aggregate`
- For find ensure that let can be supported both in projection and query.
- For aggregate, so far this is only supported in `$match`.

In subsequent PRs other stages will be reviewed and modified.

Note that this is a precursor to $lookup with let since Lookup's inner pipeline can reference any of those stages that can use `let`

Since this is a partial feature this is guarded with a feature flag.
2024-07-11 17:56:53 +03:00
Parag Jain 0922eac1d7 Merged PR 1391214: [Feature] support for $merge aggregation stage part -2
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
No

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
support of `merge` Action in case of `whenMatched` in `$merge` aggregation stage.

Related work items: #1827100
2024-07-11 17:56:53 +03:00
Deepak Pemmaraju f3b17295f5 Merged PR 1381452: changing schema for user CRUD
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

changing schema for user CRUD
2024-07-05 13:16:04 +03:00
Parag Jain 833980da3f Merged PR 1344998: [Feature] support for $merge aggregation stage part -1
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
No
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
NA
### Description:

**Things this PR has:**
1. support for $merge stage for unsharded collection.
- whenNotMatched : support for `fail` , `discard` and `insert` option.
- whenMatched : support for `replace`, `fail` and `keep-existing` option.

**What next In upcoming PR's :**
1. support for 'merge' option whenMathced
2. support for 'pipeline' option whenMatched.
3. support for sharded collection.
4. support of co-located collection. (one case : when source and taget is same)
5. Index push down for join condition.
6. optimizations in case of sharded collection and unsharded collection for single aggregation stage when we can directly refer source shard_key_column.

Related work items: #3085830
2024-07-05 13:16:04 +03:00
Santiago Fernandez Madero ebf487f2a4 Merged PR 1370848: Bump main to 1.19
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

Bump main to 1.19
2024-06-21 18:21:38 +03:00
Deepak Pemmaraju c44eb4ae23 Merged PR 1359979: Adding a read only role in the backend
### Does this PR have any customer impact?
By default, no. Not unless they choose to use the read only role.

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
New Feature

### Description
Adding a readonly role in the backend. The goal is to enable native readonly users in Mongo VCore

Related work items: #3141137, #3158558
2024-06-21 18:21:38 +03:00
Lucas Borges Fernandes 025a9452dc Merged PR 1321307: Merge Objects accumulator for group by aggregation
### Description

Implementation of the $mergeObjects accumulator.

Reference: https://www.mongodb.com/docs/manual/reference/operator/aggregation/mergeObjects/

Related work items: #2988409
2024-06-21 18:21:38 +03:00
Deepak Bisht 05c3a40f06 Merged PR 1316036: [GeoNear] Part 2 - GeoNear JSTest infra and added new distance range operator that is pushed to indexes
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
Added support for $geoNear aggregation stage pipeline.

### Description
Part 2  for supporting geoNear stage

This fixes following TODOs from the last PR:

1. Push `$minDistance` and `$maxDistance` to index similar to mongo. ✅
=================
Created a new range distance query operator `@|><|` which is supported by both `2d` and `2dsphere` indexes. $geoNear now uses this instead of float8 distance less than / greater than comparisions.

1. [TODO] Make the stage require an index always. ❌
=================
$geoNear queries always require a geo index in mongo

1. Fix projection for nested array fields. ✅
=================
$geoNear projection for `distanceField` or `includeLocs` overrides the intermediate array nodes instead of recursing into arrays and updating each element.

1. Make common test setup for running $geoNear queries in JSTests which don't require key in mongo. ✅
=================
Mongo JSTests mostly create single geo index on the collection and misses the `key` option for $geoNear stage, so we override couple of mongo functions to make sure if we are using the $geoNear stage then we can update the pipeline to contain the respective `key` field, so that tests can run without failure.

1. More testing and validations with other stages that can use $geoNear. ✅
1. Test batching and cursor support. ✅
1. Overall testing and validations ✅
1. $geoNear Tests passed: **Out of 23 test dependent on $geonear now 9 are passing**, and rest are again dependednt on $near and $nearSphere which is a subset of $geonear

Related work items: #2782870, #2782872
2024-06-03 13:12:17 +03:00
Vinod Sridharan 67519437d4 Merged PR 1362229: Fix lookup for paths that have operator like syntax
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Fix scenarios where the documents may have $prefixed nested fields.
When doing lookup this was treated as an operator instead of a field throwing an error
Introduce a bson_dollar_merge_documents which ignores operators and simply merges the documents directly.
2024-06-03 13:12:17 +03:00
Javier Figueroa 1123f4f2a2 Merged PR 1356960: Bumping schema version for 1.17 release
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
DevOps

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
No

### Description
Bumping schema version for 1.17 release
2024-05-24 10:41:23 +03:00
Lucas Borges Fernandes 533dc34b1a Merged PR 1347886: Moving $addToSet to 1.15-1
Adding $addToSet in 1.15-1.
Removing it from 1.14-7.
2024-05-13 13:23:03 +03:00
Lucas Borges Fernandes 51042e6473 Merged PR 1334895: AddToSet accumulator for group by aggregation stage
### Description
Implementation of the $addToSet group accumulator.

Reference: https://www.mongodb.com/docs/manual/reference/operator/aggregation/addToSet/

Related work items: #2988403
2024-05-13 13:23:03 +03:00
Javier Figueroa 01d68d4efa Merged PR 1343345: Commit 7e3533e6: Bumping schema for 1.17 release
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
DevOps

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
No

### Description
Commit 7e3533e6: Bumping schema for 1.17 release
2024-05-03 13:34:41 +03:00
Vinod Sridharan d4d9f5cda9 Merged PR 1333485: [Negation] Add support for pushdown of the $not operator sometimes
### Does this PR have any customer impact?
Yes

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
Adds index pushdown for $not operators
### Description
Adds index pushdown for $not operators
2024-04-26 16:19:32 +03:00
Vinod Sridharan cf88b82fe7 Merged PR 1332248: Selectivity Part 1: Implement operator statistics behind a flag
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
N/A
### Description
Implement statistics for bson Dollar functions behind a feature flag. This uses the default Postgres operator statistics using Most Common Value and Histogram stats if available to compute.

As a fallback it also uses operator data to provide default values if needed.

Sets up an ANALYZE function on bson so that this can be improved (since the ANALYZE is not there yet). This will be done in a subsequent change.
2024-04-26 16:19:32 +03:00
Vinod Sridharan 8c7173a2d9 Merged PR 1325613: Lookup Index pushdown
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Lookup Index pushdown: Allow index pushdown of lookup queries for both left and right queries.
For _id joins, push to the B-tree index and use the object_id column.
For RUM indexes, introduce a new function and support that allows pushing to the index based on the specified path rather than the document itself.

This caused a customer $lookup query to go from taking 45+ mins and timing out to finishing in under 3 mins
2024-04-19 12:03:52 +03:00
Deepak Bisht 58afe0aa8f Merged PR 1275776: [Pgmongo] - Part 1 GeoNear aggregation stage, add documentation and necessary operators and functions
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
Support $geoNear aggregation stage

### Description
Add the design doc and necessary operators and functions needed for $geoNear

Also,
1-  Implement the $geoNear aggregation stage skeleton and support all options.
     - `distanceField`,
     - `near` (both legacy, geojson point)
     -`includeLocs`,
     -`minDistance` & `maxDistance`
     - `distanceMultiplier`
     - `key`
2- Support index pushdown of the stage to `2d` and `2dspher` index.
    - spherical / non-spherical distance support for `2d` index.
    - spherical distance support for `2dsphere` index

Things still a TODO.
1- ~~Update the mongo explain plans for geonear stage~~.
2- Make the stage require an index always, currently it can be executed without index too.
3- Fix projection for nested array fields.
4- Make common test setup for running $geoNear queries in JSTests which don't require key in mongo.
5- More testing and validations with other stages that can use $geoNear.
6- Test batching and cursor support.
7- Overall testing and validations

Related work items: #3000189, #3065270
2024-04-19 12:03:52 +03:00
Mehmet Yilmaz 1151d0fcdc Merged PR 1318097: [OSS] Move coll_stats func
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
UDF

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Move coll_stats funcs to oss
2024-04-19 12:03:52 +03:00
Vinod Sridharan 2da6633812 Merged PR 1314099: Colocation Support Part 2
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
2024-04-05 13:43:51 +03:00
Santiago Fernandez Madero 2ad7381f79 Merged PR 1291973: Add inverseMatch aggregation stage
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

Add inverseMatch aggregation stage

Currently it supports a spec like:
```
{ path: <path>, input: <doc or array of docs/path expression>, defaultResult: [optional | <bool> ]}
```

So if we want to fetch the input to the document query from an external collection, a lookup stage would need to be added prior to the inverseMatch.

The next stage of this feature is to support a spec like:
```
{ path: <path>, fromCollection: <coll>, pipelines: [$match/$project/$limit], defaultResult: [optional | <bool> ]}
```
2024-03-29 10:49:37 +03:00
Vinod Sridharan 1f8b17e477 Merged PR 1311551: [Part 1: API] Reduce OSS Public API surface
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
OSS
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
N/A
### Description
Start bringing down the Public surface area with back compat for helio_api
2024-03-29 10:49:37 +03:00
Manish Gaur bb2655dcba Merged PR 1303061: Version update for next release
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
2024-03-22 11:28:50 +03:00
Lucas Borges Fernandes 2b7b1706af Merged PR 1302307: Supporting group $push in 1.13/1.14 clusters
### Description
2024-03-22 11:28:50 +03:00
Lucas Borges Fernandes ab11777a85 Merged PR 1290864: $push accumulator for $group pipeline stage
### Description
Implementation of the $push accumulator for $group pipeline stage.

Related work items: #2988400
2024-03-22 11:28:50 +03:00
Santiago Fernandez Madero a770e351e6 Merged PR 1289897: Add inverse match UDF to 1.14 schema to be able to use it sooner in prod
Add inverse match UDF to 1.14 schema to be able to use it sooner in prod

This UDF will be used for the new $inverseMatch aggregation stage which will take the document to apply the match on, the path where the document defines the query and the input value to evaluate the query against.

Since we need a UDF to do this against every document in a query like so:

```
SELECT document FROM mongo_data.documents_1_102045 collection WHERE helio_api.bson_dollar_inverse_match(document,{ "path": "a", "input" : [ { "foo" : { "$numberInt" : "3" } } ] }'::mongo_catalog.bson)
```

This is schema dependent, let's add a no-op UDF so that we can use the 1.14 train and that way in the 1.15 branch it will be already available to be used by the binary since we ship schema n-1 always.
2024-03-08 14:01:20 +03:00
Santiago Fernandez Madero dba20f6b85 Merged PR 1288356: Move main to 1.15 binary and 1.14 schema and fix rename collection version
### Description

Move main to 1.15 binary and 1.14 schema and fix rename collection version

Rename collection PR went in after we forked the 1.14 release branch but before main was bumped to the new schema and binary version, so fix that in this PR as well.
2024-03-04 09:53:14 +03:00
Lucas Borges Fernandes b635c244d8 Merged PR 1249041: [OSS] Adding C impl for rename_collection UDF 2024-03-04 09:53:14 +03:00
Tanvi Bhandari 8d461c2b27 Merged PR 1271218: CurrentOp showing the progress for Index Build in background
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Issue:
Since the owner of scheduled cron job for index build background is admin user i.e. 'postgres' in prod, the CREATE INDEX CONCURRETLY queries are owned by 'postgres' user in pg_stat_activity.
And non-admin user who requests for currentOp to get progress of running indexes, does not have enough privilege to read queries owned by admin user in pg_stat_activity.

Solutions:
From inside running cron-job, create LibPQ connection as the non-admin user for CREATE INDEX CONCURRENTLY. The pre-requisite for this approach is that we have the information of user-oid who has submitted the request in queue. I.e. This needs user_name column in index_queue table.

Related work items: #2961369
2024-03-01 17:24:28 +03:00
Vinod Sridharan 3b6e07f30f Merged PR 1281490: Query operator native support part 1: Enable expansion of the query operator
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
Yes (OSS)

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Today in OSS the query operator was evaluated at runtime - move this to expansion of hte operator at planning time using the simplify function.
2024-03-01 17:24:28 +03:00
Mehmet Yilmaz 1224a287d6 Merged PR 1269280: [OSS] Move auth_scram functions to OSS
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Move auth_scram functions to OSS
2024-02-23 14:49:18 +03:00
Mehmet Yilmaz e329534b27 Merged PR 1265532: [OSS] Refactor some pgmongo references in OSS
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Refactor some pgmongo references in OSS
2024-02-16 13:49:10 +03:00
Santiago Fernandez Madero e67907c67d Merged PR 1271505: Bump main to 1.14
Bump main to 1.14
2024-02-14 11:28:20 +03:00
Lucas Borges Fernandes 0a88ba774d Merged PR 1269659: [OSS] Changing helio major version to 1 for compatibility
### Description

Bumping major version to 1 in order to prevent future errors with version check utility functions.
2024-02-14 11:28:20 +03:00
Mehmet Yilmaz cb797d3eae Merged PR 1261782: [OSS] Move drop_database function to OSS
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
UDF

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Move drop_database function to OSS
2024-02-07 23:20:07 +03:00
Manish Gaur ac6ae0a7c6 Merged PR 1259477: Updating version for 1.12 release
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

Updating version for 1.12 release
2024-02-07 23:20:07 +03:00
Mehmet Yilmaz e2f438d8ba Merged PR 1244566: Refactor allow_nested_distributed_execution conf in oss
Refactor allow_nested_distributed_execution conf in oss
2024-01-31 12:06:30 +03:00