### Does this PR have any customer impact?
yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
bugfix
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
no
### Are you introducing any new config? If yes, do you have tests with and without them being set?
no
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
### Description
- enable building both locally & in docker
- fix the hardcoded paths
----
#### AI description (iteration 1)
#### PR Classification
Enhancement: Updates to build scripts and Docker configuration to enable building the gateway both locally and in Docker.
#### PR Summary
This pull request revises several scripts and configuration files to integrate local and containerized build processes for the gateway.
- **`/oss/scripts/emulator_entrypoint.sh`**: Refactors the configuration file handling by defining a variable and updating file permissions and copy paths.
- **`/oss/scripts/build_and_start_gateway.sh`**: Introduces a local build process using Cargo (clean, build, and binary copy) and adjusts the binary execution path.
- **`/oss/.github/containers/Build-Ubuntu/Dockerfile_gateway`**: Updates copy paths to place the gateway binary and configuration into the new subfolder.
- **`/oss/rust-toolchain.toml` & `/oss/pg_documentdb_gw/.gitignore`**: Adds the Cargo component to the toolchain and ensures the built binary is ignored.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?
yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
feature
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
no
### Are you introducing any new config? If yes, do you have tests with and without them being set?
no
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
Provide Rust dev environment in devcontainer
### Description
- install toolchain from rustup
- ensure we run the proper profile in dockerfile
- make sure tests are working
----
#### AI description (iteration 1)
#### PR Classification
This pull request introduces a new feature to enhance the Rust development environment in the OSS repository.
#### PR Summary
The changes streamline the Rust dev experience by updating the build infrastructure, adding Rust-specific tools, and fine-tuning test configurations. Key updates include:
- **`Dockerfile_gateway`**: Adjusts the Docker workflow by switching to a non-root user, installing Rust via a new `install_rustup.sh` script, and building with the custom `release-with-symbols` profile.
- **`devcontainer.json`**: Incorporates the Rust analyzer extension and detailed Rust settings for improved performance, code formatting, and project linking.
- **`Cargo.toml`**: Introduces a new release profile (`release-with-symbols`) with tailored debug and optimization settings.
- **Test and common modules**: Updates test assertions and runtime configuration to correctly handle error messages and dynamically set the Postgres system user.
- **New files and minor updates**: Adds `install_rustup.sh` and `rust-toolchain.toml` for robust Rust toolchain management, and updates `.gitignore` by removing `.vscode/`.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?
yes
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
bugfix
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
no
### Are you introducing any new config? If yes, do you have tests with and without them being set?
no
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
- drop unused environment variable `ENFORCE_SSL` in dockerfile *[Bugfix]* [issue 313](https://github.com/documentdb/documentdb/issues/313)
### Description
drop unused environment variable
Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
### Does this PR have any customer impact?
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
### Are you introducing any new config? If yes, do you have tests with and without them being set?
### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
### Description
----
#### AI description (iteration 1)
#### PR Classification
This pull request improves the packaging workflow by updating and fixing the signing tool integration for both DEB and RPM packages.
#### PR Summary
The changes streamline the build and signing processes in the OSS workflows, ensuring more consistent and simplified key generation and package verification.
- `oss/.github/workflows/build_packages.yml`: Updates the DEB package signing by removing branch-specific conditions, switching to a quick GPG key generation method, revising the dpkg-sig installation process, updating the cosign verify regex to support branches and tags, and reducing the OS matrix to ubuntu22.04.
- `oss/.github/workflows/build_rpm_packages.yml`: Refines RPM package signing by eliminating branch checks, replacing the batch key generation with a quick-gen-key command, and modifying the RPM signing macros for a cleaner signing command.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
Require local auth if external IP is flag is enabled:
If the allowExternalAccess flag is enabled on the Postgres server, this will require to use username password to connect.
Require Password to Start Docker Container
password should always be provided to protect the database
$ docker run doc:001 --username guanzhou
Error: PASSWORD is required. Please provide a password using --password argument or PASSWORD environment variable.
----
#### AI description (iteration 1)
#### PR Classification
This pull request implements a security enhancement by enforcing mandatory password authentication and stronger connection methods for external access.
#### PR Summary
The pull request ensures that a password must be provided at startup and updates the authentication configuration for PostgreSQL in external access scenarios.
- `oss/scripts/emulator_entrypoint.sh`: Added a validation check that exits when PASSWORD is missing and updated the CLI help to mark the password as REQUIRED.
- `oss/scripts/start_oss_server.sh`: Changed PostgreSQL settings to use scram-sha-256 for external IP connections instead of the "trust" method.
- `oss/.github/containers/Build-Ubuntu/Dockerfile_gateway`: Removed the default PASSWORD value, requiring explicit configuration.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
* first commit
* update action
* fix
* fix
* test push
* test push
* test push
* test push
* test push
* test push
* add lables
* update tag
* clean up
* try to resolve unknown/unknown
* clean up
* test sign
* clean up
* update action name
* fix bug
* clean up
---------
Co-authored-by: root <root@CPC-yangq-FRSGK>
Optimize the fix for postgis 3.5
Related postgis issue: https://trac.osgeo.org/postgis/ticket/5652
----
This pull request is a bug fix that updates the PostGIS 3.5 build process.
The changes modify the installation script and update Docker container configurations to resolve build issues for PostGIS 3.5 by removing unused GDAL components and avoiding parallel build race conditions.
- `oss/scripts/install_setup_postgis.sh`: Added a call to `./autogen.sh`, updated configuration options by removing GDAL use and adding `--without-raster`, and replaced parallel `make` with serial execution.
- Multiple Dockerfiles (`.pipelines/Containers/Container-Base-Mariner/Dockerfile`, `/.devcontainer/Dockerfile`, `/.pipelines/Containers/Container-Base-Ubuntu/Dockerfile`, `/oss/.devcontainer/Dockerfile`, and `/oss/.github/containers/Build-Ubuntu/Dockerfile`): Removed GDAL development dependencies to align with the updated build process.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
* init
* fix
* revert
* Create docker-image.yml
* Update docker-image.yml
* Update docker-image.yml
* set image
* clean up
* image tag and build
* typo
* fix naming
* rename
* arm64
* add arm64
* update
* set default data path
* generate cert files in start up scripts
* set up for cloud native PG
* init
* gateway
* using ENV to set arguments
* remove password
* set name
* add citus back
* update trigger
* push to GHCR
* generate manifest to GHCR
* update image name
* Revert "update image name"
This reverts commit 0279d40a56adce11920f512854190c19edfaa13b.
* rename
* rename
* set image name and tags
* set input for manifest tag
* make documentdb version configurable
* add port check
* use matrix for clearance
* set default value
* set env
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Infra
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
### Description
We have validation scripts we run internally. Run a subset of them on OSS
----
#### AI description (iteration 1)
#### PR Classification
New feature
#### PR Summary
This pull request adds validation scripts to the OSS setup to ensure code quality and consistency.
- Added `oss/scripts/validate_extension_file_state.sh` to validate no Citus references and check SQL file versions.
- Updated `/.pipelines/templates/oss_documentdb_build.yml` to include the new validation script in the build process.
- Modified `/oss/.github/workflows/regress_tests.yml` to run the new validation script during regression tests.
- Updated messages in `oss/scripts/validate_headers.sh` for clarity.
[OSS] fix regress sql test workflow
----
Bug fix for the SQL test workflow.
This pull request fixes the regression in the SQL test workflow by updating the GitHub Actions configuration and related scripts.
- Updated `.github/workflows/regress_tests.yml` to improve the build matrix, install dependencies, and run tests for multiple PostgreSQL versions and Ubuntu versions.
- Modified `scripts/start_oss_server.sh` to correct the help message and default directory.
- Minor cleanup in `scripts/install_setup_pg_cron.sh` to remove an unnecessary blank line.
* Merged PR 1557887: Change error file mapping name
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactor
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
### Description
Rename error file mapping
----
#### AI description (iteration 1)
#### PR Classification
Code cleanup
#### PR Summary
This pull request updates the name of the error mapping file and adjusts references accordingly.
- `Makefile`: Updated references from `mongoerrors.csv` to `error_mappings.csv`.
- `docs/pgmongo/design_docs/data_plane/backend_common/error_management.md`: Updated documentation to reflect the new file name.
- `src/utils/documentdb_conversion_codes.c`: Updated comment to reference the new file name.
- Moved `oss/mongoerrors.csv` to `oss/error_mappings.csv` without code changes.
* Merged PR 1555452: [OSS] Enable distributed tests in documentdb
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))
### Description
Enable distributed tests in documentdb, and clean up helio references from documentdb_distributed
Add substitution to make pg_helio_distributed tests work
Add substitution to make pgmongo/helio_compat tests work
Move udfs bson_aggregation_redact to documentdb
----
#### AI description (iteration 1)
#### PR Classification
New feature
#### PR Summary
This pull request enables distributed tests in DocumentDB by adding a new set of regression tests for BSON aggregation comparison operators.
- Added `bson_aggregation_comparison_operators_tests.out` with extensive test cases for BSON comparison operators (`$cmp`, `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`).
* Merged PR 1555841: [OSS] fix regress sql test workflow
[OSS] fix regress sql test workflow
----
Bug fix for the SQL test workflow.
This pull request fixes the regression in the SQL test workflow by updating the GitHub Actions configuration and related scripts.
- Updated `.github/workflows/regress_tests.yml` to improve the build matrix, install dependencies, and run tests for multiple PostgreSQL versions and Ubuntu versions.
- Modified `scripts/start_oss_server.sh` to correct the help message and default directory.
- Minor cleanup in `scripts/install_setup_pg_cron.sh` to remove an unnecessary blank line.
---------
Co-authored-by: Vinod Sridharan <visridha@microsoft.com>
Co-authored-by: Zinan Ren <zinanren@microsoft.com>
Move test files from pg_helio_distributed to pg_documentdb_distributed
Clean up pg_documentdb
----
Code cleanup
This pull request cleans up the `helio_api` references and updates them to `documentdb`.
- `custom_query_scan.c` and `custom_scan_continuation.c`: Updated custom scan names from `HelioApi` to `DocumentDBApi`.
- Deleted `commands_coll_stats.out` and `commands_coll_stats.sql` test files.
- `start_oss_server.sh`: Removed `helio` server type and related logic.
- `setup_psqlrc.sh`: Updated search path and settings from `helio` to `documentdb`.
- `.devcontainer/Dockerfile` and `.github/containers/Build-Ubuntu/Dockerfile`: Updated user and labels from `heliodb` to `documentdb`.
Started running regression tests for multiple ubuntu and pg versions
tested os: ubuntu20.04, ubuntu22.04
tested pg : 15, 16
it revealed a test failure at pg_helio_distributed extension tests,
should be fixed soon
Shuai Tian
Andrew Khoma
yangqiaoandYang Qiao
Shuai Tian
Guanzhou Song
Guanzhou Song
Copilot
German EichbergerandGerman
Vinod Sridharan
Alexey Palazhchenko
Siddhesh Vethe
Murat Tuncer
Murat Tuncer
gledis69