50 Commits
Author SHA1 Message Date
Shuai Tian 56ef6ff121 Add packaging CI workflow into every PR check and fix rpm errors (#336)
* Refactor CI workflows to conditionally set OS and PostgreSQL version for pull requests

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* exclude specific internal directories from Debian and RPM packages

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* fix

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* skip check-extended-rum

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* Include check-extended-rum target and tests in building

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* fix

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* update sql tests

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* fix

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* use exclude

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* more

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* use template

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* feedback

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

---------

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
2025-10-09 10:41:56 -07:00
Shuai Tian 65b48ced1b [workflow] Fix build errors on arm64 and refine the yaml files (#323)
* Update PostgreSQL repo setup in Dockerfiles for RHEL 8 and RHEL 9 to use --disablerepo=* option

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

* Remove CMake installation step from regression tests workflow

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>

---------

Signed-off-by: Shuai Tian <shuaitian@microsoft.com>
2025-09-22 10:24:29 -07:00
Shuai Tian c66a3576ac Package: update changelog/spec when building packages (#310) 2025-09-18 22:52:06 -07:00
yangqiaoandYang Qiao b849c222c6 build: skip trigger when PR is draft (#304)
Co-authored-by: Yang Qiao <yangqiao@microsoft.com>
2025-09-08 12:21:35 +08:00
Shuai Tian 2b29847e6f RPM package: support arm64 and update build workflow (#297) 2025-09-05 15:51:09 +08:00
Shuai Tian 92d04371a1 Deb package: support deb:trixie and update the workflow (#296) 2025-09-05 11:31:54 +08:00
Shuai Tian 9e414a7d1f Rename job names in workflows (#278) 2025-08-24 22:26:14 +08:00
Shuai Tian 155eb41bfe Merged PR 1741151: [oss] Improve workflow of packaging and fix signing tool
### Does this PR have any customer impact?

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)

### Are you introducing any new config? If yes, do you have tests with and without them being set?

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

----
#### AI description  (iteration 1)
#### PR Classification
This pull request improves the packaging workflow by updating and fixing the signing tool integration for both DEB and RPM packages.

#### PR Summary
The changes streamline the build and signing processes in the OSS workflows, ensuring more consistent and simplified key generation and package verification.
- `oss/.github/workflows/build_packages.yml`: Updates the DEB package signing by removing branch-specific conditions, switching to a quick GPG key generation method, revising the dpkg-sig installation process, updating the cosign verify regex to support branches and tags, and reducing the OS matrix to ubuntu22.04.
- `oss/.github/workflows/build_rpm_packages.yml`: Refines RPM package signing by eliminating branch checks, replacing the batch key generation with a quick-gen-key command, and modifying the RPM signing macros for a cleaner signing command.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-28 13:14:05 +08:00
Copilot 57706dd4e0 Add RPM package support for Red Hat-based distributions (#204)
Add RPM package support for RHEL distributions
2025-06-18 15:49:03 +08:00
German EichbergerandGerman 0280d20be8 Bump baseversion to 1.0.4 (#220)
Co-authored-by: German <geeichbe@microsoft.com>
2025-06-16 09:08:25 -07:00
yangqiaoandroot ce8cda5b1d Update readme to use new container registry (#217)
* update

* test

* test

* update readme

---------

Co-authored-by: root <root@CPC-yangq-FRSGK>
2025-06-16 14:41:54 +08:00
yangqiaoandroot 3217a57219 Add prebuild image tasks in GH action (#212)
* first commit

* update action

* fix

* fix

* test push

* test push

* test push

* test push

* test push

* test push

* add lables

* update tag

* clean up

* try to resolve unknown/unknown

* clean up

* test sign

* clean up

* update action name

* fix bug

* clean up

---------

Co-authored-by: root <root@CPC-yangq-FRSGK>
2025-06-10 11:15:45 +08:00
Shuai Tian a948405ed4 Update codeql.yml to use ubuntu22.04 (#214) 2025-06-06 15:23:11 +08:00
Guanzhou Songandsajeetharan 5c240146b1 Add Documentations and Deployment Workflow (#187)
* Add doc site

* update deploy workflow

* add gateway doc

* update structure and add gateway doc

* update missing image

---------

Co-authored-by: sajeetharan <sajeetharan@outlook.com>
2025-05-20 11:20:17 -04:00
Guanzhou Song 30572d9ddf Add Steps in GIthub Action to Sign the Docker Image and Workflow to Release (#183)
* Add steps to sign the docker image

* clean up

* add workflow to release latest manifest

* release

* manual trigger

* trigger

* update

* update

* create
2025-05-14 23:43:44 -04:00
Guanzhou Song b2080caee5 Add Gateway Integration (#166)
* init

* fix

* revert

* Create docker-image.yml

* Update docker-image.yml

* Update docker-image.yml

* set image

* clean up

* image tag and build

* typo

* fix naming

* rename

* arm64

* add arm64

* update

* set default data path

* generate cert files in start up scripts

* set up for cloud native PG

* init

* gateway

* using ENV to set arguments

* remove password

* set name

* add citus back

* update trigger

* push to GHCR

* generate manifest to GHCR

* update image name

* Revert "update image name"

This reverts commit 0279d40a56adce11920f512854190c19edfaa13b.

* rename

* rename

* set image name and tags

* set input for manifest tag

* make documentdb version configurable

* add port check

* use matrix for clearance

* set default value

* set env
2025-05-12 10:38:55 -04:00
shuaitian 916a6b423f use 3.5.2 and install raster 2025-05-09 14:08:53 +08:00
Shuai Tian 48e250ab6d Merged PR 1641717: [oss] add arm64 to test workflow and update geo tests
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Testing

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))
None

### Description
arm64 and amd64 handle floating-point math differently, which caused tiny differences for some geo related tests. Reduce the decimal positions to fix them.

test failure messages

**geospatial_extract_2d_geometries.out**

```
SELECT public.ST_ASTEXT(
     public.ST_SUBDIVIDE(
         public.ST_MAKEVALID(public.ST_SEGMENTIZE('SRID=4326;POLYGON((-180 40, -157.5 40, -157.5 55, -180 55, -180 40))'::public.geography, 500000)::public.geometry),
         8 -- 8 is the default number of max vertices a divide segment can have
     )::public.geography
 );
                                                                                                    st_astext
 ---------------------------------------------------------------------
- POLYGON((-163.1021234145348 40.4105866220272,-160.29991441874373 40.205209224804754,-180 40,-174.39787658546524 40.4105866220272,-168.75000000000003 40.54826391836536,-163.1021234145348 40.4105866220272))
+ POLYGON((-163.1021234145348 40.4105866220272,-160.29991441874373 40.205209224804754,-180 40,-174.39787658546524 40.41058662202719,-168.75000000000003 40.54826391836536,-163.1021234145348 40.4105866220272))
```

**bson_aggregation_pipeline_tests_geonear.out**

```
SELECT * FROM documentdb_api_catalog.bson_aggregation_pipeline('db', '{ "aggregate": "geonear_lookup_1", "pipeline": [ { "$geoNear": {"near": {"type": "Point", "coordinates": [0, 1]}, "distanceField": "distance", "spherical": true, "key": "a"}}, {"$lookup": {"from": "geonear_lookup_2", "localField": "x", "foreignField": "x", "as": "new"}}] } ');
                                                                                                                                                                     document
 ---------------------------------------------------------------------
- { "_id" : { "$numberInt" : "1" }, "a" : [ { "$numberInt" : "10" }, { "$numberInt" : "10" } ], "x" : { "$numberInt" : "5" }, "distance" : { "$numberDouble" : "1491297.403229492018" }, "new" : [ { "_id" : { "$numberInt" : "5" }, "foo" : "bar", "a" : [ { "$numberInt" : "5" }, { "$numberInt" : "5" } ], "x" : { "$numberInt" : "5" } } ] }
+ { "_id" : { "$numberInt" : "1" }, "a" : [ { "$numberInt" : "10" }, { "$numberInt" : "10" } ], "x" : { "$numberInt" : "5" }, "distance" : { "$numberDouble" : "1491297.4032294917852" }, "new" : [ { "_...
2025-04-17 18:40:16 +08:00
Shuai Tian 1badc1d7d5 fix debian workflow (#145) 2025-04-14 15:16:20 +08:00
Shuai Tian c70966ead9 Add scripts to generate deb packages and set up new workflow (#125) 2025-04-14 11:09:35 +08:00
Shuai Tian 666e5529ed Update regress_tests.yml to enable tests for PG17 (#80)
* Update regress_tests.yml

* Update setup_versions.sh

* fix test_mutate_17

* upload logs

* fix

* fix

* include subdirectory

* if: always()

* name

* debug

* debug

* fix the bug

* fix double checking

* revert change in .out

* fix
2025-02-27 09:53:22 -08:00
Shuai Tian a1ce474e5e Update regress_tests.yml to remove ubuntu-20.04 (#75) 2025-02-21 12:51:50 -08:00
Vinod Sridharan e3c7e3c71f Merged PR 1564344: [OSS] Add validation scripts into the OSS set up as well
### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Infra
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
We have validation scripts we run internally. Run a subset of them on OSS

----
#### AI description  (iteration 1)
#### PR Classification
New feature

#### PR Summary
This pull request adds validation scripts to the OSS setup to ensure code quality and consistency.
- Added `oss/scripts/validate_extension_file_state.sh` to validate no Citus references and check SQL file versions.
- Updated `/.pipelines/templates/oss_documentdb_build.yml` to include the new validation script in the build process.
- Modified `/oss/.github/workflows/regress_tests.yml` to run the new validation script during regression tests.
- Updated messages in `oss/scripts/validate_headers.sh` for clarity.
2025-01-30 10:35:08 -08:00
Shuai Tian b42a46e5f4 cleanup and renaming helio (#7) 2025-01-22 22:26:11 -08:00
Shuai Tian b16399b1af Merged PR 1555841: [OSS] fix regress sql test workflow
[OSS] fix regress sql test workflow

----
Bug fix for the SQL test workflow.
This pull request fixes the regression in the SQL test workflow by updating the GitHub Actions configuration and related scripts.
- Updated `.github/workflows/regress_tests.yml` to improve the build matrix, install dependencies, and run tests for multiple PostgreSQL versions and Ubuntu versions.
- Modified `scripts/start_oss_server.sh` to correct the help message and default directory.
- Minor cleanup in `scripts/install_setup_pg_cron.sh` to remove an unnecessary blank line.
2025-01-22 22:24:35 -08:00
Siddhesh Vethe 361c56bd92 Revert "Documentdb sync 2025-01-23 (#6)" (#9)
This reverts commit 451119c809.
2025-01-22 22:21:30 -08:00
451119c809 Documentdb sync 2025-01-23 (#6)
* Merged PR 1557887: Change error file mapping name

### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactor
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description
Rename error file mapping

----
#### AI description  (iteration 1)
#### PR Classification
Code cleanup

#### PR Summary
This pull request updates the name of the error mapping file and adjusts references accordingly.
- `Makefile`: Updated references from `mongoerrors.csv` to `error_mappings.csv`.
- `docs/pgmongo/design_docs/data_plane/backend_common/error_management.md`: Updated documentation to reflect the new file name.
- `src/utils/documentdb_conversion_codes.c`: Updated comment to reference the new file name.
- Moved `oss/mongoerrors.csv` to `oss/error_mappings.csv` without code changes.

* Merged PR 1555452: [OSS] Enable distributed tests in documentdb

### Does this PR have any customer impact?
No
### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Refactoring
### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No
### Are you introducing any new config? If yes, do you have tests with and without them being set?
No
### Release Note (Refer [Template](../docs/release_notes/pgmongo_template.md))

### Description

Enable distributed tests in documentdb, and clean up helio references from documentdb_distributed
Add substitution to make pg_helio_distributed tests work
Add substitution to make pgmongo/helio_compat tests work
Move udfs bson_aggregation_redact to documentdb

----
#### AI description  (iteration 1)
#### PR Classification
New feature

#### PR Summary
This pull request enables distributed tests in DocumentDB by adding a new set of regression tests for BSON aggregation comparison operators.
- Added `bson_aggregation_comparison_operators_tests.out` with extensive test cases for BSON comparison operators (`$cmp`, `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`).

* Merged PR 1555841: [OSS] fix regress sql test workflow

[OSS] fix regress sql test workflow

----
Bug fix for the SQL test workflow.
This pull request fixes the regression in the SQL test workflow by updating the GitHub Actions configuration and related scripts.
- Updated `.github/workflows/regress_tests.yml` to improve the build matrix, install dependencies, and run tests for multiple PostgreSQL versions and Ubuntu versions.
- Modified `scripts/start_oss_server.sh` to correct the help message and default directory.
- Minor cleanup in `scripts/install_setup_pg_cron.sh` to remove an unnecessary blank line.

---------

Co-authored-by: Vinod Sridharan <visridha@microsoft.com>
Co-authored-by: Zinan Ren <zinanren@microsoft.com>
2025-01-22 22:20:31 -08:00
Murat Tuncer 1147fb8042 Fix sql test make check run (#67)
Started running regression tests for multiple ubuntu and pg versions
tested os: ubuntu20.04, ubuntu22.04
tested pg : 15, 16

it revealed a test failure at pg_helio_distributed extension tests,
should be fixed soon
2024-10-04 19:32:05 +03:00
Murat Tuncer 609da18acb Update regression test defines 2024-09-05 18:18:23 +03:00
Murat Tuncer dd76765703 Add python to codeql checks (#54) 2024-09-05 09:36:42 +03:00
Murat Tuncer aea74d4beb Fix codeql workflow failures (#53)
new package dependencies are added to fix build failure
codeql api version is bumped to v3 due to planned deprecation of v2
2024-09-04 08:35:38 +03:00
Murat Tuncer 6b6a10e03e Add new lib dependencies and fix build breaks 2024-08-26 13:12:13 +03:00
Murat Tuncer 3c3d57e6ab switch to PG16 2024-08-16 18:24:09 +03:00
gledis69 7e95cf5232 Redo the dependecy install 2024-08-16 18:24:09 +03:00
gledis69 caf2fe2899 Add libbson install back 2024-08-16 18:24:09 +03:00
gledis69 b3f6ba96ed Elevate all the install permissions 2024-08-16 18:24:09 +03:00
gledis69 0295952ce9 Give elevated premissions to pcre2 install 2024-08-16 18:24:09 +03:00
gledis69 c8d6c53993 Try without mongo-c-driver 2024-08-16 18:24:09 +03:00
gledis69 0551c58e7e Fix install step 2024-08-16 18:24:09 +03:00
gledis69 c848e1ddc2 Print intall_setup contents 2024-08-16 18:24:09 +03:00
gledis69 e9ef818084 Test installing without sudo 2024-08-16 18:24:09 +03:00
gledis69 a072014215 Switch up install order 2024-08-16 18:24:09 +03:00
gledis69 bf5fa8a4cb Split the script steps in order to find where the issue is 2024-08-16 18:24:09 +03:00
gledis69 06e0c08826 Add debug step 2024-08-16 18:24:09 +03:00
gledis69 c26840d9b0 Try making sudo non interactive for the scripts 2024-08-16 18:24:09 +03:00
gledis69 59bed7d6d9 Try specifying shell 2024-08-16 18:24:09 +03:00
gledis69 285ad323bf Fix workflow syntax 2024-08-16 18:24:09 +03:00
gledis69 d63eb10749 Run sql tests without caching container 2024-08-16 18:24:09 +03:00
Murat Tuncer 8d9bef2a79 Create codeql.yml 2024-03-14 13:51:15 +03:00
Santiago Fernandez Madero ca7fe5ea32 Merged PR 1265227: Add container build and test github action to heliodb
Tested on my own fork: https://github.com/safern/heliodb/actions
2024-02-07 23:20:07 +03:00