2 Commits
Author SHA1 Message Date
Amit Prabhudesai 6ac8a8c563 Merged PR 1734798: [Background worker] Dynamically scheduled job intervals via hooks
### Does this PR have any customer impact?
No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Bugfix

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No

### Are you introducing any new config? If yes, do you have tests with and without them being set?
No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Dynamically scheduled job intervals via hooks

----
#### AI description  (iteration 1)
#### PR Classification
This pull request implements a new feature that enables dynamically scheduled job intervals for background worker jobs via hook functions.

#### PR Summary
The pull request refactors background worker job scheduling by replacing the static schedule interval with a hook that dynamically determines the interval. A default hook is provided if none is set, and all related validations and logging have been updated accordingly.
- `oss/pg_documentdb/src/background_worker/background_worker.c`: Updated functions (ValidateJob, RegisterBackgroundWorkerJob, and CanExecuteJob) to retrieve and validate the schedule interval using a hook, including logging when using the default interval.
- `oss/pg_documentdb/include/background_worker/background_worker_job.h`: Replaced the static schedule interval field with a hook function pointer and added a typedef for the scheduling hook.
- Added a default hook function, `GetDefaultScheduleIntervalInSeconds`, that returns a default interval of 60 seconds for jobs without a custom hook.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
2025-07-23 15:56:08 +08:00
Lucas Borges Fernandes 1cf6873426 Merged PR 1661393: Add OSS background_worker initial jobs framework
### Does this PR have any customer impact?
No.

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc)
Feature.

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes)
No.

### Are you introducing any new config? If yes, do you have tests with and without them being set?
Yes.

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description

Add OSS background_worker initial jobs framework. For this initial version, only schedulable jobs are allowed (i.e. jobs that run on timed intervals). The background worker loop has a latch timeout with a predefined duration of 10 seconds that is configured through a GUC. Every time this timeout occurs, it iterates on the JobRegistry array.

1. Checks if the job is running and if it timed out (i.e. running longer than timeoutInSeconds). If yes, cancels the query and finishes libpq connection.
2. Checks if the job finished running (i.e. libpq connection isn't busy). If yes, finishes the libpq connection.
3. Runs the job if its scheduled interval has reached (i.e. currentTime - lastStartTime >= scheduledInterval). Each job execution has its own libpq connection to the database.

In order to add a new job callers must pass a BackgroundWorkerJob structure to RegisterBackgroundWorkerJob.

----
#### AI description  (iteration 1)
#### PR Classification
This pull request introduces a new feature by adding an OSS background worker jobs framework.

#### PR Summary
The changes implement a framework for scheduling, executing, and managing background jobs within the background worker process. This provides structured job processing with timeout, cancellation, and configuration support.

- **`src/background_worker/background_worker.c`**: Added job management functions (e.g., ManageJobs, ExecuteJob, CanExecuteJob, CheckJobCompletion, CancelJobIfTimeIsUp, ValidateJob) and integrated job scheduling in the main loop.
- **`include/background_worker/background_worker_job.h`**: Introduced new structures for defining background worker jobs and their execution objects.
- **`src/api_hook_impl.c`**: Added a hook function to retrieve background worker jobs for registration.
- **`src/configs/background_job_configs.c`**: Defined new configuration variables (GUCs) to enable background worker jobs.
- **API Hooks and Connection Management Files**: Updated declarations and functions (e.g., in `include/api_hooks_def.h` and connection management files) to support connection handling necessary for job execution.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #4225917
2025-07-09 18:56:11 +08:00