Feature: add support for jobs to run sequentially #2461

rushatgabhane · 2026-01-18T21:28:23Z

Details

For Jobs plugin, added a new sequentialKey param. If multiple jobs have the same sequentialKey, then they will run one at a time, in the order they were created in.

Added sequentialKey column to jobs table
Jobs created with a sequentialKey check for existing active jobs with the same key
If found, new job is set to WAITING state instead of QUEUED
When a job completes (FinishJob, FailJob, DeleteJob), the next WAITING job with the same key is promoted to QUEUED

Unsupported stuff

I couldn't think of a usecase / requirement at the moment for these

Using repeat with sequentialKey - A repeating job never gets deleted, so WAITING jobs won't be promoted.
Child jobs used with sequentialKey - Child jobs are marked FINISHED (not deleted) when complete so WAITING jobs won't be promoted.

Fixed Issues

See - https://expensify.slack.com/archives/C086VHWPPUM/p1768336706994209?thread_ts=1768271555.676269&cid=C086VHWPPUM

Part of https://github.com/Expensify/Expensify/issues/568339

Tests

Unit tests added that verify

First job with sequentialKey is QUEUED
Second job with same key becomes WAITING when first is QUEUED
Second job with same key becomes WAITING when first is RUNNING
FinishJob promotes next WAITING job to QUEUED
FailJob promotes next WAITING job to QUEUED
DeleteJob promotes next WAITING job to QUEUED
Three jobs with same key run in order
Jobs with different keys don't block each other

Internal Testing Reminder: when changing bedrock, please compile auth against your new changes

Copilot

Pull request overview

This PR adds support for sequential job execution in the Jobs plugin through a new sequentialKey parameter. Jobs sharing the same sequential key will execute one at a time in creation order, with subsequent jobs waiting until the active job completes.

Changes:

Added sequentialKey column to the jobs table
Jobs with an active sequential key are created in WAITING state instead of QUEUED
When jobs complete via FinishJob, FailJob, or DeleteJob, the next WAITING job with the same key is promoted to QUEUED

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
plugins/Jobs.cpp	Added sequentialKey column to schema, logic to set jobs to WAITING when sequential key exists, and promotion logic in FinishJob/FailJob/DeleteJob
plugins/Jobs.md	Documented the sequentialKey parameter and its limitations with repeat and parentJobID
test/tests/jobs/CreateJobTest.cpp	Added comprehensive tests for sequential job creation and ordering
test/tests/jobs/FailJobTest.cpp	Added test verifying FailJob promotes waiting jobs
test/tests/jobs/DeleteJobTest.cpp	Added test verifying DeleteJob promotes waiting jobs
test/clustertest/tests/FinishJobTest.cpp	Added test verifying FinishJob promotes waiting jobs

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-18T22:10:26Z

plugins/Jobs.cpp

+                // If sequentialKey is set, check if there are other pending jobs with the same key.
+                // If so, this job should wait for them to complete before running by setting the initial state to WAITING.
+                const string& safeSequentialKey = SContains(job, "sequentialKey") ? SQ(job["sequentialKey"]) : SQ("");
+                if (safeSequentialKey != SQ("")) {
+                    SQResult result;
+                    if (!db.read("SELECT 1 FROM jobs "
+                                 "WHERE sequentialKey=" + safeSequentialKey + " "
+                                 "AND state IN ('QUEUED', 'RUNQUEUED', 'RUNNING', 'WAITING') "
+                                 "LIMIT 1;", result)) {
+                        STHROW("502 Select failed");
+                    }
+                    if (!result.empty()) {
+                        initialState = "WAITING";
+                    }
+                }


Missing validation to prevent unsupported combinations of sequentialKey with repeat or parentJobID. The PR description states these are unsupported because jobs with repeat or parentJobID don't get deleted on completion, preventing WAITING jobs from being promoted. However, there's no validation to prevent users from creating jobs with these combinations, which would lead to unexpected behavior where WAITING jobs never get promoted to QUEUED.

Copilot · 2026-01-18T22:10:27Z

plugins/Jobs.cpp

                               "parentJobID INTEGER NOT NULL DEFAULT 0, "
-                               "retryAfter  TEXT NOT NULL DEFAULT \"\")",
+                               "retryAfter  TEXT NOT NULL DEFAULT \"\", "
+                               "sequentialKey TEXT NOT NULL DEFAULT \"\")",


Missing database index for sequentialKey column. The queries at lines 617-620, 1224-1230, 1330-1336, and 1383-1389 filter by sequentialKey without an index, which will cause performance issues as the jobs table grows. An index should be added similar to other indexed columns.

Do we need an index? I thinkk let's add it when it becomes a problem

Copilot · 2026-01-18T22:10:27Z

plugins/Jobs.cpp

+                // If sequentialKey is set, check if there are other pending jobs with the same key.
+                // If so, this job should wait for them to complete before running by setting the initial state to WAITING.
+                const string& safeSequentialKey = SContains(job, "sequentialKey") ? SQ(job["sequentialKey"]) : SQ("");
+                if (safeSequentialKey != SQ("")) {
+                    SQResult result;
+                    if (!db.read("SELECT 1 FROM jobs "
+                                 "WHERE sequentialKey=" + safeSequentialKey + " "
+                                 "AND state IN ('QUEUED', 'RUNQUEUED', 'RUNNING', 'WAITING') "
+                                 "LIMIT 1;", result)) {
+                        STHROW("502 Select failed");
+                    }
+                    if (!result.empty()) {
+                        initialState = "WAITING";
+                    }
+                }


The sequentialKey logic can override the parentJobID state logic, which could break parent-child job semantics. When both parentJobID and sequentialKey are set (an unsupported combination per the PR description), a child job that should be PAUSED (lines 605-610) can be set to WAITING (line 624), preventing it from running when the parent finishes. This is another reason why the combination should be explicitly validated and rejected.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2daa99573f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-01-19T23:33:24Z

plugins/Jobs.cpp

                               "data        TEXT NOT NULL, "
                               "priority    INTEGER NOT NULL DEFAULT " + SToStr(JOBS_DEFAULT_PRIORITY) + ", "
                               "parentJobID INTEGER NOT NULL DEFAULT 0, "
-                               "retryAfter  TEXT NOT NULL DEFAULT \"\")",
+                               "retryAfter  TEXT NOT NULL DEFAULT \"\", "
+                               "sequentialKey TEXT NOT NULL DEFAULT \"\")",


Add migration for sequentialKey column

This change only updates the CREATE TABLE jobs SQL. verifyTable doesn’t alter existing tables, so an upgraded deployment with a pre-existing jobs table will still be missing sequentialKey, and subsequent statements like INSERT INTO jobs (..., sequentialKey) or the new SELECT ... sequentialKey will fail with “no such column,” breaking job creation/finish on upgrade. Please add an explicit schema migration (e.g., addColumn) or a guarded alter step for existing databases.

Useful? React with 👍 / 👎.

we don't do this is code right? have to manually run Alter table before deploying this code

rushatgabhane added 5 commits January 18, 2026 22:23

add sequentialkey column with default empty

cf0f1d2

add sequentialkey support to queue dependents

90893ca

add sequentialkey to select and bind to result

69c2f22

promote next waiting job in sequence to queued

de8b677

read sequentialkey and promote next waiting job

58e06af

melvin-bot bot assigned rushatgabhane Jan 18, 2026

rushatgabhane changed the title ~~Sequential jobs~~ Feature: add support for jobs to run sequentially Jan 18, 2026

rushatgabhane added 16 commits January 18, 2026 22:32

fix style

30d62cc

add tests for sequential job queuing and promotion

2876b99

fix style

49b64a2

add sequentialkey param to createjob

22d2b00

add sequential_key queueing and promote waiting

b6e277a

add test for finish job promotes waiting job

a2faf7f

test: promote next waiting with same key

e4f34ae

add sequential key tests for create job

2d81e34

add tests for sequentialKey queuing behavior

331ce81

add test for deletejobpromoteswaitingjob

1e2c79b

add test delete first promotes waiting job

968c24e

add test for failjobpromoteswaitingjob

a571f6e

add test: fail promotes next waiting with same key

5699abb

remove sequential job tests

c20d8b7

add test for sequential jobs run in order

16cc3f6

add test for three sequential jobs in order

2daa995

rushatgabhane requested a review from Copilot January 18, 2026 22:06

Copilot started reviewing on behalf of rushatgabhane January 18, 2026 22:06 View session

Copilot AI reviewed Jan 18, 2026

View reviewed changes

rushatgabhane marked this pull request as ready for review January 19, 2026 23:28

rushatgabhane requested a review from marcaaron January 19, 2026 23:28

chatgpt-codex-connector bot reviewed Jan 19, 2026

View reviewed changes

rushatgabhane requested a review from Gonals January 20, 2026 04:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: add support for jobs to run sequentially #2461

Feature: add support for jobs to run sequentially #2461

rushatgabhane commented Jan 18, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 18, 2026

Uh oh!

Copilot AI Jan 18, 2026

Uh oh!

rushatgabhane Jan 18, 2026

Uh oh!

Copilot AI Jan 18, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Jan 19, 2026

Uh oh!

rushatgabhane Jan 19, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Feature: add support for jobs to run sequentially #2461

Are you sure you want to change the base?

Feature: add support for jobs to run sequentially #2461

Conversation

rushatgabhane commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Unsupported stuff

Fixed Issues

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

rushatgabhane Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

rushatgabhane Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rushatgabhane commented Jan 18, 2026 •

edited

Loading

rushatgabhane Jan 19, 2026 •

edited

Loading