@gavel/e2e
The scenario language and runner, property tests, load harness, browser specs and the real-sandbox specs.
packages/e2e tests the packages together. It has a small YAML scenario language, two runners (in process and as separate processes), property tests, a load harness, browser specs and the specs that talk to the real PayPal sandbox.
It sits outside the package dependency rule: it may import every other package, and nothing imports it.
Responsibilities
Fifteen scenarios, S01 to S15, in scenarios/*.yaml. Each file declares an auction, actors, timed steps, faults and expectations. The schema in src/dsl/schema.ts rejects a file that does not match, and a step or fault a runner cannot drive throws; it is never skipped.
- In process (
src/runner/inproc.ts): the real engine, room, settlement saga and the fake PayPal server on a manual clock. No browser, no real sleeping. - Stack (
src/runner/stack.ts): the real server (fake payments, file store) as a child process, killed with SIGKILL and restarted on the same data folder.externalFakeruns the fake PayPal in its own process so a server kill leaves orders and idempotency keys alive. ScenariosS03,S09andS15declaremodes: [inproc, stack]; the others run in process only.
Scenario files
| Id | Title in the file |
|---|---|
S01-happy-path | Full auction happy path |
S02-voice-replay | Spoken bids with a clear utterance, a declined and a confirmed ambiguous one |
S03-crash-resume | Server dies during settlement and the restart finishes it with one capture (also stack mode) |
S04-duplicate-webhooks | Fifty copies of one capture webhook change nothing |
S05-out-of-order-webhooks | Void before capture and a stale capture event leave the same result |
S06-runner-up-fallback | Winner hold voided externally, runner-up is charged |
S07-hold-voided-all-invalid | Every hold voided externally, nothing is charged |
S08-hold-expired | Winner hold expires before the hammer |
S09-reconnect | Bidder drops during bidding and resumes without losing state (also stack mode) |
S10-abusive-input | Malformed and hostile frames are each refused with a code |
S11-last-100ms-race | Twenty bids in the last 100 ms then hammer |
S12-soft-close-extends | Bids inside the soft close window then hammer |
S13-capture-5xx-retry | Capture fails with 503 twice then succeeds |
S14-void-failure-retry | Void of the losing hold fails three times then succeeds |
S15-restart-persisted-ledger | Server restart keeps registered bidders, bids and price (also stack mode) |
The titles are the title field of each file; open the YAML for the exact steps.
The five invariants
After a run, src/runner/invariants.ts evaluates five checks on what was observed:
| Id | Check |
|---|---|
| I1 | At most one capture across all holds, and the fake PayPal's own invariant check passed |
| I2 | Nothing captured above the winner's ceiling |
| I3 | Every loser hold is voided and never captured |
| I4 | Held money equals captured plus released |
| I5 | The ledger hash chain verifies |
A scenario lists the invariants it expects in its expect block next to ledger, settlement, rejection, PayPal call count and bid count expectations.
Scenario shape
id: S04-duplicate-webhooks
modes: [inproc]
seed: 4
clock: { startMs: 1793750400000 }
auction: { itemId: lot-1, reserve: 5000, minIncrement: 500, softCloseMs: 0, mode: english }
actors: { ana: { funding: wallet }, ben: { funding: wallet } }
steps:
- { t: 0, actor: ana, do: register, ceiling: 50000 }
- { t: 2000, actor: ana, do: bid, amount: 6000 }
- { t: 7000, actor: auctioneer, do: hammer }
expect:
- ledger: { winner: ana, captured: 10000, released: 40000, voided: [ben] }(Abridged from scenarios/S04-duplicate-webhooks.yaml.) Faults available in the language: PayPal capture errors (500, 502, 503, 429) and a timeout after commit, void failures, authorization expiry, external void or decline, webhook duplicate, delay and reorder, a server kill at a named checkpoint, speech-to-text timeout, garbage or low confidence, and a WebSocket drop.
Public interface
src/index.ts exports the DSL types, ScenarioSchema, StepSchema, FaultSchema, ExpectSchema and the loaders loadScenarioFile, parseScenarioYaml, validateScenario with ScenarioError. The runners are used through their CLI (src/runner/cli.ts) and by the tests.
Key files
Scripts
| Script | What it does |
|---|---|
npm run test -w @gavel/e2e | Unit tests of the DSL, runners, report and harnesses |
npm run test:browser -w @gavel/e2e | Playwright specs (CI) |
npm run test:sandbox -w @gavel/e2e | Real-sandbox specs; skipped without secrets |
npm run measure:sandbox -w @gavel/e2e | Per-call latency against the sandbox |
npm run measure:settlement -w @gavel/e2e | Hammer to captured and settled on the sandbox |
npm run report:sandbox -w @gavel/e2e | Builds the redacted sandbox report |
npm run gate:g5 -w @gavel/e2e | The G5 live harness; needs recordings and a provider key |
From the repository root, npm run test:scenarios runs the project unit:e2e and npm run test:property runs the property projects.
Tests that guard it
test/dsl.test.ts (schema), runner-inproc.test.ts and runner-process.test.ts (the runners), scenarios.test.ts, s02-s05-s15-scenarios.test.ts, s06-s09-scenarios.test.ts, s11-s14-scenarios.test.ts and s10-abusive-input.test.ts (the scenarios), stack-mode.test.ts and stack-scenarios.test.ts (child-process mode), property-harness.test.ts, sandbox-harness.test.ts, sandbox-report.test.ts, flakes.test.ts and repo-contracts.test.ts, which reads the repo files to check that workflows and scripts stay wired (for example that the dependency audit runs and stays outside the required roll-up).
- packages/e2e/src/dsl/schema.tsthe scenario schema
- packages/e2e/src/runner/inproc.tsin-process runner
- packages/e2e/src/runner/stack.tschild-process runner with kill and restart
- packages/e2e/src/runner/invariants.tsI1 to I5
- packages/e2e/scenariosthe fifteen scenarios
- packages/e2e/playwright.config.tsbrowser projects
- packages/e2e/vitest.sandbox.config.tssandbox spec config
Last updated on