Skip to content

PayPal sandbox only. No real money moves.

BuildersBuildersPackages

@gavel/voice

Audio framing, speech-to-text providers, the bid parser and the decision step that turns speech into a confirm prompt or a bid.

Partialmeasured; some of the bar is met, with limits stated on the page

Page last changed 7 Nov 2026 (from the git history)

packages/voice turns speech into a structured bid, or into a question for the bidder. It never decides whether a bid is valid: the engine does that. Its job ends at a decision (auto, confirm or drop) and, for an accepted one, a PlaceBid command the server submits to the room.

G5 PARTIAL Speech accuracy was measured on synthetic audio only (PARTIAL: the first run missed the threshold, the fixes were chosen on the same clips), so spoken bids always go through the confirm prompt. See Voice and the confirm gate and Gate G5.

Responsibilities

Capture and frame audio

src/audio frames PCM (20 ms frames, 100 ms batches), resamples, finds the end of an utterance and queues it. src/browser holds the browser capture state machine, the mute controller (push-to-talk and open mic, with a keepalive every 5 s while muted) and the degraded-mode handling.

Talk to a speech provider

src/stt has one interface (SttProvider) and five implementations: Deepgram (ReconnectingDeepgramStt), Groq (GroqStt), Web Speech (browser only), FakeStt and ReplayStt (cached transcripts). FailoverStt chains providers and selectProvider picks them from the environment.

Parse and decide

src/parser reads the amount and the paddle or speaker from a transcript with rules. src/decide scores the reading, merges it with an optional model reading, applies the policy thresholds and writes the confirm prompt. src/pipeline/pipeline.ts ties segmentation, parsing and deciding together; bid-command.ts maps a decision to an engine PlaceBid.

Limit the stream

src/session.ts enforces the limits of the /audio socket (byte and message rates, stream caps, idle sweep). It holds no socket and no timer, so it is a pure state holder that the server drives.

Where it sits

The dependency rule in .dependency-cruiser.cjs lets voice import only shared. The server imports voice, and the web client is allowed to. The engine and payments are not imported as values; the package ESLint config allows type-only imports of them.

Public interface

Everything is exported from src/index.ts (main and types both point at it). The groups:

GroupExports
ParsingparseUtterance, parseUtteranceOutcome, ParseResult, PARSE_FLAGS
Decidingdecide, CONFIRM_TTL_MS, Decision, ConfirmPrompt, VOICE_REASONS, buildAudit
PipelineVoicePipeline, STALE_SEGMENT_MS, RETRACT_ADVISORY_MS, toPlaceBid, resolveConfirm, commandIdFromKey
ProvidersselectProvider, STT_NAMES, FailoverStt, ReplayStt, SttProvider, SttEvent
WireencodeBatch, decodeBatch, VoiceFrameError
SessioncreateVoiceSession, AUDIO_ERROR_CODES, AUDIO_IDLE_MS
AudittoAuditRecord, auditRecordSchema, voiceEvidenceSchema
BudgetBUDGET_LIMITS_MS, budgetCheck, latencyOf, percentile

The exported surface is recorded in etc/voice.api.md; a change to it needs the interface label (see Contributing).

Key files

provider.ts
select.ts
failover.ts
deepgram-reconnect.ts
groq.ts
replay.ts
config.ts
policy.ts
decide.ts
validate.ts
pipeline.ts
bid-command.ts
faults.ts
latency.ts
session.ts
privacy.ts
index.ts
FileWhat it holds
src/stt/select.tsselectProvider: unset VOICE_STT tries Deepgram then Groq; a named provider is used or startup fails with unknown_provider, missing_key or unavailable. Errors never contain a key
src/stt/failover.tsFailoverStt: switches provider on a closed connection, an error or a silent stream. Ambient room noise does not arm the silence timer: only a mean-square level above about -35 dBFS counts as speech
src/stt/deepgram-reconnect.tsReconnecting Deepgram client; the audio of the last seconds is replayed after a switch
src/decide/config.tsdefaultConfig: auto 0.85, confirm 0.5
src/decide/decide.tsThe decision step; CONFIRM_TTL_MS is 8000
src/pipeline/pipeline.tsVoicePipeline; a segment older than STALE_SEGMENT_MS (5000) is treated as stale, RETRACT_ADVISORY_MS is 8000
src/pipeline/faults.tsSpeech health: status degraded when no final transcript arrives for 8 s while the level meter shows speech, down after 15 s; transcript sanity checks (2000 characters, character runs)
src/pipeline/latency.tsLatency budget: components sum to the total; limits p50 1200 ms and p95 2000 ms with a model, p50 700 ms and p95 2000 ms rules-only
src/privacy.tsPRIVACY_NOTICE and PERSISTED_VOICE_KEYS, the only voice fields that are stored

Thresholds and limits

Prop

Type

Only the auctioneer and console roles may open an audio stream (AUDIO_ROLES); screens and bidders never can.

Invariants

  • A spoken bid never skips the engine. toPlaceBid sets commandId with commandIdFromKey (a hash of the auction id and the utterance idempotency key), so a replayed utterance maps to the same id and books once, even if two providers heard it. The provider name is not part of the key.
  • Below the auto threshold, the bidder is asked. With no model reading wired into the running pipeline, every bid scores below 0.85 and reaches a confirm prompt (stated in the package README).
  • Only audit fields in PERSISTED_VOICE_KEYS are persisted. The test test/privacy.test.ts guards the list against the audit schema.
  • No key in an error. SttConfigError messages name the provider, never the key.
  • The held-out text set is frozen. fixtures/text/HELDOUT.sha256 is checked in CI against the base branch.

Tests that guard it

AreaTest files
Provider selection and failovertest/stt/select.test.ts, select-deepgram.test.ts, failover.test.ts, deepgram-reconnect.test.ts, groq-retry.test.ts, groq-timeout.test.ts
Decision policy and promptstest/decide/policy.test.ts, score.test.ts, validate.test.ts, prompts.test.ts, wording.test.ts, audit.test.ts
Pipelinetest/pipeline/pipeline.test.ts, faults.test.ts, latency.test.ts, prompt-expiry.test.ts, prompt-identity.test.ts, crosstalk.test.ts
Bid command mappingtest/pipeline-bid-command.test.ts, test/pipeline-bounds.test.ts
Stream limitstest/session.test.ts, test/session-idle.test.ts
Parser corpustest/corpus.test.ts, test/labels.test.ts
Privacy and held-out guardtest/privacy.test.ts, test/heldout-guard.test.ts
Evaluation harnesstest/eval/gate.test.ts, metrics.test.ts, report.test.ts, cli.test.ts
Browser capture and mutetest/browser/capture.test.ts, mute.test.ts, degraded.test.ts, capture-state.test.ts

Scripts

ScriptWhat it does
npm run test -w @gavel/voicePackage unit tests
npm run eval -w @gavel/voice -- --mode live --provider deepgramScores recordings through a real provider; needs a key
npm run scan:secrets -w @gavel/voiceScans the package for vendor key patterns (runs in the ci.yml voice job)
npm run check:heldout -w @gavel/voiceGuards the held-out text set
npm run size -w @gavel/voiceBundle size limit
Edit on GitHub

Last updated on

server verified

The single process that hosts the rooms, the HTTP API, the WebSocket gateway and settlement.

@gavel/web verified

The browser client: routes, the transport seam, the payment approval port, the copy rules and the bundle gate.