Voice Agent Framework

Deterministic vs Non-Deterministic Pipeline Architecture

Project: noboxAI Voice Receptionist Date: 2026-04-06 Status: LIVE — 4 Tenants Tests: 104 passing

The Core Tension

The LLM produces excellent conversation but can't reliably execute actions. Deterministic code guarantees business-critical outcomes. They run in parallel, not in series.

Non-Deterministic — LLM Owns

Conversation Quality

Natural dialogue, gate qualification, knowledge weaving, caller name capture. 100% quality across 25 scenarios.

Deterministic — Code Owns

Business-Critical Actions

Transfer, hangup, callback booking, post-transfer silence. 100% reliability via 4-layer fallback.

100%

Conversation Quality

100%

Transfer Reliability

~0%

Native Tool Call

Re-Ask Rate

293ms

Avg Latency

104

Tests Passing

Audio Processing Pipeline

Caller audio passes through 4 deterministic defense layers before reaching the non-deterministic STT and LLM. Each layer reduces noise at a different level.

DTLN
Noise Filter
+8ms

→

Silero VAD
Energy Gate
0.38 threshold

→

Whisper STT
Speech→Text
hallucination risk

→

Confidence
Filter
<0.4 rejected

→

SlotTracker
13 patterns
regex extract

→

PolicyEngine
State Machine
6 actions

→

Qwen3-8B
Conversation
gate flow

→

Kokoro TTS
Text→Speech
1.15x speed

Full Pipeline — Deterministic (blue) wraps Non-Deterministic (amber)

Component Classification

Component	Type	Role	Failure Mode
DTLN	Deterministic	Noise amplitude reduction	Passes noise through (graceful)
Silero VAD	Deterministic	Speech energy detection	Threshold tradeoff: sensitivity vs noise
Whisper STT	Non-Deterministic	Speech → text	Hallucinate text from ambient noise
STT Confidence	Deterministic	Reject low-confidence text	Dormant (confidence=0.0 from plugin)
SlotTracker	Deterministic	Extract caller info (13 regex)	False positive extraction
PolicyEngine	Deterministic	Transfer/callback/hangup rules	Regex coverage gaps for new phrasings
Qwen3-8B LLM	Non-Deterministic	Conversation + gate flow	Re-asks, 0% tool calls, instruction drift
Prompt Gates	Non-Deterministic	3-gate qualification	May skip, re-ask, or bypass
Kokoro TTS	Deterministic	Text → speech	Pronunciation issues

The Parallel Processing Model

When a caller speaks, two independent systems process simultaneously. The deterministic path (PolicyEngine) runs synchronously in the event handler. The LLM path runs asynchronously. By setting the transfer flag before creating the async task, the deterministic path always wins the race.

Parallel Processing — Deterministic path wins the race

Design Patterns

Pattern 1: Gate as New-Information Question

Problem

Qwen re-asks classification questions ("business or personal?") even when the caller already said "at my business." Re-ask rate was 60%.

Solution

Frame gates around information the caller hasn't provided. SlotTracker injects filled slots into LLM instructions: "Caller told you: service=shredding. Do NOT re-ask." Re-ask rate → 4%.

Slot Injection — Prevents re-asking at the LLM instruction level

Pattern 2: Decision Point → Disarm on Callback

Problem

"Yes" confirming phone number was triggering transfer (false positive).

Solution

PolicyEngine tracks decision state. When caller chooses callback, decision_offered resets to False. Subsequent "Yes" is phone confirmation, not transfer.

Pattern 3: 4-Layer Transfer Fallback

Layer	Type	Trigger	Latency	Success Rate
1: Native tool call	Non-Det	Qwen fires transfer_call	0ms	~0%
2a: Caller demand	Deterministic	"Connect me now" on speech	<100ms	~95%
2b: Caller confirm	Deterministic	"Yes" after decision point	<100ms	~95%
3: Agent text fallback	Deterministic	"Let me connect you" in output	5-30s	~80%

Pattern 4: LLM-Level Silence via Generator Override

After transfer fires, llm_node override returns empty async generator — stops LLM generation at the source. No text generated → no TTS → no audio → no "ghost talking" to ambient noise.

tts_node gate provides defense-in-depth: drains text stream if transfer flag set mid-generation.

PolicyEngine State Machine

INIT │ ├─ caller: "connect me now" ──→ ACTION: transfer (demand) │ ├─ agent: "connect now or callback?" ──→ decision_offered = True │ │ │ ├─ caller: "call me back" ──→ callback_chosen = True, decision_offered = False │ │ │ │ │ ├─ agent: "I'll call you at 10" ──→ callback_confirmed = True │ │ │ │ │ │ │ └─ agent: "have a great day" ──→ ACTION: end_call │ │ │ │ │ └─ caller: "yes" ──→ ACTION: none (phone confirm, NOT transfer) │ │ │ └─ caller: "yes / go ahead" ──→ ACTION: transfer (confirm) │ └─ agent: "let me connect you" ──→ ACTION: transfer (fallback) └─ UNLESS callback context ──→ ACTION: none (suppressed)

Scorecard Baseline

25 scenarios across 4 tenants (GDS Atlanta, CRU Insurance, noboxAI, North Group RE). Streaming mode, production prompts.

Component Reliability — Deterministic vs Non-Deterministic Contribution

Metric	Deterministic Contribution	Non-Deterministic Contribution
Transfer reliability	100% (PolicyEngine fallback)	8% (native tool call)
Gate flow	83% (SlotTracker injection)	83% (LLM follows instructions)
Conversation quality	0%	100% (LLM excels here)
Noise rejection	~80% (DTLN + VAD + interruption)	0% (Whisper hallucinates)
Re-ask prevention	~96% (slot injection)	4% (LLM still re-asks)

VAD Tuning History

Version	Threshold	min_speech	Noise Events	Missed Speech	Verdict
v1 (original)	0.30	0.05s	6+/call	None	Too noisy
v2	0.35	0.15s	5+/call	None	Still noisy
v3 (aggressive)	0.45	0.25s	1-2/call	YES	Too aggressive
v4 (current)	0.38	0.15s	1-2/call	TBD	Balanced

Configuration Decisions

1. VAD threshold per tenant?

Recommended: B — Per-tenant from Redis. Noisy callers need higher threshold. Quiet environments can be more sensitive.

Options: A) Global (0.38) — simple. B) Per-tenant from Redis — adaptive. C) Dynamic per-call — complex.

2. STT confidence data source?

Recommended: B — Custom Whisper wrapper. Highest control, moderate effort. Exposes no_speech_prob + avg_logprob.

Options: A) Wait for OpenAI plugin — passive. B) Custom Whisper wrapper — active, 4-8h effort. C) Switch to AssemblyAI — cloud dependency.

3. Transfer confirmation audio?

Recommended: A — session.say() before silence. Best CX per warm transfer best practices. Already deployed.

Options: A) session.say() — verbal confirm, bypasses LLM. B) Silent transfer — no confirmation. C) Pre-recorded file — static.

4. Callback timeout behavior?

Recommended: B — 15s timeout → end_call. Prevents orphan calls where caller hung up but room stays open.

Options: A) Wait indefinitely. B) 15s timeout. C) 30s timeout.

5. Slot patterns: universal or tenant-specific?

Recommended: A for now. Universal 13 patterns. Migrate to B when false positives emerge per-tenant.

Options: A) Universal (current). B) Tenant-specific subset from config. Budget slot is already $-prefix-only to prevent false positives.

Deferred Priorities & Roadmap

Item	Priority	Gate	Status
Small transfer classifier (replaces regex)	Medium	100+ call summaries	Designed
Post-generation repetition filter	Low	Re-ask > 10%	Not needed (4%)
Tool-tuned SLM evaluation	Watch	Qwen3.1 / Llama 4 release	External
Custom Whisper wrapper (confidence)	High	Decision #2 above	Designed
Callback timeout (15s)	Medium	Decision #4 above	Designed

Dependency Chain — What unblocks what

Voice Agent Framework Architecture Briefing — noboxAI, 2026-04-06

Source: 35 commits, 15+ live calls, 25 scorecard scenarios, 104 unit tests

Generated from docs/value-delivered-2026-04-06.md + docs/scorecard-baseline-2026-04-05.md + docs/findings-ambient-noise-2026-04-06.md