ADW Pipeline System Specification

Agent DevOps Worker — System Architecture & Decision Briefing

Date: 2026-03-07 Author: Jonathan Gatlit Type: Internal Infrastructure
Pipeline: Operational
15/15 Issues Complete
5 Decisions Open
22 Repos Enrolled

System Status

100%
Completion Rate
15
Issues Processed
22
Repos in Allowlist
270
Test Count
11
Pipeline Steps

Pipeline Architecture

The 11-step pipeline processes GitHub issue events from webhook receipt through autonomous PR creation. Each step has explicit verification before proceeding to the next.

Webhook-to-PR Pipeline

Loading pipeline diagram...
Webhook Receive
Redis Queue
Classify
Plan Phase
Implement
Validate
Safety Gate
Create PR

Module Architecture

Module Interaction Map

Loading module diagram...
Module Responsibility Role
adw_pipeline.py Pipeline orchestration, step sequencing, output extraction Core
worker.py Job processing, issue-level dedup (Redis SETNX), cleanup Core
agent.py Claude CLI invocation, JSONL parsing, template execution Core
safety_gate.py Pre-PR 3-tier validation (forbidden files, diff size, secrets) Guard
preflight.py Repo prep, build step, submodule handling Setup
config.py Multi-repo routing, workdir resolution, env vars Config
github_ops.py GitHub API calls (issue fetch, comments, PR creation) IO
webhook.py / main.py FastAPI webhook endpoint, event validation IO

Template System

Template Selection Flow

Loading template diagram...
TemplatePhasePurpose
bug.mdPlanningBug fix planning with reproduction steps
feature.mdPlanningFeature planning with scope definition
chore.mdPlanningMaintenance/chore task planning
implement.mdImplementationCode implementation with scope discipline
commit.mdCommitCommit with submodule push
pull_request.mdPR CreationPR with 8 positional arguments
compliance_check.mdVerificationPost-implementation compliance check
frontend_handoff.mdConditionalVisual validation handoff

Trigger Mechanisms

Activation Paths

Loading trigger diagram...
TriggerGitHub EventExample
/adw commentissue_commentComment "/adw" on any issue
adw-* labelissues.labeledApply "adw-ready" label
/adw in bodyissues.openedOpen issue containing "/adw"

Infrastructure Stack

Component Distribution

Loading infrastructure diagram...

Architectural Decisions

D1: Should the pipeline support batched issue processing?
Recommended: Keep single-issue (Option A)
100% success rate at current volume. Batch adds complexity (context overflow risk, priority ordering) without demonstrated need. Revisit at 50+ issues/week.
D2: How should the pipeline handle long-running implementations?
Recommended: Pre-flight decomposition (Option C)
Detect scope complexity before committing to implementation. Split oversized issues into sub-issues automatically. Better than fighting context limits mid-execution.

D2: Decomposition Flow

Loading decision diagram...
D3: What retry/backoff strategy for GitHub API failures?
Recommended: Exponential backoff + jitter (Option B)
Standard resilience pattern. Prevents thundering herd on rate limit recovery. Max 5 retries, max delay 32s + jitter, give up with failure comment.
D4: Should ADW completion update CSA handoff manifest?
Recommended: Post-PR comment trigger (Option C)
Natural extension of step 11 (completion comment). Adds manifest update as step 11.1. Low friction, high traceability, enables GitReport flywheel detection.
D5: How should the repo allowlist scale?
Recommended: Hybrid convention + denylist (Option C)
Convention file (.adw-enabled) auto-enrolls repos. Denylist for explicit exclusion. Manual curation doesn't scale past 30 repos. Hybrid maintains safety with reduced friction.

D5: Allowlist Scaling Comparison

Loading allowlist diagram...

Deferred Priorities

LangGraph Migration Path
Activation: Pipeline volume exceeds 50 issues/week OR complex dependency chains require graph-aware scheduling.
Bridge: Current sequential processing handles present volume.
Multi-Worker Parallelism
Activation: Average queue depth consistently exceeds 3 OR p95 latency exceeds SLA.
Bridge: PM2 can scale instances. Redis already supports competing consumers.
Frontend Validation Automation
Activation: ADW processes 3+ frontend-heavy repos regularly.
Bridge: Manual review via PR screenshots.

Phase Transition Gates

  • - Volume exceeds 50 issues/week → activate multi-worker
  • - 3+ context overflow incidents → evaluate LangGraph
  • - 30+ repos in allowlist → convention-based discovery
  • - 3+ frontend PRs/week → Playwright validation

Appendix

A1: Safety Gate Implementation (3-Tier)
TierCheckThresholdAction on Fail
1Forbidden files.env, secrets.*, CI configsBlock PR
2Diff sizeMax lines changed (configurable)Warning + manual review
3Secrets scanPattern-based (API keys, tokens)Block PR
A2: Redis Queue Schema
KeyTypePurpose
adw:queueLISTFIFO job queue
adw:lock:{repo}:{issue}SET (NX, TTL 1h)Issue-level dedup

Job payload: {"repo", "issue_number", "event_type", "trigger", "enqueued_at"}

A3: Convention Files (Per-Repo Opt-In)
FilePurposeFormat
.adw-tests.jsonCustom test command{"command": "..."}
.adw-build.jsonBuild step config{"command": "...", "timeout": N}
.adw-security.jsonSecurity scan patterns{"patterns": [...]}
_handoff.jsonCSA → ADW manifestSee handoff spec
A4: Multi-Repo Routing Examples
GitHub RepoLocal Workdir
jgatlit/HHE-app~/projects/HHE-app/
jgatlit/JuxtMedia-agent~/projects/JuxtMedia-agent/
jgatlit/secondact-*~/projects/secondact/