ADW Pipeline System

Operational Metrics

Performance Baseline

Issues Processed

100%

Completion Rate

Allowed Repos

Active Client Repos

270

Test Suite Size

~7.5

Avg Comments/Issue

Issue Classification Distribution

Architecture

Pipeline Architecture

The ADW pipeline is a GitHub webhook-driven CI/CD automation system. It receives GitHub issue/comment events, queues them in Redis, and dispatches Claude CLI workers that autonomously plan, implement, test, and PR code changes.

End-to-End Pipeline Flow

Infrastructure Stack

Component	Config	Notes
Process Manager	`PM2: agent-hook-worker`	Auto-restart on failure
Reverse Proxy	Caddy (root-owned)	HTTPS termination
Queue	Redis on port 6383	Issue-level dedup via SETNX
API	FastAPI on port 8200	Webhook receiver
Repo Allowlist	22 repos (manual)	Curated by config.py

Trigger Mechanisms

/adw Comment

Comment /adw on any issue in an allowed repo. Event: issue_comment.created

adw-* Label

Apply any label prefixed adw- to an issue. Event: issues.labeled

/adw in Body

Open a new issue with /adw in the body text. Event: issues.opened

Pipeline

11-Step Pipeline Sequence

Each issue flows through a rigid 11-step pipeline. The compliance layer (Step 9c) self-retriggers on PARTIAL/FAIL results. The safety gate (Step 9e) validates all changes before PR creation.

Pipeline Step Sequence

Classify

→

Branch

→

Plan

→

Implement

→

Validate

→

Safety Gate

→

Modules

Key Modules & Templates

Core Modules

Module	Purpose
`adw_pipeline.py`	Pipeline orchestration, step sequencing, output extraction
`worker.py`	Job processing, issue-level dedup (Redis SETNX), cleanup
`agent.py`	Claude CLI invocation, JSONL parsing, template execution
`safety_gate.py`	Pre-PR 3-tier validation (forbidden files, diff size, secrets)
`preflight.py`	Repo prep, build step, submodule handling
`config.py`	Multi-repo routing, workdir resolution, env vars
`github_ops.py`	GitHub API calls (issue fetch, comments)
`webhook.py` / `main.py`	FastAPI webhook endpoint

Convention Files (Per-Repo Opt-In)

.adw-tests.json

Custom test command override

.adw-build.json

Build step configuration

.adw-security.json

Security scan patterns

_handoff.json NEW

CSA-to-ADW handoff manifest

Templates (`.claude/commands/`)

Template	Purpose
`bug.md`, `feature.md`, `chore.md`	Planning templates (per issue type)
`implement.md`	Implementation with scope discipline
`commit.md`	Commit with submodule push
`pull_request.md`	PR creation (8 positional args)
`compliance_check.md`	Post-implementation verification
`frontend_handoff.md`	Visual validation handoff

Safety

Safety Gate Architecture

Step 9e implements a 3-tier validation gate that blocks PR creation if any tier fails. This is the last checkpoint before code reaches the PR stage.

3-Tier Safety Gate

Tier 1: Forbidden Files

Blocks modifications to protected infrastructure files (PM2 configs, Caddy, systemd).

Tier 2: Diff Size Check

Flags oversized diffs that may indicate scope creep or unintended changes.

Tier 3: Secrets Scan

Detects API keys, tokens, passwords, and other secrets in the diff.

Decision Points

Strategic Decisions

Five architectural decision points for future ADW scaling. All have recommendations — confirm or adjust via the input form.

D1: Should the pipeline support batched issue processing?

Recommended: Option A — Single-issue (current)

Throughput is not a bottleneck at 15 issues total. Single-issue processing keeps failures isolated and logs clean. Batching adds shared context contamination risk without proportionate value.

D2: How to handle implementations exceeding Claude CLI context?

Recommended: Option B — Checkpoint + Resume (v2)

Checkpoint at step boundaries and resume from last successful step. Current behavior (fail + notify) loses progress. Decomposition into sub-issues (Option C) requires issue graph logic — a future consideration.

D3: What retry/backoff strategy for GitHub API failures?

Recommended: Option B — Exponential backoff (3 retries)

Standard pattern: 1s/2s/4s delays. Current no-retry behavior is fragile on transient errors. Circuit breaker (Option C) is overkill for current scale.

D4: Should ADW auto-update CSA _handoff.json back-link?

Recommended: Option B — Auto-update manifest

ADW writes adw_issues_created array and updates status: handed-off on completion. Full traceability. GitReport can reconcile independently as a secondary check.

D5: How should the allowlist scale?

Recommended: Option B — Convention-based auto-discovery (v2)

Repos with .adw-pipeline.json auto-enroll. Allowlist becomes deny-list for exceptions. Good for current 22 repos, essential when team grows or repo count exceeds 30.

Deferred

Deferred Priorities

Item	Activation Trigger	Bridge Strategy
DP-1: Checkpoint/Resume	First context exhaustion failure	Manual `--resume` with issue link
DP-2: Auto-Discovery	Allowlist exceeds 30 repos OR team grows	Manual addition (<1 minute)
DP-3: Parallel Workers	Queue depth regularly exceeds 5	Not needed (current depth < 2)

Dependency Map

Decision-to-Deferred Dependencies

Phase Transition Gates

○ DP-1 activates when: context exhaustion failure occurs AND D2 decision confirmed
○ DP-2 activates when: allowlist > 30 repos OR team grows beyond 1
○ DP-3 activates when: queue monitoring shows sustained depth > 5

Risk Register

Risk Assessment

Risk Positioning (Likelihood vs Impact)

Risk	Likelihood	Impact	Mitigation
Context exhaustion mid-implementation	Medium	High	DP-1: Checkpoint/resume (deferred)
Redis queue corruption	Low	High	PM2 auto-restart + queue flush script
GitHub API rate limiting	Low	Medium	D3: Exponential backoff (recommended)
Secrets in PR diff	Low	Critical	Safety gate Tier 3 (active)
Allowlist drift (stale repos)	Medium	Low	Periodic audit (manual)

Appendix: CSA-to-ADW Handoff Convention

The _handoff.json manifest bridges CSA pipeline output to ADW pipeline intake. Lifecycle: draft (CSA complete) → ready (user promotes) → handed-off (ADW processes) → archived (engagement complete).

HITL constraint: CSA never auto-triggers ADW without explicit user instruction.

Field	Type	Purpose
`engagement`	string	CSA slug identifier
`status`	enum	draft \| ready \| handed-off \| archived
`target_repo`	string	ADW target (e.g. owner/repo)
`adw_handoff_authorized`	boolean	Explicit user authorization
`quality_gate`	object	Spec exists, artifact count, diagrams, deployment status
`adw_issues_created`	array	Backfilled by ADW on completion

Appendix: Issue Timeline (All 15 Issues)

#	Type	Title	Created	Comments
17	PR	chore: #16 — Add pre-PR safety gate	2026-02-27	0
16	Issue	Add pre-PR safety gate — validate diff size, forbidden files, secrets	2026-02-27	8
15	PR	chore: #14 — Add pre-PR rebase step	2026-02-27	0
14	Issue	Add pre-PR rebase step to prevent merge conflicts	2026-02-27	8
13	PR	bug: #12 — Retrigger runs produce zero commits	2026-02-27	1
12	Issue	Retrigger runs produce zero commits — trigger_comment not passed	2026-02-27	14
11	PR	bug: #10 — Compliance retrigger creates duplicate branches/PRs	2026-02-26	0
10	Issue	Compliance retrigger creates duplicate branches/PRs	2026-02-26	8
9	PR	feat: #8 — Add compliance layer with retrigger	2026-02-26	0
8	Issue	Add compliance layer (Step 9c) with retrigger and security	2026-02-26	8
7	PR	bug: #6 — Pipeline failures: stale workers, commit timeout	2026-02-25	0
6	Issue	ADW pipeline failures: stale workers, commit timeout, dirty workdir	2026-02-25	7
5	Issue	Add deterministic build step (.adw-build.json convention)	2026-02-20	6
4	PR	feat: #3 — Multi-repo routing via convention + allowlist	2026-02-19	0
3	Issue	Multi-repo routing via convention + allowlist	2026-02-19	7