CSA Pipeline System Specification

Internal infrastructure review and architecture decisions

2026-03-07 From: Jonathan Gatlit Internal Infrastructure
18Engagements
2Active Flywheels
5Decisions Needed
12Phases

Pipeline Architecture

The CSA pipeline transforms unstructured project inputs into deployed HTML artifacts with structured decision capture through a 12-phase rigid skill chain.

HandoffPhase 0-2
Spec AuthPhase 3
Visual ArtifactPhase 4
ValidatePhase 5-6
DeployPhase 7
QA + PublishPhase 8-11
18

Total Engagements

212+

Files Produced

16 MB

Workspace Size

4

Ext Dependencies

Fig 1: Pipeline phase flow with dependency classification

External Dependencies

DependencyEndpointCritical?SLA
FTP Serverftp://151.106.96.76YES~99.5%
Mermaid APIchart.chem.dev/api/v1YES~99.9%
Slack Notifyio.noboxai.com/notify.phpNO~99.5%
PlaywrightLocalNOLocal

Actionable Now

1.1 Partial Failure Handling

Critical

The pipeline has 4 external dependencies. Currently, any single failure blocks the entire run. Implement graceful degradation with critical/non-critical classification.

Fig 2: Graceful degradation flow for partial failures

  • Implement try/catch wrapper for Slack notification (Phase 10.5)
  • Add --skip-qa flag for re-deploy scenarios
  • Add retry logic (max 2, 5s backoff) for FTP failures

1.2 Dashboard Auto-Generation

Recommended

Dashboard is manually maintained. At 18 projects and growing, a hybrid manifest-driven approach would reduce maintenance while preserving control.

Fig 3: Manifest-driven dashboard generation

  • Define _manifest.json schema (reconcile with _handoff.json)
  • Build generate-dashboard.py script
  • Preserve manual summary/status overrides

1.3 Handoff Manifest Rollout

Required for New

Phase 10.1 was just added. _handoff.json should be required for all new engagements, with optional backfill for existing 18.

Fig 4: _handoff.json rollout timeline

  • Define _handoff.json schema
  • Validate schema against GitReport flywheel needs
  • Write for all new engagements
  • Backfill active engagements (HHE, JuxtMedia, SecondAct, FLY, Melody Keel)

5 Architectural Decisions

These decisions shape the next iteration of the CSA pipeline system. Each has a recommendation based on current operational patterns.

#DecisionOptionsRecommended
D1Partial failure handlingHalt-all / Graceful degrade / Retry queueGraceful degrade
D2_handoff.json requirementAll / New only / OptionalNew only
D3Flywheel version numberingSequential / Date-based / HybridSequential
D4Quality thresholdsQA only / Score+age / Score+resolved%Score + resolved %
D5Dashboard generationFull auto / Manual / Hybrid manifestHybrid manifest

Fig 5: Quality threshold decision flow for flywheel vs. fresh run

Decision capture: These 5 decisions are available in the companion Input Form. Responses drive the next implementation phase for pipeline improvements.

Current → Deferred Unlock Chain

Current ItemUnlocksGate
D1 (Partial failure)LangGraph reimplementation error modelFailure handling pattern defined
D2 (Handoff manifest)GitReport CSA-ADW conversion detection_handoff.json schema finalized
D5 (Dashboard gen)Response analytics dashboardManifest includes response metadata
1.3 (Manifest rollout)Multi-operator supportManifest = portable engagement state

Phase Transition Gates

Future Priorities

LangGraph Reimplementation

Scale-Dependent

Code-based CSA pipeline with typed state, interrupt gates, and parallel fan-out. Already spec'd at io.noboxai.com/csa-lg-architecture/.

Activation: When 3+ decisions resolved AND skill-chain hits scale ceiling (>30 concurrent engagements or >3 failures/week).

Multi-Operator Support

Future

Pipeline is single-operator. No auth, no role separation. Activates when a second operator needs independent CSA pipeline access.

Response Analytics Dashboard

Future

Cross-engagement response analytics. Activates when 5+ engagements complete flywheel cycles with captured responses.

Operational Risks

RiskImpactLikelihoodMitigation
FTP host downtimeDeploy blockedLowStage locally, retry after 30min
Mermaid API breaking changeDiagrams failLowPin server version, test before upgrade
SKILL.md drift from behaviorPipeline errorsMediumMonthly full-pipeline integration test
Dashboard row countUX degradationMediumAuto-gen with pagination at 30 projects
_handoff.json schema changeInconsistencyLowVersion schema, migrate on read