SecondActSociety RAG Pipeline V2 — Implementation Planning

v1 Outcomes

All 5 Architecture Decisions Confirmed

Every decision from the v1 briefing was confirmed by Skip. The architecture is validated and ready for implementation sprint planning.

Latency Target

Connection Pool Warming

Observability Stack

LangSmith Only

Enhancement Priority

Conversation Memory (LIVE)

Graph Traversal Depth

3 Chunks

Fallback Strategy

Broad Match

Architecture Update

Conversation Memory is LIVE

Spec Correction

The v1 briefing listed conversation memory as a deferred enhancement priority. Skip confirms this is already implemented — the 5 most recent messages/turns are included in retrieval context via Redis session state. This corrects the v1 specification and removes conversation memory from the enhancement backlog.

Updated Pipeline — Conversation Memory Active

Redis session persistence is active. Each query includes the 5 most recent conversational turns, enabling context-aware retrieval and response generation. The session state informs intent classification confidence and provides continuity across multi-turn interactions.

Implementation

Confirmed Implementation Priorities

Latency: Connection Pool Warming

What: Pre-warm database connections on service startup plus periodic keepalive to maintain warm connection pools.

Why: Railway's ephemeral containers can go idle. When a container has been idle, the first user request incurs a cold-start penalty as database connections are re-established. Connection pool warming is the confirmed #1 latency optimization target.

How: FastAPI startup event initializes connection pools for PostgreSQL/pgvector, Neo4j, and Redis. A background keepalive task periodically pings each connection to prevent idle timeout.

Observability: LangSmith (Current Stack)

What: LangSmith tracing confirmed as sufficient for current observability needs.

Config: LANGSMITH_TRACING=true, project second-act-learning-v2

No additional tooling required at current scale. LangSmith provides trace-level visibility into LangGraph node execution, retrieval quality, and response generation latency.

Configuration

Confirmed Parameters (No Changes)

All retrieval parameters are locked at their current values following v1 confirmation. No tuning required for implementation sprint.

Parameter	Confirmed Value	Status
`PERSONA_RETRIEVAL_COUNT`	3	Locked
`FALLBACK_THRESHOLD`	3	Locked
`VECTOR_WEIGHT`	0.5	Locked
`RRF_K`	60	Locked
`MAX_RAG_RESULTS`	12	Locked

Your Input

Next Enhancement Priority

With conversation memory shipped, the next enhancement slot is open. The following candidates are ranked by estimated impact.

Enhancement Candidates — Weighted Priority

Your Input

With conversation memory shipped, what's the next enhancement priority?

A) Intent Classification Refinement — Improve 7-intent accuracy, reduce false clarification triggers. The Feb 28 clarification routing bug revealed over-triggering on vague/personal queries from known users. Better intent classification has the highest immediate UX impact.
B) A/B Testing Framework — Experiment with retrieval parameters (RRF weights, phase ratios, fallback thresholds). Requires sufficient query volume for statistical significance.
C) Authority Taxonomy Evolution — Add new authority levels or reclassify sources as content library grows. Currently: primary, supplementary, illustrative, supporting.
D) Multi-Modal Content Ingestion — Add video/image content paths to the RAG pipeline. Activates when Jay produces visual content materials.

Your Input

Connection Pool Warming Scope

Your Input

For connection pool warming, which implementation scope?

A) Service Startup Only — Pre-warm all connections (PostgreSQL/pgvector, Neo4j, Redis) on FastAPI startup event. Simple and reliable.
B) Startup + Periodic Keepalive — Pre-warm on startup PLUS background keepalive task to maintain warm connections. Prevents Railway idle timeout from causing cold-start penalty.
C) Lazy + Circuit Breaker — No pre-warming. Instead, add circuit breaker pattern for graceful degradation when connections are cold.