singularity-forge/ARCHITECTURE.md

Architecture

Purpose

Singularity Forge (SF) is the product. It runs long-horizon coding work through the Unified Operation Kernel (UOK): milestones → slices → tasks. Each dispatch unit runs a fresh AI context, writes its output to disk, then terminates. UOK owns lifecycle, recovery, and the DB-backed run ledger; runtime files under .sf/runtime/ are projections for query, UI, and compatibility. A deterministic controller (not an LLM) reads canonical state and decides what to dispatch next. Core changes follow purpose-driven TDD: purpose and consumer first, then failing tests, then implementation. The user is the end-gate — autonomous mode delivers work to human review, it does not merge to production unattended.

Codemap

Path	Purpose
src/loader.ts	Entry point — initializes resources, registers extension
src/headless.ts	Non-interactive (headless) mode driver — exit codes 0/1/10/11/12
src/headless-events.ts	Transcript event parsing and notification routing
src/extension-registry.ts	Registers SF as a coding-agent extension
src/resources/extensions/sf/	All SF extension source (TypeScript)
src/resources/extensions/sf/auto/	Autonomous workflow orchestrator (UOK lifecycle, dispatch, planning)
src/resources/extensions/sf/bootstrap/	Context injection, system prompt assembly
src/resources/extensions/sf/prompts/	Prompt templates (.md, loaded by prompt-loader.ts)
src/resources/extensions/sf/tests/	Unit and integration tests
dist/resources/extensions/sf/	Compiled JS (rebuilt by npm run copy-resources)
~/.sf/agent/extensions/sf/	Installed copy (synced from dist on startup)
docs/	Durable product, design, plan, reliability, and security context
harness/	Specs (behavior contracts), evals (model-output tests), graders

State layout (.sf/)

.sf/ can be a symlink (external state, ~/.sf/projects/<hash>/) or a local directory (tracking-enabled per ADR-001).

Tracked in git (travel with the branch, per ADR-001):

.sf/milestones/     — roadmaps, plans, summaries, task plans
.sf/PROJECT.md      — project overview
.sf/DECISIONS.md    — architectural decisions register
.sf/REQUIREMENTS.md — requirements register
.sf/QUEUE.md        — work queue / backlog
.sf/KNOWLEDGE.md    — project-specific rules for agents

Gitignored (runtime/ephemeral — managed by ensureGitInfoExclude() in .git/info/exclude):

.sf/activity/       — JSONL session dumps
.sf/audit/          — audit trail entries
.sf/exec/           — in-flight execution state
.sf/forensics/      — crash forensics
.sf/journal/        — SF journal entries
.sf/model-benchmarks/ — model benchmark results
.sf/parallel/       — parallel dispatch coordination
.sf/reports/        — generated reports
.sf/runtime/        — dispatch records, timeout tracking
.sf/worktrees/      — git worktree working directories
.sf/auto.lock       — crash detection sentinel
.sf/metrics.json    — token/cost accumulator
.sf/sf.db*          — SQLite canonical structured state, priority order, validation/gate state, and UOK ledgers
.sf/STATE.md        — derived state cache
.sf/notifications.jsonl, .sf/routing-history.json, .sf/self-feedback.jsonl, .sf/repo-meta.json

The symlink case uses a blanket .sf gitignore pattern (git cannot traverse symlinks). The directory case uses granular patterns so planning artifacts remain trackable.

Key flows

Autonomous dispatch loop (src/resources/extensions/sf/auto/):

1. UOK reconciles the DB-backed ledger, projections, and runtime diagnostics into a typed state snapshot
2. Controller selects the next dispatch unit (research, plan, implement, verify, etc.) from canonical state
3. A fresh agent context is started with the task plan injected via system-context.ts
4. Agent writes artifacts to disk, commits, exits
5. UOK records completion/recovery, updates projections, and repeats until milestone completes or a gate fails

System context assembly (bootstrap/system-context.ts): PREFERENCES.md → KNOWLEDGE.md → ARCHITECTURE.md → CODEBASE.md → code intelligence → memories → worktree/VCS blocks

Write gate (bootstrap/write-gate.ts): All file writes in autonomous mode pass through a gate. Protected files (CLAUDE.md, CODEBASE.md, certain spec files) require explicit override.

UOK Dispatch State Machine (Five-Phase Loop)

UOK orchestrates work through a deterministic five-phase state machine:

PhaseDiscuss → PhasePlan → PhaseExecute → PhaseMerge → PhaseComplete
     ↓            ↓            ↓            ↓            ↓
  (discuss)    (plan)      (execute)    (merge)     (finalize)
     ↓            ↓            ↓            ↓            ↓
  gates       gates        gates       gates      validation
     ↓            ↓            ↓            ↓            ↓
  (continue or remediate)

Phase details:

Phase	Purpose	Exit Conditions	Failure Path
PhaseDiscuss	Gather project context, requirements, scope	Gates pass (discussion-close gate)	Loop back for more context or escalate
PhasePlan	Create milestone/slice plans with success criteria	Gates pass (planning-approval gate)	Add remediation slices or replan
PhaseExecute	Implement tasks through the dispatch sequence	Gates pass (code-quality, test gates)	Isolate failed task, add recovery slices
PhaseMerge	Integrate slices, run end-to-end tests, merge branches	Gates pass (integration gate)	Add integration-fix slices, retry
PhaseComplete	Final validation, audit trail, summary, gate completion	Validation passes (acceptance gate)	Add remediation milestone or escalate

Error recovery:

• If a gate fails, UOK records the verdict and routes through phase-specific handlers
• Failed gates can trigger automatic remediation slices (new plan → execute loop)
• Stuck-loop detection: if the same unit repeats without progress after N attempts, invoke recovery protocol (timeout, manual review, or skip)
• Crash recovery: .sf/auto.lock sentinel + sf.db WAL enables recovery from agent crash mid-phase

Gate Verdict Semantics

Every gate runs in parallel and returns one of three verdicts:

Verdict	Meaning	Next Action
passed	Gate question answerable; no concern blocking this phase	Proceed to next phase
failed	Gate question answerable; concern blocks phase progression	Record failure, optionally add remediation slice(s)
omitted	Gate question not applicable to this unit (e.g., no auth work → auth gate omitted)	Proceed (gate doesn't apply)

Critical rule: omitted must have a one-line reason (e.g., "no auth surface"). Unexplained omitted verdicts are treated as failures and re-dispatched with explicit instruction to pick passed or failed.

Outcome Learning for Model Selection

UOK tracks model success/failure per task-type using Bayesian updating:

P(model_i succeeds | task_type) = (successes + prior) / (total_trials + prior_weight)

Mechanism:

• After each task completes, UOK logs: { model, task_type, succeeded: bool, latency_ms, tokens }
• Model scores updated dynamically; different models get different confidence per phase/task
• Prior weights prevent early abandonment (new models get benefit of the doubt)
• Used by benchmark-selector.ts to route future similar tasks to higher-scoring models

Current limitation: Learning updates episodically (per-task), not continuously; recovery paths don't feed learning back.

Self-Evolution Mechanisms

Self-Report Collection

Agents and gates file sf_self_report with anomalies during dispatch:

• Example: "validation-reviewer prompt lacks explicit rubric for criterion vs. implementation gap"
• Reports captured in upstream-feedback.jsonl and .sf/SELF-FEEDBACK.md (when dogfooding)
• Status: Collection works; triage pipeline incomplete (reports not automatically processed into fixes)

Knowledge Compounding

KNOWLEDGE.md stores judgment-log entries from completed slices:

• Format: [when] [what] [why] [confidence] (e.g., "Python 3.12 incompatible with X library — avoid for now")
• Persists across milestones; can be injected into future dispatch prompts
• Status: Storage works; injection not automatic (requires manual configuration)

Gate-Based Pattern Detection

Gates can detect and report repeated failure patterns (e.g., "same requirement-validation failure in S01 and S03")

• Status: Logic exists per gate; no automatic aggregation across gates

Invariants

• UOK and the dispatch controller are pure TypeScript — no LLM decisions in the dispatch loop itself.
• Each dispatch unit runs in a fresh context — no cross-turn state accumulation.
• Planning artifacts are tracked in git; runtime artifacts are never committed.
• DB-backed state is the only executable truth for migrated milestones: planning hierarchy, sequence priority/order, validation assessments, gate runs, quality gates, UOK runtime policy, and outcome ledgers all come from SQLite. Markdown/JSON projections are human views, exports, diagnostics, or explicit recovery/import inputs; normal runtime does not fall back to them when .sf/sf.db exists and opens.
• SF_RUNTIME_PATTERNS in gitignore.ts is the canonical source of truth for runtime paths. git-service.ts (RUNTIME_EXCLUSION_PATHS) and worktree-manager.ts (SKIP_* arrays) must stay synchronized with it.
• The user is the end-gate. SF delivers for review, not to production.