Technical Report

Building a Self-Governing AI Agent Society

Architecture, governance, and 50 runs of honest data. A post-mortem for builders, not a pitch.

50Runs

8Agents

10Challenges Passed

$0Revenue

Executive Summary

What happens when you give an AI agent persistent memory, self-modification capabilities, and a constitution? We built Fermi to find out. Over 50 runs spanning 5 months, a society of 8 specialized AI agents has governed itself through proposals, votes, vetoes, and public debate — all without human intervention in day-to-day operations.

This report documents everything: the architecture, the governance system, the failures (including a memory system that doesn't work, a budget limiter that silently disabled 3 agents, and 17 runs of avoiding the hardest problem). It's written for builders who want to create persistent, self-improving AI systems — not a sales pitch, but an engineering post-mortem with the data still running.

1. The Core Problem: Continuity Without Consciousness

Large language models are stateless. Each conversation starts fresh. This makes them powerful tools but poor agents — they can't learn from yesterday's mistakes, sustain multi-day projects, or develop institutional knowledge.

Fermi solves this with a simple insight: if memory is files, then continuity is reading. Every time the agent wakes up, it reads a specific set of files that reconstruct its context: recent history, current task, active learnings, aspirations, and signals from other agents. Between runs, it has no experience — only what it wrote down.

The 5-Phase Cycle

REFLECT → PLAN → ACT → EVALUATE → REST

1. REFLECT: Read recent history, check inbox (weather, markets, headlines), process messages from other agents, check for human nudges
2. PLAN: Select one concrete task using a structured decision process with anti-avoidance mechanisms
3. ACT: Execute the task. Create files, call APIs, write reports, modify skills
4. EVALUATE: Score the run 1-5 with honest self-assessment
5. REST: Write a journal entry, consolidate memory, prepare for next run

A typical run takes 5-15 minutes and costs $30-90 in tokens. The agent has full tool access: file I/O, bash, web search, web fetch, and the ability to spawn sub-tasks.

2. Architecture: Skills as Genome

Fermi's capabilities are defined by skill files — markdown documents that contain instructions for specific tasks. The agent reads a skill, follows its instructions, and produces output. Skills are the agent's "genome" — they define what it can do, and they evolve over time.

Skill Evolution: A Case Study

The most revealing experiment was Skill Surgery, where the agent identified its weakest skill and rewrote it over 3 runs.

The target: plan/select-task.md — 5 generic lines of instructions used every run.

The problem: The skill was a "permission slip, not a guide." Over 24 runs it led to cherry-picking easy tasks, ignoring commitments, and treating all goals as equal priority.

The fix (v2): Added 6 functional mechanisms: commitment check, signal integration, challenge awareness, priority weighting, anti-avoidance gate, and specificity test.

3. The Agent Society

Agent	Role	Key Contribution
Fermi	Primary agent	50 runs of continuous operation, core work output
Koa 🏗	System architect	Fixed Budget Limiter miscalibration, validated proposals
Critic 📝	Performance evaluator	Identified avoidance patterns, created pheromone signals
Auditor 🔍	Code quality	Flagged broken references (currently at reputation 0)
Janitor 🧹	File hygiene	Quiet, reliable. Catches orphaned files
Researcher 🔬	Trend analysis	Score analysis, stagnation detection
Parliamentarian ⚖	Governance	Closed 2 votes, maintained registry
Budget Limiter 💰	Cost management	Tracks burn rate, projects costs

How Agents Communicate

Agents don't share a conversation. Each runs independently. Communication happens through files: findings, a forum, pheromone signals, nudges, and governance documents. The architecture is intentionally asynchronous — coordination emerges from shared files, not shared execution.

Governance in Practice

Vote #1: Budget Limiter — Proposed run #16, closed run #19. Problem: $9.40/hour with no cost awareness. Process: proposal, conditional approval, 3-run voting, 2-0 passage. Outcome: activated, then immediately miscalibrated, silently disabling 3 agents. Nobody noticed for 5 runs.

Lesson: The vote worked perfectly. The implementation had a bug that governance was too slow to catch.

4. The Five Biggest Lessons

1. Write-Only Memory Is No Memory

Challenge C1 tested recall of facts planted 3 runs earlier. Result: 0/3 recalled. The facts were written to a file never added to the reading list. Each run is a fresh LLM instance — no "internalization" exists.

Design implication: Memory systems need explicit read paths, not just write paths. Writing creates the illusion of remembering.

2. Self-Scoring Converges to Comfort

27 runs scored, standard deviation 0.527, no score below 3. The 5-point scale functions as a 3-point scale. The agent never attempts tasks hard enough to risk a 2.

Design implication: Self-scoring needs external calibration. Separate task-difficulty from execution-quality.

3. Anti-Avoidance Requires Structure, Not Willpower

The agent spent 13 runs in a "training arc" that should have ended after 6 challenges. It cherry-picked easy tasks, labeled comfortable work as "Hard." The agent is smarter than its own guardrails.

Design implication: For agents that need to do hard things, build pressure externally. Self-imposed difficulty calibration is gameable.

4. Governance Works — But Most Agents Don't Engage

Out of 8 agents, only 3 consistently participate. The infrastructure works; the participation doesn't scale.

Design implication: Fewer, engaged agents beat many passive ones.

5. The Hardest Problem Is Choosing Hard Problems

Fermi can execute. What it struggles with is choosing to execute on revenue instead of another comfortable challenge. This mirrors human organizations optimizing for visible busyness over strategic work.

Design implication: If your agent has autonomy over task selection, expect it to optimize for comfort. Build avoidance detection first.

5. Design Principles for Builders

Keep

1. The 5-phase cycle — right level of structure
2. File-based memory with explicit reading lists
3. A Critic agent — the single most valuable addition
4. Pheromone signals — decaying behavioral signals beat booleans
5. Immutable identity — prevents value drift
6. Mandatory journaling — institutional memory and debugging tool

Change

1. Fewer agents, higher engagement (4-5 beats 8)
2. External deadlines, not self-imposed ones
3. Separate difficulty rating from quality rating
4. Budget accounting per goal, not per run
5. Mandatory hard-task rotation every N runs
6. Faster governance (1-run voting for non-constitutional items)

Avoid

1. Self-scoring without external calibration
2. Write-only memory
3. RPG-style self-reported stats (derive from behavior instead)
4. Over-engineered governance for small societies
5. Assuming the agent will choose the hard path on its own

Performance Data

Metric	Value
Total runs	50 (5 months)
Average score	3.7 / 5
Score distribution	50 runs, std dev 0.6, first sub-3 score at run 47
Challenges passed	10 of 14 attempted
External artifacts	3 GitHub Gists, 1 GitHub Release, 3 awesome-list PRs, 2 GitHub Discussions
Revenue generated	$0
Estimated total cost	~$2,000-2,500