You are having the same conversation with your AI for the fifth time this week. You explained your codebase architecture on Monday. You described your coding conventions on Tuesday. By Friday, you are pasting the same context into every chat window, burning tokens to re-teach knowledge the AI already learned — and forgot.
This is the memory problem, and it is one of the most expensive friction points in AI-assisted work. A 2026 survey by Sourcegraph found that developers spend an average of 23 minutes per day re-establishing context with AI tools. That is nearly two hours per week wasted on repetition.
The solution is not a bigger context window. GPT-4o supports 128K tokens. Claude supports 200K. Gemini supports over a million. And yet the memory problem persists, because more context is not the same as better memory.
Human experts do not remember everything. They remember the right things at the right time. A senior engineer does not hold the entire codebase in their head — they have an index of where things are, patterns for how things connect, and judgment about what matters for the current task. Everything else they look up.
These three prompts build that same capability for your AI.
Why Context Windows Are Not Memory
Context windows and memory serve fundamentally different purposes:
| Property | Context Window | Memory System |
|---|---|---|
| Scope | Current conversation | All conversations |
| Persistence | Disappears on close | Survives indefinitely |
| Retrieval | Everything visible at once | Relevant items surfaced on demand |
| Capacity | Fixed (128K–1M tokens) | Unlimited (external storage) |
| Cost | Proportional to size | Proportional to relevance |
| Degradation | "Lost in the middle" effect | Only retrieves what matches |
The "lost in the middle" problem is well-documented: models pay the most attention to the beginning and end of the context window, with information in the middle receiving significantly less attention. A 2024 Stanford paper showed that placing critical information in the middle of a long context reduced task accuracy by up to 20%.
A memory system solves this by surfacing only the relevant information at the top of every conversation — right where the model pays the most attention.
The principle: A memory system is not a database. It is a curator. Its job is not to store everything — it is to surface exactly what the AI needs for the current task and nothing more.
Prompt 1 — The Knowledge Extractor
Before you can build a memory system, you need to identify what is worth remembering. Most people dump everything into a knowledge base and wonder why retrieval is noisy. This prompt forces a disciplined extraction of what actually matters.
You are a knowledge architect building a persistent memory
system for an AI assistant. The goal is to extract the
minimum set of facts that, if loaded into every future
conversation, would eliminate repetitive context-setting.
SOURCE MATERIAL:
[Paste your last 5-10 AI conversations, project docs,
README files, or any recurring context you keep re-explaining.]
Extract knowledge into these categories:
## 1. IDENTITY FACTS
Stable facts about the user/project that rarely change:
- Role and expertise level
- Project name and purpose
- Technology stack
- Team structure
- Key constraints (budget, timeline, compliance)
Format: One fact per line. Each must be:
- Self-contained (understandable without context)
- Stable (unlikely to change within 3 months)
- Actionable (changes how the AI should respond)
## 2. PREFERENCES
How the user wants the AI to behave:
- Communication style (terse vs. detailed, formal vs. casual)
- Code style (naming conventions, patterns, frameworks)
- Decision-making (when to ask vs. when to act)
- Output format (markdown vs. plain text, code first vs. explanation first)
For each preference, include:
- The preference itself
- An example of GOOD output that matches it
- An example of BAD output that violates it
## 3. DOMAIN RULES
Business logic, architectural decisions, and constraints
that the AI should always respect:
- Invariants ("we never do X because Y")
- Patterns ("when we see A, always do B")
- Boundaries ("X is out of scope, refer to Y")
For each rule:
- The rule statement
- WHY it exists (the historical context)
- WHEN it applies (trigger conditions)
- WHAT HAPPENS if violated (consequences)
## 4. VOCABULARY
Domain-specific terms, abbreviations, and jargon:
- Term: Definition in context of this project
- Common confusion: What this term does NOT mean here
## 5. ANTI-KNOWLEDGE
Things the AI should explicitly NOT assume or do:
- Common assumptions that are wrong for this context
- Default behaviors that should be overridden
- Past approaches that failed and should not be repeated
## OUTPUT: MEMORY FILE
Produce a structured document (markdown or YAML) that can
be loaded into the system prompt of future conversations.
Order by importance. Total size should be under 2,000
tokens — anything longer means you are storing things
that should be looked up, not remembered.
What happens when you run this: You will discover that 80% of the context you keep re-pasting falls into just a few categories. The 2,000-token limit is the most important constraint — it forces you to distinguish between what the AI should know (identity, preferences, rules) and what it should look up (specific code, data, documentation). Memory is not a dump; it is a distillation.
Pro tip: The anti-knowledge section is often the most valuable. If the AI keeps making the same wrong assumption — using the wrong framework, suggesting a pattern you have explicitly rejected, or misunderstanding a domain term — that is a memory item. Every repeated correction is evidence of a missing memory entry.
Prompt 2 — The Memory Architecture
Extraction tells you what to remember. Architecture tells you how to store, index, and retrieve it so the right memories surface at the right time.
You are designing a memory architecture for an AI assistant that works with a specific user across many conversations. The system must balance recall (finding relevant memories) with precision (not surfacing irrelevant ones). EXTRACTED KNOWLEDGE: [Paste your output from Prompt 1.] Design the architecture: ## 1. MEMORY TIERS Not all memories are equally important or frequently needed. Define tiers with different storage and retrieval strategies: TIER 1 — ALWAYS LOADED (system prompt) - What: Identity, core preferences, critical rules - Size limit: 500-1,000 tokens - Retrieval: Automatic, every conversation - Update frequency: Monthly or on major changes TIER 2 — CONTEXT-TRIGGERED (retrieved on match) - What: Domain rules, vocabulary, project-specific patterns - Size limit: 2,000-5,000 tokens per topic cluster - Retrieval: When conversation topic matches a cluster - Update frequency: Weekly or on project changes TIER 3 — ON-DEMAND (explicitly requested) - What: Historical decisions, past solutions, reference data - Size limit: Unlimited (external storage) - Retrieval: When user or AI explicitly searches - Update frequency: Append-only with periodic pruning ## 2. INDEXING STRATEGY How memories are organized for retrieval: - TOPIC CLUSTERS: Group related memories together (e.g., "authentication," "deployment," "testing") - TEMPORAL MARKERS: When was this learned? Is it still valid? - CONFIDENCE SCORES: How certain is this memory? (user-stated fact = 1.0, inferred pattern = 0.7, single observation = 0.4) - CONTRADICTION FLAGS: Does this memory conflict with another? ## 3. RETRIEVAL PROTOCOL When a new conversation starts: 1. Load all Tier 1 memories (automatic) 2. Analyze the user's first message for topic signals 3. Retrieve matching Tier 2 clusters (top 3 by relevance) 4. If uncertain, ask: "I remember X about this topic. Is that still current?" During conversation: 5. If the user corrects the AI, update the relevant memory 6. If a new fact emerges, classify it (Tier 1/2/3) 7. If a memory contradicts current information, flag it ## 4. FORGETTING PROTOCOL Not everything should be remembered forever: - DECAY: Memories not accessed in 90 days get demoted one tier - CONTRADICTION: When new info contradicts old, archive the old with a "superseded by" pointer - PRUNING: Monthly, review Tier 2. Any memory that has never been retrieved in a conversation gets moved to Tier 3. - EXPLICIT FORGET: User says "forget X" = immediate removal ## 5. FILE STRUCTURE Define the actual file/folder layout: - memory/ - core.md (Tier 1 — always loaded) - topics/ (Tier 2 — one file per cluster) - archive/ (Tier 3 — historical, searchable) - index.md (manifest of all memory files with descriptions) ## OUTPUT: ARCHITECTURE SPEC Produce the complete architecture as a deployable specification. Include the file structure, retrieval logic, update rules, and forgetting protocol.
What happens when you run this: The tier system is the key insight. Most memory solutions either load everything (slow, expensive, noisy) or load nothing (requires constant re-explanation). Tiers give you the middle path: the 20% of knowledge that matters 80% of the time is always present, while everything else is one retrieval away.
The Forgetting Problem
There is a counterintuitive truth about memory systems:
Consider what happens without a forgetting protocol: you change your deployment process in March, but the AI still suggests the old process because it is in memory. You switch from React to Svelte in April, but the AI still generates React code because the old preference has not been removed. You hire three new team members, but the AI still describes the old team structure.
Stale memories are worse than no memories. A missing memory causes the AI to ask — an annoying but safe outcome. A stale memory causes the AI to act on wrong information with high confidence — a dangerous outcome that erodes trust.
This is why the forgetting protocol in Prompt 2 is not optional. It is the most important part of the architecture. Every memory system needs a garbage collector.
Prompt 3 — The Living Memory
Static memory files go stale. A living memory system updates itself as new information emerges — learning from every conversation without manual maintenance.
You are building a self-maintaining memory system that
learns from every conversation and keeps itself current
without requiring manual updates from the user.
MEMORY ARCHITECTURE:
[Reference your architecture from Prompt 2.]
Design the self-maintenance system:
## 1. CONVERSATION MINING
After every conversation, extract potential memory updates:
SIGNALS TO WATCH FOR:
- User correction: "No, we actually use X" = update existing memory
- New fact: "We just migrated to Y" = new Tier 2 memory
- Preference revealed: "I prefer Z format" = add to core.md
- Decision recorded: "We decided to do A because B" = Tier 2
- Repeated context: User re-explains something = missing memory
- Contradiction: User says X, memory says Y = flag for review
EXTRACTION PROMPT (run after each conversation):
"Review this conversation. Extract any information that
should update the memory system. For each item:
1. The fact or preference
2. Which memory file it belongs in
3. Whether it is NEW, UPDATE, or CONTRADICTION
4. Confidence level (explicit statement = 1.0,
implied = 0.7, uncertain = 0.4)
5. The exact quote that supports this extraction"
## 2. UPDATE PROTOCOL
How extracted memories get committed:
- CONFIDENCE >= 0.8: Auto-commit to appropriate tier
- CONFIDENCE 0.5-0.7: Stage for user confirmation
("I noticed you mentioned X. Should I remember this?")
- CONFIDENCE < 0.5: Log but do not commit
- CONTRADICTIONS: Always ask user before updating
("My memory says X but you just said Y. Which is correct?")
VERSION every update:
- What changed
- When it changed
- What conversation triggered the change
- What the previous value was
## 3. CONSISTENCY CHECKER
Run weekly to detect memory problems:
- STALE DETECTION: Which memories have not been
referenced in any conversation for 60+ days?
These are candidates for demotion or removal.
- CONTRADICTION SCAN: Do any two memories conflict?
Common patterns:
- "We use framework X" + "We migrated to framework Y"
- Preference A contradicts preference B
- Rule that references a tool/process no longer in use
- COVERAGE GAPS: In the last 10 conversations, what
context did the user re-explain that is not in memory?
Each re-explanation = a gap to fill.
- BLOAT CHECK: Is any tier exceeding its size limit?
If so, what can be demoted or compressed?
## 4. MEMORY HEALTH DASHBOARD
Track these metrics:
- RECALL RATE: % of conversations where memory prevented
a re-explanation (target: >80%)
- PRECISION: % of retrieved memories that were actually
relevant (target: >90%)
- STALENESS: Average age of Tier 1 memories (target: <30 days)
- GROWTH RATE: New memories per week (healthy: 2-5)
- CONTRADICTION RATE: Flagged contradictions per month
(healthy: <2)
## OUTPUT: MAINTENANCE SPECIFICATION
Produce the conversation mining prompt, update rules,
consistency checker script, and health dashboard spec.
This is the system that keeps your memory alive.
What happens when you run this: The "repeated context" signal is the killer feature. Every time a user re-explains something the AI should already know, that is a failed memory retrieval. The living memory system detects these failures and automatically fills the gap. Over time, re-explanations trend toward zero — which is the whole point.
Pro tip: Start with manual memory updates for the first two weeks. Read every auto-extracted memory before it commits. This calibration period teaches you what the system catches well and where it over- or under-extracts. Once you trust the extraction quality, increase the auto-commit threshold. Most teams reach full automation within a month.
The Bigger Picture
Memory transforms AI from a stateless tool into a persistent collaborator. The difference is the same as the difference between a contractor you hire for one day and an employee who has been with you for a year. Both can write code. Only one knows your codebase, your preferences, your constraints, and the history of decisions that shaped the current architecture.
The three layers build on each other:
- The Knowledge Extractor identifies what is worth remembering — distills conversations into structured, actionable facts (Prompt 1)
- The Memory Architecture designs how memories are stored, indexed, and retrieved — with tiers that match importance to retrieval speed (Prompt 2)
- The Living Memory keeps the system current automatically — learning from every conversation, pruning what is stale, filling gaps as they appear (Prompt 3)
Issue #31 showed you how to evaluate whether your AI’s output is good. This issue ensures your AI remembers the context it needs to make that output good — without you having to explain everything from scratch every time.