Reflection: From Experiences to Beliefs

A coding agent has been working with the same user for three months. The long-term episodic store holds 4,200 segmented episodes; each one a clean, salience-scored unit, every one of them admitted through a strict write-policy gate. Retrieval works. Recall is high. The agent can find the right episode for almost any query. And yet the agent feels flat. It remembers that the user reverted three pull requests last month, but it never noticed that all three reverts touched the auth subsystem and that the user has stopped trusting the auth refactor; it remembers eleven separate conversations about deploy retries, but it never abstracted “this user always wants to see logs before retrying”; it remembers each individual debugging session but doesn’t recognize the pattern in how this particular user debugs. The episodes are individually intact. The understanding is missing. The layer that turns raw observations into beliefs is the operation cognitive scientists call consolidation and the agent-memory literature, following Park et al. (2023), calls reflection.

Reflection as a materialized view

The cleanest distributed-systems parallel is the materialized view in a transactional database. In Postgres, a MATERIALIZED VIEW is a precomputed query result stored as a table; reads against the view are O(1) instead of O(table-scan); writes to the underlying tables don’t update the view automatically; a REFRESH MATERIALIZED VIEW has to be triggered. The trade-off is canonical: stale-but-fast reads for free, fresh reads cost a refresh. Production systems live on this trade-off; every analytics dashboard, every “top-K customers by revenue last month” query, every leaderboard, is the same shape.

Agent reflection recapitulates this exactly. The episodic store is the underlying table (every meaningful turn, append-only, expensive to aggregate). The reflection table is the materialized view (one row per derived belief, indexed for retrieval, fast to query). The reflection pass is the refresh operation; it reads from the episodic table, computes the aggregate, and writes to the view. Reads on the hot path query the view, not the underlying table; the cost of the higher-order answer is amortized over many reads. Three further parallels worth naming.

Refresh policy is the design surface, just like in databases. Postgres ships REFRESH MATERIALIZED VIEW CONCURRENTLY (locks the old version until the new one is built, then atomic swap) for live-traffic systems and REFRESH MATERIALIZED VIEW (blocks reads until done) for simpler workloads. The same trade-off shows up in reflection: reflect-on-trigger-with-concurrent-reads keeps the agent responsive while the reflection runs in the background; reflect-on-trigger-with-blocking-reads is simpler but means the next call after a trigger pays the reflection latency. Generative Agents implemented the latter (reflection runs inline at the importance-threshold breach); production systems lean toward the former.

View invalidation is the conflict-resolution problem. When a new episode contradicts a fact a reflection depends on, the view is stale. Postgres handles this manually; you have to REFRESH or set up triggers. Agent memory has to handle it more carefully because reflections aren’t typed enough to mechanically invalidate; the production answer in A-MEM and Memory-R1 is to keep a reflection’s evidence_ids as back-references to source episodes and run a background invalidation pass. A provenance walk and bi-temporal staleness gate can then walk dependents to a bounded depth and revalidate or mark them stale when a source becomes invalid.

Cascading materialized views; reflection on reflections. Generative Agents introduced a reflection tree: the leaves are episodes, the next layer is reflections over episodes, the next is reflections over reflections. The same shape exists in database systems (a materialized view over another materialized view), and the same caveat holds: the deeper the tree, the more compounded the staleness. The depth-2 reflection in Generative Agents is the canonical example; “I’m a researcher who values cleanliness” is a belief derived from beliefs (“I clean my workspace daily,” “I publish frequently,” “I attend conferences regularly”) that were themselves derived from raw episodes. Production reflection systems rarely go beyond depth-2 because each layer multiplies the invalidation surface.

Reflection loop

The mechanism Park et al. (2023) introduced is the canonical reference for every production reflection pass; knowing it by its components is what makes the implementations downstream legible.

Step 1; Trigger. Maintain a running sum of importance scores for every episode admitted since the last reflection. When the sum crosses a threshold (the paper uses 150 with their 1-10 scale, so roughly 15-20 high-importance episodes worth of accumulated signal), fire a reflection. The threshold is the consolidation analogue: not enough new signal, no reflection; enough new signal, refresh the view. Agents in the paper reflect about two-to-three times per simulated day under realistic interaction load.

Step 2; Salient question generation. Pull the 100 most recent records from the episodic store and prompt a large model: “Given only the information above, what are the 3 most salient high-level questions we can answer about the subjects in the statements?” This is the consolidation analogue of “what would be worth knowing about this period?”; the agent is generating its own retrieval queries against its own recent experience. The questions are not the output of reflection; they’re the index into the next step.

Step 3; Evidence retrieval per question. For each generated question, run a retrieval pass against the full episodic store (not just the recent window) and pull the top relevant episodes. This is the consolidation analogue of “what specific memories does this generalization rest on?”; the reflection process has to ground itself in concrete evidence, not in pure model intuition. The retrieval pass is the standard recency × importance × similarity rerank, scoped to the question.

Step 4; Insight generation with evidence references. Prompt the model: “What 5 high-level insights can you infer from the above statements? (example format: insight (because of 1, 5, 3)).” The output is a list of insights, each with explicit citation back to the source-episode IDs. This is the load-bearing structural pattern; every insight is anchored to its evidence, and the evidence IDs are stored alongside the insight. Without the citations, the insight is a free-floating claim; with them, it’s an audit-trail-bearing belief.

Step 5; Persist as a memory. The insights are written into the same episodic store, with a type=reflection tag, an evidence_ids field listing the source episodes, and a normal salience/importance score (often higher than typical raw episodes, since reflections are derived from many evidence pieces). On future retrieval, reflections are returned alongside raw episodes; the agent’s read path doesn’t have to know whether a returned memory is a raw observation or a reflection over many observations. The structural property: reflections sit in the same address space as the episodes they were derived from, which means recency × importance × similarity reranking works on both transparently.

The implementation is a five-step pipeline with explicit handoffs, not a single “summarize my recent memories” call.

Threshold-triggered reflection in Python

The smallest interesting build: an importance-sum-triggered reflector that runs the five-step Generative Agents loop against an existing Chroma-backed episodic store. Writes reflections back into the same store with the type=reflection tag and an evidence_ids field. Uses the Anthropic SDK for the question-generation and insight-generation calls. Install: pip install anthropic chromadb.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196

import json
import time
import uuid
from dataclasses import dataclass, field
from anthropic import Anthropic

import chromadb

client = Anthropic()
chroma = chromadb.PersistentClient(path="./memory_store")
episodes = chroma.get_or_create_collection("episodes")

# --------- tunables ---------
IMPORTANCE_THRESHOLD = 15.0     # sum of normalized [0,1] importance to trigger
RECENT_WINDOW = 100             # records prompted to the salient-question step
QUESTIONS_PER_REFLECTION = 3
INSIGHTS_PER_REFLECTION = 5
EVIDENCE_PER_QUESTION = 8
MODEL = "claude-haiku-4-5"      # cost-effective for the inner reflection loop


@dataclass
class ReflectionState:
    """Per-user accumulator. Persist this if the agent process is long-lived."""
    user: str
    pending_importance: float = 0.0
    last_reflection_ts: float = 0.0
    reflections_run: int = 0


def on_new_episode(state: ReflectionState, importance_normalized: float) -> bool:
    """Call after every successful episode write. Returns True if reflection should fire."""
    state.pending_importance += importance_normalized
    return state.pending_importance >= IMPORTANCE_THRESHOLD


# --------- Step 2: salient-question generation ---------
def generate_salient_questions(recent_records: list[dict]) -> list[str]:
    transcript = "\n".join(
        f"[{i}] {r['document']}" for i, r in enumerate(recent_records, start=1)
    )
    prompt = (
        f"Given only the information below, what are the {QUESTIONS_PER_REFLECTION} most "
        "salient high-level questions we can answer about the subjects in the statements? "
        "Return a JSON array of strings. Only the questions.\n\n"
        f"Statements:\n{transcript}"
    )
    resp = client.messages.create(
        model=MODEL,
        max_tokens=400,
        temperature=0,
        messages=[{"role": "user", "content": prompt}],
    )
    raw = resp.content[0].text.strip()
    try:
        return json.loads(raw[raw.index("[") : raw.rindex("]") + 1])
    except (ValueError, json.JSONDecodeError):
        return []


# --------- Step 3: evidence retrieval per question ---------
def evidence_for(user: str, question: str, k: int = EVIDENCE_PER_QUESTION) -> list[dict]:
    hits = episodes.query(
        query_texts=[question],
        n_results=k,
        where={"user": user},
    )
    if not hits["ids"][0]:
        return []
    return [
        {"id": eid, "document": doc, "metadata": meta}
        for eid, doc, meta in zip(
            hits["ids"][0], hits["documents"][0], hits["metadatas"][0]
        )
    ]


# --------- Step 4: insight generation with citations ---------
INSIGHT_PROMPT = """Below are statements from the agent's memory, each with a numeric ID.

Statements:
{statements}

What {n} high-level insights can you infer from the statements? Each insight must be
anchored to the specific statement IDs that support it.

Return JSON: a list of objects, each {{"insight": "<one sentence>", "evidence_ids": [int]}}.

Rules:
- The insight must say something the statements do not literally say; you are generalizing,
  not paraphrasing.
- Cite at least 2 evidence IDs per insight.
- If you cannot ground an insight in at least 2 statements, do not emit it.
"""


def generate_insights(evidence_records: list[dict], n: int = INSIGHTS_PER_REFLECTION) -> list[dict]:
    # Renumber statements for the model with a stable local index, but track real IDs.
    local_to_real = {}
    statements_block = []
    for local_id, rec in enumerate(evidence_records, start=1):
        local_to_real[local_id] = rec["id"]
        statements_block.append(f"[{local_id}] {rec['document']}")
    prompt = INSIGHT_PROMPT.format(statements="\n".join(statements_block), n=n)
    resp = client.messages.create(
        model=MODEL,
        max_tokens=900,
        temperature=0,
        messages=[{"role": "user", "content": prompt}],
    )
    raw = resp.content[0].text.strip()
    try:
        parsed = json.loads(raw[raw.index("[") : raw.rindex("]") + 1])
    except (ValueError, json.JSONDecodeError):
        return []
    # Remap local citation IDs to the real episode IDs.
    results = []
    for item in parsed:
        local_ids = item.get("evidence_ids", [])
        real_ids = [local_to_real[i] for i in local_ids if i in local_to_real]
        if len(real_ids) < 2:
            continue  # enforce the citation floor — drop unsupported insights
        results.append({"insight": item["insight"], "evidence_ids": real_ids})
    return results


# --------- Step 5: persist ---------
def persist_reflection(user: str, insight: str, evidence_ids: list[str]) -> str:
    rid = str(uuid.uuid4())
    episodes.add(
        ids=[rid],
        documents=[insight],
        metadatas=[
            {
                "user": user,
                "type": "reflection",
                "importance": 0.85,  # reflections inherit high importance by default
                "ts": time.time(),
                "last_read": time.time(),
                "evidence_ids": ",".join(evidence_ids),
                "depth": 1,
            }
        ],
    )
    return rid


# --------- The full reflection pass ---------
def reflect(state: ReflectionState) -> list[str]:
    """Run one full reflection cycle for a user. Returns the new reflection IDs."""
    # Pull the most recent records from this user's episodic store.
    all_records = episodes.get(where={"user": state.user})
    records = list(
        zip(all_records["ids"], all_records["documents"], all_records["metadatas"])
    )
    records.sort(key=lambda r: r[2].get("ts", 0), reverse=True)
    recent = [
        {"id": rid, "document": doc, "metadata": meta}
        for rid, doc, meta in records[:RECENT_WINDOW]
    ]
    if len(recent) < 5:
        return []  # not enough signal; reset the accumulator and skip

    questions = generate_salient_questions(recent)
    if not questions:
        return []

    new_reflection_ids: list[str] = []
    for question in questions:
        evidence = evidence_for(state.user, question)
        if len(evidence) < 2:
            continue  # no point reflecting without grounding
        insights = generate_insights(evidence)
        for ins in insights:
            rid = persist_reflection(state.user, ins["insight"], ins["evidence_ids"])
            new_reflection_ids.append(rid)

    # Reset the accumulator regardless — we paid the reflection cost, mark the cycle done.
    state.pending_importance = 0.0
    state.last_reflection_ts = time.time()
    state.reflections_run += 1
    return new_reflection_ids


# --------- minimal usage sketch ---------
if __name__ == "__main__":
    state = ReflectionState(user="alice")

    # Imagine an agent writing episodes through the normal path; importance arrives normalized.
    sample_importance_stream = [0.6, 0.9, 0.4, 0.85, 0.7, 0.95, 0.55, 0.8, 0.5, 0.9,
                                 0.65, 0.75, 0.85, 0.6, 0.9, 0.7, 0.8, 0.55, 0.95, 0.85]
    for imp in sample_importance_stream:
        # ... the normal write-path call goes here; we just track the accumulator ...
        if on_new_episode(state, imp):
            new_ids = reflect(state)
            print(f"Reflection cycle #{state.reflections_run}: wrote {len(new_ids)} insights")

the citation floor is structural, not stylistic; insights that cite fewer than two evidence IDs are dropped. This is the most-important quality-control knob in a reflection pipeline; without it, the model will emit free-floating claims (closer to a generic “the user seems thoughtful” than to a grounded “the user prioritizes API stability across these three PRs”). The renumber-then-remap pattern is what makes citations reliable; the model sees local 1-N integers that are easy to reference; the persistence layer remaps to real episode IDs that survive future store rewrites. The accumulator resets unconditionally after a cycle runs; even if zero insights are emitted, the cycle has consumed compute and the policy says “we tried.” Without the reset, a perpetual-fail state can spin the reflection pass in a tight loop and burn the budget; the unconditional reset is the circuit-breaker pattern for the reflection layer.

When reflection helps

Reflection is not always the right read-time substrate. The three regimes where it pays off, and the two where raw episodic recall wins.

Reflection wins when the query is about a pattern, not an instance. “What does this user usually want at deploy time?” is a pattern question; the right answer is a generalization over many past deploys, and surfacing one specific deploy episode misses the point. A reflection that says “this user wants logs and a manual approval gate before any deploy” is the right answer; the eight raw episodes that grounded it are noise on the read path. Pattern queries are the canonical reflection workload, and they’re also the queries where pure cosine recall over raw episodes underperforms most dramatically; the right answer is a synthesis, not a single hit.

Reflection wins when the working set of beliefs is small and the underlying episode set is large. A user who has 5,000 episodes in their long-term store probably has 30-50 stable beliefs the agent should know about them; preferences, recurring patterns, durable identity facts. The compression ratio is two orders of magnitude. Pre-paging the 30-50 beliefs into the system prompt is feasible; pre-paging the 5,000 episodes is not. Reflection is what makes the pinned-belief tier in hierarchical memory operationally tractable.

Reflection wins when the read path is latency-sensitive and the consolidation can run in the background. The cost of computing “what does this user usually want at deploy time?” from raw episodes is a multi-hop retrieval + summarization at read time; 1-3 seconds and a non-trivial token budget. The cost of reading a pre-computed reflection is one vector lookup; sub-50ms and a single short token block. For interactive agents where p95 latency matters, the only practical answer is to compute the higher-order answer once, materialize it, and serve it cheaply forever after.

Raw recall wins when the query is about a specific event. “When did the user mention they were vegetarian?” is an instance query; the right answer is a specific date and turn. A reflection that says “this user is vegetarian” is technically true but misses the actual question; the raw episode with the original turn and timestamp is the right answer. Mixing the two layers; surfacing a reflection when the user wanted the source episode; is a common production failure mode, and the fix is to type the read path: instance queries route to raw episodes, pattern queries route to reflections (with raw episodes as fallback or evidence).

Raw recall wins when the underlying episodes are still in flux. A user the agent has been working with for one week has too few episodes to consolidate from; the reflections that fire will be over-confident generalizations from too little evidence. The cognitive analogy is the same; systems consolidation is a slow process precisely because the brain doesn’t generalize from a single experience. Set a floor: the user has to have at least N high-salience episodes (50 is a defensible starting point) before reflection fires at all, or the early reflections will pollute the belief tier with shaky claims that have to be invalidated later.