$ cat ai-engineering/memory-retrieval-policies.md

Memory Retrieval Policies: Recency, Relevance, Importance

How recency, importance, and similarity combine to rank memories for the current query.

Jatin Bansal@blog:~/ai-engineering$ open memory-retrieval-policies

A memory store can contain the right episode and still rank it below stale or keyword-heavy records. Retrieval policy corrects that ordering by combining semantic similarity with time, importance, explicit validity, and a bounded context budget.

The retrieval-policy contract

A retrieval policy recalls a coarse candidate set, computes comparable signal values, reranks the candidates, trims them to a prompt budget, and updates read-driven state. Vector similarity supplies recall; recency and importance change the order after recall.

Signals need normalization because cosine, decay, and importance can have different ranges. Recall size, rerank cost, and the final prompt contribution also need explicit bounds so retrieval does not grow with the store.

Chunking chooses storage units, the write policy controls admission, and hierarchical paging chooses a tier. Retrieval policy operates after those decisions on the candidates available for the current read.

Combine recency, importance, and similarity

The formulation from Park et al. (2023), §A.1, is the canonical reference, and knowing each piece in detail is what makes the implementations downstream legible.

Recency. Exponential decay over the time since the episode was last retrieved (not just last written). The paper uses a decay factor of 0.995 per sandbox hour, which sounds modest until you compound it: after 24 sandbox hours an episode’s recency term is 0.995^24 ≈ 0.886; after a week (168 hours) it is 0.995^168 ≈ 0.43; after a month it is 0.995^720 ≈ 0.027. The choice of decay constant is the load-bearing knob. Too aggressive (0.99 per hour) and any episode older than a few days drops out of the top-K regardless of importance; too gentle (0.999 per hour) and recency becomes effectively constant and the policy collapses to importance-plus-similarity. A defensible default for real-world wall-clock time is a half-life in the 7-to-30-day range, set explicitly; recency = exp(-Δt / τ) with τ chosen so that recency(τ) = 0.5.

Importance. The anchored 1-10 salience score, normalized to [0, 1] by dividing by 10. The dependency on the salience scorer producing a distributed score (and not the all-7s collapse) is what makes this term informative; the salience-collapse failure mode named in the segmentation piece is also the most common way the retrieval-side rerank silently regresses to similarity-plus-recency.

Similarity. Cosine similarity between the query embedding and the episode embedding, no normalization needed since cosine is already in [-1, 1] and effectively in [0, 1] for reasonable embedding models. The choice of embedding model matters here in the same way it matters everywhere; the text-embeddings article named the trade-offs; the relevant gotcha for retrieval policies specifically is that some embedding models have very tight cosine distributions (everything between 0.7 and 0.9 for any reasonable query/document pair), which silently squashes the similarity term’s dynamic range. Mitigation: rescale the observed cosine band to [0, 1] by min-max normalization over the recalled candidates, not over the global range.

Combination. Linear weighted sum: α·recency + β·importance + γ·similarity. The paper’s default is equal weights and the empirical result is that equal weights outperform any single-signal baseline by a wide margin. Workload-specific tuning matters at the margins; a customer-support agent where importance is the strongest signal benefits from β > α, γ; a coding agent where similarity-to-current-code matters most benefits from γ > α, β; a personal-assistant agent in long conversation where the most-recent preferences supersede older ones benefits from α > β, γ. But the variance from tuning is dwarfed by the variance between equal-weights and any single-signal baseline; get to equal-weights first, tune second.

Top-K selection. After computing the combined score for every candidate from the coarse recall (typically K_recall = 20-50), sort and take the top N (typically N_final = 3-10). The 4-to-1 oversampling ratio between recall and final is the standard pattern from the reranking article; recall has to be wide enough that the right episode is in the candidate set, even if it’s not the highest cosine hit; the rerank uses the additional signals to find it.

Update state after a read

Every successful retrieval should write back to the store. Three updates worth doing on every retrieval:

Update last_read_at. Each top-N episode gets its last_read_at advanced to the current time, which refreshes its recency term for the next retrieval. This is the LRU update, and it’s what keeps useful episodes from decaying out of the top-K just because they were written long ago. An episode written six months ago but read fifty times in the last week should have a recency term of nearly 1, not nearly 0; the last_read_at update is what makes that the case.

Increment retrieval_count. A monotonic counter on each episode, incremented on every successful retrieval. Optionally folded into the importance term as importance' = importance + δ · log(1 + retrieval_count) (logarithmic to avoid letting a few highly-retrieved episodes dominate the entire policy). This is the LFU contribution; frequency-of-use is a meaningful signal that the static write-time importance doesn’t capture.

Conditional importance boost. When an episode is retrieved and the agent’s subsequent answer cites it (or simply succeeds against a downstream evaluation), boost the importance. When an episode is retrieved and the answer goes badly, optionally demote. This is the read-driven salience update MemoryBank introduced; the Ebbinghaus-curve-inspired strengthening of memories that prove useful. It is the closest agent analogue to the brain’s reactivation-strengthens-memory consolidation pattern, and the closest cache analogue to the priority boost in priority-aware cache replacement.

The discipline that separates a useful read-driven update from a counterproductive one is bounded magnitude. A 5% boost per retrieval keeps the policy responsive to actual usage; a 50% boost per retrieval makes the importance term swing wildly on every read and the rerank becomes unstable. The Generative Agents paper uses a 0.995 per-hour decay on the recency term and effectively a constant 1.0 per-retrieval refresh on last_read_at; MemoryBank uses an Ebbinghaus-shaped S = S_0 · exp(-t/η) where η grows with retrieval count, so each retrieval extends the half-life rather than ratcheting up a separate score. Both are defensible; the wrong answer is to compound multiple unbounded boosts and watch the importance term diverge.

A normalize-and-blend implementation

The smallest interesting build: a retrieval policy that runs cosine recall, normalizes the three signals, blends them with tunable weights, updates the read-driven state, and returns the top-N. Uses Chroma as the substrate. Install: pip install chromadb numpy.

python

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
# pip install chromadb numpy
import math
import time
import uuid
from dataclasses import dataclass

import chromadb
import numpy as np

chroma = chromadb.PersistentClient(path="./memory_store")
episodes = chroma.get_or_create_collection("episodes")


@dataclass
class RetrievalPolicy:
    # Half-life of the recency term in seconds (default ~14 days).
    recency_half_life_s: float = 14 * 86_400
    # Linear weights on the three signals. Equal-weights is the Park et al. default.
    alpha_recency: float = 1.0
    beta_importance: float = 1.0
    gamma_similarity: float = 1.0
    # Logarithmic retrieval-count boost on importance. 0 disables LFU.
    delta_retrieval_count: float = 0.05
    # Recall/final fan-out.
    recall_k: int = 20
    final_n: int = 5
    # Minimum gap between top-K refreshes of `last_read_at`. Without this every
    # retrieval flattens the recency term across the entire candidate set.
    last_read_refresh_floor_s: float = 60.0


def _decay(age_s: float, half_life_s: float) -> float:
    """Exponential decay normalized so that decay(half_life) == 0.5."""
    return math.pow(0.5, age_s / half_life_s)


def _minmax(xs: list[float]) -> list[float]:
    """Min-max normalize a list of scores to [0, 1]. Robust to constant vectors."""
    if not xs:
        return []
    lo, hi = min(xs), max(xs)
    if hi - lo < 1e-9:
        return [0.5 for _ in xs]
    return [(x - lo) / (hi - lo) for x in xs]


def retrieve(
    user: str,
    query: str,
    policy: RetrievalPolicy | None = None,
) -> list[dict]:
    """Two-stage retrieval: cosine recall, then multi-signal rerank with read-driven update."""
    policy = policy or RetrievalPolicy()
    now = time.time()

    # ----- 1. Coarse recall (cosine similarity, tenant-scoped) -----
    hits = episodes.query(
        query_texts=[query],
        n_results=policy.recall_k,
        where={"user": user},
    )
    if not hits["ids"][0]:
        return []

    ids = hits["ids"][0]
    docs = hits["documents"][0]
    metas = hits["metadatas"][0]
    distances = hits["distances"][0]

    # ----- 2. Compute each signal -----
    # Recency: exponential decay since last read.
    age_seconds = [now - (m.get("last_read", m.get("ts", now))) for m in metas]
    recency_raw = [
        _decay(age, policy.recency_half_life_s) for age in age_seconds
    ]

    # Importance: normalized salience (already in [0, 1] from write path).
    # LFU boost via log(1 + retrieval_count).
    importance_raw = [
        m.get("importance", 0.5)
        + policy.delta_retrieval_count
        * math.log1p(m.get("retrieval_count", 0))
        for m in metas
    ]

    # Similarity: convert Chroma L2 distance to a [0, 1] proxy, then min-max
    # across the candidate set so a tight cosine band doesn't squash the term.
    similarity_raw = [1.0 / (1.0 + d) for d in distances]

    # ----- 3. Per-candidate normalization (critical for blending) -----
    recency = _minmax(recency_raw)
    importance = _minmax(importance_raw)
    similarity = _minmax(similarity_raw)

    # ----- 4. Linear blend -----
    scored: list[tuple[float, str, str, dict]] = []
    for i, (eid, doc, meta) in enumerate(zip(ids, docs, metas)):
        score = (
            policy.alpha_recency * recency[i]
            + policy.beta_importance * importance[i]
            + policy.gamma_similarity * similarity[i]
        )
        scored.append((score, eid, doc, meta))
    scored.sort(reverse=True, key=lambda t: t[0])

    top = scored[: policy.final_n]

    # ----- 5. Read-driven state update (LRU/LFU/Ebbinghaus contributions) -----
    updates_ids: list[str] = []
    updates_metas: list[dict] = []
    for _, eid, _, meta in top:
        last_read = meta.get("last_read", 0.0)
        # Refresh-floor: don't bump last_read more often than the floor allows.
        # Prevents a hot query loop from flattening the recency curve.
        if (now - last_read) < policy.last_read_refresh_floor_s:
            continue
        meta = dict(meta)
        meta["last_read"] = now
        meta["retrieval_count"] = int(meta.get("retrieval_count", 0)) + 1
        updates_ids.append(eid)
        updates_metas.append(meta)
    if updates_ids:
        episodes.update(ids=updates_ids, metadatas=updates_metas)

    return [
        {
            "id": eid,
            "text": doc,
            "score": score,
            "meta": meta,
        }
        for score, eid, doc, meta in top
    ]


# ----- write-side helper (just for the demo) -----
def write_episode(
    user: str, text: str, importance: float = 0.5, type_: str = "event"
) -> str:
    eid = str(uuid.uuid4())
    now = time.time()
    episodes.add(
        ids=[eid],
        documents=[text],
        metadatas=[
            {
                "user": user,
                "type": type_,
                "importance": importance,
                "ts": now,
                "last_read": now,
                "retrieval_count": 0,
            }
        ],
    )
    return eid


if __name__ == "__main__":
    # Seed three episodes with different importance + age profiles.
    write_episode(
        "alice", "User strongly prefers Postgres for any relational workload.", 0.9
    )
    # Older but mundane:
    eid_old = write_episode(
        "alice", "User asked what time it was in UTC.", 0.2
    )
    # Force the "old" episode to look stale:
    episodes.update(
        ids=[eid_old],
        metadatas=[
            {
                "user": "alice",
                "type": "event",
                "importance": 0.2,
                "ts": time.time() - 30 * 86_400,
                "last_read": time.time() - 30 * 86_400,
                "retrieval_count": 0,
            }
        ],
    )
    write_episode(
        "alice", "User mentioned they are deploying to Cloudflare Pages.", 0.7
    )

    # Equal-weights retrieve against a Postgres-flavored query.
    results = retrieve("alice", "What database does the user like?")
    for r in results:
        print(f"score={r['score']:.3f}  imp={r['meta']['importance']:.2f}  text={r['text']}")

Per-candidate min-max normalization makes the three signals comparable. Without it, a tight cosine band such as [0.6, 0.8] has much less range than recency and contributes little to the result. last_read_refresh_floor_s prevents a loop from refreshing every top-N episode on every second. delta_retrieval_count supplies a small LFU contribution; large values lock in old popular episodes even with logarithmic scaling. The policy uses a dataclass so weights and half-life can be calibrated per workload instead of being buried in the retrieval function.

Handle temporal queries and stale facts

Two non-obvious refinements show up in production retrieval policies and are worth knowing by name.

The anti-recency boost. Sometimes you want to surface old content; for example, when answering “what did we decide three months ago about X?” the answer is by construction old, and the recency term will push it down. The fix is to recognize the temporal intent of the query and invert the recency term for those queries; multiply by (1 - recency) rather than by recency; or, more often, drop the recency term entirely (set α = 0) when the query carries explicit temporal markers (“yesterday,” “last week,” “three months ago,” “originally”). Mem0 and Letta both expose temporal-intent flags on retrieval for this reason.

The staleness gate. Some content has explicit validity intervals; a knowledge-graph edge with valid_until set, a reflection whose source episodes have been superseded, a fact tagged with a confidence that has been demoted by a sleep-time consolidator. The retrieval policy should filter out (not just down-weight) any candidate that fails an explicit staleness check before scoring. A down-weighted stale candidate can still surface above a fresh one if the down-weight is small relative to the other signals; a hard filter is what enforces the staleness boundary.

The diversity penalty. A top-K dominated by five near-duplicates of the same episode wastes 4/5ths of the prompt budget on redundant content. The mitigation is a maximum-marginal-relevance (MMR) pass after the score-and-rank; for each candidate, deduct a fraction of its score for each higher-ranked candidate it’s textually similar to, so near-duplicates are spread across the top-K rather than concentrated at the top. The same MMR trick the reranking article named for static-corpus search applies here without modification.