$ cat ai-engineering/memory-conflict-and-forgetting.md

Memory Conflict, Forgetting, and Embedding Drift

Agent-memory conflict resolution, active forgetting, and embedding-model migration.

Jatin Bansal@blog:~/ai-engineering$ open memory-conflict-and-forgetting

Long-lived memory stores fail in several distinct ways. Facts can conflict, unused material can crowd retrieval, and an embedding-model upgrade can make old vectors unreachable by new queries. These failures need separate policies even though they operate on the same records.

Three maintenance problems

Memory conflict resolution is the discipline of detecting when two facts in the store disagree and choosing how to surface the resolution to the read path. Three operational modes. Detect on write; when a new fact lands, scan the store for facts that overlap its category, scope, and validity window; if any contradict, fire the resolver. Detect on consolidation; a background pass over the store finds pairs of facts whose embeddings cluster above a similarity threshold and whose claims disagree on inspection; resolve and write the merged outcome. Detect on read; when retrieval surfaces two candidates that contradict, the rerank either picks one with a recorded resolution rule, returns both with explicit divergence flags, or asks the user. Production systems combine all three: write-time catches the obvious cases cheaply, consolidation catches what write-time missed, and the read-time check is the safety net for the pairs neither earlier pass resolved.

Forgetting is the discipline of actively removing or down-weighting memories that the system no longer benefits from carrying. Forgetting is not the same as unlearning. Forgetting is a soft-state property; the agent doesn’t retrieve the memory because the policy has decayed it below a usefulness threshold, but the underlying data is still in the store and could in principle be recovered. Unlearning is a hard property; the underlying data is gone, in a way that satisfies a verifiable deletion guarantee (the GDPR Article 17 case). Most production discussion of “forgetting” is the soft kind; the hard kind needs its own deletion pipeline, audit trail, and verification step, and getting it right is a separate engineering exercise from designing a decay policy.

Embedding drift is the cluster of failure modes that happens when the embedding model underneath a vector store changes versions. Five sub-failures, each with its own mitigation. Coordinate-system mismatch; a v1 vector and a v2 vector are not comparable; even for the same text, their cosines drift unpredictably. Recall collapse; the most acute symptom: a v2 query against a v1 index returns near-random nearest neighbors. Hybrid skew; partial migration leaves the index with mixed v1 and v2 vectors, and reranking sees a coordinate-system mixture rather than a semantic ordering. Cost blast; full re-embedding of a large store is expensive in both compute and operational risk. Index rebuild downtime; re-embedding invalidates the ANN index, which has to be rebuilt against the new vectors and brought online without dropping production traffic. The mitigation menu; dual-index migration, alias swaps, Drift-Adapter affine maps; exists because no single approach wins on all four axes.

Resolve contradictions at write time

The cleanest formalization of write-time conflict resolution is the Mem0 ADD/UPDATE/DELETE/NOOP classifier. For every candidate write, an LLM-driven resolver compares the candidate against the closest existing facts and decides:

ADD; no semantically equivalent fact exists. Write the candidate as a new entry. The default case.
UPDATE; an existing fact is closely related and the new fact augments it (an additional detail, a refined version, a constraint added). Merge into the existing entry; do not write a new one.
DELETE; an existing fact is contradicted by the new one. Mark the old as INVALID (do not physically remove), and write the new as a separate entry with a supersedes pointer back to the old.
NOOP; the new fact is already represented in the store with no meaningful change. Discard.

The classifier is a single LLM call with the candidate plus the top-K (K ≈ 5-10) closest existing facts in the prompt; the model returns the operation and the target fact ID. Cost is bounded; one resolver call per write, against a small candidate set. The Mem0 paper reports the resolver running on every memory update and being the primary mechanism by which the store stays internally consistent.

Three failure modes worth handling explicitly.

The false-negative cascade. The resolver decides ADD when it should have decided DELETE; a real contradiction is missed because the embedding similarity between the old and new facts was below the candidate-retrieval threshold. The contradiction sits in the store until something surfaces both facts in the same retrieval and the rerank trips. The mitigation is the sleep-time consolidation pass; a periodic background job that runs the resolver across every fact pair within a category, not just the top-K-similar ones, and catches the contradictions that the per-write retrieval missed. Mem0 documents this as a consolidation job; it reports the consolidator catching 5-10% additional contradictions per pass on a steady-state store.

The high-density contradiction cliff. Most workloads have contradiction density (fraction of new writes that supersede an existing fact) in the single-digit-percent range, and the resolver handles them comfortably. Some workloads; onboarding flows where the user is rapidly updating their profile, debugging conversations where the model is iterating on a hypothesis; spike into the 20-30% range, and the resolver’s load doubles and its accuracy drops (more candidates means more chances of confusion in the classification prompt). Mem0’s benchmark numbers show the resolver staying above threshold up to 30% contradiction density with timestamp-aware features turned on; beyond that, the policy needs to either widen the candidate window (more LLM cost) or accept some contradictions in the store and rely on the consolidator. The pattern is what queueing theory calls “the saturation knee”; the operating curve is fine up to a load and then degrades sharply.

The user-vs-system arbitration problem. A new user assertion contradicts an existing system-derived belief. The user is usually right; they’re the source of truth on their own preferences, history, and intent. But sometimes they mis-remember or mis-state, and the year-old belief (with corroborating provenance from multiple sources) is the more trustworthy one. The arbitration policy is a classifier on top of the resolver: for high-stakes categories (medical, financial, legal), confirm explicitly before overriding; for low-stakes categories (food preferences, casual mentions), update silently. The friction trade-off is workload-specific; production agents that don’t make the choice explicitly default to “always trust the user” and accumulate quiet errors when the user is wrong.

Decay and eviction

The Ebbinghaus forgetting curve; a memory’s retention decays exponentially with time, slowed by each repetition; is the standard frame. The agent-memory port carries three knobs the cognitive-psychology version doesn’t have: per-category half-lives, an importance multiplier, and a verification boost.

Per-category half-lives. The decay rate is not global. A name fact (half_life = ∞) shouldn’t decay at all. A food-preference fact decays over a multi-year horizon. An event fact (half_life ≈ 30 days) decays fast. Setting the half-lives by introspection; “events feel like they decay over a month”; is the default and is wrong as often as right. The defensible approach is to measure; fit half-lives against the empirical distribution of valid_to - valid_from intervals in your store, conditional on category. Most projects skip the measurement step and ship the introspected values; the failure mode is silent until an audit catches it.

Importance multiplier. Two facts in the same category can have very different long-run value. “The user’s job title is Staff Engineer” decays at the job-title half-life, but a high-importance variant (“the user’s promotion to staff was contingent on the Q2 launch”) deserves to decay slower. The standard form is strength(t) = importance · exp(-Δt / half_life), with importance ∈ [0, 1] set by the salience scorer at write time.

Verification boost. A fact that’s been re-confirmed by the user (the last_verified timestamp got updated) gets a strength boost; strength *= 1 + δ · indicator(recently_verified). The signal captures the behavioral component of memory: a fact the user re-mentioned last week is current regardless of when it was first ingested. The mechanic shows up in the MemoryBank read-driven strength updates, and it’s the same shape as the verification term in yesterday’s temporal-reasoning piece.

The combined eviction score:

text

1
score(fact) = importance · exp(-(now - last_access) / half_life[category]) · (1 + δ · recently_verified)

Facts whose score falls below a threshold (or whose total count exceeds a budget) get evicted. Eviction is not deletion in the unlearning sense; the row is moved to a cold tier, marked archived, and excluded from the default retrieval path. A subsequent explicit query can still find it. The pattern is exactly the hierarchical-memory demotion from working → episodic → cold; “forgetting” is the demotion at the storage layer.

FadeMem ships this with a dual-layer hierarchy and reports the 45% storage reduction at comparable benchmark accuracy. The biological-fidelity argument is interesting but not load-bearing; the engineering argument is that the score is principled (every term has a measurable workload analogue), the policy is cheap (one score per fact per pass), and the failure modes are localized (a wrongly-evicted fact is recoverable from the cold tier, unlike a wrongly-deleted fact).

The hard form of forgetting; machine unlearning; is a separate engineering exercise. GDPR Article 17 erasure (“delete everything the agent knows about user X”) requires the data to be provably gone from every derived artifact: the vector store row, the reflection that cited it, the embedding cache, the prompt-caching prefix, the trace logs. The audit boundary is much wider than the soft-forgetting boundary. The 2025 machine unlearning literature covers the technique mix; retraining, targeted parameter editing, output filtering; but most production deployments handle the vector-store side with a hard delete plus a reflection-walk that prunes dependent claims, accepting some residual influence in the model weights themselves. The right discipline is to treat unlearning as a deletion pipeline, not a memory-policy tuning exercise, and to instrument it with a verification step that confirms each derived artifact actually got pruned.

Migrating embedding models

When the embedding model under the store changes; text-embedding-3-small → text-embedding-4-small, voyage-2 → voyage-3-large, an open-source model swap; the vectors already in the store are not directly usable by the new model’s queries. Four migration patterns, in increasing order of cost-vs-recall trade-off.

The naive re-embedding. Re-encode every fact in the store with the new model. Rebuild the ANN index. Switch reads. Cost: linear in store size, O(N) model calls. Operational risk: high; the index rebuild needs to happen offline, and during the rebuild window the read path is either down or serving stale results. For a million-fact store at ~$0.00002 per embedding, the re-encoding bill is $20, but the index rebuild and the validation pass dominate operationally. Defensible for small stores; impractical at scale.

Dual-index migration. Run v1 and v2 indexes in parallel. Encode every new write with both models, store both vectors. Backfill the v2 index from the existing v1 corpus in the background. Validate by shadow-querying both indexes and comparing recall against a held-out query set. When v2 recall ≥ v1 recall, swap the read alias from v1 to v2. Decommission v1 after a grace window. Cost: doubled write traffic and doubled storage during the migration window; one round of re-encoding. Operational risk: low; every step is rollback-safe, the read path never breaks, and the validation is principled. The pattern is standard in production vector deployments and is documented across Pinecone, Weaviate, and Qdrant operational guides. The cost is high but bounded; the safety is essentially complete.

Alias-based versioning. A refinement on the dual-index pattern: every index is named with the model version and a date (memories_v1_20260201, memories_v2_20260415), and the application references a stable alias (memories_current) that points to whichever version is live. The alias swap is atomic; the rollback is an alias-flip. The cost is the same as dual-index; the operational ergonomics are dramatically better because every deployment that depends on the index references an alias rather than a version, and the alias is the single point of cutover.

Drift-Adapter affine maps. The 2025 result that changes the cost calculus. Instead of re-embedding every vector, train a small transformation T: ℝᵈ¹ → ℝᵈ² on a sample of paired old/new embeddings; encode a few thousand documents with both models, fit an orthogonal Procrustes map (or a low-rank affine, or a small residual MLP) from v1-space to v2-space. At query time, the new model encodes the query into v2-space, the adapter projects the query into v1-space, and retrieval runs against the existing v1 index. The Drift-Adapter paper (Vejendla, EMNLP 2025) reports the affine map recovering 95-99% of the recall a full re-embedding would have produced, on MTEB text corpora and a 1M-item CLIP image upgrade, with <10µs of added query latency and 100× less recompute than the dual-index path. The trade-off is the 1-5% recall gap; Drift-Adapter is a pragmatic migration that defers the re-embedding rather than eliminating it, and the residual gap matters for workloads where the top-K is the entire signal. For most application-side retrieval, the gap is well below other sources of noise.

The choice between dual-index and Drift-Adapter is a workload question, not an architecture question. If recall@10 is the dominant metric and the 1-5% gap is visible in evals, run the full dual-index. If query latency and migration cost dominate, the adapter is the better path. Production deployments that have done both report using the adapter as a bridge; deploy it on the day of the model upgrade to keep recall flat, then run the dual-index migration in the background over the following weeks, and decommission the adapter when the v2 index is fully populated and validated.

Memory Conflict, Forgetting, and Embedding Drift

Three maintenance problems

Resolve contradictions at write time

Decay and eviction

Migrating embedding models

Further reading