$ cat ai-engineering/knowledge-graphs-memory.md

Knowledge Graphs as Structured Memory

How knowledge graphs represent entities, relations, temporal validity, and hybrid graph-vector retrieval.

Jatin Bansal@blog:~/ai-engineering$ open knowledge-graphs-memory

Vector retrieval can find separate episodes about a person, a forecast, and a model version without connecting them. A question such as “Which model did Priya’s team use for the March forecast?” requires entity resolution, graph traversal, and a temporal filter. Graph memory adds those operations while retaining vector search for fuzzy recall.

What graph memory stores

A memory graph stores typed entities and relations. Relations can carry validity intervals, provenance, and confidence. Entity resolution maps “Priya,” “my manager Priya,” and a matching CRM record to one node. Edges such as MANAGED_BY are directly traversable, and valid_from plus valid_to turns “what was true in March?” into a structured temporal query. Vector search remains useful for finding a starting node or a fuzzy text match; graph traversal handles exact relations and multi-hop questions.

This design combines fuzzy vector lookup with exact graph structure. It also differs from Microsoft’s GraphRAG, which extracts and summarizes a static corpus for global document queries. An agent memory graph is updated incrementally and serves entity-grounded queries over a growing history. Structured tags on flat facts do not provide the traversable edges that make the graph useful.

When graphs beat vectors (and when they don’t)

The 2026 empirical picture is sharper than it was a year ago. Three question classes where graph memory clearly outperforms vector-only memory:

Multi-hop relational queries. “What was the version of the forecast model used by the team Priya manages?” requires traversing user→manages→team→uses→model. Pure-vector retrieval has no way to compose three independent retrievals into a coherent answer; the best it can do is retrieve top-K episodes for the surface form and hope the model can stitch them. Graph traversal walks the edges directly and returns a single, grounded answer.

Temporal point-in-time queries. “What was the user’s role at company X as of March 2026?” is unanswerable from a vector store that retrieves on recency-weighted similarity alone; the most recent role will always rank higher, regardless of the asked-about date. A bi-temporal graph filters by valid_from ≤ date ≤ valid_to and returns the structurally correct answer. The Zep paper benchmark reports an 18.5% improvement over MemGPT on the Deep Memory Retrieval benchmark and an 18% accuracy lift on LongMemEval, with the largest gains concentrated in temporal-reasoning categories.

Entity-centric aggregation. “Summarize everything I know about Priya” is a query against the neighborhood of a single entity node. A graph returns the entity and its first-hop edges in one traversal. A vector store has to retrieve every episode mentioning “Priya,” paginate through them, and deduplicate; the cost grows linearly with the user’s history while the graph version stays constant in the size of the entity’s neighborhood.

Three question classes where vectors still win:

Open-ended semantic recall. “Have I ever mentioned anything about my food preferences?” is a fuzzy question whose target facts don’t fit any specific predefined relation type. The graph would need a MENTIONS_PREFERENCE edge and an exhaustive ontology of preference types; the vector store just retrieves on cosine similarity over “food preferences” and works.

Long-tail facts the extractor missed. Entity-and-relation extraction is itself an LLM call with non-zero error rate. Facts the extractor misclassified (or skipped entirely) are invisible in the graph but still searchable in the raw vector store. Pure-graph systems lose recall on exactly the long tail that pure-vector systems handle best.

Low-cardinality or short-lived agents. A graph carries an irreducible per-episode write cost: entity extraction, relation extraction, deduplication against the existing graph. For a single-session agent, or a workload where the user history never exceeds a few hundred turns, the graph’s payoff is smaller than its overhead. The Mem0 paper’s empirical comparison (Chhikara et al., 2025) shows the Mem0g variant adds roughly 2 percentage points of accuracy on LOCOMO over the pure-vector Mem0 (68.4% vs 66.9%) at 53% higher p50 latency (1.09s vs 0.71s) and meaningfully higher token cost; worth it for relational workloads, overkill for short-conversation chatbots.

Most graph-memory systems retain both graph and vector retrieval. Structural queries use graph traversal, fuzzy queries use semantic search, and mixed queries can fuse both result sets.

The bi-temporal model in detail

The single most underappreciated detail in graph memory is the bi-temporal part. Most engineers reach for graphs to encode entities and relations; they often skip the temporal half because their first benchmark doesn’t stress it. Six months later, the same agent is silently returning stale answers, and the team rediscovers what temporal-database designers have known for forty years.

Two clocks are required:

Valid time; when the fact was true in the world. “Priya managed user-42 from 2025-09-15 to 2026-04-01.” This is the clock that answers “what was true in March?”

Transaction time; when the system learned the fact. “We ingested the ‘Priya managed user-42 ending 2026-04-01’ update at 2026-04-08 10:33.” This is the clock that answers “what did the system believe on March 30?”

The two clocks separate because corrections happen out of order. The user might tell the agent on April 15 that Priya stopped being their manager on April 1; the world-time fact is “true until 2026-04-01,” but the system only knew that as of 2026-04-15. An audit query “what did the agent think on April 5?” must return the old belief (Priya still the manager), not the corrected one. Single-temporal systems can’t make that distinction. Graphiti and the broader Zep architecture ship with both clocks; most hand-rolled graph implementations don’t, and then have to bolt the second clock on later when the first audit-driven question gets asked.

When a fact changes, close the old edge’s valid interval and insert a replacement while retaining transaction history. Contradiction detection may use an LLM to recognize statements such as “X is now Y,” so extraction errors need review and provenance. Temporal reasoning and provenance covers as-of filters, staleness, and source walks.

Hybrid retrieval: graph + vector + keyword

The 2026 production pattern is consistent across every framework that has published numbers: a hybrid read path that runs three retrievals in parallel and fuses the results.

Graph traversal: Start from one or more entity nodes (extracted from the query), walk up to N hops, return the subgraph with edges valid at the query time.
Vector similarity: Embed the query, retrieve top-K episodes/facts by cosine similarity from the vector index (filtered by tenant).
Keyword/BM25: Sparse-retrieval pass over the same corpus to catch exact-term matches the embedding might miss; same idea as hybrid search in the RAG subtree.

The fusion step is some form of reciprocal-rank fusion (RRF) over the three result lists, optionally followed by a reranker. Zep’s published architecture follows this shape and reports a P95 retrieval latency of ~300ms with no LLM calls in the read path itself (the LLM work is concentrated at write time, in the entity-and-relation extraction). The trade-off is explicit and exactly what the context-engineering article framed as the JIT-vs-AOT question: pay LLM cost at write time so the read path can stay cheap, or pay LLM cost at read time and skip the heavyweight write pipeline. Graph memory makes the write-heavy choice.

The query-routing question; “which retrieval path do I trust for this question?”; has two defensible answers. The conservative answer is always run all three and fuse (Zep’s default); the aggressive answer is to classify the query first (with a small-model call) into “structural,” “fuzzy,” or “mixed,” and run only the relevant path. The conservative answer wastes some retrieval cost on every read; the aggressive answer adds a small-model latency tax to every read and can mis-route on ambiguous queries. Production systems lean conservative because the fusion cost is cheap relative to the model call that follows.

Code: Python; Graphiti for entity-and-temporal memory

The smallest interesting build: ingest episodic content into a temporal knowledge graph, then run a hybrid retrieval over it. Uses Graphiti (the open-source temporal-graph engine from Zep) against a Neo4j backend. Install: pip install graphiti-core neo4j and run docker run -d -p 7687:7687 -p 7474:7474 -e NEO4J_AUTH=neo4j/password neo4j:5. You also need OPENAI_API_KEY set for the entity-extraction LLM calls (Graphiti uses OpenAI by default and supports Anthropic via configuration).

python

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
import asyncio
from datetime import datetime, timezone
from graphiti_core import Graphiti
from graphiti_core.nodes import EpisodeType

graphiti = Graphiti(
    uri="bolt://localhost:7687",
    user="neo4j",
    password="password",
)

# --------- write path: ingest episodes ---------
async def ingest_episode(name: str, body: str, ref_time: datetime, group_id: str):
    """An episode is the unit of write. Graphiti extracts entities and relations
    from the body, deduplicates against existing nodes, and stamps every new edge
    with bi-temporal (valid_time, transaction_time) intervals."""
    await graphiti.add_episode(
        name=name,
        episode_body=body,
        source=EpisodeType.text,
        source_description="conversation turn",
        reference_time=ref_time,  # this is the valid-time anchor for extracted facts
        group_id=group_id,        # tenant scope; never skip this in multi-user systems
    )

# --------- read path: hybrid graph + vector retrieval ---------
async def recall(query: str, group_id: str, top_n: int = 5):
    """search() runs entity extraction on the query, walks the graph from matched
    entities, and fuses graph results with semantic similarity. Single call,
    no LLM in the read path itself."""
    results = await graphiti.search(
        query=query,
        group_ids=[group_id],
        num_results=top_n,
    )
    return [
        {
            "fact": r.fact,
            "valid_from": r.valid_at,
            "valid_to": r.invalid_at,  # None if still valid
            "score": r.score,
        }
        for r in results
    ]

async def main():
    await graphiti.build_indices_and_constraints()

    # September: Priya becomes the user's manager
    await ingest_episode(
        name="onboarding-2025-09-15",
        body="My new manager is Priya Sharma. She runs the forecasting team and "
             "uses Llama-3-70B for the Q3 model.",
        ref_time=datetime(2025, 9, 15, 10, 0, tzinfo=timezone.utc),
        group_id="user-42",
    )

    # March: still Priya, new model
    await ingest_episode(
        name="checkin-2026-03-01",
        body="Priya approved the Q1 forecast yesterday. The team switched to "
             "Claude Opus 4 for the new run.",
        ref_time=datetime(2026, 3, 1, 10, 0, tzinfo=timezone.utc),
        group_id="user-42",
    )

    # April: Priya leaves the role
    await ingest_episode(
        name="reorg-2026-04-15",
        body="Priya moved to head of platform. My new manager is Devansh. The "
             "forecasting team now reports to Devansh too.",
        ref_time=datetime(2026, 4, 15, 10, 0, tzinfo=timezone.utc),
        group_id="user-42",
    )

    # The query a vector-only store can't answer correctly:
    # "Who managed the forecasting team in March?"
    hits = await recall(
        "Who managed the forecasting team in March 2026?",
        group_id="user-42",
    )
    for h in hits:
        print(f"{h['fact']}  [valid {h['valid_from']} -> {h['valid_to']}]  score={h['score']:.2f}")

    await graphiti.close()

asyncio.run(main())

reference_time is the event’s valid-time anchor, so historical conversations should use their original timestamp rather than now(). group_id scopes every read and write to a tenant. The search() read path uses graph, vector, and keyword retrieval; entity and relation extraction occurred during ingestion.

When the read runs against the third (April) episode, Graphiti’s contradiction-detection pass stamps the old MANAGES edge from Priya with invalid_at = 2026-04-15, creates a new MANAGES edge from Devansh with valid_from = 2026-04-15, and preserves both. The “who managed in March?” query filters on valid_from ≤ March ≤ valid_to and returns Priya correctly; the kind of answer pure-vector retrieval cannot produce regardless of how many cosine neighbors it pulls.

Production frameworks: how each draws the line

Three frameworks worth knowing by their stance on the graph/vector boundary.

Graphiti / Zep is graph-first. The temporal knowledge graph is the source of truth; the vector index and BM25 index live inside the graph as auxiliary structures over node and edge content. Entity extraction and bi-temporal stamping happen on every write; the read path is pure traversal-plus-search, no LLM in the loop. Best fit: workloads where relational structure dominates (CRM, compliance, healthcare, support escalation) and where audit trails and temporal point-in-time queries are first-class requirements. Cost: heavyweight write pipeline (entity + relation extraction + deduplication + contradiction detection on every episode), proportional to a small-model call per write.

Mem0 with graph_store enabled (Mem0g) is graph-augmented. The vector store remains the primary substrate; the graph is a secondary index that gets populated on the same add() call and consulted on the same search() call. Easier to bolt onto an existing Mem0 deployment; the API surface doesn’t change. Best fit: teams that already have working vector-backed memory and want the multi-hop and entity-resolution lift without rewriting their write pipeline. Cost: marginally higher write latency, marginally higher accuracy on relational queries, no fundamental architectural change.

Microsoft GraphRAG / LazyGraphRAG is graph-as-corpus-index, not graph-as-memory. The system runs a one-shot extraction-and-community-detection pipeline over a static document corpus, then answers global (“summarize the main themes”) and local (“what does the corpus say about X?”) questions over the resulting graph. It is the closest published architecture to “GraphRAG done at scale” but it is not an agent memory framework; there’s no incremental update model, no bi-temporal validity, no per-user tenant boundary. Best fit: RAG over a large but stable corpus where global summarization queries matter (think: “synthesize what 10,000 documents say about climate policy”). Worst fit: agent memory, because the assumption of a stable corpus breaks the moment the agent ingests its first conversation turn.

The pragmatic choice for most teams in 2026 is one of two paths: start with Mem0g if you’re already on Mem0 or want graph-augmented vector memory with minimal architectural change; start with Graphiti if you’re greenfield and your workload’s defining feature is relational/temporal complexity. Both pay for themselves on the right workloads and are over-engineered on the wrong ones; the decision is downstream of how much of your memory’s value lives in entity graphs vs. open-ended episodes.