RAG and agent memory often run on the exact same vector database, the same index, the same nearest-neighbor query. People argue about which to use as if they were rival technologies. They are not even competing at the layer everyone points to. The only thing that separates them is one direction of data flow: whether the agent can ever write back what it just learned. RAG reads from a corpus someone else wrote. Agent memory writes down what the agent itself learned, and that write path is what this post walks step by step.
The slogan RAG is stateless and memory is stateful describes a symptom, not a mechanism. Both systems sit on the same vector store and run the same nearest-neighbor search at read time. What actually separates them is whether anything ever flows backward into that store, and what happens when a stored fact stops being true. The write path below has three movements: how a fact gets extracted, stored, deduplicated, and reconciled when it changes.
The one-line difference: RAG reads, memory writes
RAG reads; memory writes. RAG retrieves passages from an authoritative corpus and hands them to the model as context. Memory captures facts from the interaction itself and saves them so a later session can use them. RAG treats its corpus as ground truth the agent reads. Memory treats its store as a notebook the agent keeps writing to.
The original RAG paper from Patrick Lewis and colleagues in 2020 framed it as combining parametric memory, the knowledge baked into model weights, with non-parametric memory, a dense vector index of Wikipedia accessed by a neural retriever. The vector index there is static. You build it once over a fixed document set and query it. Nothing the model produces gets written back into Wikipedia.
Memory inverts that flow. The store starts empty. It fills up with whatever the agent extracts from conversations and actions: a user's stated preference, a deadline, a correction to something said earlier. The corpus is not authoritative content from outside; it is the agent's own accumulated record. For a deeper grounding on how that record is built and recalled across sessions, see our explainer on how AI long-term memory works.
Do RAG and agent memory use the same vector database?
Both RAG and memory embed text into vectors and run approximate nearest-neighbor search. The retrieval primitive is identical. If you opened the database behind a RAG pipeline and the database behind a memory layer, you might see the same engine, the same index type, the same top-k query. The divergence is lifecycle, not storage.
- RAG lifecycle: ingest a fixed corpus once (or on a scheduled rebuild), chunk it, embed it, index it. At query time, retrieve and read. The store is read-only with respect to the agent.
- Memory lifecycle: start empty, then write continuously during use. Every interaction can add, change, or delete entries. The store is read-write, and the agent is the author.
This is why the embedding mechanics carry over but the operational shape does not. If you want the storage primitive itself demystified, our guide to vector embeddings explained without math covers how text becomes a vector and why similar meanings land near each other. That part is shared. The write path is where the two diverge.
The clean test: ask whether the agent's own output can ever change what the store returns next time. If no, you have RAG. If yes, you have memory. Everything else, statelessness, personalization, persistence, follows from that one property.
The write path, step by step: extract, store, deduplicate
Memory's write path has three core stages: extract a fact from raw text, decide whether it is new, and store or merge it. RAG has none of these because nothing gets written. Mem0's published architecture is a clear reference implementation, so the steps below follow its extract-and-update pipeline.
Step 1: Extract
An LLM reads the latest message pair and pulls out salient, atomic facts. Not the whole conversation, just the durable claims worth keeping. A message like, my flight got moved to the 9am out of Newark, becomes a structured fact such as user departs from Newark at 9am. Mem0 feeds the extractor both a running conversation summary and recent messages so the fact has enough context to be unambiguous.
Step 2: Find candidates
Before writing, the system embeds the new fact and retrieves semantically similar memories already in the store. This is the same nearest-neighbor query RAG would run, but here it serves a different purpose: not to answer a question, but to ask, do I already know something about this? The returned candidates are what the next stage reconciles against.
Step 3: Decide the operation
For each extracted fact, Mem0's update phase uses an LLM to choose one of four operations against the retrieved candidates: ADD a genuinely new memory, UPDATE an existing one with more detail, DELETE a memory the new fact contradicts, or NOOP when the fact is already known and nothing changes. Deduplication is not a separate cleanup job. It is the default path: if the candidate already covers the fact, the operation is NOOP and no duplicate is written.
The same vector retrieval appears in both worlds. In RAG it is the whole job. In memory it is one step inside a write loop whose real work is the decision: add, update, delete, or do nothing. That decision is what RAG structurally cannot make, because RAG never writes.
Conflict resolution: when a stored fact changes
Here is what most explainers miss: the hard part of memory is not remembering, it is forgetting on purpose. RAG never has to unlearn anything, because its corpus cannot contradict itself. The instant your store can write, every fact you keep is a fact you might one day have to retract, and that retraction logic, not the vector search, is the real engineering.
When a stored fact stops being true, memory has to invalidate the old version, not just append the new one. This is the hardest part of the write path, and one a document corpus never confronts. A document corpus does not contradict itself across time the way a user does when they say, actually, cancel that, move me to Tuesday.
There are two common strategies. The first is destructive: detect the contradiction and DELETE or overwrite the stale memory, which is the approach in Mem0's update phase. The store always reflects the current truth, at the cost of losing history. The second is temporal: keep both versions but timestamp when each was valid.
Zep's Graphiti engine takes the temporal route with a bi-temporal model. Every fact carries two timelines: when it was true in the world (a valid-from and a valid-until time) and, separately, when the system learned it and when the system retired it. When a new fact supersedes an old one, the system does not delete the old edge; it invalidates it by stamping the moment it stopped being true. The agent can then reason over history: what did the user prefer last month versus now. That distinction, between forgetting a fact and recording that it stopped being true, has no analogue in RAG.
If your facts rarely change, destructive overwrite is simpler and cheaper. If you need to answer, what was true at the time, or audit how a decision was made, you need temporal invalidation. Choose before you build; retrofitting time into a destructive store means reconstructing history you already threw away.
The downsides of agent memory: what statefulness costs
Statefulness is not free. A read-only RAG corpus is easy to reason about: rebuild it and every query is reproducible. A mutable memory store carries costs RAG does not, so name them before you commit to a memory layer.
- Write-time LLM calls: extraction and the add/update/delete decision each cost a model call per interaction. RAG spends nothing at write time because there is no write.
- Non-reproducibility: the same conversation can leave the store in different states depending on extraction quality. Two users with identical histories may not have identical memories.
- Drift and corruption: a bad extraction writes a wrong fact that then influences future retrievals. Errors compound because the store feeds itself.
- Privacy surface: a store that remembers personal facts across sessions needs deletion, scoping, and retention controls that a public document corpus does not.
- Conflict-resolution edge cases: deciding whether two facts truly contradict is itself an LLM judgment that can be wrong.
The upside is measurable. Mem0 reports that selectively storing and retrieving facts, rather than stuffing the full conversation into context, cut p95 latency by 91% and token cost by more than 90% versus a full-context baseline, while scoring 26% higher in relative terms than OpenAI's built-in memory on the LoCoMo long-conversation benchmark. The point is not that memory always wins; statefulness buys efficiency and personalization at the cost of operational complexity.
A worked example: one fact through RAG vs through memory
Follow a single sentence through each system. The user says, I am allergic to penicillin, in a healthcare assistant.
Through RAG: nothing happens to the store. The sentence is part of the current prompt, possibly used to retrieve documents about penicillin alternatives from a medical corpus. When the session ends, the fact is gone. Next week the user opens a new session and must say it again, because RAG's corpus only contains documents someone curated in advance, not anything the user told it.
Through memory: the extractor produces an atomic fact, user is allergic to penicillin. The system embeds it, retrieves similar existing memories, finds none, and runs ADD. Next week the fact surfaces automatically when the agent considers a prescription. If the user later says, my allergy testing came back clear, the write path retrieves the stored allergy fact, detects the contradiction, and either DELETEs it or, under a temporal model, sets invalid_at so the record shows the allergy was believed true until that date. One sentence, two different fates, decided entirely by whether the system writes.
| Property | RAG | Agent Memory |
|---|---|---|
| Data source | Fixed external corpus someone curated | Facts the agent extracts from its own use |
| Direction | Read-only | Read and write |
| Write path | None at query time | Extract, dedup, add/update/delete |
| When a fact changes | Corpus is rebuilt out of band | Old fact overwritten or invalidated in place |
| State across sessions | None; each query is independent | Persists; later sessions reuse earlier facts |
| Reproducibility | High; rebuild gives same store | Lower; store evolves with interactions |
| Shared primitive | Vector nearest-neighbor search | Vector nearest-neighbor search |
When you genuinely need both layers
Most production agents need both, because they answer two different kinds of questions. RAG answers, what does the documentation say, grounded in authoritative content the agent should not invent. Memory answers, what do I know about this specific user or task, grounded in history the agent accumulated.
A support agent is the canonical case. It reads product docs through RAG to give correct, current answers, and it writes to memory to remember that this customer is on the enterprise plan, already tried the reset, and prefers email. Neither layer can do the other's job. RAG cannot remember the customer; memory should not be the source of truth for how the product works. Getting the chunking right on the RAG side is its own discipline, covered in our RAG chunking strategy recipe; getting the write path right is the memory side this post is about.
Separate memory from the context window. The context window is the model's short-term working set for a single call, bounded and transient. Memory is the durable store that survives across calls and decides what enters that window. Our piece on context window vs memory draws that line in full; the short version is that a bigger context window does not give you memory, because nothing persists once the call returns.
Decision checklist: pick memory, RAG, or both
- The knowledge is authoritative, external, and changes on a publish schedule (docs, policies, papers): use RAG.
- The knowledge is generated during use and specific to a user, session, or task: use memory.
- Facts in your domain get superseded and you must answer what was true when: use memory with temporal invalidation, not destructive overwrite.
- You need both correct external grounding and per-user continuity: run both layers and keep their stores separate.
- You only ever need single-turn answers over static content with no personalization: RAG alone is enough, and the write-path cost of memory is wasted.
- Reproducibility and auditability dominate, and the data never mutates: prefer RAG; a mutable store is harder to reason about.
Once you decide you need the write path, the engineering question is who owns extraction, deduplication, and conflict resolution. Building that loop in-house means maintaining the extraction prompts, the add/update/delete logic, and the invalidation rules yourself. If what you want is that memory for yourself rather than inside an app you are shipping, MemX handles the extract-store-reconcile path as a finished consumer product, a personal memory over your own documents, photos, and notes, so you are not building the loop at all.
01Is agent memory just RAG with extra steps?
They share the same vector retrieval, but memory adds a write path RAG lacks. RAG reads a fixed corpus; memory extracts facts from use, deduplicates them, and reconciles them when they change. The retrieval is identical; the lifecycle, read-only versus read-write, is the real difference.
02Can RAG remember things between sessions?
No. Standard RAG retrieves from a static corpus and writes nothing back, so each query is independent and anything the user said is gone when the session ends. Remembering across sessions requires a memory layer that writes facts to a store the next session can read.
03What happens when a stored fact becomes wrong?
Memory either overwrites it destructively, deleting the stale fact so the store reflects current truth, or invalidates it temporally, keeping the old fact with a timestamp marking when it stopped being valid. Temporal invalidation lets the agent answer what was true at a given time; destructive overwrite is simpler but loses history.
04Do I need both RAG and agent memory?
Often yes. RAG grounds answers in authoritative external content like documentation, while memory tracks per-user and per-task facts the agent learned. They answer different questions and run as separate stores. Use RAG alone only when content is static and no personalization is needed.
05Does a bigger context window replace memory?
No. The context window is transient working memory for one model call; nothing persists after it returns. Memory is a durable store that survives across calls and decides what enters the window. A larger window holds more at once but still forgets everything when the call ends.
