To begin, let's explain what Mem0 and Zep are. They are specialized artificial intelligence tools that help AI remember and organize information from conversations, as if it had a memory; Mem0 stores facts and summaries in a simple vector system, while Zep creates a more complex graph of relationships between things, which can result in more automatic updates and higher costs.
When you look at systems like Mem0 or Zep, they promise cost savings of up to 90% and faster responses. But in real-world deployments, engineers often shake their heads—these tools are cumbersome, cause delays, and do not work in practice as advertised. The author of the article decided to investigate this more deeply. He took two popular systems—Zep with its graph structure and Mem0 as a universal memory—and tested them against the 2025 MemBench benchmark. It tests reflective memory and reasoning in conversations.
The goal was to simulate 4,000 conversational cases using the gpt-5-nano model. The task was simple: conduct a conversation, extract facts, and recall them later. That is exactly the type of work these systems are designed for.
The results were shocking. The basic long-context approach achieved an accuracy of 84.6%, averaging 4,232 input tokens, with a latency of 7.8 seconds and a total cost of $1.98 (approximately CZK 45 at an exchange rate of CZK 23 per dollar). Mem0 with vectors achieved an accuracy of only 49.3%, averaging 7,319 tokens, with a latency of 154.5 seconds and a cost of $24.88 (approximately CZK 572). Zep with Graphiti achieved 51.6% accuracy but consumed an average of 1.17 million tokens per case, with a latency of 224 seconds and an estimated cost of around $152.60 (approximately CZK 3,510)—the test had to be stopped after 1,730 cases due to high expenses after 9 hours.
At first, the author thought there was an error in his setup because a million tokens for a simple conversation sounds absurd. But after examining the logs, he found that it was not an error—it was how these systems work.
Architecture: LLM-on-Write
These systems do not function merely as simple storage. They use a method the author calls LLM-on-Write. They capture every message and run artificial intelligence processes in the background to extract its meaning.
With Mem0, which relies on facts and summaries, three parallel processes run for every interaction: updating the conversation timeline to create a narrative summary, identifying facts to store in the vector database, and checking for contradictions to modify or delete old facts. Each of these processes calls the LLM separately. Thus, a single message from the agent triggers three separate inferences.
Zep with Graphiti goes even further. When you say "I am working on Project X," it does not merely try to store the text. It runs an extractor that identifies the "User" entity and the "Project X" entity and creates a "Works on" edge. It then searches the graph to determine whether this conflicts with other facts, such as "User works on Project Y." If it finds a conflict, it triggers another LLM call to resolve it. In the experiment, this led to a chain reaction of updates—one change triggered an update to a neighboring node, which prompted the edge to be summarized again. On average, this meant 1,028 LLM calls per case and a total of 2 billion input tokens.
A Shared Flaw: Fact Extraction
Although Mem0 (vectors) and Zep (graph) differ, they both suffer from the same problem: reliance on LLM-based fact extraction. These models interpret raw data as "facts," which works for personalization, such as remembering that a user likes the color blue. But it does not work for autonomous agents that need to reduce costs and latency.
The extractor is nondeterministic, meaning it can rewrite "I was sick last year" as "current status: sick." The error occurs during writing, so the data is corrupted before it is even stored. No search optimization can fix this because the database is full of hallucinations. The main LLM then depends on the accuracy of this extractor.
On top of this comes the N+1 tax—every message triggers a series of calls, such as extraction, summarization, and graph updates. This increases latency and costs because you are layering LLM upon LLM and paying for noise at every layer.
Marketing Conceals the True Costs
Vendors of these systems emphasize the cost of a single retrieval—how cheap it is to load a small context instead of the entire history. But in practice, you pay for the entire conversation, including the extraction tax and recursive updates. Zep is more honest about this than most because it focuses on temporal graphs for personalization and business context. Even so, the costs are too high at production scale, even for its intended use case. People often use these systems for tasks they were not designed for, such as tracking an agent's execution state.
The hype around "universal memory" is appealing because it sounds amazing—infinite context for nothing. But the architecture does not allow it.
Working Memory Versus Semantic Memory
The experiment shows that universal memory does not exist. We are trying to solve two different problems with one tool. Semantic memory serves the user—it tracks preferences, long-term history, and relationships. It is meant to be fuzzy, extracted, and graph-based. Working memory serves the agent—it tracks file paths, variable names, and immediate error logs. It must be lossless, temporal, and precise.
Using a semantic tool such as Zep or Mem0 for working tasks is not merely a compromise—it is the wrong choice. It is like running a database on a lossy compression algorithm. State is the core of an application and cannot be reliably compressed. Semantic memory is excellent for personalization across sessions but disastrous for state during a task. They need to be separated and used independently.
