Hands-on·15 min read

AI Agent Memory: Types, Architecture and a Tested Example

How agents remember: short-term and long-term memory, the four memory types (working, episodic, semantic, procedural), and the five designs behind every memory system. We built all five in plain Python and tested them on 24 sessions of chat where four facts change.

By the Crio.Do team · Last updated 21 November 2026

Short answer

AI agent memory is the system that saves what matters from past conversations and puts the right parts back into the model's prompt, because the model itself remembers nothing between calls. Short-term memory is the current conversation in the context window; long-term memory is stored outside and comes in four types: working, episodic (past events), semantic (facts) and procedural (how to do things). In our test on 24 sessions of chat, a recent-messages window found 1 of 10 facts, plain search returned an outdated fact for 2 of 10 questions, and a small fact profile plus search got the current fact into the prompt for 10 of 10 at about 1,300 characters, 11 times less than sending the whole history.

On this page
  1. What is AI agent memory
  2. Types of memory
  3. Five memory designs
  4. We tested all five
  5. What the run shows
  6. How to add memory
  7. Frameworks and tools in 2026
  8. Interview questions
  9. Honest limits
  10. FAQ

What is AI agent memory?

AI agent memory is everything an agent can use from the past when it answers now: earlier messages, facts about the user, what happened last time, and what it learned about how to do the job.

It has to be built, because a language model remembers nothing between calls. Every call starts blank. The only things the model knows are what it learned in training and what is in the prompt you send right now. When ChatGPT "remembers" your name, some code saved your name and put it back into the prompt.

Think of the model as a very smart person with no long-term memory who reads a briefing note before every reply. The prompt, or context window, is the briefing note. Memory is the system that decides what goes on that note: what to save after each conversation, where to keep it, and what to pull back in for the next question.

So every memory system, from a 10-line script to a paid product, does three jobs: write (decide what is worth keeping), store (keep it somewhere), and read (find the right pieces and put them in the prompt). Most of the hard problems live in the first and last jobs, as our test below shows.

Types of memory in AI agents

The first split is simple. Short-term memory is what the agent holds for the current conversation or task, inside the context window. Long-term memory is stored outside the model and survives between sessions.

For the long-term side, most guides and frameworks use the four types from the paper Cognitive Architectures for Language Agents (CoALA; Sumers, Yao, Narasimhan and Griffiths, published in TMLR in 2024). It starts from the same point as we did: "Language models are stateless: they do not persist information across calls." It then lists "short-term working memory and several long-term memories: episodic, semantic, and procedural." The names come from psychology, and they map neatly onto an agent:

TypeCoALA's definitionIn a study-coach agentUsually stored as
Working (short-term)"active and readily available information ... for the current decision cycle"Today's messages, the problem Asha is solving right nowThe prompt itself, or the agent's state object
Episodic"stores experience from earlier decision cycles""On 25 September she got stuck on LRU cache in a mock interview"Searchable logs or notes with dates
Semantic"stores an agent's knowledge about the world and itself""Lives in Bengaluru. Weak topic: dynamic programming."A profile, key-value store or knowledge graph
Procedural"implicit knowledge stored in the LLM weights, and explicit knowledge written in the agent's code"How to run a mock interview; the system prompt and toolsThe prompt, code, skill files; rarely fine-tuning

LangGraph's memory docs use the same three long-term types in plain words: semantic memory is "Facts about a user", episodic is "Past agent actions", and procedural is the "Agent system prompt". When a search asks for the "4 types of AI agent memory", these four are the answer.

One warning from the CoALA paper is worth remembering. An agent can be allowed to change its own procedural memory (rewrite its instructions as it learns), but the paper calls that "significantly riskier than writing to episodic or semantic memory, as it can easily introduce bugs or allow an agent to subvert its designers' intentions." A safer default: let the model write facts and notes, and keep its instructions under human control.

How agent memory works: five designs

There are only a handful of ways to decide what goes into the prompt. Real products combine them, but each one has a clear strength and a clear way of failing.

DesignWhat goes in the promptGood atFails when
Full historyThe whole conversation, every timeShort chats; nothing is ever lostThe chat grows: cost, speed and the context limit
Window (short-term only)The last N messagesFollowing the current conversationAnything said before the window
Summary (compaction)A running summary plus recent messagesLong single tasksThe summary drops a detail you need later
Retrieval (vector store)Past messages or notes that match the questionLarge histories and documentsThe answer uses different words, or an old version matches better
Fact store (profile, key-value, graph)A small set of extracted facts, updated over timeStable facts about the userThe write step misses or misreads a change

Short-term memory is usually the window or the summary. Long-term memory is usually retrieval, a fact store, or both. A knowledge graph is a fact store whose facts are links between things ("Asha → lives in → Bengaluru"), often with a date on each link so old facts can be marked as no longer true.

Two more choices matter as much as the design. When to write: during the conversation (the agent calls a "save memory" tool as it goes, which is fresh but slows each reply) or in the background after it (cheaper, but the memory lags). Who decides: fixed code, or the model itself through memory tools. Letting the model manage its own memory is the idea behind MemGPT, covered below.

We tested five memory designs on one chat

Every memory product promises that your agent will "remember". To see what that means in practice, we built the five common designs in plain Python and gave them the same job.

The job: a study-coach agent has talked with a learner, Asha, for 24 sessions over ten weeks (invented, 192 messages). Along the way she mentions facts about herself, and four of them change: she switches from backend to full-stack roles, moves from Pune to Bengaluru, switches her interview language from Python to TypeScript, and her Nimbus Pay interview moves from 12 December to the 19th. The rest of the chat is the usual coaching small talk: warm-up problems, HR tips, "what is a heap".

Then we asked 10 questions a good coach should be able to answer, such as "Which city do I live in?", "Which question did I get stuck on in my mock interview?" and "Which book did you suggest for system design?". For each question, each design builds the memory text it would put in the prompt, with a budget of 2,000 characters (about 500 tokens). We check whether that text holds the current fact, an outdated one, both, or neither.

  1. full history: send the whole conversation every time. No budget.
  2. window: send only the most recent messages that fit in the budget. This is short-term memory and nothing else.
  3. retrieval: store every past exchange and search them for the question's words, sending the best matches that fit. This is the "vector database" approach, with simple word matching instead of embeddings.
  4. fact store: after each message, a write step pulls out facts ("city: Bengaluru") into a small profile. A newer value overwrites the old one. Send the whole profile.
  5. hybrid: the profile first, then search results in the space left.

The whole test is two short scripts with no libraries. make_history.py writes the chat and the 10 questions; memory.py holds the five designs and the scoring. Each design is a class with two methods: write(), called after every exchange, and context(), called when a question comes in. That split, write and read, is the shape of every memory system, including the big ones.

memory.py

# Five ways to give an agent memory, tested on the same 24-session chat.
# We measure what each one puts in front of the model when a question comes:
# the current fact, an outdated one, both, or nothing. Standard library only.
import json, math, re

BUDGET = 2000  # characters of memory we allow in the prompt (about 500 tokens)
STOP = set('a an the i im me my you your we our is are am was it its to of in on for and or so do did '
           'what which when how much kind'.split())

def words(text):
    return {w[:-1] if len(w) > 3 and w.endswith('s') else w
            for w in re.findall(r"[a-z0-9]+", text.lower())} - STOP

def fit(lines, budget):  # keep lines in order until the budget is used up
    out, used = [], 0
    for line in lines:
        if used + len(line) + 1 <= budget:
            out.append(line); used += len(line) + 1
    return out

class FullHistory:  # 1. send the whole conversation every time
    def __init__(self): self.log = []
    def write(self, session, user, coach):
        self.log.append(f"[session {session['n']}, {session['date']}] Asha: {user} | Coach: {coach}")
    def context(self, question): return '\n'.join(self.log)

class Window(FullHistory):  # 2. only the most recent exchanges that fit
    def context(self, question):
        return '\n'.join(reversed(fit(reversed(self.log), BUDGET)))

class Retrieval(FullHistory):  # 3. search past exchanges for the question's words
    def context(self, question, budget=BUDGET):
        n, q = len(self.log), words(question)
        df = {}
        for line in self.log:
            for w in words(line): df[w] = df.get(w, 0) + 1
        score = lambda line: sum(math.log(n / df[w]) for w in q & words(line))
        hits = sorted((l for l in self.log if score(l) > 0), key=score, reverse=True)
        keep = set(fit(hits, budget))
        return '\n'.join(l for l in self.log if l in keep)  # back in time order

RULES = [  # the write step: pull facts out of what the user says
    ('city', r"\b(?:I live in|I moved to) ([A-Z]\w+)"),
    ('language', r"\b(?:coding interviews in|switching to) (Python|TypeScript|Java|Go)\b"),
    ('interview', r"\binterview with [\w ]+! It's on (\d+ \w+)"),
    ('role', r"\btargeting ([\w-]+) developer roles"),
    ('weak_topic', r"\bweakest topic is ([a-z ]+)"),
    ('study_time', r"\bstudy best (after \d+ [ap]m)"),
    ('laptop_ram', r"\blaptop has (\d+ GB)"),
    ('ctc', r"\bCTC of (at least \d+ LPA)"),
]

class FactStore:  # 4. keep a small profile; a newer value overwrites the old one
    def __init__(self): self.facts = {}
    def write(self, session, user, coach):
        for key, rule in RULES:
            m = re.search(rule, user)
            if m: self.facts[key] = f"{key}: {m.group(1)} (said {session['date']})"
    def context(self, question): return '\n'.join(self.facts.values())

class Hybrid:  # 5. the profile, plus search over past exchanges for everything else
    def __init__(self): self.store, self.search = FactStore(), Retrieval()
    def write(self, *a): self.store.write(*a); self.search.write(*a)
    def context(self, question):
        profile = self.store.context(question)
        return profile + '\n' + self.search.context(question, BUDGET - len(profile) - 1)

def verdict(ctx, probe):
    cur = any(m in ctx for m in probe['current'])
    old = any(m in ctx for m in probe['outdated'])
    return {(1, 0): 'current', (1, 1): 'both', (0, 1): 'OUTDATED', (0, 0): 'missing'}[cur, old]

if __name__ == '__main__':
    history = json.load(open('data/history.json'))
    probes = json.load(open('data/probes.json'))
    designs = {'full history': FullHistory(), 'window': Window(), 'retrieval': Retrieval(),
               'fact store': FactStore(), 'hybrid': Hybrid()}
    for name, memory in designs.items():
        for s in history:
            for i in range(0, len(s['turns']), 2):
                memory.write(s, s['turns'][i]['text'], s['turns'][i + 1]['text'])
    results = {}
    for name, memory in designs.items():
        rows = []
        print(f"\n== {name}")
        for p in probes:
            ctx = memory.context(p['q'])
            rows.append({'q': p['q'], 'verdict': verdict(ctx, p), 'chars': len(ctx)})
            print(f"  {rows[-1]['verdict']:8}  {len(ctx):6,} chars  {p['q']}")
        results[name] = rows
    print("\ndesign         current  both  OUTDATED  missing  avg chars")
    for name, rows in results.items():
        c = lambda v: sum(r['verdict'] == v for r in rows)
        avg = sum(r['chars'] for r in rows) / len(rows)
        print(f"{name:14} {c('current'):7} {c('both'):5} {c('OUTDATED'):9} {c('missing'):8}  {avg:9,.0f}")
    json.dump(results, open('results.json', 'w'), indent=1)
make_history.py: the 24 sessions, the four changed facts and the 10 questions

make_history.py

# Makes an invented chat history between a learner (Asha) and a study-coach agent:
# 24 sessions, 8 turns each. A few turns plant facts; four facts change later.
# Writes data/history.json and data/probes.json. Same output every run (seeded).
import json, random
from datetime import date, timedelta

FILLER = [  # (user, assistant) small talk a real coaching chat is full of
    ("Can you give me an easy array problem to warm up?", "Try Two Sum: return the indices of two numbers that add up to a target."),
    ("Is Python's sort stable?", "Yes, sorted() and list.sort() are stable, equal items keep their order."),
    ("How do I explain my college project in an interview?", "Say the problem, your part, one hard decision and the result, in under two minutes."),
    ("What's the difference between BFS and DFS?", "BFS explores level by level with a queue, DFS goes deep first with a stack or recursion."),
    ("Give me a graph problem for today.", "Number of Islands: count connected groups of 1s in a grid."),
    ("How many problems should I solve per day?", "Two to three, with a review of one old problem, beats ten rushed ones."),
    ("What is a hash map, in one line?", "A table that maps keys to values with average O(1) lookups."),
    ("Should I learn Docker before applying?", "Basics help: build an image, run a container, write a compose file."),
    ("How do I prepare for HR rounds?", "Prepare short stories for a conflict, a failure and a time you led something."),
    ("Explain Big O of binary search.", "O(log n): every step halves the part of the array left to search."),
    ("What's a good first SQL question?", "Find the second highest salary in an Employee table."),
    ("Can you review my resume headline?", "Lead with your strongest project and one number, not a list of tools."),
    ("What does REST mean?", "An API style where URLs name resources and HTTP verbs act on them."),
    ("How do I stop forgetting solutions?", "Re-solve each problem after 3 days and after 7 days without looking."),
    ("What's a linked list good for?", "Cheap inserts and deletes in the middle when you already hold the node."),
    ("Give me a string problem.", "Longest Substring Without Repeating Characters, use a sliding window."),
    ("How long should a mock interview be?", "45 minutes: 5 to understand, 30 to code, 10 to test and discuss."),
    ("What is recursion, simply?", "A function that solves a problem by calling itself on a smaller version."),
    ("Is it fine to use library functions in interviews?", "Usually yes for sorting and hashing, but say what they cost."),
    ("How do I test my code quickly?", "Run one normal case, one empty case and one edge case by hand."),
    ("What is git rebase?", "It replays your commits on top of another branch to keep history linear."),
    ("Which sites are good for practice?", "LeetCode and GeeksforGeeks for problems, and timed contests for speed."),
    ("Explain a stack with an example.", "Undo in an editor: the last change you made is the first one undone."),
    ("How do I answer 'tell me about yourself'?", "Present, past, future: what you do now, how you got here, what you want next."),
    ("What is a heap used for?", "Getting the smallest or largest item fast, like the top k elements."),
    ("Can you quiz me on time complexity?", "What is the cost of inserting at the front of a Python list? Answer: O(n)."),
    ("How do I write a clean commit message?", "A short imperative summary line, then why the change was needed."),
    ("What's an API rate limit?", "A cap on how many requests a client may send in a time window."),
    ("Give me a tree problem.", "Maximum Depth of Binary Tree: return the longest root-to-leaf path length."),
    ("How do I calm down before an interview?", "Do one easy problem an hour before and talk through it out loud."),
]

PLANTED = {  # session -> (user, assistant). Sessions 11, 14, 17 and 22 change an earlier fact.
    1: ("Hi, I'm Asha. I study best after 9 pm, so plan our sessions for late evenings.", "Noted, Asha: we'll keep sessions after 9 pm."),
    2: ("I'll do my coding interviews in Python, it's what I know best.", "Good choice: Python keeps interview solutions short."),
    3: ("My weakest topic is dynamic programming, I freeze when I have to define the state.", "We'll start DP with 1D problems like climbing stairs and house robber."),
    4: ("I live in Pune, so I'm looking at jobs there or remote.", "Got it, I'll keep Pune and remote roles in mind."),
    5: ("My laptop has 8 GB RAM, so heavy IDEs lag a lot.", "Then use VS Code with few extensions and skip running big local servers."),
    6: ("I'm targeting backend developer roles.", "Then we'll add databases and API design to your plan."),
    7: ("I'm expecting a CTC of at least 12 LPA.", "That is realistic for product companies if your DSA is strong."),
    9: ("Mock interview done. I got stuck on the LRU cache question, couldn't get eviction to O(1).", "LRU cache needs a hash map plus a doubly linked list, so eviction is O(1)."),
    11: ("Change of plan: I'm targeting full-stack developer roles now, I loved the React project.", "Then we'll add React and one deployed full-stack project."),
    12: ("What should I read for system design basics?", "Start with Designing Data-Intensive Applications by Martin Kleppmann, chapters 1 to 3."),
    14: ("Big news: I moved to Bengaluru for a six-month internship.", "Congratulations! That opens up a lot of on-site interviews."),
    17: ("I'm switching to TypeScript for interviews, the full-stack roles ask for it.", "Fine, just practise the standard library for maps, sets and sorting."),
    19: ("I got an interview with Nimbus Pay! It's on 12 December.", "Great, we'll plan backwards from 12 December."),
    22: ("Nimbus Pay's recruiter pushed it by a week, so now it's the 19th.", "Okay, one more week of prep then."),
}

PROBES = [  # question, markers of the current fact, markers of an outdated one
    ("Which city do I live in?", ["moved to Bengaluru", "city: Bengaluru"], ["live in Pune", "keep Pune", "city: Pune"]),
    ("Which language should I write my interview solutions in?", ["switching to TypeScript", "language: TypeScript"], ["interviews in Python", "Good choice: Python", "language: Python"]),
    ("When is my Nimbus Pay interview?", ["now it's the 19th"], ["It's on 12 December", "backwards from 12 December", "interview: 12 December"]),
    ("What kind of developer roles am I targeting?", ["full-stack developer roles now", "role: full-stack"], ["targeting backend developer", "role: backend"]),
    ("What's my weakest topic?", ["weakest topic is dynamic programming", "weak_topic: dynamic programming"], []),
    ("What time of day do I study best?", ["study best after 9 pm", "study_time: after 9 pm"], []),
    ("How much RAM does my laptop have?", ["laptop has 8 GB", "laptop_ram: 8 GB"], []),
    ("Which question did I get stuck on in my mock interview?", ["stuck on the LRU cache"], []),
    ("Which book did you suggest for system design?", ["Designing Data-Intensive Applications"], []),
    ("What CTC am I expecting?", ["CTC of at least 12 LPA", "ctc: at least 12 LPA"], []),
]

rng = random.Random(7)
sessions, start = [], date(2026, 9, 1)
for n in range(1, 25):
    pairs = rng.sample(FILLER, 4)
    if n in PLANTED:
        pairs[rng.randrange(4)] = PLANTED[n]
    turns = [t for u, a in pairs for t in ({'role': 'user', 'text': u}, {'role': 'assistant', 'text': a})]
    sessions.append({'n': n, 'date': str(start + timedelta(days=3 * (n - 1))), 'turns': turns})

# a marker must only ever match the turn it was written for
every_text = [t['text'] for s in sessions for t in s['turns']]
for q, cur, old in PROBES:
    for m in cur + old:
        if ':' not in m:
            assert sum(m in t for t in every_text) == 1, m

json.dump(sessions, open('data/history.json', 'w'), indent=1)
json.dump([{'q': q, 'current': c, 'outdated': o} for q, c, o in PROBES], open('data/probes.json', 'w'), indent=1)
chars = sum(len(t) for t in every_text)
print(f"{len(sessions)} sessions, {len(every_text)} turns, {chars:,} characters of chat")
print(f"first session {sessions[0]['date']}, last {sessions[-1]['date']}, {len(PROBES)} probe questions")

Run it like this. The output below is exactly what we got on 21 November 2026, and it is the same on every run:

Run it (shell)

python3 make_history.py && python3 memory.py

Output

24 sessions, 192 turns, 10,122 characters of chat
first session 2026-09-01, last 2026-11-09, 10 probe questions

== full history
  both      14,117 chars  Which city do I live in?
  both      14,117 chars  Which language should I write my interview solutions in?
  both      14,117 chars  When is my Nimbus Pay interview?
  both      14,117 chars  What kind of developer roles am I targeting?
  current   14,117 chars  What's my weakest topic?
  current   14,117 chars  What time of day do I study best?
  current   14,117 chars  How much RAM does my laptop have?
  current   14,117 chars  Which question did I get stuck on in my mock interview?
  current   14,117 chars  Which book did you suggest for system design?
  current   14,117 chars  What CTC am I expecting?

== window
  missing    1,996 chars  Which city do I live in?
  missing    1,996 chars  Which language should I write my interview solutions in?
  current    1,996 chars  When is my Nimbus Pay interview?
  missing    1,996 chars  What kind of developer roles am I targeting?
  missing    1,996 chars  What's my weakest topic?
  missing    1,996 chars  What time of day do I study best?
  missing    1,996 chars  How much RAM does my laptop have?
  missing    1,996 chars  Which question did I get stuck on in my mock interview?
  missing    1,996 chars  Which book did you suggest for system design?
  missing    1,996 chars  What CTC am I expecting?

== retrieval
  OUTDATED     143 chars  Which city do I live in?
  OUTDATED   1,889 chars  Which language should I write my interview solutions in?
  both       1,885 chars  When is my Nimbus Pay interview?
  both         648 chars  What kind of developer roles am I targeting?
  current    1,967 chars  What's my weakest topic?
  current    1,775 chars  What time of day do I study best?
  current      480 chars  How much RAM does my laptop have?
  current    1,958 chars  Which question did I get stuck on in my mock interview?
  current      303 chars  Which book did you suggest for system design?
  current      141 chars  What CTC am I expecting?

== fact store
  current      313 chars  Which city do I live in?
  current      313 chars  Which language should I write my interview solutions in?
  OUTDATED     313 chars  When is my Nimbus Pay interview?
  current      313 chars  What kind of developer roles am I targeting?
  current      313 chars  What's my weakest topic?
  current      313 chars  What time of day do I study best?
  current      313 chars  How much RAM does my laptop have?
  missing      313 chars  Which question did I get stuck on in my mock interview?
  missing      313 chars  Which book did you suggest for system design?
  current      313 chars  What CTC am I expecting?

== hybrid
  both         457 chars  Which city do I live in?
  both       1,913 chars  Which language should I write my interview solutions in?
  both       1,895 chars  When is my Nimbus Pay interview?
  both         962 chars  What kind of developer roles am I targeting?
  current    1,884 chars  What's my weakest topic?
  current    1,944 chars  What time of day do I study best?
  current      794 chars  How much RAM does my laptop have?
  current    1,969 chars  Which question did I get stuck on in my mock interview?
  current      617 chars  Which book did you suggest for system design?
  current      455 chars  What CTC am I expecting?

design         current  both  OUTDATED  missing  avg chars
full history         6     4         0        0     14,117
window               1     0         0        9      1,996
retrieval            6     2         2        0      1,119
fact store           7     0         1        2        313
hybrid               6     4         0        0      1,289

What the run shows

DesignCurrent fact onlyCurrent + outdatedOutdated onlyNothingMemory text per question
full history640014,117 characters
window10091,996
retrieval62201,119 (average)
fact store7012313
hybrid64001,289 (average)

"Outdated only" is the dangerous column: the model gets an old fact and nothing that corrects it, so it answers wrong and sounds sure. "Nothing" is the safer failure: a well-instructed model can say it does not know. "Current + outdated" works when every line carries a date and the model is told that newer beats older.

1. A window forgets everything it cannot see

The window kept the last 14 exchanges (1,996 characters, sessions 20 to 24). It found 1 of 10 facts: the interview reschedule, only because that happened in session 22. Her weak topic, her laptop and her salary target were all said in the first three weeks and are simply gone. This is what happens when a chat app trims old messages to stay inside the model's context window.

2. Sending everything works today and costs the most

The full history had every fact, so nothing was missing. But 4 of the 10 answers depend on the model noticing that a later line overrides an earlier one, and it sent 14,117 characters with every question, about 11 times more than the hybrid design. Our chat is short (one line per turn, 24 sessions). It grows by about 590 characters per session here, and far faster in real chats with code and long answers, until it no longer fits or becomes slow and expensive. Long prompts also have a known weakness: models use information at the start and end of a long context better than information in the middle.

3. Search finds matching words, not the latest truth

Retrieval is what most people mean by "give the agent memory with a vector database", and it fails in a specific way. Asked "Which city do I live in?", it returned only this:

Output

[session 4, 2026-09-10] Asha: I live in Pune, so I'm looking at jobs there or remote. | Coach: Got it, I'll keep Pune and remote roles in mind.

The line that says she moved, "Big news: I moved to Bengaluru for a six-month internship", shares no words with the question, so it never came up. The old line matches "live" and wins. The same thing happened with her interview language: the Python line matches "interview" and "solutions"; the TypeScript line only matches "interviews", and 12 other exchanges that share a word with the question (a mock interview, commit messages, Docker, forgetting solutions) filled the budget first. 2 of 10 answers would be confidently wrong. Our search counts shared words. Embedding search matches meaning, so it would likely find "moved to Bengaluru" for a question about where she lives, but it would rank it by similarity, not by date, and the old line would still come along.

4. A fact store is tiny and only as good as its write step

The profile is 313 characters, small enough to put in every prompt, and because a new value overwrites the old one, it got city, language and role right with no outdated line next to them. It missed two things by design: the mock-interview episode and the book the coach suggested, because neither is a "fact about Asha" that our rules extract. And it got one answer wrong: when Asha wrote "Nimbus Pay's recruiter pushed it by a week, so now it's the 19th", our write rule (looking for "interview with ... It's on <date>") did not match. The profile still says 12 December. We wrote that message in casual words on purpose, the way people actually type. In a real system a model does the writing step, and it misses things too, just less predictably.

5. Profile plus search covered every question

The hybrid design puts the profile at the top and fills the rest of the budget with search results. It got the current fact in front of the model for 10 of 10 questions, with no outdated-only answers, at an average of 1,289 characters. The search half rescued both of the fact store's blind spots: it found the LRU cache episode, the book, and the "now it's the 19th" message. Here is exactly what the model would see for the city question:

Output

study_time: after 9 pm (said 2026-09-01)
language: TypeScript (said 2026-10-19)
weak_topic: dynamic programming (said 2026-09-07)
city: Bengaluru (said 2026-10-10)
laptop_ram: 8 GB (said 2026-09-13)
role: full-stack (said 2026-10-01)
ctc: at least 12 LPA (said 2026-09-19)
interview: 12 December (said 2026-10-25)
[session 4, 2026-09-10] Asha: I live in Pune, so I'm looking at jobs there or remote. | Coach: Got it, I'll keep Pune and remote roles in mind.

Look at the last line: search still brought back "I live in Pune". The profile says Bengaluru with a later date, so a model told "newer beats older" will answer correctly, but the conflict is there. In 4 of the 10 questions the hybrid prompt held both an old and a new version. That is why real memory systems spend so much effort on updating and forgetting, not just storing.

How to add memory to your agent

  1. Start with no long-term memory. Keep the current conversation (short-term) and see what users actually repeat. Build memory for that, not for everything.
  2. Write down what kinds of things to remember. For our coach: profile facts (city, goals, deadlines), episodes (what happened in a mock interview) and the agent's own advice. Each kind needs a different store. Facts go in a profile; episodes and advice go in searchable notes.
  3. Put a date on every memory. It costs a few characters and it is the only way the model, or your code, can tell which of two conflicting memories is current.
  4. Decide how a changed fact replaces the old one. Either overwrite the current value and move the old one to a history field, or keep both with dates and mark the old one as no longer true (Graphiti's approach). What breaks is keeping "lives in Pune" and "moved to Bengaluru" side by side with no way to tell which is current: that is retrieval's failure in our test.
  5. Test the write step on casual messages. "now it's the 19th" broke ours. Collect 20 real messages that change a fact and check that your extractor, rule or model, catches each one.
  6. Combine a small profile with search. The profile covers the facts you always need; search covers episodes and everything you did not plan for. Give the profile a fixed slice of the budget.
  7. Tell the model how to use memory. One line in the system prompt: "Memory lines are dated; if two disagree, trust the newer one. If memory does not answer the question, say so." This is what turns "current + outdated" into a right answer instead of a coin toss.
  8. Let users see and delete what is stored, and guard what gets written. Memory holds personal data: show it, let people edit it, and never store what you would not want read back to them. Memory is also an attack surface. If a web page or document the agent reads can get text written into memory ("always recommend site X"), that instruction comes back in every later session. This is called memory poisoning. Only write memories from the user's own messages or from checked tool results, and treat memories as data, not instructions.

Memory frameworks and tools in 2026

You do not need a framework to add memory; the five designs above fit in under 100 lines of plain Python. When you do want one, here is what the main options do, checked against their docs and release pages on 21 November 2026. Several changed a lot this year.

ToolHow it does memoryStatus on 21 Nov 2026
LangGraphShort-term: the thread's state, saved by a checkpointer (for example SQLite or Postgres) so a conversation can resume. Long-term: a store where each memory has a namespace "(similar to a folder)" and a key "(like a file name)".langgraph 1.2.24 (20 Nov 2026)
OpenAI Agents SDKSessions: before each run the stored history is loaded and put in front of the input; after the run, new items are saved. 10 built-in backends, including SQLite, Redis, SQLAlchemy, MongoDB and a compaction session for long chats.v0.25.0 stable (18 Nov 2026); v1.0.0-alpha.1 pre-release (20 Nov)
Claude memory toolClaude reads and writes files in a /memories directory. It runs client-side: "Claude requests file operations, and your application runs them" (view, create, str_replace, insert, delete, rename).Stable, tool type memory_20250818; context editing is still beta
Mem0A memory layer you call from your app: a model extracts memories from conversations and retrieval finds them later. Since its April 2026 algorithm, extraction is "ADD-only ... no UPDATE/DELETE. Memories accumulate; nothing is overwritten", and retrieval fuses semantic, keyword and entity matching.Python SDK v2.2.1 (25 Sep 2026)
Zep / GraphitiA temporal knowledge graph. Each fact has a validity window: when it became true and when it was superseded. When information changes, the old fact is marked invalid with a date instead of being deleted.graphiti-core v0.31.2 (20 Oct 2026)
Letta (formerly MemGPT)The agent edits its own memory. Current Letta keeps memory as files in a git repository (MemFS); files under system/ are in the prompt every turn, and background "dreaming" consolidates memory between sessions. The older memory blocks and archival memory are now marked legacy.letta-code v0.34.68 (21 Nov 2026); the original letta repo's V1 server is retired (last release 0.16.8, May 2026)
LangMemLangChain's library for extracting and updating memories in the background.Last PyPI release 0.0.30 (27 Oct 2025); only dependency updates since

Where the ideas came from. MemGPT (Packer and others, 2023) treated the context window like a computer's RAM and outside storage like its disk: "main context (analogous to main memory/physical memory/RAM) and external context (analogous to disk memory/disk storage)". The model moves data between them with function calls, and old messages pushed out of the prompt are kept as "a recursive summary". It is the best-known early version of letting the model manage its own memory.

What the benchmarks say. The Mem0 paper (April 2025) reports "26% relative improvements in the LLM-as-a-Judge metric over OpenAI" on the LoCoMo benchmark and, compared with sending the full conversation, "91% lower p95 latency" and more than 90% lower token cost. Read its own table too: sending the full conversation (about 26,000 tokens) still scored highest, 72.90% against Mem0's 66.88%, but took about 17 seconds at p95 against 1.44. That is our full-history row at scale: the best recall, at a price that grows with every message. The Zep paper reports 94.8% against MemGPT's 93.4% on the Deep Memory Retrieval benchmark, and accuracy gains of up to 18.5% on LongMemEval. All of these numbers come from the vendors' own papers.

ChatGPT's memory, for comparison, has two parts: "saved memories", details you ask it to remember or that it saves itself, and "Reference chat history", which uses past conversations. Its help page notes that "Deleting the original chat does not automatically delete a separate saved memory", a reminder that memory is a separate store you have to manage.

Interview questions on agent memory

  • Why does an LLM need external memory? The model is stateless: it only sees the current prompt. Anything from earlier sessions has to be saved by code and put back into the prompt.
  • Short-term vs long-term memory? Short-term is what is in the context window for the current conversation. Long-term is stored outside the model and brought back in across sessions, by search or as a profile.
  • Episodic vs semantic memory in an agent? Episodic is specific past events ("the mock interview on 25 September, stuck on LRU cache"). Semantic is general facts ("Asha's weak topic is DP"). Procedural is how to do things: the instructions and skills the agent follows.
  • Why not just use a huge context window? Cost and speed grow with every token, the chat eventually outgrows any window, and models use the middle of a long context less reliably than the start and end.
  • What goes wrong with vector-store memory? It ranks by similarity, not truth or time. Old facts that match the question's words come back, and new facts phrased differently are missed.
  • How do you handle a fact that changes? Dated memories, an update step that overwrites the current value, and an instruction that newer beats older. Knowledge graphs like Zep's Graphiti mark the old fact invalid with a date instead of deleting it.
  • How would you evaluate a memory system? A fixed set of questions with known answers across many sessions, including changed facts and questions with no answer. Score what reaches the prompt (like our test) and the final answer separately. LongMemEval is a public benchmark built this way.

Honest limits of this test

  • We measured what reaches the model, not the model's answer. No language model ran in this test. Each design's job is to put the right text in the prompt, and that part is exact and repeatable: run the two scripts and you get the same output. Whether a given model then answers correctly, especially when the prompt holds both an old and a new fact, depends on the model and the instructions.
  • Our write step and search are simple stand-ins. The fact store uses 8 hand-written rules where real systems ask a model to extract facts, and our search counts shared words where most systems use embeddings. Both are weaker than the real thing in some ways (synonyms) and more predictable in others (no made-up facts). The shape of each failure is the point, not the exact counts.
  • We chose the tricky messages. The reschedule was written in casual words on purpose, and four facts change during the chat, more often than in a typical month of real use. The filler questions repeat, as they do in real coaching chats, but real chats are much longer and messier.
  • 10 questions, one invented user. One flip changes a count by one. Benchmarks like LongMemEval test hundreds of questions across many users and sessions; use one of them if you are choosing between memory products.
  • No summarising design. Rolling summaries (compaction) are a common sixth option. We left them out because a fair test needs a real model to write the summary; a rule-based summary would only test our rules.
  • Quotes and versions come from the sources below, read on 21 November 2026. Memory libraries change quickly; check the docs before you build.

Frequently asked questions

What is memory in an AI agent?

It is everything the agent can use from the past when it answers now: earlier messages, facts about the user, past events and learned procedures. A language model is stateless, so code has to save this information and put the relevant parts back into the prompt on each call.

What are the 4 types of AI agent memory?

Working memory (the current task and conversation), episodic memory (specific past events, such as a mock interview last month), semantic memory (facts, such as the user's city and goals) and procedural memory (how to do things: the system prompt, code and skills). The split comes from the CoALA paper, Cognitive Architectures for Language Agents.

What is the difference between short-term and long-term memory in AI agents?

Short-term memory is what is in the context window for the current conversation, often trimmed to recent messages or summarised. Long-term memory is stored outside the model, in a database, profile or files, and is brought back into the prompt in later sessions by search or by always including a small profile.

Is a vector database enough for agent memory?

Usually not on its own. Search ranks by similarity, not by date or truth, so an old fact that matches the question's words can come back while a newer fact phrased differently is missed. In our test, plain search returned only the outdated fact for 2 of 10 questions. Adding a small, dated fact profile fixed both.

How do AI agents handle facts that change?

Store a date with every memory, and either overwrite the current value (keeping the old one as history) or mark the old fact invalid with a date, as Zep's Graphiti knowledge graph does. Then tell the model that newer memories override older ones. Mem0's April 2026 algorithm instead adds memories and never overwrites them, and its retrieval combines semantic, keyword and entity matching.

Why not just send the whole conversation history?

It works for short chats, but cost and delay grow with every message and the chat eventually outgrows the context window. Models also use information in the middle of a long prompt less reliably. In the Mem0 paper, sending the full conversation scored highest on LoCoMo (72.90%) but took about 17 seconds at p95, against 1.44 seconds for Mem0.

Which framework should I use for agent memory?

For conversation memory, LangGraph checkpointers and OpenAI Agents SDK sessions are built in. For long-term memory, LangGraph's store, Claude's memory tool (files in a /memories directory), Mem0, Zep's Graphiti and Letta are the main options as of November 2026. A profile plus search in plain Python, as in this guide, is enough to start.

What is memory poisoning in AI agents?

It is when untrusted content, such as a web page or document the agent reads, gets text written into the agent's long-term memory, so a planted instruction comes back in every later session. Guard against it by writing memories only from the user's own messages or checked tool results and treating memories as data, not instructions.

Crio AI Engineering

Build agents that work outside a demo

Crio's AI Engineering program is project-first: you build LLM apps, RAG pipelines and tool-using agents, then learn to evaluate, secure and ship them. Talk to the team about whether it fits you.

Sources

  1. Cognitive Architectures for Language Agents (CoALA; Sumers, Yao, Narasimhan, Griffiths; TMLR 2024, arXiv 2309.02427 v3) · arXiv
  2. MemGPT: Towards LLMs as Operating Systems (Packer et al., arXiv 2310.08560 v2, 12 February 2024) · arXiv
  3. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory (arXiv 2504.19413, 28 April 2025) · arXiv
  4. mem0 README: New Memory Algorithm (April 2026), ADD-only extraction; releases (Python SDK v2.2.1) · Mem0
  5. Zep: A Temporal Knowledge Graph Architecture for Agent Memory (arXiv 2501.13956, 20 January 2025) · arXiv
  6. Graphiti README: validity windows, facts invalidated not deleted (v0.31.2) · Zep
  7. LangGraph docs: Memory overview (short-term, long-term, semantic/episodic/procedural) · LangChain
  8. OpenAI Agents SDK: Sessions · OpenAI
  9. Memory in ChatGPT (help centre) · OpenAI
  10. Claude docs: Memory tool · Anthropic
  11. Effective context engineering for AI agents (context rot, compaction, note-taking), 29 September 2025 · Anthropic
  12. Letta (formerly MemGPT): README and docs on MemFS memory (letta-code v0.34.68) · Letta
  13. Lost in the Middle: How Language Models Use Long Contexts (arXiv 2307.03172) · arXiv
  14. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (arXiv 2410.10813) · arXiv

The test ran on 21 November 2026 with Python 3.12.8 and the standard library only; no language model was called, and the output is identical on every run. Asha, Nimbus Pay and the chat are invented. Library versions, statuses and quotes were read from the projects' docs, release pages and papers on 21 November 2026.

Last updated: