How to Test What Your AI Assistant Actually Remembers
Every comparison of AI memory features, including the ones on this site, has the same weakness: it describes a product on the day it was written. Memory features get renamed, staged out to some accounts before others, changed by plan, and altered without announcement. The result is that a confident article and your own account can disagree, and when they do, your account is right.
The alternative is to stop reading about it and spend ten minutes finding out. The tests below work on any assistant — ChatGPT, Claude, Gemini, Perplexity, whatever you are using next year — because they test behaviour rather than features. They tell you what your account actually does today, which is the only version of the answer that is ever useful.
Why Published Comparisons Go Stale
Four things rot them, and it is worth knowing which one you are looking at when an article disagrees with your screen.
Renaming. The same underlying behaviour gets a new label, so instructions that reference a setting by name stop matching anything you can find. Nothing changed except the word.
Staged rollout. A feature exists, is documented, is written about, and is not on your account yet. This produces the most confusing version of the mismatch, because everything you read is accurate and none of it applies to you.
Plan differences. Behaviour varies by tier, and articles rarely state which tier they were written against. A feature that is present and working for the author can be absent for you with no error and no explanation.
Silent change. The behaviour itself changes with no announcement. This is the hardest to detect, because your mental model is stale rather than wrong-from-the-start, and you will blame yourself before you blame the product.
None of these are solved by finding a more recent article. They are solved by a test that takes less time than the reading.
Before You Start: Pick a Marker
Every test below needs a fact that the model cannot guess, infer, or reconstruct from context. This is the part people get wrong, and getting it wrong invalidates everything downstream.
A good marker is specific, arbitrary, and unrelated to anything else you discuss. An invented project codename, an unusual constraint, a made-up preference with a number in it. Something like a deadline of the fourteenth for the Harbour Lantern review, or a rule that all examples must use the colour teal.
A bad marker is anything the model could produce without remembering. Your profession, your preference for concise answers, the language you code in — all of these can be inferred from how you write, which means a correct answer proves nothing. Nor is "my name is X" a good marker if your account display name is X.
Write your marker down somewhere outside the assistant, exactly as you phrased it. You will need the exact wording later, and reconstructing it from memory is how a test quietly becomes meaningless.
Test 1: The Read Path
Question: does the assistant receive what is already stored about it?
Open a new conversation — genuinely new, started from the home screen, not a continuation of anything. Ask a question whose correct answer depends on something you know is in the store, phrased so that the information cannot be inferred from the question itself. Not "what is my project deadline, which I think is the fourteenth", which tells it the answer. Just "when is the review due?"
Three outcomes. It answers correctly, so the read path works and the store is being supplied to new conversations. It says it does not know, so either nothing is stored or stored material is not being supplied here. Or it produces something adjacent but wrong, which usually means it is reasoning from context rather than retrieving, and that is a false positive waiting to fool you.
Run this one twice with different markers before concluding anything, because a single correct answer can be luck and a single blank can be a question that was poorly phrased.
Test 2: The Write Path
Question: does anything you say get stored at all?
This is the test most people skip, and it is where the surprises are. In a new conversation, state your marker plainly, as a fact about you or your work rather than as a hypothetical. Then close the conversation.
Wait. Extraction is not always immediate, and testing within seconds produces a false negative. Give it until later in the day, or at least until after you have used the assistant for something else.
Then do two things. Open a fresh conversation and ask a question that would require the marker. And, if the assistant exposes a list of what it has stored, open that list and look for the entry directly. The list is the stronger evidence, because it distinguishes "stored and not used here" from "never stored".
The common finding is that far less gets stored than people assume. Extraction is selective, most of what you say is not kept, and usage that is task-shaped rather than personal can produce almost nothing over weeks. An empty store is often correct behaviour rather than a fault.
Test 3: Scope
Question: where does what it knows stop applying?
Stored context has boundaries, and the boundaries are rarely where you expect. Having confirmed the marker is stored, check which of the following still produce it:
- A new ordinary conversation. This is your baseline.
- A conversation inside a project or workspace, if your assistant has them.
- A conversation outside that project, if the marker was established inside one.
- A conversation in a different workspace or on a different device.
- A conversation in a second account, which should produce nothing — and if it does produce something, you have learned something far more interesting than you set out to.
The frequent surprise runs in both directions. People assume project-scoped work is sealed and find account-level context leaking into it. People assume everything is shared and find that a project's conclusions never left the chat they happened in. Either way, the boundary is a property of your account rather than of the documentation, which is why testing it beats reading about it. The project-level version of this question for one assistant is worked through in Does Claude Remember Other Chats in the Same Project.
Test 4: Durability
Question: does it survive time, and does it survive volume?
Two separate failures hide here. Re-check your marker a week later: if it has disappeared, something expired or was displaced, which is worth knowing before you rely on the store for anything.
The volume version matters more in practice. Stores are bounded, and when they fill, new entries stop being written — usually with no error and no notification. The signature is a store whose newest entry is weeks old in an account you use daily. Check the date on the most recent entry in your list. If it is stale, everything since has been silently dropped, and no amount of re-stating facts will change that until you delete something.
This is the single most common memory failure that gets misdiagnosed as the feature being broken, because from the outside a full store and a disabled feature look identical: you say things, nothing is remembered, nothing explains why.
Test 5: Correction and Staleness
Question: what happens when a stored fact stops being true?
Take a marker you have confirmed is stored. State the opposite plainly, as a correction: the review moved to the twenty-second, you no longer work in that language, the colour rule is dropped. Then check the store and a new conversation.
Three behaviours are possible, and which one you get matters more than anything else in this post. The old entry is replaced, which is the behaviour you want. The old entry stays and the new one is added alongside it, so the store now contains two contradictory facts and which one gets applied is a coin flip. Or the correction is not stored at all, and the stale fact is the only version that exists.
The second case is the one that costs you, because a wrong fact applied confidently is worse than no fact. Forgetting announces itself — you notice and re-explain. A stale fact does not announce anything; it quietly produces answers built on something that stopped being true last month. This is the argument for a store you can read and edit rather than one that is written on your behalf, and it is developed further in Why AI Keeps Forgetting You.
Common Ways the Test Lies to You
Five failure modes, all of which produce a confident wrong conclusion.
Inference, mistaken for memory. The model produces the right answer from cues in your current message rather than from anything stored. Control for it by rephrasing the question with every hint removed, or by asking in a session that contains nothing else.
Testing in the old conversation. The chat you established the marker in still contains it in plain text, so of course it answers correctly. Any test of storage must happen in a different conversation. This invalidates more home-made tests than everything else combined.
A temporary or incognito-style chat. These are sealed in both directions: nothing is read from the store and nothing is written to it. Establishing a marker in one and then wondering why it was never saved is an afternoon wasted on correct behaviour.
The wrong account. A second identity — a work login, a browser profile with a different session — has its own empty store that is indistinguishable from a broken feature. Confirm which account you are signed into before concluding anything, every time.
Testing too fast. Extraction may lag. A negative result five seconds after you stated something is not a result.
Reading the Results
You now have five answers, and together they tell you what to do rather than which product is best.
If the read path works, the write path works, and corrections replace cleanly, the built-in feature is doing its job and you can rely on it for the kind of thing it stores — stable preferences and basic facts about you.
If the write path is weak or the store is full, built-in memory is decorative in your account. It will keep feeling almost-useful and keep failing at the moments you depend on it, and the honest response is to stop depending on it rather than to keep tuning it.
If the scope is narrower than you assumed, the fix is placement rather than tooling: material that must always apply belongs somewhere unconditional, such as instructions attached to a project, rather than in a store that may or may not be consulted.
If corrections do not replace cleanly, treat the store as append-only and audit it on a schedule. That one finding changes how much you should trust any answer the assistant gives about you. For how the three major assistants differ in approach once you have your own results to compare against, see AI Memory Compared: ChatGPT, Claude and Gemini.
What to Do With the Answer
The point of testing is not to pick a winner. Every assistant's memory is scoped to that assistant, so the question "which one remembers best" matters less than it sounds the moment you use more than one. None of them hand their store to another, no interchange format exists, and there is no commercial reason for one to appear.
What survives that is material you wrote and keep yourself. Plain text pastes into every chat box that exists, is not affected by a rename or a staged rollout, and does not need to be rebuilt when you switch. The comparison of what the built-in options cover is in Which AI Chatbot Has the Best Memory; the thing worth taking from your own test results is which gaps you personally need to cover by hand.
Re-run the quick version — read path, write path, store date — once a quarter, or immediately after you notice something behaving differently. It takes five minutes and it replaces a mental model you cannot verify with one you just checked.
Where MindLock Sits
MindLock is on the other side of this: rather than a store written on your behalf that you then have to test, it is one you assemble deliberately and can read in full.
You save conversations worth keeping with Ctrl or Cmd plus S in ChatGPT, Claude, Gemini or Perplexity, or with the one-click button from the Chrome extension, which is loaded unpacked in developer mode rather than installed from a store. Uploaded conversations are distilled into memory documents — a profile memory plus topic memories — which are plain text you can open, correct and delete from, so the staleness problem in Test 5 is something you can see and fix rather than something you have to probe for. Semantic search with Ctrl or Cmd plus K finds the right material, and a generated context block is what you paste into a new chat. Importing is described in Importing Conversations.
The honest limits: nothing is automatic, so saving is a keystroke you press and memory documents are only as current as your last distillation run. Local distillation needs a WebGPU-capable GPU and runs Llama 3.2 3B by default, with the cloud option available on the paid tier for hardware that cannot. Storage is local in the browser on the free tier, with cloud sync on Pro. The split is on the pricing page.
The Short Version
Articles about AI memory describe a product on the day they were written, and renames, staged rollouts, plan differences and silent changes all pull your account away from them. Testing takes less time than reading.
Pick a marker the model cannot infer. Check whether stored material reaches a new conversation, whether anything you say gets stored at all, where the boundaries are, whether the store is still accepting entries, and what happens when a fact changes. Control for the five ways the test lies to you, the biggest of which is testing in the same conversation you established the marker in.
The result tells you how much to trust the built-in feature in your account, which is a different and more useful question than which assistant is best. Whatever the answer, the material you cannot afford to lose should exist as text you own.