Mem0 mem0ai 2.2.1 (open-source Python SDK): our test log

Mem0's open-source SDK 2.2.1 stored a fact from one chat message with a local 3B model and retrieved it with a semantic search (2.04 s) on the second attempt; the first two runs failed on a missing package and an API change.

Result at a glance

What we found

• The memory was extracted as one sentence: 'User is vegetarian and allergic to peanuts, lives in Austin, and has a dog named Biscuit' (a single combined memory rather than four separate facts).
• The search returned that memory with a similarity score of 0.643.
• It worked fully offline with local models; no key or account was used.

What we ran, step by step

# Install with uv (cold, no cache)
uv venv -q -p 3.13 work/py/mem0 && VIRTUAL_ENV=work/py/mem0 uv pip install --no-cache -q mem0ai && du -sm work/py/mem0 | cut -f1 && VIRTUAL_ENV=work/py/mem0 uv pip list 2>/dev/null | grep -i -E '^(mem0ai|mem0ai) '

# Task command, first run
work/py/mem0/bin/python tasks3/mem0_task.py

# Install ollama client (needed for local models)
VIRTUAL_ENV=work/py/mem0 uv pip install --no-cache -q ollama

# Task command, second run
work/py/mem0/bin/python tasks3/mem0_task.py

# Task command, third run (search with filters)
work/py/mem0/bin/python tasks3/mem0_task.py

Raw output (excerpts)

{
 "add": {
  "results": [
   {
    "id": "d727bea0-e15e-4df5-abd0-884db27f30dd",
    "memory": "User is vegetarian and allergic to peanuts, lives in Austin, and has a dog named Biscuit",
    "event": "ADD"
   }
  ]
 },
 "search_hits": [
  [
   "User is vegetarian and allergic to peanuts, lives in Austin, and has a dog named Biscuit",
   0.643
  ]
 ]
}

import json, time, os, shutil
os.environ["MEM0_TELEMETRY"] = "False"
shutil.rmtree("/tmp/mem0-qdrant", ignore_errors=True)
from mem0 import Memory
cfg = {
  "llm": {"provider": "ollama", "config": {"model": "qwen2.5-3b", "temperature": 0, "max_tokens": 400, "ollama_base_url": "http://127.0.0.1:11434"}},
  "embedder": {"provider": "ollama", "config": {"model": "nomic-embed-text", "ollama_base_url": "http://127.0.0.1:11434", "embedding_dims": 768}},
  "vector_store": {"provider": "qdrant", "config": {"path": "/tmp/mem0-qdrant", "embedding_model_dims": 768, "on_disk": True}},
  "history_db_path": "/tmp/mem0-history.db",
}
t = time.perf_counter(); m = Memory.from_config(cfg); init_s = round(time.perf_counter() - t, 2)
res = {"init_s": init_s}
t = time.perf_counter()
r = m.add([{"role": "user", "content": "I'm vegetarian and allergic to peanuts. I live in Austin and my dog is called Biscuit."}], user_id="alex")
res["add_s"] = round(time.perf_counter() - t, 2); res["add_result"] = r
t = time.perf_counter()
s = m.search("What food does this person avoid?", filters={"user_id": "alex"}, limit=3)
res["search_s"] = round(time.perf_counter() - t, 2)
items = s["results"] if isinstance(s, dict) else s
res["hits"] = [(x.get("memory"), round(x.get("score", 0), 3)) for x in items]
txt = " ".join(h[0].lower() for h in res["hits"])
res["ok"] = ("peanut" in txt) or ("vegetarian" in txt)
print("RESULT_JSON " + json.dumps(res))

Gotchas and things that broke

• The Ollama embedder import failed until we installed the separate 'ollama' package.
• search() rejects user_id=; filters={'user_id': ...} is required in 2.2.1, so older tutorials fail with a ValueError.
• On start it logged that spaCy and fastembed are not installed and that keyword (BM25) search is disabled; those are optional extras we did not install.

What we did not test

• Mem0's hosted platform, pricing tiers and quotas: only the open-source SDK was run.
• Graph memory, multi-user isolation, updates and deletes, and the optional extras (spaCy, fastembed).
• Retrieval quality over more than one fact: a single message and a single query is a smoke test, not a benchmark.
• Timing: add() ran while another model job loaded the machine.

Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

uv venv -p 3.13 .venv && VIRTUAL_ENV=.venv uv pip install mem0ai ollama
# serve a local chat model and nomic-embed-text with Ollama, then run the script above

How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the testing rig for the hardware, the local model and the rules we follow, and the test hub for every other test.

Related tests: agent frameworks and sdks

• OpenAI Agents SDK (Python): OpenAI Agents SDK (Python) 0.24.0 passed our one-tool task on a local 3B model: 3 of 3 trials; cold uv install 1.7 s.
• OpenAI Agents SDK for JavaScript: OpenAI Agents SDK for JS (0.21.0) passed 2 of 5 (Node 20) and 1 of 5 (Node 24) at default sampling but 5 of 5 on both Node versions once temperature was pinned to 0.
• OpenAI Python SDK: The plain OpenAI Python SDK (v3.37.0) with a hand-written tool loop passed 3 of 3 trials against a local model; it is our control for every framework test.
• Pydantic AI: Pydantic AI 2.61.0 passed our one-tool task on a local 3B model: 3 of 3 trials; cold uv install 3.7 s.
• Semantic Kernel: Semantic Kernel 1.45.0 returned an empty reply with our standard prompt (0 of 3) but worked once the prompt used its plugin-prefixed tool name (3 of 3); a naming trap with small models.
• smolagents: smolagents 1.26.0 passed our one-tool task on a local 3B model: 6 of 6 trials; cold uv install 1.5 s.

All test logs · How we test

FAQ

Sources

• Mem0 repository (fetched 2026-10-26)

Related

• All test logs
• Methodology and testing rig
• Mem0 facts and sources