Aider 0.86.2: our test log

Aider 0.86.2 implemented a missing function with a local 3B model in 3 of 3 trials (27 to 32 s each); it did not install on Python 3.13 without a C++ compiler, and it left a stray placeholder comment in the file.

Result at a glance

What we found

• 3 of 3 trials produced a working clamp (max(low, min(value, high))) and the test printed ok, in 26.8 to 32.2 s each on a 3B local model, using the whole-file edit format.
• Each result also contained the literal comment "# ... entire content of the file ..." that the model copied from the edit-format instructions. Functionally harmless, but it is junk in your file and shows a small model following the template too literally.
• Aider asks the model to emit a diff-style block for every file in the chat; the 3B model produced an empty block for the test file, and aider logged it as an applied edit. The test file was unchanged (verified with git).

What we ran, step by step

# Install on Python 3.12 with uv
uv venv -q -p 3.12 work/aider312 && VIRTUAL_ENV=work/aider312 uv pip install --no-cache -q aider-chat

# Install on Python 3.13 (first attempt)
uv pip install aider-chat  # on Python 3.13.5

# Verification rerun after resetting the repo
git checkout -- mathutil.py && aider ... && git status --short

Raw output (excerpts)

OPENAI_API_KEY=x OPENAI_API_BASE=http://127.0.0.1:11434/v1 aider --model openai/qwen2.5-3b --edit-format whole --no-show-model-warnings --no-check-update --analytics-disable --no-auto-commits --no-gitignore --no-pretty --no-stream --yes-always --no-detect-urls --map-tokens 0 --message 'Implement clamp in mathutil.py so test_mathutil.py passes. Return value limited to [low, high].' mathutil.py

def clamp(value, low, high):
    """Return value limited to the range [low, high]."""
    return max(low, min(value, high))

# ... entire content of the file ...

from mathutil import clamp
assert clamp(5, 0, 10) == 5
assert clamp(-3, 0, 10) == 0
assert clamp(99, 0, 10) == 10
print("ok")

```



mathutil.py

```diff
@@ -1,3 +1,5 @@
 def clamp(value, low, high):
     """Return value limited to the range [low, high]."""
-    raise NotImplementedError
+    return max(low, min(value, high))
+
+# ... entire content of the file ...
```



This implementation ensures that the `clamp` function returns the value limited 
to the range `[low, high]`.

Tokens: 757 sent, 178 received.
Applied edit to test_mathutil.py
Applied edit to mathutil.py
---
ok

Gotchas and things that broke

• aider-chat did not install on Python 3.13 here: a dependency had no prebuilt wheel for 3.13 and needed a C++ compiler we do not have. Python 3.12 worked. If you use uv, pass -p 3.12, or install a C++ toolchain.
• Small local models cannot use aider's diff edit formats reliably; we forced --edit-format whole, which resends the entire file.
• We disabled analytics, update checks and the repository map to keep the run local and short; defaults were not measured.

What we did not test

• Any hosted model (aider is mostly used with them).
• Repo-map, git auto-commit, multi-file edits, voice and the browser UI.
• A single toy task; three trials is a smoke test, not a benchmark.

Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

uv venv -p 3.12 .venv && VIRTUAL_ENV=.venv uv pip install aider-chat
# serve a local model at http://127.0.0.1:11434/v1 (we used Qwen2.5-3B-Instruct Q4_K_M, name qwen2.5-3b)
# create mathutil.py and test_mathutil.py as shown, git init and commit them, then run the command above

How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the testing rig for the hardware, the local model and the rules we follow, and the test hub for every other test.

Related tests: coding agents and clis

• Claude Code: Claude Code 2.1.310 edited calc.py through its Edit tool with a local 3B model on the second prompt (85.9 s); the first, vaguer prompt made no edit.
• Cline CLI: Cline CLI 3.0.73 edited calc.py correctly with a local 3B model, then failed on every follow-up command with 'Invalid input' until the session was aborted at 68 s.
• Codex CLI: Codex CLI 0.164.0 changed calc.py by running a sed command in its sandbox with a local 3B model; an earlier plain-language edit request ran no edit.
• Continue CLI: Continue CLI 1.5.47 answered a question and fixed a one-line bug with a local 3B model through a YAML config; a 1.6 s, 81 MB install.
• goose: goose 1.56.0 edited calc.py on the second headless run with a local 3B model (19.4 s); the first run called the edit tool but its text did not match, so the file did not change.
• opencode: opencode 1.19.4 edited calc.py on the second headless run with a local 3B model; the first run tried the wrong path (src/calc.py) and failed.

All test logs · How we test

FAQ

Sources

• Aider repository (fetched 2026-10-25)

Related

• All test logs
• Methodology and testing rig
• Study: install size and first-task results across 16 frameworks
• Aider facts and sources