Cline CLI 3.0.73: our test log

Cline CLI 3.0.73 edited calc.py correctly with a local 3B model, then failed on every follow-up command with 'Invalid input' until the session was aborted at 68 s.

Result at a glance

What we found

• The core edit worked: calc.py was changed during the run (file time inside the run window) and read return a + b afterwards.
• Every attempt to verify the change with the run_commands tool failed with {"error":"✖ Invalid input"}; the model retried the same call twice more and the session then ended with an abort message. We did not find out whether that comes from the 3B model emitting malformed arguments or from Cline's tool schema; we did not dig into it.
• We count this as Partly worked, so the tool entry on /tools stays Reported.

What we ran, step by step

# npm install on Node 20 (cold)
npm install cline

Raw output (excerpts)

HOME=work/cl-home DO_NOT_TRACK=1 cline --yolo -P ollama -m qwen2.5-3b -k ollama --config cfg --data-dir data -t 150 'The function add in calc.py subtracts instead of adding. Fix it by editing calc.py.'

⎿ error: {"error":"✖ Invalid input"}
I was unable to verify the edit immediately. To proceed, I will run a simple test case to ensure that the `add` function works as expected.

Let's test the `add` function with a couple of test cases:

- `add(5, 3)` should return `8`.
- `add(-1, 1)` should return `0`.

I will execute the tests and read the output to confirm the function works as intended.


[run_commands]
   ⎿ error: {"error":"✖ Invalid input"}
It seems I encountered an issue again with the tests. Let me re-run the test cases to ensure the function `add` in `calc.py` behaves as expected.

- Test case 1: `add(5, 3)` should return `8`.
- Test case 2: `add(-1, 1)` should return `0`.

I will run these tests to verify the functionality.


[run_commands]
   ⎿ error: {"error":"✖ Invalid input"}
[abort] aborted by another client

---
def add(a, b):
    return a + b

Gotchas and things that broke

• The npm package starts a native binary; run it with an isolated --data-dir and HOME so it does not read your real settings.
• One trial only, so the failure pattern is not a rate.

What we did not test

• The VS Code extension, which is how most people use Cline.
• Any hosted model; a repeat of the failed tool call.

Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

npm install cline
# create calc.py with `return a - b`, then run the command above with a local model served at 127.0.0.1:11434

How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the testing rig for the hardware, the local model and the rules we follow, and the test hub for every other test.

Related tests: coding agents and clis

• Codex CLI: Codex CLI 0.164.0 changed calc.py by running a sed command in its sandbox with a local 3B model; an earlier plain-language edit request ran no edit.
• Continue CLI: Continue CLI 1.5.47 answered a question and fixed a one-line bug with a local 3B model through a YAML config; a 1.6 s, 81 MB install.
• goose: goose 1.56.0 edited calc.py on the second headless run with a local 3B model (19.4 s); the first run called the edit tool but its text did not match, so the file did not change.
• opencode: opencode 1.19.4 edited calc.py on the second headless run with a local 3B model; the first run tried the wrong path (src/calc.py) and failed.
• OpenHands CLI: OpenHands 1.16.0 installed (14 s, 537 MB) and started its agent, but with a local 3B model it made 0 edits in 2 trials: calc.py was unchanged.
• Aider: Aider 0.86.2 implemented a missing function with a local 3B model in 3 of 3 trials (27 to 32 s each); it did not install on Python 3.13 without a C++ compiler, and it left a stray placeholder comment in the file.

All test logs · How we test

FAQ

Sources

• Cline repository (fetched 2026-10-25)

Related

• All test logs
• Methodology and testing rig
• Study: install size and first-task results across 16 frameworks
• Study: the silent version downgrade on older Node, and other install traps