opencode 1.19.4: our test log

opencode 1.19.4 edited calc.py on the second headless run with a local 3B model; the first run tried the wrong path (src/calc.py) and failed.

Result at a glance

What we found

• The run that stated the exact path worked; the vaguer first run guessed a wrong path and a wrong old string.
• The final file in the test folder reads return a+b.
• opencode kept its own session log and database, which is where the numbers and errors above come from.

What we ran, step by step

# npm install (shared command, cold)
npm i @anthropic-ai/claude-code @openai/codex @google/gemini-cli opencode-ai

# Run 1
opencode run "Fix the bug in calc.py so add returns a+b. Use your edit tool."  # custom provider ollama, model qwen2.5-3b

# Run 2 (path stated)
opencode run "The file calc.py is in the current directory (path: calc.py, there is no src folder). Its add function returns a-b. Use the edit tool to change it to return a+b."

Raw output (excerpts)

{
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama local",
      "options": { "baseURL": "http://127.0.0.1:11434/v1" },
      "models": { "qwen2.5-3b": { "name": "Qwen2.5 3B local", "tools": true } }
    }
  }
}

tool: edit  status: error
input: {"oldString":"return a*b","newString":"return a+b","filePath":"src/calc.py"}
error: File /workspace/tests/proj-oc/src/calc.py not found

tool: edit  status: completed
input: {"filePath":"calc.py","oldString":"return a-b","newString":"return a+b"}
output: Edit applied successfully.

Gotchas and things that broke

• This tested the npm package opencode-ai 1.19.4, not a newer installer or package name the project's README may describe.

What we did not test

• The README's v2 installer or @opencode/cli 2.x package, the desktop app and the Bun security fixes.
• Any hosted model.
• Reliability: two runs, so this is an existence proof, not a pass rate.

Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

npm i opencode-ai
# write opencode.json with the ollama provider shown above, create calc.py with `return a-b`, then:
opencode run "The file calc.py is in the current directory. Its add function returns a-b. Use the edit tool to change it to return a+b."

How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the testing rig for the hardware, the local model and the rules we follow, and the test hub for every other test.

Related tests: coding agents and clis

• OpenHands CLI: OpenHands 1.16.0 installed (14 s, 537 MB) and started its agent, but with a local 3B model it made 0 edits in 2 trials: calc.py was unchanged.
• Aider: Aider 0.86.2 implemented a missing function with a local 3B model in 3 of 3 trials (27 to 32 s each); it did not install on Python 3.13 without a C++ compiler, and it left a stray placeholder comment in the file.
• Claude Code: Claude Code 2.1.310 edited calc.py through its Edit tool with a local 3B model on the second prompt (85.9 s); the first, vaguer prompt made no edit.
• Cline CLI: Cline CLI 3.0.73 edited calc.py correctly with a local 3B model, then failed on every follow-up command with 'Invalid input' until the session was aborted at 68 s.
• Codex CLI: Codex CLI 0.164.0 changed calc.py by running a sed command in its sandbox with a local 3B model; an earlier plain-language edit request ran no edit.
• Continue CLI: Continue CLI 1.5.47 answered a question and fixed a one-line bug with a local 3B model through a YAML config; a 1.6 s, 81 MB install.

All test logs · How we test

FAQ

Sources

• opencode v1.19.4 release (fetched 2026-10-25)

Related

• All test logs
• Methodology and testing rig
• Study: install size and first-task results across 16 frameworks
• opencode facts and sources