Codex CLI 0.164.0: our test log

Codex CLI 0.164.0 changed calc.py by running a sed command in its sandbox with a local 3B model; an earlier plain-language edit request ran no edit.

Result at a glance

What we found

• With an explicit instruction to run sed, Codex executed the command in its workspace-write sandbox and the command exited 0.
• The plain-language request (run 2) did not edit the file in the log: the small model went to a sub-agent tool call and produced a parse error.
• We kept two Codex logs, so this is two runs, not a pass rate. We did not store a separate file-content check for run 3; the exit code and the command are what the log shows.

What we ran, step by step

# npm install (shared command, cold)
npm i @anthropic-ai/claude-code @openai/codex @google/gemini-cli opencode-ai

# Run 2: plain-language edit request
codex exec --oss --local-provider ollama -s workspace-write "The file calc.py is in the current directory. Its add function returns a-b. Use the apply_patch or shell tool to change it so it returns a+b. Do not ask questions; make the edit."

# Run 3: told to run a sed command
codex exec --oss --local-provider ollama -s workspace-write --json   # the prompt told it to run the sed command (exact wording not saved)

Raw output (excerpts)

OpenAI Codex v0.164.0
model: qwen2.5-3b:latest
provider: ollama
sandbox: workspace-write [workdir, /tmp, $TMPDIR]
collab: SpawnAgent
...
2026-10-25T22:45:01Z ERROR codex_core::tools::router: error=failed to parse function arguments: missing field `type` at line 1 column 336

{"type":"item.completed","item":{"type":"command_execution","command":"/bin/bash -lc \"sed -i 's/a-b/a+b/' calc.py\"","exit_code":0,"status":"completed"}}
{"type":"turn.completed","usage":{"input_tokens":7934,"cached_input_tokens":3930,"output_tokens":152}}

Gotchas and things that broke

• Codex warned that metadata for qwen2.5-3b:latest was not found and used fallback metadata; the model name needs the :latest tag.
• The sed command did the edit, not Codex's apply_patch tool, so this does not show that apply_patch works with a small model.

What we did not test

• ChatGPT sign-in, OpenAI-hosted models, presets, /model and /permissions arguments, MCP sign-in and image previews.
• The apply_patch tool, and any run where the model was left to choose its own editing method successfully.
• Reliability: two runs, so this is not a pass rate.

Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

npm i @openai/codex
# create calc.py with `return a-b`, run a local Ollama server, then:
codex exec --oss --local-provider ollama -s workspace-write --json "Run this command in the current directory: sed -i 's/a-b/a+b/' calc.py"

How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the testing rig for the hardware, the local model and the rules we follow, and the test hub for every other test.

Related tests: coding agents and clis

• Continue CLI: Continue CLI 1.5.47 answered a question and fixed a one-line bug with a local 3B model through a YAML config; a 1.6 s, 81 MB install.
• goose: goose 1.56.0 edited calc.py on the second headless run with a local 3B model (19.4 s); the first run called the edit tool but its text did not match, so the file did not change.
• opencode: opencode 1.19.4 edited calc.py on the second headless run with a local 3B model; the first run tried the wrong path (src/calc.py) and failed.
• OpenHands CLI: OpenHands 1.16.0 installed (14 s, 537 MB) and started its agent, but with a local 3B model it made 0 edits in 2 trials: calc.py was unchanged.
• Aider: Aider 0.86.2 implemented a missing function with a local 3B model in 3 of 3 trials (27 to 32 s each); it did not install on Python 3.13 without a C++ compiler, and it left a stray placeholder comment in the file.
• Claude Code: Claude Code 2.1.310 edited calc.py through its Edit tool with a local 3B model on the second prompt (85.9 s); the first, vaguer prompt made no edit.

All test logs · How we test

FAQ

Sources

• GitHub release rust-v0.164.0 (fetched 2026-10-25)

Related

• All test logs
• Methodology and testing rig
• Study: install size and first-task results across 16 frameworks
• Codex CLI facts and sources