# OpenHands CLI 1.16.0 (SDK 1.21.0): our test log

Canonical: https://theagenticweekly.com/tests/openhands-cli · Last verified: 2026-10-25 · Published: 2026-10-25 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Tested by us on 2026-10-25. Local open-weight model, not the vendor's hosted models. A smoke test on a CPU-only box, not a benchmark.

> OpenHands 1.16.0 installed (14 s, 537 MB) and started its agent, but with a local 3B model it made 0 edits in 2 trials: calc.py was unchanged.

## Result at a glance

Table: OpenHands CLI test record

| Field | Value |
|---|---|
| Result | Failed |
| Version tested | 1.16.0 (SDK 1.21.0) |
| Date and time | 2026-10-25, 23:06 CDT (America/Chicago) |
| Task | In a throwaway folder, give the tool a small bug and ask a local model to fix it through the tool's own file-editing. calc.py contained `def add(a, b): return a - b`; we asked the OpenHands CLI (headless mode) to fix it. Pass means calc.py ends as `return a + b`. Two trials. |
| Model used | Qwen2.5-3B-Instruct (Q4_K_M) through Ollama 0.40.11 |
| Hardware | Linux x86_64 virtual machine (Debian 13), 8 vCPU (Intel Xeon, 1 thread per core), 15 GiB RAM, no GPU |
| Install (uv, Python 3.12) | 14.1 s, 537 MB |
| Edit task trials passed | 0 of 2 |

## What we found

- 0 of 2 trials changed calc.py; the agent produced one empty message and stopped without calling its file editor.
- The agent registers 7 tools and a very long system prompt with many built-in skills; a 3B model may simply be too small to drive it. That is a limit of the model we could run, not necessarily of OpenHands.

## What we ran, step by step

Table: Commands run, wall-clock time and exit code

| Step | Wall time | Exit code | Notes |
|---|---|---|---|
| Install with uv on Python 3.12 | 14.1 s | 0 | openhands 1.16.0, openhands-sdk 1.21.0, openhands-tools 1.21.0; 537 MB. Load 5.35. |
| Edit task, trials 1 and 2 | 10 s | 0 | About 10 s each. The agent initialised and finished with 1 agent message that was empty; calc.py still read `return a - b` afterwards. |

Exact commands (paths shortened)

```
# Install with uv on Python 3.12
uv venv -p 3.12 work/py/openhands-cli && VIRTUAL_ENV=work/py/openhands-cli uv pip install --no-cache -q openhands

# Edit task, trials 1 and 2
openhands CLI in headless mode with its LLM set to openai/qwen2.5-3b at http://127.0.0.1:11434/v1 (exact flags not preserved)
```

## Raw output (excerpts)

Console output (the conversation summary)

```
Initializing agent...
✓ Agent initialized with model: openai/qwen2.5-3b
Agent is working
───────────────────────────── CONVERSATION SUMMARY ─────────────────────────────
Number of agent messages: 1
Last message sent by the agent:
(empty)

```

calc.py after the run

```
def add(a, b):
    return a - b
```

## Gotchas and things that broke

- The log of the first trial is large because the CLI prints the whole system prompt in a panel.
- Counts as Failed in this setup, so the tool entry stays Reported.

## What we did not test

- A larger or hosted model, the GUI/Docker runtime and sandboxed execution.
- Why the model returned an empty message.

## Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

```
uv venv -p 3.12 .venv && VIRTUAL_ENV=.venv uv pip install openhands
# configure an OpenAI-compatible endpoint at http://127.0.0.1:11434/v1 with model openai/qwen2.5-3b and run the CLI headless on a folder containing calc.py
```

## How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the [testing rig](https://theagenticweekly.com/methodology#testing-rig) for the hardware, the local model and the rules we follow, and the [test hub](https://theagenticweekly.com/tests) for every other test.

## FAQ

### Can OpenHands run on a 3B local model?

Not in our two trials: it started but made no edit. We did not test larger local models.

## Sources

- [OpenHands repository](https://github.com/OpenHands/OpenHands) (fetched 2026-10-25)

## Related

- [All test logs](https://theagenticweekly.com/tests)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)
- [Study: install size and first-task results across 16 frameworks](https://theagenticweekly.com/research/agent-framework-install-and-first-task)

## How to cite

Agentic Weekly, "OpenHands CLI 1.16.0 (SDK 1.21.0): our test log", last updated 2026-10-25, https://theagenticweekly.com/tests/openhands-cli