# goose 1.56.0: our test log

Canonical: https://theagenticweekly.com/tests/goose · Last verified: 2026-10-25 · Published: 2026-10-25 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Tested by us on 2026-10-25. Local open-weight model, not the vendor's hosted models. A smoke test on a CPU-only box, not a benchmark.

> goose 1.56.0 edited calc.py on the second headless run with a local 3B model (19.4 s); the first run called the edit tool but its text did not match, so the file did not change.

## Result at a glance

Table: goose test record

| Field | Value |
|---|---|
| Result | Partly worked |
| Version tested | 1.56.0 |
| Date and time | 2026-10-25, 17:45 to 17:46 CDT (America/Chicago) |
| Task | In a throwaway folder, give the tool a small bug and ask a local model to fix it through the tool's own editing. calc.py contained `def add(a,b): return a-b`. Pass means the file ended as `return a+b` after the tool's own edit. |
| Model used | Qwen2.5-3B-Instruct (Q4_K_M) through Ollama 0.40.11 |
| Hardware | Linux x86_64 virtual machine (Debian 13), 8 vCPU (Intel Xeon, 1 thread per core), 15 GiB RAM, no GPU |
| Run 1 | 27.6 s, file unchanged, 3,404 tokens |
| Run 2 | 19.4 s, file edited, 3,317 tokens |
| Version | 1.56.0 |

## What we found

- Run 2 edited the file; run 1 did not, because the model guessed the text to replace.
- goose's own log marks both edit tool calls as completed; the first completed without changing the file, so a tool "success" is not proof of a changed file.
- Both headless sessions finished with exit type normal (27.6 s and 19.4 s).

## What we ran, step by step

Table: Commands run, wall-clock time and exit code

| Step | Wall time | Exit code | Notes |
|---|---|---|---|
| Unpack the release archive | n/a | n/a | `goose --version` printed 1.56.0. The release archive is the one the official script downloads; the script was read first. |
| Run 1 | 27.6 s | 0 | The edit tool ran with a `before` text that did not exist in the file: "No match found for the specified text." calc.py was unchanged. 3,404 tokens. |
| Run 2 (exact before and after strings) | 19.4 s | 0 | "Edited calc.py (1 lines -> 1 lines)". 3,317 tokens. |

Exact commands (paths shortened)

```
# Unpack the release archive
# unpacked goose-x86_64-unknown-linux-gnu.tar.bz2 with Python, because the official download_cli.sh needs bzip2, which is not installed on this box

# Run 1
goose run --no-session  # provider ollama, model qwen2.5-3b

# Run 2 (exact before and after strings)
goose run --no-session  # same provider and model
```

## Raw output (excerpts)

Run 1 (goose-run1.log, trimmed)

```
▸ edit
  path calc.py
  before: def add(a, b):
  after: def add(a, b): return a + b

No match found for the specified text.
```

Run 2 (goose-run2.log)

```
▸ edit
  path calc.py
  before:     return a-b
  after:     return a+b

Edited calc.py (1 lines -> 1 lines)
```

## Gotchas and things that broke

- Giving goose the exact before and after strings is what made it work; a plain request failed.

## What we did not test

- Subagents, resume and limits, the desktop app, and any hosted model.
- Reliability: two runs, one worked, so this is an existence proof, not a pass rate.

## Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

```
# download the release archive for your platform from the goose releases page and unpack it
goose --version
# configure the ollama provider with a local model, create calc.py with `return a-b`, then:
goose run --no-session -t "In calc.py replace '    return a-b' with '    return a+b' using the edit tool"
```

## How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the [testing rig](https://theagenticweekly.com/methodology#testing-rig) for the hardware, the local model and the rules we follow, and the [test hub](https://theagenticweekly.com/tests) for every other test.

## FAQ

### Does goose work with a local model?

In our run it did on the second attempt, with exact strings, through the ollama provider. The first attempt did not change the file.

## Sources

- [goose v1.56.0 release](https://github.com/aaif-goose/goose/releases/tag/v1.56.0) (fetched 2026-10-25)

## Related

- [All test logs](https://theagenticweekly.com/tests)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)
- [Study: install size and first-task results across 16 frameworks](https://theagenticweekly.com/research/agent-framework-install-and-first-task)
- [goose facts and sources](https://theagenticweekly.com/tools/goose)

## How to cite

Agentic Weekly, "goose 1.56.0: our test log", last updated 2026-10-25, https://theagenticweekly.com/tests/goose