# Ollama 0.40.11: our test log

Canonical: https://theagenticweekly.com/tests/ollama · Last verified: 2026-10-25 · Published: 2026-10-25 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Tested by us on 2026-10-25. Local open-weight model, not the vendor's hosted models. A smoke test on a CPU-only box, not a benchmark.

> Ollama 0.40.11 served three Qwen2.5 sizes over its OpenAI-compatible API on 8 CPU cores: all three answered simple prompts; the 0.5B model failed JSON extraction while 1.5B and 3B passed, and tool calls worked 5 of 5 on all three.

## Result at a glance

Table: Ollama test record

| Field | Value |
|---|---|
| Result | Passed |
| Version tested | 0.40.11 |
| Date and time | 2026-10-25, 22:50 to 22:52 CDT (America/Chicago) |
| Task | Send the same four prompts (a one-word fact, arithmetic, JSON extraction, a two-sentence explanation) and a one-tool function-calling request to Qwen2.5-Instruct at 0.5B, 1.5B and 3B (Q4_K_M GGUF) through Ollama's /v1/chat/completions, temperature 0. Three runs per prompt, five for the tool call. |
| Model used | Qwen2.5-3B-Instruct (Q4_K_M) through Ollama 0.40.11 |
| Hardware | Linux x86_64 virtual machine (Debian 13), 8 vCPU (Intel Xeon, 1 thread per core), 15 GiB RAM, no GPU |
| Pass counts (capital, arithmetic, JSON, tool call) | 0.5B: 3/3, 3/3, 0/3, 5/5. 1.5B: 3/3, 3/3, 3/3, 5/5. 3B: 3/3, 3/3, 3/3, 5/5 |
| Median seconds, two-sentence answer | 0.21 / 0.94 / 2.63 (0.5B / 1.5B / 3B) |
| Load average | 2.6 to 5.4 |

## What we found

- Capability steps are visible at this scale: the 0.5B model answered the fact and arithmetic prompts and called the tool correctly 5 of 5 times, but it returned "Order 8841" where the field should be "8841", so our JSON check failed 3 of 3. The 1.5B and 3B models passed it 3 of 3.
- Median time for the two-sentence answer: 0.21 s (0.5B), 0.94 s (1.5B), 2.63 s (3B) on 8 CPU threads, roughly 109.5, 41.8 and 20.3 completion tokens per second (completion tokens divided by wall time, so including prompt processing; Ollama's /v1 response did not include its own tokens-per-second figure).
- Tool calling through /v1 worked on every size for this one trivial tool; that says nothing about harder schemas.

## What we ran, step by step

Table: Commands run, wall-clock time and exit code

| Step | Wall time | Exit code | Notes |
|---|---|---|---|
| Serve the 3B model (already installed) | n/a | 0 | The 3B model was loaded from a local GGUF file. `ollama pull` from the public registry was blocked on this machine, so all three sizes were imported from GGUF files downloaded from Hugging Face with `ollama create`. |
| Run the comparison script | n/a | 0 | 51 timed requests in all. One warm-up request (not recorded) loads each model first. |

Exact commands (paths shortened)

```
# Serve the 3B model (already installed)
ollama serve  # model qwen2.5-3b from a local GGUF

# Run the comparison script
python modelcmp.py ollama-qwen2.5-3b http://127.0.0.1:11434/v1 qwen2.5-3b  # and 1.5b, 0.5b
```

## Raw output (excerpts)

Results: pass counts and median wall seconds for the writing prompt

```
label                     capital arith json  toolcall write_med_s  write_tok/s  start
ollama-qwen2.5-0.5b       3/3     3/3   0/3   5/5              0.21        109.522:50 CDT
ollama-qwen2.5-1.5b       3/3     3/3   3/3   5/5              0.94         41.822:50 CDT
ollama-qwen2.5-3b         3/3     3/3   3/3   5/5              2.63         20.322:51 CDT
llamacpp-qwen2.5-0.5b     3/3     3/3   0/3   5/5              0.37        124.322:51 CDT
llamacpp-qwen2.5-1.5b     3/3     3/3   3/3   5/5              2.03         41.622:51 CDT
llamacpp-qwen2.5-3b       3/3     3/3   3/3   5/5              2.68         16.422:52 CDT
```

What the 0.5B model returned for JSON extraction (fails the check)

```
```json
{
  "order": "Order 8841",
  "city": "Austin",
  "date": "2026-03-02"
}
```
```

What the 3B model returned for the same prompt (passes)

```
{"order": "8841", "city": "Austin", "date": "2026-03-02"}
```

## Gotchas and things that broke

- Ollama's OpenAI-compatible response omitted native timing fields, so tokens per second here are derived. For native generation speed see the llama.cpp log.
- Timings are from a shared machine whose 1-minute load average ranged from 2.6 to 5.4 during these runs.

## What we did not test

- Larger models, GPUs, concurrency, context lengths beyond a few hundred tokens and quality of long answers.
- A "write" quality judgement: we only timed it.
- Three runs per prompt is a smoke test, not a benchmark.

## Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

```
# import a GGUF (we could not use `ollama pull` here): write a Modelfile with `FROM ./qwen2.5-3b-instruct-q4_k_m.gguf` then
ollama create qwen2.5-3b -f Modelfile
python modelcmp.py ollama-qwen2.5-3b http://127.0.0.1:11434/v1 qwen2.5-3b
```

## How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the [testing rig](https://theagenticweekly.com/methodology#testing-rig) for the hardware, the local model and the rules we follow, and the [test hub](https://theagenticweekly.com/tests) for every other test.

## FAQ

### Is a 0.5B model good enough for tool calling?

For a single trivial tool it worked 5 of 5 times in our run, but the same model failed a simple JSON-extraction prompt that 1.5B and 3B passed.

## Sources

- [Ollama repository](https://github.com/ollama/ollama) (fetched 2026-10-25)
- [Qwen2.5 model family](https://github.com/QwenLM/Qwen2.5) (fetched 2026-10-25)

## Related

- [All test logs](https://theagenticweekly.com/tests)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)
- [Study: Ollama and llama.cpp compared on three Qwen2.5 sizes](https://theagenticweekly.com/research/ollama-vs-llama-cpp-qwen-sizes)

## How to cite

Agentic Weekly, "Ollama 0.40.11: our test log", last updated 2026-10-25, https://theagenticweekly.com/tests/ollama