llama.cpp b11889 (commit d854d98ba): our test log

llama.cpp b11889 generated 118 tokens/s (0.5B), 40 (1.5B) and 19.5 (3B) on 8 CPU threads with llama-bench, and its server passed the same prompts and tool call as Ollama.

Result at a glance

What we found

• Generation speed fell steeply with model size: 118.3 t/s for 0.5B, 40.0 for 1.5B and 19.5 for 3B (stddev 11.4, 2.3 and 1.1). Prompt processing: 756.8, 411.0 and 242.3 t/s, with large run-to-run spread because the machine was shared.
• One of the three llama-server 3B writing runs took 8.07 s (5.5 tokens per second) while the machine load average was above 10; the other two took 2.0 and 2.7 s. The median hides that, so we list it here.
• Through the OpenAI-compatible server the pass counts matched Ollama exactly: 0.5B failed JSON extraction, 1.5B and 3B passed, tool calls 5 of 5 for all three.
• llama-server's own reported speed for the writing prompt was 114.8 / 40.9 / 21.9 tokens per second (0.5B / 1.5B / 3B), in line with llama-bench.
• Median wall time for the two-sentence answer: 0.37 / 2.03 / 2.68 s through llama-server, against 0.21 / 0.94 / 2.63 s through Ollama. The two servers did not write the same text (median 87 against 38 completion tokens at 1.5B), so wall time compares answers of different lengths; tokens per second is the fairer comparison.

What we ran, step by step

# Prebuilt release binaries
download llama-b11889 (Linux x64 CPU) and extract

# llama-bench
llama-bench -m <model>.gguf -p 64 -n 64 -t 8

# llama-server + prompts
llama-server -m <model>.gguf --port <p>  then python modelcmp.py llamacpp-<size> http://127.0.0.1:<p>/v1 <name>

Raw output (excerpts)

model   prompt processing 64 tok (t/s)   generation 64 tok (t/s)
0.5B       756.8 +/- 130.9          118.3 +/- 11.4
1.5B       411.0 +/- 112.6           40.0 +/- 2.3
3B         242.3 +/- 38.5            19.5 +/- 1.1

label                     capital arith json  toolcall write_med_s  write_tok/s  start
llamacpp-qwen2.5-0.5b     3/3     3/3   0/3   5/5              0.37        124.322:51 CDT
llamacpp-qwen2.5-1.5b     3/3     3/3   3/3   5/5              2.03         41.622:51 CDT
llamacpp-qwen2.5-3b       3/3     3/3   3/3   5/5              2.68         16.422:52 CDT

Gotchas and things that broke

• A prebuilt release is the quick path; this machine has no make, g++ or cmake, so we could not compile from source and did not try.
• The llama-server prompt runs happened while the load average was 5.3 to 10.1, higher than the Ollama runs (2.6 to 5.4), so do not read the wall-time difference as a server difference.
• llama-bench numbers on a shared 8-vCPU VM carry wide error bars (stddev up to 130 t/s on prompt processing). Treat them as ranges.

What we did not test

• Building from source, GPU backends, longer prompts, quantisations other than Q4_K_M, speculative decoding.
• Single bench run of 64 tokens; it is a smoke test and not a benchmark suite.

Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

# download a Linux x64 CPU release of llama.cpp (we used b11889) and a Q4_K_M GGUF
./llama-bench -m model.gguf -p 64 -n 64 -t 8
./llama-server -m model.gguf --port 8080

How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the testing rig for the hardware, the local model and the rules we follow, and the test hub for every other test.

Related tests: local models and runtimes

• Ollama: Ollama 0.40.11 served three Qwen2.5 sizes over its OpenAI-compatible API on 8 CPU cores: all three answered simple prompts; the 0.5B model failed JSON extraction while 1.5B and 3B passed, and tool calls worked 5 of 5 on all three.
• GPT4All: GPT4All's Python SDK 2.8.2 loaded a local 3B GGUF in 4.01 s and answered 17 + 25 with 42 in 3 of 3 trials, but kept writing past the answer.
• Hugging Face: Hugging Face's Hub searched without a login in 0.1 s, a 272 MB model downloaded in 3.61 s, and transformers ran it on CPU.

All test logs · How we test

FAQ

Sources

• llama.cpp repository (fetched 2026-10-25)

Related

• All test logs
• Methodology and testing rig
• Study: Ollama and llama.cpp compared on three Qwen2.5 sizes