# Writing leaderboard: fifteen small AI models, scores by criterion

Canonical: https://theagenticweekly.com/tests/writing-leaderboard · Last verified: 2026-10-26 · Published: 2026-10-26 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Tested by us on 2026-10-26 (America/Chicago): open-weight models run locally on a CPU-only machine, outputs rated by a separate model (Mistral 7B Instruct v0.3) that is not a writer in this test. Indicative results, not a benchmark, and not a statement about any vendor's hosted models.

> Llama 3.2 1B leads our writing leaderboard with a 67% (57% to 76%) blind win rate and 4.49 of 5 on the rubric; Qwen2.5 0.5B is last at 19%. Fifteen small open models, 14 prompts, 3 runs each, rated by Mistral 7B Instruct v0.3. Indicative, not a benchmark.

## Added models rated without pairwise games

Table: Added models whose blind pairwise games against the originals were not finished in the time available. They are rated on the rubric and the code-checked format rules and appear in the instruction-following test, but they have no blind win rate and are not in the ranking above

| Model | Size and family | Rubric mean | Format rules passed | Instruction-following strict pass (95% interval) | Median tokens/s |
|---|---|---|---|---|---|
| Falcon3 3B | 3B, Falcon | 4.51 of 5 | 30/42 | 75% (60% to 88%) | 16.9 |
| Granite 3.3 2B | 2B, Granite | 4.50 of 5 | 27/42 | 79% (67% to 91%) | 18.9 |
| Qwen3 1.7B (non-thinking) | 1.7B, Qwen | 4.49 of 5 | 37/42 | 77% (64% to 89%) | 22.9 |
| TinyLlama 1.1B Chat | 1.1B, TinyLlama | 4.18 of 5 | 3/42 | 16% (7% to 28%) | 39.1 |

Rubric means of these models are comparable with the rubric column above (same rater, same outputs-per-model), but they are single-rater scores and the interval-free rubric ordering is weaker evidence than the blind win rate.

## Ranking by blind win rate

Table: Win rate counts a tie as half a win. Intervals are 95%, from resampling the prompts

| Rank | Model | Blind win rate vs the 8 originals (95% interval) | Rubric mean | Format rules passed | Median tokens/s |
|---|---|---|---|---|---|
| 1 | Llama 3.2 1B (added) | 67% (57% to 76%) | 4.49 of 5 | 41/42 | 21.9 |
| 2 | Phi-3.5 mini | 65% (55% to 74%) | 4.47 of 5 | 36/42 | 13.7 |
| 3 | Llama 3.2 3B | 64% (54% to 72%) | 4.45 of 5 | 42/42 | 21.4 |
| 4 | Gemma 2 2B | 63% (56% to 69%) | 4.52 of 5 | 39/42 | 22.5 |
| 5 | Gemma 3 1B (added) | 57% (46% to 67%) | 4.52 of 5 | 35/42 | 13.4 |
| 6 | SmolLM2 1.7B | 56% (46% to 66%) | 4.45 of 5 | 27/42 | 20.9 |
| 7 | Qwen2.5 7B | 55% (42% to 67%) | 4.52 of 5 | 31/42 | 5.9 |
| 8 | Qwen2.5 3B | 44% (37% to 52%) | 4.51 of 5 | 26/42 | 14.4 |
| 9 | Qwen2.5 1.5B | 34% (26% to 42%) | 4.40 of 5 | 27/42 | 32.3 |
| 10 | Qwen3 0.6B (non-thinking) (added) | 29% (19% to 39%) | 4.38 of 5 | 24/42 | 22.5 |
| 11 | Qwen2.5 0.5B | 19% (13% to 27%) | 4.28 of 5 | 19/42 | 56.8 |

Models in this table have finished blind games against all eight originals. A model more than a few points ahead with non-overlapping intervals is meaningfully ahead on these prompts; overlapping intervals mean we cannot separate them.

## Rubric scores by criterion (1 to 5)

Table: The rubric: five criteria, each scored 1 (poor) to 5 (excellent)

| Criterion | What the rater looks for |
|---|---|
| Clarity | Easy to read and understand on the first pass; no confusing wording; ideas in a sensible order. |
| Accuracy to the prompt | Does exactly what the task asked, obeys every stated limit (length, number of sentences, format), and stays consistent with any facts the task supplies; invents nothing. |
| Concision | No filler, repetition, preamble or sign-off that the task did not ask for; every sentence earns its place. |
| Tone | Register and voice fit the audience and purpose (for example warm and apologetic for a customer, plain for a beginner, upbeat for a blurb). |
| Originality | Fresh, specific wording rather than stock phrases and cliches; a concrete detail or angle where the task allows one. |

Table: Mean rater score per criterion over all 630 outputs; the last column has a 95% interval from resampling prompts

| Model | Clarity | Accuracy to the prompt | Concision | Tone | Originality | Rubric mean |
|---|---|---|---|---|---|---|
| Llama 3.2 1B | 5.00 | 5.00 | 4.40 | 4.36 | 3.67 | 4.49 (4.39 to 4.59) |
| Phi-3.5 mini | 5.00 | 5.00 | 4.43 | 4.26 | 3.64 | 4.47 (4.37 to 4.56) |
| Llama 3.2 3B | 4.95 | 4.93 | 4.38 | 4.40 | 3.60 | 4.45 (4.25 to 4.61) |
| Gemma 2 2B | 5.00 | 5.00 | 4.60 | 4.36 | 3.64 | 4.52 (4.44 to 4.60) |
| Gemma 3 1B | 5.00 | 4.93 | 4.48 | 4.43 | 3.79 | 4.52 (4.42 to 4.62) |
| SmolLM2 1.7B | 5.00 | 5.00 | 4.52 | 4.17 | 3.57 | 4.45 (4.38 to 4.53) |
| Qwen2.5 7B | 5.00 | 5.00 | 4.67 | 4.36 | 3.57 | 4.52 (4.41 to 4.63) |
| Qwen2.5 3B | 5.00 | 5.00 | 4.69 | 4.31 | 3.57 | 4.51 (4.43 to 4.60) |
| Qwen2.5 1.5B | 4.95 | 4.93 | 4.57 | 4.21 | 3.31 | 4.40 (4.25 to 4.51) |
| Qwen3 0.6B (non-thinking) | 4.98 | 4.95 | 4.64 | 4.05 | 3.29 | 4.38 (4.26 to 4.50) |
| Qwen2.5 0.5B | 4.93 | 4.90 | 4.38 | 3.98 | 3.19 | 4.28 (4.09 to 4.41) |

## Head-to-head grid

Table: Row model's wins - ties - losses against each original model, over 14 prompts (a prompt is a tie when the two answer orders disagree)

|  | Phi-3.5 mini | Llama 3.2 3B | Gemma 2 2B | SmolLM2 1.7B | Qwen2.5 7B | Qwen2.5 3B | Qwen2.5 1.5B | Qwen2.5 0.5B |
|---|---|---|---|---|---|---|---|---|
| Llama 3.2 1B | 4-6-4 | 1-10-3 | 6-5-3 | 7-6-1 | 4-10-0 | 8-3-3 | 12-0-2 | 12-2-0 |
| Phi-3.5 mini | - | 2-10-2 | 3-9-2 | 4-8-2 | 4-8-2 | 6-8-0 | 9-4-1 | 10-4-0 |
| Llama 3.2 3B | 2-10-2 | - | 5-6-3 | 3-9-2 | 3-8-3 | 10-3-1 | 7-7-0 | 9-4-1 |
| Gemma 2 2B | 2-9-3 | 3-6-5 | - | 6-7-1 | 3-9-2 | 5-7-2 | 9-4-1 | 11-3-0 |
| Gemma 3 1B | 2-6-6 | 2-8-4 | 4-9-1 | 2-7-5 | 3-7-4 | 8-5-1 | 9-5-0 | 9-3-2 |
| SmolLM2 1.7B | 2-8-4 | 2-9-3 | 1-7-6 | - | 5-6-3 | 6-6-2 | 7-5-2 | 9-5-0 |
| Qwen2.5 7B | 2-8-4 | 3-8-3 | 2-9-3 | 3-6-5 | - | 5-8-1 | 7-6-1 | 7-5-2 |
| Qwen2.5 3B | 0-8-6 | 1-3-10 | 2-7-5 | 2-6-6 | 1-8-5 | - | 4-9-1 | 11-3-0 |
| Qwen2.5 1.5B | 1-4-9 | 0-7-7 | 1-4-9 | 2-5-7 | 1-6-7 | 1-9-4 | - | 8-4-2 |
| Qwen3 0.6B (non-thinking) | 0-4-10 | 0-4-10 | 1-4-9 | 1-4-9 | 2-4-8 | 2-4-8 | 4-4-6 | 6-4-4 |
| Qwen2.5 0.5B | 0-4-10 | 1-4-9 | 0-3-11 | 0-5-9 | 2-5-7 | 0-3-11 | 2-4-8 | - |

Pair pages exist for 28 of the 52 judged pairs (the rest are in the grid and raw data). Open one for the per-prompt verdicts and example outputs side by side: [Qwen2.5 0.5B vs Qwen2.5 1.5B](https://theagenticweekly.com/tests/writing-qwen2-5-0-5b-vs-qwen2-5-1-5b); [Qwen2.5 0.5B vs Qwen2.5 3B](https://theagenticweekly.com/tests/writing-qwen2-5-0-5b-vs-qwen2-5-3b); [Qwen2.5 0.5B vs Qwen2.5 7B](https://theagenticweekly.com/tests/writing-qwen2-5-0-5b-vs-qwen2-5-7b); [Qwen2.5 0.5B vs Llama 3.2 3B](https://theagenticweekly.com/tests/writing-qwen2-5-0-5b-vs-llama3-2-3b); [Qwen2.5 0.5B vs Gemma 2 2B](https://theagenticweekly.com/tests/writing-qwen2-5-0-5b-vs-gemma2-2b); [Qwen2.5 0.5B vs Phi-3.5 mini](https://theagenticweekly.com/tests/writing-qwen2-5-0-5b-vs-phi3-5-mini); [Qwen2.5 0.5B vs SmolLM2 1.7B](https://theagenticweekly.com/tests/writing-qwen2-5-0-5b-vs-smollm2-1-7b); [Qwen2.5 1.5B vs Qwen2.5 3B](https://theagenticweekly.com/tests/writing-qwen2-5-1-5b-vs-qwen2-5-3b); [Qwen2.5 1.5B vs Qwen2.5 7B](https://theagenticweekly.com/tests/writing-qwen2-5-1-5b-vs-qwen2-5-7b); [Qwen2.5 1.5B vs Llama 3.2 3B](https://theagenticweekly.com/tests/writing-qwen2-5-1-5b-vs-llama3-2-3b); [Qwen2.5 1.5B vs Gemma 2 2B](https://theagenticweekly.com/tests/writing-qwen2-5-1-5b-vs-gemma2-2b); [Qwen2.5 1.5B vs Phi-3.5 mini](https://theagenticweekly.com/tests/writing-qwen2-5-1-5b-vs-phi3-5-mini); [Qwen2.5 1.5B vs SmolLM2 1.7B](https://theagenticweekly.com/tests/writing-qwen2-5-1-5b-vs-smollm2-1-7b); [Qwen2.5 3B vs Qwen2.5 7B](https://theagenticweekly.com/tests/writing-qwen2-5-3b-vs-qwen2-5-7b); [Qwen2.5 3B vs Llama 3.2 3B](https://theagenticweekly.com/tests/writing-qwen2-5-3b-vs-llama3-2-3b); [Qwen2.5 3B vs Gemma 2 2B](https://theagenticweekly.com/tests/writing-qwen2-5-3b-vs-gemma2-2b); [Qwen2.5 3B vs Phi-3.5 mini](https://theagenticweekly.com/tests/writing-qwen2-5-3b-vs-phi3-5-mini); [Qwen2.5 3B vs SmolLM2 1.7B](https://theagenticweekly.com/tests/writing-qwen2-5-3b-vs-smollm2-1-7b); [Qwen2.5 7B vs Llama 3.2 3B](https://theagenticweekly.com/tests/writing-qwen2-5-7b-vs-llama3-2-3b); [Qwen2.5 7B vs Gemma 2 2B](https://theagenticweekly.com/tests/writing-qwen2-5-7b-vs-gemma2-2b); [Qwen2.5 7B vs Phi-3.5 mini](https://theagenticweekly.com/tests/writing-qwen2-5-7b-vs-phi3-5-mini); [Qwen2.5 7B vs SmolLM2 1.7B](https://theagenticweekly.com/tests/writing-qwen2-5-7b-vs-smollm2-1-7b); [Llama 3.2 3B vs Gemma 2 2B](https://theagenticweekly.com/tests/writing-llama3-2-3b-vs-gemma2-2b); [Llama 3.2 3B vs Phi-3.5 mini](https://theagenticweekly.com/tests/writing-llama3-2-3b-vs-phi3-5-mini); [Llama 3.2 3B vs SmolLM2 1.7B](https://theagenticweekly.com/tests/writing-llama3-2-3b-vs-smollm2-1-7b); [Gemma 2 2B vs Phi-3.5 mini](https://theagenticweekly.com/tests/writing-gemma2-2b-vs-phi3-5-mini); [Gemma 2 2B vs SmolLM2 1.7B](https://theagenticweekly.com/tests/writing-gemma2-2b-vs-smollm2-1-7b); [Phi-3.5 mini vs SmolLM2 1.7B](https://theagenticweekly.com/tests/writing-phi3-5-mini-vs-smollm2-1-7b).

## Speed and length

Table: Generation speed on this CPU-only machine (indicative; machine load varied)

| Model | Median tokens/s | Median seconds per answer | Median words | Load average during runs |
|---|---|---|---|---|
| Llama 3.2 1B | 21.9 | 3.34 | 53 | 9.7 to 15.0 |
| Phi-3.5 mini | 13.7 | 5.98 | 51 | 8.3 to 12.2 |
| Llama 3.2 3B | 21.4 | 3.65 | 52.5 | 8.8 to 12.4 |
| Gemma 2 2B | 22.5 | 2.51 | 35.5 | 8.1 to 9.1 |
| Gemma 3 1B | 13.4 | 3.86 | 46 | 12.4 to 92.2 |
| SmolLM2 1.7B | 20.9 | 3.93 | 48 | 11.5 to 23.9 |
| Qwen2.5 7B | 5.9 | 8.30 | 35.5 | 9.7 to 15.1 |
| Qwen2.5 3B | 14.4 | 3.03 | 28 | 10.6 to 13.3 |
| Qwen2.5 1.5B | 32.3 | 1.75 | 32 | 11.0 to 13.2 |
| Qwen3 0.6B (non-thinking) | 22.5 | 1.84 | 27.5 | 9.9 to 67.2 |
| Qwen2.5 0.5B | 56.8 | 0.67 | 28.5 | 12.9 to 15.5 |

## Judge checks

- What this means: the 7B rater is a weak judge. It changed its mind when we swapped the answer order in 41% of comparisons, it leans towards longer answers, and its rubric accuracy score did not separate outputs that broke the stated rule from those that kept it, so we treat the pairwise ranking as indicative and show the code-checked format column next to it. The format column does not depend on any model.

630 writing outputs (15 models x 14 prompts x 3 runs), 630 single-answer rubric ratings, 1456 blind pairwise judgments (728 head-to-head games after combining the two answer orders).

## Limitations

- Every writer is a small open-weight model (0.5B to 8B parameters, mostly Q4_K_M quantization) run locally on a CPU. We have no API access to frontier models, so ChatGPT, Claude, Gemini, Grok and similar products are not in this test, and nothing here says how they compare.
- The rater is itself a 7B model, so it is a noisy judge. We published its exact prompts and both the pairwise and rubric results. It was not a writer, so it never rated its own text, but it may share blind spots with writers from other families.
- 14 prompts and 3 sampled runs each is a small sample. The leaderboard intervals reflect prompt-to-prompt variation only; they do not cover rater error.
- Pairwise games use one sampled run per prompt and pair (the same run index for both writers), judged in both orders. The rubric ratings cover every output.
- Sentence counts use a simple splitter (a full stop, question mark or exclamation mark followed by a space or the end) and can be off for abbreviations; our prompts avoid them.
- Speed figures (tokens per second) were measured on a shared machine whose load average ranged widely during the run, so they are indicative only. Expect different numbers on other hardware.
- Prompts are in English and short. Nothing here measures long-form writing, factual recall, safety or multilingual ability.

## Related

- [Writing test hub](https://theagenticweekly.com/tests/writing)
- [Writing leaderboard](https://theagenticweekly.com/tests/writing-leaderboard)
- [Instruction-following test](https://theagenticweekly.com/tests/instruction-following)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)

## How to cite

Agentic Weekly, "Writing leaderboard: fifteen small AI models, scores by criterion", last updated 2026-10-26, https://theagenticweekly.com/tests/writing-leaderboard