Reported News
llama.cpp v0.9.0 adds FP8 E4M3 quantization, Mistral Large 4 and Chatterbox TTS
llama.cpp v0.9.0, released November 4, 2026, adds an OCP FP8 E4M3 quantization type, support for four new architectures (Clover, Limite, Metis-1.6 and Mistral Large 4), Chatterbox TTS and Nemotron 3.5 Transcribe, and updates its ggml library to v0.29.0.
- Changes reported
- 6
- listed below, each from a source we read
- We tested
- Not tested yet
- reported from sources; we did not run it ourselves
- Gaps we flag
- 3
- things we could not verify
- Sources
- 2
- 1 primary, linked below
What changed
| # | What the source says | Status |
|---|---|---|
| 01 | Quantization: a new OCP FP8 E4M3 tensor type (saturating mode) with conversion helpers and a new file type, exposed in llama.h as LLAMA_FTYPE_MOSTLY_F8_E4M3. | Reported |
| 02 | A new flash_attn_ext_rows op makes the unified KV cache efficient for multi-sequence inference; the release notes say it restores non-unified KV prefill performance. | Reported |
| 03 | Four new architectures: the Clover-35B-A3B-Preview diffusion model (AR mode), Limite 1B, Metis-1.6 (Praxis ~2.9B and Logos ~10.2B, hybrid Mamba-2 and attention) and Mistral Large 4. | Reported |
| 04 | Audio: Chatterbox TTS support, Nemotron 3.5 Transcribe for offline transcription in 40 languages with speaker attribution, and predefined Pocket TTS voices; the server's GET /props now lists a model's predefined voices. | Reported |
| 05 | The bundled ggml library moves to v0.29.0, which adds initial multi-node tensor parallelism over NCCL plus backend work across CPU, CUDA, SYCL, Metal, Vulkan, OpenCL and Hexagon. | Reported |
| 06 | Gemma 4 E2B and E4B get graph optimizations the notes describe as massive prefill gains, and the web UI adds MCP group checkboxes in its tools settings. | Reported |
reported by the sourcemeasured or run by usnot verified
Primary source: llama.cpp v0.9.0 release notes (GitHub). We are the fast, independent index; the vendor’s own page is the source of record for this release.
Who should care
Anyone serving several sequences at once from one llama.cpp server should look at the unified KV change, and anyone who wants FP8 weights in GGUF form now has a file type for it. People who want to run Mistral Large 4, Chatterbox TTS or Nemotron 3.5 Transcribe locally get first support in this release. Embedders calling llama.h or mtmd.h directly should read the API changes list, which adds new enums and voice functions. If you only run small chat models through a wrapper such as Ollama, nothing here needs action until your wrapper picks up the new version.
In the source’s words
“This release adds OCP FP8 E4M3 quantization support”
“Four new architectures are supported (Clover, Limite, Metis-1.6 and Mistral Large 4)”
What we have not verified
- We did not run this release, so we cannot confirm the prefill gains or multi-sequence speedups the notes describe; no figures are given for them.
- FP8 E4M3 model quality and memory savings compared with existing GGUF quant types are not stated in the release notes.
- Which hardware backends accelerate the new FP8 type end to end is not spelled out in the llama.cpp notes.
This brief is Reported: written from the sources below, not from running the release ourselves.
Sources
- llama.cpp v0.9.0 release notes (GitHub)primary source · fetched 2026-11-04
- ggml v0.29.0 release notes (GitHub)source · fetched 2026-11-04
Related
Spotted an error? See our corrections policy or email the address on the contact page. Markdown version for agents: /news/llama-cpp-v0-9-0-fp8-mistral-large-4.md
Want these briefs in your Google Top Stories? Add Agentic Weekly as a preferred source on Google.
Newsletter
The Weekly Verdict. Free, every Monday.
A free email every Monday: the week's few big AI releases, then 5 short use cases for the new models, each with a prompt you can paste.
Free. Double opt-in. One click to unsubscribe.