# Transformers v5.21 adds Pocket TTS, Flint and MiniCPM-OCR, and changes tensor-parallel plans

Canonical: https://theagenticweekly.com/news/transformers-v5-21-pocket-tts-flint-breaking-changes · Evidence: Reported · First seen: 2026-10-26 18:36 UTC · Source published: 2026-10-26 18:17 UTC · Brief published: 2026-10-27 12:05 UTC · Updated: 2026-10-27 12:05 UTC · Author: Agentic Weekly

> Transformers v5.21.0 (October 26, 2026) adds MiniCPM-OCR, Flint, UltraVBert, Gradium Phonon TTS, Pocket TTS and its audio tokenizer, an fp8 QuantizedCache backend, and breaking changes to how tensor parallelism shards embeddings and MoE experts.

## What changed

- New model support: MiniCPM-OCR (document reading), Flint (a mixture-of-experts language model that routes each token to 6 of 256 experts), UltraVBert (visual document retrieval), Gradium Phonon TTS, and Kyutai's 100M-parameter Pocket TTS with its audio tokenizer.
- Breaking: tensor parallelism gives embeddings their own style and shards input embeddings by default, so ColwiseParallel and RowwiseParallel now apply only to linear layers; custom TP plans that used them on embeddings must switch.
- Breaking: every MoE tensor-parallel and expert-parallel plan now shards its experts, and custom or downstream MoE plans must add expert sharding to pass the new test.
- QuantizedCache gains an "fp8" backend that stores KV states in torch.float8_e4m3fn with one scale per tensor, enabled with cache_implementation="quantized" and cache_config={"backend": "fp8"}.
- Cache fixes: the notes say Flash Attention now works with static caches, BLOOM no longer crashes with static cache or compiled generate, and chunked prefill supports cache continuation.

## In the source's words

> "Flash Attention now works with static caches, BLOOM no longer crashes with static cache or compiled generate, chunked prefill supports cache continuation"
> (Transformers v5.21.0 release notes, https://github.com/huggingface/transformers/releases/tag/v5.21)

## Early verdict

Anyone with custom tensor-parallel or MoE parallel plans should review them before upgrading; others mainly get new models and cache fixes. (Basis: Based on the breaking-changes section of the release notes; we have not run v5.21 or any of the new models.)

## What we have not verified

- We did not install v5.21.0 or run any of the newly supported models.
- Model descriptions are the release notes' own wording; we did not check them against the model cards or papers.
- We did not measure the fp8 KV cache's memory savings or accuracy.

## Timeline

- Source published: 2026-10-26 18:17 UTC
- First seen by our watcher: 2026-10-26 18:36 UTC (via GitHub releases: huggingface/transformers), 18 minutes after the source's timestamp
- Brief published: 2026-10-27 12:05 UTC

## Sources

- [huggingface/transformers v5.21.0 release notes (GitHub)](https://github.com/huggingface/transformers/releases/tag/v5.21) (primary source) (fetched 2026-10-27)

## Related

- [Hugging Face: verdict and setup guide](https://theagenticweekly.com/tools/hugging-face)
- [Alternatives to Hugging Face](https://theagenticweekly.com/alternatives/hugging-face)
- [All news briefs](https://theagenticweekly.com/news)
- [This week's releases](https://theagenticweekly.com/this-week)
- [New AI models this week](https://theagenticweekly.com/models/this-week)

## How to cite

Agentic Weekly, "Transformers v5.21 adds Pocket TTS, Flint and MiniCPM-OCR, and changes tensor-parallel plans", published 2026-10-27 12:05 UTC, https://theagenticweekly.com/news/transformers-v5-21-pocket-tts-flint-breaking-changes