Reported News
Together Compiler: agent-driven continual compilation for production inference
Together AI announced Together Compiler on November 4, 2026: agents propose GPU forward-pass changes that a persistent engine verifies before shipping. Compiled kernels are on by default for supported serverless models at the same price; Together claims 1.3 to 3x Blackwell throughput vs the best open engines after under 24 hours of compilation.
- Changes reported
- 6
- listed below, each from a source we read
- We tested
- Not tested yet
- reported from sources; we did not run it ourselves
- Gaps we flag
- 4
- things we could not verify
- Sources
- 2
- 2 primary, linked below
What changed
| # | What the source says | Status |
|---|---|---|
| 01 | Launch: Together Compiler is announced as a production agentic optimization loop that treats inference tuning as a search problem: agents propose forward-pass changes, and a persistent engine hot-reloads and measures them for speed and correctness before anything ships. | Reported |
| 02 | Availability: compiled kernels are on by default for supported serverless models at the same price, with no flag to set. Together lists GLM 5.3, GLM 5.3 Flash and DeepSeek V4.1 Flash on serverless today, with more to follow. | Reported |
| 03 | Dedicated path: Dedicated Model Inference and Dedicated Container Inference customers can ask Together to apply the compiler to their endpoint; self-serve optimization for dedicated workloads is described as coming, with limited beta access mentioned on the research blog. | Reported |
| 04 | Claimed gains: on NVIDIA Blackwell, Together says the compiler reaches 1.3 to 3x higher throughput than the best open engines in its benchmarks after under 24 hours of compilation on two nodes per model, and that at matched per-user speed the compiled engine serves up to 2.6x tokens per GPU. | Reported |
| 05 | Scope: agents optimize the whole forward pass (including vendor kernels), keep a shared champion registry of winning kernels, and are pitched as GPU-agnostic; the product page also shows AMD MI355X and AWS Trainium2 results. | Reported |
| 06 | Framing: Together positions the compiler as the first step of Polymorphic Inference, arguing static one-config engines cannot keep up when 80% of OpenRouter traffic goes to models released in the last three months. | Reported |
reported by the sourcemeasured or run by usnot verified
Primary source: Together AI research blog: From Static to Polymorphic Inference. We are the fast, independent index; the vendor’s own page is the source of record for this release.
In the source’s words
“Compiled kernels are on by default for supported serverless models, at the same price.”
“On NVIDIA Blackwell GPUs, Together Compiler reaches 1.3 to 3x higher throughput than the best open engines in our benchmarks, after under 24 hours of compilation.”
“Together Compiler is running in production today on some of our most popular serverless endpoints, where it has found further gains on top of already highly optimized code.”
What we have not verified
- We did not run Together Compiler or reproduce the 1.3–3x / 2.6x throughput claims; those are Together's own benchmarks against open engines and InferenceX.
- The research blog gives only the calendar date (November 4, 2026), so the exact release time is unknown.
- Which serverless endpoints beyond GLM 5.3, GLM 5.3 Flash and DeepSeek V4.1 Flash already run compiled kernels is not fully listed.
- Limited-beta and self-serve timelines for dedicated deployments are marketing language; we did not confirm access terms or pricing changes beyond "same price" for supported serverless.
This brief is Reported: written from the sources below, not from running the release ourselves.
Sources
- Together AI research blog: From Static to Polymorphic Inferenceprimary source · fetched 2026-11-05
- Together AI: Polymorphic Inference / Together Compiler product pageprimary source · fetched 2026-11-05
Related
Spotted an error? See our corrections policy or email the address on the contact page. Markdown version for agents: /news/together-compiler-continual-compilation.md
Want these briefs in your Google Top Stories? Add Agentic Weekly as a preferred source on Google.
Newsletter
The Weekly Verdict. Free, every Monday.
A free email every Monday: the week's few big AI releases, then 5 short use cases for the new models, each with a prompt you can paste.
Free. Double opt-in. One click to unsubscribe.