|
Issue 1 · Week of Oct 19 to 26, 2026 · The Weekly Verdict · about 6 min
In this issue: the setting that fixed two frameworks · Anthropic's S-1 · 4 updates to make now · Kimi K3.1 Pro · one MCP tool to try
|
The Verdict
Before you blame your agent framework, pin the temperature.
We gave two JavaScript agent frameworks the same one-tool task. At default settings Mastra passed 1 of 6 trials and the OpenAI Agents SDK for JS passed 3 of 10, every failure an empty reply with no tool call. With temperature set to 0: 10 of 10 and 10 of 10. The framework was never the problem.
Based on: 16 default-setting trials and 20 pinned trials, on Node 20 and Node 24, with a small local 3B model. One task, so a smoke test and not a benchmark.
|
| |
Also today, Oct 26 Reported
Anthropic files its S-1: a Nasdaq listing as ANT, with the price still blank
• Revenue was $16.3 billion for the six months to June 30, 2026, up from $1.2 billion a year earlier. Operating loss was $693 million, down from $2.1 billion. • Price range and share count are still blank. The filing calls Claude Code its clearest example of what it terms Virtual Collaborators.
Our read: Nothing here changes what you should install. Check any figure you see in a headline against the filing.
Basis: We read the cover, summary, risk-factor and business sections; we did not read the whole filing or analyse the financial statements. Not investment advice. Source (Oct 26)
Read our Anthropic IPO hub → Our brief
|
4 Update now | | 3 Worth a look | | 1 Skip | | 57 Tests run so far |
|
Shipped this weekEach release gets one call: update now, worth a look, or skip |
Eight releases, one call each. Reported means we read the source and did not run it; Tested means we ran it, and the Basis line says what we did not try.
| |
Update now MCP reference servers v2.1.0 Reported
Eight vulnerabilities fixed across the filesystem, everything and fetch servers. Date-versioned installs will not be offered the upgrade by your package manager.
Basis: Release notes read. We did not run it, reproduce any vulnerability or read the advisories. Source (Oct 25)
|
| |
Update now Claude Code v2.1.310 Tested
A hardening release: Monitor now works on Bedrock, Vertex and Foundry, and gateway retries are fixed. Update if you use the sandbox or a cloud provider.
Basis: We installed 2.1.310 and ran one headless edit through a local model. We did not test the sandbox, permission fixes or any cloud provider. Source (Oct 23)
|
| |
Update now FastMCP v4.1.3 Tested
Upgrade, then read the security notices: the OAuth issuer protection only works once you set the issuer yourself.
Basis: We installed 4.1.3 and called a two-tool server over stdio, HTTP and in memory. We did not test OAuth issuer settings or the trust-store fix. Source (Oct 20)
|
| |
Update now Open WebUI v0.12.4 Reported
The release notes say it contains security fixes that are not itemised and recommend updating every deployment soon.
Basis: Release notes read. We did not run it, so we cannot say what the fixes cover or how severe they are. Source (Oct 25)
|
| |
Worth a look Ollama v0.40.11 Reported
Speech generation on macOS and the new d3 decision models in 2B, 4B, 9B and 27B sizes.
Basis: Release notes read. We did not run the speech feature or the d3 models, so how well they work is open. Source (Oct 24)
|
| |
Worth a look Replit Monitoring Agent Reported
A new agent that investigates production issues from your app's logs and metrics and tries to fix them. Read the credit and overage rules first.
Basis: Vendor update notes read. We did not run it, and Core's price is not confirmed because the pricing page blocked our fetch. Source (Oct 23)
|
| |
Worth a look Claude Duet research preview Reported
A new app where Claude works as a proactive agent, for Pro and Max only. Treat it as a preview.
Basis: Release notes read. We did not use Duet and cite no benchmarks for it. Source (Oct 21)
|
| |
Skip Gemini CLI v0.65.0 Reported
Skip it as a new starter on a consumer plan: Google says it stopped serving Pro, Ultra and free individual requests on June 18, 2026. Enterprise licenses and paid API keys only.
Basis: Release notes and Google's notice read. We did not run it. Source (Oct 20)
|
Also on our radar Superset is now Paradev Reported the parallel coding-agent app renamed itself, shipped desktop v2.0.0 and says it raised an $11.5M seed. Per the company, accounts and the superset command carry over; we did not install it.
|
The LabWhat we ran ourselves, with the scope and the failures |
Every Tested label links to a page that says what we ran and what we did not. 57 public tests so far, with the failures left in.
Every test we have published: 57 hands-on runs
48 passed 6 partial 2 failed 1 blocked
A pass means the task as we defined it worked, with its scope stated. It is not a benchmark score. Same task, one setting changed: trials passed
One-tool task on a local 3B model, Node 20 and Node 24 combined.
Mastra |
| Default setting | | 1 of 6 |
| Temperature 0 | | 10 of 10 | OpenAI Agents SDK for JS |
| Default setting | | 3 of 10 |
| Temperature 0 | | 10 of 10 |
|
EXACT SCOPE One small local 3B model, one tool-calling task, a handful of trials per setting, on Node 20 and Node 24. It shows a model setting can look like a framework fault. It does not rank the frameworks. Not tested: frontier hosted models; multi-agent flows, memory and streaming; throughput or quality benchmarks. Limit: a local 3B model says little about how these tools behave with frontier hosted models. |
|
Model WatchOne model worth knowing this week, from our /models page |
|
Kimi K3.1 Pro Reported
Moonshot AI
Moonshot's successor to Kimi K3, live in the Kimi apps and the API since Oct 23.
Released Oct 23, 2026 | Context 1,048,576 tokens | Price per 1M $2 in · $10 out | Weights Due by Nov 4 |
Our read: Worth a trial through the API if you run long-context coding agents. Do not build around the weights yet: the license is not published, so we do not call it open-weights until both are out.
Caveat: The 65.7% on Terminal-Bench 4.0 is Moonshot's own run; Artificial Analysis measured 67% in its own and ranked the model 4th on its Intelligence Index (55, behind Claude Opus 5.5 at 58). We did not run the model, and Moonshot does not confirm its input and output modalities.
Sources: Moonshot blog · Pricing · Artificial Analysis · Full model page
|
|
Pick of the weekOne tool, one reason, one thing to try |
| |
MCP Inspector 2.12.0 Tested
The quickest way to see what an MCP server exposes before you wire it into an agent. In our run it listed the filesystem server's 14 tools from the command line in under a second.
Try it in a minute $ npm install @modelcontextprotocol/inspector $ node node_modules/.bin/mcp-inspector --cli node /path/to/server.js --method tools/list |
Gotcha: Run it on Node 22.19 or newer. On Node 20, npm silently installs the old 0.15.0.
Not tested: the browser UI, OAuth flows, SSE and streamable HTTP; only tools/list was run, once on each version. Source · Full entry
|
|
Your moveOne reply makes the next issue better |
Reply with the one tool or model you want us to test next, or one fact we got wrong. It goes to a real inbox, and we fix mistakes and say so. |
One honest note: We make no claim to have been first on any story here. We read the primary source for every fact and link it; where we did not run something, we say so.
Model tracker (88 models from 14 vendors, each dated and sourced) · AI tools cheat sheet (our pick for 8 everyday tasks) · How we label things
Our record so far: 57 public tests, 48 passed, 6 partial, 2 failed, 1 blocked, failures left in.
|