# Research: original measurements from our own test rig

Canonical: https://theagenticweekly.com/research · Last verified: 2026-10-25 · Published: 2026-10-25 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Measured by us on 2026-10-25 on one shared 8 vCPU machine; smoke tests, not benchmarks.

> We publish 5 data studies based on tests we ran ourselves on a CPU-only Linux box: install size and first-task results across 16 agent frameworks, MCP server latency, Ollama against llama.cpp, a voice round trip, and silent npm downgrades. Every number links back to a test log.

## Studies

Table: Studies and their one-line answers

| Study | Answer |
|---|---|
| [Install size and a first tool-calling task across 16 agent frameworks: what we measured](https://theagenticweekly.com/research/agent-framework-install-and-first-task) | We installed 16 Python agent frameworks from an empty cache on one 8-core CPU box. With uv the installs took 0.8 to 8.4 s and used 26 MB (openai-python) to 819 MB (crewai); 15 of 16 passed a one-tool task 3 of 3 with a local Qwen2.5-3B model and one (Semantic Kernel) passed only after a prompt change. In JavaScript, two frameworks looked unreliable until we pinned the sampling temperature. |
| [MCP servers over stdio: startup time, call latency and what broke when we ran 15 of them](https://theagenticweekly.com/research/mcp-servers-latency-and-failures) | Over stdio on one 8-core box, local MCP servers started in about 150 to 540 ms and answered a simple tool call in 0.4 to 3.2 ms at the median; browser-driven servers took tens to hundreds of milliseconds per call and the Context7 lookup took about 1.2 s because it calls a hosted API. Two failures were instructive: the Fetch server refuses localhost by default, and the 2.x reference servers need Node 24 and are silently replaced by 1.x on Node 20. |
| [npm on an older Node installs old versions without telling you: four measured cases and other install traps](https://theagenticweekly.com/research/older-node-silent-downgrades) | Running the same npm install on Node 20.19.2 and Node 24.22.1 gave different major versions of four packages (ai, @mastra/core, the MCP filesystem server and the MCP Inspector) because newer releases declare a higher Node requirement and npm picks the newest version that fits. There was no error. We also hit an undeclared dependency, an unpinned dependency that broke a package, a missing prebuilt wheel, and a sampling setting that looked like a framework failure. |
| [Ollama and llama.cpp on a CPU-only box: three Qwen2.5 sizes, same prompts](https://theagenticweekly.com/research/ollama-vs-llama-cpp-qwen-sizes) | On 8 CPU threads, generation speed was 118 tokens/s for the 0.5B model, 40 for 1.5B and 19.5 for 3B (llama-bench). Ollama and llama-server gave matching pass/fail results on our prompts: the 0.5B model failed JSON extraction, the 1.5B and 3B passed it, and all three called a simple tool correctly 5 of 5 times. Tokens per second through the two servers were similar. |
| [A voice round trip on CPU: Piper speaks, faster-whisper listens, and int8 changes the answer](https://theagenticweekly.com/research/voice-round-trip-piper-faster-whisper) | Piper 1.8.0 turned 10 sentences (41.29 s, 124 words) into audio on CPU in about 1.4 s each. faster-whisper 1.2.1 transcribed it back at 2.4 percent word error with base.en and small.en in float32 on every run, but the int8 setting gave anywhere from 2.4 to 59.7 percent on the same file, run to run. |

## How to use these

Each study states its hardware, versions and what it did not test, and links to the per-tool test logs that hold the commands and raw output. Treat the numbers as measurements of one machine on one day; verify anything that matters on your own hardware.

The full test-log index is at [/tests](https://theagenticweekly.com/tests) and the rig is described in [the methodology](https://theagenticweekly.com/methodology#testing-rig).

## FAQ

### Are these benchmarks?

No. They are small, dated measurements on one shared CPU-only machine with local models, designed to show how things install and fail, not to rank tools.

## Related

- [All test logs](https://theagenticweekly.com/tests)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)

## Changelog

- 2026-10-25: First published.

## How to cite

Agentic Weekly, "Research: original measurements from our own test rig", last updated 2026-10-25, https://theagenticweekly.com/research