# Install size and a first tool-calling task across 16 agent frameworks: what we measured

Canonical: https://theagenticweekly.com/research/agent-framework-install-and-first-task · Last verified: 2026-10-25 · Published: 2026-10-25 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Measured by us on 2026-10-25. Local open-weight models on a CPU-only box; nothing is vendor-reported. Smoke tests, not benchmarks.

> We installed 16 Python agent frameworks from an empty cache on one 8-core CPU box. With uv the installs took 0.8 to 8.4 s and used 26 MB (openai-python) to 819 MB (crewai); 15 of 16 passed a one-tool task 3 of 3 with a local Qwen2.5-3B model and one (Semantic Kernel) passed only after a prompt change. In JavaScript, two frameworks looked unreliable until we pinned the sampling temperature.

## What we did

For each framework we created an empty environment, installed the package with no cache, recorded the time, the disk size and the number of installed distributions, and then ran the same task: the model is given one tool, add_numbers(a, b), and asked to compute 17 + 25 and report the result. A trial passes only if the function was called with 17 and 25 and the answer contains 42. There are three trials per framework, with temperature 0 wherever the framework exposes it. The model is Qwen2.5-3B-Instruct (Q4_K_M) served by Ollama 0.40.11 through its OpenAI-compatible API.

All numbers come from one Linux virtual machine (Debian 13, 8 vCPU Intel Xeon, 15 GiB RAM, no GPU) that also ran other people's jobs, so timings are indicative and the 1-minute load average is recorded next to each run in the underlying test logs.

This is a smoke test of installing and wiring a tool, not a benchmark of agent quality. A 3B local model says little about how any framework behaves with a frontier model, a long task or a team deployment. Each framework has its own test log with the exact script, command and output.

## Python frameworks: install and first task

Table: Cold installs (uv and pip, no cache) and the one-tool task, sorted by uv install time

| Framework | Version | uv install (s) | Size (MB) | Packages | pip install (s) | Task passes | Task median (s) |
|---|---|---|---|---|---|---|---|
| [langchain](https://theagenticweekly.com/tests/langchain) | 1.4.7 | 0.8 | 77 | 44 | 13.3 | 3 of 3 | 3.4 |
| [openai-python](https://theagenticweekly.com/tests/openai-python) | 3.37.0 | 1.1 | 26 | 14 | 7.2 | 3 of 3 | 2.8 |
| [instructor](https://theagenticweekly.com/tests/instructor) | 1.17.0 | 1.4 | 49 | 41 | 7.3 | 3 of 3 | 3.7 |
| [smolagents](https://theagenticweekly.com/tests/smolagents) | 1.26.0 | 1.5 | 54 | 28 | 5.6 | 6 of 6 | 8.2 |
| [openai-agents-sdk](https://theagenticweekly.com/tests/openai-agents-sdk) | 0.24.0 | 1.7 | 57 | 38 | 11.3 | 3 of 3 | 4.9 |
| [google-adk](https://theagenticweekly.com/tests/google-adk) | 2.13.0 | 1.7 | 71 | 48 | 13.2 | 3 of 3 | 7.6 |
| [agno](https://theagenticweekly.com/tests/agno) | 3.1.10 | 1.8 | 69 | 37 | 13.2 | 3 of 3 | 3.9 |
| [autogen](https://theagenticweekly.com/tests/autogen) | 0.7.5 | 1.9 | 59 | 28 | 4.8 | 3 of 3 | 4.7 |
| [haystack](https://theagenticweekly.com/tests/haystack) | 3.4.1 | 2.3 | 111 | 43 | 8.4 | 3 of 3 | 4.0 |
| [litellm](https://theagenticweekly.com/tests/litellm) | 1.106.4 | 2.3 | 225 | 58 | 14.1 | 3 of 3 | 6.7 |
| [langgraph](https://theagenticweekly.com/tests/langgraph) | 1.2.17 | 2.5 | 76 | 43 | 8.1 | 3 of 3 | 3.2 |
| [llamaindex](https://theagenticweekly.com/tests/llamaindex) | 0.14.25 | 3.4 | 193 | 69 | 12.2 | 3 of 3 | 5.9 |
| [pydantic-ai](https://theagenticweekly.com/tests/pydantic-ai) | 2.61.0 | 3.7 | 141 | 102 | 25.9 | 3 of 3 | 5.7 |
| [dspy](https://theagenticweekly.com/tests/dspy) | 3.4.1 | 3.7 | 231 | 66 | 23.3 | 3 of 3 | 23.7 |
| [semantic-kernel](https://theagenticweekly.com/tests/semantic-kernel) | 1.45.0 | 4.6 | 381 | 95 | 24.1 | 0 of 3 | 3.5 |
| [crewai](https://theagenticweekly.com/tests/crewai) | 1.15.30 | 8.4 | 819 | 137 | 32.2 | 3 of 3 | 5.9 |

Install columns are for the bare package named in each test log. Sizes are the on-disk size of the virtual environment after installing. The task median includes interpreter start and imports. Where a framework needs an extra for local models (for example smolagents), the log says so.

## What stood out

- uv installed 16 of the 16 faster than pip in our runs; the pip-to-uv time ratio ranged from 2.5x to 16.6x. Both were run with caches disabled. This is about install tooling, not about the frameworks.
- Install weight varies by more than 10x: openai-python at 26 MB against crewai at 819 MB (137 packages). Everything else, including LangChain and LangGraph, sat between those.
- 15 of 16 passed 3 of 3. smolagents passed with both of its agent styles (tool-calling and code-writing), and DSPy needed its caches switched off before timings meant anything.
- Semantic Kernel returned an empty reply 3 of 3 times with our standard prompt. We traced it to the plugin-prefixed tool name it sends to the model (math-add_numbers) while the prompt said add_numbers; with a prompt that used the prefixed name it passed 3 of 3. That is a framework-and-small-model interaction, and we label it Partly worked.
- Litellm needed us to install the mcp package ourselves before it would run, because that dependency was not declared by the package version we tested.

## JavaScript frameworks and the temperature trap

Table: Install on Node 20 / Node 24 (npm, empty cache)

| Framework | Resolved version | Install time (s) | Size (MB) | Packages |
|---|---|---|---|---|
| [Vercel AI SDK](https://theagenticweekly.com/tests/vercel-ai-sdk) | 6.0.315 / 7.0.153 | 1.6 / 1.3 | 26 / 27 | 21 / 21 |
| [LangChain.js](https://theagenticweekly.com/tests/langchainjs) | 1.5.19 / 1.5.19 | 3.7 / 4.0 | 101 / 115 | 36 / 36 |
| [OpenAI Agents SDK (JS)](https://theagenticweekly.com/tests/openai-agents-js) | 0.21.0 / 0.21.0 | 2.6 / 2.4 | 84 / 84 | 34 / 34 |
| [Mastra](https://theagenticweekly.com/tests/mastra) | 0.24.9 / 1.79.0 | 9.9 / 5.1 | 236 / 151 | 376 / 196 |

Table: One-tool task: default sampling (Node 20 / Node 24 runs) and after pinning temperature to 0

| Framework | Default sampling, Node 20 | Default sampling, Node 24 | Temperature pinned to 0 |
|---|---|---|---|
| Vercel AI SDK | 3 of 3 | 3 of 3 | not needed (set in the call) |
| LangChain.js | 3 of 3 | 3 of 3 | not needed (set in the call) |
| OpenAI Agents SDK (JS) | 2 of 5 | 1 of 5 | 5 of 5 on both Node versions |
| Mastra | 0 of 3 | 1 of 3 | 5 of 5 on both Node versions |

OpenAI Agents SDK (JS) and Mastra first passed only 1 or 2 times in 3 to 5 trials, always with an empty reply and no tool call. We had not pinned the temperature in those two scripts; the other scripts did. After adding modelSettings with temperature 0, all four combinations (both frameworks, both Node versions) passed 5 of 5. In short, a sampling setting, not the framework, explained the failures. If you test a framework with a small model, pin the temperature before you conclude anything.

## Caveats and a discarded run

- The machine was shared. The load average before individual runs ranged from about 0.5 to 11 across the recorded runs; install and task timings are indicative and the pass or fail results and sizes are the more reliable parts.
- A first pip pass was discarded. Its times were inconsistent with an immediate rerun (for example OpenHands 121.8 s against 56.8 s), consistent with other jobs interfering. We did not isolate the cause. The raw file is kept and the rerun is what this page uses.
- The first full task run was also noisy (load averages 1.3 to 8.5 before its trials) and was re-run; both files are kept in our raw records.
- We did not test hosted models, multi-agent flows, memory, streaming, observability or anything beyond one tool and one question.
- Versions are the latest that installed on the day, on Python 3.13.5 unless a log says otherwise. Some packages resolve to different versions on other Python or Node releases; see the research page on silent downgrades.

## Reproduce it

Per framework (see each test log for the exact script)

```
uv venv -p 3.13 .venv && VIRTUAL_ENV=.venv uv pip install --no-cache <package>
# serve a tool-calling model at http://127.0.0.1:11434/v1 (we used Ollama with qwen2.5-3b)
.venv/bin/python task.py   # each test log shows the full script
```

Every framework named in the tables has a log under /tests with the commands, versions, timings per trial and the raw result lines.

## FAQ

### Which agent framework is the lightest to install?

In our uv measurements the OpenAI Python client with a hand-written loop was the lightest at 26 MB, and among agent frameworks LangChain was small too; CrewAI was the heaviest at 819 MB. See the table for the full list.

### Did every framework work with a local model?

15 of 16 passed a one-tool task 3 of 3 with a local 3B model. Semantic Kernel passed only after the prompt used its plugin-prefixed tool name. This tests wiring, not agent quality.

### Why do the JavaScript results mention temperature?

Two JavaScript scripts did not pin temperature and looked unreliable. With temperature 0 both passed 5 of 5 on Node 20 and Node 24.

## Related

- [All research](https://theagenticweekly.com/research)
- [All test logs](https://theagenticweekly.com/tests)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)
- [Study: install traps and silent version downgrades on older Node](https://theagenticweekly.com/research/older-node-silent-downgrades)

## Changelog

- 2026-10-25: First published.

## How to cite

Agentic Weekly, "Install size and a first tool-calling task across 16 agent frameworks: what we measured", last updated 2026-10-25, https://theagenticweekly.com/research/agent-framework-install-and-first-task