# smolagents 1.26.0: our test log

Canonical: https://theagenticweekly.com/tests/smolagents · Last verified: 2026-10-25 · Published: 2026-10-25 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Tested by us on 2026-10-25. Local open-weight model, not the vendor's hosted models. A smoke test on a CPU-only box, not a benchmark.

> smolagents 1.26.0 passed our one-tool task on a local 3B model: 6 of 6 trials; cold uv install 1.5 s.

## Result at a glance

Table: smolagents test record

| Field | Value |
|---|---|
| Result | Passed |
| Version tested | 1.26.0 |
| Date and time | 2026-10-25, 23:18 to 23:19 CDT (America/Chicago) |
| Task | Give a local model one Python function, add_numbers(a, b), and ask it to use that tool to compute 17 + 25 and report the result. Pass means the function was called with 17 and 25 and the final answer contains 42. Three trials per tool, all on the same local model. |
| Model used | Qwen2.5-3B-Instruct (Q4_K_M) through Ollama 0.40.11 |
| Hardware | Linux x86_64 virtual machine (Debian 13), 8 vCPU (Intel Xeon, 1 thread per core), 15 GiB RAM, no GPU |
| Installed size (uv) | 54 MB in 28 packages |
| Install time (uv, cold) | 1.5 s |
| Install time (pip, cold) | 5.6 s |
| Installed size (pip) | 85 MB in 29 packages |
| Task trials passed | 6 of 6 |
| Task wall time (median, includes imports) | 8.2 s (range 4.2 to 11.9 s) |
| Machine load during trials | 10.3 to 11.2 (1-minute load average, 8 vCPU) |

## What we found

- 6 of 6 task trials passed: the model's tool call reached the framework, the function ran with 17 and 25, and the final answer contained 42.
- Cold install took 1.5 s with uv and 5.6 s with pip, and left 54 MB in 28 packages (uv).
- We ran both agent styles: ToolCallingAgent (the model emits JSON tool calls) and CodeAgent (the model writes Python that calls the tool). Per-style pass counts are in the step table.
- CodeAgent executes model-written Python in a local interpreter; we did not test its sandbox options.

## What we ran, step by step

Table: Commands run, wall-clock time and exit code

| Step | Wall time | Exit code | Notes |
|---|---|---|---|
| Install with uv (cold, no cache) | 1.5 s | 0 | Machine load average before the run: 0.59. |
| Install with pip (cold, no cache) | 5.6 s | 0 | Load average before: 1.35. Second run; the first pip run was discarded (see the research page). |
| Task command (all trials) | n/a | n/a | Run from a directory that contains common.py (shown below). Telemetry switches were set for tools that have them. |
| Task trial 1 (smolagents-toolcalling) | 6.6 s | 0 | Tool called with [[17, 25]]; answer: The result of adding 17 and 25 is 42. Load 11.20. |
| Task trial 2 (smolagents-toolcalling) | 5.2 s | 0 | Tool called with [[17, 25]]; answer: The result of adding 17 and 25 is 42. Load 10.94. |
| Task trial 3 (smolagents-toolcalling) | 4.2 s | 0 | Tool called with [[17, 25]]; answer: 42 Load 10.71. |
| Task trial 1 (smolagents-code) | 9.9 s | 0 | Tool called with [[17, 25]]; answer: The result of 17 + 25 is: 42 Load 10.89. |
| Task trial 2 (smolagents-code) | 9.9 s | 0 | Tool called with [[17, 25]]; answer: 42 Load 10.59. |
| Task trial 3 (smolagents-code) | 11.9 s | 0 | Tool called with [[17, 25]]; answer: 42 Load 10.27. |

Exact commands (paths shortened)

```
# Install with uv (cold, no cache)
uv venv -q -p 3.13 work/py/smolagents && VIRTUAL_ENV=work/py/smolagents uv pip install --no-cache -q smolagents

# Install with pip (cold, no cache)
uv venv -q --seed -p 3.13 work/pip/smolagents && work/pip/smolagents/bin/pip install --no-cache-dir -q smolagents

# Task command (all trials)
OTEL_SDK_DISABLED=true work/py/smolagents/bin/python tasks/smolagents_agent.py
```

## Raw output (excerpts)

tasks/smolagents_agent.py, the whole script for this test

```
from common import *
from smolagents import ToolCallingAgent, OpenAIServerModel, tool
@tool
def add_numbers_sm(a: int, b: int) -> int:
    """Add two integers and return the sum.

    Args:
        a: first integer
        b: second integer
    """
    return add_numbers(a, b)
t = add_numbers_sm
agent = ToolCallingAgent(tools=[t], model=OpenAIServerModel(model_id=MODEL, api_base=BASE, api_key=KEY), verbosity_level=0, max_steps=4)
report(agent.run(TASK))
```

common.py, the shared helper every framework script imports

```
import json, os
BASE = "http://127.0.0.1:11434/v1"
MODEL = os.environ.get("MODEL", "qwen2.5-3b")
KEY = "ollama"
TASK = "Use the add_numbers tool to compute 17 + 25, then tell me the result."
CALLS = []
def add_numbers(a: int, b: int) -> int:
    """Add two integers and return the sum."""
    CALLS.append((a, b))
    return int(a) + int(b)
def report(final):
    ok = any({int(x), int(y)} == {17, 25} for x, y in CALLS) and "42" in str(final)
    print("RESULT_JSON " + json.dumps({"calls": CALLS, "final": str(final)[:300], "ok": ok}))
```

Result line printed by trial 1 (2026-10-25 23:18:33 CDT)

```
RESULT_JSON {"calls": [[17, 25]], "final": "The result of adding 17 and 25 is 42.", "ok": true}
```

tasks/smolagents_code.py, the CodeAgent variant

```
from common import *
from common import add_numbers as _add
from smolagents import CodeAgent, OpenAIServerModel, tool
@tool
def add_numbers(a: int, b: int) -> int:
    """Add two integers and return the sum.

    Args:
        a: first integer
        b: second integer
    """
    return _add(a, b)
t = add_numbers
agent = CodeAgent(tools=[t], model=OpenAIServerModel(model_id=MODEL, api_base=BASE, api_key=KEY), verbosity_level=0, max_steps=4)
report(agent.run(TASK))
```

## Gotchas and things that broke

- The tool function needs a docstring that describes every argument, or smolagents raises DocstringParsingException (Cannot generate JSON schema ... the docstring has no description for the argument) at definition time.
- On a bare `pip install smolagents`, constructing OpenAIServerModel raises ModuleNotFoundError: Please install 'openai' extra (smolagents[openai]). The install size and time above are for the bare package; we did not time the extra separately.

## What we did not test

- Any vendor-hosted model: everything ran on a local 3B model, so tool-call reliability with frontier models is not measured here.
- Multi-agent flows, memory, long conversations, streaming, tracing and cloud deployment.
- Anything beyond one tool and one question; three trials is a smoke test, not a benchmark.
- The CodeAgent sandboxes (Docker, E2B, Modal) and the hub tools.

## Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

```
# 1. Serve any local model that supports tool calling behind an OpenAI-compatible endpoint, registered as qwen2.5-3b:
#    http://127.0.0.1:11434/v1  (we used Ollama with a Qwen2.5-3B-Instruct Q4_K_M GGUF)
uv venv -p 3.13 .venv && VIRTUAL_ENV=.venv uv pip install smolagents
# 2. Save common.py (above) and the task script next to each other, then:
.venv/bin/python smolagents_agent.py
```

## How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the [testing rig](https://theagenticweekly.com/methodology#testing-rig) for the hardware, the local model and the rules we follow, and the [test hub](https://theagenticweekly.com/tests) for every other test.

## FAQ

### Does smolagents work with a local model?

In our run it did for this one task: 6 of 6 trials passed against a local Qwen2.5-3B model served by Ollama. We did not test other models, and results depend heavily on the model's tool-calling ability.

### How big is a fresh smolagents install?

54 MB in 28 packages with uv on 1.26.0, measured on 2026-10-25.

## Sources

- [smolagents repository](https://github.com/huggingface/smolagents) (fetched 2026-10-25)

## Related

- [All test logs](https://theagenticweekly.com/tests)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)
- [Study: install size and first-task results across 16 frameworks](https://theagenticweekly.com/research/agent-framework-install-and-first-task)
- [smolagents facts and sources](https://theagenticweekly.com/tools/smolagents)

## How to cite

Agentic Weekly, "smolagents 1.26.0: our test log", last updated 2026-10-25, https://theagenticweekly.com/tests/smolagents