# Instructor 1.17.0: our test log

Canonical: https://theagenticweekly.com/tests/instructor · Last verified: 2026-10-25 · Published: 2026-10-25 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Tested by us on 2026-10-25. Local open-weight model, not the vendor's hosted models. A smoke test on a CPU-only box, not a benchmark.

> Instructor 1.17.0 returned a validated Pydantic object from a local 3B model in 3 of 3 trials (structured extraction, not a tool call); cold uv install 1.4 s.

## Result at a glance

Table: Instructor test record

| Field | Value |
|---|---|
| Result | Passed |
| Version tested | 1.17.0 |
| Date and time | 2026-10-25, 23:22 to 23:22 CDT (America/Chicago) |
| Task | Give a local model one Python function, add_numbers(a, b), and ask it to use that tool to compute 17 + 25 and report the result. Pass means the function was called with 17 and 25 and the final answer contains 42. Three trials per tool, all on the same local model. Instructor is not an agent framework, so its task is structured extraction instead: parse 'Order 8841 shipped to Austin on 2026-03-02' into a validated Pydantic object; pass means order 8841, city Austin, date 2026-03-02. |
| Model used | Qwen2.5-3B-Instruct (Q4_K_M) through Ollama 0.40.11 |
| Hardware | Linux x86_64 virtual machine (Debian 13), 8 vCPU (Intel Xeon, 1 thread per core), 15 GiB RAM, no GPU |
| Installed size (uv) | 49 MB in 41 packages |
| Install time (uv, cold) | 1.4 s |
| Install time (pip, cold) | 7.3 s |
| Installed size (pip) | 89 MB in 42 packages |
| Task trials passed | 3 of 3 |
| Task wall time (median, includes imports) | 3.7 s (range 3.3 to 3.8 s) |
| Machine load during trials | 5.3 to 5.9 (1-minute load average, 8 vCPU) |

## What we found

- 3 of 3 task trials passed: the model's tool call reached the framework, the function ran with 17 and 25, and the final answer contained 42.
- Cold install took 1.4 s with uv and 7.3 s with pip, and left 49 MB in 41 packages (uv).
- Different task from the others: structured extraction, not a tool call. We asked for an Order model (order int, city str, date str) from the sentence 'Order 8841 shipped to Austin on 2026-03-02' with JSON mode and max_retries=2.
- Pass here means the validated object equaled order 8841, city Austin, date 2026-03-02.

## What we ran, step by step

Table: Commands run, wall-clock time and exit code

| Step | Wall time | Exit code | Notes |
|---|---|---|---|
| Install with uv (cold, no cache) | 1.4 s | 0 | Machine load average before the run: 0.59. |
| Install with pip (cold, no cache) | 7.3 s | 0 | Load average before: 1.94. Second run; the first pip run was discarded (see the research page). |
| Task command (all trials) | n/a | n/a | Run from a directory that contains common.py (shown below). Telemetry switches were set for tools that have them. |
| Task trial 1 | 3.7 s | 0 | Tool called with []; answer: {"order":8841,"city":"Austin","date":"2026-03-02"} Load 5.28. |
| Task trial 2 | 3.3 s | 0 | Tool called with []; answer: {"order":8841,"city":"Austin","date":"2026-03-02"} Load 5.58. |
| Task trial 3 | 3.8 s | 0 | Tool called with []; answer: {"order":8841,"city":"Austin","date":"2026-03-02"} Load 5.94. |

Exact commands (paths shortened)

```
# Install with uv (cold, no cache)
uv venv -q -p 3.13 work/py/instructor && VIRTUAL_ENV=work/py/instructor uv pip install --no-cache -q instructor

# Install with pip (cold, no cache)
uv venv -q --seed -p 3.13 work/pip/instructor && work/pip/instructor/bin/pip install --no-cache-dir -q instructor

# Task command (all trials)
OTEL_SDK_DISABLED=true work/py/instructor/bin/python tasks/instructor_task.py
```

## Raw output (excerpts)

tasks/instructor_task.py, the whole script for this test

```
import json, instructor
from openai import OpenAI
from pydantic import BaseModel
class Order(BaseModel):
    order: int
    city: str
    date: str
c = instructor.from_openai(OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="ollama"), mode=instructor.Mode.JSON)
o = c.chat.completions.create(model="qwen2.5-3b", response_model=Order, max_retries=2, temperature=0, messages=[{"role":"user","content":"Extract the fields from: Order 8841 shipped to Austin on 2026-03-02"}])
ok = o.order == 8841 and o.city == "Austin" and o.date == "2026-03-02"
print("RESULT_JSON " + json.dumps({"calls": [], "final": o.model_dump_json(), "ok": ok}))
```

common.py, the shared helper every framework script imports

```
import json, os
BASE = "http://127.0.0.1:11434/v1"
MODEL = os.environ.get("MODEL", "qwen2.5-3b")
KEY = "ollama"
TASK = "Use the add_numbers tool to compute 17 + 25, then tell me the result."
CALLS = []
def add_numbers(a: int, b: int) -> int:
    """Add two integers and return the sum."""
    CALLS.append((a, b))
    return int(a) + int(b)
def report(final):
    ok = any({int(x), int(y)} == {17, 25} for x, y in CALLS) and "42" in str(final)
    print("RESULT_JSON " + json.dumps({"calls": CALLS, "final": str(final)[:300], "ok": ok}))
```

Result line printed by trial 1 (2026-10-25 23:22:21 CDT)

```
RESULT_JSON {"calls": [], "final": "{\"order\":8841,\"city\":\"Austin\",\"date\":\"2026-03-02\"}", "ok": true}
```

## What we did not test

- Any vendor-hosted model: everything ran on a local 3B model, so tool-call reliability with frontier models is not measured here.
- Multi-agent flows, memory, long conversations, streaming, tracing and cloud deployment.
- Anything beyond one tool and one question; three trials is a smoke test, not a benchmark.

## Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

```
# 1. Serve any local model that supports tool calling behind an OpenAI-compatible endpoint, registered as qwen2.5-3b:
#    http://127.0.0.1:11434/v1  (we used Ollama with a Qwen2.5-3B-Instruct Q4_K_M GGUF)
uv venv -p 3.13 .venv && VIRTUAL_ENV=.venv uv pip install instructor
# 2. Save common.py (above) and the task script next to each other, then:
.venv/bin/python instructor_task.py
```

## How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the [testing rig](https://theagenticweekly.com/methodology#testing-rig) for the hardware, the local model and the rules we follow, and the [test hub](https://theagenticweekly.com/tests) for every other test.

## FAQ

### Does Instructor work with a local model?

In our run it did for this one task: 3 of 3 trials passed against a local Qwen2.5-3B model served by Ollama. We did not test other models, and results depend heavily on the model's tool-calling ability.

### How big is a fresh Instructor install?

49 MB in 41 packages with uv on 1.17.0, measured on 2026-10-25.

## Sources

- [Instructor repository](https://github.com/567-labs/instructor) (fetched 2026-10-25)
- [Instructor documentation](https://python.useinstructor.com/) (fetched 2026-10-25)

## Related

- [All test logs](https://theagenticweekly.com/tests)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)
- [Study: install size and first-task results across 16 frameworks](https://theagenticweekly.com/research/agent-framework-install-and-first-task)

## How to cite

Agentic Weekly, "Instructor 1.17.0: our test log", last updated 2026-10-25, https://theagenticweekly.com/tests/instructor