GPT4All gpt4all 2.8.2 (Python SDK): our test log

GPT4All's Python SDK 2.8.2 loaded a local 3B GGUF in 4.01 s and answered 17 + 25 with 42 in 3 of 3 trials, but kept writing past the answer.

Result at a glance

What we found

• 3 of 3 trials contained 42.
• Each reply was '42' followed by an invented 'Human:' turn until the 20-token limit, so the model did not stop after the answer: a raw GGUF loaded this way did not get the chat template the model expects, and we did not try to fix that.
• The two-sentence sky explanation was accurate and fluent.

What we ran, step by step

# Install with uv (cold, no cache)
uv venv -q -p 3.13 work/py/gpt4all && VIRTUAL_ENV=work/py/gpt4all uv pip install --no-cache -q gpt4all && du -sm work/py/gpt4all | cut -f1 && VIRTUAL_ENV=work/py/gpt4all uv pip list 2>/dev/null | grep -i -E '^(gpt4all|gpt4all) '

# Task command (all trials)
work/py/gpt4all/bin/python tasks3/gpt4all_task.py

Raw output (excerpts)

42

Human: Given the sentence "A man in a red shirt and blue jeans plays an

import time, json, os, sys
t0 = time.perf_counter()
from gpt4all import GPT4All
m = GPT4All("qwen2.5-3b-instruct-q4_k_m.gguf", model_path="/workspace/tests2/work/gpt4all/models", allow_download=False, device="cpu", n_ctx=2048)
load_s = round(time.perf_counter() - t0, 2)
out = []
for i in range(3):
    t = time.perf_counter()
    with m.chat_session():
        r = m.generate("What is 17 + 25? Reply with only the number.", max_tokens=20, temp=0.0)
    s = round(time.perf_counter() - t, 2)
    out.append(dict(trial=i + 1, seconds=s, text=r.strip()[:80], ok="42" in r))
t = time.perf_counter()
with m.chat_session():
    r = m.generate("Write two sentences explaining why the sky is blue.", max_tokens=80, temp=0.7)
gs = round(time.perf_counter() - t, 2)
print("RESULT_JSON " + json.dumps(dict(load_s=load_s, trials=out, gen_seconds=gs, gen_text=r.strip()[:300])))

Gotchas and things that broke

• The Linux wheel printed 'Failed to load libllamamodel-mainline-cuda' warnings on this CPU-only machine and then ran on CPU as asked.
• Loading a GGUF that is not in GPT4All's own model list means you supply the file name and folder yourself with allow_download=False; stop behaviour depended on the model's template, which was not applied here.

What we did not test

• The GPT4All desktop app and its LocalDocs feature: only the Python SDK was run.
• GPT4All's own downloadable model catalogue and its download speed; we used a file already on disk.
• Speed: the machine was heavily shared during the run, so the timings are not a measure of the SDK.
• A single prompt and three trials is a smoke test, not a benchmark.

Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

uv venv -p 3.13 .venv && VIRTUAL_ENV=.venv uv pip install gpt4all
# put a GGUF in ./models, then:
.venv/bin/python -c "from gpt4all import GPT4All; m=GPT4All('your-model.gguf', model_path='./models', allow_download=False, device='cpu'); print(m.generate('What is 17 + 25? Reply with only the number.', max_tokens=20, temp=0))"

How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the testing rig for the hardware, the local model and the rules we follow, and the test hub for every other test.

Related tests: local models and runtimes

• Hugging Face: Hugging Face's Hub searched without a login in 0.1 s, a 272 MB model downloaded in 3.61 s, and transformers ran it on CPU.
• llama.cpp: llama.cpp b11889 generated 118 tokens/s (0.5B), 40 (1.5B) and 19.5 (3B) on 8 CPU threads with llama-bench, and its server passed the same prompts and tool call as Ollama.
• Ollama: Ollama 0.40.11 served three Qwen2.5 sizes over its OpenAI-compatible API on 8 CPU cores: all three answered simple prompts; the 0.5B model failed JSON extraction while 1.5B and 3B passed, and tool calls worked 5 of 5 on all three.

All test logs · How we test

FAQ

Sources

• GPT4All repository (fetched 2026-10-26)

Related

• All test logs
• Methodology and testing rig
• GPT4All facts and sources