# Continue CLI 1.5.47: our test log

Canonical: https://theagenticweekly.com/tests/continue-cli · Last verified: 2026-10-25 · Published: 2026-10-25 · Author: Agentic Weekly (operated independently from Texas, US) · Evidence: Tested by us on 2026-10-25. Local open-weight model, not the vendor's hosted models. A smoke test on a CPU-only box, not a benchmark.

> Continue CLI 1.5.47 answered a question and fixed a one-line bug with a local 3B model through a YAML config; a 1.6 s, 81 MB install.

## Result at a glance

Table: Continue CLI test record

| Field | Value |
|---|---|
| Result | Passed |
| Version tested | 1.5.47 |
| Date and time | 2026-10-25, 23:03 to 23:04 CDT (America/Chicago) |
| Task | In a throwaway folder, give the tool a small bug and ask a local model to fix it through the tool's own file-editing. Two checks: (1) a one-shot question, 'What is 17 + 25? Answer with just the number.'; (2) calc.py contained `def add(a, b): return a - b` and we asked it to fix the bug by editing the file. Pass means the answer was 42 and calc.py ended as `return a + b`. |
| Model used | Qwen2.5-3B-Instruct (Q4_K_M) through Ollama 0.40.11 |
| Hardware | Linux x86_64 virtual machine (Debian 13), 8 vCPU (Intel Xeon, 1 thread per core), 15 GiB RAM, no GPU |
| Install (npm, cold) | 1.6 s, 81 MB |
| Question wall time | 13.2 s (first), 1.8 s (second) |
| Edit task | fixed in 26.7 s, 1 trial |

## What we found

- The one-shot question returned 42 twice (13.2 s the first time, which includes starting up, then 1.8 s).
- The edit task rewrote calc.py correctly in 26.7 s with --auto (which lets the agent act without asking). Checked by reading the file.
- Install is small (81 MB, 13 packages) because the CLI is a thin client.

## What we ran, step by step

Table: Commands run, wall-clock time and exit code

| Step | Wall time | Exit code | Notes |
|---|---|---|---|
| npm install on Node 20 (cold) | 1.6 s | 0 | 81 MB in 13 packages; load 3.06. |
| Question, run 1 (includes first start) | 13.2 s | 0 | Answer: 42 |
| Question, run 2 | 1.8 s | 0 | Answer: 42 |
| Edit task | 26.7 s | 0 | calc.py ended as `return a + b`; the CLI printed "The function has been updated successfully." Load 5.07. |

Exact commands (paths shortened)

```
# npm install on Node 20 (cold)
npm install @continuedev/cli
```

## Raw output (excerpts)

config.yaml used (a local Ollama model, nothing hosted)

```
name: local-test
version: 1.0.0
schema: v1
models:
  - name: qwen-local
    provider: ollama
    model: qwen2.5-3b
    apiBase: http://127.0.0.1:11434
    roles: [chat]
```

Edit task command

```
cn --auto --config config.yaml -p 'The function add in calc.py has a bug: it subtracts. Fix it so it adds, by editing calc.py.'
```

Edit task output

```
The function has been updated successfully.
---
def add(a, b):
    return a + b
```

## Gotchas and things that broke

- We set HOME and CONTINUE_GLOBAL_DIR to a scratch folder and turned telemetry off with DO_NOT_TRACK=1 so the test did not touch real config or phone home; the defaults were not measured.
- This is the Continue command-line tool (npm package @continuedev/cli), not the VS Code or JetBrains extension, which we did not run.

## What we did not test

- The IDE extensions, hub-hosted configs and any hosted model.
- Reliability: the edit task ran once, so this is an existence proof, not a pass rate.

## Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

```
npm install @continuedev/cli
# write config.yaml as shown, create calc.py with the bug, then:
cn --auto --config config.yaml -p "The function add in calc.py has a bug: it subtracts. Fix it so it adds, by editing calc.py."
```

## How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the [testing rig](https://theagenticweekly.com/methodology#testing-rig) for the hardware, the local model and the rules we follow, and the [test hub](https://theagenticweekly.com/tests) for every other test.

## FAQ

### Can Continue run offline with a local model?

The CLI did in our run, with an Ollama model configured in YAML. We did not test the IDE extension.

## Sources

- [Continue repository](https://github.com/continuedev/continue) (fetched 2026-10-25)

## Related

- [All test logs](https://theagenticweekly.com/tests)
- [Methodology and testing rig](https://theagenticweekly.com/methodology#testing-rig)
- [Study: install size and first-task results across 16 frameworks](https://theagenticweekly.com/research/agent-framework-install-and-first-task)
- [Study: the silent version downgrade on older Node, and other install traps](https://theagenticweekly.com/research/older-node-silent-downgrades)

## How to cite

Agentic Weekly, "Continue CLI 1.5.47: our test log", last updated 2026-10-25, https://theagenticweekly.com/tests/continue-cli