LangChain.js 1.5.19: our test log

LangChain.js 1.5.19 passed our one-tool task 3 of 3 on Node 20 and 3 of 3 on Node 24; about 101 MB in 36 packages.

Result at a glance

What we found

• 3 of 3 on Node 20 and 3 of 3 on Node 24 using createAgent from langchain 1.5.19.
• Install size is about 100 MB in 36 packages on both Node versions, roughly four times the Vercel AI SDK.

What we ran, step by step

# npm install on Node 20.19.2 (cold, empty cache)
cd work/npm/langchainjs && npm install --no-audit --no-fund --cache /tmp/npmcache --loglevel=error langchain @langchain/openai @langchain/core zod

# npm install on Node 24.22.1 (cold, empty cache)
cd work/npm24/langchainjs && npm install --no-audit --no-fund --cache /tmp/npmcache --loglevel=error langchain @langchain/openai @langchain/core zod

# Task command (all trials)
node task.mjs

Raw output (excerpts)

import { createAgent, tool } from "langchain";
import { ChatOpenAI } from "@langchain/openai";
import { z } from "zod";
const calls = [];
const add = tool(async ({ a, b }) => { calls.push([a, b]); return String(a + b); }, { name: "add_numbers", description: "Add two integers and return the sum.", schema: z.object({ a: z.number(), b: z.number() }) });
const agent = createAgent({ model: new ChatOpenAI({ model: "qwen2.5-3b", temperature: 0, apiKey: "ollama", configuration: { baseURL: "http://127.0.0.1:11434/v1" } }), tools: [add] });
const r = await agent.invoke({ messages: [{ role: "user", content: "Use the add_numbers tool to compute 17 + 25, then tell me the result." }] });
const final = String(r.messages.at(-1).content);
const ok = calls.some(([a, b]) => (a === 17 && b === 25) || (a === 25 && b === 17)) && final.includes("42");
console.log("RESULT_JSON " + JSON.stringify({ calls, final: final.slice(0, 300), ok }));

Gotchas and things that broke

• The agent API is createAgent from the langchain package, not the older AgentExecutor patterns shown in many tutorials.

What we did not test

• Any vendor-hosted model: everything ran against one local 3B model.
• Multi-agent flows, memory, streaming and deployment.
• Anything beyond one tool and one question; five trials is a smoke test, not a benchmark.

Reproduce it

Versions change. These commands install whatever is latest on the day you run them unless a version is pinned, so expect different results later. The version we measured is in the table above.

# serve a tool-calling local model at http://127.0.0.1:11434/v1 (we used Ollama with Qwen2.5-3B-Instruct, model name qwen2.5-3b)
mkdir t && cd t && npm init -y && npm install langchain @langchain/openai @langchain/core zod
# save task.mjs (above) and run it:
node task.mjs

How to read this test

Every number on this page was measured on this one machine, which also ran other jobs, so timings are indicative and vary run to run. A pass means our small task worked once or as stated, a fail means it did not work as we ran it; neither says how the tool behaves with a vendor's frontier model, on a GPU, or at scale. Where a tool could not run without an account or a GPU, we say so instead of guessing.

See the testing rig for the hardware, the local model and the rules we follow, and the test hub for every other test.

Related tests: agent frameworks and sdks

• LangGraph: LangGraph 1.2.17 passed our one-tool task on a local 3B model: 3 of 3 trials; cold uv install 2.5 s.
• LiteLLM: LiteLLM 1.106.4 passed our one-tool task on a local 3B model: 3 of 3 trials; cold uv install 2.3 s.
• LlamaIndex: LlamaIndex 0.14.25 passed our one-tool task on a local 3B model: 3 of 3 trials; cold uv install 3.4 s.
• Mastra: Mastra passed 5 of 5 on Node 20 (0.24.9) and 5 of 5 on Node 24 (1.79.0) with temperature pinned to 0, after failing 5 of 6 earlier trials at default sampling.
• Mem0: Mem0's open-source SDK 2.2.1 stored a fact from one chat message with a local 3B model and retrieved it with a semantic search (2.04 s) on the second attempt; the first two runs failed on a missing package and an API change.
• OpenAI Agents SDK (Python): OpenAI Agents SDK (Python) 0.24.0 passed our one-tool task on a local 3B model: 3 of 3 trials; cold uv install 1.7 s.

All test logs · How we test

FAQ

Related

• All test logs
• Methodology and testing rig
• Study: install size and first-task results across 16 frameworks
• Study: the silent version downgrade on older Node, and other install traps
• LangChain facts and sources