# Haystack 3.5.0 adds OpenAI Decisions API routers and Agent structured output, changes default model

Canonical: https://theagenticweekly.com/news/haystack-3-5-0-decisions-api-structured-output · Evidence: Reported · First seen: 2026-11-10 12:50 UTC · Source published: 2026-11-10 12:34 UTC · Brief published: 2026-11-10 12:52 UTC · Updated: 2026-11-10 12:52 UTC · Author: Agentic Weekly

> Haystack v3.5.0, released on GitHub on November 10, 2026 (UTC), adds three components that, in the release notes' words, "use the OpenAI Decisions API to choose how to handle your data without instructions." It also changes the default OpenAI chat model from gpt-5-mini to gpt-5.6-terra, which the notes say OpenAI shuts down on December 11, 2026.

## What changed

- Three new components use the OpenAI Decisions API: OpenAIDecisionsTextRouter sends text to a named output per label, OpenAIDecisionsImageRouter routes images and can send low-confidence decisions to a fallback branch, and OpenAIDecisionsDocumentClassifier stores choice, score and yes/no answers in document metadata.
- A new StructuredOutputHook validates an Agent's final reply against a Pydantic model or JSON schema and returns it under result["structured_output"]; on a failed parse it sends the error back to the model, up to max_retries times (2 by default).
- The experimental StepBudgetReminderHook tells the model how many steps it has left as it nears max_agent_steps, so it wraps up instead of being cut off. max_agent_steps is now a reserved state key and redefining it raises a ValueError.
- Document gains a multivector_embedding field for late-interaction models such as ColBERT, a new MaxSimRanker reranks candidates with those vectors, and DocumentJoiner(join_mode="merge_embeddings") can merge dense, sparse and multivector embeddings.
- OpenAIChatGenerator, OpenAIResponsesChatGenerator and the LLM-based evaluators now default to gpt-5.6-terra instead of gpt-5-mini; pipelines saved with model: gpt-5-mini keep using it, and the notes say OpenAI shuts that model down on December 11, 2026.
- Upgrade notes: the minimum openai SDK rises from 2.6.0 to 3.26.0, pydantic>=2.12 is now required, and Variadic or multi-sender list inputs now receive values in Pipeline.connect() order rather than the order senders ran.

## Who should act before upgrading

Teams with hybrid retrieval should check this first: the release notes say order-dependent components such as DocumentJoiner with weights can now return different results, because a socket receives values in the order the senders were connected. Pipelines loaded from YAML were saved with connections grouped by sender, so the weights may now apply to a different retriever. Reorder those connections entries before trusting scores after the upgrade.

Anyone who relies on the default OpenAI model gets a different model on upgrade, and anyone with saved pipelines pinned to gpt-5-mini has a deadline the notes put at December 11, 2026. The openai SDK floor also jumps to 3.26.0, which can conflict with other packages in the same environment.

## What the release means for builders

Routing and classification without a hand-written prompt now sit inside the framework, but the three Decisions components tie that step to one vendor's API. StructuredOutputHook adds a validate-and-retry loop that many teams wrote themselves, and multivector support is explicitly only a first step: the embedders that produce multivector embeddings are still in progress in Haystack Core Integrations, per the notes.

## In the source's words

> "Three new components use the OpenAI Decisions API to choose how to handle your data without instructions."
> (Haystack v3.5.0 release notes, https://github.com/deepset-ai/haystack/releases/tag/v3.5.0)

> "This is the first step toward multivector retrieval in Haystack"
> (Haystack v3.5.0 release notes, multivector section, https://github.com/deepset-ai/haystack/releases/tag/v3.5.0)

> "connect the senders in the order you want the values to arrive."
> (Haystack v3.5.0 release notes, Upgrade Notes, https://github.com/deepset-ai/haystack/releases/tag/v3.5.0)

## What we have not verified

- We did not run this release ourselves.
- The gpt-5.6-terra recommendation and the December 11, 2026 gpt-5-mini shutdown date come from the Haystack release notes; we did not confirm them on OpenAI's own deprecation page.
- We did not check Decisions API pricing or availability, or how routing accuracy compares with a prompted classifier.
- We did not test how many existing YAML pipelines change results under the new connection ordering.

## Timeline

- Source published: 2026-11-10 12:34 UTC
- First seen by our watcher: 2026-11-10 12:50 UTC (via GitHub releases: deepset-ai/haystack)
- Brief published: 2026-11-10 12:52 UTC

## Sources

- [GitHub release: deepset-ai/haystack v3.5.0](https://github.com/deepset-ai/haystack/releases/tag/v3.5.0) (primary source) (fetched 2026-11-10)

## Related

- [Haystack: verdict and setup guide](https://theagenticweekly.com/tools/haystack)
- [LlamaIndex: verdict and setup guide](https://theagenticweekly.com/tools/llamaindex)
- [LangChain: verdict and setup guide](https://theagenticweekly.com/tools/langchain)
- [All news briefs](https://theagenticweekly.com/news)
- [New AI models this week](https://theagenticweekly.com/models/this-week)

## How to cite

Agentic Weekly, "Haystack 3.5.0 adds OpenAI Decisions API routers and Agent structured output, changes default model", published 2026-11-10 12:52 UTC, https://theagenticweekly.com/news/haystack-3-5-0-decisions-api-structured-output