Open source · 3 November 2026
Zyphra AI launches ZHA, a tree-indexed sparse attention method with 4B-token distillation
ZHA retrofits frozen LLMs with a tree-indexed sparse attention mechanism and 4 billion tokens of distillation, matching dense attention on reasoning tasks and 256 K retrieval benchmarks.
Benchmarks are the company's own; not independently verified.
Why it matters
ZHA’s 4 B-token context lets you build features that handle longer inputs without quadratic cost.
For product managers
For your product
What does this mean for your product?
Try it on your product More from this issueAll AI news from 3 Nov →
Get this every morning, written for your product.
Deltaday reads the whole AI news cycle and tells you what changed, whether it's real yet, and what to do about it.