Open source · 8 November 2026
Buoyant launches Kinference, open‑source inference control plane for vLLM on Kubernetes
Kinference adds declarative routing policies, KV‑cache‑aware load balancing, per‑tenant metrics, and reduces median time‑to‑first‑token by 95 % in Buoyant’s own benchmark.
Benchmarks are the company's own; not independently verified.
Why it matters
Consider adding Kinference‑style routing and load‑balancing to cut latency for multi‑tenant LLM services.
For product managers
For your product
What does this mean for your product?
Try it on your product More from this issueAll AI news from 8 Nov →
Get this every morning, written for your product.
Deltaday reads the whole AI news cycle and tells you what changed, whether it's real yet, and what to do about it.