arXiv / 2026-11-10

Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference

arXiv:2611.10193v1 Announce Type: new Abstract: Large language models (LLMs) increasingly support Web-based services that must balance response quality and inference cost across heterogeneous requests. Conventional routing selects one model per prompt, while fixed ensembles cannot adapt their composition and cost to individual requests. We introduce PISA, a prompt-adaptive, cost-aware single-model router, and PISA-E, its extension to ensemble routing. At the core of both methods, PISA combines s