ProRouter: 비용 효율적인 LLM 협업을 위한 프로젝션 기반 라우터 : SVD를 이용한 라우팅 = ProRouter: A Projection-Based Router for Cost-Efficient LLM Collaboration

다국어 초록 (Multilingual Abstract)

Recent advances in large language models (LLMs) have greatly improved performance across natural language tasks, but their substantial resource cost poses serious challenges for practical deployment. Although numerous routing methods have been proposed to reduce this cost, most rely on output scores, estimated difficulty, or heuristic rules, and often require additional fine-tuning of the base models. In this paper, we present ProRouter, a projection-based routing framework that efficiently selects between large and small language models (SLMs) using subspace projections of query embeddings. Given an input query, ProRouter first computes its dense embedding using a pre-trained encoder. For each candidate model, it applies singular value decomposition (SVD) to the embeddings of queries labeled according to whether the model’s predictions were correct or incorrect, constructing low-dimensional subspaces that capture the distinctive patterns of success and failure. At inference time, a new query embedding is projected onto these subspaces to obtain similarity- and projection-based features, including cosine similarity to mean embeddings, top-𝑘 similarity, and projection magnitude. These features are concatenated and fed into a lightweight routing classifier to select the most appropriate model, without retraining or fine-tuning of the underlying LLMs. By routing simpler queries to SLMs and complex ones to LLMs, ProRouter substantially reduces computation and latency while maintaining competitive accuracy. Experimental results show that ProRouter achieves large reductions in LLM usage, demonstrating the effectiveness of subspace-projection features for scalable and cost-efficient multi-model collaboration.

목차 (Table of Contents)

• 1 Introduction 5
• 2 Related Work 9
• 2.0.1 Cost-Efficient LLM Cascades and Ensembling 9
• 2.0.2 LLM Routing Models and Expert Selection 9
• 2.0.3 Mixture-of-Experts and Model Merging 10
• 3 Proposed Method 11
• 3.0.1 Query Embedding Extraction 11
• 3.0.2 Model-Specific Correct/Incorrect Subspace Construction 12
• 3.0.3 Subspace Projection and Feature Engineering 13
• 3.0.4 Label Construction: Soft and Strict Targets 16
• 3.0.5 Dynamic Multi-Task Loss 16
• 4 Experiments 18
• 4.0.1 Experimental Setup 21
• 4.0.2 Evaluation Metrics 21
• 4.0.3 Results and Analysis 22
• 5 Conclusion 25
• A Experiment Settings 26
• A.1 Ablation study 26
• A.2 Cost model for IRT-specific comparison 26
• A.3 Supervised data used to build SVD subspaces and train the router 26
• A.4 Additional OOD Evaluation Results 26
• A.5 Feature Construction and Embedding 28
• A.6 Router Model and Training 28
• A.7 Compute and Infrastructure 29
• B Prompt Templates 30
• B.1 CommonsenseQA (CQA) Prompt 30
• B.2 OpenbookQA Prompt 30
• B.3 GSM8K Prompt 31
• B.4 RACE Prompt 33