MambaVLA: A Scalable and Efficient Vision-Language- Action Model with State Space Architecture = 상태 공간 아키텍처를 활용한 확장 가능하고 효율적인 비전 언어 행동 모델 MambaVLA

다국어 초록 (Multilingual Abstract)

Recent advancements in multimodal learning have demonstrated the promise of Vision Language Action (VLA) systems for robotic reasoning and real-world interaction. However, most existing VLA models rely on Transformer based backbones that suffer from quadratic complexity and limited scalability for long sequences. We present MambaVLA, a scalable VLA framework built on the structured state space architecture of Mamba, enabling efficient linear time sequence modeling. MambaVLA integrates an Eagle visual encoder with the Qwen 7B Chat Int4 language model to fuse fine grained visual cues and language representations effectively. To bridge multimodal understanding and robotic control, we incorporate a diffusion flow matching module that aligns visual and linguistic embeddings with continuous action representations. Extensive evaluations on standard VLA benchmarks show that MambaVLA achieves competitive or superior performance compared to Transformer based models while significantly reducing computational overhead and inference latency, demonstrating its suitability for real world embodied intelligence. https://sainavaneet.github.io/MambaVLA.gihub.io/

목차 (Table of Contents)

• 1 INTRODUCTION 1
• 2 RELATED WORKS 3
• 2.1 VISION-LANGUAGE-ACTION MODELS FOR ROBOTICS 3
• 2.2 TRANSFORMER-BASED ARCHITECTURES AND THEIR LIMITATIONS 3
• 2.3 STATE SPACE MODELS AND THE MAMBA ARCHITECTURE 4
• 2.4 MULTIMODAL FUSION AND VISUAL ENCODERS 5
• 2.5 GENERATIVE MODELS FOR ACTION GENERATION 6
• 2.6 COMPUTATIONAL EFFICIENCY AND SCALABLE ARCHITECTURES 7
• 3 BACKGROUND AND TECHNICAL FOUNDATIONS 8
• 3.1 TRANSFORMER ARCHITECTURE AND ITS LIMITATIONS 8
• 3.1.1 TRANSFORMER CORE COMPONENTS 8
• 3.1.2 COMPUTATIONAL AND MEMORY LIMITATIONS 9
• 3.2 STATE SPACE MODELS (SSMS) 9
• 3.2.1 CONTINUOUS-TIME STATE SPACE FORMULATION 9
• 3.2.2 DISCRETIZATION VIA ZERO-ORDER HOLD 10
• 3.2.3 CONVOLUTIONAL FORM VIA KERNEL 10
• 3.3 MAMBA: SELECTIVE STATE SPACE MODEL 10
• 3.3.1 INPUT-DEPENDENT SELECTIVITY 11
• 3.3.2 SELECTIVE DISCRETIZATION 12
• 3.3.3 COMPUTATIONAL EFFICIENCY 12
• 3.3.4 GATING MECHANISM 13
• 3.4 VISION ENCODERS: EAGLE BACKBONE 13
• 3.4.1 ARCHITECTURE OVERVIEW 13
• 3.4.2 PATCH EMBEDDING AND HIERARCHICAL FEATURES 13
• 3.4.3 MULTI-SCALE FEATURE OUTPUT 14
• 3.5 LANGUAGE MODELS: QWEN-7B-CHAT-INT4 14
• 3.5.1 MODEL ARCHITECTURE 14
• 3.5.2 ROTARY POSITION EMBEDDINGS (ROPE) 14
• 3.5.3 QUANTIZATION TO INT4 15
• 3.5.4 TOKEN GENERATION VIA AUTOREGRESSIVE SAMPLING 15
• 3.6 FLOW MATCHING FOR ACTION GENERATION 15
• 3.6.1 DIFFUSION MODELS AND FLOW-BASED GENERATIVE MODELS 16
• 3.6.2 OPTIMAL TRANSPORT PATH 16
• 3.6.3 FLOW MATCHING OBJECTIVE. 16
• 3.6.4 ACTION GENERATION VIA INVERSE FLOW 16
• 3.6.5 CONDITIONAL ACTION GENERATION 17
• 4 MAMBAVLA: SYSTEM ARCHITECTURE AND DESIGN 18
• 4.1 OVERALL FRAMEWORK OVERVIEW 18
• 4.1.1 MULTIMODAL PERCEPTION MODULE 19
• 4.1.2 ACTION-STATE CONDITIONING MODULE 20
• 4.2 VISUAL ENCODING MODULE 20
• 4.2.1 ARCHITECTURE OVERVIEW 21
• 4.2.2 HIERARCHICAL FEATURE EXTRACTION 21
• 4.2.3 FEATURE AGGREGATION AND PROJECTION 22
• 4.2.4 ADVANTAGES 23
• 4.3 LANGUAGE ENCODING MODULE 23
• 4.3.1 MODEL OVERVIEW 24
• 4.3.2 ROTARY POSITIONAL EMBEDDING (ROPE) 25
• 4.3.3 QUANTIZATION TO INT4 PRECISION 25
• 4.3.4 SEMANTIC REPRESENTATION OF OUTPUT EMBEDDINGS 26
• 4.3.5 ADVANTAGES FOR VLA TASKS 26
• 4.4 DIFFUSION FLOW MATCHING MODULE 27
• 4.4.1 MOTIVATION AND BACKGROUND 28
• 4.4.2 CONTINUOUS FLOW FORMULATION 28
• 4.4.3 TRAINING OBJECTIVE 29
• 4.4.4 INTEGRATION WITH CONDITIONAL MAMBA BLOCKS 29
• 4.4.5 ADVANTAGES OF FLOW MATCHING FOR ACTION GENERATION 30
• 4.5 ACTION DECODER AND CONTROL OUTPUT 30
• 4.5.1 DECODING ARCHITECTURE 30
• 4.5.2 EULER INTEGRATION FOR ACTION REFINEMENT 31
• 4.5.3 CONTROL OUTPUT AND EXECUTION 31
• 4.5.4 ADVANTAGES OF THE DECODING DESIGN 32
• 5 IMPLEMENTATION DETAILS 33
• 5.1 MODEL ARCHITECTURE SPECIFICATIONS 33
• 5.2 TRAINING METHODOLOGY AND OBJECTIVES 34
• 5.2.1 SUPERVISED DEMONSTRATION LEARNING 34
• 5.2.2 FLOW-MATCHING OBJECTIVE 35
• 5.2.3 AUXILIARY LOSSES 35
• 5.3 DATA PREPROCESSING AND AUGMENTATION 35
• 5.3.1 VISUAL DATA: 35
• 5.3.2 LANGUAGE DATA: 36
• 5.3.3 ACTION DATA: 36
• 5.4 HYPER PARAMETERS AND TRAINING CONFIGURATION 36
• 5.4 COMPUTATIONAL EFFICIENCY OPTIMIZATIONS 36
• 6 EXPERIMENTAL EVALUATION 38
• 6.1 SIMULATION BENCHMARKS 38
• 6.1.1 LIBERO BENCHMARK 38
• 1) Libero-Object 38
• 2) Libero-Spatial 39
• 3) Libero-Goal 41
• 4) Libero-Long 42
• 6.1.2 METAWORLD BENCHMARK 43
• 6.1.3 ROBOCASA BENCHMARK 44
• 6.1.4 SIM CUBE TRANSFER (MUJOCO) 45
• 6.2 REAL WORLD SETUP 47
• 6.2.1 EXPERIMENTAL CONFIGURATION 47
• 6.2.2 TASK DESCRIPTION 48
• 6.3 BASELINE MODELS AND COMPARISON METHODS 49
• 6.3.1 TRANSFORMER BASED BASELINES 50
• 6.4 PERFORMANCE METRICS AND EVALUATION PROTOCOL 51
• 6.4.1 EVALUATION METRICS 51
• 6.4.2 EVALUATION PROTOCOL 52
• 6.5 QUANTITATIVE RESULTS 53
• 6.6 COMPUTATIONAL EFFICIENCY ANALYSIS 53
• 6.7 ABLATION STUDIES 54
• 7 RESULTS AND ANALYSIS 55
• 7.1 MAIN PERFORMANCE COMPARISONS 55
• 7.2 EFFICIENCY GAINS: INFERENCE SPEED AND MEMORY FOOTPRINT 56
• 7.3 SCALABILITY ANALYSIS 57
• 7.4 QUALITATIVE ANALYSIS: LEARNED REPRESENTATIONS 58
• 7.5 FAILURE CASES AND LIMITATIONS 59
• 7.6 SUMMARY OF CONTRIBUTIONS 59
• 8 DISCUSSION 61
• 8.1 IMPACT ON EMBODIED AI AND ROBOTICS 61
• 8.2 FUTURE RESEARCH OPPORTUNITIES 61
• 9 FINAL REMARKS 63
• REFERENCES 64