Human analogical reasoning violates geometric assumptions of vector-based models

다국어 초록 (Multilingual Abstract)

Analogical reasoning is identifying relationships between two things and mapping those relationships onto novel pairs. Computational approaches often model analogical reasoning using vector-based word embeddings, where relational similarity is represented geometrically. A key example is the parallelogram model, which assumes that relationships between word pairs can be captured as vector differences. These models assume that relational similarity follows geometric constraints such as triangle inequality, which in mathematics means that the sum of any two sides of a triangle is greater than or equal to the third side. In analogy, if A is similar to B and B to C, then A should be similar to C. However, such models assume a pair’s similarity is stable and context-independent. We hypothesized that when there are two types of relationships for one pair, it would lead to context-dependent violations of the triangle inequality and challenge the assumptions of vector-based models. To investigate this, we developed an analogy task where participants revealed four masked images one by one through fixation and then rated the similarity between pairs. We created 18 questions, each of four pairs of related items labeled A, B, C, and D. Pairs AB, AC, and BC share the same relation; pair AD has a different relation, while pairs BD and CD have no meaningful relation. This configuration formed two comparison triangles: one with AB, AC, BC, and another with AB, AD, and BD. This enabled us to test whether relational similarity judgements followed the triangle inequality within a consistent relation, and whether that pattern was violated when a different relation was introduced. Human behavior confirmed that the triangle inequality was often violated in the mixed-relation triangle but preserved in the consistent-relation triangle. These findings suggest that human relational similarity judgements are context-sensitive. To test whether this effect depended on semantic meaning, a second experiment used geometric stimuli defined by visual features as shape, edge color, and fill color. Violations occurred more frequently when pair A shared two features (same shape and same fill color), while B and C shared only the fill color, and D shared only the shape. This suggests that humans selectively emphasize one relational dimension at a time, compressing multi-feature information possibly due to attentional limits or prioritization. Furthermore, when the same questions were tested on large language models, there were slightly fewer violations in the main experiment and minimal violations in the control task. These results suggest that, unlike humans, the large language models might maintain access to all relational dimensions, leading to no feature compression.

국문 초록 (Abstract)

초록 기존의 벡터 기반 계산 모델(Vector-based computational models)들은 유추 추론이 삼각 부등식(Triangle inequality)과 같은 기하학적 제약(Geometric constraints)을 따른다고 가정하며, 이는 관계적 유사성이 문맥과 무관하게 독립적(Context-independent)임을 전제합니다. 본 연구는 인간의 추론 과정이 본질적으로 문맥 의존적(Context-sensitive)이라는 가설을 바탕으로 이러한 기하학적 가정을 검증했습니다. '일관된(Consistent)' 관계와 '혼합된(Mixed)' 관계를 구분한 유추 과제 실험 결과, 인간은 관계적 문맥이 변화할 때 삼각 부등식을 빈번하게 위반하는 것으로 나타났습니다. 기하학적 도형을 이용한 후속 실험을 통해, 이러한 위반이 인간의 주의력 한계(Attentional limits)로 인해 특정 차원만을 선택적으로 처리하는 '특징 압축(Feature compression)' 기제에서 기인함을 확인했습니다. 반면, 대규모 언어 모델(LLM)은 본 실험에서는 인간보다 다소 적은 위반을 보였으며, 기하학적 통제 실험에서는 위반이 거의 나타나지 않았습니다. 이는 인간과 달리 LLM 이 주의력 병목 현상 없이 고차원적 특징 정보에 동시에 접근하여 처리하고 있음을 시사합니다.

목차 (Table of Contents)

• Chapter 1. INTRODUCTION 1
• 1.1 Cognition and Concepts 1
• 1.2 Concepts and Relational Similarity 2
• 1.3 Analogy 6
• 1.3.1 Formalization of Analogical Reasoning in Humans 8
• 1.3.2 Formalization of Analogical Reasoning in Computational Models 10
• 1.3.3 Analogical Reasoning in Large Language Models 10
• Chapter 2. MOTIVATION AND RATIONALE 14
• Chapter 3. MATERIALS AND METHODS 16
• 3.1 Participants 16
• 3.2 Stimuli 16
• 3.3 Task Design 19
• 3.4 Control Experiment 25
• 3.5 Large Language Models 28
• 3.5.1 Model Background and Architecture 28
• 3.5.2 Experimental Implementation 30
• 3.6 Data Analysis 31
• 3.6.1 Human vs. LLM Comparison 32
• 3.6.2 Correlation Matrix 32
• 3.6.3 Product Test 33
• Chapter 4. RESULTS 35
• 4.1 Main Experiment 35
• 4.2 Control Experiment 42
• Chapter 5. DISCUSSION AND FUTURE DIRECTIONS 47
• References 50
• Appendix 54
• 논문요약 60