A Multi-touch Gesture Vocabulary Design Methodology for Mobile Devices

다국어 초록 (Multilingual Abstract)

This study aims to propose a methodology for designing multi-touch gesture vocabularies applicable on mobile devices. The specific research objectives are 1) Proposing an integrated procedure for designing gesture vocabularies; 2) Proposing a systematic framework to analyze and create gesture vocabularies; 3) Validating the proposed methodology through a case study for designing mobile web browsing gestures; 4) Suggesting best multi-touch gesture vocabularies for mobile web browsers (MWBs).
The integrated procedure involved both empirical approach and analytical approach. The empirical approach collected intuitive gestures from end-users for mobile web browsing commands. Users defined appropriate gestures intuitively themselves. On the other hand, the analytical approach created gestures based on a systematic framework. The framework consisted of gesture features and gesture elements, and the two attributes were combined to create various gestures. Designers generated gesture vocabularies analytically based on the framework. The analytical approach followed the empirical approach in the procedures.
Basic gesture elements that constitute multi-touch gestures were identified. Extrinsic postures or motions were decomposed into the smallest units, and the categorized units became gesture elements. The gesture elements allowed observers to describe gestures concretely, and moreover the elements were used to analyze the usage patterns of gestures. Taxonomy of multi-touch gestures was also developed based on the basic elements. This study identified 6 basic elements of multi-touch gestures in mobile devices. The proposed elements were ‘posture’, ‘location’, ‘pose’, ‘touch’, ‘path’, and ‘device’.
The usefulness and practicality of the methodology were validated through a case study. The case study identified mobile web browsing gestures to improve short-cut interactions. The initial gestures for the commands were collected from 36 end-users who did not have any experience with gesture interaction on web browsers. A total of 642 gestures were collected in the empirical approach. Six volunteers created gestures based on the systematic design framework, and they generated 314 gestures in the analytical approach. The proposed methodology helped designers to find appropriate gesture vocabularies for various commands on mobile devices.
Gesture candidates were selected from the gestures which were collected by the empirical approach and created by the analytical approach. The selected candidates were evaluated by the 22 subjects. Their intuitiveness and subjective satisfaction scores are compared with each other to identify the most suitable gesture vocabularies. Finally, 11 best gestures and 38 top group gestures were identified for mobile web browsing commands.
This study tried to find cognitively and physically appropriate web browsing gestures to end-users. However, technological issues such as confliction between different commands and restriction of recognition algorithm were not considered. Based on gestures identified in this study, further study considering these issues should be conducted in the future.

목차 (Table of Contents)

• Chapter 1. Introduction 1
• 1.1. Background 1
• 1.1.1. Multi-touch gestures for natural user interface 1
• 1.2. Motivation 6
• 1.3. Thesis objectives 8
• 1.4. Thesis structures 9
• Chapter 2. Related works 10
• 2.1. Taxonomies of gesture 10
• 2.1.1. Semantic labels 11
• 2.1.2. Functional labels 13
• 2.1.3. Descriptive labels 14
• 2.1.4. Communicative labels 16
• 2.2. Approaches to design gesture vocabulary 18
• 2.2.1. Gesture design procedures 19
• 2.2.2. Gesture designers 21
• 2.2.3. Gesture design solution methods 24
• Chapter 3. Methodology for designing gesture vocabulary 30
• 3.1. Design procedures of multi-touch gesture vocabulary 30
• 3.2. Collecting gestures from end users 32
• 3.3. Analyzing basic elements that consist of multi-touch gestures 33
• 3.4. Creating gestures by designers 35
• 3.5. Evaluating gesture candidates 36
• Chapter 4. Intuitive gesture collection by empirical approach 37
• 4.1. Finding gesture commands 37
• 4.2. Collecting intuitive gestures from end users 41
• 4.2.1. Methods 41
• 4.2.1.1. Subjects 41
• 4.2.1.2. Apparatus 42
• 4.2.1.3. Procedure 43
• 4.2.2. Consensus among users 44
• 4.2.3. Top gestures for web browsing commands 46
• 4.2.4. Difficulties of gesture observation 51
• Chapter 5. Analysis of the multi-touch gesture elements 52
• 5.1. Surveying examples of multi-touch gestures 52
• 5.1.1. Literature survey 52
• 5.1.2. User survey 55
• 5.2. Classifying elements of multi-touch gestures from users 55
• 5.2.1. Elements of multi-touch gesture 55
• 5.2.2. Taxonomy of multi-touch gesture in mobile devices 59
• 5.2.3. Method for describing gesture 60
• 5.3. Analyzing dominant patterns of using gesture elements 63
• 5.3.1. Dominant elements of web browsing gestures 63
• 5.3.2. Combination of gesture elements 69
• Chapter 6. Gesture creation by analytical approach 71
• 6.1. Extracting gesture features 71
• 6.2. Creation of gestures by combining gesture features and elements 77
• Chapter 7. Evaluation of gesture candidates 82
• 7.1. Design principles for multi-touch gesture interface 82
• 7.2. Screening gesture candidate 86
• 7.3. Evaluating gesture candidates to find best gesture vocabularies 89
• 7.3.1. Method 89
• 7.3.1.1. Subjects 89
• 7.3.1.2. Apparatus 89
• 7.3.1.3. Procedure 90
• 7.3.2. Intuitiveness of gesture candidates 93
• 7.3.3. Subjective satisfaction of gesture candidates 95
• Chapter 8. General discussion 99
• 8.1. Best gestures for mobile web browsers 99
• 8.2. Effectiveness of the proposed design methodology 100
• 8.2.1. Effectiveness basic elements for observing gestures 100
• 8.2.2. Effectiveness of the analytical approach for creating gestures 101
• 8.2.3. Necessity of the usability evaluation 102
• 8.3. Comparison between empirical approach and analytical approach 103
• 8.3.1. Number of gestures 103
• 8.3.2. Subjective satisfaction of gestures 105
• 8.4. Usage of design methodology for multi-touch gestures 107
• Chapter 9. Conclusions 110
• References 115
• Appendices 122