Skeleton-Based Parent-Child Play Activity Recognition using a Single Camera = 스켈레톤기반 단일 카메라를 이용한 부모-자녀의 놀이 행동 인식

다국어 초록 (Multilingual Abstract)

This study presents a nonverbal activity recognition framework that identifies parent and child play activities using time-series skeleton data extracted from CCTV footage. The system employs pre-trained YOLOv8 model for object detection, localization, and consistent ID assignment across frames, while DeepSORT is simultaneously incorporated for real-time tracking of each detected subject through the consistent IDs over time. YOLOv11 is used for skeleton joint extraction. Two feature extraction methods were compared: 1D-CNN embeddings (Method 1) and the proposed Fusion-CNNFE (Method 2), which integrates CNN-based embeddings with engineered features. Several machine learning classifiers were used for play activity recognition. Experimental results show that Fusion-CNNFE achieved higher accuracy, precision, recall, and F1-score than the baseline 1D-CNN. The best results were obtained using LightGBM for the parent dataset (F1 = 87.81%) and Random Forest for the child dataset (F1 = 70.09%) under the K-Means SMOTE data augmentation. The findings confirm that fusing spatial–temporal features with balanced training data improves recognition robustness, providing a foundation for nonverbal behavior monitoring in playroom and educational environments. Keywords Human interaction recognition, Parent and child play activity, 1D-Convolutional Neural Network (1D-CNN), Feature Engineering, Data Interpolation, Sliding Window, YOLO

목차 (Table of Contents)

• LIST OF TABLES ⅲ
• LIST OF FIGURES ⅴ
• Abstract ⅵ
• 1. INTRODUCTION 1
• 2. RELATED WORK 4
• 3. METHOD 6
• 3.1. Dataset 6
• 3.2. Proposed Methodology 7
• 3.3. Preprocessing 8
• 3.3.1. Data Labelling 9
• 3.3.2. Training 15
• 3.3.3. Parent and child classification 15
• 3.3.4. Skeleton Data Extraction 16
• 3.4. Feature Extraction 18
• 3.4.1. Method 1: 1D-CNN-Based Feature 18
• 3.4.2. Feature Engineering Extraction 21
• 3.4.3. Method 2: 1D-CNN Feature Concatenated with Features Engineer (Fusion-CNNFE) 24
• 3.5. Classifiers 25
• 3.6. Evaluation Model 25
• 4. EXPERIMENTAL RESULTS 27
• 4.1. Data Distributions 27
• 4.2. Performance Evaluation of Window Sizes 29
• 4.3. Evaluation of Feature Extraction Methods 31
• 4.4. Results 32
• 4.4.1. 1D-CNN-Based Feature 33
• 4.4.2. Fusion-CNNFE 37
• 5. DISCUSSION 43
• 6. CONCLUSION 44
• REFERENCES 46