Deep Learning of Postural and Ocular Dynamics to Predict Engagement and Learning of Audiovisual Materials.

다국어 초록 (Multilingual Abstract)

Engagement with educational instruction and related materials is an important part of learning and contributes to test performance. There are various measures of engagement including self-reports, observations, pupil diameter, and posture. With the challenges associated with obtaining accurate engagement levels, such as difficulties with measuring variations in engagement, the present study used a novel approach to predict engagement from posture by using deep learning. Deep learning was used to analyze a labeled outline of the participants and extract key points that are expected to predict engagement. In the first experiment two short lectures were presented and participants were tested on a lecture to motivate engagement. The next experiment had videos that varied in interest to understand whether a more interesting presentation engages participants more, therefore helping participants achieve higher comprehension scores. In a third experiment, one video was presented to attempt to use posture to predict comprehension rather than engagement. The fourth experiment had videos that varied in level of difficulty to determine whether a challenging topic versus an easier topic affects engagement. T-tests revealed that the more interesting Ted Talk was rated as more engaging, and for the fourth study, the more difficult video was rated as more engaging. Comparing average pupil sizes did not reveal significant differences that would relate to differences in the engagement scores, and average pupil dilation did not correlate with engagement. Analyzing posture through deep learning resulted in three accurate predictive models and a way to predict comprehension. Since engagement relates to learning, researchers and educators can benefit from accurate engagement measures.