Routing Mixture-of-Experts Based on Ensembled Spatiotemporal Representations for Engagement Estimation in Online Courses

EasyChair Preprint 14673

Abstract

Automatic estimation of students’ engagement provides real-time feedback to the teachers in online courses. Although some deep learning methods have shown success in engagement estimation, most of them are developed based on convolutional neural networks (CNNs), which fail to capture long-range spatial and temporal dependencies in video data. Even when both temporal and spatial representations are extracted, they are not fully utilized, decreasing the accuracy of en-gagement estimation. To address these issues, we propose a novel Mixture-of-Experts (MoE) method that effectively ensembles spatial and temporal representations. Specifically, we introduce a Routing Mixture-of-Experts (RMoE) method designed to capture comprehensive and discriminative spatiotemporal representations. The method uses a routing mechanism to dynamically select the most relevant experts for a given input, ensuring accurately capture both spatial and temporal representations. We evaluated the effectiveness of our model using the Dataset for Affective States in E-Environments (DAiSEE). Experimental results show that our model significantly outperforms several state-of-the-art methods, highlighting its potential to improve the accuracy of student engagement estimation in online learning environments.

Keyphrases: Engagement estimation, Spatiotemporal Representations, ensemble learning, online courses

@booklet{EasyChair:14673,
  author    = {Hongqiang Shen and Jun Wang and Juncheng Li and Jun Shi},
  title     = {Routing Mixture-of-Experts Based on Ensembled Spatiotemporal Representations for Engagement Estimation in Online Courses},
  howpublished = {EasyChair Preprint 14673},
  year      = {EasyChair, 2024}}