Publisher DOI: 10.48550/arXiv.2410.03331
Title: EmojiHeroVR : a study on facial expression recognition under partial occlusion from head-mounted displays
Language: English
Authors: Ortmann, Thorben  
Wang, Qi 
Putzar, Larissa 
Keywords: facial expressions; emotion recognition; virtual reality; affective game
Issue Date: 4-Oct-2024
Publisher: arxiv.org
Journal or Series Name: De.arxiv.org 
Conference: International Conference on Affective Computing and Intelligent Interaction 2024 
Abstract: 
Emotion recognition promotes the evaluation and enhancement of Virtual Reality (VR) experiences by providing emotional feedback and enabling advanced personalization. However, facial expressions are rarely used to recognize users’ emotions, as Head-Mounted Displays (HMDs) occlude the upper half of the face. To address this issue, we conducted a study with 37 participants who played our novel affective VR game EmojiHeroVR. The collected database, EmoHeVRDB (EmojiHeroVR Database), includes 3,556 labeled facial images of 1,778 reenacted emotions. For each labeled image, we also provide 29 additional frames recorded directly before and after the labeled image to facilitate dynamic Facial Expression Recognition (FER). Additionally, EmoHeVRDB includes data on the activations of 63 facial expressions captured via the Meta Quest Pro VR headset for each frame. Leveraging our database, we conducted a baseline evaluation on the static FER classification task with six basic emotions and neutral using the EfficientNet-B0 architecture. The best model achieved an accuracy of 69.84% on the test set, indicating that FER under HMD occlusion is feasible but significantly more challenging than conventional FER.
URI: https://hdl.handle.net/20.500.12738/17122
Review status: Only preprints: This version has not yet been reviewed
Institute: Department Medientechnik 
Fakultät Design, Medien und Information 
Type: Preprint
Appears in Collections:Publications without full text

Show full item record

Page view(s)

21
checked on Mar 14, 2025

Google ScholarTM

Check

HAW Katalog

Check

Add Files to Item

Note about this record


This item is licensed under a Creative Commons License Creative Commons