Identification of Facial Information Used for Emotion Recognition in Children and Adolescents with Non-Suicidal Self-Injurious Behavior, utilizing ResNet50 Models and Layerwise Relevance Propagation for Analysis
Wed-H6-Talk 9-9703
Presented by: Alexandra Otto
Alexandra Otto 1, Irina Jarvers 1, Stephanie Kandsperger 1, Romuald Brunner 1, Robert Bosek 2, Jens Schwarzbach 3, Gregor Volberg 2
1 Department of Child and Adolescent Psychiatry and Psychotherapy, University of Regensburg, Regensburg, Germany, 2 Cognitive Neuroscience, University of Regensburg, Regensburg, Germany, 3 Department of Biomedical Imaging, University of Regensburg, Regensburg, Germany
Approximately 18-22% of adolescents engage in non-suicidal self-injury (NSSI). While the consequences of NSSI are well-documented, emotion recognition mechanisms in NSSI and which facial information’s are used remain unclear. Therefore, this study explores the emotion recognition process preceding NSSI. Participants, comprising 42 patients and 43 controls, judged images of faces for emotional or neutral expressions across sessions featuring happy and sad valences. Employing the bubble technique, gaussian apertures at random locations unveiled different facial areas in distinct spatial frequencies in each trial for emotion classification. With over 33,000 trials per group and valence, images from correct emotion classification were used to train an image classifier (ResNet50) distinguishing emotional vs. neutral facial expressions. Cross-validation demonstrated deviating performances between the NSSI and control model in classifying sad facial expressions, with the NSSI model misclassifying sad expressions more frequently as neutral. Through layer-wise relevance propagation, facial features around the left eye and mouth regions were identified as contributors to the lower accuracy of emotion perception in the NSSI model. Conversely, the right eye and forehead region contributed to a more accurate classification of the control model. Further analyses revealed components of emotional competence, particularly attitudes towards emotions, as predictors for the non-utilization of specific facial regions for classification of sad expressions. The observed disparity in identifying sad emotions suggests a potential avoidance strategy at perception among adolescents with NSSI, wherein visual facial information related to sad emotions might be disregarded as a mechanism for emotion regulation and a divergent attitude toward emotions.
Keywords: Emotion Perception, NSSI, machine learning