Deep Learning in Personalized Music Emotion Recognition

Music has a unique power to evoke strong emotions in us-bringing us to tears, lifting us into ecstasy or triggering vivid memories. Often described as a universal language, it conveys feelings that transcend words. But are machines, too, able to understand this language and capture emotions conveyed in music?

This book delves into the field of Musical Emotion Recognition (MER), aiming to develop a mathematical model to predict the emotional content of music. It explores the fundamentals of this interdisciplinary research area, including the relationship between music and emotions, mathematical representations of music and deep learning algorithms. Two MER models are developed and evaluated: one employing handcrafted audio features with a long short-term memory architecture and the other using embeddings from the pre-trained music understanding model MERT. Results show that MERT embeddings can enhance predictions compared to traditional handcrafted features. Additionally, driven by the subjectivity of musical emotions and the low inter-rater agreement of annotations, this book investigates personalized emotion recognition. The findings suggest that personalized models surpass the limitations of general MER systems and can even outperform a theoretically perfect general MER system.

About the author

Yannik Venohr is a Ph.D. candidate at the University of Würzburg and works with Prof. Christof Weiß in the Emmy Noether group on developing robust methods for computational musicology.