Associated with the Linz Institute of Technology and the Institute of Computational Perception @ JKU Linz, Austria

Emotion-aware Music Tower Blocks (EmoMTB): An Intelligent Audiovisual Interface for Music Discovery and Recommendation

IJMIR_EMOMTB Teaser

Abstract

Music listening has experienced a sharp increase during the last decade thanks to music streaming and recommendation services. While they offer text-based search functionality and provide recommendation lists of remarkable utility, their typical mode of interaction is unidimensional, i.e., they provide lists of consecutive tracks, which are commonly inspected in sequential order by the user. The user experience with such systems is heavily affected by cognition biases (e.g., position bias, human tendency to pay more attention to first positions of ordered lists) as well as algorithmic biases (e.g., popularity bias, the tendency of recommender systems to overrepresent popular items). This may cause dissatisfaction among the users by disabling them to find novel music to enjoy. In light of such systems and biases, we propose an intelligent audiovisual music exploration system named EmoMTB. It allows the user to browse the entirety of a given collection in a free non-linear fashion. The navigation is assisted by a set of personalized emotion-aware recommendations which serve as starting points for the exploration experience. EmoMTB adopts the metaphor of a city, in which each track (visualized as a colored cube) represents one floor of a building. Highly similar tracks are located in the same building, moderately similar ones form neighborhoods that mostly correspond to genres. Tracks situated between distinct neighborhoods create a gradual transition between genres. Users can navigate this music city using their smartphones as control devices. They can explore districts of well-known music or decide to leave their comfort zone. In addition, EmoMTB integrates an emotion-aware music recommendation system that re-ranks the list of suggested starting points for exploration according to the user’s self-identified emotion or the collective emotion expressed in EmoMTB’s Twitter channel. Evaluation of EmoMTB has been carried out in a three-fold way: by quantifying the homogeneity of the clustering underlying the construction of the city, by measuring the accuracy of the emotion predictor, and by carrying out a web-based survey composed of open questions to obtain qualitative feedback from users.


Citation

Alessandro B. Melchiorre, David Penz, Christian Ganhör, Oleg Lesota, Vasco Bezold Rosner Fragoso, Florian Friztl, Emilia Parada-Cabaleiro, Franz Schubert, Markus Schedl
Emotion-aware Music Tower Blocks (EmoMTB): An Intelligent Audiovisual Interface for Music Discovery and Recommendation
International Journal of Multimedia Information Retrieval (IJMIR), 12(13): doi:10.1007/s13735-023-00275-8, 2023.

BibTeX

@article{Melchiorre2023IJMIR_EMOMTB,
    title = {Emotion-aware Music Tower Blocks (EmoMTB): An Intelligent Audiovisual Interface for Music Discovery and Recommendation},
    author = {Melchiorre, Alessandro B. and Penz, David and Ganhör, Christian and Lesota, Oleg and Fragoso, Vasco Bezold Rosner and Friztl, Florian and Parada-Cabaleiro, Emilia and Schubert, Franz and Schedl, Markus},
    journal = {International Journal of Multimedia Information Retrieval (IJMIR)},
    doi = {10.1007/s13735-023-00275-8},
    url = {https://doi.org/10.1007/s13735-023-00275-8},
    volume = {12},
    number = {13},
    year = {2023}
}