Abstract

Traditional recommender systems rely on collaborative filtering (CF), using past user-item interactions to help users discover new items in a vast collection. In cold start, i.e., when interaction histories of users or items are unavailable, content-based recommender systems (CBRSs) use side information instead, most commonly, descriptions of items for item cold start. Hybrid recommender systems (HRSs) often employ multimodal learning to combine collaborative and user and item side information, which we jointly refer to as modalities. Though HRSs can provide recommendations when some modalities are missing, their quality degrades. In this work, we utilize single-branch neural networks equipped with weight sharing, modality sampling, and contrastive loss to provide accurate recommendations even in missing modality scenarios, including cold start, by closing the modality gap. We compare these networks with multi-branch alternatives and conduct extensive experiments on the MovieLens 1M, Music4All-Onion, and Amazon Video Games datasets. Six accuracy-based and four beyond-accuracy-based metrics help assess the recommendation quality and popularity bias for the different training paradigms and their hyperparameters on single- and multi-branch networks in warm-start and missing modality scenarios. We quantitatively and qualitatively study the effects of these different aspects on bridging the modality gap. Our results show that single-branch networks provide competitive recommendation quality in warm start, and a significantly better one in missing modality scenarios. Moreover, weight sharing, modality sampling, and contrastive loss lead to modalities of an item being closer together in embedding space than to other items.

Citation

BibTeX

@misc{Ganhör2024master-thesis,
    title = {Single-Branch Network Architectures to Close the Modality Gap in Multimodal Recommendation},
    author = {Ganhör, Christian},
    school = {Johannes Kepler University Linz},
    year = {2024}
}