Associated with the Linz Institute of Technology and the Institute of Computational Perception @ JKU Linz, Austria

Multiobjective Hyperparameter Optimization of Recommender Systems

MultiObjectiveHyperOpt Teaser

Abstract

The quality of recommendations can be evaluated in terms of accuracy and beyond-accuracy metrics; this renders recommendation a multiobjective task. Several works apply multiobjective optimization techniques for training recommender systems (RSs) or for late fusion of recommendations. However, for the hyperparameter selection, only accuracy is considered. In this paper, we include metrics for accuracy, coverage, novelty, and fairness of recommendations towards groups of users of different activity, and items of different popularity, in the hyperparameter optimization of RSs. We apply the concept of Pareto dominance to select the optimal hyperparameter configurations. Then, by performing linear regressions of the values of beyond-accuracy metrics on the values of NDCG for the optimal hyperparameter configurations, we quantify the interplay of accuracy and beyond-accuracy metrics in terms of the the slope of the lines of best fit. Furthermore, by performing experiments in the domains of movie rating, music streaming, and food and household delivery and with four recommendation algorithms we provide insight in the generalizability of the interplay between accuracy and beyond-accuracy metrics. Our analysis shows that for 8 out of 12 combinations of algorithms and domains, the line of best fit for at least one beyond-accuracy metric has a negative slope, indicating a trade-off relationship and supporting the multiobjective hyperparameter optimization. Our analysis further shows that both the sign and the absolute value of the slope of the line of best fit depend on the recommendation algorithm as well as the recommendation domain, indicating the non-generalizability of the interplay between accuracy and beyond-accuracy metrics in the hyperparameter optimization.


Citation

Marta Moscati, Yashar Deldjoo, Giulio Davide Carparelli, Markus Schedl
Multiobjective Hyperparameter Optimization of Recommender Systems
Proceedings of the 3rd Workshop on Perspectives on the Evaluation of Recommender Systems co-located with the 17th ACM Conference on Recommender Systems (RecSys 2023), Singapore, Singapore., 3476: 2023.

BibTeX

@proceedings{Moscati2023MultiObjectiveHyperOpt,
    title = {Multiobjective Hyperparameter Optimization of Recommender Systems},
    author = {Moscati, Marta and Deldjoo, Yashar and Carparelli, Giulio Davide and Schedl, Markus},
    booktitle = {Proceedings of the 3rd Workshop on Perspectives on the Evaluation of Recommender Systems co-located with the 17th ACM Conference on Recommender Systems (RecSys 2023), Singapore, Singapore.},
    editor = {Said, Alan and Zangerle, Eva and Bauer, Christine},
    publisher = {CEUR-WS.org},
    url = {https://ceur-ws.org/Vol-3476/paper3.pdf},
    volume = {3476},
    year = {2023}
}