Calibrating Recommendations on Ordinal Attributes

Abstract

A recommendation list is said to be calibrated with respect to an item attribute if the distribution of the attribute over recommended items matches the distribution over the user's consumed items. Calibration techniques were introduced for non-ordinal attributes such as genres, and later applied to ordinal ones like popularity. However, when the support of the distributions is ordinal, i.e., when the order of bins matters, current calibration techniques may fall short in capturing distribution displacements along the ordinal axis. To address this limitation, we propose to use order-sensitive measures of divergences between distributions. With quantitative experiments on movie and music recommendation, we show that standard, order-insensitive calibration metrics are not able to fully capture the miscalibration of ordinal attributes in recommendation lists. We then propose order-sensitive objectives for post-processing calibration and compare their impact on recommendations with that of order-insensitive approaches. We show that order-sensitive approaches reach a better accuracy-calibration tradeoff. With this work we aim to start the discussion on how to appropriately model order-sensitive item attribute for calibrated recommendations.

Citation

Marta Moscati, Varvara Toloknova, Oleg Lesota, Markus Schedl
Calibrating Recommendations on Ordinal Attributes
Proceedings of the 20th ACM Conference on Recommender Systems (RecSys 2026), 2026.

BibTeX

@inproceedings{Moscati2026OrdinalCalibration,
    title = {Calibrating Recommendations on Ordinal Attributes},
    author = {Moscati, Marta and Varvara Toloknova and Lesota, Oleg and Schedl, Markus},
    booktitle = {Proceedings of the 20th ACM Conference on Recommender Systems (RecSys 2026)},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    location = {Minneapolis, Minnesota, US},
    month = {September},
    year = {2026}
}