Abstract

This thesis evaluates whether coreset-based data selection (where a coreset is a subset of the original training set) can reduce demographic performance gaps in encoder-only transformer text classifiers without substantially reducing balanced accuracy. We compare four selection strategies (Active Learning (AL), K-Center Greedy (KCG), K-Means, and a Hybrid strategy combining KCG and AL) using a fixed coreset size of 25% across five encoder-only transformer backbones on three classification datasets (PAN16, FDCL18, and BIOS). Each strategy is evaluated against the corresponding full-training-set baseline classifier for each backbone. We quantify demographic disparity using balanced-accuracy gaps between groups under predefined demographic train–test scenarios, and we report overall balanced accuracy alongside these gap metrics. The results show that coreset selection can meaningfully change demographic disparities, but the direction and magnitude of the effect vary across datasets and demographic variables. Overall, KCG and Hybrid emerge most consistently as strong accuracy–fairness trade-offs, though no single strategy dominates across all datasets. These findings suggest that coreset selection can be a useful data-centric intervention in this setting, while highlighting limits to generalization beyond the tasks and metrics evaluated here.

Citation

BibTeX

@misc{MahmoudMohammadArefBarham2026master-thesis,
    title = {The Effect of Coresets on Demographic Bias in Encoder-Only Transformer Models for Text Classification},
    author = {Mahmoud Mohammad Aref Barham},
    school = {Johannes Kepler University Linz},
    year = {2026}
}