Data Pruning and Classification Bias in Deep Learning

Table of Links

Abstract and 1 Introduction

2 Data Pruning & Fairness

3 Method & Results

4 Theoretical Analysis

5 Discussion, Acknowledgments and Disclosure of Funding, and References

A Implementation Details

B Theoretical Analysis for a Mixture of Gaussians

2 Data Pruning & Fairness

As deep learning models become more data-hungry, researchers and practitioners are focusing extensively on improving data efficiency. Indeed, there is much room for improvement: it is generally accepted that large-scale datasets have many redundant, noisy, or uninformative samples that may even have a negative effect on learning, if any [He et al., 2023, Chaudhuri et al., 2023]. Thus, it is imperative to carefully curate a subset of all available data for learning purposes. The corresponding literature is exceptionally rich, with a few fruitful and relevant research threads. Dataset distillation replaces the original examples with synthetically generated samples that bear compressed, albeit not as much interpretable, training signal [Sucholutsky and Schonlau, 2021, Cazenavette et al., 2022, Such et al., 2020, Zhao and Bilen, 2023, Nguyen et al., 2021, Feng et al., 2024]. Coreset methods select representative samples that jointly capture the entire data manifold [Sener and Savarese, 2018, Guo et al., 2022, Zheng et al., 2023, Agarwal et al., 2020, Mirzasoleiman et al., 2020, Welling, 2009]; they yield weak generalization guarantees for non-convex problems and are not too effective in practice [Feldman, 2020, Paul et al., 2021]. Active learning iteratively selects an informative subset of a larger pool of unlabeled data for annotation, which is ultimately used for supervised learning [Tharwat and Schenck, 2023, Ren et al., 2021, Beluch et al., 2018, Kirsch et al., 2019]. Subsampling deletes instances of certain groups or classes when datasets are imbalanced, aiming to reduce bias and improve robustness of the downstream classifiers [Chawla et al., 2002, Barua et al., 2012, Chaudhuri et al., 2023].

Fairness & Evaluation. Fairness in machine learning concerns the issue of non-uniform accuracy over the data distribution [Caton and Haas, 2023]. The majority of research focuses on group fairness where certain, often under-represented or sensitive, groups of the population enjoy worse predictive performance [Hashimoto et al., 2018, Thomas McCoy et al., 2020]. In general, groups are subsets of classes, and worst-group accuracy is a standard optimization criterion in this domain [Sagawa* et al., 2020, Kirichenko et al., 2023, Rudner et al., 2024]. The focus of this study is a special case of group fairness, classification bias, where groups are directly defined by class attributions. Classification bias commonly arises in the context of imbalanced datasets where tail classes require upsampling or importance weighting to produce models with strong worst-class performance [Cui et al., 2019, Barua et al., 2012, Tan et al., 2020, Chaudhuri et al., 2023]. For balanced datasets, classification bias is found to be exacerbated by adversarial training [Li and Liu, 2023, Benz et al., 2021, Xu et al., 2021, Nanda et al., 2021, Ma et al., 2022] and network pruning [Paganini, 2020, Joseph et al., 2020, Tran et al., 2022, Good et al., 2022]. These works measure classification bias in various ways ranging from visual inspection to confidence intervals for the slope of the linear regression that fits accuracy deviations around the mean across classes. Another common approach is to measure the class- or group-wise performance gaps relative to the baseline model (without adversarial training or pruning) [Hashemizadeh et al., 2023, Dai et al., 2023, Lin et al., 2022]. However, this metric ignores the fact that this original model can be and often is highly unfair itself, and counts any potential improvement over it as unfavorable. Instead, we utilize a simpler and more natural suite of metrics to measure classification bias. Given accuracy (recall) rk for each class k ∈ [K], we report (1) worst-class accuracy mink rk, (2) difference between the maximum and the minimum recall maxk rk-mink rk [Joseph et al., 2020], and (3) standard deviation of recalls stdkrk [Ma et al., 2022].

Data Pruning Meets Fairness. Although existing data pruning techniques have proven to achieve strong average generalization performance, no prior work systematically evaluated their robustness to classification bias. Among related works, Pote et al. [2023] studied how EL2N pruning affects the class-wise performance and found that, at high data density levels (e.g., 80–90% remaining data), under-performing classes improve their accuracy compared to training on the full dataset. Zayed et al. [2023] propose a modification of EL2N to achieve fairness across protected groups with two attributes (e.g., male and female) in datasets with counterfactual augmentation, which is a rather specific setting. In this study, we eliminate this blind spot in the data pruning literature and analyze the trade-off between the average performance and classification bias exhibited by some of the most common data pruning methods: EL2N and GraNd [Paul et al., 2021], Influence [Yang et al., 2023], Forgetting [Toneva et al., 2019], Dynamic Uncertainty [He et al., 2023], and CoreSet [Sener and Savarese, 2018]. We present the experimental details and hyperparameters in Appendix A.

Data Pruning is Currently Unfair. We conduct the first comprehensive comparative study of data pruning through the lens of class-wise fairness (Figure 1). First, we note that no pruning method is uniformly state-of-the-art, and the results vary considerably across model-dataset pairs. Among all algorithms, Dynamic Uncertainty with VGG-19 on CIFAR-100 presents a particularly interesting and instructive case. While it is arguably the best with respect to the average test performance, it fails miserably across all fairness metrics. Figure 2 reveals that it actually removes entire classes already at 10% of CIFAR-100. Therefore, it seems plausible that Dynamic Uncertainty simply sacrifices the most difficult classes to retain strong performance on the easier ones. Figure 3 confirms our hypothesis: in contrast to other algorithms, at low density levels, Dynamic Uncertainty tends to prune classes with lower baseline accuracy (obtained from training on the full dataset) more aggressively, which entails a catastrophic classification bias hidden underneath a deceptively strong average accuracy. This observation presents a compelling argument for using criteria beyond average test performance for data pruning algorithms, particularly emphasizing classification bias.

Overall, existing algorithms exhibit poor robustness to bias, although several of them improve ever slightly over the full dataset. In particular, EL2N and GraNd achieve a relatively low classification bias, closely followed by Forgetting. At the same time, Forgetting has a substantially stronger average

test accuracy compared to these three methods, falling short of Random only after pruning more than 60% from CIFAR-100. Forgetting produces more balanced datasets than EL2N and GraNd at low densities (Figure 2), and tends to prune “easier” classes more aggressively compared to all other methods (Figure 3). These two properties seem to be beneficial, especially when the available training data is scarce.

While the results in Figure 1 uncover strong classification bias, existing literature hints that data pruning should in fact be effective in mitigating the classification bias because it bears similarity to some fairness-driven optimization approaches. The majority of well-established and successful algorithms that optimize for worst-group performance apply importance weighting to effectively re-balance the long-tailed data distributions (cf. cost-sensitive learning [Elkan, 2001]). Taking this paradigm to an extreme, data pruning assigns zero weights to the pruned samples in the original dataset, trading some learning signal for faster convergence, and resource and memory efficiency. Furthermore, similarly to EL2N or Forgetting, the majority of cost-sensitive methods estimate classor group-specific weights according to their difficulty. Thus, Sagawa* et al. [2020] optimize an approximate minimax DRO (distributionally robust) objective using a weighted sum of group-wise losses, putting higher mass on high-loss groups; Sinha et al. [2022] weigh samples by the current class-wise misclassification rates measured on a holdout validation set; Liu et al. [2021] pre-train a reference model to estimate the importance coefficients of groups for subsequent re-training. Similar cost-weighting strategies are adopted for robust knowledge distillation [Wang et al., 2023, Lukasik et al., 2022] and online batch selection [Kawaguchi and Lu, 2020, Mindermann et al., 2022]. These techniques compute weights at the level of classes or sensitive groups and not for individual training samples. In the context of data pruning, this would correspond to selecting the appropriate density ratios across classes but subsampling randomly otherwise. While existing pruning algorithms automatically balance classes according to their validation errors fairly well (Figure 3), their overly scrupulous filtering within classes may be suboptimal. In support of this view, Ayed and Hayou [2023] formally show that integrating random sampling into score-based pruning procedures improves their average performance. In fact, random pruning is regarded as a notoriously strong baseline in active learning and data pruning (see Figure 1). In the next section, we propose and evaluate a random pruning algorithm that respects difficulty-based class-wise ratios.