Vol. 6 No. 2 (2026): July−December 2026
Articles

A Comparative Evaluation of Rejection Inference Methods for Credit Application Screening: Inverse Probability Weighting, Hard-Cutoff Augmentation, and Semi-Supervised Self-Training

View PDF
  • Zhouting Mo,
  • Zhi Luo
Zhouting Mo
Computer Technology, Fudan University, Shanghai, China
Zhi Luo
Business Analytics, Columbia University, NY, USA

Published 2026-08-27

Keywords

  • rejection inference,
  • credit scoring,
  • sample selection bias,
  • semi-supervised learning

How to Cite

Mo, Z., & Luo, Z. (2026). A Comparative Evaluation of Rejection Inference Methods for Credit Application Screening: Inverse Probability Weighting, Hard-Cutoff Augmentation, and Semi-Supervised Self-Training. Spectrum of Research, 6(2). Retrieved from https://spectrumofresearch.com/index.php/sr/article/view/60

Copyright (c) 2026 Spectrum of Research

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Credit acquisition models are trained on applicants whose repayment outcomes are observable, a group that excludes everyone the underwriting policy declined. The resulting selection bias can skew risk estimates toward the accepted population and away from the full applicant pool. We compare three correction strategies, inverse probability weighting, hard-cutoff augmentation, and semi-supervised self-training, against an uncorrected baseline on three public credit-scoring datasets under two simulated selection mechanisms, missing at random (MAR) and missing not at random (MNAR), the latter driven directly by the hidden outcome, as the limiting case of an underwriter acting on off-record judgment that the model cannot observe. Under MAR, the baseline already approaches an oracle trained on all labels, and correction movements are small, the largest being a 0.018 drop for inverse probability weighting on German Credit, with most movements within a few thousandths, echoing the classical finding that reject inference offers limited gains when selection is ignorable. Under MNAR, the baseline degrades sharply and no correction fully recovers the oracle, with the ranking of methods shifting across datasets. The degradation is concentrated among thin-file applicants, whose predicted default rate is understated by more than half under MNAR, and the augmentation methods are sensitive to their confidence cutoff. The results indicate that the value of a rejection-inference correction depends less on the method family than on whether selection is ignorable, and that method choice matters most precisely where the data are least informative.

View PDF

References

  1. Crook, J. N., & Banasik, J. (2004). Does reject inference really improve the performance of application scoring models? *Journal of Banking & Finance*, 28(4), 857–874.
  2. Ehrhardt, A., Biernacki, C., Vandewalle, V., Heinrich, P., & Beben, S. (2021). Reject inference methods in credit scoring. *Journal of Applied Statistics*, 48(13–15), 2734–2754.
  3. Smith, A., & Elkan, C. (2004). A Bayesian network framework for reject inference. *Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, 286–295.
  4. Goel, N., Amayuelas, A., Deshpande, A., & Sharma, A. (2021). The importance of modeling data missingness in algorithmic fairness: A causal perspective. *Proceedings of the AAAI Conference on Artificial Intelligence*, 35(9), 7564–7573.
  5. Banasik, J., & Crook, J. N. (2005). Credit scoring, augmentation and lean models. *Journal of the Operational Research Society*, 56(9), 1072–1081.
  6. Chen, G. G., & Åstebro, T. (2012). Bound and collapse Bayesian reject inference for credit scoring. *Journal of the Operational Research Society*, 63(10), 1374–1387.
  7. Anderson, B. (2019). Using Bayesian networks to perform reject inference. *Expert Systems with Applications*, 137, 349–356.
  8. Heckman, J. J. (1979). Sample selection bias as a specification error. *Econometrica*, 47(1), 153–161.
  9. Zadrozny, B. (2004). Learning and evaluating classifiers under sample selection bias. *Proceedings of the 21st International Conference on Machine Learning*, 114–121.
  10. Huang, J., Smola, A. J., Gretton, A., Borgwardt, K. M., & Schölkopf, B. (2006). Correcting sample selection bias by unlabeled data. *Advances in Neural Information Processing Systems*, 19, 601–608.
  11. Sugiyama, M., Nakajima, S., Kashima, H., von Bünau, P., & Kawanabe, M. (2007). Direct importance estimation with model selection and its application to covariate shift adaptation. *Advances in Neural Information Processing Systems*, 20, 1433–1440.
  12. Kanamori, T., Hido, S., & Sugiyama, M. (2009). A least-squares approach to direct importance estimation. *Journal of Machine Learning Research*, 10, 1391–1445.
  13. Bickel, S., Brückner, M., & Scheffer, T. (2009). Discriminative learning under covariate shift. *Journal of Machine Learning Research*, 10, 2137–2155.
  14. Grandvalet, Y., & Bengio, Y. (2004). Semi-supervised learning by entropy minimization. *Advances in Neural Information Processing Systems*, 17, 529–536.
  15. Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. (2000). Text classification from labeled and unlabeled documents using EM. *Machine Learning*, 39(2–3), 103–134.
  16. Blum, A., & Mitchell, T. (1998). Combining labeled and unlabeled data with co-training. *Proceedings of the 11th Annual Conference on Computational Learning Theory*, 92–100.
  17. Liao, J., Wang, W., Xue, J., Lei, A., Han, X., & Lu, K. (2022). Combating sampling bias: A self-training method in credit risk models. *Proceedings of the AAAI Conference on Artificial Intelligence*, 36(11), 12566–12572.
  18. Kozodoi, N., Katsas, P., Lessmann, S., Moreira-Matias, L., & Papakonstantinou, K. (2019). Shallow self-learning for reject inference in credit scoring. *Machine Learning and Knowledge Discovery in Databases*, 516–532.
  19. Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., & Raffel, C. A. (2019). MixMatch: A holistic approach to semi-supervised learning. *Advances in Neural Information Processing Systems*, 32, 5050–5060.
  20. Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C. A., Cubuk, E. D., Kurakin, A., & Li, C.-L. (2020). FixMatch: Simplifying semi-supervised learning with consistency and confidence. *Advances in Neural Information Processing Systems*, 33.
  21. Tran, V.-T., & Aussem, A. (2015). A practical approach to reduce the learning bias under covariate shift. *Machine Learning and Knowledge Discovery in Databases*, 71–86.
  22. Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. *Advances in Neural Information Processing Systems*, 29.
  23. Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. Z. (2017). Algorithmic decision making and the cost of fairness. *Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, 797–806.