Vol. 6 No. 1 (2026): January−June 2026
Articles

Predictive Power or Feature Stability? A Comparative Study of Filter, Embedded, and Wrapper Feature Selection for Consumer Credit Pre-Approval

View PDF
  • Zihan Li,
  • Kun Qian
Zihan Li
Computer Science, Northeastern University, CA, USA
Kun Qian
Business Intelligence, Engineering School of Information and Digital Technologies, Villejuif, France

Published 2026-06-09

Keywords

  • feature selection,
  • credit pre-approval,
  • selection stability,
  • dimensionality reduction

How to Cite

Li, Z., & Qian, K. (2026). Predictive Power or Feature Stability? A Comparative Study of Filter, Embedded, and Wrapper Feature Selection for Consumer Credit Pre-Approval. Spectrum of Research, 6(1). Retrieved from https://spectrumofresearch.com/index.php/sr/article/view/52

Copyright (c) 2026 Spectrum of Research

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Credit pre-approval scores prospective borrowers from candidate pools that can reach several hundred variables, and the feature-selection step that trims these pools shapes both predictive value and the reproducibility of the resulting scorecard. This study compares eight procedures spanning the filter, embedded, and wrapper families, together with a no-selection baseline, on three real public credit datasets that differ in dimensionality and class imbalance. Each method is assessed along three axes: discrimination measured by rank-based indices, selection stability measured by a corrected agreement index over bootstrap resamples, and efficiency measured by the retained feature fraction and selection runtime. Discrimination differences among the methods are moderate, with mean area under the curve between 0.727 and 0.737 for every method except the redundancy-minimizing filter, which trails at 0.704. Stability separates the methods far more sharply: the information-value filter yields the most reproducible subsets, random-forest importance is the most stable embedded method, and the L1-regularized, gradient-boosting, and recursive-elimination methods produce the least stable subsets, with the gap widening on the highest-dimensional data. The analysis indicates that for auditable pre-approval screening a univariate filter offers the most defensible balance of predictive value, stability, and cost, and that discrimination alone is an insufficient basis for choosing among selection families.

View PDF

References

  1. [1] Lessmann, S., Baesens, B., Seow, H. V., & Thomas, L. C. (2015). Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research. European Journal of Operational Research, 247(1), 124-136.
  2. [2] Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., & Vanthienen, J. (2003). Benchmarking state-of-the-art classification algorithms for credit scoring. Journal of the Operational Research Society, 54(6), 627-635.
  3. [3] Luo, Z. (2026). Fairness-Aware Credit Evaluation: Bias Detection and Mitigation Techniques for Inclusive Lending Practices. Journal of Sustainability, Policy, and Practice, 2(2), 78-89.
  4. [4] Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157-1182.
  5. [5] Zhao, F., & Tang, T. (2026). AI-Based Sentiment Analysis for Stock Market Prediction: A Systematic Literature Review. Journal of Sustainability, Policy, and Practice, 2(3), 115-124.
  6. [6] Li, Y., Zhao, F., & Hu, J. (2026). Identifying Cross-Market Risk Contagion Amplifiers via Graph Attention Networks: Empirical Evidence from US Financial Stress Periods. Journal of Computing Innovations and Applications, 4(1), 164-175.
  7. [7] Nogueira, S., Sechidis, K., & Brown, G. (2018). On the stability of feature selection algorithms. Journal of Machine Learning Research, 18(174), 1-54.
  8. [8] Weng, H., Yang, Q., Fu, H., Pan, H., & Lv, X. (2026). When Retrieval Hurts Code Completion: A Diagnostic Study of Stale Repository Context. arXiv preprint arXiv:2605.14478.
  9. [9] Jiang, Z., & Wang, M. (2025). Evaluation and Analysis of Chart Reasoning Accuracy in Multimodal Large Language Models: An Empirical Study on Influencing Factors. Pinnacle Academic Press Proceedings Series, 3, 43-58.
  10. [10] Wang, M., Xiao, P., & Yu, M. (2026). Sparse, Dense, or Hybrid? Comparing Retrieval Strategies for Biomedical Question Answering with Retrieval-Augmented Generation. Journal of Science, Innovation & Social Impact, 2(2), 141-152.
  11. [11] Han, M., Jin, Z., & Zou, D. (2026). Comparative Evaluation of Self-Supervised Pretraining Strategies for Few-Shot Medical Image Analysis. Artificial Intelligence and Machine Learning Review, 7(2), 23-42.
  12. [12] Peng, H., Long, F., & Ding, C. (2005). Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1226-1238.
  13. [13] Brown, G., Pocock, A., Zhao, M. J., & Lujan, M. (2012). Conditional likelihood maximisation: A unifying framework for information theoretic feature selection. Journal of Machine Learning Research, 13, 27-66.
  14. [14] Tang, T., & Yu, M. (2024). A Comparative Empirical Study of Semantic Signal Enhancement Methods for User Interest Features in CTR Prediction: Applicability of TF-IDF Weighting, Sentence-BERT Embeddings, and LDA Topic Fusion. Journal of Computing Innovations and Applications, 2(1), 165-174.
  15. [15] Tang, T., & Yu, M. (2024). A Comparative Evaluation of LLM-Generated Semantic Tags versus Classical Text Features (TF-IDF, LDA, BERT Embeddings) for User-Interest Enrichment in Short-Video Recommendation. Artificial Intelligence and Machine Learning Review, 5(1), 129-140.
  16. [16] Tang, T., Fu, X., & Luo, C. (2026). An Empirical Comparison of High-Order Feature Interaction Operators for Conversion Rate Prediction in Sparse, High-Cardinality Message-Ads Traffic: Accuracy, Efficiency, and Offline-Online Consistency. Journal of Science, Innovation & Social Impact, 2(3), 12-22.
  17. [17] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B, 58(1), 267-288.
  18. [18] Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97(1-2), 273-324.
  19. [19] Ming, Y., & Fu, H. (2020, July). Variational Bayesian sparsification for distillation compression. In 2020 IEEE International Conference on Multimedia and Expo (ICME) (pp. 1-6). IEEE.
  20. [20] Ming, Y., Fu, H., Jiang, Y., & Yu, H. (2020). Variational Bayesian Group-Level Sparsification for Knowledge Distillation. IEEE Access, 8, 126628-126636.
  21. [21] Fu, H., Ming, Y., Jiang, Y., & Fan, C. (2018, August). Effective SVD-based deep network compression for automatic speech recognition. In International Workshop on Face and Facial Expression Recognition from Real World Videos (pp. 35-47). Cham: Springer International Publishing.
  22. [22] Zhang, R., Fan, C., Ming, Y., Fu, H., & Meng, X. (2020, October). An Effective Hierarchical Resolution Learning Method for Low-Resolution Targets Tracking. In 2020 IEEE International Conference on Image Processing (ICIP) (pp. 2076-2080). IEEE.
  23. [23] Kalousis, A., Prados, J., & Hilario, M. (2007). Stability of feature selection algorithms: A study on high-dimensional spaces. Knowledge and Information Systems, 12(1), 95-116.
  24. [24] Hu, J., & Long, X. (2024). Graph Learning-Based Behavioral Detection for Software Supply Chain Attacks. Journal of Advanced Computing Systems, 4(4), 49-60.
  25. [25] Long, X., Hu, J., & Ling, Z. (2026). A Comparative Analysis of Telemetry-Driven Anomaly Detection Approaches for Dual-Purpose Operational and Security Optimization in Edge Computing Infrastructure. Journal of Computing Innovations and Applications, 4(1), 79-88.
  26. [26] Chen, Y., & Hu, J. (2026). Graph Neural Network-Based Cascading Disruption Path Identification in Multi-Tier Rare Earth Processing Networks. Journal of Global Engineering Review, 4(1), 99-112.
  27. [27] Hu, J., Wang, X., & Lai, J. (2026). Benchmarking Learned Cardinality Estimation Techniques for Analytical Query Processing in Data Warehouses. Journal of Computer Technology and Applied Mathematics, 3(3), 1-8.
  28. [28] Cao, H., Hu, J., & Luo, C. (2025). Behavioural Feature Analysis for Anomalous Click Detection in Mobile Advertising Environments: Toward In-App Browser-Specific Detection. Journal of Computing Innovations and Applications, 3(2), 96-105.
  29. [29] Long, L., & Hu, J. (2026). Multi-Objective Particle Swarm Optimization for Site Selection and Policy Subsidy Maximization of Foreign Renewable Energy Enterprises in the United States. Artificial Intelligence and Machine Learning Review, 7(2), 54-69.
  30. [30] Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B, 67(2), 301-320.
  31. [31] Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32.
  32. [32] Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794.
  33. [33] Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765-4777.
  34. [34] Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1-3), 389-422.
  35. [35] Luo, C., & Wang, X. (2026). Latency-Throughput Tradeoffs of ONNX Runtime, TensorRT-LLM, vLLM, and Triton: An Empirical Comparison on 1B-3B Parameter LLM Inference. Spectrum of Research, 6(1).
  36. [36] Nogueira, S., & Brown, G. (2016). Measuring the stability of feature selection. Machine Learning and Knowledge Discovery in Databases (ECML PKDD), LNCS 9852, 442-457.
  37. [37] Sechidis, K., Papangelou, K., Nogueira, S., Weatherall, J., & Brown, G. (2019). On the stability of feature selection in the presence of feature correlations. Machine Learning and Knowledge Discovery in Databases (ECML PKDD), LNCS 11906, 327-342.
  38. [38] Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S. I. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1), 56-67.
  39. [39] Meinshausen, N., & Buhlmann, P. (2010). Stability selection. Journal of the Royal Statistical Society: Series B, 72(4), 417-473.
  40. [40] Chen, Z., & Wang, M. (2025). Evaluating the Quality of Large Language Model-Generated Explanations in Recommendation Tasks: A Multi-Dimensional Comparative Analysis. Journal of Global Engineering Review, 3(1), 54-63.
  41. [41] Chen, Y., & Tang, T. (2026). Evaluating Prompt Engineering Strategies for Few-Shot Cyber Threat Intelligence Entity and Relation Extraction from Multi-Source Reports. Journal of Science, Innovation & Social Impact, 2(2), 153-164.
  42. [42] Fu, X., Tang, T., & Luo, C. (2026). An Empirical Comparison of ReAct, Reflexion, Plan-and-Solve, and Tree-of-Thought Planning Strategies on Financial Question Answering and Numerical Reasoning Tasks. Journal of Science, Innovation & Social Impact, 2(3), 23-34.
  43. [43] Wang, M., & Zhu, L. (2024). Linguistic analysis of verb tense usage patterns in computer science paper abstracts. Academia Nexus Journal, 3(3).
  44. [44] Wang, M., Jiang, Z., & Zhou, S. (2024). Cross-cultural semantic differences in emoji usage on social media platforms. Journal of Advanced Computing Systems, 4(5), 55-66.
  45. [45] Guo, Y., Li, Y., Chen, S., Wu, Y., Poll, O., Ren, Z., ... & Baran, P. S. (2025). Scalable total synthesis of saxitoxin and related natural products. Nature, 646(8084), 351-357.
  46. [46] Guo, Y., Li, Y., Chen, S., Poll, O., Wu, Y., Prier, C., ... & Baran, P. (2025). Scalable, convergent total synthesis of (+)-saxitoxin and related natural products.
  47. [47] Li, Y., Buchberger, B., Liu, Y., Capone, C., Adak, T., Ojha, S., ... & Baran, P. S. (2026). General Access to C-Glycosides via Redox-Neutral Radical Cross-Coupling of Glycohydrazides.