Vol. 4 No. 2 (2024): July−December 2024
Articles

An Empirical Study on the Quality Assessment of Code Comments Generated by Large Language Models for Different Programming Paradigms

View PDF
  • Pengfei Li,
  • Wenyan Liu,
  • Qichang Zheng
Pengfei Li
Software Engineering, Duke University, NC, USA
Wenyan Liu
Electrical & Computer Engineering, Carnegie Mellon University, PA, USA
Qichang Zheng
Computational Social Science, University of Chicago, IL, USA

Published 2024-09-24

Keywords

  • Large Language Models,
  • Code Comment Generation,
  • Programming Paradigms,
  • Quality Assessment

How to Cite

Li, P., Liu, W., & Zheng, Q. (2024). An Empirical Study on the Quality Assessment of Code Comments Generated by Large Language Models for Different Programming Paradigms. Spectrum of Research, 4(2). Retrieved from https://spectrumofresearch.com/index.php/sr/article/view/32

Copyright (c) 2024 Spectrum of Research

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

The proliferation of Large Language Models (LLMs) in software engineering has introduced unprecedented opportunities for automated code documentation. This empirical study investigates the quality variations of code comments generated by contemporary LLMs across distinct programming paradigms including object-oriented, functional, and procedural approaches. Through systematic evaluation of comment generation performance using established quality metrics, this research examines the paradigm-specific strengths and limitations of current LLM architectures. The study employs a comprehensive experimental framework encompassing code samples from multiple programming languages and domains, with quality assessment based on accuracy, completeness, readability, and contextual relevance. Findings reveal significant disparities in comment quality across programming paradigms, with object-oriented constructs demonstrating superior documentation fidelity compared to functional programming patterns. The research contributes empirical evidence regarding LLM capabilities in automated documentation generation and provides actionable insights for improving code comment quality in software development workflows. These results have substantial implications for the integration of AI-assisted documentation tools in diverse programming environments and highlight critical areas requiring targeted improvements in current LLM training methodologies.

View PDF

References

  1. H. McNichols, M. Zhang, and A. Lan, "Algebra error classification with large language models," in International Conference on Artificial Intelligence in Education, Springer Nature Switzerland, June 2023, pp. 365-376.
  2. Y. Li, X. Jiang, and Y. Wang, "TRAM-FIN: A Transformer-Based Real-time Assessment Model for Financial Risk Detection in Multinational Corporate Statements," Journal of Advanced Computing Systems, vol. 3, no. 9, pp. 54-67, 2023.
  3. Z. Wang, X. Wang, and H. Wang, "Temporal Graph Neural Networks for Money Laundering Detection in Cross-Border Transactions," Academia Nexus Journal, vol. 3, no. 2, 2024.
  4. M. Wang, P. Xue, Y. Li, and Z. Wu, "Distilling the documents for relation extraction by topic segmentation," in Document Analysis and Recognition–ICDAR 2021: 16th International Conference, Lausanne, Switzerland, September 5–10, 2021, Proceedings, Part I 16, Springer International Publishing, 2021, pp. 517-531.
  5. Y. Chen, C. Ni, and H. Wang, "AdaptiveGenBackend A Scalable Architecture for Low-Latency Generative AI Video Processing in Content Creation Platforms," Annals of Applied Sciences, vol. 5, no. 1, 2024.
  6. A. Kang, J. Xin, and X. Ma, "Anomalous Cross-Border Capital Flow Patterns and Their Implications for National Economic Security: An Empirical Analysis," Journal of Advanced Computing Systems, vol. 4, no. 5, pp. 42-54, 2024.
  7. K. Mo et al., "Precision kinematic path optimization for high-dof robotic manipulators utilizing advanced natural language processing models," in 2024 5th International Conference on Electronic Communication and Artificial Intelligence (ICECAI), IEEE, May 2024, pp. 649-654.
  8. J. Y. Shih and Z. H. Chin, "A Fairness Approach to Mitigating Racial Bias of Credit Scoring Models by Decision Tree and the Reweighing Fairness Algorithm," in 2023 IEEE 3rd International Conference on Electronic Communications, Internet of Things and Big Data (ICEIB), IEEE, April 2023, pp. 100-105.
  9. Y. Zhao et al., "Unit operation combination and flow distribution scheme of water pump station system based on Genetic Algorithm," Applied Sciences, vol. 13, no. 21, p. 11869, 2023.
  10. M. Zhang, Z. Wang, R. G. Baraniuk, and A. S. Lan, "Scientific formula retrieval via tree embeddings," in 2021 IEEE International Conference on Big Data (Big Data), IEEE, December 2021, pp. 1493-1503.
  11. J. Wang, L. Guo, and K. Qian, "LSTM-based heart rate dynamics prediction during aerobic exercise for elderly adults," 2025.
  12. S. Zhang, T. Mo, and Z. Zhang, "LightPersML: A Lightweight Machine Learning Pipeline Architecture for Real-Time Personalization in Resource-Constrained E-commerce Businesses," Journal of Advanced Computing Systems, vol. 4, no. 8, pp. 44-56, 2024.
  13. D. Qi et al., "Anomaly explanation using metadata," in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, March 2018, pp. 1916-1924.
  14. A. A. H. Raji, A. H. F. Alabdoon, and A. Almagtome, "AI in Credit Scoring and Risk Assessment: Enhancing Lending Practices and Financial Inclusion," in 2024 International Conference on Knowledge Engineering and Communication Systems (ICKECS), IEEE, April 2024, vol. 1, pp. 1-7.
  15. C. Ni, J. Wu, and H. Wang, "Energy-Aware Edge Computing Optimization for Real-Time Anomaly Detection in IoT Networks," Applied and Computational Engineering, vol. 139, pp. 42-53, 2025.
  16. G. Wei et al., "Lateral bracing of beams provided by standing seam roof system: concepts and case study," 2020.
  17. T. K. Trinh and H. Wang, "Behavioral Responses to AI Financial Advisors: Trust Dynamics and Decision Quality Among Retail Investors," Applied and Computational Engineering, vol. 144, pp. 69-79, 2025.
  18. S. Wu et al., "Knowledge-Aware Dialogue Generation via Hierarchical Infobox Accessing and Infobox-Dialogue Interaction Graph Network," in IJCAI, August 2021, pp. 3964-3970.
  19. S. Zhang, Z. Feng, and B. Dong, "LAMDA: Low-Latency Anomaly Detection Architecture for Real-Time Cross-Market Financial Decision Support," Academia Nexus Journal, vol. 3, no. 2, 2024.
  20. D. Chowdhury and P. Kulkarni, "Application of data analytics in risk management of fintech companies," in 2023 International Conference on Innovative Data Communication Technologies and Application (ICIDCA), IEEE, March 2023, pp. 384-389.
  21. J. Fan, T. K. Trinh, and H. Zhang, "Deep Learning-Based Transfer Pricing Anomaly Detection and Risk Alert System for Pharmaceutical Companies: A Data Security-Oriented Approach," Journal of Advanced Computing Systems, vol. 4, no. 2, pp. 1-14, 2024.
  22. C. Zhu, J. Xin, and T. K. Trinh, "Data Quality Challenges and Governance Frameworks for AI Implementation in Supply Chain Management," Pinnacle Academic Press Proceedings Series, vol. 2, pp. 28-43, 2025.
  23. S. Wu et al., "Improving the applicability of knowledge-enhanced dialogue generation systems by using heterogeneous knowledge from multiple sources," in Proceedings of the fifteenth ACM international conference on WEB search and data mining, February 2022, pp. 1149-1157.
  24. C. Zhu, C. Cheng, and S. Meng, "DRL PricePro: A Deep Reinforcement Learning Framework for Personalized Dynamic Pricing in E-commerce Platforms with Supply Constraints," Spectrum of Research, vol. 4, no. 1, 2024.
  25. M. R. Eatherton et al., "Considering ductility in the design of bare deck and concrete on metal deck diaphragms," in The 17th World Conference on Earthquake Engineering, Sendai, Japan.
  26. M. Zhang, T. Mathew, and B. Juba, "An improved algorithm for learning to perform exception-tolerant abduction," in Proceedings of the AAAI Conference on Artificial Intelligence, February 2017, vol. 31, no. 1.
  27. L. Yan, Y. Wang, L. Guo, and K. Qian, "Enhanced Spatio-Temporal Attention Mechanism for Video Anomaly Event Detection," 2025.
  28. G. Wei et al., "Seismic Design of Diaphragms for Steel Buildings Considering Diaphragm Inelasticity," Journal of Structural Engineering, vol. 149, no. 7, p. 04023077, 2023.
  29. J. Chen and Z. Lv, "Graph Neural Networks for Critical Path Prediction and Optimization in High-Performance ASIC Design: A ML-Driven Physical Implementation Approach," in Global Conference on Advanced Science and Technology, April 2025, vol. 1, no. 1, pp. 23-30.
  30. C. Zhu, J. Xin, and D. Zhang, "A Deep Reinforcement Learning Approach to Dynamic E-commerce Pricing Under Supply Chain Disruption Risk," Annals of Applied Sciences, vol. 5, no. 1, 2024.
  31. S. Yan, "Design of Obstacle Avoidance System for the Blind based on Fuzzy Control," Netinfo Security, 2014.
  32. C. Ni, K. Qian, J. Wu, and H. Wang, "Contrastive Time-Series Visualization Techniques for Enhancing AI Model Interpretability in Financial Risk Assessment," 2025.
  33. T. K. Trinh and Z. Wang, "Dynamic Graph Neural Networks for Multi-Level Financial Fraud Detection: A Temporal-Structural Approach," Annals of Applied Sciences, vol. 5, no. 1, 2024.
  34. G. Wei et al., "Investigating partial tension field action in gable frame panel zones," Journal of Constructional Steel Research, vol. 162, p. 105746, 2019.
  35. L. Zhu, H. Yang, and Z. Yan, "Extracting temporal information from online health communities," in Proceedings of the 2nd International Conference on Crowd Science and Engineering, July 2017, pp. 50-55.
  36. J. Wu, H. Wang, K. Qian, and E. Feng, "Optimizing Latency-Sensitive AI Applications Through Edge-Cloud Collaboration," Journal of Advanced Computing Systems, vol. 3, no. 3, pp. 19-33, 2023.
  37. Z. Wang et al., "Temporal Evolution of Sentiment in Earnings Calls and Its Relationship with Financial Performance," Applied and Computational Engineering, vol. 141, pp. 195-206, 2025.
  38. H. Foroughi et al., "Seismic Demands on Steel Diaphragms for 3D Archetype Buildings with Concentric Braced Frames."
  39. H. Wang et al., "Automated Compliance Monitoring: A Machine Learning Approach for Digital Services Act Adherence in Multi-Product Platforms," Applied and Computational Engineering, vol. 147, pp. 14-25, 2025.
  40. M. Li, W. Liu, and C. Chen, "Adaptive Financial Literacy Enhancement through Cloud-Based AI Content Delivery: Effectiveness and Engagement Metrics," Annals of Applied Sciences, vol. 5, no. 1, 2024.
  41. H. Foroughi et al., "Seismic response predictions from 3D steel braced frame building simulations."
  42. M. Zhang, S. Baral, N. Heffernan, and A. Lan, "Automatic short math answer grading via in-context meta-learning," arXiv preprint arXiv:2205.15219, 2022.
  43. C. Ju and G. Rao, "Analyzing Foreign Investment Patterns in the US Semiconductor Value Chain Using AI-Enabled Analytics: A Framework for Economic Security," Pinnacle Academic Press Proceedings Series, vol. 2, pp. 60-74, 2025.
  44. J. Wu et al., "Graph Neural Networks for Efficient Clock Tree Synthesis Optimization in Complex SoC Designs," Applied and Computational Engineering, vol. 150, pp. 101-111, 2025.
  45. M. Zhang, N. Heffernan, and A. Lan, "Modeling and Analyzing Scorer Preferences in Short-Answer Math Questions," arXiv preprint arXiv:2306.00791, 2023.
  46. M. Zhang, Z. Wang, R. Baraniuk, and A. Lan, "Math operation embeddings for open-ended solution analysis and feedback," arXiv preprint arXiv:2104.12047, 2021.
  47. Xiong, K., Wu, Z., & Jia, X. (2025). DeepContainer: A Deep Learning-based Framework for Real-time Anomaly Detection in Cloud-Native Container Environments. Journal of Advanced Computing Systems, 5(1), 1-17.
  48. Wu, Z., Wang, S., Ni, C., & Wu, J. (2024). Adaptive Traffic Signal Timing Optimization Using Deep Reinforcement Learning in Urban Networks. Artificial Intelligence and Machine Learning Review, 5(4), 55-68.
  49. Wu, Z., Feng, E., & Zhang, Z. (2024). Temporal-Contextual Behavioral Analytics for Proactive Cloud Security Threat Detection. Academia Nexus Journal, 3(2).
  50. Wu, Z., Feng, Z., & Dong, B. (2024). Optimal Feature Selection for Market Risk Assessment: A Dimensional Reduction Approach in Quantitative Finance. Journal of Computing Innovations and Applications, 2(1), 20-31.
  51. Zhang, Z., & Wu, Z. (2023). Context-Aware Feature Selection for User Behavior Analytics in Zero-Trust Environments. Journal of Advanced Computing Systems, 3(5), 21-33.
  52. Wu, Z., Zhang, Z., Zhao, Q., & Yan, L. (2025). Privacy-Preserving Financial Transaction Pattern Recognition: A Differential Privacy Approach. Applied and Computational Engineering, 146, 30-40.
  53. Rao, G., Zheng, S., & Guo, L. (2025). Dynamic Reinforcement Learning for Suspicious Fund Flow Detection: A Multi-layer Transaction Network Approach with Adaptive Strategy Optimization. Applied and Computational Engineering, 145, 1-11.
  54. Rao, G., Wang, Z., & Liang, J. (2025). Reinforcement Learning for Pattern Recognition in Cross-Border Financial Transaction Anomalies: A Behavioral Economics Approach to AML. Applied and Computational Engineering, 142, 116-127.
  55. Ju, C., & Rao, G. (2025). Analyzing Foreign Investment Patterns in the US Semiconductor Value Chain Using AI-Enabled Analytics: A Framework for Economic Security. Pinnacle Academic Press Proceedings Series, 2, 60-74.
  56. Rao, G., Lu, T., Yan, L., & Liu, Y. (2024). A Hybrid LSTM-KNN Framework for Detecting Market Microstructure Anomalies:: Evidence from High-Frequency Jump Behaviors in Credit Default Swap Markets. Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online), 3(4), 361-371.
  57. Rao, G., Trinh, T. K., Chen, Y., Shu, M., & Zheng, S. (2024). Jump Prediction in Systemically Important Financial Institutions' CDS Prices. Spectrum of Research, 4(2).
  58. Rao, G., Ju, C., & Feng, Z. (2024). AI-Driven Identification of Critical Dependencies in US-China Technology Supply Chains: Implications for Economic Security Policy. Journal of Advanced Computing Systems, 4(12), 43-57.
  59. Liu, W., Rao, G., & Lian, H. (2023). Anomaly Pattern Recognition and Risk Control in High-Frequency Trading Using Reinforcement Learning. Journal of Computing Innovations and Applications, 1(2), 47-58.