Vol. 4 No. 1 (2024): January−June 2024
Articles

Cultural Bias Mitigation in Vision-Language Models for Digital Heritage Documentation: A Comparative Analysis of Debiasing Techniques

View PDF
  • Caiqian Cheng,
  • Yue Xi,
  • Suchuan Xing,
  • Yizhe Chen
Caiqian Cheng
Computer Science, University of California, San Diego, CA, USA
Yue Xi
Information Systems, Northeastern Unversity, WA, USA
Suchuan Xing
Electrical and Computer Engineering, Duke university, NC, USA
Yizhe Chen
Computer Science, University of California, San Diego, CA, USA

Published 2024-05-20

Keywords

  • Vision-language models,
  • cultural bias mitigation,
  • digital heritage documentation,
  • cross-modal adapters

How to Cite

Cheng, C., Xi, Y., Xing, S., & Chen, Y. (2024). Cultural Bias Mitigation in Vision-Language Models for Digital Heritage Documentation: A Comparative Analysis of Debiasing Techniques. Spectrum of Research, 4(1). Retrieved from https://spectrumofresearch.com/index.php/sr/article/view/10

Copyright (c) 2024 Spectrum of Research

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Although vision-language models have demonstrated remarkable capabilities in digital heritage documentation, they exhibit persistent cultural biases that compromise equitable representation of diverse cultural traditions. This study presents a systematic comparative analysis of debiasing techniques for vision-language models in heritage documentation contexts, categorizing approaches into data-level interventions, model-level modifications, and post-processing methods. We introduce Heritage-Bias, a specialized dataset containing 18,750 digitized artifacts from 15 cultural traditions with controlled variation in artifact attributes and contextual descriptions. Quantitative evaluation across multiple bias dimensions demonstrates that cross-modal adapter approaches achieve superior performance in preserving cultural nuance while reducing bias (47.2% reduction with 0.87 cultural attribute preservation). Combined interventions integrating counterfactual data generation with cross-modal adapters yield the most substantial improvements (53.8% overall bias reduction). Geo-cultural bias proves more resistant to mitigation than gender or skin tone bias, requiring specialized interventions incorporating domain expertise. Implementation analysis reveals context-dependent effectiveness patterns, with balanced dataset construction and output calibration serving as effective initial interventions for resource-constrained heritage institutions. Our findings establish a methodological framework for evaluating and addressing cultural bias in computational heritage documentation, promoting more equitable representation of global cultural heritage in digital preservation efforts.

View PDF

References

  1. Yang, J., Li, Z., Xie, S., Zhu, W., Yu, W., & Li, S. (2024, July). Cross-modal adapter: Parameter-efficient transfer learning approach for vision-language models. In 2024 IEEE International Conference on Multimedia and Expo (ICME) (pp. 1-6). IEEE.
  2. Wang, Y., & Liu, H. (2023, July). De-Biasing Methods in Neural Networks: A Survey. In 2023 International Conference on Machine Learning and Cybernetics (ICMLC) (pp. 458-463). IEEE.
  3. Mu, J., Niu, L., & Zhang, Y. (2024, May). Multimodal Recognition of Landmarks Based on Vision Language Model. In 2024 6th International Conference on Communications, Information System and Computer Engineering (CISCE) (pp. 455-458). IEEE.
  4. Yuan, Z., Xie, F., & Ji, T. (2024, September). Patrol Agent: An Autonomous UAV Framework for Urban Patrol Using on Board Vision Language Model and on Cloud Large Language Model. In 2024 6th International Conference on Robotics and Computer Vision (ICRCV) (pp. 237-242). IEEE.
  5. Huang, D., Yang, M., & Zheng, W. (2024). Using Deep Reinforcement Learning for Optimizing Process Parameters in CHO Cell Cultures for Monoclonal Antibody Production. Artificial Intelligence and Machine Learning Review, 5(3), 12-27.
  6. Jiang, C., Zhang, H., & Xi, Y. (2024). Automated Game Localization Quality Assessment Using Deep Learning: A Case Study in Error Pattern Recognition. Journal of Advanced Computing Systems, 4(10), 25-37.
  7. Huang, T., Xu, Z., Yu, P., Yi, J., & Xu, X. (2025). A Hybrid Transformer Model for Fake News Detection: Leveraging Bayesian Optimization and Bidirectional Recurrent Unit. arXiv preprint arXiv:2502.09097.
  8. Weng, J., Jiang, X., & Chen, Y. (2024). Real-time Squat Pose Assessment and Injury Risk Prediction Based on Enhanced Temporal Convolutional Neural Networks.
  9. Xu, X., Yu, P., Xu, Z., & Wang, J. (2025). A hybrid attention framework for fake news detection with large language models. arXiv preprint arXiv:2501.11967.
  10. Bi, W., Trinh, T. K., & Fan, S. (2024). Machine Learning-Based Pattern Recognition for Anti-Money Laundering in Banking Systems. Journal of Advanced Computing Systems, 4(11), 30-41.
  11. Shen, Q., Zhang, Y., & Xi, Y. (2024). Deep Learning-Based Investment Risk Assessment Model for Distributed Photovoltaic Projects. Journal of Advanced Computing Systems, 4(3), 31-46.
  12. Ma, X., Bi, W., Li, M., Liang, P., & Wu, J. (2025). An Enhanced LSTM-based Sales Forecasting Model for Functional Beverages in Cross-Cultural Markets. Applied and Computational Engineering, 118, 55-63.
  13. Wang, J., Zhao, Q., & Xi, Y. (2025). Cross-lingual Search Intent Understanding Framework Based on Multi-modal User Behavior. Annals of Applied Sciences, 6(1).
  14. Ju, C. (2023). A Machine Learning Approach to Supply Chain Vulnerability Early Warning System: Evidence from US Semiconductor Industry. Journal of Advanced Computing Systems, 3(11), 21-35.
  15. Ma, X., & Fan, S. (2024). Research on Cross-national Customer Churn Prediction Model for Biopharmaceutical Products Based on LSTM-Attention Mechanism. Academia Nexus Journal, 3(3).
  16. Chen, Y., Feng, E., & Ling, Z. (2024). Secure Resource Allocation Optimization in Cloud Computing Using Deep Reinforcement Learning. Journal of Advanced Computing Systems, 4(11), 15-29.
  17. Yu, P, Yi, J., Huang, T., Xu, Z., & Xu,X.(2024).Optimizationof Transformer heart disease prediction model based onparticle swarm optimization algorithm. arXiv preprintarXiv:2412.02801.
  18. Yu, P., Xu, Z., Wang, J., & Xu, X. (2025). The Application of Large Language Models in Recommendation Systems. arXiv preprint arXiv:2501.02178.
  19. Xu, X., Yu, P., Xu, Z., & Wang, J. (2025). A hybrid attention framework for fake news detection with large language models. arXiv preprint arXiv:2501.11967.
  20. Yi, J., Xu, Z., Huang, T., & Yu, P. (2025). Challenges and Innovations in LLM-Powered Fake News Detection: A Synthesis of Approaches and Future Directions. arXiv preprint arXiv:2502.00339.
  21. Huang, T., Xu, Z., Yu, P., Yi, J., & Xu, X. (2025). A Hybrid Transformer Model for Fake News Detection: Leveraging Bayesian Optimization and Bidirectional Recurrent Unit. arXiv preprint arXiv:2502.09097.
  22. Chen, J., Yan, L., Wang, S., & Zheng, W. (2024). Deep Reinforcement Learning-Based Automatic Test Case Generation for Hardware Verification. Journal of Artificial Intelligence General science (JAIGS) ISSN: 3006-4023, 6(1), 409-429.
  23. Rao, G., Lu, T., Yan, L., & Liu, Y. (2024). A Hybrid LSTM-KNN Framework for Detecting Market Microstructure Anomalies:: Evidence from High-Frequency Jump Behaviors in Credit Default Swap Markets. Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online), 3(4), 361-371.
  24. Yan, L., Zhou, S., Zheng, W., & Chen, J. (2024). Deep Reinforcement Learning-based Resource Adaptive Scheduling for Cloud Video Conferencing Systems.
  25. Wang, S., Chen, J., Yan, L., & Shui, Z. (2025). Automated Test Case Generation for Chip Verification Using Deep Reinforcement Learning. Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online), 4(1), 1-12.
  26. Guo, L., Li, Z., Qian, K., Ding, W., & Chen, Z. (2024). Bank credit risk early warning model based on machine learning decision trees. Journal of Economic Theory and Business Management, 1(3), 24-30.
  27. Lu, W., Ni, C., Wang, H., Wu, J., & Zhang, C. (2024). Machine learning-based automatic fault diagnosis method for operating systems.
  28. Wu, J., Wang, H., Ni, C., Zhang, C., & Lu, W. (2024). Case Study of Next-Generation Artificial Intelligence in Medical Image Diagnosis Based on Cloud Computing. Journal of Theory and Practice of Engineering Science, 4(02), 66-73.
  29. Wu, J., Wang, H., Ni, C., Zhang, C., & Lu, W. (2024, March). Data Pipeline Training: Integrating AutoML to Optimize the Data Flow of Machine Learning Models. In 2024 7th International Conference on Advanced Algorithms and Control Engineering (ICAACE) (pp. 730-734). IEEE.
  30. Zhang, C., Lu, W., Wu, J., Ni, C., & Wang, H. (2024). SegNet network architecture for deep learning image segmentation and its integrated applications and prospects. Academic Journal of Science and Technology, 9(2), 224-229.
  31. Ni, C., Wu, J., Wang, H., Lu, W., & Zhang, C. (2024, June). Enhancing cloud-based large language model processing with elasticsearch and transformer models. In International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024) (Vol. 13180, pp. 1648-1654). SPIE.