International Journal of Computational and Electronic Aspects in Engineering
Volume 7 · Issue 2 · April 2026 · pp. 154-175
Research Article · Peer Reviewed
Received: February 8, 2026 · Accepted: April 2, 2026 · Published: April 22, 2026
DOI: 10.26706/ijceae.7.2.20260306
Open Access · CC BY 4.0

A Hybrid Framework for Automated Mechanistic Interpretability and Self-Evolving Adversarial Robustness in Vision-Language Models

Aws Hamed Hamad*
Biotechnology Research Center, Al-Nahrain University, Baghdad, Iraq.

*Corresponding author: aws.hamed@nahrainuniv.edu.iq
Download PDF Cite this article Certificate Volume 7 Issue 2

Abstract

The last significant milestones in visual-language models (LLMs) are truly impressive; they are challenged to study the relationship between visual and textual information. Nevertheless, despite their outstanding predictive capability, these models can still be subject to adversarial attacks and are usually not very transparent in the way they arrive at their decisions internally. Having no credible mechanisms towards describing what is happening inside, it becomes quite difficult to identify the parts of the internal that are doing the predictions of the model. Contrasting changes are also known to have a detrimental effect on performance particularly in applications where security is paramount. In order to solve these problems, a new hybrid framework named HMIR-VLM is proposed in this study. This system is a combination of automated mechanistic interpretation and a self-evolution approach to enhance the adversary robustness. The approach applies a dual-encoder transformer-based system to train multimodal images. It also has an interpretation analyzer; this analyzer monitors attentional structures and internal activations in order to identify the most important computational clues. These interpretation cues then cause the adaptive adversary defense module to be in control. This module is used to reinforce the weakest areas of the model in the course of training. Some of the popular multimodal benchmark data sets, including MSCOCO, Flickr30K and VQA v2, were used to test the framework. The experimental findings indicate that the suggested approach does not only enhance the performance of multimodal prediction but also its resistance to adversarial attacks is drastically increased. In addition, the interpretation unit produces more regular patterns of explanation than conventional vision-language models, enabling a clearer understanding of the model’s internal reasoning. These results indicate that combining interpretability-oriented analysis with adversarial training is a promising direction for making multimodal AI systems more trustworthy and secure.

Keywords

Vision–Language Models Multimodal Learning Mechanistic Interpretability Adversarial Robustness Explainable Artificial Intelligence

References

  1. S. Danish et al., “A comprehensive survey of vision–language models: Pretrained models, fine-tuning, prompt engineering, adapters, and benchmark datasets,” Information Fusion, vol. 126, Art. no. 103623, Feb. 2026, doi: 10.1016/j.inffus.2025.103623.
  2. Y. Fang et al., “Alignment in large vision language models: A survey,” Information Fusion, in press, 2026, Art. no. 104294, doi: 10.1016/j.inffus.2026.104294.
  3. R. Kazmierczak et al., “Explainability and vision foundation models: A survey,” Information Fusion, vol. 122, Art. no. 103184, Oct. 2025, doi: 10.1016/j.inffus.2025.103184.
  4. L. Kästner and B. Crook, “Explaining AI through mechanistic interpretability,” European Journal for Philosophy of Science, vol. 14, Art. no. 52, 2024, doi: 10.1007/s13194-024-00614-4.
  5. H. Choi, S. Jin, and K. Han, “ICEv2: Interpretability, comprehensiveness, and explainability in vision transformer,” International Journal of Computer Vision, vol. 133, pp. 2487–2504, 2025, doi: 10.1007/s11263-024-02290-6.
  6. L. Yu et al., “eX-ViT: A novel explainable vision transformer for weakly supervised semantic segmentation,” Pattern Recognition, vol. 142, Art. no. 109666, Oct. 2023, doi: 10.1016/j.patcog.2023.109666.
  7. S. Nadweh et al., “A hybrid approach based on artificial intelligence and model predictive control for enhancing stability and efficiency of complex dynamic systems,” Journal of Robotics and Control, vol. 6, no. 5, pp. 2426–2435, 2025, doi: 10.18196/jrc.v6i5.25789.
  8. Y. Qian, X. Zhuang, and M. Wang, “Head information bottleneck (HIB): Leveraging information bottleneck for efficient transformer head attribution and pruning,” EURASIP Journal on Audio, Speech, and Music Processing, vol. 2025, Art. no. 25, 2025, doi: 10.1186/s13636-025-00411-8.
  9. D. Kong et al., “Patch is enough: Naturalistic adversarial patch against vision-language pre-training models,” Visual Intelligence, vol. 2, Art. no. 33, 2024, doi: 10.1007/s44267-024-00066-7.
  10. X. Ren et al., “JDA-attack: Leveraging joint multimodal data augmentation to enhance adversarial transferability of vision-language pre-training models,” Complex & Intelligent Systems, vol. 11, Art. no. 444, 2025, doi: 10.1007/s40747-025-02036-y.
  11. D. Liu et al., “Generating transferable attacks across large vision-language models using adversarial deformation learning,” Pattern Recognition, vol. 176, Art. no. 113194, Aug. 2026, doi: 10.1016/j.patcog.2026.113194.
  12. C. Zhou et al., “PromptGuard: Safeguarding large vision-language models via adversarial prompt tuning,” Knowledge-Based Systems, vol. 338, Art. no. 115498, 2026, doi: 10.1016/j.knosys.2026.115498.
  13. S. Pelekis et al., “Adversarial machine learning: A review of methods, tools, and critical industry sectors,” Artificial Intelligence Review, vol. 58, Art. no. 226, 2025, doi: 10.1007/s10462-025-11147-4.
  14. I. I. Al Barazanchi et al., “Secure trust node acquisition and access control for privacy-preserving expertise trust in WBAN networks,” in Lecture Notes in Networks and Systems. Cham, Switzerland: Springer, 2024, pp. 265–275, doi: 10.1007/978-3-031-62881-8_22.
  15. A. L. Khalaf et al., “Real-time pedestrian and object detection using enhanced YOLO integrated with learning complexity-aware cascades,” TELKOMNIKA, vol. 22, no. 2, pp. 362–371, Apr. 2024, doi: 10.12928/TELKOMNIKA.v22i2.24854.
  16. F. Ahmed et al., “Rock-paper-scissors image classification using transfer learning,” in Proc. 2023 Int. Conf. Business Analytics for Technology and Security (ICBATS), Mar. 2023, pp. 1–6, doi: 10.1109/ICBATS57792.2023.10111433.
  17. S. Nadweh et al., “A reinforcement learning framework for intelligent detection of bad data in power system state estimation,” in Proc. 2025 3rd Int. Conf. Cyber Resilience (ICCR), Jul. 2025, pp. 1–7, doi: 10.1109/ICCR61069.2025.10987654.
  18. A. S. Abdulbaqi et al., “Scalable networking protocols for efficient communication in IoT-enabled smart cities,” in Proc. 2025 3rd Int. Conf. Business Analytics for Technology and Security (ICBATS), May 2025, pp. 1–8, doi: 10.1109/ICBATS66542.2025.11258186.
  19. A. M. Ali et al., “Intelligent traffic model for unmanned ground vehicles based on DSDV-AODV protocol,” Sensors, vol. 23, no. 14, Art. no. 6426, Jul. 2023, doi: 10.3390/s23146426.
  20. H. A. H. Al Naffakh et al., “Optimizing IoT device communication using machine learning for real-time data processing in smart environments,” in Proc. 2025 3rd Int. Conf. Business Analytics for Technology and Security (ICBATS), May 2025, pp. 1–6, doi: 10.1109/ICBATS66542.2025.11258174.
  21. F. A. Al-Ibraheemi et al., “Hybrid machine learning models for predicting user behavior in big data-driven web technologies,” in Proc. 2025 3rd Int. Conf. Business Analytics for Technology and Security (ICBATS), May 2025, pp. 1–8, doi: 10.1109/ICBATS66542.2025.11258496.
  22. S. R. Bora and S. K. Anapu, “Security threats in the model context protocol: A comprehensive survey and trust boundary mitigation framework for agentic AI systems,” International Journal of Science and Research, pp. 990–998, Mar. 2026, doi: 10.21275/sr26316110418.
  23. S. A. Shawkat and I. Al-Barazanchi, “A proposed model for text and image encryption using different techniques,” TELKOMNIKA, vol. 20, no. 4, pp. 858–866, Aug. 2022, doi: 10.12928/TELKOMNIKA.v20i4.23367.
  24. I. Al-Barazanchi et al., “Proposed new framework scheme for path loss in wireless body area network,” Iraqi Journal for Computer Science and Mathematics, vol. 3, no. 1, pp. 11–21, 2022, doi: 10.52866/ijcsm.2022.01.01.002.
  25. S. Nadweh, N. Mohammed, and S. Mekhilef, “Intelligent prediction of photovoltaic inverter failures for improved reliability and control in smart grid systems,” Electric Power Systems Research, vol. 255, Art. no. 112735, 2026, doi: 10.1016/j.epsr.2026.112735.
  26. B. M. Salih et al., “Quantum-inspired optimization algorithms for scalable machine learning models,” International Journal of Intelligent Engineering & Systems, vol. 18, no. 10, 2025.
  27. H. R. Abdulshaheed et al., “A survey on the use of WiMAX and Wi-Fi on vehicular ad-hoc networks (VANETs),” in IOP Conf. Ser.: Materials Science and Engineering, vol. 870, no. 1, Jul. 2020, Art. no. 012122, doi: 10.1088/1757-899X/870/1/012122.
  28. A. S. Abdullah et al., “Improving face recognition by Elman neural network using curvelet transform and HSI color space,” vol. 7, no. 2, pp. 430–437, 2019. [Online]. Available: http://pen.ius.edu.ba
  29. W. Kong et al., “Towards a robust adversarial patch attack against RGB-T crowd counting,” Pattern Recognition, vol. 176, Art. no. 113149, Aug. 2026, doi: 10.1016/j.patcog.2026.113149.
  30. A. M. Ali et al., “Enhanced QoS routing protocol for an unmanned ground vehicle based on the ACO approach,” Sensors, vol. 23, no. 3, Art. no. 1431, Jan. 2023, doi: 10.3390/s23031431.
  31. Y. Chen et al., “A traffic-aware Q-network enhanced routing protocol based on GPSR for unmanned aerial vehicle ad-hoc networks,” Frontiers of Information Technology & Electronic Engineering, vol. 21, no. 9, pp. 1308–1320, Sep. 2020, doi: 10.1631/fitee.1900401.
  32. A. H. Hamad et al., “A secure sharing control framework supporting elastic mobile cloud computing,” International Journal of Electrical and Computer Engineering, vol. 13, no. 2, pp. 2270–2277, Apr. 2023, doi: 10.11591/ijece.v13i2.pp2270-2277.