International Journal of Computational and Electronic Aspects in Engineering
Volume 7 · Issue 2 · May 2026 · pp. 189-199
Research Article · Peer Reviewed
Received: March 2, 2026 · Accepted: May 7, 2026 · Published: May 20, 2026
DOI: 10.26706/ijceae.7.2.20260308
Open Access · CC BY 4.0

Transformer-Based Malware Classification for Modern Threat Intelligence: An Explainable Multi-Modal Framework

Hasan A. Abdulla*
Technical Engineering College of Mosul, Northern Technical University, Mosul, Iraq.

*Corresponding author: hasan.alsarraf@ntu.edu.iq
Download PDF Cite this article Certificate Volume 7 Issue 2

Abstract

Malware classification is an important problem in modern cybersecurity since malware has become increasingly elusive, polymorphic, obfuscated, and incorporated in the workflow of cyber threat intelligence. While conventional signature-based techniques and manually-designed machine learning models remain effective in static environments, they may perform poorly when faced with packed, obfuscated, tampered, or temporally distant malware samples. In this paper, we propose an interpretable and multimodal transformer-based approach for malware classification. Our approach employs raw-byte sequences along with metadata features using a fusion module. Our model can be used for binary malware detection, malware family classification, and behavior-informed prediction. To improve interpretability, our approach leverages byte-level saliency maps and metadata importance estimation by using explainable artificial intelligence techniques. We evaluate our proposed framework using benchmarking, temporal generalization, robustness to adversarial challenges, ablation experiments, and explanation veracity assessments. We observe that the incorporation of byte-level transformer-based learning, metadata fusion, focal loss, and multi-task learning results in better performance in terms of macro-F1, PR-AUC, robustness, and explainability compared to baseline models that employ only metadata, only bytes, or classical machine learning algorithms.

Keywords

Malware Classification Transformer Raw Bytes Explainable AI Metadata Fusion Cyber Threat Intelligence Multi-Task Learning

References

  1. D. Ucci, L. Aniello, and R. Baldoni, “Survey of machine learning techniques for malware analysis,” Computers & Security, vol. 81, pp. 123–147, 2019, doi: 10.1016/j.cose.2018.11.001.
  2. E. Raff and C. Nicholas, “A survey of machine learning methods and challenges for Windows malware classification,” arXiv preprint arXiv:2006.09271, 2020.
  3. M. G. Schultz, E. Eskin, F. Zadok, and S. J. Stolfo, “Data mining methods for detection of new malicious executables,” in Proc. IEEE Symposium on Security and Privacy, 2001, pp. 38–49, doi: 10.1109/SECPRI.2001.924286.
  4. K. Rieck, T. Holz, C. Willems, P. Düssel, and P. Laskov, “Learning and classification of malware behavior,” in Detection of Intrusions and Malware, and Vulnerability Assessment, LNCS 5137, Berlin, Heidelberg: Springer, 2008, pp. 108–125, doi: 10.1007/978-3-540-70542-0_6.
  5. J. Z. Kolter and M. A. Maloof, “Learning to detect and classify malicious executables in the wild,” Journal of Machine Learning Research, vol. 7, no. 99, pp. 2721–2744, 2006.
  6. I. Santos, F. Brezo, J. Nieves, Y. K. Penya, B. Sanz, C. Laorden, and P. G. Bringas, “Idea: Opcode-sequence-based malware detection,” in Engineering Secure Software and Systems, LNCS 5965, Berlin, Heidelberg: Springer, 2010, pp. 35–43, doi: 10.1007/978-3-642-11747-3_3.
  7. H. S. Anderson and P. Roth, “EMBER: An open dataset for training static PE malware machine learning models,” arXiv preprint arXiv:1804.04637, 2018.
  8. R. Harang and E. M. Rudd, “SOREL-20M: A large scale benchmark dataset for malicious PE detection,” arXiv preprint arXiv:2012.07634, 2020.
  9. R. J. Joyce et al., “EMBER2024: A benchmark dataset for holistic evaluation of malware classifiers,” in Proc. 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’25), 2025, doi: 10.1145/3711896.3737431.
  10. E. Raff, J. Barker, J. Sylvester, R. Brandon, B. Catanzaro, and C. Nicholas, “Malware detection by eating a whole EXE,” arXiv preprint arXiv:1710.09435, 2017.
  11. E. Raff, J. Sylvester, and C. Nicholas, “Learning the PE header, malware detection with minimal domain knowledge,” in Proc. 10th ACM Workshop on Artificial Intelligence and Security (AISec ’17), 2017, pp. 121–132, doi: 10.1145/3128572.3140442.
  12. A. Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, 2017.
  13. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186.
  14. S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in Advances in Neural Information Processing Systems, vol. 30, 2017.
  15. M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in Proc. International Conference on Machine Learning, 2017, pp. 3319–3328.
  16. L. Nataraj, S. Karthikeyan, G. Jacob, and B. S. Manjunath, “Malware images: Visualization and automatic classification,” in Proc. International Symposium on Visualization for Cyber Security, 2011, pp. 1–7, doi: 10.1145/2016904.2016908.
  17. S. Freitas, R. Duggal, and D. H. Chau, “MalNet: A large-scale image database of malicious software,” in Proc. 31st ACM International Conference on Information & Knowledge Management (CIKM ’22), 2022, pp. 3948–3952, doi: 10.1145/3511808.3557533.
  18. T. Bilot, N. El Madhoun, K. Al Agha, and A. Zouaoui, “A survey on malware detection with graph representation learning,” ACM Computing Surveys, vol. 56, no. 11, Art. no. 278, 2024, doi: 10.1145/3664649.
  19. R. Caruana, “Multitask learning,” Machine Learning, vol. 28, no. 1, pp. 41–75, 1997, doi: 10.1023/A:1007379606734.
  20. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 318–327, 2020, doi: 10.1109/TPAMI.2018.2858826.
  21. T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 785–794, doi: 10.1145/2939672.2939785.
  22. G. Ke et al., “LightGBM: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems, vol. 30, 2017.
  23. D. S. Berman, A. L. Buczak, J. S. Chavis, and C. L. Corbett, “A survey of deep learning methods for cyber security,” Information, vol. 10, no. 4, Art. no. 122, 2019, doi: 10.3390/info10040122.
  24. D. Gibert, C. Mateu, and J. Planes, “The rise of machine learning for detection and classification of malware: Research developments, trends and challenges,” Journal of Network and Computer Applications, vol. 153, Art. no. 102526, 2020, doi: 10.1016/j.jnca.2019.102526.
  25. N. Šrndic and P. Laskov, “Practical evasion of a learning-based classifier: A case study,” in Proc. IEEE Symposium on Security and Privacy, 2014, pp. 197–211, doi: 10.1109/SP.2014.20.
  26. Z. S. Karam, R. H. Ali, and B. O. Al-Nashy, “Exploring emerging strategies for countering computer malware attacks: A comprehensive survey of tools and techniques,” International Journal of Computational and Electronic Aspects in Engineering, vol. 4, no. 2, pp. 25–37, 2023, doi: 10.26706/ijceae.4.2.20239750.
  27. M. F. Mahdi, “Enhancing cloud security through artificial intelligence: Detecting advanced cyber-attacks and analyzing anomalous patterns,” International Journal of Computational and Electronic Aspects in Engineering, vol. 6, no. 3, pp. 108–120, 2025, doi: 10.26706/ijceae.6.3.20250601.
  28. Y. M. Abdall, “AI-driven privacy shield: A secure and privacy-preserving federated learning framework,” International Journal of Computational and Electronic Aspects in Engineering, vol. 6, no. 3, pp. 180–189, 2025, doi: 10.26706/ijceae.6.3.20250608.