Structure-Aware Dual-View Contrastive Swin Transformer for Fine-Grained Gastrointestinal Endoscopic Image Classification

Fadhil, Fajer F.

doi:10.26706/ijceae.7.4.20260814

Abstract

The classification of gastrointestinal endoscopic images is still a difficult task due to the visual similarity of the related abnormalities and the high class imbalance. This study introduces a novel framework, SADC-Swin (Structure-Aware Dual-View Contrastive Swin), based on Swin Transformer V2, which incorporates dual-view supervised contrastive learning, ordinal supervision for ulcerative-colitis grades, and structure-aware supervision for normal Z-line and oesophagitis classes.

The final number of images for experiments was 10,479 images from 16 classes after discarding classes with less than 100 images. Official dataset folds were kept and no exact duplicate images were found in or between folds using SHA-256 analysis. The conventional Swin-V2-T baseline model achieved 93.06% accuracy and 87.43% Macro-F1 on the validation fold, while SADC-Swin achieved 93.87% and 89.02%, respectively. On the independently reserved fold, the accuracy of SADC-Swin was 93.39% with a Macro-F1 score of 87.74% and the baseline was 92.82% and 86.11% respectively. The highest F1 improvements were seen in ulcerative-colitis grade 1 and oesophagitis-A. The results suggest that the clinically structured supervision could benefit the fine-grained endoscopic classification, especially for visually similar and under-represented classes.

Keywords

HyperKvasir Swin Transformer gastrointestinal endoscopy contrastive learning medical image classification

References

H. Ali et al., “Artificial intelligence in gastrointestinal endoscopy: A comprehensive review,” Annals of Gastroenterology, vol. 37, no. 2, pp. 133–141, 2024, doi: 10.20524/aog.2024.0861.
I. S. Seger, “Evaluating and selecting optimal CNN architectures for accurate pneumonia detection in chest X-rays,” International Journal of Computational and Electronics Aspects in Engineering, vol. 5, no. 4, pp. 183–193, Dec. 2024, doi: 10.26706/ijceae.5.4.20241106.
H. Borgli et al., “HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy,” Scientific Data, vol. 7, Art. no. 283, 2020, doi: 10.1038/s41597-020-00622-y.
N. Waisi, “Deep feature fusion method for images classification,” International Journal of Computational and Electronics Aspects in Engineering, vol. 5, no. 4, pp. 148–153, Dec. 2024, doi: 10.26706/ijceae.5.4.20241103.
Z. Liu et al., “Swin Transformer V2: Scaling up capacity and resolution,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 12009–12019, doi: 10.1109/CVPR52688.2022.01170.
M. K. Yousif, “Enhanced diabetic retinopathy classification using hybrid EfficientNetB0 with CBAM,” International Journal of Computational and Electronics Aspects in Engineering, vol. 7, no. 2, pp. 176–188, May 2026, doi: 10.26706/ijceae.7.2.20260307.
R. W. Stidham et al., “Performance of a deep learning model vs human reviewers in grading endoscopic disease severity of patients with ulcerative colitis,” JAMA Network Open, vol. 2, no. 5, Art. no. e193963, 2019, doi: 10.1001/jamanetworkopen.2019.3963.
B. Gutierrez Becker et al., “Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data,” Therapeutic Advances in Gastrointestinal Endoscopy, vol. 14, 2021, doi: 10.1177/2631774521990623.
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 18661–18673.
K. Ramamurthy, T. T. George, Y. Shah, and P. Sasidhar, “A novel multi-feature fusion method for classification of gastrointestinal diseases using endoscopy images,” Diagnostics, vol. 12, no. 10, Art. no. 2316, 2022, doi: 10.3390/diagnostics12102316.
R. Espantaleón-Pérez, I. Jiménez-Velasco, R. Muñoz-Salinas, and M. J. Marín-Jiménez, “Empirical study of attention-based models for automatic classification of gastrointestinal endoscopy images,” in Computer Analysis of Images and Patterns (CAIP 2023), Lecture Notes in Computer Science, vol. 14185, 2023, pp. 98–108, doi: 10.1007/978-3-031-44240-7_10.
W. Wang, X. Yang, and J. Tang, “Vision transformer with hybrid shifted windows for gastrointestinal endoscopy image classification,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 9, pp. 4452–4461, 2023, doi: 10.1109/TCSVT.2023.3277462.
H. Ge et al., “Development and validation of deep learning models for the multiclassification of reflux esophagitis based on the Los Angeles classification,” Journal of Healthcare Engineering, vol. 2023, Art. no. 7023731, 2023, doi: 10.1155/2023/7023731.
H. Guo, S. A. Somayajula, R. Hosseini, and P. Xie, “Improving image classification of gastrointestinal endoscopy using curriculum self-supervised learning,” Scientific Reports, vol. 14, Art. no. 6100, 2024, doi: 10.1038/s41598-024-53955-8.
Z. Wang, C. Liu, S. Zhang, and Q. Dou, “Foundation model for endoscopy video analysis via large-scale self-supervised pre-train,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, Lecture Notes in Computer Science. Cham, Switzerland: Springer, 2023, pp. 101–111, doi: 10.1007/978-3-031-43996-4_10.
J. Lasington and L. S. M. Lasington, “Can deep learning architectures reliably classify the full spectrum of gastrointestinal endoscopic findings? A comparative study of vision transformers and CNNs on 23-class classification,” Gut, vol. 75, suppl. 2, p. A373, 2026, doi: 10.1136/gutjnl-2026-IDDF.264.
R. Albasrawi, M. Ilyas, and Q. Althebyan, “Modified YOLOv8x model for coronary stenosis detection and troponin risk stratification,” Discov. Comput., vol. 29, Art. no. 34, 2026, doi: 10.1007/s10791-026-09913-1.
R. Albasrawi, M. Ilyas, and O. Bayat, “A modified YOLO-based deep learning model for accurate ST-elevation myocardial infarction detection in ECG images,” Neural Comput. Appl., vol. 38, Art. no. 617, 2026, doi: 10.1007/s00521-026-12347-x.
A. B. Salih, R. B. Mohammed, A. A. Othman, S. A. Mostafa, and M. Kamalrudin, “YOLOv8-based real-time pedestrian detection model for urban traffic surveillance systems,” Int. J. Inform. Vis., vol. 10, no. 2, pp. 556–564, Mar. 2026.
S. I. Hamad, “Utilizing convolutional neural networks for the identification of lung cancer,” International Journal of Computational and Electronics Aspects in Engineering, vol. 6, no. 1, pp. 35–41, Mar. 2025, doi: 10.26706/ijceae.6.1.20250206.
F. Fadhil, M. Abdulghani, A. Salih, and M. Ghazal, “Traffic surveillance: Vehicle detection and pose estimation based on deep learning,” Przegląd Elektrotechniczny, vol. 99, no. 2, pp. 131–134, 2023, doi: 10.15199/48.2023.02.22.
N. Waisi, “Comparative evaluation of ResNet-50 and EfficientNet-B1 for pneumonia detection in chest X-ray images using transfer learning,” International Journal of Computational and Electronics Aspects in Engineering, vol. 6, no. 2, pp. 89–97, Jun. 2025, doi: 10.26706/ijceae.6.2.20250405.
M. Ghazal, M. Al-Ghadhanfari, and F. F. Fadhil, “Classification of breast cancer using a precise deep learning model architecture,” International Journal of Informatics and Communication Technology, vol. 14, no. 3, pp. 933–940, 2025, doi: 10.11591/ijict.v14i3.pp933-940.
M. K. Yousif, “Enhanced diabetic retinopathy classification using hybrid EfficientNetB0 with CBAM,” International Journal of Computational and Electronics Aspects in Engineering, vol. 7, no. 2, pp. 176–188, May 2026, doi: 10.26706/ijceae.7.2.20260307.

[1] H. Ali et al., “Artificial intelligence in gastrointestinal endoscopy: A comprehensive review,” Annals of Gastroenterology, vol. 37, no. 2, pp. 133–141, 2024, doi: 10.20524/aog.2024.0861.

[2] I. S. Seger, “Evaluating and selecting optimal CNN architectures for accurate pneumonia detection in chest X-rays,” International Journal of Computational and Electronics Aspects in Engineering, vol. 5, no. 4, pp. 183–193, Dec. 2024, doi: 10.26706/ijceae.5.4.20241106.

[3] H. Borgli et al., “HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy,” Scientific Data, vol. 7, Art. no. 283, 2020, doi: 10.1038/s41597-020-00622-y.

[4] N. Waisi, “Deep feature fusion method for images classification,” International Journal of Computational and Electronics Aspects in Engineering, vol. 5, no. 4, pp. 148–153, Dec. 2024, doi: 10.26706/ijceae.5.4.20241103.

[5] Z. Liu et al., “Swin Transformer V2: Scaling up capacity and resolution,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 12009–12019, doi: 10.1109/CVPR52688.2022.01170.

[6] M. K. Yousif, “Enhanced diabetic retinopathy classification using hybrid EfficientNetB0 with CBAM,” International Journal of Computational and Electronics Aspects in Engineering, vol. 7, no. 2, pp. 176–188, May 2026, doi: 10.26706/ijceae.7.2.20260307.

[7] R. W. Stidham et al., “Performance of a deep learning model vs human reviewers in grading endoscopic disease severity of patients with ulcerative colitis,” JAMA Network Open, vol. 2, no. 5, Art. no. e193963, 2019, doi: 10.1001/jamanetworkopen.2019.3963.

[8] B. Gutierrez Becker et al., “Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data,” Therapeutic Advances in Gastrointestinal Endoscopy, vol. 14, 2021, doi: 10.1177/2631774521990623.

[9] P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 18661–18673.

[10] K. Ramamurthy, T. T. George, Y. Shah, and P. Sasidhar, “A novel multi-feature fusion method for classification of gastrointestinal diseases using endoscopy images,” Diagnostics, vol. 12, no. 10, Art. no. 2316, 2022, doi: 10.3390/diagnostics12102316.

[11] R. Espantaleón-Pérez, I. Jiménez-Velasco, R. Muñoz-Salinas, and M. J. Marín-Jiménez, “Empirical study of attention-based models for automatic classification of gastrointestinal endoscopy images,” in Computer Analysis of Images and Patterns (CAIP 2023), Lecture Notes in Computer Science, vol. 14185, 2023, pp. 98–108, doi: 10.1007/978-3-031-44240-7_10.

[12] W. Wang, X. Yang, and J. Tang, “Vision transformer with hybrid shifted windows for gastrointestinal endoscopy image classification,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 9, pp. 4452–4461, 2023, doi: 10.1109/TCSVT.2023.3277462.

[13] H. Ge et al., “Development and validation of deep learning models for the multiclassification of reflux esophagitis based on the Los Angeles classification,” Journal of Healthcare Engineering, vol. 2023, Art. no. 7023731, 2023, doi: 10.1155/2023/7023731.

[14] H. Guo, S. A. Somayajula, R. Hosseini, and P. Xie, “Improving image classification of gastrointestinal endoscopy using curriculum self-supervised learning,” Scientific Reports, vol. 14, Art. no. 6100, 2024, doi: 10.1038/s41598-024-53955-8.

[15] Z. Wang, C. Liu, S. Zhang, and Q. Dou, “Foundation model for endoscopy video analysis via large-scale self-supervised pre-train,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, Lecture Notes in Computer Science. Cham, Switzerland: Springer, 2023, pp. 101–111, doi: 10.1007/978-3-031-43996-4_10.

[16] J. Lasington and L. S. M. Lasington, “Can deep learning architectures reliably classify the full spectrum of gastrointestinal endoscopic findings? A comparative study of vision transformers and CNNs on 23-class classification,” Gut, vol. 75, suppl. 2, p. A373, 2026, doi: 10.1136/gutjnl-2026-IDDF.264.

[17] R. Albasrawi, M. Ilyas, and Q. Althebyan, “Modified YOLOv8x model for coronary stenosis detection and troponin risk stratification,” Discov. Comput., vol. 29, Art. no. 34, 2026, doi: 10.1007/s10791-026-09913-1.

[18] R. Albasrawi, M. Ilyas, and O. Bayat, “A modified YOLO-based deep learning model for accurate ST-elevation myocardial infarction detection in ECG images,” Neural Comput. Appl., vol. 38, Art. no. 617, 2026, doi: 10.1007/s00521-026-12347-x.

[19] A. B. Salih, R. B. Mohammed, A. A. Othman, S. A. Mostafa, and M. Kamalrudin, “YOLOv8-based real-time pedestrian detection model for urban traffic surveillance systems,” Int. J. Inform. Vis., vol. 10, no. 2, pp. 556–564, Mar. 2026.

[20] S. I. Hamad, “Utilizing convolutional neural networks for the identification of lung cancer,” International Journal of Computational and Electronics Aspects in Engineering, vol. 6, no. 1, pp. 35–41, Mar. 2025, doi: 10.26706/ijceae.6.1.20250206.

[21] F. Fadhil, M. Abdulghani, A. Salih, and M. Ghazal, “Traffic surveillance: Vehicle detection and pose estimation based on deep learning,” Przegląd Elektrotechniczny, vol. 99, no. 2, pp. 131–134, 2023, doi: 10.15199/48.2023.02.22.

[22] N. Waisi, “Comparative evaluation of ResNet-50 and EfficientNet-B1 for pneumonia detection in chest X-ray images using transfer learning,” International Journal of Computational and Electronics Aspects in Engineering, vol. 6, no. 2, pp. 89–97, Jun. 2025, doi: 10.26706/ijceae.6.2.20250405.

[23] M. Ghazal, M. Al-Ghadhanfari, and F. F. Fadhil, “Classification of breast cancer using a precise deep learning model architecture,” International Journal of Informatics and Communication Technology, vol. 14, no. 3, pp. 933–940, 2025, doi: 10.11591/ijict.v14i3.pp933-940.

[24] M. K. Yousif, “Enhanced diabetic retinopathy classification using hybrid EfficientNetB0 with CBAM,” International Journal of Computational and Electronics Aspects in Engineering, vol. 7, no. 2, pp. 176–188, May 2026, doi: 10.26706/ijceae.7.2.20260307.