International Journal of Computational and Electronic Aspects in Engineering
Volume 7 · Issue 2 · April 2026 · pp. 130-139
Research Article · Peer Reviewed
Received: January 5, 2026 · Accepted: March 9, 2026 · Published: April 02, 2026
DOI: 10.26706/ijceae.7.2.20260302
Open Access · CC BY 4.0

Evaluating Generative AI Models in Software Engineering: A Metrics-Based Analysis of Java Code Quality

Marwah Najm Abed1*
1Assistant Lecturer, Department of Networks and Computer Software Techniques, Mosul Polytechnic College, Northern Technical University, Mosul, Iraq.
ORCID: 0009-0003-2853-141X

*Corresponding author: marwa.najm@ntu.edu.iq
Download PDF Cite this article Certificate Volume 7 Issue 2

Abstract

This study offers a thorough analysis of AI technologies in software engineering, focusing on generative AI models and their effects on software quality indicators. Eight various artificial intelligence tools, including Claude, ChatGPT, CommandR, and Codeium, were evaluated using established software engineering indicators: weighted methods per class (WMC), lines of code (LOC), depth of inheritance tree (DIT), and lack of cohesion of methods (LCOM). Our experimental results indicate that several artificial intelligence tools are utilized at various levels; some of these tools generate code with low complexity scores, whereas others produce code with medium to high complexity. This study helps us better understand the actual consequences of including AI tools in software development procedures and provides developers and researchers with guidance on how to select the best AI assistants for software engineering projects.

Keywords

Code Generation AI Tools Software Quality Metrics Generative AI Software Engineering

References

  1. IBM, “What Is Generative AI,” [Online]. Available: https://www.ibm.com/think/topics/generative-ai.
  2. S. Feuerriegel, J. Hartmann, C. Janiesch, and P. Zschech, “Generative AI,” Business & Information Systems Engineering, Sep. 2023.
  3. P. Kaliraj and T. Devi, Artificial Intelligence: Theory, Models, and Applications, 1st ed. New York, NY, USA: Auerbach Publications, Oct. 2021. doi: 10.1201/9781003175865.
  4. K. Singh, “Principles of Generative AI: A Technical Introduction,” Carnegie Mellon University, 2023. [Online]. Available: https://www.cmu.edu/intelligentbusiness/expertise/genai-principles.pdf.
  5. P. Kaliraj and T. Devi, Eds., Artificial Intelligence: Theory, Models, and Applications. Boca Raton, FL, USA: CRC Press, 2023.
  6. OpenAI, “Introducing ChatGPT,” [Online]. Available: https://openai.com/index/chatgpt/.
  7. S. Chaudhary and P. Gupta, “A Comprehensive Study on ChatGPT,” International Journal of Emerging Technologies and Innovative Research (JETIR), vol. 10, no. 10, pp. b196–b201, Oct. 2023. [Online]. Available: http://www.jetir.org/papers/JETIR2310119.pdf.
  8. ResearchGate, “Software Quality Measurement State of the Art,” [Online]. Available: https://www.researchgate.net/publication/344880012_Software_Quality_Measurement_State_of_the_Art.
  9. S. R. Chidamber and C. F. Kemerer, “A metrics suite for object-oriented design,” IEEE Transactions on Software Engineering, vol. 20, no. 6, pp. 476–493, Jun. 1994.
  10. T. J. McCabe, “A complexity measure,” IEEE Transactions on Software Engineering, vol. SE-2, no. 4, pp. 308–320, Dec. 1976.
  11. A. J. Albrecht and J. E. Gaffney, “Software function, source lines of code, and development effort prediction: A software science validation,” IEEE Transactions on Software Engineering, vol. SE-9, no. 6, pp. 639–648, Nov. 1983.
  12. CodeMR Team, “CodeMR Eclipse Plugin Installation Guide,” CodeMR, 2023. [Online]. Available: https://marketplace.eclipse.org/content/codemr-static-code-analyser.
  13. S. Bhatia, T. Gandhi, D. Kumar, and P. Jalote, “Unit test generation using generative AI: A comparative performance analysis of autogeneration tools,” arXiv preprint, arXiv:2312.10622, 2023. [Online]. Available: https://arxiv.org/abs/2312.10622.
  14. S. Treude and M. Gerosa, “How developers interact with AI: A taxonomy of human-AI collaboration in software engineering,” Empirical Software Engineering, vol. 30, no. 2, pp. 1–30, 2025.
  15. IEEE Std 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology, 1990.
  16. BLACKBOX AI Documentation, “About BLACKBOX AI,” [Online]. Available: https://docs.blackbox.ai/getting-started/about-blackbox-ai.
  17. Anthropic, “Claude AI,” [Online]. Available: https://www.anthropic.com/claude.
  18. Cohere, “Command R Model,” [Online]. Available: https://cohere.com/command.
  19. Cohere Documentation, “Command R Documentation,” [Online]. Available: https://docs.cohere.com/v2/docs/command-r.
  20. S. Singh, S. Bansal, A. El Saddik, and M. Saini, “From ChatGPT to DeepSeek AI: A comprehensive analysis of evolution, deviation, and future implications in AI-language models,” arXiv, arXiv:2504.03219, Apr. 2025.
  21. Mistral AI, “About Us,” [Online]. Available: https://mistral.ai/about.
  22. H.-C. Tsai, Y.-F. Huang, and C.-W. Kuo, “Comparative analysis of automatic literature review using Mistral large language model and human reviewers,” Research Square, preprint, 2024. [Online]. Available: https://doi.org/10.21203/rs.3.rs-4022248/v1.
  23. Windsurf (Codeium), “About Codeium,” [Online]. Available: https://windsurf.com/about.
  24. Tabnine, “Docs: Overview,” [Online]. Available: https://docs.tabnine.com/main.
  25. ISO/IEC 25010:2011, Systems and software engineering—Systems and software Quality Requirements and Evaluation (SQuaRE)—System and software quality models. Geneva, Switzerland: International Organization for Standardization, 2011.
  26. N. S. Ubale, K. K. Bandagar, N. M. Dasari, A. N. Parde, and M. M. Kulkarni, “Smart Cab Barcode,” International Journal of Computational and Electronic Aspects in Engineering, vol. 2, no. 3, pp. 76–80, 2021, doi: 10.26706/ijceae.2.3.20210603.
  27. P. D. Armarkar, V. C. Asole, and S. S. Thul, “Online Examination System – A Web-Based Application,” International Journal of Computational and Electronic Aspects in Engineering, vol. 2, no. 3, pp. 46–50, 2021.
  28. A. N. Hasoon, “Using Machine Learning Models to Evaluate the Performance of Website in Iraqi Universities,” International Journal of Computational and Electronic Aspects in Engineering, vol. 3, no. 4, pp. 63–69, 2022, doi: 10.26706/ijceae.3.4.2211496.
  29. I. S. Seger, “Evaluating and Selecting Optimal CNN Architectures for Accurate Pneumonia Detection in Chest X-Rays,” International Journal of Computational and Electronic Aspects in Engineering, vol. 5, no. 4, pp. 185–193, 2024.
  30. Z. S. Karam, “A Comprehensive Survey of Tools and Techniques,” International Journal of Computational and Electronic Aspects in Engineering, vol. 4, no. 2, 2023.