Artificial Intelligence for Software Engineering (AI4SE): A Review of Capabilities, Evaluation, and Verification Challenges

Artificial Intelligence for Software Engineering (AI4SE): A Review of Capabilities, Evaluation, and Verification Challenges

Avdoshin S.M. (NRU HSE, Moscow, Russia)
Pesotskaya E. Y. (NRU HSE, Moscow, Russia)

Abstract

Artificial intelligence has significantly reshaped contemporary software engineering practice, changing how software systems are designed, implemented, tested, and maintained. Recent advances in large language models and AI-assisted development tools have expanded the role of AI from supporting isolated tasks to participating in multiple stages of the software lifecycle. In many development settings, software is now created from high-level specifications and contextual requirements, while developers increasingly focus on supervision, validation, and coordination of generated outputs. Despite rapid growth in Artificial Intelligence for Software Engineering (AI4SE), the field remains fragmented, with no widely accepted framework connecting lifecycle stages, model capabilities, autonomy, context, and evaluation. This paper presents a review of AI4SE from a systems perspective and proposes an analytical model for structuring AI4SE analysis that links these dimensions within a single analytical framework. Based on this model, the study proposes a taxonomy of AI capabilities in software engineering, including code generation, transformation, analysis, testing, and interactive assistance, together with their practical limitations. The paper also examines recent developments in AI-assisted development workflows, formal verification of generated code, and multi-agent approaches to automated testing. Special focus is placed on evaluation challenges, including reproducibility, limited transferability of benchmark results to real development environments, and the decreasing usefulness of heavily optimized benchmark datasets. To support more balanced assessment of AI-assisted software engineering systems, the paper proposes a multi-dimensional utility model combining correctness, efficiency, maintainability, and operational risk. The analysis indicates a shift from isolated code-generation tools toward more integrated and partially autonomous development workflows. Reliable adoption of such systems will depend on stronger validation and verification practices, better integration of project context, and evaluation methods that more accurately reflect real software engineering conditions.

Keywords

Artificial Intelligence for Software Engineering (AI4SE); large language models; software engineering evaluation; AI-assisted software development; formal verification.

Edition

Proceedings of the Institute for System Programming, vol. 38, issue 6, part 2, 2026, pp. 133-146

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2026-38(6)-29

For citation

Avdoshin S.M., Pesotskaya E. Y. Artificial Intelligence for Software Engineering (AI4SE): A Review of Capabilities, Evaluation, and Verification Challenges. Proceedings of the Institute for System Programming, vol. 38, issue 6, part 2, 2026, pp. 133-146 DOI: 10.15514/ISPRAS-2026-38(6)-29.

Full text of the paper in pdf Back to the contents of the volume