Rethinking Correctness and Efficiency in AI-Assisted Code Generation
21th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2026, Benidorm, Spain, 22 - 24 May 2026, vol.1, pp.200-209, (Full Text)
- Publication Type: Conference Paper / Full Text
- Volume: 1
- Doi Number: 10.5220/0014971000004015
- City: Benidorm
- Country: Spain
- Page Numbers: pp.200-209
- Keywords: Artificial Intelligence (AI), Code Generation, Correctness, Efficiency, Large Language Model, Productivity, Software Development
- Hacettepe University Affiliated: Yes
Abstract
Large Language Models (LLMs) have significantly advanced the software engineering life cycle by automating code generation, yet evaluations have traditionally relied heavily on functional correctness metrics like Pass@k. Such correctness-only assessments often mask critical deficiencies in computational efficiency, including computational overhead, memory footprint, and long-term maintainability. To address this gap, this study systematically surveys eight rigorous benchmarking studies to review the current evaluation landscape of AI-assisted code generation. By analyzing benchmarking methodologies, datasets, and multi-dimensional performance metrics, we highlight the severe efficiency gap between LLM-generated code and optimal human expert baselines. Ultimately, this paper outlines the current boundaries of generative AI in software development and identifies critical research gaps, advocating for a paradigm shift toward rigorous, hardware-agnostic compound metrics and stress testing to achieve truly scalable and efficient AI-assisted programming.