How Does Generative AI Usage Affect the Coding Performance of Developers?
This project examines how Generative AI tools impact developers' coding performance, using empirical analysis and proprietary data from an IT organization.
Generative AI (GenAI) tools, such as GitHub Copilot, have emerged as transformative technologies in software development, offering the potential to enhance coding efficiency through real-time suggestions. However, the impact of GenAI on developers' coding performance remains underexplored. This study investigates the effects of GenAI usage on coding productivity by leveraging a difference-in-differences (DID) analysis of 27 weeks of proprietary data from a mid-sized global information technology organization. Results reveal a significant increase in the number of user stories completed per week among developers using GenAI tools. Additionally, we examine the moderating role of developers' working experience, identifying nuanced mechanisms by which GenAI tools influence performance. These findings contribute to theoretical advancements in understanding GenAI's role in software development and offer practical insights for optimizing AI tool adoption in professional settings. The paper, sharing the same title as this project, will be presented at the Hawaii International Conference on System Sciences (HICSS) in January 2025, a premier academic research conference on information systems.
