It’s been about two years since the explosive release of ChatGPT. We went through a classic hype cycle – from the initial rush to a little disillusionment, maybe some enlightenment, and even a plateau of stability. For software developers specifically, foundational models and LLMs have become helpful tools. But they are far from making us 10x more productive. In this essay we will explore why AI seems to have reached a productivity cap for devs and what needs to change to break through that ceiling.
Please note that this view is written from personal experience as a developer of enterprise backend software. I am well aware that there are other fields, such as lightweight frontends, one-off projects, or refactoring legacy applications where my views don’t hold true.
Code is written for humans
Any developer will tell you that writing code that works at least once is not the difficult part of software engineering. It’s maintaining code, evolving code in a changing business environment, maintaining feature velocity, managing complexity, preventing regression, detecting critical bugs, and all the other non-functional requirements that make software development hard.
Over the decades we, as a profession, have developed many tools and tricks to manage complexity in software systems. From object-oriented programming to domain-driven design, from onion architecture to interface contracts, from microservices to ports-and-adapters – there are so many patterns that help us maintain the, well, maintainability of the software we write. But we should be very clear that all these methods only exist for one purpose: to help humans maintain the systems they build.
The computer doesn’t care about your monoliths or microservices. Nor does it care about interfaces versus implementation, strong typing versus dynamic typing, inline comments, universal binaries, or named arguments. It only cares about the code it’s being fed by the compiler, and faithfully executes it.
No, code is written for humans. A common saying is that every line of code is read ten times more often than it is written or modified. Architectural patterns like (micro-) services and domains exist to allow multiple teams to work on the same application environment at the same time, with minimal side-effects. Foundational concepts like stable interfaces, modularity, cohesion, and coupling are so important for exactly the same reason. Principles like testability and strong typing help humans prevent mistakes in the very human code they write. The popularity of Rust is largely due to the fact that it prevents humans from making a whole swathe of human errors. The list goes on and it paints a clear picture: almost all the tools we use are there for us humans, not the system.
The role of AI in software development
Now let’s look at the role of AI in software development. Initially we used LLMs outside our IDEs to help us write pieces of code we need, and copy-pasted them where we wanted them. Later AI became integrated into IDEs like Cursor and Zed. This close integration removed a lot of the early speed bumps, and allows the AI to autocomplete our lines and blocks of code as we type them. It also provides deep integration between your workspace and the AI, giving it much more context for the tasks it is given. The general consensus is that these tools improve our productivity, reduce drudge work, and help us solve complex problems faster.
But there are more pessimistic sounds as well. People complain that less-experienced developers blindly accept the AI code suggestions without checking for correctness, architectural fit, implications, or side-effects. For more experienced engineers, LLMs are said to stump their critical thinking and creativity. And maybe the most common refrain is that asking a question, checking the answer, asking follow-up questions, and refactoring the result may be about as efficient as writing the entire solution from scratch. In fact, there is research which found the use of coding assistants yielded “no significant gains” in developer productivity.
So what gives? What makes developers say AI makes them more productive, while research shows no net win? I believe this is because AI doesn’t solve our basic human developer needs.
What AI cannot do
Modern foundational models can solve most technical problems we pose to it. But they will generally answer the exact question and not take secondary requirements into account. This is very different from a human solving a problem. An experienced developer thinks not only about the literal goal but also how a solution fits into the architecture, whether the responsibilities are in the right place, if style conventions are followed, and maybe if a decision warrants an inline comment for future readers. A simple example: I often see an AI putting the imports it needs for a solution inline with the function, while a human would always put them at the top of the file. Another example is separating interfaces from implementation – a human might find that important, while an AI does not care about these details. A third example is when an AI writes a comment, it generally describes what the code does, while a good developer will describe why the code is written as it is. As a result we often end up refactoring the AI’s solution.
I believe this is the direct cause why AI-powered productivity has reached a ceiling: it helps us perform bare bones software development (writing code that works) slightly faster. But it does not solve the other 80-90% of software engineering: making sure the code is maintainable and evolvable. In fact it makes this work slightly more difficult, which might explain the reported net-zero productivity gains. Side note: one alternative is using AI to solve problems, and not refactoring their solutions. This inevitably leads to unmaintainable balls of mud and the associated loss of productivity – which would be worse than the time lost on refactoring.
In a nutshell, an AI cannot understand the structure and non-functional requirements us humans have. It can try to infer it from the structure it detects in the existing codebase, but generally does a bad job at this – because our needs are not clearly expressed in code. They are hidden in the seams, in the folder structure, in the naming conventions of our files, folders, classes, and methods. They are expressed in abstract classes, shared libraries, and documentation. All of this cannot be understood by an AI. In other words: an AI cannot generate code with human needs as its top priority, because we cannot feasibly express those needs.
The two roads from here
I believe there are two ways AI in software development will evolve. Most likely in parallel, until one approach eclipses the other. The first is the path we’re already following: using foundational models and large language models to support human productivity. On this road there are still many improvements ahead. The AI will get better at understanding and serving our needs – maybe through training, maybe through instruction, maybe through specialized models, or maybe through some unforeseen breakthrough. But this path might just be today’s equivalent of creating faster horses.
The second path is to remove the limiting factor from the equation. If the effectiveness of AI is limited by human needs, we need a new approach in which the human needs become irrelevant. This might sound dystopian, but it really isn’t. What I’m suggesting is a new form of abstraction in computer software – one in which we no longer try to understand how the AI-generated system works internally, but only care about its outcomes. This is not so different from earlier iterations of abstraction. Not too long ago computer scientists needed to write assembly code to instruct a machine to do work. This was very inefficient, so we invented languages like Cobol, Pascal, C, and C++ as an abstraction layer. As a result most of us no longer need to write or even understand assembly. Then we iterated further and invented new languages like Java, C#, Python, JavaScript, and Go. These further abstracted away the CPU and instruction sets, and made developing software look almost like natural language. Maybe the next level of abstraction is defining outcomes, goals, or API specs instead of writing code. We feed these into an AI, and it generates an opaque system which achieves the stated goals. It might be able to deploy and monitor the system itself, and adjust it if it moves outside predefined tolerances like error rates or latency values. Developers might change the parameters or system goals, but they would have no way to see or understand the internal working of the software. This would truly be prompt engineering.
The future of software development
Regardless of the path we’ll follow, the software engineering profession will change, again. In the past, developers had to adjust to use assembly instead of punch cards, lower-level programming languages instead of assembly, higher-level languages instead of lower-level languages, and so on. We have always adapted, and we will adapt again. Maybe we will need to learn how to effectively use AI to serve the non-functional needs of software developers. Maybe we will all need to become software architects, and let the AI write the implementation for us. Or maybe we need to learn new languages and paradigms to tell AI what to build, and let go of our needs to understand the system.
Most likely, we will see a little bit of everything. There will always be engineers who write every single line of code by hand. For some critical systems this might even be the preferable approach. Then there will be the engineers who use AI to speed up their work, but remain in control of the structure and boundaries. There will also be a new generation of engineers who no longer feel inclined to understand the inner workings of the systems they create – much like almost nobody understands byte code generated by compilers anymore. And then there is a special class of engineers: they who will create the new AI languages and systems used by other engineers. These developers are like the designers of modern day compilers. They unlock the potential of an entirely new generation of AI-native engineers, and they might turn out to be the only ones who still understand how the system works – hopefully.
So what can you do to prepare for what’s to come? The main goal is to stay adaptable. The future will change, and it will change faster and faster every year. What is new today will be old tomorrow. Prioritize the skill to learn new skills. Keep a flexible mind. Don’t be afraid to let go of patterns that worked in the past. Experiment. And most of all, make sure the new tools work for you, the human.
Luc van Donkersgoed's Notes 