This is my experience too. All these LLMs and agents are great for bootstrapping projects faster than ever. They help immensely with bringing down the activation energy. So I do a lot more tinkering around in multiple languages that I wouldn’t have otherwise.
That said, once the project goes beyond a certain threshold, LLMs offer little more than a highly capable autocomplete. In larger codebases, the current context window is too small for the tools to even answer questions properly, let alone make any non-trivial changes.
Productivity starts going down once you try to use LLMs in a large codebase and expect them to be as helpful as they were in a smaller one. Often, these authoritative little shits will whirl you around in a doom loop and still won’t find any useful solution. And suddenly you find yourself having to clean up the mess.
I’d still say the productivity benefit is positive, but at the same time, the hype around these is bonkers. Employers are holding onto the hype cycle to bring down wages through FUD.
>Productivity starts going down once you try to use LLMs in a large codebase and expect them to be as helpful as they were in a smaller one.
Exactly this. I've really tried to find use for LLMs in my big tech company SWE job and I just can't. The context is just too large, and not just the code context. In the time that I can "explain" everything to the LLM, keep iterating until it spits out something semi-useful and massage that into something I can merge, I'd rather just do the whole thing myself.
But it's amazing for greenfield personal projects.
You need to setup the problem for the llm. If I am getting an error, I can normally piece together the 10k lines of relevant code much more quickly than I can track down the bug.
If you see an error and can't immediately Cmd+Shift+F an error code or something and jump to the exact line of code that threw the error, that's an engineering problem.
I'm decently excited about the new world of hugely increased quantity of terrible code. Not absolutely sure how to go mining yet but it has opportunity written all over it.
We as engineers have let the recruiters and VC funding brainwash us into lower salaries. Kind of looking forward to a rebound.