The developer pipeline is the big question here. My experience using these tools is that you absolutely have to know what you’re doing in order to evaluate the code LLMs produce. Right now we have a big pool of senior developers who can wrangle these tools productively and produce good code using them because they understand what the proper solution should look like. However, if new developers start out using these tools directly, without building prior experience by hand, then it might be a lot harder for them to build such intuition for problem solving.
The developer pipeline is the big question here. My experience using these tools is that you absolutely have to know what you’re doing in order to evaluate the code LLMs produce. Right now we have a big pool of senior developers who can wrangle these tools productively and produce good code using them because they understand what the proper solution should look like. However, if new developers start out using these tools directly, without building prior experience by hand, then it might be a lot harder for them to build such intuition for problem solving.