3 ms·

The author's stance also used to be my stance for a long time. Then, earlier this week, I tried out AutoGPT/evo.ninja (which uses GPT-4 under the hood) and told

The author's stance also used to be my stance for a long time. Then, earlier this week, I tried out AutoGPT/evo.ninja (which uses GPT-4 under the hood) and told it to "Write me a snake game in HTML, CSS and JavaScript" and, voilà, it wrote me a game that I could open in my browser and it worked perfectly out of the box. I also tried this with Tic Tac Toe – it didn't work at first due to a bug in the CSS but the agent was able to fix that bug in the second round. It knew the context, knew the files it had created in the first round, and knew what I was talking about when I said "the Xs and Os are not visible", etc. So, yes, program specs might still be needed in the beginning, when a piece of software gets written from scratch. But AI will often just fill in missing steps, and once the code has been written, the AI will simply iterate on it, and so the code will take on the role of the spec, just like it does nowadays. No need for the product manager to write detailed specs anymore. All in all, I think the second direction mentioned in the article (AI becoming a full-blown software developer) is a lot more likely than the author thinks.

__loam 3y ago
"I gave an LLM an extremely small programming task that is a common interview question in a widely used language and I'm shocked it got it right"
- codethief 3y ago
  > extremely small programming task Maybe, but what I was trying to get at was: If you grant an LLM access to your file system and to the internet (like evo.ninja does), it is already very powerful and can work independently. It will read in and analyze files, remember its original goal even in the middle of some subsubtask it is executing, will self-correct and iterate, et cetera, so context size and the quality of how it solves one-shot tasks are no longer the main limitation. Of course I am aware we are not "there" yet and I am sure an agent like evo.ninja will still struggle with large, unknown code bases in many cases. (Can't try it on our proprietary code, unfortunately.) But if the past year has taught me anything then it's to not underestimate the future.
- dontupvoteme 3y ago
  The fact that an LLM can code up the game of life/snake/etc faster than you could find a download link is an interesting thing. If you give it some documentation, background info, code and traceback info (and know to avoid context window poisoning) you can find it's capable of even helping solve rather tedious CUDA/etc issues.
floating-io 3y ago
Disclaimer: I have not delved into AI to any great degree. Where this kind of statement ("Holy crap! It wrote me a program and it worked!") tends to make me skeptical is this: the program always seems to be something that's already been done a million times before. IOW, something that was probably somewhere in the model's input dataset. I would want to see it produce something truly unique that I'm trying to do for my business -- something that's never been done before. Then we can talk about AI's ending coding. Until then, they're going to be a tool used by actual coders IMO.
- codethief 3y ago
  > the program always seems to be something that's already been done a million times before Most software being written today, in its essence, has been done a million times before. It's just that, from case to case, the tech stack, interfaces, and data structures always differ slightly, which says little about the difficulty of the software problem being solved and is more a function of your organization's history, the developers' background & personal preferences, industry trends, et cetera. In other words: If zoom out a little, the wheel is being re-invented on the daily.

__loam 3y ago

"I gave an LLM an extremely small programming task that is a common interview question in a widely used language and I'm shocked it got it right"

codethief 3y ago
> extremely small programming task Maybe, but what I was trying to get at was: If you grant an LLM access to your file system and to the internet (like evo.ninja does), it is already very powerful and can work independently. It will read in and analyze files, remember its original goal even in the middle of some subsubtask it is executing, will self-correct and iterate, et cetera, so context size and the quality of how it solves one-shot tasks are no longer the main limitation. Of course I am aware we are not "there" yet and I am sure an agent like evo.ninja will still struggle with large, unknown code bases in many cases. (Can't try it on our proprietary code, unfortunately.) But if the past year has taught me anything then it's to not underestimate the future.
dontupvoteme 3y ago
The fact that an LLM can code up the game of life/snake/etc faster than you could find a download link is an interesting thing. If you give it some documentation, background info, code and traceback info (and know to avoid context window poisoning) you can find it's capable of even helping solve rather tedious CUDA/etc issues.

floating-io 3y ago

Disclaimer: I have not delved into AI to any great degree. Where this kind of statement ("Holy crap! It wrote me a program and it worked!") tends to make me skeptical is this: the program always seems to be something that's already been done a million times before. IOW, something that was probably somewhere in the model's input dataset. I would want to see it produce something truly unique that I'm trying to do for my business -- something that's never been done before. Then we can talk about AI's ending coding. Until then, they're going to be a tool used by actual coders IMO.

codethief 3y ago
> the program always seems to be something that's already been done a million times before Most software being written today, in its essence, has been done a million times before. It's just that, from case to case, the tech stack, interfaces, and data structures always differ slightly, which says little about the difficulty of the software problem being solved and is more a function of your organization's history, the developers' background & personal preferences, industry trends, et cetera. In other words: If zoom out a little, the wheel is being re-invented on the daily.