3 ms·

"I gave an LLM an extremely small programming task that is a common interview question in a widely used language and I'm shocked it got it right"

• codethief 3y ago
  > extremely small programming task Maybe, but what I was trying to get at was: If you grant an LLM access to your file system and to the internet (like evo.ninja does), it is already very powerful and can work independently. It will read in and analyze files, remember its original goal even in the middle of some subsubtask it is executing, will self-correct and iterate, et cetera, so context size and the quality of how it solves one-shot tasks are no longer the main limitation. Of course I am aware we are not "there" yet and I am sure an agent like evo.ninja will still struggle with large, unknown code bases in many cases. (Can't try it on our proprietary code, unfortunately.) But if the past year has taught me anything then it's to not underestimate the future.
• dontupvoteme 3y ago
  The fact that an LLM can code up the game of life/snake/etc faster than you could find a download link is an interesting thing. If you give it some documentation, background info, code and traceback info (and know to avoid context window poisoning) you can find it's capable of even helping solve rather tedious CUDA/etc issues.