4 ms·

How are you surprise that getting an LLM to spit out a clone of a very common starter project is evidence of it being able to generate non trivial and valuable

How are you surprise that getting an LLM to spit out a clone of a very common starter project is evidence of it being able to generate non trivial and valuable code - as in not a clone of overabundant codebases - on demand?

fragmede 2y ago
because in actually doing the exercise, and not just talking about it, you'd come up with your own tweak on the Todo app that couldn't be directly be in the training data. you, as a smart human, could come up with a creative feature for your Todo app to have, that no one else would make, showing that these things can compose between the things in their training data and produce a unique combination that didn't exist before. copying example-todo.app to my-todo.app isn't what's impressive, having it able to add features that aren't in the example app is what is. If it only has a box of Lego and can only build things from them, and can't invent new Lego blocks, there's still a large amount of things it can be told to build. That it can assemble those blocks together into a new model that isn't in the instruction manual might not be the most surprising thing in the world, but when that's what most software development is, is the fact that it can't invent new blocks really going to hold it back that much?
- namaria 2y ago
  I think you're getting too distracted with the analogies. LLMs have been shown to spit out easily code for patterns that are very common such as todo apps. That isn't very impressive in itself. LLMs are cool and show some interesting emergent properties, but knowing why they work getting them to make a very common type of app doesn't show a great deal of emergent ability beyond searching a compressed space and reproducing a common pattern found in it. It's easy to underestimate the amount and variability of the training data used. There are possibly tens of thousands of variations on todo apps available on github alone.
  - fragmede 2y ago
    So be creative and ask it to make something that isn't a Todo app with thousands examples on github. Knowing that it's really really really advanced autocomplete based on matrix math that does have some funny edge cases because of a numbering subreddit is an interesting degenerate case, but the sheer vastness of the training data it has picked up makes it able to dig into a simultaneously deep and shallow Lego box. it does fall over and go in circles, and having learned to program before the Internet was mainstream, I'm able to go into the code a fix it manually, doesn't impeach it's ability to get that far in the first place. if it's only able to do the first 90% of the work, and I have to do the last 90% of the work, it's still saved me from doing that first 90% of the work.
    - namaria 2y ago
      I may be wrong but I coming to the conclusion that the promise that a piece of software can generate any piece of software you can possibly desire and describe is basically saying that P=NP and the halting problem can be solved by a Turing machine that approaches infinite speed.
      - fragmede 2y ago
        I'm not claiming that it can generate any piece of software you can possibly desire, but that there are enough examples of quite a lot of pieces of them, and that it can compose them into something that it hasn't directly seen before. Like an astronaut on a horse that was popular for Stablediffusion. It didn't have that directly in the training data but it had enough of the pieces that it's able to create a reasonable looking version of one. That it produces too many fingers on hands and has no concept of writing is a glaring obvious shortcoming, just like LLMs generating buggy inefficient code is another one. Catching it's fuckups is missing the point. My point is that it's such a game changer that you ignore it at your own peril. Just go into a side project, half-cocked and get shit built. Yes the code will be ugly but it got built. Maybe. It has its limits, as does the operator's patience, so it's entirely possible you'll run into a bug it can't fix. But a smart operator knows when to stop it and dig into the problem and fix it manually. Funnily enough though, if you give it some toy code that doesn't ever complete, like a Fibonacci number generator and ask it if it will halt, it's able to point out that it won't. That, of course, is because those are in the training data, but it's cute nonetheless.

fragmede 2y ago

because in actually doing the exercise, and not just talking about it, you'd come up with your own tweak on the Todo app that couldn't be directly be in the training data. you, as a smart human, could come up with a creative feature for your Todo app to have, that no one else would make, showing that these things can compose between the things in their training data and produce a unique combination that didn't exist before. copying example-todo.app to my-todo.app isn't what's impressive, having it able to add features that aren't in the example app is what is. If it only has a box of Lego and can only build things from them, and can't invent new Lego blocks, there's still a large amount of things it can be told to build. That it can assemble those blocks together into a new model that isn't in the instruction manual might not be the most surprising thing in the world, but when that's what most software development is, is the fact that it can't invent new blocks really going to hold it back that much?

namaria 2y ago
I think you're getting too distracted with the analogies. LLMs have been shown to spit out easily code for patterns that are very common such as todo apps. That isn't very impressive in itself. LLMs are cool and show some interesting emergent properties, but knowing why they work getting them to make a very common type of app doesn't show a great deal of emergent ability beyond searching a compressed space and reproducing a common pattern found in it. It's easy to underestimate the amount and variability of the training data used. There are possibly tens of thousands of variations on todo apps available on github alone.
- fragmede 2y ago
  So be creative and ask it to make something that isn't a Todo app with thousands examples on github. Knowing that it's really really really advanced autocomplete based on matrix math that does have some funny edge cases because of a numbering subreddit is an interesting degenerate case, but the sheer vastness of the training data it has picked up makes it able to dig into a simultaneously deep and shallow Lego box. it does fall over and go in circles, and having learned to program before the Internet was mainstream, I'm able to go into the code a fix it manually, doesn't impeach it's ability to get that far in the first place. if it's only able to do the first 90% of the work, and I have to do the last 90% of the work, it's still saved me from doing that first 90% of the work.
  - namaria 2y ago
    I may be wrong but I coming to the conclusion that the promise that a piece of software can generate any piece of software you can possibly desire and describe is basically saying that P=NP and the halting problem can be solved by a Turing machine that approaches infinite speed.
    - fragmede 2y ago
      I'm not claiming that it can generate any piece of software you can possibly desire, but that there are enough examples of quite a lot of pieces of them, and that it can compose them into something that it hasn't directly seen before. Like an astronaut on a horse that was popular for Stablediffusion. It didn't have that directly in the training data but it had enough of the pieces that it's able to create a reasonable looking version of one. That it produces too many fingers on hands and has no concept of writing is a glaring obvious shortcoming, just like LLMs generating buggy inefficient code is another one. Catching it's fuckups is missing the point. My point is that it's such a game changer that you ignore it at your own peril. Just go into a side project, half-cocked and get shit built. Yes the code will be ugly but it got built. Maybe. It has its limits, as does the operator's patience, so it's entirely possible you'll run into a bug it can't fix. But a smart operator knows when to stop it and dig into the problem and fix it manually. Funnily enough though, if you give it some toy code that doesn't ever complete, like a Fibonacci number generator and ask it if it will halt, it's able to point out that it won't. That, of course, is because those are in the training data, but it's cute nonetheless.