4 ms·

I saw inputs like "please write C code that writes lisp code that writes pascal code" and stuffs like "please generate some songs in style of spongebob or KJV"

• czzr 4y ago
  Turns out, yes. The hard thing is internalising the sheer scale of data it was trained on.
  • visarga 4y ago
    infinite recursion on language, that's what it internalises, it's not a simple storage
• simonw 4y ago
  It turns out it is! That's what I find so interesting about large language models: they consistently demonstrate abilities that were not predicted when they were first invented. I don't think anyone working on them initially expected them to be able to write Lisp and Pascal, certainly not as well as they can already.
  • visarga 4y ago
    >> Is the power of predictive text that crazy? > It turns out it is! Intelligence was in the data, we were just its vehicles.
• pfortuny 4y ago
  When you think about the speed of light, you think you understand it. Then you realize what it truly means and go “what?”. The same happens with the huge volume of text this model has been trained off. We cannot even begin to grasp its size. It is more. More than that. Yes, even more.
• snowwrestler 4y ago
  Hard to say without knowing what it was trained on. Let’s say you train a predictive engine on the entirety of fiction and nonfiction works of the 19th century. You ask it to produce C code, and it does! That would be obviously crazy. On the other extreme, imagine a predictive engine that has been trained on every bit of C code that has ever been written. You ask to produce C code, and it does! Maybe not so crazy this time.