3 ms·

I don't consider that a knowledge model. (Does a calculator have a knowledge model of multiplication?) But I agree that it's something more than Markov chains.

• mannykannot 3y ago
  I think we have to take seriously the proposition that this is what a Markov chain can do when it is based on statistics from a vast corpus of human language use, and consider the possibility that a similar process plays a greater role in human intelligence than many of us (well, me, at least) would have thought. On the other hand, all our prior experience with language of the quality sometimes produced by LLMs has been produced by humans, so LLMs mess with our intuitions and may lead us to anthropomorphize them excessively.
  • eli 3y ago
    Yes, I think the scale that the LLMs operate at makes them interesting and different more than anything else. It's not really a new idea. There's a theory going back decades that consciousness is an emergent property and that if we could build a neural network in a lab that was big and complex enough it would become conscious. (Not really sure I buy it, but it's interesting)
    • mannykannot 3y ago
      Yes - emergence is the relatively straightforward observation that large systems can have complex features that are not discernible at the detailed level. My go-to example is Darwinian fitness, which is not discernible at the atomic level. The trickiest part of making a large neural network conscious would be in how you train it. Our brains have been trained by a half-billion years of evolution since the first neurons emerged, and while evolution is slow, that's still a long time.