4 ms·
A lot of people dismiss LLMs based on how ridiculously simple next token prediction is. Human nerve cells look ridiculously simple relative to what a clump of
by richrichie 3y ago
A lot of people dismiss LLMs based on how ridiculously simple next token prediction is.
Human nerve cells look ridiculously simple relative to what a clump of them can do. So does a transistor that CPUs are built out of.
- nicklecompte 3y ago[deleted]
- catchnear4321 3y agothe claim was not made that something is simple. (other than llm output.) “look.” which can be done with an untrained eye. A hand looks simple to a child. to a doctor? orthopedic surgeon? worse, to an untrained eye, your response looks like it is from an llm. it would explain missing the nuance. not the only explanation by far, but when combined with talk of genetic effects in the footnote…
- nicklecompte 3y agoI deleted my comment, it was a mistake to get involved in this conversation.
- catchnear4321 3y agothat wasn’t the mistake.
- deleted 3y ago[deleted]
- visarga 3y agoThey are complicated because they got to do something much harder than neural nets - they have to build themselves starting from a single cell. When a neural net will be able to secrete GPUs to enlarge itself like the brain, then it will do the same thing with brains. But then it would have to carry extra apparatus. A neural net doesn't need to operate for 80-90 years without servicing the hardware, brain builds redundancy from the start to compensate. And brains keep an energy budget of just a few watts, not kilowatts. Out of all that complex biology in the brain maybe just a small part of it is contributing to intelligent behavior. Most of it is for support or redundancy. So I don't care you can equal a neuron with a 3 layer MLP with 1000 neurons or more. A model that fits on a DVD can speak fluent language, write code, and solve tasks - that is the useful capacity we are looking for.
- nicklecompte 3y agoI deleted my comment, it was a mistake to get involved in this conversation.
- jokellum 3y ago> No, human nerve cells are not "ridiculously simple." The idea that neurons = transistors is old-fashioned That's not what the OP or the literal quote you pulled from them said. They said a single neuron is ridiculously simple _relative_ to what a clump of them can do.
- nicklecompte 3y agoI deleted my comment, it was a mistake to get involved in this conversation.
- batch12 3y agoI don't dismiss LLMs. They're amazing tools. I dismiss the idea that they are sentient.
- vidarh 3y agoVery few people claim they are sentient. The more common claim is that they have shown qualities that makes it reasonable to think that we are getting close to it - for very varying estimates of "close". If you were to look at human brain cells without ever having seen a functioning human brain built from one, you might well question if they'd be enough too.
- astrobe_ 3y agoI don't think anyone claims that. The claim that one hears more often is the AI "understands" what it talks about.
- stormfather 3y agoI don't get that. To truly predict the next token you have to understand everything in the universe. Maybe people haven't thought enough about what a powerful learning objective that is. The fact that its simply defined is deceptive because as a heuristic people associate complex definitions with powerful tools.
- krapp 3y ago>To truly predict the next token you have to understand everything in the universe. What do "truly" and "understand" mean here, when LLMs are known for making errors and generating semantically correct but contextually meaningless results? Do they truly understand everything in the universe, or is the appearance of understanding the result of that universe being composed of data created by humans, so that relevant and meaningful results are statistically more likely than not?