4 ms·
So much disagreement in this thread over statements like LLMs “just predict the next token”. The thing to keep in mind - we know very little about how the huma
by LASR 3y ago
So much disagreement in this thread over statements like LLMs “just predict the next token”.
The thing to keep in mind - we know very little about how the human mind works.
Thinking that somehow humans are special and then making conclusions about LLMs truly work based on that it just pointless IMO.
Who knows. Maybe consciousness itself is a next word predictor.
All we know now is that LLMs exhibit some emergent behaviors that allows them to do very useful kinds of reasoning that was previously only possible by humans. Engaging in heated debate derails the eventual goal of putting these capabilities into use.
This is like “computers are just calculators”.
My team has been neck-deep in applications this past year. We have nobody on the team with ML experience and largely do not know how any of this works under the hood. But what we have built over the past months is considerable familiarity with the “personalities” of LLMs. Like this emotional context improving performance is a basic learning at this point.
- ipunchghosts 3y agoEXACTLY
- SamFold 3y agoOne interesting point here is that humans can learn to do things without language at all. If you raised a human baby and never exposed it to any language it would still learn skills and behaviors. So while intelligence and reasoning in humans does still seem to be linked to language it’s not quite as simple as all knowledge and reasoning simply being encoded in language. Whereas (obviously) ChatGPT is completely based on language and can’t do anything without language or anything that isn’t derived directly from language.
- mastercheif 3y agoIn college I tried a medication called Topamax (Topiramate) for migraine prevention. Topamax has a low-occurrence side effect of “language impairment” which I was particularly sensitive to. It was a terrifying experience, but it was also a valuable one as it changed the way I view intelligence. 10 days in my writing and speech skills had devolved to that of a primary school aged child. I tried to type a text message and struggled to come up with simple words like “and”. My speech slowed down considerably. The terrifying thing was that my internal world was still as complex and meaningful as it was before. All of the emotions I felt were real and legitimate. My cognition outside of lqnguage was intact, I could do math just fine and conceptualize and abstract problems. In spite of this, I was unable to convey what I was feeling and thinking to the outside world. It felt like I was trapped inside my own body. After recovering, my intuitive understanding of the link between language and intelligence was changed forever. See the following for more info on Topamax/Topiramate language impairment: https://link.springer.com/article/10.1007/s10072-008-0906-5 https://link.springer.com/article/10.1007/s10072-008-0906-5
- RoyalHenOil 3y agoI do not experience an internal monologue (unless I make a conscious effort at it), and I also struggle to find the right words for things. For this, and for a few other reasons (like taking a longer time than usual to understand speech), I suspect I may have a mild language processing disorder. But my inner world is still extremely rich. I have very complex thoughts, can solve complex puzzles, etc., all without thinking any words. It sounds like your medication gave you a particularly intense experience of this. Many people have historically downplayed the consciousness of animals and babies on account of their inability to understand language, but it seems language likely has nothing to do with consciousness.
- famouswaffles 3y agoI'll do you one better. It doesn't matter. GPT doesn't need to be a brain anymore than a plane needs to flap wings and have feathers to fly.
- lucasjanon 3y agoI'd like to read about your team's learnings on LLMs during the last months. Did you write any article/HN comment about it?
- LASR 3y agoSome clues in my comments. But for the most part, I can’t share. We’ve built some cool stuff, but not enough to be “moaty” at the moment.