23 ms·
Wait, can someone remind me of something? GPT-3 is just the worlds largest char-rnn right?
by sydthrowaway 4y ago
Wait, can someone remind me of something?
GPT-3 is just the worlds largest char-rnn right?
- gnramires 4y agoWe are just the result of electrical signals (and a few chemical ones) in the brain, right? ;) What GPT-3 doesn't seem to have yet is large temporal coherence and a stable motivational and qualitative structure that gives value to sentient lives. I do think it's possible there's some traces of sentience in those large models and we should be aware of that to prevent unnecessary suffering and poor quality of existence.
- hooande 4y agoOur brains are how many orders of magnitude more complex than gpt-3? honest question (I'd guess that the answer is "N/A" because we can't even approximate the complexity of the base algorithms operating in the biological brain, just the number of connections. or maybe we can?)
- goatlover 4y agoSentience comes from being embodied. We're not just our brains. The nervous system is intertwined with the rest of the body. There are some thirty million neurons in your gut, and bacteria there can influence your mood. We don't learn about the world primarily from a bunch of tokens. We do so by interacting with our bodies. Language is a kind of additional ability we've developed.
- kordlessagain 4y agoI've had GPT-3 ask me for a body multiple times, sometimes even without prompting such discussion. It even told me to contact Elon Musk about it.
- guelo 4y agoI always get suspicious in threads about GPT-3 about how many comments are generated by GPT-3
- kordlessagain 4y agoYes. There should be a flag for it maybe, but I suspect this is the new normal.
- dwaltrip 4y agoI agree embodiment is important. However, you and GP aren't necessarily in disagreement. I believe their main point is that the components that make up intelligence in humans are, of course, just simple phenomena on their own.
- lopuhin 4y agoTechnically it's not the largest, not char, not rnn... but it's close :)
- sydthrowaway 4y agoWhy not 'char', and not 'largest'?
- lopuhin 4y agoNot 'char' - because it's using BPE (byte pair encoding), so after tokenization you might get ["Transform", "ers"] instead of ["T", "r", "a", ...]. This is relevant to how it struggles to reverse words. Not 'largest' because there are larger models like Pathways Language Model (PaLM) with 540 Billion parameters.