4 ms·
On the one hand, it is clearly true that much of the requirements for neural networks (in silicon) to achieve intelligence of a general sort, is to simply be bi
by rossdavidh 4y ago
On the one hand, it is clearly true that much of the requirements for neural networks (in silicon) to achieve intelligence of a general sort, is to simply be big enough.
On the other hand, anyone acquainted with the facts of biological brain size vs. intelligence, can see that this cannot be all that is going on. Bison are not clearly more intelligent than crows, not even when the part of the brain involved in operating the body parts and processing raw sensory data is excluded. Something other than scale must be involved, even if scale is a necessary condition.
- api 4y ago> not clearly more intelligent than crows There are two possibilities: they're not more intelligent than crows, or they are in a way that is not clear to us.
- jeffreyrogers 4y agoWell brain size is clearly not the main driver behind intelligence because it's only roughly correlated with intelligence in humans.
- api 4y agoThere are some extreme cases too: https://www.rifters.com/crawl/?p=6116 https://www.rifters.com/crawl/?p=6116 That one is downright spooky, especially for its bizarre implications if you take it seriously.
- FeepingCreature 4y agoThough see gwern's skeptical take. https://www.gwern.net/Hydrocephalus#problems-with-the-case-studies https://www.gwern.net/Hydrocephalus#problems-with-the-case-s...
- lumost 4y agoFundamentally, it may not matter. Just like biological brains are intelligent in some way - an extremely large Language Model may be intelligent in some way. If current scaling trends hold, it may not be possible to distinguish a hypothetical GPT-6 vs. a human over reasonably sized conversations/text production tasks. Granted, such a GPT-6 model may be larger than any commercial application could support, and possibly beyond the range of research oriented funding as of today.
- astrange 4y agoIf GPT-6 was only larger, it would still only know the things in its training set, whereas a human can take arbitrary time to think, can go out and perform experiments, can Google/GPT things themselves, etc. GPT can never be an intelligent agent because it’s not an agent. Also, the increased training time to produce a really big model makes it less like an intelligence, which has low marginal cost to learn something new. The reason GPT looks intelligent is because you’re the one providing that; it’s just a big blob of inanimate knowledge.
- lumost 4y agoWhile a LLM doesn't have the ability to perform real world experiments, it remains to be seen whether an LLM can "reason" over a contextual input, combined with its own outputs. There is no fundamental reason it can't, and no fundamental reason to think that it can. Anecdotal observation from AI Dungeon and similar tools would put this as a "maybe".
- astrange 4y agoI believe it can do that, but there's two periods where it can "think". 1. During training time, which is very expensive in terms of input text and money - plus, model training often completely fails which is why you have to do checkpoints, hyper parameter searches, etc. 2. During inference, but it doesn't have arbitrary thinking and memory abilities like a human does, it only has however much space is in the input tokens + its weights. There are thoughts that can't fit even in an optimal model. GPT isn't just a model, it also has that sampling system which is a regular computer program (as opposed to a learned one), which does give it extra abilities.
- astrange 4y agoIf bison were intelligent, they wouldn’t be cooperative considering how we genocided them - so it’d be hard to tell if they are or not.
- pmoriarty 4y agoI think this is anthropomorphising them. We don't really know what other creatures think. The best we can do is make guesses based on their behavior. But it's still just a guess.
- astrange 4y agoI think we have good evidence for animals learning not to cooperate with us, though; completely ignoring us is associated with animals living in uninhabited places, like penguins, and with ones we killed or ate to death, like historical emus and megafauna.
- dragonwriter 4y ago> On the other hand, anyone acquainted with the facts of biological brain size vs. intelligence, can see that this cannot be all that is going on. I don't think that's true, if big enough is measured in connections, not mass. > Bison are not clearly more intelligent than crows, not even when the part of the brain involved in operating the body parts and processing raw sensory data is excluded. While it's true that crows, despite a high EQ [0] for birds, are lower than some apparently less-intelligent large mammals, birds have lower cell size and thus potentially more connections than mass-based measures like EQ account for when comparing to mammals. So, it's not clear at all that network size (in surplus or ratio to basic functional support of the body) isn't the key thing for intelligence. [0] https://en.m.wikipedia.org/wiki/Encephalization_quotient https://en.m.wikipedia.org/wiki/Encephalization_quotient