3 ms·
> The scaling hypothesis regards the blessings of scale as the secret of AGI: intelligence is ‘just’ simple neural units & learning algorithms applied to divers
by ZeroGravitas 4y ago
> The scaling hypothesis regards the blessings of scale as the secret of AGI: intelligence is ‘just’ simple neural units & learning algorithms applied to diverse experiences at a (currently) unreachable scale. As increasing computational resources permit running such algorithms at the necessary scale, the neural networks will get ever more intelligent.
I think this is what I believe i.e. that animals and humans are just evolved machines, with no divine spark. Not sure that I agree that it's unpopular or that it allows you to make decades long predictions on progress that'll come true.
I'm also not sure why you'd want to do this if smaller models are meeting your needs. Feels a bit like the future of flight being predicted as a man with flapping wings rather than a jet engine.
- abriosi 4y agoIf you think humans are only interpolating machines then I agree with your statement. Personally, I don't think humans or animals are only capable of doing interpolation Nonetheless, assuming that these models are only doing interpolation it is astonishing Now, a question arises; is there some emerging weak extrapolation going on? I have no idea, until very recently I though I had a good grasp of what interpolation/extrapolation meant in "human conceptual space". Not anymore.
- Valgrim 4y agoCan you explain what interpolation and extrapolation mean in this context?
- abriosi 4y agoI wish I could do a good job at that. Let me rob a definition from this paper "Learning in High Dimension Always Amounts to Extrapolation" https://arxiv.org/pdf/2110.09485.pdf https://arxiv.org/pdf/2110.09485.pdf : Definition 1. Interpolation occurs for a sample x whenever this sample belongs to the convex hull of a set of samples X, {x1, . . . , xN }, if not, extrapolation occurs.
- paskozdilar 4y ago> I think this is what I believe i.e. that animals and humans are just evolved machines, with no divine spark. I think so, too. It seems to me that a fundamental intelligent activity is the act of "association" - associating two stimuli that occur at the same time/sequentially, which is what NNs already do (I suppose?).