3 ms·

LLMs don't even come close to the complexity of the human mind though. They're a pastiche of human language, a fuzzy jpeg of the Internet. The human mind is _s

LLMs don't even come close to the complexity of the human mind though. They're a pastiche of human language, a fuzzy jpeg of the Internet. The human mind is _so much more_ than a prediction machine, and incredibly complex... All that's before you get into the way the endocrine system interacts with your mind. A single neuron has an average of 250000 connections in some parts of the brain. The speed at which neuronal signals travel varies neuron to neuron from 2.5m/s to 200m/s. Human minds are more than just prediction. The anterior lateral prefrontal cortex has the sole responsibility of prediction (not that nothing else does, just that the ALPC seems solely dedicated to that task) and is extremely good at it. Prediction can influence all sorts of mental processes such as most all forms of perception... But it is _not_ the same as _being_ all forms of perception. If something unpredictable enough happens in front of you: you'll still see it. Sure there are limits to that: when focused on a task the predictive parts of sight tend to filter out visual data that doesn't match the signal you're looking for (see: basketball players passing the ball and a moon walking man in an ape suit) but if every basketball player turned into Spaghetti-os and started screaming you'd still hear the sounds and see the O's. So sure: LLMs do a good job at basic prediction but they're nowhere near the complexity of the human mind, of which prediction is only a small piece. (And let's not even get into efficiency... A brain runs on 20W of power)

moffkalast 3y ago
Of course, it's only a small piece of the puzzle, but I do wonder what kind of existing organism would roughly match it by complexity, at least in terms of problem solving capacity. A bunch of cells likely have more than a few GB of data between them, but most of it is just instructions on how to fold proteins to replicate themselves. Is that intelligent behavior or just mindlessly executing a script, albeit in DNA form? I would lean towards the latter. I suppose it's hard to compare biological intelligent systems directly since they all have to process all available sensory info in a timely manner, not just very narrow text input. Once there are more LMMs trained on vision, sound, and kinematic data it should get far easier to draw comparisons. > but if every basketball player turned into Spaghetti-os and started screaming you'd still hear the sounds and see the O's Unless of course someone had set up a Somebody else's problem field generator around them :)
pixl97 3y ago
>(And let's not even get into efficiency... A brain runs on 20W of power) At the end of the day this is mostly meaningless. Biological creatures have to be efficient because they need to harvest energy from the environment around them. But to limit machines to that would be like saying we have to feed biomass to airplanes.
- dontwearitout 3y ago
  Their point was the insane efficiency, not the low consumption. Modern LLMs are many orders of magnitude less energy efficient; despite all our progress computational biology makes our silicon looks primitive in some metrics.
nickpsecurity 3y ago
And they are generated by their host piece by piece (brain growth). And they last longer than AI processors with basic maintenance anyone can do (eat/sleep). And they usually continue working when their programs break (eg hardware failures or gamma rays) while NN’s and GA’s don’t. There’s hardly a comparison. If they want to, I’d like to see them start with using dynamic chunking instead of tokenization like the mind does. Also, they can tell me what pre-training method uses 20W per second for 1+ trillion parameters. I could use that right now.

moffkalast 3y ago

Of course, it's only a small piece of the puzzle, but I do wonder what kind of existing organism would roughly match it by complexity, at least in terms of problem solving capacity. A bunch of cells likely have more than a few GB of data between them, but most of it is just instructions on how to fold proteins to replicate themselves. Is that intelligent behavior or just mindlessly executing a script, albeit in DNA form? I would lean towards the latter. I suppose it's hard to compare biological intelligent systems directly since they all have to process all available sensory info in a timely manner, not just very narrow text input. Once there are more LMMs trained on vision, sound, and kinematic data it should get far easier to draw comparisons. > but if every basketball player turned into Spaghetti-os and started screaming you'd still hear the sounds and see the O's Unless of course someone had set up a Somebody else's problem field generator around them :)

pixl97 3y ago

>(And let's not even get into efficiency... A brain runs on 20W of power) At the end of the day this is mostly meaningless. Biological creatures have to be efficient because they need to harvest energy from the environment around them. But to limit machines to that would be like saying we have to feed biomass to airplanes.

dontwearitout 3y ago
Their point was the insane efficiency, not the low consumption. Modern LLMs are many orders of magnitude less energy efficient; despite all our progress computational biology makes our silicon looks primitive in some metrics.

nickpsecurity 3y ago

And they are generated by their host piece by piece (brain growth). And they last longer than AI processors with basic maintenance anyone can do (eat/sleep). And they usually continue working when their programs break (eg hardware failures or gamma rays) while NN’s and GA’s don’t. There’s hardly a comparison. If they want to, I’d like to see them start with using dynamic chunking instead of tokenization like the mind does. Also, they can tell me what pre-training method uses 20W per second for 1+ trillion parameters. I could use that right now.