5 ms·
It makes me wonder, what is the energy consumption of a what it takes to nurture a human to be an expert in something? Discussions of the intelligence in the L
by dpflan 4y ago
It makes me wonder, what is the energy consumption of a what it takes to nurture a human to be an expert in something?
Discussions of the intelligence in the LLMs as an emergent property of differentiating the model on troves of quality data, raise the question of how a human gains its intelligence. Starting from scratch, a child and a LLM, how can you compare their intelligence when provided with the same training data? What happens? What is the comparable intelligence of the two?
We sit here analyzing the energy cost of training a LLM, developing artificial intelligence. We have to compare that to the energy costs of human intelligence. One may argue we are here because of the previous energy use of civilizations prior.
- jstx1 4y agoA child doesn't start from scratch, they have billions of years of evolution encoded in their DNA.
- pixl97 4y agoAn LLM doesn't start from scratch, it's ran on computer hardware... The particular problem with talking about humanity is our biological functions are deeply intertwined with our intelligence functions. Some things are programmed in, like cellular behavior, organ building, what to do with these systems after you eat food. The two bottom parts of the brain are rather well developed at birth that control these functions. The cerebrum is rather undeveloped in comparison, and that's where the "thinking" happens, at least in terms of what we call intelligence and consciousness. While it comes preprogrammed with any number of subroutines, the amount of what we would consider data is minimal, and in our young years a huge amount of energy is spent exercising the brain so it works properly in the reality it exists. In children that is deprived from stimulation and real world experience (think extreme abuse), the brain underdevelops and it can become impossible for them to learn some things or think in particular ways. Do not underestimate the amount of work that must be performed for an infant to become a thinking machine.
- dpflan 4y agoThank you for bringing up the hardware perspective for LLM intelligence development.
- dpflan 4y agoCan one argue that the random initialization of a model is akin to this starting point? A human brain has reached a morphogenetic starting point thanks to much evolution, a process full of randomness.
- jstx1 4y ago> Can one argue that the random initialization of a model is akin to this starting point? It's a starting point but it's random, unlike the starting point of a human being which is very very not random.
- dpflan 4y agoAll right, let’s dig deeper, I like your responsiveness. Imagine if the LLM were randomly provided the same starting point as human brain…what happens? Is evolution a differentiable process?