6 ms·

1. Measured how?

• famouswaffles 3y ago
  The closest biological analogue for an artificial parameter is a synapse. Even then, Synapses, and the brain in general, have more degrees of freedom than just a single floating point parameter allows. Humans have some 50 to 100 trillion synapses. Who knows how far scaling goes in a transformer but simply increasing the parameter count increases performance so far. Either way, we're not really close to emulating the complexity of the brain especially when taking neurons into account (one human neuron is far more complex than any one artificial parameter) In other words, simpler building blocks and smaller sie. Of course, how much complexity is needed for Intelligence is unknown.
• civilitty 3y ago
  The connectome [1]. There are about 100B +- 20B neurons in the average human brain. Each neuron has thousands of synapses and each synapse is made up of dozens to hundreds of neurotransmitter channels. That’s at least a quadrillion parameters which is at least three orders of magnitude bigger than SOTA LLMs assuming each of those synapse channels maps to a parameter. That is an absurd assumption given neuroplasticity, which operates at a high level to adapt the neural network as it learns. See the plasticity section in the linked wikipedia articles: connections between neurons can grow or even get removed on the scale of hours and days. The human brain has entire biological systems supporting intelligence for which there is no real ML equivalent because our ML architectures are static. Given a family member’s research on neurotransmitter potentiation, I’d estimate 10 to 100 parameters per channel to get full fidelity of the human brain. (This is absurdly speculative - we have no idea at which fidelity will intelligence emerge) [1] https://en.m.wikipedia.org/wiki/Connectome https://en.m.wikipedia.org/wiki/Connectome