3 ms·
> Humans are still 22x more efficient, which is not that far considering the rate of progress in this area. Based on a human output rate of 3.3 tok/s, which se
by nojs 1mo ago
> Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.
Based on a human output rate of 3.3 tok/s, which seems questionable as a means of comparison
- falcor84 1mo agoWhat exactly are you questioning?
- nojs 1mo agoThe claim that tok/s independent of quality is a useful comparison (I can get thousands of tok/s on a suitable small model), and secondarily that humans can’t output “tokens” faster than than in some sense, which I am less confident about
- xyzsparetimexyz 1mo agoI do believe that this is the trade off. We are more efficient but slower in terms of thinking (at the same level of intelligence). Some animals go much further in terms of that trade off, see https://en.wikipedia.org/wiki/Portia_(spider) https://en.wikipedia.org/wiki/Portia_(spider) for example.
- freakynit 1mo agoJust checked wikipedia page... they have like 100K neurons only.. wtf!!! How can nature cramp all senses, including spatial, motion, life maintenance and general thinking into just 100K neurons?
- levocardia 1mo agoA lot of the low level stuff is outsourced to biochemistry: the physical properties of proteins, and the various self-regulating biochemical systems of an animal, can "encode" a lot of intelligence, easing up on the computational demands of the brain proper.
- freakynit 1mo agoHmm... is there something that can estimate how many bits of information in portia, each neuron+synapse combination might be encoding?
- madaxe_again 1mo ago100k is already a lot. Plenty of insects get by with a couple of thousand, and pack a whole bunch of complex behaviour, including flight, into that.
- tesnorindian 1mo agoOur brain is more like a MoE model activating only a few neurons for specific activities making it more efficient unlike a dense model activating all the params. Also brain produces quality tokens @ 3.3 tps instead of fast generating hallucinated tokens by certain models. Thus MTP can produce low quality tokens at 2x speed. Patience pays.