4 ms·
> but we can’t ignore the information that’s been trained into our DNA There's around 600MB in our DNA. Subtract this from the size of any LLM out there and se
by marcosdumay 2y ago
> but we can’t ignore the information that’s been trained into our DNA
There's around 600MB in our DNA. Subtract this from the size of any LLM out there and see how much you get.
- myownpetard 2y agoA more fair comparison would be subtract it from the size the of source code required to represent the LLM.
- nick3443 2y agoMore like the source code AND the complete design for a 200+ degree of freedom robot with batteries etc. pretty amazing. It's like a 600mb demoscene demo for Conway's game of life!
- Terr_ 2y agoThat's underselling the product, a swarm of nanobots that are (literally, currently) beyond human understanding that are also the only way to construct certain materials and systems. Inheritor of the Gray Goo apocalypse that covered the planet, this kind constructs an enormous mobile mega-fortress with a literal hive-mind, scouring the environment for raw materials and fending off hacking attempts by other nanobots. They even simulate other hive-minds to gain an advantage.
- marcosdumay 2y agoThe source code is the weights. That's what they learn.
- myownpetard 2y agoI disagree. A neural network is not learning it's source code. The source code specifies the model structure and hyperparameters. Then it compiled and instantiated into some physical medium, usually a bunch of GPUs, and weights are learned. Our DNA specifies the model structure and hyperparameters for our brains. Then it is compiled and instantiated into a physical medium, our bodies, and our connectome is trained. If you want to make a comparison about the quantity of information contained in different components of an artificial and a biological system, then it only makes sense if you compare apples to apples. DNA:Code :: Connectome:Weights