The largest AI models currently in use have around 0.5 trillion connections between artificial neurons. Each connection is represented and quantified by a weigh

The largest AI models currently in use have around 0.5 trillion connections between artificial neurons. Each connection is represented and quantified by a weight parameter, i.e., a single floating-point number that is learnable in training. For example, inside an AI model, a linear layer that transforms an input vector with 1024 elements to an output vector with 2048 elements has a weight matrix with 1024×2048 elements, or approximately 2M. Each element of the weight is a parameter specifying by how much each element in the input vector contributes to or subtracts from each element in the output vector. Each output vector element is a weighted sum (AKA a linear combination), of each input vector element. A human brain has an estimated 100-500 trillion synapses connecting biological neurons. Each synapse is quite a complicated biological structure[a], but if we oversimplify things and assume that every synapse can be modeled as a single parameter in a weight matrix, then the largest AI models today have approximately 100T to 500T ÷ 0.5T = 200x to 1000x fewer connections between neurons that the human brain. It remains to be seen how AI models will perform in IQ tests if we are able to increase the number of connections, or parameters, by 10x, 100x, 1000x, and beyond. --- [a] https://en.wikipedia.org/wiki/Synapse https://en.wikipedia.org/wiki/Synapse