4 ms·

GPT-3: I need 17 billion parameters to create text

• visarga 6y ago
  On the other hand how many parameters do we need to create text? A simple calculation of number of parameters in the human brain divided by seconds of lifetime yields: 10^14 synapses / (72.6 * 365 * 24 * 3600 seconds) =~ 43000 synapses/second, not all of them related to language and reasoning. So we use tens of thousands of synapses for each second of our life, which is quite a contrast to the 100 bits of information per second which is estimated for conscious processing.