4 ms·

GPT-What: Intro to the next big thing

• dane-pgp 6y ago
  > What are those 175 billion parameters? > According to Geoffrey Hinton — the godfather of ‘deep learning’ — parameter can be thought of as a synapse in a human brain ... > They’re far, far smaller, the biggest ones we have right now have about a billion synapses. That’s about a million times smaller than the brain. So his estimate is now off by a factor of about 100? How long will it take for the next couple of 100-fold increases in parameter count?