7 ms·
What if it just takes 100B weights to show interesting behavior? Maybe it's not the method that is bad, but the problems that are hard. This recent paper shows
by visarga 5y ago
What if it just takes 100B weights to show interesting behavior? Maybe it's not the method that is bad, but the problems that are hard. This recent paper shows how at 10^11 weights the network starts behaving much better, as if going through a phase change.
https://twitter.com/AnthropicAI/status/1494352855972540418 https://twitter.com/AnthropicAI/status/1494352855972540418