6 ms·
Just try GPT-3...
by tmabraham 6y ago
Just try GPT-3...
- speedgoose 6y agoSo dubious bash completion requiring a cluster of GPUs in a a closed cloud and the energy of a hydroelectric dam to run?
- melling 6y agoDoes it really matter how it’s solved the first time? The second version can require three orders of magnitude less hardware. Premature optimization is the root of all evil.
- speedgoose 6y agoIsn't the whole point of GPT-3 to have tons of data and hope for the best by training it with a huge amount of time and energy ?
- setr 6y agoFor training -- using the trained algorithm is cheap. It's a one-time cost (well, one-time in the final version)
- DSingularity 6y agoYou do realize that reducing costs means dropping parameters. So what you are suggestion is reversing the very technique that enables this result. Color me doubtful.
- jnwatson 6y agoIt doesn't. There exist far more sophisticated techniques to reduce the size of a neural network while minimizing loss of accuracy.
- jnwatson 6y agoThe energy required to use GPT-3 is a small fraction of the energy required to train it.
- msapaydin 6y agoIndeed, the demos of GPT-3 should specifically tackle this.