4 ms·
Does it really matter how it’s solved the first time? The second version can require three orders of magnitude less hardware. Premature optimization is the roo
by melling 6y ago
Does it really matter how it’s solved the first time?
The second version can require three orders of magnitude less hardware. Premature optimization is the root of all evil.
- speedgoose 6y agoIsn't the whole point of GPT-3 to have tons of data and hope for the best by training it with a huge amount of time and energy ?
- setr 6y agoFor training -- using the trained algorithm is cheap. It's a one-time cost (well, one-time in the final version)
- DSingularity 6y agoYou do realize that reducing costs means dropping parameters. So what you are suggestion is reversing the very technique that enables this result. Color me doubtful.
- jnwatson 6y agoIt doesn't. There exist far more sophisticated techniques to reduce the size of a neural network while minimizing loss of accuracy.