3 ms·
I love how I inevitably see a comment in threads like this along the lines of "we're going to hit diminishing returns on Moore's law soon and so we can't expect
by kromem 5y ago
I love how I inevitably see a comment in threads like this along the lines of "we're going to hit diminishing returns on Moore's law soon and so we can't expect to see much more advancement like we've seen."
I've been hearing some form of the "we're approaching diminishing returns" argument for decades now, and if anything, I've recently been seeing advancement compounding in ways it hasn't in the past.
As just one example, nearly every major tech school connected VC group I'm aware of has a photonics play in progress for enhancing AI workloads.
I wonder if those "things won't significantly change" folks are just broken records with it, or if there's new blood adopting the position each year while the old blood sees they were wrong and approach predicting future stagnation with greater trepidation.
(I'm sure it's the former, but the idealist in me likes to imagine it's the latter.)
- breck 5y agoIn the comments Karpathy wrote this: > ...today's computers are terrible for neural nets. We are running in total emulation mode and there are many orders of magnitude of improvement to claim from a redesign of the architectures with neural nets in mind.