4 ms·

Was it very shocking when you woke up today and found yourself transported from the year 2000, when Moore's Law still held?

• gleenn 2y ago
  Your comment is a bit snarky and I understand what you're saying but my understanding was that everyone thought Moore's Law was over when the computers stopped getting 2x faster every 18 months but that's not what Moore said. He said the transistor count would double amd it just so happened that that meant computers got a lot faster too because they benefited from all the xtra transistors. But GPUs do still get faster with more transistors because of the embarrassingly parallel nature of many matrix operations. So doubling the transistor count could still effectively significantly improve the speed and reduce time of AI training which is dominated by matrix math. Am I wrong? Did the transistor count doubling also slow down?
  • timschmidt 2y ago
    There's always someone out there claiming Moore's law to be dead. It never is. Chiplets and chip stacking in particular have enabled big increases in transistor count without associated downsides like decreased yield. So long as there is innovation and competition in the semiconductor industry, density / $, which you correctly point out was Moore's original metric, will continue to increase. The potential for increasing performance and decreasing cost of LLM inference is even higher than Moore's law suggests, as the algorithms improve and are pushed closer to ideal hardware to run them.