3 ms·

>* The speedup in a one-off gain, the death of Moore's law is equally dead for "AI chips" and CPU's This does not seem to be true in reality. The H100 is about

• _nalply 3y ago
  The difference seems to be: more units for parallel calculation, but the speed of calculation in itself doesn't double anymore. In other words: Moore's law has stopped for raw speed and perhaps other areas, but is still alive in other areas. This has some weird consequences: Some models can't be processed in smaller chips (because swapping in parts is too slow to be useful), but after a threshold is crossed, suddenly the large models run efficiently. Probably we will see usage of large AI models in smaller devices the next years because there's another way to optimize: use more efficient representations of the model weights. I think about posits, a different floating point system where even 6 bits are perhaps usable. When models can switch to 6 bit posits from f16 (half floats), hardware can load more than three times larger models. We will see whether hardware for this will be mass-produced.
  • daveguy 3y ago
    Moore's law was never about raw speed. It is about transistors per unit area and it is still very much active. That transistor count is going into cache and core count, but it can just as easily go into specialized linear algebra units, which is what AI chips do.
  • raincole 3y ago
    Why from 16 to 6 is "more than three times larger"? Why not 16/6=2.667 times?