3 ms·

I’m confused as to why you think a GPU is necessary? It’s just linear algebra.

• oreoftw 1y ago
  most likely he was referring the fact that you need plenty of GPU-fast memory to keep the model, and GPU cards have it.
  • deleted 1y ago
    [deleted]
  • adastra22 1y ago
    There is nothing magical about GPU memory though. It’s just faster. But people have been doing CPU inference since the first llama code came out.