3 ms·
A lot of people are running fairly powerful models directly on the CPU these days... seems like inference will not be a GPU exclusive activity going forward. Gi
by adam_arthur 4y ago
A lot of people are running fairly powerful models directly on the CPU these days... seems like inference will not be a GPU exclusive activity going forward. Given that RAM is the main bottleneck at this point, running on CPU seems more practical for most end users
See:
https://news.ycombinator.com/item?id=35602234 https://news.ycombinator.com/item?id=35602234