3 ms·
The most time/cost optimal solution is probably to buy 32 or 64 gigs of ram. That'll still be slow but most people are already half way there.
by moffkalast 4y ago
The most time/cost optimal solution is probably to buy 32 or 64 gigs of ram. That'll still be slow but most people are already half way there.
- esperent 4y agoDoesn't it need to be GPU ram?
- nl 4y agoThey are saying you can run it on a CPU by doing this: > However, code changes are necessary to achieve that, although they won't be crazy complex. This is technically true. It will be very slow though. However, give it 6 months and I think we might see an order of magnitude increase in speed on CPUs. This will still be too slow to be very useful though.
- koheripbal 4y agoThat will be very VERY slow. Pcie bandwidth is way too slow.
- moffkalast 4y agoShould be like an order of magnitude faster than trying to run it from a NVMe still, no? I've ran some small flan models from RAM and it was fine, but yeah it's not exactly realtime.