2 ms·
At this rate it may not be needed in a few months to run a model of equivalent performance. At work I've been successfully running llama2 variants on Mac M2 sy
by hereonout2 3y ago
At this rate it may not be needed in a few months to run a model of equivalent performance.
At work I've been successfully running llama2 variants on Mac M2 systems with more than adequate throughput for small experiments and examples. Can also boot a 4x Nvidia T4 instance on AWS (g4dn.12xlarge) to larger models.
I feel you'd need to be doing inference on a 24/7 basis for at least a year to justify the price of this machine.