4 ms·
12 tok/s and almost instant response on M1 Max Mac Studio (with faster SSD than laptops) are impressive – gives hope that large models may run locally from SSDs
by nvch 2mo ago
12 tok/s and almost instant response on M1 Max Mac Studio (with faster SSD than laptops) are impressive – gives hope that large models may run locally from SSDs instead of memory.
- gitpusher42 2mo agoThanks for sharing! SSD read speed is the biggest limiting factor here, unfortunately