4 ms·
35b A3b runs ~100 tokens a second on the best M5 Max gpu setup.
by jaggederest 4mo ago
35b A3b runs ~100 tokens a second on the best M5 Max gpu setup.
- ctkhn 4mo agoI got around 50-60 on my m3 max so 100tps seems very realistic for 2 gens later of chip and double the ram