3 ms·

Running a 35B on iPhone at 1 tok/s with 2.5 GB RAM… this is the future of on-device inference. Insane work.