3 ms·
What are the specs of your laptop and what tokens per second do you get?
by bckr 2mo ago
What are the specs of your laptop and what tokens per second do you get?
- nozzlegear 2mo agoIt's just a Macbook Air with the base M4 and 16gb ram, but I'm hosting the models on a Mac Studio with M1 Ultra and 64gb ram that I had purchased when it came out. I get about 45-55 tokens per second with this setup. I think I could get more if I spent some time fiddling with the parameters, but I don't really know what I'm doing there so I've just left most of it on oMLX's defaults.