4 ms·
Seems like it's extremely high performance for me on an M1 running the 7B model. Totally usable.
by eigenvalue 4y ago
Seems like it's extremely high performance for me on an M1 running the 7B model. Totally usable.
- DesiLurker 4y agojust because something is high performance does not means it cannot be improved by say another 2X. My point was not whether its usable or not, it is that if you are going to run on CPU vectorization is rather important part and its odd that the landing page has no remark about it.