3 ms·
What's the speed? I heard of llamafile before.
by 3abiton 3y ago
What's the speed? I heard of llamafile before.
- discardedrefuse 3y agoI wasn't really timing it, but it felt fine, especially since it prints out while generating. The 13B model on the 5600X is about 10 seconds. Tho once the conversation gets too long (about 10 replies) it will go sideways. I gotta play with the default settings. IIRC the 7B on the Fx8350 was closer to 25 seconds per response.