3 ms·
If you can run it as a AI Horde worker, and the home usage is sporadic, you could definitely support more than one person. Otherwise ~1.5 tokens/s is definitel
by brucethemoose2 3y ago
If you can run it as a AI Horde worker, and the home usage is sporadic, you could definitely support more than one person.
Otherwise ~1.5 tokens/s is definitely the minimum you'd want streaming tokens to a single person.