2 ms·
The difference is computers post text terminal are latency and throughput dependent to the user. LLMs are not particularly.
by kcb 6mo ago
The difference is computers post text terminal are latency and throughput dependent to the user. LLMs are not particularly.
- marcus_holmes 6mo agoSorry, I don't understand that comment. Can you clarify, please?
- kcb 6mo agoMy point is LLMs aren't more usable if the hardware is in your room versus a few states away. Personal computers still to this day aren't great when the hardware is fully remote.
- marcus_holmes 6mo agoAgreed. But you couldn't do much on a PC when they launched, at least compared to a mainframe. The hardware was slow, the memory was limited, there was no networking at all, etc. If you wanted to do any actual serious computing, you couldn't do that on a PC. And yet they ate the world. I can easily see the advantage, even now, of running the LLM locally. As others have said in this topic. I think it'll happen. edit: thanks for clarifying :)