2 ms·
They're probably 2-3 years from being able to run worthwhile LLM inference on device assuming they started a week after chatgpt dropped - and that's pushing the
by baq 3y ago
They're probably 2-3 years from being able to run worthwhile LLM inference on device assuming they started a week after chatgpt dropped - and that's pushing the timelines. I'll be surprised if next year's pro can do it. I won't be surprised if the 2025 does. I'll be surprised if the 2026 doesn't.
- themeiguoren 3y agoWith 8 GB RAM, I wouldn’t be surprised if these pros will be able to do it with 2026’s algorithms.