> I don’t think it’s crazy to believe that people will also be running local inference on their phones in the next 5 years. How about now? https://apfel.franza

> I don’t think it’s crazy to believe that people will also be running local inference on their phones in the next 5 years. How about now? https://apfel.franzai.com/ https://apfel.franzai.com/ (iOS/MacOS, runs the 3B param model already bundled for Siri) https://github.com/alichherawalla/off-grid-mobile-ai https://github.com/alichherawalla/off-grid-mobile-ai (Android, runs ~7B models on flagship phones at 15-30tok/s) Foundation model investment feels like the bubble around fiber optics circa 2001 - a great technology being pushed forward by a speculative mania as it seems like it'll be useful in some profitable way, but nobody's quite sure how.