4 ms·

Never thought I'd run an LLM released in 2025, on my phone, in full BF16. With ~80tps on an iPhone 16 pro btw.

• elAhmo 1y ago
  How do you actually run this on an iPhone?
  • CharlesW 1y ago
    With something like PocketPal AI (https://github.com/a-ghorbani/pocketpal-ai https://github.com/a-ghorbani/pocketpal-ai). I'd love hear HN'ers opinions on the "best" LM Studio-like option for iOS devices.
    • sogen 1y ago
      For iOS, OpenCat. Has iCloud sync, and one universal app for MacOS and iOS devices.
  • lemonish97 1y ago
    I use PocketPal. Can run any gguf model off hf.
    • rshemet 1y ago
      you can also run it on Cactus - either in Cactus Chat from the App/Play Store or by using the Cactus framework to integrate it into your own app
• deleted 1y ago
  [deleted]