4 ms·
Vicuna on iPhone
- glimmeringpenny 4y agoMy iPhone 12 carries 3.62 gb of RAM, won't run :(. Maybe I will spend time and fix the code. Big maybe very on the fence.
- simooooo 4y agoWhat the heck is it?
- isoprophlex 4y agoFantastic effort, I'm on holidays but I'm going to burn through a significant chunk of my mobile internet traffic allotment to test this... I seriously thought it'd take at least until summer for someone to make this happen (LLMs on mobile devices) Edit: too bad, it crashes all the time without generating output.
- simonw 4y agoThis is pretty fun. It's pretty competent for a model that runs on a phone - I did get a great hallucination out of it when I asked it about myself: https://twitter.com/simonw/status/1652363265979318272 https://twitter.com/simonw/status/1652363265979318272
- Operyl 4y agoHmm. I just get it to consistently crash on my 14 Pro Max, latest stable iOS build. At least TestFlight will send those reports on over.
- isoprophlex 4y agoMe too. I thought it was my crap ass iphone 11, but if you're also suffering on a 14 pro... maybe it's a bug, not a hw constraint.
- skykooler 4y agoSame thing on 12 mini. I think 6gb of ram is a little low for this.
- InvaderFizz 4y ago13 Pro Max, no crashes but definitely went OOM and slowed to a crawl. My phone also got very warm. I wasn't able to get much useful output for the things I normally use ChatGPT to help with. It insisted on giving general steps instead of code for basic tasks I threw at it. (Certificate generation using openssl or python).
- carterschonwald 4y agoBummer it crashes on my iPhone 11pro.
- rock_artist 4y agoIt also crashes on my SE. This is most likely due to memory as it suggests. Update : Also other users with beefier hardware comment about it.
- batrat 4y agoAre there any Android alternatives?
- homarp 4y agosee https://news.ycombinator.com/item?id=35763483 https://news.ycombinator.com/item?id=35763483 also
- aledalgrande 4y agoEncode 20.5 tok/s and decode 6.6 tok/s on my 14 Pro (non-Max) Did pretty good on an okonomiyaki recipe Edit: the coding capabilities are pretty hilarious. I asked about partials in Turbo streams and it correctly answered, but then when I asked about a code sample it gave me a PHP+MySQL query? Who knows what happened there.