4 ms·
Apple would need to stick an m4 in the next iPhone to even hope to run something like this and I bet that GPT4o would run either slowly, poorly, or not at all o
by TOMDM 2y ago
Apple would need to stick an m4 in the next iPhone to even hope to run something like this and I bet that GPT4o would run either slowly, poorly, or not at all on a top spec m4.
- tiagod 2y agoOf course GPT 4, or even 3, are impossible to run on any consumer product. As far as I know it's an ensemble of several models which are huge by themselves, with enormous hardware requirements. But there's a lot of smaller LLMs, and my point is that these models can already run in mobile phones.
- xcv123 2y agoYou can run a SLM (small language model) on an iPhone, and not much else. It would use up all of the resources. The iPhone only has 6 GB RAM.
- tiagod 2y agoWhere do you draw the line? GPT2 was introduced as a LLM, and you can easily run it on more limited devices than a recent iPhone. Did it stop being an LLM when bigger models were released? Is llama 7B an LLM or an "SLM"?
- xcv123 2y agoRelatively speaking. It's like the definition of a supercomputer 30 years ago is a cheap Android phone in your pocket today. You can certainly run a transformer model or any other neural network based model on an iPhone. Siri is probably some kind of neural network. But obviously a model running on device is nowhere near comparable to the current state of the art LLM's. Can't fit a $40k GPU in your pocket (yet). A transformer running on an iPhone would be roughly 2 orders of magnitude smaller than the state of the art LLM (GPT4 with a trillion parameters)
- meiraleal 2y ago> Did it stop being an LLM when bigger models were released? Yes it does make sense that what is considered L for a LLM is a moving target.