4 ms·

5 years? Bit of a long stretch with how much focus is going into AI. I’d be surprised if it’s not possible within two iterations of the new iPhone.

• Gigachad 4y ago
  GPT-3 was said to require something like 150gb of VRAM. I don't see that gap being bridged in phones within 2 years.
  • mlboss 4y ago
    LLaMA already runs on M2 and is comparable to gpt 3
    • crooked-v 4y ago
      I gave a try at Alpaca-LoRA with tloen's tuning and it definitely feels in reach of GPT 3. Not quite as good, but some of that may just be in whatever's going on with OpenAI's hidden prompts encouraging lots of text out of the model.
  • PaulDavisThe1st 4y ago
    Not impossible that the gap will be bridged in the other direction, by GPT-N or a cousin requiring much less VRAM ....
  • pbronez 4y ago
    Alpaca already works in just 4GB of RAM. This stuff is moving incredibly fast.
    • astrange 4y ago
      It doesn't work well at all. (Also alpaca.cpp uses 8GB of RAM for a 4GB model, possibly because they don't know how mmap works.)
      • dachryn 4y ago
        alpaca lora is quite capable actually and is suddenly gaining a hell of a lot of traction in the corporate world
  • dragonwriter 4y ago
    > GPT-3 was said to require something like 150gb of VRAM. By all accounts GPT-3 is wildly inefficient in resource use; OpenAI runs like a company that’s concerned with the functionality it can achieve by calendar date, and has an almost infinite bankroll to do it. But, other actors in the field have different priorities, and the various open source or, at least, available-to-use models seem to be far more efficient than the OpenAI models of similar function (though they are behind the newest OpenAI models in function.)
  • whywhywhywhy 4y ago
    Dall-e 2 was claimed to need A100s. Stable Diffusion runs on 6 year old gamer cards.