5 ms·

> The key moment for me is that I've now personally seen a GPT-3 scale model running on my own personal laptop. I hate to pooh-pooh it for everyone, but this w

> The key moment for me is that I've now personally seen a GPT-3 scale model running on my own personal laptop. I hate to pooh-pooh it for everyone, but this was possible before LLaMa. GPT-J-125m/6b have been around for a while, and are frankly easier to install and get results out of. The smaller pruned model even fits on an iPhone. The problem is more that these smaller models won't ever compete with GPT-scale APIs. Tomorrow's local LLaMa might beat yesterday's ChatGPT, but I think those optimistic for the democratization of chatbot intelligence are setting their hopes a bit high. LLaMa really isn't breaking new ground.

stonebraker 4y ago
Has anyone compared performance between LLaMa.cpp vs Alpaca.cpp?
simonw 4y ago
I'm not particularly interested in beating ChatGPT: I'm looking for a "calculator for words" which I can use for things like summarization, term extraction, text rephrasing etc - maybe translation between languages too. There are all kinds of things I want to be able to do with a LLM that are a lot tighter than general chatbots. I'd love to see a demo of GPT-J on an iPhone!
- sroussey 4y ago
  In a decade, these models will be baked into a SOC.
Vetch 4y ago
There was only ever a 6B GPT-J, you must be thinking of GPT-neo's for smaller sizes. GPT-J was the best of its kind for a long while but even just the 7b version of LLaMa soundly surpasses it in how well it follows examples to solve problems and do useful work like summarization, keyword extraction and a mini search agent. Being able to run something on your laptop that matches the original davinci GPT3 and can be tuned to go beyond is nothing to sniff at. The best models will be by API but there is a threshold of utility for LLMs, below which they're just bad story telling toys, that hasn't been available for home users until recently, with FlanT5s and LLaMas.
- simonw 4y ago
  What prompt are you using to get LLaMA to do summarization?
  - BryceSchroeder 4y ago
    Would also like to know.
ShamelessC 4y ago
GPT-J is cool and all, but it’s been out for awhile and hasn’t seen anywhere near this amount of network effects. Further the Eleuther team is a bit detached from the “average user” in my experience, being all-in on TPU and JAX (which is still suffering from a similar lack of maturity making it difficult to run on non-standard setups). I’m sure someone did the work you suggest to let folks run on this on more reasonable setups- but I never saw it on the front page here. Did they achieve GPT-3 performance on anything at that model size? Happy to be corrected, but my understanding was llama does in fact outperform GPT-3 even at the smaller parameter-counts on certain downstream tasks due to using the chinchilla training regime (and other tricks).

stonebraker 4y ago

Has anyone compared performance between LLaMa.cpp vs Alpaca.cpp?

simonw 4y ago

I'm not particularly interested in beating ChatGPT: I'm looking for a "calculator for words" which I can use for things like summarization, term extraction, text rephrasing etc - maybe translation between languages too. There are all kinds of things I want to be able to do with a LLM that are a lot tighter than general chatbots. I'd love to see a demo of GPT-J on an iPhone!

sroussey 4y ago
In a decade, these models will be baked into a SOC.

Vetch 4y ago

There was only ever a 6B GPT-J, you must be thinking of GPT-neo's for smaller sizes. GPT-J was the best of its kind for a long while but even just the 7b version of LLaMa soundly surpasses it in how well it follows examples to solve problems and do useful work like summarization, keyword extraction and a mini search agent. Being able to run something on your laptop that matches the original davinci GPT3 and can be tuned to go beyond is nothing to sniff at. The best models will be by API but there is a threshold of utility for LLMs, below which they're just bad story telling toys, that hasn't been available for home users until recently, with FlanT5s and LLaMas.

simonw 4y ago
What prompt are you using to get LLaMA to do summarization?
- BryceSchroeder 4y ago
  Would also like to know.

ShamelessC 4y ago

GPT-J is cool and all, but it’s been out for awhile and hasn’t seen anywhere near this amount of network effects. Further the Eleuther team is a bit detached from the “average user” in my experience, being all-in on TPU and JAX (which is still suffering from a similar lack of maturity making it difficult to run on non-standard setups). I’m sure someone did the work you suggest to let folks run on this on more reasonable setups- but I never saw it on the front page here. Did they achieve GPT-3 performance on anything at that model size? Happy to be corrected, but my understanding was llama does in fact outperform GPT-3 even at the smaller parameter-counts on certain downstream tasks due to using the chinchilla training regime (and other tricks).