3 ms·

Quick thoughts - Major arch changes are not that major, mostly GQA and tokenizer improvements. Tokenizer improvement is a under-explored domain IMO. 15T token

Quick thoughts - Major arch changes are not that major, mostly GQA and tokenizer improvements. Tokenizer improvement is a under-explored domain IMO. 15T tokens is a ton! 400B model performance looks great, can’t wait for that to be released. Might be time to invest in a Mac studio! OpenAI probably needs to release GPT-5 soon to convince people they are still staying ahead.

anentropic 3y ago
> Might be time to invest in a Mac studio! it's wild isn't it for so long a few years old macbook is fine for everything, in desperation Apple waste their time with VR goggles in search of a use-case... then suddenly ChatGPT etc comes along and despite relatively weak GPU Apple accidentally have stuff worth upgrading to imagine when they eventually take the goggles off and start facing in the right direction...
oezi 3y ago
The money making opportunity is releasing PCs/laptops with 10x as much RAM.
Manabu-eo 3y ago
> Might be time to invest in a Mac studio! The highest end Mac Studio with 196GB of ram won't even be enough to run a Q4 quant of the 400B+ (don't forget the +) model. At this point, one have to consider an Epyc for CPU inference or costlier gpu solutions like the "popular" 8xA100 80GB... An if it's a dense model like the other llamas, it will be pretty slow..
- nilsherzig 3y ago
  It's a dense one, zuck confirms this a couple minutes into the interview posted in this thread
- urbandw311er 3y ago
  Just FYI on the podcast video Zuck seems to let it slip that the exact number is 405B. (2-3mins in)
- buildbot 3y ago
  It might be large enough Q2 won't impact performance too much (not guaranteed), but that's a great point. Time to invest in a Milan-X! (Already have the 512GB of DDR4)