29 ms·
The fact that OpenAI documents theirs is already a big improvement over Anthropic. But, also, the OpenAI tokenizer got more efficient when they last updated it,
by SwellJoe 3mo ago
The fact that OpenAI documents theirs is already a big improvement over Anthropic. But, also, the OpenAI tokenizer got more efficient when they last updated it, rather than less. https://mdstudio.app/o200k-base-tokenizer https://mdstudio.app/o200k-base-tokenizer
- ianberdin 3mo ago[dead]
- alansaber 3mo agoInteresting. New models are estimated at ~5T params, so 45,000x increase over BERT base (110m). But vocab size of 200k, so only an increase of 7x over BERT base (30k).
- kamranjon 3mo agoWhere do these estimates come from?
- cyanydeez 3mo agoa dark anal cavity somewhere next to marketing.
- satvikpendem 3mo agoInteresting that Grok 4.5 nears or exceeds OpenAI and Anthropic models in some benchmarks at only 1.5 trillion parameters per their announcement post.
- vlian2088 3mo agoBERT is not an LLM, it's an encoder-only model, i.e. it doesn't generate new text. the first somewhat useful publicly accessible LLM was GPT-3 with 175B params, and it was also the last frontier model whose parameter count was disclosed.