3 ms·
> GPT-4 boasts 100 trillion parameters Source? I’ve never heard this. I’ve heard 1 trillion and that it’s a 8 x 175B ensemble.
by jasonjmcghee 3y ago
> GPT-4 boasts 100 trillion parameters
Source? I’ve never heard this.
I’ve heard 1 trillion and that it’s a 8 x 175B ensemble.
- behnamoh 3y agoThe entire article looks like an amalgam of stitched together pieces taken from different sources without much care.
- malfist 3y agoAlmost like an AI written article
- samhita-alla 3y agoWe have many existing sources on LLMs. I referenced a couple of them that I find great. Repeating the same content doesn't offer much value. :) And I wrote the entire content myself.
- gwright 3y agoMy observation from this is that the process by which humans assemble and summarize information, at least at a somewhat high level, is pretty darn close to the way that LLMs do it. I think that falls apart when you want to talk about deeper learning, drawing inferences and so on but if you are just trying to pull together an executive summary on XYZ, an LLM with some fact checking gets you pretty far IMHO.
- jasondrowley 3y agoUp until recently I was editor at an AI company, and parts of this read exactly like some of the outputs I'd get out of GPT-4/ChatGPT Plus. Many of the linked references are too recent to be in the training corpus (Llama 2, for example), so unless there's some web-search component to this it looks like an LLM wrote the first draft, and a human went through to edit, add links, and populate with images, etc.
- samhita-alla 3y agoOpenAI hasn't confirmed it yet, so as of now, there are no reliable sources to rely on for this information. I just removed that part.
- brandall10 3y agoThere are some people close to the company who claim 8 x 220B MOE.
- brandall10 3y agoI believe it's actually 8 x 220B. Just want to make it clear it's not simply a MOE GPT-3.5.