2 ms·
> GPT 4 is based on the same architecture, but at 8*222B. Do we actually either no that it is MoE or that size? IIRC both if those started as outsidr guesses t
by dragonwriter 3y ago
> GPT 4 is based on the same architecture, but at 8*222B.
Do we actually either no that it is MoE or that size? IIRC both if those started as outsidr guesses that somehow just became accepted knowledge without any actual confirmation.
- moffkalast 3y agoIirc some of the other things the same source stated were later confirmed, so this is likely to be true as well, but I might be misremembering.