4 ms·
It's not true we know nothing. We know a little bit by using the two models from their API. Given the time per inference and the limit on messages per day for G
by typon 4y ago
It's not true we know nothing. We know a little bit by using the two models from their API. Given the time per inference and the limit on messages per day for GPT4, I'm willing to bet it's doing around 10x more compute than GPT3.5. If that's because it has 10x more weights, I don't know. But it wouldn't be a terrible guess.
- feanaro 4y agoSo your estimate is that GPT4 has 1.75 trillion weights?
- dwaltrip 4y agoIs there anything that affects inference compute time besides the number of parameters? Assuming same hardware, etc.
- typon 4y agoYes - for example adding memory to the attention mechanism (similar to RETRO or Memorizing Transformers paper)