3 ms·
> I see GPT as being ~13-14 years ahead of its time. What do you think is missing from GPT that will take more than ten years to add?
by theptip 6y ago
> I see GPT as being ~13-14 years ahead of its time.
What do you think is missing from GPT that will take more than ten years to add?
- petters 6y agoI certainly would not want to bet on it, but adding the ability to perform computation and access storage could take 10 years. Right now, GPT-3 "sees" 2048 tokens, does a forward pass, and outputs the next word. We would like to to be able to say "hold on while I think for a while" but how to do that with current gradient-based deep learning methods is unclear.