I certainly would not want to bet on it, but adding the ability to perform computation and access storage could take 10 years. Right now, GPT-3 "sees" 2048 tok

I certainly would not want to bet on it, but adding the ability to perform computation and access storage could take 10 years. Right now, GPT-3 "sees" 2048 tokens, does a forward pass, and outputs the next word. We would like to to be able to say "hold on while I think for a while" but how to do that with current gradient-based deep learning methods is unclear.