3 ms·
Well, I would have agreed with you a year or two ago. I have been experimenting with GPT-3, via OpenAI beta APIs. The generative (not extractive) summarization
by mark_l_watson 5y ago
Well, I would have agreed with you a year or two ago. I have been experimenting with GPT-3, via OpenAI beta APIs. The generative (not extractive) summarization is impressive. Question answering functionality is also good.
Fast forward ten years when we have: 1. Much faster, with much more memory, custom neural processing hardware. 2. Advances in modeling techniques, similar to how Transformer models seemed revolutionary a few years ago.
Wait ten years.
- visarga 5y ago> Fast forward ten years My laundry list for GPT-4 is: - multi-modal - image + text + other modalities - multi-task - the training data should include all useful supervised datasets and knowledge bases besides next token prediction - longer context window - using external search and an external memory bank - so it is up to date without retraining, and has episodic memory This kind of general model could be hooked up to a virtual body and trained on a diverse set of RL tasks, the problem is that it's probably too slow for real time use.