3 ms·
I doubt it. I would expect that if GPT's "knowledge" reaches some limit, OpenAI could just hire subject matter experts to write books for the model to read, and
by dpkirchner 4y ago
I doubt it. I would expect that if GPT's "knowledge" reaches some limit, OpenAI could just hire subject matter experts to write books for the model to read, and then use those to (effectively) down-rank material that differs substantially.
If we're talking about the modeling itself, I think we're a long long ways from any limits. There's room for improvements such as cheap personalized results, faster hardware with more memory for lower costs, bot-to-bot communication allowing my instance to interface with yours to come up with mutual understanding, etc. Pretty much all of the components for this exist already in some form, they just need to be optimized and implemented on a large scale and made available to us humans.
- JieJie 4y agoI found this Veritasium video interesting, about a company that is paring down models so they fit on analog chips, so the chips can be used in self-contained devices. It also appears that quantum computers may be even better for matrix math that transformers rely on. If all these hardware advances pan out... https://www.youtube.com/watch?v=GVsUOuSjvcg https://www.youtube.com/watch?v=GVsUOuSjvcg https://phys.org/news/2023-01-analog-quantum-previously-unsolvable-problems.html https://phys.org/news/2023-01-analog-quantum-previously-unso...
- ThrowawayR2 4y agoIt takes months to years to write a well researched and in-depth book and subject matter experts are not necessarily good authors. Beyond that, each book only moves the needle a little bit as well because of the size of the training corpus. If there's going to be improvement from that path, it will be at best slow and incremental.
- dpkirchner 4y agoYou're right, maybe a whole book is unrealistic. I guess it's more likely that they'd have someone make small corrections, like if enough people flag a common result as wrong or poor.