6 ms·
> "Bing is going to have to come up with some kind of radical revenue sharing too if they want anything written after 2023." ChatGPT doesn't include anything w
by hooande 4y ago
> "Bing is going to have to come up with some kind of radical revenue sharing too if they want anything written after 2023."
ChatGPT doesn't include anything written after 2021. I certainly wouldn't use it to find an air fryer. The results will be from over a year ago. I would want to see what the newest air fryer options are and it would be really important to have to up to date pricing.
AFAIK there is not a way to update a large language model in real time. You have to train on the entire dataset to do a meaningful update, just like with most forms of neural networks. For ChatGPT that takes days and costs hundreds of thousands of dollars. Every time.
It's great for explanations of concepts, and programming, and a few other things. But with the huge caveat that all of the information you're looking at is from one year ago and may have changed in that time. This really limits the utility of ChatGPT for me.
- wepple 4y agoThat’s a very solvable problem though. If Microsoft decides to integrate ChatGPT with bing, they have the resources to retrain the model on a more recent data set, and even do it somewhat regularly
- typon 4y agoYou don't even need to retrain if you use retrieval transformers. That is the real revolution waiting to happen. Deepmind already unlocked it with RETRO, but I don't know why a public version hasn't been released - hooked into the live internet.
- IanCal 4y agoOpenAI have webgpt too https://openai.com/blog/webgpt/ https://openai.com/blog/webgpt/
- jacooper 4y agoOpenAI is already working on solving this https://openai.com/blog/webgpt/ https://openai.com/blog/webgpt/
- sho_hn 4y agoNeat! I've seen so many discussions of the cost of continually retraining ChatGPT with new knowledge (and the energy efficiency of that, etc.) but had a similar thought that you can probably use a GPT-like approach to do "next word prediction" for a command-based web crawler to gather up to date data and then use the GPT-we-already-have to combine/integrate found content using the classic next word prediction. Sometimes I feel that what makes humans cool is that we (well, some of us!) have good internal scoring on when we lack knowledge and must go in search of it which makes us go down different branches of next-action-in-line.
- int_19h 4y agoSomeone pointed out that the energy cost of training GPT is roughly on par with a single transcontinental flight. If so, I don't think this is a limiting factor in any meaningful sense - you could spend that much energy daily, and it would still be a drop in the bucket overall for any moderately large business.
- bamboozled 4y agoI would imagine trying new datasets on daily basis wouldn’t be trivial ?
- Xelynega 4y agoThe bottleneck would be the number of workers on sites like mechanical turk available to create the datasets. Might take a few more years before amazon and facebook get enough third world countries to the point they can exploit their labour online to create daily training sets.