2 ms·
Training a model on new text is expensive. I don't think it would be feasibly to constantly retrain on an index that's being fed by a live crawl of the web. Wh
by simonw 4y ago
Training a model on new text is expensive. I don't think it would be feasibly to constantly retrain on an index that's being fed by a live crawl of the web.
What would make sense though is setting up a mechanism whereby the language model can consult a traditional search engine any time it needs to. I've heard about people building demos of that kind of thing on top of GPT-3 already - you take the user's input, use it to generate a search term, execute that search, then feed the results back into the language model so it can use them to influence its reply.
For example: you could ask "Who won the superbowl?"
Language model turns that into a search against Google or Bing or similar and grabs the text from the first page of results.
Then internally executes a completion something like this:
"The search results for the search 'who won the superbowl' are: <paste in search results>. Now answer the question 'who won the superbowl?'"
- simonw 4y agoHere's an example of someone implementing exactly that for ChatGPT: https://twitter.com/zswitten/status/1598855548581339138 https://twitter.com/zswitten/status/1598855548581339138
- georgehill 4y agoGot it! But what will happen if those search engines return complete SEO spam, which they are?