3 ms·
As someone who understands how these models are built and what they do, let me just say that almost all of what you think these models can do is wrong. For one
by tensor 4y ago
As someone who understands how these models are built and what they do, let me just say that almost all of what you think these models can do is wrong.
For one, you can't just "hook up" a language model to some other task, or to the web. ChatGPT is specifically built and trained to have good conversations. The fact that it can sort of appear to do other things is a happy coincidence.
To do any of what you want, new algorithms need to be built, and none of that is "easy". And finally, these models take A LOT of cpu time. They are not going to be reading thousands of posts in a second without serious and expensive compute hardware backing it, and that level of compute isn't remotely feasible to give out to individual users.
Even chatGPT, which is doing a fraction of the tasks you are listing, costs millions of dollars worth of hardware a day. The only reason it exists for free is because Microsoft has donated all that time.
- agentultra 4y agoAnd because OpenAI exploits labour markets in places like Kenya that have weaker labour protection laws and lower minimum wages than in developed countries. They had to pay someone to label data and filter the worst "content" humanity has to offer. Otherwise it would've ended up like numerous other attempts at exposing "AI" to the Internet. So it also has a huge human cost that OpenAI is not properly accounting for (and another reason why dreaming up potential use-cases for the technology as if it will be miniaturized and become a commodity in the near future takes some wilful ignorance).
- bbor 4y agoInteresting reply, thanks for taking the time to share your expertise. I definitely wasn’t considering the economics of the question, and I tend to agree with you - supporting LLM queries with ad dollars seems impossible at their current state. But chatgpt is already pretty darn good at summarizing and communicating. By “hook up” I literally just mean feed text from the internet into the prompt followed by whatever you want it to do with it - summarize, rank, whatever. Ignoring the economics for a moment (paid search engines?) and assuming that GPT 3.5 is the very best LLM we’ll ever get for simplicity: would you still say you don’t think tweaked versions of such models would MASSIVELY improve search?
- tensor 4y agoThese technologies can and likely will help improve search in the future, but there is still a ton of work to be done both on how specifically to use them and also on scaleability. There is also the business model to sort out. Right now search is primarily driven by ads, which I doubt will cover the costs of the sort of ultra-personalization that you're thinking about. Also, reducing the time you spend on a search engine or looking through results will further reduce ad revenue. However, I can see paid search engines perhaps leading to this. So yes, eventually these models can help improve search, just not in the form that we have today. In a few years the story could well be different. I'm quite interested in seeing how Bing integrates chatgpt technology. They claim that they've created some new model on top of it that somehow links it to search results.
- Jarwain 4y agoKinda loose spit balling idea but couldn't you ask Chatgpt to produce a (set of) query for a search engine that would help a given person find the information they're looking for? Wouldn't "hooking up" ultimately just be a matter of translating intent to a known set of commands?
- tensor 4y agoThat’s a good idea but I’m guessing that using it for query expansion will only lead to marginal improvements as you are still limited by the main search engine.