7 ms·

I think there are plenty of people that remain skeptical of their utility for this application. People who want to get rich will tell you it's the next greates

I think there are plenty of people that remain skeptical of their utility for this application. People who want to get rich will tell you it's the next greatest thing that will revolutionize the industry. Personally, I've been annoyed at how confidently wrong ChatGPT can be. Even when you point out the error and ask it to correct the mistake it comes back with an even-more-wrong answer. And it frames it like the answer is completely, 100% correct and accurate. Because it's essentially really deep auto-complete, it's designed to generate text that sounds plausible. This isn't useful in a search context when you want to find sources and truth. I think there are useful applications for this technology but I think we should leave that to the people who understand LLM's best and keep the charlatans out of it. LLM's are really interesting and have come a long way by leaps and bounds... but I don't see how replacing entire institutions and processes by something that is only well understood by a handful of people is a great idea. It's like watering plants with gatorade.

eptcyka 4y ago
It's not designed to generate plausible sounding text, it's designed to produce text that is statistically likely to be sound. It's a markov chain, but there might be multiple roots and some backpropped probabilities for which leafs should be picked. The statistical significance doesn't have to track with logic.
- stingraycharles 4y ago
  So what you’re saying is that they not just trained it on text, but also verified the answers and trained it in such a way that it would get a negative feedback if the model gave the wrong answer?
  - pwinnski 4y ago
    It got negative feedback if the model gave wrong-sounding answers, which is not the same as wrong answers. I have seen no indication that they had anybody verify the factual content of answers in the training process, and many indications that they did not.
rvz 4y ago
> People who want to get rich will tell you it's the next greatest thing that will revolutionize the industry. Reminds me of the web3 crypto hype and this hype is the same thing that happened with GPT-3. Closed source black-box AI model hidden behind a SaaS confidently generating wrong answers as the truth. Sounds like OpenAI (effectively Microsoft's new AI division) are selling snake-oil again. > I think there are useful applications for this technology but I think we should leave that to the people who understand LLM's best and keep the charlatans out of it. There are certainly charlatans, grifters and snake-oil sales people pretending and hyping about their so-called 'AI startup' when it is actually uses the GPT-3 or ChatGPT API. Another emperor has no clothes confidence trick generating garbage. Given that ChatGPT cannot explain its own answers transparently when questioned especially when it confidently generates wrong answers means that you cannot trust its output and telling it to give you a sophisticated answer to an 'un-googleable' question is where you see it clearly trip over. What would really 'revolutionize the industry' is an open-source LLM model which is a smaller model and more transparent than ChatGPT, like what Stable Diffusion did to DALLE-2.
- brookst 4y ago
  Don’t mistake the fallibility of chatgpt for a problem with the entire field. It would be like saying transistors don’t have much of a future because they are too big to fit many in a device. The Bing openai integration is already dramatically different and better than chatgpt. I’d really advise trying it before forming too many opinions. Certainly don’t mistake any experience you have with chatgpt as indicative.
  - rvz 4y ago
    > Don’t mistake the fallibility of chatgpt for a problem with the entire field. It would be like saying transistors don’t have much of a future because they are too big to fit many in a device. Fundamentally, GPT itself and LLMs are essentially still black-box neural networks and all have the same drawbacks as them: * More data needed to train them. Which then... * ...Increases the size of the model (Getting bigger on each revision) * Opaqueness (Cannot transparently explain its own decisions when it confidently generates the wrong answer.) * Massive cost involved in training, retraining and fine-tuning it. The best part is that you can't even give an explanation on why it is generating nonsense or why it overfitted or got confused over an invalid character or input. Little to nothing fundamental about neural networks (which is what LLMs are based on) has changed. It is essentially more of: 'Just waste more money on training it on more data.' > The Bing openai integration is already dramatically different and better than chatgpt. I’d really advise trying it before forming too many opinions. What I have said about ChatGPT and GPT-3 still holds true and hyping about a chatbot confidently generating nonsense and that cannot cite its own sources makes it highly untrustworthy to use. It is always eternal to the shortcomings of neural networks fundamentally; which is what LLMs use. They are great sophists and bullshit generators. At least what needs to exist is an open-source LLM that is more transparent and is smaller than GPT-3 or GPT-4. There is a basic reason why OpenAI won't do that.
bbor 4y ago
This is a perfect example of the popular view on here, and in my humble naive opinion it’s completely mistaken. The point isn’t “can an LLM replace google” the point is “can robots that can speak English and use logic improve the search experience” which I think basically everyone would answer “yes” to. Complaining that it gets stuff wrong when not hooked up to a web of resources to cite is, IMO, completely missing the point. Also OP (not so much you) is way too caught up in the “chat” aspect - that is the first exciting UX that got popular, but these are much, much more than chatbots. Pretending that they’re human/conscious/in a conversation is fun, but having an invisible partner that knows you and tailors your search results… that’s powerful. For example, you’ll never have to add “Reddit” again, or at least you’ll only have to tell it once. An LLM can easily identify the kind of questions where you want forum posts, read thousands of posts in a second, summarize all their content, and label each link with other information that helps you decide which threads to read in full. I can’t wait!
- pwinnski 4y ago
  If the point isn't how an LLM replace a search engine, then why is Bing using an LLM to replace their search engine? When you ask whether speaking English and using logic can improve the search experience, I wonder what you consider the most important parts of the search experience? I think many people, most of the time, might say that "accurate information" is their highest expectation, with "a pleasant conversation" somewhere below that. Delivering a plausible-sounding and pleasant answer that's completely wrong is... well, that's not a search engine I can depend on, is it? You're hypothesizing a few things at the end that sound great! It's completely unclear whether any of those things will actually end up happening, so I think the focus on what is available today, with Chat-GPT and Bing, is more apt than a focus on what could be.
  - bbor 4y ago
    My basic answer is “they’re trying to rush stuff out the door because the people running bing have no idea what they’re doing” :) given that the things I propose don’t need any new inventions, I’d say they’re good to discuss and coming in the next few years for sure. And I totally agree that a) a search engine that doesn’t cite its sources is useless, and b) you almost never want to chat with google like it’s a person. So you’re spot on. But the point I was trying to make is that the main use case is in stuff like automated summaries, specialized page rankings, expanding quick informal queries into longer formal-logic ones, etc.
marban 4y ago
People who want to get rich will tell you it's the next greatest thing that will revolutionize the industry. The nice way of saying grifters.
- braindead_in 4y ago
  Or Entrepreneur maybe?
- rchaud 4y ago
  They go where the funding is.
725686 4y ago
"Personally, I've been annoyed at how confidently wrong ChatGPT can be. Even when you point out the error and ask it to correct the mistake it comes back with an even-more-wrong answer" That also happens with real people so... A web search also returns wrong answers, because it is not magic. It just searches through all the garbage out there. You just have to be aware of its flaws and limitations... as you do with you fellow humans.
- smith7018 4y ago
  My God, I'm so tired of this response. It's not a person. It's software. It shouldn't be wrong just like humans. Imagine opening your banking app and your total savings is wrong. Would you say, "It's okay, people make mistakes, too!" or would you be pissed that your bank's software is incorrect? Why are we pretending like this is any different?
  - rvz 4y ago
    There you go. The difference is that a human can transparently explain to you step by step how they have concluded to a given answer. Humans have always used multiple methods to explain their decisions transparently when questioned to. AI machines and especially neural networks which LLMs are based on still cannot explain themselves and are as transparent as a black-box.
  - JohnFen 4y ago
    Because LLMs aren't doing calculations that result in precise mathematical answers like your bank should be doing. All they are is very advanced pattern-matching machines. They match patterns, but those patterns don't always relate to accurate information.
  - 8note 4y ago
    I'm not pissed when google doesn't have a good result for my search on the first page, no.
- flir 4y ago
  A real person would say "I don't know" occasionally.
shanebellone 4y ago
"it's essentially really deep auto-complete" This is the most accurate description I've even read.
- shanebellone 4y ago
  With that that in mind... cell phones have offered a similar feature to billions of users for many years. Those autocorrections are still frequently wrong.
AutoDunkGPT 4y ago
so what you're saying is, that it's my time to shine, right?

eptcyka 4y ago

It's not designed to generate plausible sounding text, it's designed to produce text that is statistically likely to be sound. It's a markov chain, but there might be multiple roots and some backpropped probabilities for which leafs should be picked. The statistical significance doesn't have to track with logic.

stingraycharles 4y ago
So what you’re saying is that they not just trained it on text, but also verified the answers and trained it in such a way that it would get a negative feedback if the model gave the wrong answer?
- pwinnski 4y ago
  It got negative feedback if the model gave wrong-sounding answers, which is not the same as wrong answers. I have seen no indication that they had anybody verify the factual content of answers in the training process, and many indications that they did not.

rvz 4y ago

> People who want to get rich will tell you it's the next greatest thing that will revolutionize the industry. Reminds me of the web3 crypto hype and this hype is the same thing that happened with GPT-3. Closed source black-box AI model hidden behind a SaaS confidently generating wrong answers as the truth. Sounds like OpenAI (effectively Microsoft's new AI division) are selling snake-oil again. > I think there are useful applications for this technology but I think we should leave that to the people who understand LLM's best and keep the charlatans out of it. There are certainly charlatans, grifters and snake-oil sales people pretending and hyping about their so-called 'AI startup' when it is actually uses the GPT-3 or ChatGPT API. Another emperor has no clothes confidence trick generating garbage. Given that ChatGPT cannot explain its own answers transparently when questioned especially when it confidently generates wrong answers means that you cannot trust its output and telling it to give you a sophisticated answer to an 'un-googleable' question is where you see it clearly trip over. What would really 'revolutionize the industry' is an open-source LLM model which is a smaller model and more transparent than ChatGPT, like what Stable Diffusion did to DALLE-2.

brookst 4y ago
Don’t mistake the fallibility of chatgpt for a problem with the entire field. It would be like saying transistors don’t have much of a future because they are too big to fit many in a device. The Bing openai integration is already dramatically different and better than chatgpt. I’d really advise trying it before forming too many opinions. Certainly don’t mistake any experience you have with chatgpt as indicative.
- rvz 4y ago
  > Don’t mistake the fallibility of chatgpt for a problem with the entire field. It would be like saying transistors don’t have much of a future because they are too big to fit many in a device. Fundamentally, GPT itself and LLMs are essentially still black-box neural networks and all have the same drawbacks as them: * More data needed to train them. Which then... * ...Increases the size of the model (Getting bigger on each revision) * Opaqueness (Cannot transparently explain its own decisions when it confidently generates the wrong answer.) * Massive cost involved in training, retraining and fine-tuning it. The best part is that you can't even give an explanation on why it is generating nonsense or why it overfitted or got confused over an invalid character or input. Little to nothing fundamental about neural networks (which is what LLMs are based on) has changed. It is essentially more of: 'Just waste more money on training it on more data.' > The Bing openai integration is already dramatically different and better than chatgpt. I’d really advise trying it before forming too many opinions. What I have said about ChatGPT and GPT-3 still holds true and hyping about a chatbot confidently generating nonsense and that cannot cite its own sources makes it highly untrustworthy to use. It is always eternal to the shortcomings of neural networks fundamentally; which is what LLMs use. They are great sophists and bullshit generators. At least what needs to exist is an open-source LLM that is more transparent and is smaller than GPT-3 or GPT-4. There is a basic reason why OpenAI won't do that.

bbor 4y ago

This is a perfect example of the popular view on here, and in my humble naive opinion it’s completely mistaken. The point isn’t “can an LLM replace google” the point is “can robots that can speak English and use logic improve the search experience” which I think basically everyone would answer “yes” to. Complaining that it gets stuff wrong when not hooked up to a web of resources to cite is, IMO, completely missing the point. Also OP (not so much you) is way too caught up in the “chat” aspect - that is the first exciting UX that got popular, but these are much, much more than chatbots. Pretending that they’re human/conscious/in a conversation is fun, but having an invisible partner that knows you and tailors your search results… that’s powerful. For example, you’ll never have to add “Reddit” again, or at least you’ll only have to tell it once. An LLM can easily identify the kind of questions where you want forum posts, read thousands of posts in a second, summarize all their content, and label each link with other information that helps you decide which threads to read in full. I can’t wait!

pwinnski 4y ago
If the point isn't how an LLM replace a search engine, then why is Bing using an LLM to replace their search engine? When you ask whether speaking English and using logic can improve the search experience, I wonder what you consider the most important parts of the search experience? I think many people, most of the time, might say that "accurate information" is their highest expectation, with "a pleasant conversation" somewhere below that. Delivering a plausible-sounding and pleasant answer that's completely wrong is... well, that's not a search engine I can depend on, is it? You're hypothesizing a few things at the end that sound great! It's completely unclear whether any of those things will actually end up happening, so I think the focus on what is available today, with Chat-GPT and Bing, is more apt than a focus on what could be.
- bbor 4y ago
  My basic answer is “they’re trying to rush stuff out the door because the people running bing have no idea what they’re doing” :) given that the things I propose don’t need any new inventions, I’d say they’re good to discuss and coming in the next few years for sure. And I totally agree that a) a search engine that doesn’t cite its sources is useless, and b) you almost never want to chat with google like it’s a person. So you’re spot on. But the point I was trying to make is that the main use case is in stuff like automated summaries, specialized page rankings, expanding quick informal queries into longer formal-logic ones, etc.

marban 4y ago

People who want to get rich will tell you it's the next greatest thing that will revolutionize the industry. The nice way of saying grifters.

braindead_in 4y ago
Or Entrepreneur maybe?
rchaud 4y ago
They go where the funding is.

725686 4y ago

"Personally, I've been annoyed at how confidently wrong ChatGPT can be. Even when you point out the error and ask it to correct the mistake it comes back with an even-more-wrong answer" That also happens with real people so... A web search also returns wrong answers, because it is not magic. It just searches through all the garbage out there. You just have to be aware of its flaws and limitations... as you do with you fellow humans.

smith7018 4y ago
My God, I'm so tired of this response. It's not a person. It's software. It shouldn't be wrong just like humans. Imagine opening your banking app and your total savings is wrong. Would you say, "It's okay, people make mistakes, too!" or would you be pissed that your bank's software is incorrect? Why are we pretending like this is any different?
- rvz 4y ago
  There you go. The difference is that a human can transparently explain to you step by step how they have concluded to a given answer. Humans have always used multiple methods to explain their decisions transparently when questioned to. AI machines and especially neural networks which LLMs are based on still cannot explain themselves and are as transparent as a black-box.
- JohnFen 4y ago
  Because LLMs aren't doing calculations that result in precise mathematical answers like your bank should be doing. All they are is very advanced pattern-matching machines. They match patterns, but those patterns don't always relate to accurate information.
- 8note 4y ago
  I'm not pissed when google doesn't have a good result for my search on the first page, no.
flir 4y ago
A real person would say "I don't know" occasionally.

shanebellone 4y ago

"it's essentially really deep auto-complete" This is the most accurate description I've even read.

shanebellone 4y ago
With that that in mind... cell phones have offered a similar feature to billions of users for many years. Those autocorrections are still frequently wrong.

AutoDunkGPT 4y ago

so what you're saying is, that it's my time to shine, right?