6 ms·

The criticisms regarding chatGPT remind me of what was said about Wikipedia at its very beginning, that it was supposedly unreliable. I think we will have a goo

by davibu 4y ago

The criticisms regarding chatGPT remind me of what was said about Wikipedia at its very beginning, that it was supposedly unreliable. I think we will have a good laugh in a few years reading these first comments. There is no doubt that chatGPT is the future. It is certainly perfectible, but the existing basis is a revolution in progress. In my opinion, there are two essential things missing for chatGPT to become the perfect replacement for Wikipedia and Google: - The ability to activate a "system 2" or slow thinking (theorized by Daniel Kahneman) - The ability to cite sources And the cherry on the cake would be the ability to interact with images

zozbot234 4y ago
You know that Wikipedia was highly unreliable in its early days, right? What ChatGPT needs if it's to be genuinely useful: https://xkcd.com/285/ https://xkcd.com/285/
- SllX 4y ago
  It’s still unreliable. There’s no guarantee a given page that’s there today and good enough to serve a purpose won’t be considered not notable enough to remain tomorrow.
HarHarVeryFunny 4y ago
I think the BS-generation problem with ChatGPT goes far deeper than citing sources, for a variety of reasons. 1) It's not a search engine, even if it behaves a bit like one. It's not "retrieving answers" to your questions (from sources that it could choose to cite). ChatGPT is really just a "language model", so it has no notion that what you're typing is even a question/query .. your input is just treated as sequence of words (which ChatGPT has zero understanding of), with ChatGPT's response then being a further sequence of words that it has calculated are (one) statistically probable continuation of what you typed (you can keep asking it for alternative answers, and it'll continue generating additional alternative statistically probable continuations). The websites/etc that ChatGPT was trained on are just sources of language that it consumed in order to learn the statistics that let it make these continuation predictions. It's not memorizing "facts" from websites, just word statistics, and these are mixed in with the statistics from all the other sources it was trained on. If it generates the word "walk" as part of a response, it can't cite a source for that since there essentially is none - only a bazillion text sources it was trained on that collectively made the word "walk" a high probability continuation on the words it had generated leading up to that... 2) Even if ChatGPT had been designed to deal in "facts" (rather that words statistics) associated with specific sources, the bullshit problem isn't just knowing the varied reliability of the sources it was trained on, but how those "facts" are combined. To combine multiple facts and correctly deduce something new from them would require intelligence, but ChatGPT doesn't have any intelligence - it's just a statistical word generator, so the way it combines snippets from different sources is again just statistical word generation, with zero knowledge of the meaning of the words it is generating or whether it makes sense! What makes ChatGPT seem semi-intelligent is that a lot of what it was trained on was text written by semi-intelligent humans, so the "sequence of words" it is generating, following the statistics of human speech, seems like something a human might say... until you start paying attention to the meaning of the words and realize it's often good-sounding garbage.
- Nathanba 4y ago
  the thing is.. ChatGPT doesn't have to compete with perfectly correct information because the information you search for on Google is often wrong(=SEO spam) too and you have to sift through a lot of garbage or misleading links there too. Sometimes literally, because you get a forum link with a bunch of people saying wrong things and then finally someone says the answer. That's similar to what you have to do on ChatGPT to doublecheck or ask a follow up questions or read more or treat a piece of information with a dose of healthy doubt. Both ChatGPT/Google are very useful and they both produce imperfect results and they both require some human thought.
  - marstall 4y ago
    interesting point about SEO. Does ChatGPT somehow filter out SEO content? Is it rather selective about the domains it crawls? Because Google could certainly turn that switch too - but then it would lose its comprehensiveness ...
- Animats 4y ago
  Which is the big problem. ChatGPT will produce something reasonable if it's seen good content on the indicated subject. Otherwise, it just makes up plausible blithering, including fake references. Useful for fiction, advertising copy, and literary criticism. Not so good for fact retrieval.
- ilaksh 4y ago
  If you go to the OpenAI playground and turn down 'temperature' to 0 it seems to not BS at all as far as I can tell.
  - HarHarVeryFunny 4y ago
    Temperature is presumably referring to sampling the output probabilities. With a temperature of 0 it'll be giving you the very highest probability continuation, while with increasingly higher temperatures it'll be sampling from the possible continuations to provide more variety. In other words, the temperature is controlling the variety of output, but of course doesn't affect what was fed into it in the first place. As the saying goes, Garbage-In, Garbage-Out .. even with a temperature of zero it's still going to be bullshitting since "predict next word" (language model) is fundamentally a bullshitting technology - just keep on spewing out words regardless of meaning.
vikp 4y ago
I don't think it's a huge lift to restrict a language model to "known" good facts from search results. And to have it cite sources. I made a proof of concept this weekend - https://github.com/VikParuchuri/researcher https://github.com/VikParuchuri/researcher . There are some issues, but it's very useful.
- hooande 4y ago
  Fascinating. This isn't at all how ChatGPT works. You're not leveraging the scale of the internet + style transfer to provide answers. Instead it's doing text summarization on search results. It's a very clever proof of concept. Not exactly a large language model.
  - vikp 4y ago
    It's doing abstractive summarization over the search results, using GPT-3. The pipeline is: - Search using Google - Run some filters to exclude SEO spam, etc. - Scrape the pages that are returned - Find chunks of text likely to align with the answer (comparing embeddings) - Feed the most likely chunks into GPT-3 to get a summary It is leveraging GPT-3 to produce better summaries, and it isn't purely extractive - the LLM uses context and knowledge to generate a better summary. I want to experiment with a local model next, versus using GPT-3.
bamboozled 4y ago
"There is no doubt that chatGPT is the future." Why do you say this? It's going to be one of many similar products, even then, it's impressive, it's fun, but is it really useful yet? I think we have to wait and see? By the way, I like ChatGPT and have a lot of fun with it.
- kruxigt 4y ago
  [dead]
bambax 4y ago
> The ability to cite sources It doesn't need to cite sources, because it has learned to make them up on the fly. Liars care about the truth; they want to subvert it. Large language models like ChatGPT don't "lie", they produce bullshit that has zero connection to what is or what isn't, but just sounds like something someone would say in that context. It's possible, even likely, that ChatGPT and the like will help people formulate queries; but if it answers them and people somehow trust its answers, we're all doomed.

by davibu 4y ago

zozbot234 4y ago

You know that Wikipedia was highly unreliable in its early days, right? What ChatGPT needs if it's to be genuinely useful: https://xkcd.com/285/ https://xkcd.com/285/

SllX 4y ago
It’s still unreliable. There’s no guarantee a given page that’s there today and good enough to serve a purpose won’t be considered not notable enough to remain tomorrow.

HarHarVeryFunny 4y ago

I think the BS-generation problem with ChatGPT goes far deeper than citing sources, for a variety of reasons. 1) It's not a search engine, even if it behaves a bit like one. It's not "retrieving answers" to your questions (from sources that it could choose to cite). ChatGPT is really just a "language model", so it has no notion that what you're typing is even a question/query .. your input is just treated as sequence of words (which ChatGPT has zero understanding of), with ChatGPT's response then being a further sequence of words that it has calculated are (one) statistically probable continuation of what you typed (you can keep asking it for alternative answers, and it'll continue generating additional alternative statistically probable continuations). The websites/etc that ChatGPT was trained on are just sources of language that it consumed in order to learn the statistics that let it make these continuation predictions. It's not memorizing "facts" from websites, just word statistics, and these are mixed in with the statistics from all the other sources it was trained on. If it generates the word "walk" as part of a response, it can't cite a source for that since there essentially is none - only a bazillion text sources it was trained on that collectively made the word "walk" a high probability continuation on the words it had generated leading up to that... 2) Even if ChatGPT had been designed to deal in "facts" (rather that words statistics) associated with specific sources, the bullshit problem isn't just knowing the varied reliability of the sources it was trained on, but how those "facts" are combined. To combine multiple facts and correctly deduce something new from them would require intelligence, but ChatGPT doesn't have any intelligence - it's just a statistical word generator, so the way it combines snippets from different sources is again just statistical word generation, with zero knowledge of the meaning of the words it is generating or whether it makes sense! What makes ChatGPT seem semi-intelligent is that a lot of what it was trained on was text written by semi-intelligent humans, so the "sequence of words" it is generating, following the statistics of human speech, seems like something a human might say... until you start paying attention to the meaning of the words and realize it's often good-sounding garbage.

Nathanba 4y ago
the thing is.. ChatGPT doesn't have to compete with perfectly correct information because the information you search for on Google is often wrong(=SEO spam) too and you have to sift through a lot of garbage or misleading links there too. Sometimes literally, because you get a forum link with a bunch of people saying wrong things and then finally someone says the answer. That's similar to what you have to do on ChatGPT to doublecheck or ask a follow up questions or read more or treat a piece of information with a dose of healthy doubt. Both ChatGPT/Google are very useful and they both produce imperfect results and they both require some human thought.
- marstall 4y ago
  interesting point about SEO. Does ChatGPT somehow filter out SEO content? Is it rather selective about the domains it crawls? Because Google could certainly turn that switch too - but then it would lose its comprehensiveness ...
Animats 4y ago
Which is the big problem. ChatGPT will produce something reasonable if it's seen good content on the indicated subject. Otherwise, it just makes up plausible blithering, including fake references. Useful for fiction, advertising copy, and literary criticism. Not so good for fact retrieval.
ilaksh 4y ago
If you go to the OpenAI playground and turn down 'temperature' to 0 it seems to not BS at all as far as I can tell.
- HarHarVeryFunny 4y ago
  Temperature is presumably referring to sampling the output probabilities. With a temperature of 0 it'll be giving you the very highest probability continuation, while with increasingly higher temperatures it'll be sampling from the possible continuations to provide more variety. In other words, the temperature is controlling the variety of output, but of course doesn't affect what was fed into it in the first place. As the saying goes, Garbage-In, Garbage-Out .. even with a temperature of zero it's still going to be bullshitting since "predict next word" (language model) is fundamentally a bullshitting technology - just keep on spewing out words regardless of meaning.

vikp 4y ago

I don't think it's a huge lift to restrict a language model to "known" good facts from search results. And to have it cite sources. I made a proof of concept this weekend - https://github.com/VikParuchuri/researcher https://github.com/VikParuchuri/researcher . There are some issues, but it's very useful.

hooande 4y ago
Fascinating. This isn't at all how ChatGPT works. You're not leveraging the scale of the internet + style transfer to provide answers. Instead it's doing text summarization on search results. It's a very clever proof of concept. Not exactly a large language model.
- vikp 4y ago
  It's doing abstractive summarization over the search results, using GPT-3. The pipeline is: - Search using Google - Run some filters to exclude SEO spam, etc. - Scrape the pages that are returned - Find chunks of text likely to align with the answer (comparing embeddings) - Feed the most likely chunks into GPT-3 to get a summary It is leveraging GPT-3 to produce better summaries, and it isn't purely extractive - the LLM uses context and knowledge to generate a better summary. I want to experiment with a local model next, versus using GPT-3.

bamboozled 4y ago

"There is no doubt that chatGPT is the future." Why do you say this? It's going to be one of many similar products, even then, it's impressive, it's fun, but is it really useful yet? I think we have to wait and see? By the way, I like ChatGPT and have a lot of fun with it.

kruxigt 4y ago
[dead]

bambax 4y ago

> The ability to cite sources It doesn't need to cite sources, because it has learned to make them up on the fly. Liars care about the truth; they want to subvert it. Large language models like ChatGPT don't "lie", they produce bullshit that has zero connection to what is or what isn't, but just sounds like something someone would say in that context. It's possible, even likely, that ChatGPT and the like will help people formulate queries; but if it answers them and people somehow trust its answers, we're all doomed.