4 ms·

ChatGPT in its current form can't replace google but plausible to think in its next few incarnations it will get there. Why can't it have a mode where it retur

ChatGPT in its current form can't replace google but plausible to think in its next few incarnations it will get there. Why can't it have a mode where it returns more rigorous results, and includes some system of links as citation? It can include footnotes for various passages of it's answer to indicate ambiguity, varying levels of confidence, etc.

enjoylife 4y ago
This is the mental leap folks need to be making, the question is when will this type of configurably factual large language model be research benchmarked and productionized. I have read many similar comments to the grandparent, all saying the same thing, “I tried it, and it turned up wrong facts” all the while they forgot that they were deeply engaged with the content it was producing. Just think of how much our attention is monetized already, money will be poring into this space from here on. It’s only a matter of time.
- lolinder 4y ago
  My argument isn't that I tried it and it didn't work, my argument is that an LLM alone fundamentally cannot work for reliable information retrieval. Producing hallucinations or not isn't just a setting you can tweak in the model. The entire function of GPT-3 is to guess the next word from the words that came before. It's a hallucination machine that has been trained on so much real-world data and has such a large parameter set that for commonly known and discussed information it does a remarkably good job at creating factual sentences. But as soon as you get out into territory that it doesn't have memorized, it will do the next best thing and produce credible-sounding new material that may or may not be complete nonsense. Again, this isn't a setting you can tune, it's just what a transformer does. It models human language, not all of human cognition.
  - astrange 4y ago
    ChatGPT isn’t trained on next word prediction alone, it was trained on human feedback. The evaluation is next word search, but the model has evolved to give “correct” answers when evaluated that way. So it’s not just the most likely answer anymore. Also, since we can watch the model evaluation and see all its weights, there’s research that shows you can determine if any answer is a “retrieval” of existing knowledge or a hallucination.
lolinder 4y ago
For the same reason why it might produce a bogus answer: it's a language model, not a general AI. Prompt it for footnotes and it will generate footnotes for books that don't exist or that don't actually contain the citation. They'll match the form of a footnote, because that's what the model was trained to do, but the citations won't resolve. For example, I asked who the first president of Argentina was. It gave the wrong name (Juan Manuel de Rosas) with a half-true bio. I then asked it for books where I can learn more about this person. It produced five books. None of the book-author pairings were correct, though three of the books were real and four of the authors actually wrote books. Of the four authors that existed, three actually did write about Argentina's history, while one is a professor of Spanish and gender studies. This is my point: ChatGPT was trained to produce text, not to produce facts. What I'm seeing here isn't just something that can be optimized away, it's fundamental to the model's design. It was basically told to produce text that looks like it could have been written on the internet. Its text is really convincing: when I saw the list of books I thought for a minute it had actually done it! But it's still just inventing text that reads well, because that's what its training was optimizing for. It could be paired with a different system that searches for facts, but then it's just a friendly abstraction layer on top of Google or some other system, not a revolution in search.
- hasmanean 4y ago
  That makes sense. The only thing about gpt that impressed me was when I asked it to condense an answer into one sentence, it was able to do it. Other than that it betrayed no understanding of the material. It rather resembled a high school kid who had read a few too many books and spouted platitudes about any and all subjects without being aware of what was really going on.
- bandrami 4y ago
  OTOH I could imagine a hybrid situation where the AI Bot searches Google for you and restates the millions of results Google gives it
  - SV_BubbleTime 4y ago
    This was my immediate thought as well. I don’t need millions of searches done, I need 20, then a language model to look through them and collect text that helps me find what I’m looking for. The lucky thing for Google if they got it working, is they are half there already. As much as I despise Google, I have to admit GPT won’t be their death.
godelski 4y ago
Because to make it focus on always producing accurate results (or rather a very low false positive rate) you need to dramatically change and tune the model. Which is much harder than what Google does because it means you'd also need to vet a lot of facts as well (by weighting certain sources, not individually). Or allowing it to search the internet (which also comes with other issues and if it is using Google to do that then it doesn't really kill Google). I do think Google's knowledge graph is more in line with what you're actually thinking. Lots of Google searches can be done without actually clicking links. There's also the ethical question. For example, duckduckgo will show you stack overflow answers. Google could easily do this but they have already gotten flack for displaying answers and taking away clicks from sites, which means reduced ad revenue for the resource actually being used. We could make even stronger arguments about services like ChatGPT in their current form, let alone the one we're talking about. If we think a bit further in the future there's another extension that might not be what we want. Suppose our model contains all current human knowledge and can accurately give you answers and even produce paragraphs at will. This kills off a lot of websites. Is this a good thing? Suppose we incorporate an image generator along similar lines and now we have text and image search. That's a huge centralization force. There are a lot of great benefits to this that I don't want to downplay, but we should also think about if this is actually what we want because there are downsides too. I thought I should also add that I had an at length discussion of these LLMs being Stochastic Parrots (an how we know they are) in a thread from 2 days ago https://news.ycombinator.com/item?id=34113011 https://news.ycombinator.com/item?id=34113011
- prox 4y ago
  What we want is not what we get unless it makes bank. Nobody (generalizing here) works to better humanity, they work to get money. if it aligns well that’s just shear luck. The world needs a lot more people who want see the world to succeed.
  - godelski 4y ago
    I think this is a highly pessimistic view. While I know many that are profit motivated and I won't argue that this isn't the majority, I know many people that just want to pursue knowledge for the sake of knowledge. I'll tell you that personally my dream is to just be able to research and not worry about constraints like publishing (I'd rather open blog) or computational constraints (don't need a data center. A few dozen A100s is more than enough). There are a lot of people like me and a lot that would be like that if they didn't have financial constraints. Having this dream is a privilege and even I didn't have it when I was living paycheck to paycheck. So I think if you want to see more people with my dream then push towards a post scarce world. We'll still have plenty of resource constraints, but it does change the game a lot. I think you'll see more artists as well. A new renaissance. But I fear the transitionary period might be quite difficult and disruptive. I do think such a future has a fairly good probability of happening within many of our lifetimes and almost guaranteed if we actively pushed for it (or solved adjacent problems like HLI). So don't be so pessimistic. There's millions of people out there that are interested in pushing the bounds of human knowledge because that's what they want. I'd argue that physicists and mathematicians wouldn't exist if this weren't true. They get paid shit for extremely difficult work. We could say the same about many sciences. While not all of these people are in it for bettering humanity (just for pursuing knowledge) it often aligns and there is a decent portion who believe this is the point of science and our purpose, as humans, in life.
dragonwriter 4y ago
> ChatGPT in its current form can’t replace google but plausible to think in its next few incarnations it will get there. I don’t think it is. > Why can’t it have a mode where it returns more rigorous results, and includes some system of links as citation? Because it is a statistical text-generation model; connection of results to specific sources is fundamentally not part of the way it works, so it can’t just be asked to supply them. You’d need a fundamentally different approach to do that.
singularity2001 4y ago
Contrary to all the naysayers I think a future version of GTP will be trained to evaluate its own certainty and express this verbally. Probably through an extra training channel.
- prox 4y ago
  It has no concept of certainty, unless it was embedded in the original data. What you are describing is an more general AI. This is a language model. I like to think of it of a cognitive intelligence: the ability to process input and compare it to a stunning array of trained data that will have many parallels to reality but has no concept of it.
  - singularity2001 4y ago
    It can be trained some form of trustfulness, and a measure of certainty is intrinsically part of the model (compare with image to text models etc "70% cheetah 10% dog") In any case it can be trained to assert confidence via a side channel

enjoylife 4y ago

This is the mental leap folks need to be making, the question is when will this type of configurably factual large language model be research benchmarked and productionized. I have read many similar comments to the grandparent, all saying the same thing, “I tried it, and it turned up wrong facts” all the while they forgot that they were deeply engaged with the content it was producing. Just think of how much our attention is monetized already, money will be poring into this space from here on. It’s only a matter of time.

lolinder 4y ago
My argument isn't that I tried it and it didn't work, my argument is that an LLM alone fundamentally cannot work for reliable information retrieval. Producing hallucinations or not isn't just a setting you can tweak in the model. The entire function of GPT-3 is to guess the next word from the words that came before. It's a hallucination machine that has been trained on so much real-world data and has such a large parameter set that for commonly known and discussed information it does a remarkably good job at creating factual sentences. But as soon as you get out into territory that it doesn't have memorized, it will do the next best thing and produce credible-sounding new material that may or may not be complete nonsense. Again, this isn't a setting you can tune, it's just what a transformer does. It models human language, not all of human cognition.
- astrange 4y ago
  ChatGPT isn’t trained on next word prediction alone, it was trained on human feedback. The evaluation is next word search, but the model has evolved to give “correct” answers when evaluated that way. So it’s not just the most likely answer anymore. Also, since we can watch the model evaluation and see all its weights, there’s research that shows you can determine if any answer is a “retrieval” of existing knowledge or a hallucination.

lolinder 4y ago

For the same reason why it might produce a bogus answer: it's a language model, not a general AI. Prompt it for footnotes and it will generate footnotes for books that don't exist or that don't actually contain the citation. They'll match the form of a footnote, because that's what the model was trained to do, but the citations won't resolve. For example, I asked who the first president of Argentina was. It gave the wrong name (Juan Manuel de Rosas) with a half-true bio. I then asked it for books where I can learn more about this person. It produced five books. None of the book-author pairings were correct, though three of the books were real and four of the authors actually wrote books. Of the four authors that existed, three actually did write about Argentina's history, while one is a professor of Spanish and gender studies. This is my point: ChatGPT was trained to produce text, not to produce facts. What I'm seeing here isn't just something that can be optimized away, it's fundamental to the model's design. It was basically told to produce text that looks like it could have been written on the internet. Its text is really convincing: when I saw the list of books I thought for a minute it had actually done it! But it's still just inventing text that reads well, because that's what its training was optimizing for. It could be paired with a different system that searches for facts, but then it's just a friendly abstraction layer on top of Google or some other system, not a revolution in search.

hasmanean 4y ago
That makes sense. The only thing about gpt that impressed me was when I asked it to condense an answer into one sentence, it was able to do it. Other than that it betrayed no understanding of the material. It rather resembled a high school kid who had read a few too many books and spouted platitudes about any and all subjects without being aware of what was really going on.
bandrami 4y ago
OTOH I could imagine a hybrid situation where the AI Bot searches Google for you and restates the millions of results Google gives it
- SV_BubbleTime 4y ago
  This was my immediate thought as well. I don’t need millions of searches done, I need 20, then a language model to look through them and collect text that helps me find what I’m looking for. The lucky thing for Google if they got it working, is they are half there already. As much as I despise Google, I have to admit GPT won’t be their death.

godelski 4y ago

Because to make it focus on always producing accurate results (or rather a very low false positive rate) you need to dramatically change and tune the model. Which is much harder than what Google does because it means you'd also need to vet a lot of facts as well (by weighting certain sources, not individually). Or allowing it to search the internet (which also comes with other issues and if it is using Google to do that then it doesn't really kill Google). I do think Google's knowledge graph is more in line with what you're actually thinking. Lots of Google searches can be done without actually clicking links. There's also the ethical question. For example, duckduckgo will show you stack overflow answers. Google could easily do this but they have already gotten flack for displaying answers and taking away clicks from sites, which means reduced ad revenue for the resource actually being used. We could make even stronger arguments about services like ChatGPT in their current form, let alone the one we're talking about. If we think a bit further in the future there's another extension that might not be what we want. Suppose our model contains all current human knowledge and can accurately give you answers and even produce paragraphs at will. This kills off a lot of websites. Is this a good thing? Suppose we incorporate an image generator along similar lines and now we have text and image search. That's a huge centralization force. There are a lot of great benefits to this that I don't want to downplay, but we should also think about if this is actually what we want because there are downsides too. I thought I should also add that I had an at length discussion of these LLMs being Stochastic Parrots (an how we know they are) in a thread from 2 days ago https://news.ycombinator.com/item?id=34113011 https://news.ycombinator.com/item?id=34113011

prox 4y ago
What we want is not what we get unless it makes bank. Nobody (generalizing here) works to better humanity, they work to get money. if it aligns well that’s just shear luck. The world needs a lot more people who want see the world to succeed.
- godelski 4y ago
  I think this is a highly pessimistic view. While I know many that are profit motivated and I won't argue that this isn't the majority, I know many people that just want to pursue knowledge for the sake of knowledge. I'll tell you that personally my dream is to just be able to research and not worry about constraints like publishing (I'd rather open blog) or computational constraints (don't need a data center. A few dozen A100s is more than enough). There are a lot of people like me and a lot that would be like that if they didn't have financial constraints. Having this dream is a privilege and even I didn't have it when I was living paycheck to paycheck. So I think if you want to see more people with my dream then push towards a post scarce world. We'll still have plenty of resource constraints, but it does change the game a lot. I think you'll see more artists as well. A new renaissance. But I fear the transitionary period might be quite difficult and disruptive. I do think such a future has a fairly good probability of happening within many of our lifetimes and almost guaranteed if we actively pushed for it (or solved adjacent problems like HLI). So don't be so pessimistic. There's millions of people out there that are interested in pushing the bounds of human knowledge because that's what they want. I'd argue that physicists and mathematicians wouldn't exist if this weren't true. They get paid shit for extremely difficult work. We could say the same about many sciences. While not all of these people are in it for bettering humanity (just for pursuing knowledge) it often aligns and there is a decent portion who believe this is the point of science and our purpose, as humans, in life.

dragonwriter 4y ago

> ChatGPT in its current form can’t replace google but plausible to think in its next few incarnations it will get there. I don’t think it is. > Why can’t it have a mode where it returns more rigorous results, and includes some system of links as citation? Because it is a statistical text-generation model; connection of results to specific sources is fundamentally not part of the way it works, so it can’t just be asked to supply them. You’d need a fundamentally different approach to do that.

singularity2001 4y ago

Contrary to all the naysayers I think a future version of GTP will be trained to evaluate its own certainty and express this verbally. Probably through an extra training channel.

prox 4y ago
It has no concept of certainty, unless it was embedded in the original data. What you are describing is an more general AI. This is a language model. I like to think of it of a cognitive intelligence: the ability to process input and compare it to a stunning array of trained data that will have many parallels to reality but has no concept of it.
- singularity2001 4y ago
  It can be trained some form of trustfulness, and a measure of certainty is intrinsically part of the model (compare with image to text models etc "70% cheetah 10% dog") In any case it can be trained to assert confidence via a side channel