8 ms·
The problem with ChatGPT as a replacement for Google is that it was not designed to produce accurate facts, and it shows. This model cut its teeth writing artic
by lolinder 4y ago
The problem with ChatGPT as a replacement for Google is that it was not designed to produce accurate facts, and it shows. This model cut its teeth writing articles about the discovery of unicorns in the Andes[0] for goodness sake! It's a language model, and a very impressive one at that, but language is used to express falsehoods and fiction just as regularly as it is used to express truth.
This doesn't mean that it can't produce accurate facts, most of the time it does! But when it does produce nonsense, it does it in exactly the same tone of authority, so if you don't already know the answer you may well walk away believing an AI hallucination.
And the trouble is it doesn't really matter if everyone here thinks "well, I would follow up each request with research to verify the answer", because most people won't! This is like the Google answer extracts, which fairly frequently mislead by extracting out-of-context quotes, except that there's no way to get the original context and there may in fact be no original context! This makes follow-up research much more complicated than with Google and therefore unlikely to happen. If ChatGPT replaces Google, the amount of nonsense on the internet will get even worse, which is something that until 2022 I never thought was possible.
[0] https://github.com/minimaxir/gpt-3-experiments/blob/master/examples/unicorn/output_0_7.md https://github.com/minimaxir/gpt-3-experiments/blob/master/e...
- Valakas_ 4y agoI can't wait for a few years, when ChatGPT is replaced by something 10^x more powerful, and then comments like this look like nonsense of someone who lived in a past era just like we now similarly make fun of comments from people that made in past eras about how certain technologies, criticized in their current state without the ability of the person to look ahead. Even though the person you're replying to mentioned ChatGPT, ChatGPT is just the FIRST of many. Imagine somebody saying in 1803 "Car is unusable because you have to keep filling the coal, it's just not convenient, and a horse will always have a future."
- MuffinFlavored 4y agodevils advocate, but Google wasn’t designed to make sure the links it aggregates at the top are accurate/isn’t just blog spam designed to cater to specialized SEO tricks
- lolinder 4y agoI agree that Google is terrible at its job and produces garbage most of the time. That's the only reason why an LLM chatbot is seen by anyone as a viable replacement. But at the very least Google always gives you the source, which allows you to read it for yourself and judge its trustworthiness. No such option for ChatGPT.
- dmix 4y ago> Google is terrible at its job and produces garbage most of the time Are we using the same search engine? I use DuckDuckGo most of the time but Google is hardly terrible, even if it could be improved. I find what I’m looking for in the top results 90-95% of the time. The other 5-10% also contains times where there simply isn’t a good link that solves what I’m looking for even after digging hard.
- imron 4y ago> Are we using the same search engine? I use DuckDuckGo most of the time Then you are not using the same engine! (I also use DDG and find its results superior to google).
- dmix 4y agoDDG is much more straight forward vs Google often tries too hard and fails. It's like how Twitter's default feed changed to try to tell you what you wanted to see, while the old chronological feed was the content you originally asked to follow. The simpler feed was always better and less contentious for non-newbie/casual users. Google is better at deciphering what you want, but DDG is straight up: here's the simplest results based on your query. Maybe the solution is both? An evolution of the ML approach that Google uses (ala ChatGPT style mimicking a human) AND offering a tab, or box, or whatever interface, with 'simple' straightforward old-school search results. Until the former is better 99% of the time, they shouldn't abandon the latter.
- tr3ntg 4y agoThe highest placed links on Google are basically guaranteed to: 1) Not be the actual source of content 2) Not link to the original source
- echelon 4y agoThese are early days. Everything is going to move faster than you can keep up. You'll see head-turning headlines on a daily basis. Next year will see AI begin to disrupt so many businesses you'd thought were absolutely impenetrable. I don't think anything is safe anymore. By the end of the decade, I think even manufacturing and robotics will be automated. Everyone from McDonalds to Apple and Ford will feel the pressure. I've got a mind-blowing demo coming out soon that will put even more fields on notice. And that's just my team. There are so many people working on AI-first products now. Incumbents will be overwhelmed and washed away. (If ever there was a cue for a career change or opportunity to make a whole lot of money, it's this. Don't ignore this one! You're still early enough to take advantage.)
- lolinder 4y agoI'm not questioning the AI revolution, I'm questioning the applicability of large generative language models to information retrieval. We need a different kind of model for that, not just a few tweaks here and there to this one.
- echelon 4y agoGiven everything we've seen in the last year alone, I'd not gamble against that kind of model being just around the corner. I can think up architectures I could draw on a napkin that could do this. Focused research teams will nail this. It's coming.
- noitpmeder 4y ago/r/iamverysmart
- smiley1437 4y ago>If ever there was a cue for a career change or opportunity to make a whole lot of money, it's this. Don't ignore this one! You're still early enough to take advantage. Any suggestions on how to take advantage? Dive into ML development? Learn how to write prompts for AI engines? Invest in AI companies? Sadly OpenAI is private, but Microsoft is a monetizing partner I guess (Copilot)
- sillysaurusx 4y agoNeither was Wikipedia. But it works well enough.
- ClumsyPilot 4y agoWikipedia is designed to be as accurate as possible and is one of the few remaining website that produce value and content. Please notice that in current setup of 'Internet capitalism' people that produce value and content earn nothing, and people that harvest that content for profit or engage in outright fraud make fortunes.
- deleted 4y ago[deleted]
- solarkraft 4y agoI believe this to be a huge deal. In no time a huge part of the internet will be AI-generated and we will pollute our data set for research and training AI with things that are subtly or completely false at times and we won't be able to tell. It's a lot like iRobot or other stories in which the robots will start to subtly misbehave, like a program you can't debug.
- cortesoft 4y agoHow is this different than regular intelligence? We grow up trained on a polluted dataset, being taught things that are subtly or completely false, and we often struggle to tell which things we have learned are false.
- ClumsyPilot 4y agoAnd what was the result of unchecked faulty learning on a corrupt dataset by natural intelligence? Dark ages where millions died in pointless wars over who's imaginary diety is the true one. A third of europe dying of preventable plague. It took a rigorous system of citation and a publically funded system of education to crawl out of the drakness. Should we try AI powred dark ages but with resource shortage and nuclear weapons?
- deleted 4y ago[deleted]
- solarkraft 4y agoOnce in a while somebody does some actual on-the-ground research and people are convinced to change their minds through evidence and reasoning. So far AI can't do that, it'll just parrot whatever it hears more often, (apparently) with some degree of randomness. Maybe it'll learn to adopt a new view, but for now it won't be through actual reasoning.
- astrange 4y agoGPT has had up/downvotes for years. They were used to train ChatGPT. It’s not just “what’s the most likely answer”. (The most likely answer is the letter “e”.)
- Cypher 4y agoget the journals open source and add it as a module, although you'd have to strip out the unreproducible social science studies but that could be a good thing.
- solardev 4y agoWhat makes you think Google is good at finding facts?
- verdenti 4y ago[dead]
- lumost 4y agoGoogle links are notoriously inaccurate. Google was always great at finding relevant/clickable links- they never tried to direct you to factual links.
- ClumsyPilot 4y ago> This is like the Google answer extracts, which fairly frequently mislead by extracting out-of-context quotes My favourite Google extracts: Q: Who invented Running A: [random guy] invented running in 1865 when he tried to walk twice at the same time. It was an overnight success Q: How much does a [model of a bycicle] weight? A: 140 KG (Maximum weight of the rider) Q: Define an english Person A: A Cunt
- thret 4y ago1/3 isn't bad.
- est 4y ago> The problem with ChatGPT as a replacement for Google is that it was not designed to produce accurate facts But chat bots could be designed to do that, no? Are there any siginificant technical barrier to prevent so?
- tsimionescu 4y agoYes: accruing a large and diverse enough training set where true and false facts are correctly labeled. The GPT models require enormous amounts of text to be trained on, so manually adding true/false labels to every part of every sentence is impossible (and automatically doing it would require exactly the AI we are trying to train). A revolution in this area would require some entirely new training algorithm that can reach GPT-levels of fidelity while using magnitudes less training material. An even bigger GPT model trained on even more text is not going to do much in approaching a solution to this problem.
- darepublic 4y agoChatGPT in its current form can't replace google but plausible to think in its next few incarnations it will get there. Why can't it have a mode where it returns more rigorous results, and includes some system of links as citation? It can include footnotes for various passages of it's answer to indicate ambiguity, varying levels of confidence, etc.
- enjoylife 4y agoThis is the mental leap folks need to be making, the question is when will this type of configurably factual large language model be research benchmarked and productionized. I have read many similar comments to the grandparent, all saying the same thing, “I tried it, and it turned up wrong facts” all the while they forgot that they were deeply engaged with the content it was producing. Just think of how much our attention is monetized already, money will be poring into this space from here on. It’s only a matter of time.
- lolinder 4y agoMy argument isn't that I tried it and it didn't work, my argument is that an LLM alone fundamentally cannot work for reliable information retrieval. Producing hallucinations or not isn't just a setting you can tweak in the model. The entire function of GPT-3 is to guess the next word from the words that came before. It's a hallucination machine that has been trained on so much real-world data and has such a large parameter set that for commonly known and discussed information it does a remarkably good job at creating factual sentences. But as soon as you get out into territory that it doesn't have memorized, it will do the next best thing and produce credible-sounding new material that may or may not be complete nonsense. Again, this isn't a setting you can tune, it's just what a transformer does. It models human language, not all of human cognition.
- astrange 4y agoChatGPT isn’t trained on next word prediction alone, it was trained on human feedback. The evaluation is next word search, but the model has evolved to give “correct” answers when evaluated that way. So it’s not just the most likely answer anymore. Also, since we can watch the model evaluation and see all its weights, there’s research that shows you can determine if any answer is a “retrieval” of existing knowledge or a hallucination.
- Nathanba 4y agoGoogle was not designed to give you accurate facts either, it was designed to calculate popularity of links based on backlinks and then show you the most referenced links first.
- joe_the_user 4y agoAbsolutely this. I just did a discussion with ChatGPT on intuitionistic logic and it gave answers that were subtly wrong in my understanding of the subject. I have an MA in math but I couldn't be sure whether my impression of its wrongness was correct until asked it for a reference and it gave a nonexistent book by Per Martin-Lof, "Intuitionistic Logic", along with a list of actual books by him. It's an impressive and scary bullshit generator at this point. And if someone can find a reference to Intuitionistic Logic by Per Martin-Lof, Oxford University Press, 1982, well, I'm wrong. But I went through Wikipedia, Google, Worldcat and Google Books.
- elicksaur 4y agoNo, you are probably right. It has no guardrails for generating a true citation. When you ask it for citations, its programming generates the form of citation. As the authors of the article discuss, it has no model of reality to understand that citations are not just a form of words and symbols put together.
- dr_dshiv 4y agoIt’s really good at showing what types of scientific books and papers should exist. GPT5 will just write them, so it isn’t lying.
- joe_the_user 4y agoIndeed and it gave several other actual citation with seemingly correct summaries. And it's definitely less prone to obvious bs than just a few weeks ago. For a lot of general things, it gives a summary paragraph that's seems a distillation of standard information - the main problems seem to appear when these "scripts" lack consistency. Edit: Once it "learns" to get citations right, you'll have to dig into those to see if it's right. It's easy to see that arms not looking good.
- thordenmark 4y agoIt seems that at the current velocity of improvement it is only a matter of time - very near future - that those problems will be mostly worked out. ChatGPT already outperforms Google search in many ways. For example I was doing some research on Scandinavian history and found the results far superior on ChatGPT, while I had to sift through pages of junk on Google. I took the results with a grain of salt just like I would with Wikipedia and verified each point, and found it to be very accurate. With this I can cut my research time down from hours to minutes on a problem.
- 55555 4y ago1. Provide citations like you.com already does. 2. Add Trustrank/Trustflow into the training set
- the_black_hand 4y agoRIP google. There's a joke that the best place to hide a murder is on page 2 of google search. The fact is GPT has at bare minimum parsed the top page of google and reasoned about the top results to give the best answer with as much context of why it reasoned so. GPT is basically a superset of google with added reasoning. I can't see it giving worse results than Google. Both will produce a bogus answer if you ask a bogus question, but GPT will never produce a bogus answer when google has the correct one.
- jchmrt 4y agoI think it's really problematic to use the word "reason" to describe what GPT does. As others have commented in this thread, GPT is a language model, which in no way reasons about anything it writes. It is simply writing whatever fits statistically to the prompt and context. Attributing reasoning or too much intelligence to this could prove quite dangerous.
- astrange 4y ago> As others have commented in this thread, GPT is a language model, which in no way reasons about anything it writes. It is simply writing whatever fits statistically to the prompt and context. Those people are wrong; statistics are a kind of reasoning (intuition) and if you try it, it obviously does the other kinds of reasoning too. I have all kinds of answers from it that are logically coherent and can’t possibly have appeared on the internet already. (Try asking it to write new SCP wiki entries.)
- jchmrt 4y agoJust because some of its answers are logically coherent due to it following statistical patterns, does not mean that the model is reasoning. Reasoning implies some type of discipline which prevents nonsensible arguments (even in uncommon situations).
- astrange 4y agoNo such thing as “is reasoning”, only “does reasoning”. Are you an unreasoning being if you make a math mistake? You can see it “does” reasoning if you say “think step by step” after a question. It can attend over its steps then.
- SillyUsername 4y agoI wasn't aware Google produces accurate facts. Cf. front page links to Wikipedia as an obvious counter.
- anonytrary 4y agoI don't think anyone who has played enough with ChatGPT is championing the notion that ChatGPT itself will replace Google. They are more so steel-manning the argument that Google might not be the best interface for search.
- creatonez 4y ago> This model cut its teeth writing articles about the discovery of unicorns in the Andes This is a different model, called GPT-3, released in 2020. ChatGPT has the same number of neurons, but is trained with both human feedback and web data, rather than just web data like GPT-3. What you're saying still holds up, though -- ChatGPT still hallucinates information.
- k__ 4y agoIsn't ChatGPT just a conversational interface for GPT-3?
- astrange 4y agoYes, the human feedback model is called InstructGPT and has been the official “GPT-3” for some time now. ChatGPT is only somewhat more trained, but unlike the playground, it’s free and has a Markdown renderer.
- creatonez 4y agoNo, it has different training, so entirely different neuron connection weights. They are very similar though.
- xbmcuser 4y agoTruthfully the internet is already full of crap a lot of it by design by different interest groups, countries spreading fud about other countries political parties doing the same for their opposition the web has become a cesspool. Majority of the people I personally assume at least 9 out 10 don't have capability or care to do the research to sift fact from fiction in the first place so this probably won't change anything. I have been trying to make it work for me and teach me some python scripting or creating regexes. If you question it correctly it outputs out something even if you might not be able to run the first time generally it works. I fed it 2 csv files/text asked it if it was possible to go convert 1 format to another got a python script that I was able to convert large number of files. Now scripting and programing might seem trivial to large parts of hackernews audience but it will be a huge change for people in general. From what I feel the AI keeps improving its answers is it being trained on the million of responses it get on the answers it gives.
- williamcotton 4y agoSearch engines are not designed to produce accurate facts either. Verification of facts requires direct experience through careful experimentation. ChatGPT’s responses can either be verified or they are as unfalsifiable as any other confidently expounded conjecture found on your computer screen.
- epolanski 4y agoChatgpt is is beta, we can safely assume the situation may change from now to 2024/2025.
- Retric 4y agoNot at a fundamental level. Even if you exclusively trained one on some reasonably authoritative source like Wikipedia it would still happily tell you complete nonsense. These models have no way to differentiate that elves are fictional while Indians actually exist.
- shaky-carrousel 4y agoGoogle was not designed to provide accurate results either, and it shows. I can find results to confirm every conspiracy theory I can think of. ChatGPT at least denies conspiracy theories.
- summerlight 4y agoIn search results, you can at least evaluate how trustful the document is, based on various factors. This resembles how academia has evolved: you can dig into all the references, see the author's publication history and even their reputation. How would you do this with ChatGPT? We can't even verify the sources of its claim and it sometimes even makes up non-existent references in a very confident tone.