6 ms·
You know, whatever the merits (and it sounds as if there were very few) of Google's presentation (or Pinchai's recent non-statement) might be, I would have expe
by PaulDavisThe1st 4y ago
You know, whatever the merits (and it sounds as if there were very few) of Google's presentation (or Pinchai's recent non-statement) might be, I would have expected the commentariat here at HN to display way more skepticism about language models as a replacement for search. If there's a crowd anywhere in the world large enough to leave 200+ comments on a post like this, you'd think thatg the one at HN would understand these models' currently unsolved problem with lying and fabulation.
And yet when I read the comments, I see this sort of calm acceptance that of course this technology is going to take over search. Why is everyone so confident? The performance of these models is simultaneously jaw-dropping and absurd. With no proposed solutions to the problems that they fundamentally face, why this level of belief that their ascendance is inevitable?
- sho_hn 4y agoI'd like to have more discussion about the attribution problem, specifically. We now have a couple of the players saying they're working on, or having demos. But from what I can see, all of these don't _actually_ attribute the source. They're just able to find _a_ source that fits to the output, working backward from it. Often the attributed source fits, but is actually disagreeing with the original output in specific details. In other words, it's not backing the output at all. At most it's a "you could check out these links and compare yourself and it might help you judge our accuracy". Is there any indication real attribution is coming?
- rchaud 4y ago> Is there any indication real attribution is coming? I wouldn't hold my breath. Showing attributions allows people to reverse engineer how the prompt was made, and identifies weaknesses in the model. Bad for business!
- sho_hn 4y agoI feel like all of the interesting stuff when it comes to search+LLM is in real hybridization that's still a few steps out from what's happening right now in terms of productization. For example: (a) Addressing the recency problem by making a model generate instructions to a browser as output token stream, so it can, well, Google for you and incorporate latest info into output by adding it to the "internal prompt". (b) Doing something similar where you teach models how to drive computational engines to make actual, computed facts and figures, not guess them from statistics over training data. (c) Maybe running a traditional search and then using LLMs to summarize over them. etc. etc. It feels increasingly clear that "complete prompt from a stale training data set" is a neat trick but doesn't solve any particular problem well enough. This also seems intuitively true: If you ask your friend a difficult question, you often don't expect them to be able to answer it on the spot. You probably expect them to have relevant experience and a rough understanding of the topics and the tools to look up or figure out the rest. Similarly, it feels intuitively wrong to expect a LLM to come ready-made with all knowledge baked in. It feels more like we'd want something with the working knowledge/method to plan and execute fact discovery and computation. It's fun to play around a bit with ChatGPT yourself and try to approximate what a more complex pipeline might be. For example, instead of asking it to answer a question, ask it to re-state the question or make a list of questions prompted by the original question, then let it "contemplate" those or propose a method for how to go after answering them, etc. It runs aground pretty often, but sometimes the results take you a lot further than the direct approach.
- PaulDavisThe1st 4y ago> This also seems intuitively true: If you ask your friend a difficult question, you often don't expect them to be able to answer it on the spot. You probably expect them to have relevant experience and a rough understanding of the topics and the tools to look up or figure out the rest. Similarly, it feels intuitively wrong to expect a LLM to come ready-made with all knowledge baked in. Not sure. In theory, this being computers and all, I sort of do expect my search engine to effectively come "ready-made with all knowledge baked in". I do not expect Google to start a new web crawl because I gave it a novel query, and I don't expect a language model (certainly not one being used for search purposes) to say "hang on, I need to go read some more before I answer that".
- sho_hn 4y agoI've seen some studies that call what I mean a "chain of thought" approach (also reasonable is just "higher-level reasoning", but maybe we need a term for the engineered implementation), and I like that terminology. I don't know, it's a gut feeling, but a one-shot prompt completion feels intuitively too simple for me - it seems like we need to build up more elaborate "thought processes" next where we use these methods for individual steps in those thought processes, but also have steps that can carry out a computation or go on a fact search. A bit similar, again, to how humans will answer plenty of questions on the spot from working knowledge, but are also able to classify when they need to drop into a slower "let's do the math" mode and then carry that out. We may be able to from these "thought processes" then generate training data sets that again reasonably cache "all possible accurate answers" as you demand, but it's not clear to me if that caching will really save computational cost or just redistribute it to different times. At that point it's probably a latency/throughput question of when you do vs. defer the "thinking".
- PaulDavisThe1st 4y agoThat's undoubtedly true for "reasoning". I don't think it is true for "search". When I ask <search engine> a question that is "demonstrably" answerable based on "reliable" sources (e.g. what is the capital of Mexico), I don't need or expect it to reason. I want it to (a) tell me the most likely answer, probably (b) provide a list a of URLs that also provide an answer, probably the same one. And that's what Google and others do right now; ignoring the results presentations being skewed by commercial interests, they do it rather well. Figuring out when a question is not answerable in this way, but requires reasoning, is certainly a part of the challenge.
- uoaei 4y agoWhat viability are top execs seeing in a business model that is so fragile as to be defeated by reverse-engineering these prompts...? There is obviously no competitive advantage here for very long. The systems must provide an additional service on top of these models, something beyond "attaching it to a publicly-available input form". It needs to aid in reasoning, not act as a god.
- deleted 4y ago[deleted]
- t-writescode 4y agoLook at the images in the announcement from yesterday: https://www.theverge.com/2023/2/7/23587454/microsoft-bing-edge-chatgpt-ai https://www.theverge.com/2023/2/7/23587454/microsoft-bing-ed... Those look like citations, to me.
- jerf 4y agoI was musing on my website about ChatGPT [1] and one of the things I noticed about it is that if you analogize it to your own brain, which has a language model in it too, we are collectively asking ChatGPT to do something we would never ask of our own language model. ChatGPT does not have the "higher level" reasoning that we're asking for, to do logic, attribution, and the other things we want. This is not necessarily a "criticism". It is obvious that ChatGPT produces an incredibly rich and meaning-filled representation of the text, as the extension of the text it can be used for demonstrates. But we should not be trying to "tweak" the language model here and there, trying to make a language model do something it isn't really suited for. We should be working on how to hook up that higher-level functionality. It is not obvious how to do that, true. But it at least stands a chance of producing the AI in reality that people think we have now, but don't. For that reason, I don't think attribution is coming without a significant addition to the architecture. Language models shouldn't be doing attribution; the mess you'd create of them to accomplish that would ruin them as language models. Can you imagine how insane you'd go if your internal language model tried to stuff down every instance of when you have seen and/or used the word "brain" every time you tried to use the word "brain"? That's not what language models are for. That's what the thing using the language model does. I expect AIs to parallel this. I see no reason at all, and abundant reasons to the contrary, to expect AIs to just be undifferentiated blobs of numbers. I expect them to have structures just as our brains do, and for the exact same reasons. If undifferentiated blobs of identical neurons was the way to go, our brains would work that way too; it's far simpler than what we actually have. [1]: https://www.jerf.org/iri/post/2023/understanding_gpt_better/ https://www.jerf.org/iri/post/2023/understanding_gpt_better/
- sho_hn 4y agoYep, this resonates heavily with my own thinking. Thanks!
- Jensson 4y agoYou don't remember where you learned the things you know from? How can you trust anything you think then? Can such people get anything done at all, seems basically impossible to learn new things then since you can't know if it is better than what you thought before? So I don't think that remembering sources is a hard problem to solve, likely making the AI work well is probably easier if it remembers sources.
- t-writescode 4y ago> Is there any indication real attribution is coming? Looking at the main article on HN yesterday around it: https://www.theverge.com/2023/2/7/23587454/microsoft-bing-edge-chatgpt-ai https://www.theverge.com/2023/2/7/23587454/microsoft-bing-ed... you can see citations in the examples provided.
- sho_hn 4y agoI've had those Bing images in mind specifically when writing the above, because I seem to recall them phrasing it very carefully as "you can look at these links for more information", not "we retrieved this info here".
- vineyardmike 4y ago> But from what I can see, all of these don't _actually_ attribute the source. They're just able to find _a_ source that fits to the output, working backward from it I feel like it should be possible to build attribution "simply" by finding good results via traditional search (or even using LLM embeddings), then asking a LLM to summarize the sources. Then you can show the attribution. This would (to me) be much more meaningful. Especially for specific searches ("I want a red oversized couch thats got good back support"). Don't tell me about couches then find a random link, find a couch then tell me about that couch. Is anyone doing this? IDK, maybe there is a reason that it's not happening, but it seems at least as easy as all the other crazy (and cool) things happening in this space.
- nmca 4y agoGopherCite, Sparrow, WebGPT and most likely the new bing (WebGPT2) all first find sources then generate output, so you're just mistaken. The first two specifically get folks check whether the selected source snippet supports the answer. Source: worked on GopherCite.
- rchaud 4y ago> And yet when I read the comments, I see this sort of calm acceptance that of course this technology is going to take over search. Why is everyone so confident? It is difficult to get a man to critically evaluate something if his future salary depends upon his unquestioning acceptance of it.
- red-iron-pine 4y agoSV and Wall Street loves a smart guy who bucks the trend, big risk taker. Mostly a question of pucker factor -- how big of a bet in favor or against, and how big of an impact on the job will you risk?
- rchaud 4y ago> SV and Wall Street loves a smart guy who bucks the trend, big risk taker. Only if they hit out of the park. You don't see a lot of founders getting a second bite of the Sand Hill Road apple if they failed the first time.
- pasquinelli 4y ago> If there's a crowd anywhere in the world large enough to leave 200+ comments on a post like this, you'd think thatg the one at HN would understand these models' currently unsolved problem with lying and fabulation. if i'm being frank, that doesn't match my experience with HN at all. chatgpt is the hot new thing, of course people here are preoccupied imagining a future powered with it. there are a bunch of different types of people here, with different perspectives that might be at odds with one another, but they don't necessarily comment on the same articles.
- PaulDavisThe1st 4y agoin general, i often post similar comments on HN, disputing the idea that there is such a thing as "HN groupthink". what puzzles me here is the the technical limitations of language models (i refuse to call them "large" because the sizes are going to change dramatically) seem so obvious and so well known, that i just find it hard to believe that there are more than a handful of people on HN who uncritically accept the idea that Bing is trying to sell.
- pixl97 4y agoWith that said I'm not sure the cynics on HN are scoring much better (we've predicted the last 5000 of 10 major failures meme). The cynic in me said crypto was overhyped by con men and mostly useless, and I was right. At the same time I said the same about most social media and yet the masses flocked to it to give their personal information away.
- PaulDavisThe1st 4y agoNo worries, I was #2 at amzn and said it would never work :)
- eric_cc 4y ago> disputing the idea that there is such a thing as "HN groupthink". There 100% is HN groupthink. Any doubts about it should have been erased by observing HN’s position on bitcoin and cryptocurrencies.
- uoaei 4y agoAny numerical system operates only on numerical, not logical/grammatical/whatever constraints. Wrongness is built into any system based solely on neural components by the simple fact that rightness is unverifiable from within these systems' own "logical" (read: associational) apparati. I share your confusion regarding the grave dearth of incredulity in the reception of these technologies.
- jakeinspace 4y agoI guess you can say the human brain operates only on the data it gets from our ear drums, optic nerve, etc. Might as well call that numerical input, it’s certainly not grammatical or logical, all that is internal and arises after many layers of neurons.
- uoaei 4y agoI agree: the human (or any organic) brain is no more "logical" than any other numerical system. We obscure this fact (mostly) successfully by virtue of the statistical regularities with which we are imprinted as a result of our experiences. This gives the appearance of logical rigor, but this rigor is only illusory.
- zzzeek 4y agosimple answer is the consensus groupthink on Hacker News is deeply wishful i made this same point at https://news.ycombinator.com/item?id=34699087 https://news.ycombinator.com/item?id=34699087 and got nothing but pushback how there is no real problem with ChatGPT and Bing just fixed all the problems (which didn't exist) anyway.
- tarsinge 4y agoThat thread is mostly about Google completely messing up their response to what Microsoft announced yesterday. I don’t see that unanimous acceptance, there are a lot of interesting discussions about the shortcomings. There are many comparisons between Google results for examples, with also potential solutions to problems you cite like https://news.ycombinator.com/item?id=34709883 https://news.ycombinator.com/item?id=34709883 It’s not about replacing search overnight, but if it can take even 10% of search volume that would have serious consequences for Google.
- PaulDavisThe1st 4y agoIf it takes even 10% of search volume (in its current technological form) then those of us working in this domain have failed, miserably. I have no idea what Microsoft is thinking. It's like a state hiring a known liar to be its star witness in some notable number of court cases. [ EDIT: OK, so one thing they might be thinking is to just this sort of technology to generate readable/listenable summaries of search results obtained by other means. I can see much less wrong with that, but it's also not really the same technology at all as ChatGPT et al. ]
- pixl97 4y agoHeh, this is where I think you may be wrong about the future.... We vote in known liars all the time. People tend to like a good story better than the truth. If this thing is good at storytelling we could be in trouble.
- jjkaczor 4y ago...It's like a state hiring a known liar... Even worse than a star witness, the US let a known liar run their entire country for 4 years... So, frankly... people will love ChatGPT and other similar clones - people love bullshit.
- 1vuio0pswjnm7 4y agoAssuming that Google is in a panic over ChatGPT, as suggested the HN commentariat and the "tech" news media, then arguably this only highlights how little confidence Google has in the quality of its search engine and the trustworthiness of the web as a source for information. For example, instead of trying to match or beat ChatGPT, whatever that would entail, Google could instead focus on illustrating the shortcomings of conversational AI versus a search engine as a reliable and trustworthy means of locating information on the web. As usual, it appears "tech" companies have an extremely warped view of web users as being non-discerning consumers of whatever garbage "tech" companies feed to them. Perhaps online advertising services companies are not a good choice to serve as arbiters of data/information search and retrieval.
- PaulDavisThe1st 4y agoAbsolutely. In part, my comment targets Google itself as much as the HN commentariat, since the corporate communications, as you note, suggest a distinct lack of confidence in what they already do, and a great deal of over-confidence in what LM's might bring.
- voisin 4y ago> arguably this only highlights how little confidence Google has in the quality of its search engine and the trustworthiness of the web as a source for information. I think they are realizing they killed their golden goose by filling it with trash ads and it only worked in the absence of something more accessible to the public. They were drunk at the wheel and crashed into (for the first time ever) a competent competitor. Now they realize they need to sober up quickly but their organization is lazy and bloated, they’ve taken their users for granted and abused them at the altar of advertisers, and generally done nothing for two entire decades to garner any form of loyalty. They’ve got some work cut out for them. From my perspective, Google could come out with a ChatGPT killer and I still wouldn’t use it, because the company has a track record of arbitrarily killing off any product I’ve actually liked.
- jozvolskyef 4y agoThe disruption isn't hypothetical. People like me are excited about the technology because it already has disrupted search for us. My brother has been using it for the majority of his queries because he doesn't speak English and ChatGPT is a gateway to the knowledge that only exists on the English-language internet.
- PaulDavisThe1st 4y agoI wasn't disputing that the disruption exists. I was questioning whether it should exist, given the technological limitations of LMs at this time.
- potatolicious 4y agoAgreed heavily - and I work with these technologies regularly. There is a mania around it much like there was for crypto (though I would maintain there's more merit here than there ever was with crypto - orders of magnitude more). There are a litany of problems that surround LLMs that are solvable, but are highly non-trivial. The present hyped demos all seem to involve mainlining the unfiltered output of LLMs directly at the user, and this approach strikes me as largely impractical, with all of the issues you've brought up and more. Ultimately I think we will land on a few learnings, some of which are IMO pretty obvious already with existing ML: - There are going to be lots and lots of use cases for LLMs where despite its additional complexity it won't out-perform simpler ML models, or even heuristic-based solutions. This has been a plague for AI startups forever - where the shameful truth is that the ML bits do not significantly outperform heuristic-based approaches, but the ML bits sure as useful for the hype machine. There will be more of these use cases than use cases where LLMs actually do significantly move the state of the art forward. - In most broadly mainstream-palatable applications the LLM will need to be intermediated by many other systems - some using other ML models, and some using heuristic-based solutions. For example for search, mediating the output through a knowledge graph with reliable provenance and accuracy. Realistically there will be vanishingly few use cases that are thin-wrappers around LLMs. Real usefulness of this tech will require very heavy lifting around the core model to make it actually accurate/reliable enough to be useful on a mass scale. The many products spinning up that seem to be premised on very thin wrappers around OpenAI's API will IMO mostly not survive, both because there is no moat and because the results won't be sufficiently useful. - Accuracy matters and will be a thorn in the side of these products for quite some time, even though it is solvable. For some use cases (mostly creative ones with a human in the loop - like creative writing or image synthesis) the lack of accuracy is livable, but for most others the propensity to hallucinate is going to be a major barrier to adoption. Right now you're dealing with an enthusiast audience that's willing to look past glaring and obvious errors in output in favor of what it could be and how far we've come, but a mainstream audience I suspect will be less forgiving. Social media virality around product failure cases will also further punish products. Overall I think what we're looking at here is legitimately a huge leap forward, but this legitimate and well-found excitement needs to be tempered by the fact that some (most?) players in the space seem to be getting in way over their skis, and that truly productionizing this technology is going to be intensely difficult. [edit] To be a bit less doom and gloom about this - I think the real winner here are natural language interfaces, and stepping ever closer towards computer systems that do not need to be actively learned before being used. There is IMO an over-focus on the knowledge-encoding part of LLMs (which is unreliable and prone to hallucination) and not enough focus on the language-encoding part of LLMs (which is what they actually do). For the most part voice/language-based interfaces have had limited traction because they honestly haven't been smart enough to understand user requests with a sufficient level of expressivity and fidelity. I suspect these products (and new ones) will have another crack at this problem.
- labrador 4y agoMicrosoft is doing it right. They display search results on the left and ChatGPT results on the right. It's up to you to decide what is accurate knowing the results on the right are supposed to be a summary of what is on the left.
- ekanes 4y agoFor me, it's that this recently-released product is already competing (in the minds of users, which is what counts) with a decades-old mature product. In terms of accuracy, the "but sometimes it's wrong" perspective reminds me of the initial reaction among some to wikipedia. Yep, ANYONE can edit it. Yep, they could lie... ¯\_(ツ)_/¯
- PaulDavisThe1st 4y agoThere are and always have been mechanisms in place to address lying on Wikipedia, which work with varying degrees of effectiveness depending on the topic. What is the mechanism to address lying by a language model?
- rendall 4y ago> There are and always have been mechanisms in place to address lying on Wikipedia Wikipedia's mechanisms are fundamentally unable to cope with motivated, coordinated gangs that conspire together to push a distorted perspective, particularly on niche articles. Given Wikipedia's prominence, this is a problem that will only get worse.
- mistymountains 4y agoIt’s difficult to disentangle people’s desire for Google to fail due to layoffs and corporate misbehavior from legitimate analysis about how LLMs will cause Google to fail.
- mark_l_watson 4y agoI am not really disagreeing with you, but just wanted to add: Probably the secret sauce to using LLMs for search and other forms of information retrieval is first taking a query and use traditional lookup to find likely information. Then aggregating this data, provide the aggregated data as context, append the query, and pass it to the LLM. Probably some post processing is a good idea also. I used to do this with squad LLMs.
- doomleika 4y agoI would argue the state of search engine(i.e. google) is worse than ChatGPT. Half of the time top of the search gives me non-answers like "it depends". I have to use `stackoverflow` `reddit` or incognito mode to force google at least give me something tangible, and I don't see google going to change in short term considering for them, me going through all 3 pages of search is engagement and ad revenue. ChatGPT might not be ideal, but they are way better than current offering.
- hahahawhat 4y ago"And yet when I read the comments, I see this sort of calm acceptance that of course this technology is going to take over search. Why is everyone so confident? The performance of these models is simultaneously jaw-dropping and absurd. With no proposed solutions to the problems that they fundamentally face, why this level of belief that their ascendance is inevitable?" Because you forgot the nr 1 rule of fight club. It doesn't matter if it has unsolvable problems. The only thing that matters is if it has the momentum and the support to create a bubble and drive economic growth. You're welcome.
- hn_throwaway_99 4y ago> I would have expected the commentariat here at HN to display way more skepticism about language models as a replacement for search. Because plenty of people don't see it necessarily as a replacement for search, but as a useful adjunct. More to the point though, why do people always want to define the "HN commentariat" as some monolithic bunch? I've seen tons of comments and articles on the front page pointing out many issues with LLMs, from attribution, to confident bullshitting, to bias issues. Heck, there was plenty of discussion about a big proposed reason for Google's stock crash yesterday was due to the ad Google used to highlight questions their LLM can answer gave a confidently incorrect response. Whenever I see comments that treat the rest of the HN audience like some monolithic group, especially in the face of plenty of evidence to the contrary, I feel like, ironically, the poster is saying "Why don't you all agree with me?"
- PaulDavisThe1st 4y agoI work really hard not to do this, and have more than once posted a message like yours. However, my point here was that at the time of posting, the comment threads for this HN post were largely devoid of any skepticism about LM+search, and were completely dominated by a calm acceptance that it was all inevitable. And I wasn't trying to say "all of HN believes <X>", but rather "I would have expected there to be more voices in the comment thread expressing skepticism, because I know there are some smart people here" Google's Bard demo certainly tweaked the situation by the end of the day.