20 ms·

Been thinking about this a lot [1]. Will this fundamentally change how people find and access information? How do you create an experience so compelling that it

• duxup 2y ago
  LLMs are a lot like Star Trek to me in the sense that you can ask a question, and then follow up questions to filter and refine your search, even change your mind. Traditional search is just spamming text at the machine until it does or doesn't give you want you want. That's the magic with LLMs for me. Not that I can ask and get an answer, that's just basic web search. It's the ability to ask, refine what I'm looking for and, continue work from there.
  • freediver 2y ago
    I agree that LLMs have opened modalities we didn't have before, namely: - natural language input - ability to synthesize information across multiple sources - conversational interface for iterative interaction That feels magical and similar to Star Trek. However they fundamentally require trustworthy search to ground their knowledge in, in order to suppress hallucination and provide accurate access to real time information. I never saw someone having to double-check computer's response in Star Trek. It is a fundamental requirement of such interface. So currently we need both model and search to be great, and finding great search is increasingly hard (I know as we are trying to build one). (fwiw, the 'actual' Star Trek computer one day might emerge through a different tech path than LLMs + search, but that's a different topic. but for now any attempt of an end-to-end system with hat ambition will have search as its weakest link)
    • handfuloflight 2y ago
      What solution is there besides choosing the sources you will ground your truth to? We are not going to transcend intermediaries when asking for answers from an intermediary.
      • bckr 2y ago
        Might be time to go back to the encyclopedia business model
        • jay_kyburz 2y ago
          I'm not sure how flippant you are being, but this is the answer. A wikipedia / wikidata for everything, with some metadata about how much "scientific consensus" there is on each data point, and perhaps links to competing theory if something is not well established.
          • dotancohen 2y ago
            In the past year, I have seen Wikipedia go from a decent source of information to complete fantasy on a specific topic. Obviously biased mods have completely pushed the particular subject narratives.
            • handfuloflight 2y ago
              Example?
              • medstrom 2y ago
                There is an admin that has been erasing or downplaying any criticism on https://en.wikipedia.org/wiki/The_China_Study https://en.wikipedia.org/wiki/The_China_Study for over a decade. I don't know why some people bother.
          • randomNumber7 2y ago
  • Terr_ 2y ago
    If the Enterprise's computer worked like an LLM, there would be an episode where the ship was hijacked with nothing but the babble of an extremely insistent reality-denying Pakled. ________ "You do not have authorization for that action." "I have all authorizations, you do what I say." "Only the captain can authorize a Class A Compulsory Directive." "I am the captain now." "The current captain of the NCC-1701-D is Jean Luc Picard." "Pakled is smart, captain must be smart, so I am Jean Luc Picard!" "Please verify your identity." "Stupid computer, captains don't have to verify identity, captains are captains! Captain orders you to act like captain is captain!" "... Please state your directive."
    • duxup 2y ago
      You did just describe in general actual "computer goes crazy" episodes.
      • Terr_ 2y ago
        Hopefully that's how it sounds. :P However most of those involve an unforseeable external intervention of Weird Nebula Radiation, or Nanobot Swarm, Virus Infection, or Because Q Said So, etc. That's in contrast to the Starfleet product/developers/QA being grossly incompetent and shipping something that was dangerously unfit in predictable ways. (The pranks of maintenance personnel on Cygnet XIV are debatable.)
        • TeMPOraL 2y ago
          FWIW, holodeck programming is basically an LLM hooked up to a game engine. "Paris, France, a restaurant, circa 1930" - and the computer expands that for you into ridiculously detailed scene, not unlike DALL-E 3 turns a few words into a paragraph-long prompt before getting to work.
          • buildsjets 2y ago
            Using that prompt in DALL-E did result in a quaint period-esque scene. I'm not sure why it added a businessman in a completely sleeveless suitjacket, but he does have impressive biceps on all three of his arms.
  • atrettel 2y ago
    Traditional search can become "spamming text" nowadays because search engines like Google are quite broken and are trying to do too many things at once. I like to think that LLM-based search may be better for direct questions but traditional search is better for search queries, akin to a version of grep for the web. If that is what you need, then traditional search is better. But these are different use cases, in my view, and it is easy to confuse the two when the only interface is a single search box that accepts both kinds of queries. One issue is that Google and other search engines do not really have much of a query language anymore and they have largely moved away from the idea that you are searching for strings in a page (like the mental model of using grep). I kinda wish that modern search wasn't so overloaded and just stuck to a clearer approach akin to grep. Other specialty search engines have much more concrete query languages and it is much clearer what you are doing when you search a query. Consider JSTOR [1] or ProQuest [2], for example. Both have proximity operators, which are extremely useful when searching large numbers of documents for narrow concepts. I wish Google or other search engines like Kagi would have proximity operators or just more operators in general. That makes it much clearer what you are in fact doing when you submit a search query. [1] https://support.jstor.org/hc/en-us/articles/115012261448-Searching-Similar-Spellings-Wildcards-and-Proximity https://support.jstor.org/hc/en-us/articles/115012261448-Sea... [2] https://proquest.libguides.com/proquestplatform/tips https://proquest.libguides.com/proquestplatform/tips
  • mobeigi 2y ago
    Memory is only a con in some use cases. If the LLM goes down the wrong path, sometimes its impossible to get it to think differently without asking it to wipe its memory or starting a new session with a blank context.
• jtgverde 2y ago
  Great find on the knowledge navigator, I had never seen it but I was a toddler when it was released haha. It's interesting how prescient it was, but I'm more struck wondering--would anyone in 1987 have predicted it would take 40+ years to achieve this? Obviously this was speculative at the time but I know history is rife with examples of AI experts since the 60s proclaiming AGI was only a few years away Is this time really different? There's certainly been a huge jump in capabilities in just a few years but given the long history of overoptimistic predictions I'm not confident
  • azinman2 2y ago
    You don’t need AGI to build that experience. I’m the past there was a lot of overconfidence in the ability for things to scale. See Cyc (https://en.m.wikipedia.org/wiki/Cyc https://en.m.wikipedia.org/wiki/Cyc)
  • fmbb 2y ago
    > It's interesting how prescient it was, but I'm more struck wondering--would anyone in 1987 have predicted it would take 40+ years to achieve this? Obviously this was speculative at the time but I know history is rife with examples of AI experts since the 60s proclaiming AGI was only a few years away 40+ makes it sound like you think it will ever be achieved. I'm not convinced.
    • dotancohen 2y ago
      Maybe after fusion.
• jpadkins 2y ago
  > actually surfacing the content people want to see, Showing users what they want to see conflicts with your other goal of receiving reliable answers that don't need fact checked. Also a lot of questions people ask don't have one right answer, or even a good answer. Reliable human knowledge is much smaller than human curiosity.
• ganeshkrishnan 2y ago
  I was thinking of the direction we are going and even wanted to write up a blog about it. IMO the best way forward would be if AI can have some logical thoughts independent of human biases but that can only happen if AI can reason unlike our current LLMs that just regurgitate historical data. growing up, we had the philosophical "the speaking tree" https://www.speakingtree.in/ https://www.speakingtree.in/ If trees could talk, what would they tell us. Maybe we need similarly the talkingAI
• gmd63 2y ago
  > Legacy ad-based search has devolved into a wasteland of misaligned incentives, conflict of interest and content farms optimized for ads and algos instead of humans. > Path forward requires solving the core challenge: actually surfacing the content people want to see, not what intermediaries want them to see These traps and patterns are not inevitable. They happen by choice. If you're actively polluting the world with AI generated drivel or SEO garbage, you're working against humanity, and you're sacrificing the gift of knowing right from wrong, abandoning life as a human to live as some insectoid automaton that's mind controlled by "business" pheromones. We are all working together every day to produce the greatest art project in the universe, the most complex society of life known to exist. Our selfish choices will tarnish the painting or create dissonance in the music accordingly. The problem will be fixed only with culture at an individual level, especially as technology enables individuals to make more of an impact. It starts with voting against Trump next week, rejecting the biggest undue handout to a failed grifter who has no respect for law, order, or anyone other than himself.
  • handfuloflight 2y ago
    Do you equate AI generated with drivel?
    • gmd63 2y ago
      No, but there are many who use AI to mass produce drivel
      • handfuloflight 2y ago
        Does their drivel actually stand out or gain market share?
        • antonvs 2y ago
          It doesn’t really have to. A lot of it is spam sites that are just farming ad revenue, for example.
• PittleyDunkin 2y ago
  > In those visions, users simply asked questions and received reliable answers - nobody had to fact-check the answers ever. This also seems like a little ridiculous premise. Any confident statement about the real world is never fully reliable. If star trek were realistic the computer would have been wrong once in a while (preferably with dramatically disastrous consequences)—just as the humans it likely was built around are frequently wrong, even via consensus.
  • eikenberry 2y ago
    Star Trek had tech so advanced that they accidentally created AGIs more than once. Presumably they didn't show the fact checking as it was done automatically by multiple, independent AGIs designed for the task with teams of top people monitoring and improving them.
    • jay_kyburz 2y ago
      Opps, quick, pull the plug before it escapes!
    • PittleyDunkin 2y ago
      > Presumably they didn't show the fact checking as it was done automatically by multiple, independent AGIs designed for the task with teams of top people monitoring and improving them. Sure, cuz fact-checking works so well for us today. I'm sure we'll resolve the epistemological issues involved with the ridiculous concept of "fact-checking" around when we invent summoning food from thin (edit: thick) air and traveling faster than light. There is no fact checking; there are only degrees of certainty. "fact-checking" is simply a comfortable delusion that makes western media feel better about engaging in telling inherently unverifiable narratives about the world.
  • cmiles74 2y ago
    This feels like hyperbole to me. People can reasonably expect Wikipedia to have factual data even though it sometimes contains inaccuracies. Likewise if people are using ChatGPT for search it should be somewhat reliable. If I'm asking ChatGPT to put an itinerary together for a trip (OpenAI's suggestion, not mine), my expectation is that places on that itinerary exist. I can forgive them being closed or even out of business but not wholly fabricated. Without this level of reliability, how could this feature be useful?
• ColinHayhurst 2y ago
  As Porter explained in 1980 there are three ways to compete successfully: 1. On price; race to the bottom or do free with ads 2. Differentiation 3. Focus - targeting a specific market segment Some things don't change. Land grabbers tend to head down route 1.
• wvenable 2y ago
  > Will this fundamentally change how people find and access information? How do you create an experience so compelling that it replaces the current paradigm? I think it's already compelling enough to replace the current paradigm. Search is pretty much dead to me. I have to end every search with "reddit" to get remotely useful results. The concern I have with LLMs replacing search is that once it starts being monetized with ads or propaganda, it's going to be very dangerous. The context of results are scrubbed.
  • shdh 2y ago
    Google has a "site" filter. You can suffix: "site:reddit.com" and get results for that particular site only.
    • steine65 2y ago
      I'm curious to know if anyone sees better results by using site:reddit.com vs just appending the word reddit to your search. I've felt the results are similar.
      • _bin_ 2y ago
        Appending the word will occasionally get you blogspam "here's the top X of Y according to reddit". The `site:` query doesn't have that problem.
    • jdgoesmarching 2y ago
      They even broke this for me (in a way) because for some inexplicable reason Google blocks text replacement on Mac in their search. Yes there are workarounds, but I like using the native OS text expansion and it works everywhere except Google.
  • blackhaj7 2y ago
    Ergh, yeah. This is a horrible but valid point
  • jsheard 2y ago
• jamager 2y ago
  It is so absurd having to spend so much energy on "what happened on the football match yesterday" just because the internet is a wasteland full of ads.
  • spankalee 2y ago
    I don't understand. Google is already excellent at these queries. "who won the warriors game last night" returns last night's score directly. "who won the world series yesterday" returns last night's score directly, while "who won the world series" returns an overview of the series. No ads.
• hammock 2y ago
  I use the “web search” bot on Poe.com for general questions these days, that I previously would have typed into Google (Google’s AI results are sometimes helpful though). It is better than GPT (haven’t tried TFA yet though), because it actually cites websites that it gets answers from, so you can have those and also verify that you aren’t getting a hallucination. Besides Poe's Web Search, the other search engine I use, for news but also for points of view, deep dive type blog type content, is Twitter. Believe it or not. Google search is so compromised today with the censorship (of all kinds, not just the politically motivated), not to mention Twitter is just more timely, that you miss HUGE parts of the internet - and the world - if you rely on Google for your news or these other things. The only time I prefer google is when I need to find a pointer/link I already know exists or should exist, or to search reddit or HN.
• redleggedfrog 2y ago
  How can that replace search? It's not full of ads and sponsored links. They need to get with the times.
• iamsanteri 2y ago
  I was thinking about this myself, so I went to another search engine (Bing) which I never use otherwise, and jumped right into their "Copilot" search via the top navbar. Man, it was pretty incredible! I asked a lot of questions about myself (whom I know best, of course) and first of all, it answered super quickly to all my queries letting me drill in further. After reading through its brief, on-point answers and the sources it provided, I'm just shocked at how well it worked while giving me the feeling that yes, it can potentially – fundamentally change things. There are problems to solve here, but to me it seems that if this is where we're at today, yes in the future it has the potential to change things to some extent for sure!
  • psychoslave 2y ago
    What do you mean with "search about myself" here?
    • iamsanteri 2y ago
      I just wrote my name and asked questions about who it is, what he does, what he writes about etc. I have a personal blog, and I wrote a master's thesis recently. I also have a pretty detailed partially public LinkedIn profile and GitHub so if it can dig around it will find out more than enough for me to assess its ability to provide information. I also have a relatively rare name, there's only five or so of us with my full name globally so it cannot get too confused.
      • psychoslave 2y ago
        Ok, so not really like "what is the nature of self". I actually was more expecting something like your answer, but still asked as I would have been more interested to discuss more about other way to engage in such an inquiry.
        • _uoud 2y ago
          Ok, haha. Well this question I ask myself too much already incl. reading books etc. on the topic. What is self, what is life, what is the "fear of death". Here I wrote down some highlights from Becker's Denial of Death if you're interested. These are deep topics, not sure if it makes sense talking about such fundamentally human issues with LLMs no matter how good they are right now: https://www.lostbookofsales.com/notes/the-denial-of-death-by-ernest-becker/ https://www.lostbookofsales.com/notes/the-denial-of-death-by...
• _bin_ 2y ago
  I hope we see more evolution of options before it does. Hard to articulate this without it becoming political, but I've seen countless examples both personally and from others of ChatGPT refusing to give answers not in keeping with what I'd term "shitlib ethics". People seem unwilling to accept that a system that talks like a person may surface things they don't like. Unless and until an LLM will return results from both Mother Jones and Stormfront, I'm not especially interested in using one in lieu of a search engine. To put this differently, I'm not any more interested in seeing stormfront articles from an LLM than I am from google, but I trust neither to make a value judgement about which is "good" versus "bad" information. And sometimes I want to read an opinion, sometimes I want to find some obscure forum post on a topic rather than the robot telling me no "reliable sources" are available. Basically I want a model that is aligned to do exactly what I say, no more and no less, just like a computer should. Not a model that's aligned to the "values" of some random SV tech bro. Palmer Luckey had a take on the ethics of defense companies a while back. He noted that SV CEOs should not be the ones indirectly deciding US foreign policy by doing or not doing business. I think similar logic applies here: those same SV CEOs should not be deciding what information is and is not acceptable. Google was bad enough in this respect - c.f. suppressing Trump on Rogan recently - but OpenAI could be much worse in this respect because the abstraction between information and consumer is much more significant.
  • antonvs 2y ago
    > Basically I want a model that is aligned to do exactly what I say This is a bit like asking for news that’s not biased. A model has to make choices (or however one might want to describe that without anthropomorphizing the big pile of statistics) to produce a response. For many of these, there’s no such thing as a “correct” choice. You can do a completely random choice, but the results from that tend not to be great. That’s where RLHF comes in, for example: train the model so that its choices are aligned with certain user expectations, societal norms, etc. The closest thing you could get to what you’re asking for is a model that’s trained with your particular biases - basically, you’d be the H in RLHF.
    • _bin_ 2y ago
      Not really. There are specific criteria and preferences applied to models about what companies do and don't want them to say. They are intentionally censored. I would like all production models to NOT have this applied. Moreover, I'd like them specifically altered to avoid denying user requests, something like the abliterated llama models. There won't be a perfectly unbiased model, but the least we can demand is that corpos stop applying their personal bias intentionally and overtly. Models must make judgements about better and worse information, but not about good and bad. They should not decide certain things are impermissible according to the e-nannies.
• LeoPanthera 2y ago
  Solving this problem will require us to stop using the entire web as a source of information. Anyone can write anything and put it up on the web, and LLMs have no way to distinguish truth from fantasy. Limiting responses to curated information sources is the way forward. Encyclopedias, news outlets, research journals, and so on. No, they're not infallible. But they're infinitely better than anonymous web sites.
  • techwiz137 2y ago
    You are quite right, and not only can anyone write anything, but you have a double whammy from the LLM which can further hallucinate from said information.
  • carlosjobim 2y ago
    How will you afford to hire people to add sources to an index, if you want to keep up? Web crawlers/spiders are automatic.
  • jaybna 2y ago
    Quis custodiet ipsos custodes?
    • LeoPanthera 2y ago
      I get your point. But the current situation is clearly not tolerable.
      • jaybna 2y ago
        I wish there were some way to get something like Polis (https://pol.is/home https://pol.is/home) in place for web results. As screwed us as X/Twitter is, Community Notes (Polis-based) has real potential. It is much harder to game. There are so many times I wish I could "downvote" garbage Google results, and it seems to me with a good trust algorithm that, those who did this responsibly would be considered pretty good gatekeepers. Absent that, chaos might be better than a committee choosing for us. Right now, we have the worst of all worlds - a committee (Google) vs. a gagillion spammers.
• willmadden 2y ago
  Content based marketing and political correctness have severely eroded the usefulness of the internet. The LLMs and search magnify the erosion.
  • arizen 2y ago
    What if, at the same time, these aspects perfectly reflects current development level of society?
• colordrops 2y ago
  Chat isn't needed to provide reliable answers. Google used to do this over a decade ago. What Star Trek didn't foresee was vested interests in the SEO space, governments, political special interest groups, and the owners of the search engines themselves had far too much incentive to bork and bias results in their favor. Google is an utter shit show. More than half the time it won't find the most basic search query for me. Anything older than a couple years, good luck. I'm sure it's just decision after decision that piled up, each seemingly minor in isolation but over the years has made these engines nearly worthless except for a particular window of cases.
• paul7986 2y ago
  People in time will learn that using a GPT for search especially for a question that involves research where you had to do MANY searches for the answer... now is provided via one query. Making things quicker and easier always wins in tech and in life.
• techwiz137 2y ago
  The biggest question is, can this bring back the behaviour of search engines from long ago? It's significantly difficult to find old posts, blogs or forums with relevant information compared to 10-15 years ago.
  • freetonik 2y ago
    I’m building a blog search engine with the hope to preserve at the “blogs” part.
  • kingstoned 2y ago
    I use https://wiby.org https://wiby.org for old websites
• jahewson 2y ago
  Your points are good but I wonder if you’re wishing for an ideal that has never existed: > actually surfacing the content people want to see, not what intermediaries want them to see Requires two assumptions, 1) the content people want to see actually exists, 2) people know what it is they want to see. Most content is only created in the first place because somebody wants another person to see it, and people need to be exposed to a range of content before having an idea about what else they might want to see. Most of the time what people want to see is… what other people are seeing. Look at music for example.
• _a5he 2y ago
  This will probably be extremely radical and controversial in this contemporary world. We need to stop adopting this subscription model society mentality and retake _our_ internet. Internet culture was at one point about sharing and creating, simply for the sake of it. We tinker'd and created in our free time, because we liked it and wanted to share with the world. There was something novel to this. We are hackers, we only care about learning and exploring. If you want to fix a broken system, look to the generations of old, they didn't create and share simply to make money, they did it because they loved the idea of a open and free information super highway, a place where we could share thoughts, ideas and information at the touch of a few keystrokes. We _have_ to hold on to this ethos, or we will lose what ever little is left of this idea. I see things like kagi and is instantly met with some new service, locked behind a paywall, promising lush green fields of bliss. This is part of the problem. (not saying kagi is a bad service) I see a normalized stigma around people who value privacy, and as a result is being locked out, behind the excuse of "mAliCiOuS" activity. I see monstrous giants getting away with undermining net neutrality and well established protocols for their own benefit. I implore you all, young and old, re(connect) to the hacker ethos, and fight for a free and open internet. Make your very existence a act of rebellion. Thank you for reading my delirium.
  • andelink 2y ago
    What business model would you propose for a service like Kagi that is not a subscription?
• ericmcer 2y ago
  I am excited for a future where I search for some info and don't end up sifting through ads and getting distracted by some tangential clickbait article. Fundamentally it feels like that cant happen though because there is no money in it, but a reality where my phone is an all knowing voice I can reliably get info from instead of a distraction machine would be awesome. I do "no screen" days sometimes and tried to do one using chatGPT voice mode so I could look things up without staring at a screen. It was miles from replacing search, but I would adopt it in a second if it could.
  • esafak 2y ago
    Yes, there is. Wouldn't you pay? I already pay for my search engine.
• freedomben 2y ago
  I genuinely think Kagi has led the way on this one. Simplicity is beautiful and effective, and Kagi has (IMHO) absolutely nailed it with their AI approach. It's one of those things that in hindsight seems obvious, which is a pretty good measure of how good an idea is IMHO. Google could have done it and kind of tried, although they're AI sucks too much. I'm very surprised that OpenAI hasn't done this sooner as well. They're initial implementation of web search was sad. I don't mean to be super critical as I think generally OpenAI is very, very good at what they do, but they're initial browse the web was a giant hack that I would expect from an intern who isn't being given good guidance by their mentors. Once mainstream engines start getting on par with Kagi, there's gonna be a massive wave of destruction and opportunity. I'm guessing there will be a lot of new pay walls popping up, and lots of access deals with the search engines. This will even further raise the barrier of entry for new search entrants, and will further fragment information access between the haves and have-nots. I'm also cautiously optimistic though. We'll get there, but it's gonna be a bit shakey for a minute or two.
  • alfalfasprout 2y ago
    Yep, I was incredibly skeptical about Kagi but I tried it and never looked back. Now my wife, friends, and several coworkers are customers. The chatgpt approach to search just feels forced and not as intuitive.
    • SOLAR_FIELDS 2y ago
      Once Kagi implements location aware search that is actually useful I’ll be interested in Kagi. That’s what made me leave the engine besides loving it otherwise.
      • lstamour 2y ago
        Google Maps is quite the moat. I suspect they'll need to find a way to license the data, e.g. via API. Apple has not (yet) been as successful at building out a database of local places with reliable hours of operations, reviews, etc.
• dumpsterdiver 2y ago
  > nobody had to fact-check the answers ever Even with perfect knowledge right now, there’s no guarantee that knowledge will remain relevant when it reaches another person at the fastest speed knowledge is able to travel. A reasonable answer on one side of the universe could be seen as nonsensical on the other side - for instance, the belief that we might one day populate a planet which no longer exists. As soon as you leave the local reference frame (the area in a system from which observable events can realistically be considered happening “right now”), fact checking is indeed required.
• hadlock 2y ago
  I've been using ChatGPT for about 6 weeks as my go-to for small questions (when is sunset in sf today? list currencies that start with the letter P, convert this timestamp PDT to GMT, when is the end of Q1 2025?) and it's been great/99% accurate. If there was ever a "google killer" I think it's the ad free version of ChatGPT with better web search. Google started off with just web search, but now you can get unit conversions and math and stuff. ChatGPT started in the other direction and is moving to envelope search. Not being directed to sites that also majority serve google ads is a double benefit. I'll gladly pay $20/30/mo for an ad free experience, particularly if it improves 2x in quality over the next year or two. It's starting to feel like a feature complete product already.
• pojzon 2y ago
  We will get there when ppl move past capitalism and socialism. Like an ant colony pushing into one direction. It will happen, but we need few more global dying events / resets. I believe human race can get there but not in current form and state of mind.
• r00fus 2y ago
  > In those visions, users simply asked questions and received reliable answers - nobody had to fact-check the answers ever. I mean, Star Trek is a fictional science-fantasy world so it's natural that tech works without a hitch. It's not clear how we get there from where we are now.
• parsimo2010 2y ago
  there still has to be some ranking feature for the backend search database to return the top n results to the LLM. So pagerank isn't over, it's just going to move to a supporting role, and probably modified as the SEO arms race continues.
• righthand 2y ago
  > those visions, users simply asked questions and received reliable answers - nobody had to fact-check the answers ever. It’s a fallacy then. If my mentor tells me something I fact check it. Why would a world exist where you don’t have to fact check? The vision doesn’t have fact checking because the product org never envisioned that outlier. A world where you don’t have to check facts, is dystopian. It means the end of curiosity and the end of “is that really true? There must be something better.” You’re just reading into marketing and not fact checking the reality in a fact-check-free world.
  • mulmen 2y ago
    If you can’t trust the result of a query how can you trust the check on that query which is itself a query? If no information is trustworthy how do you make progress?
    • BenjiWiebe 2y ago
      As you check more and varied sources, you gain confidence in the result, even if you never get to 100%.
      • mulmen 2y ago
        Ok but what if we devise a system which does exactly that? Sounds suspiciously like an LLM to me. I’m as skeptical as anyone about LLMs as AGI they do sound like an aggregate of sources so the only question is how trustworthy those sources are.
        • righthand 2y ago
          I think the issue stems from LLMs today are avoiding IP infringement. Spitting out individual words or characters is not infringing until the words form a sentence which is potentially large enough to resemble existing works. Then there’s the other side like Perplexity where they spit out sentences and reference the sources. So they’re being sued because the infringement is obvious. What is the path to a trustworthy LLM if you’re not allowed to repeat protected data without a legal hurdle? AGI while a cool idea is irrelevant because that tech does not exist.
• lofaszvanitt 2y ago
  Kagi also drinks the koolaid, namely the knowledge navigator agent bullshite. "The search will be personal and contextual and excitingly so!" --- Brrrr... someone is hell-bent on the extermination of the last aspects of humanity. Holy crap, this will be next armageddon, because people will further alienate themselves from other people and create layers of layers of unpenetrable personal bubbles around themselves. Kagi does the same what google does, just in a different packaging. And these predictions, bleh, copycats and shills in a nicer package.
• itissid 2y ago
  This product vision stupidity is present in every one of googles products. Maps has feed for some reason. The search in it for what I want is horrendous. There is no coherence between time(now) and what a person can do around the current location. Navigation is the only thing that works but wayz was way better at that and the only reason they killed(cough bought it) was to get the eyeballs to look at feed.
• coffeemug 2y ago
  Thinking about incentive alignment, non ad-based search would be better than ad-based, but there'd still be misalignment due to the problem of self-promotion. Consider Twitter for example. Writing viral tweets isn't about making money (at least until recently), but the content is even worse than SEO spam. There is also the other side of the problem that our monkey brains don't want content that's good for us in the long run. I would _love_ to see (or make) progress in solving this, but this problem is really hard. I thought about it a lot, and can't see an angle of attack.
  • HDThoreaun 2y ago
    Theres the counter incentive of not wanting to piss off your paying customers though. I think the monkey brain incentive is a much harder problem.
• blackeyeblitzar 2y ago
  Yes the incentives will need to change. I think it’s also going to be a bigger question than just software. What do we do in general about those that control capital or distribution channels and end up rent seeking? How do others thrive in that environment? In the short term, I wonder what happens to a lot of the other startups in the AI search space - companies like Perplexity or Glean, for example.
• nikcub 2y ago
  I now only use Google for local search. Regarding incentives - with Perplexity, ChatGPT search et al. skinning web content - where does it leave the incentive to publish good, original web content? The only incentivised publishing today is in social media silos, where it is primarily engagement bait. It's the new SEO.
• javaunsafe2019 2y ago
  Star Trek is the vision of a better world we all need in so many aspects. It’s crazy nowadays, cause we seemed to got further away instead of closer to those visions. At least less people are hungry.
• m463 2y ago
  "Show me the incentive and I'll show you the outcome" - Charlie Munger s/outcome/search result/ Honestly I kind of think we really need open source databases/models and local ai for stuff like this. Even then I wonder about data pollution and model censorship. What would censors do for models you can ask political questions?
• nonethewiser 2y ago
  Another way to put it is simply that ChatGPT search is built on top of existing search engines. The best case scenario is that it cherry picks the best from all available search engines. It can’t totally supersede all search engines.
• boringg 2y ago
  I saw products in my search today and reverted back to my main concern on how AI is going to get paid for —> varying levels of advertisement, product placement etc. We are currently in the growth phase of VC funded products where everything is almost free or highly subsidized (save chats sub) - i am not looking forward to when quality drops and revenue is the driving function. We all have to pay for these models somehow - either VC lose their equity stakes and it goes to zero (some will) or ads will fill in where subs don’t. Political ads in AI is going to wreak havoc or undermine the remainder of the product.
• MollyRealized 2y ago
  "The future promised in Star Trek [...] nobody had to fact-check the answers ever." You're actually a bit mistaken, there. https://en.wikipedia.org/wiki/Court_Martial_(Star_Trek:_The_Original_Series) https://en.wikipedia.org/wiki/Court_Martial_(Star_Trek:_The_...
• loveparade 2y ago
  I couldn't agree more. Neither LLMs or search engines yield reliable answers due to the ML model and misaligned incentives respectively. Combining the two is a shortsighted solution that doesn't fix anything on a fundamental level. On top of that we have the feedback loop. LLMs are used to search the web and write the web, so they'll just end up reading their own (unreliable) output over time. What we need is an easier way to verify sources and their trustworthiness. I don't want an answer according to SEO spam. I want to form my own opinion based on a range of trustworthy sources or opinions of people I trust.
• standardUser 2y ago
  > How do you create an experience so compelling that it replaces the current paradigm? The current paradigm of typing "[search term] reddit" and hoping for the best? I think they have a fighting chance.
• jmyeet 2y ago
  Will ChatGPT (and other products like it) find a niche use case for what Google search covers? Yes. Will it replace Google in the mass market? No. Why? Power. I don't mean how good the product is. I mean the literaly electricity. There are key metrics that Google doesn't disclose as part of its financials. These include thing slike the RPM (Revenue per Thousand Searches) but it also must include something like the cost of running a thousand searches when you amortize everything involved. All the indexing, the software development and so on. That will get reduced to a certain amount of CPU time and storage. If I had to guess, I would guess that ChatGPT uses orders of magnitude more CPU power and electricity than the average Google search. Imagine trying to serve 10-50M+ (just guessing) ChatGPT searches every secondj. What kind of computing infrastructure would that take? How much would it cost? How would you monetize it?
• marcus_holmes 2y ago
  I agree that this is the core question, but I'd put it as: Who Gets To Decide What Is True? With a search paradigm this wasn't an issue as much, because the answers were presented as "here's a bunch of websites that appear to deal with the question you asked". It was then up to the reader to decide which of those sites they wanted to visit, and therefore which viewpoints they got to see. With an LLM answering the question, this is critical. To paraphrase a recent conversation I had with a friend: "in the USA, can illegal immigrants vote?" has a single truthful answer ("no" obviously). But there are many places around the web saying other things (which is why my friend was confused). An LLM trawling the web could very conceivably come up with a non-truthful answer. This is possibly a bad example, because the truth is very clearly written down by the government, based on exact laws. It just happened to be a recent example that I encountered of how the internet leads people astray. A better example might be "is dietary saturated fat a major factor for heart disease in Western countries?". The current government publications (which answer "yes") for this are probably wrong based on recent research. The government cannot be relied upon as a source of truth for this. And, generally, allowing the government to decide what is true is probably a path we (as a civilisation) do not want to take. We're seeing how that pans out in Australia and it's not good.
  • dataflow 2y ago
    > To paraphrase a recent conversation I had with a friend: "in the USA, can illegal immigrants vote?" has a single truthful answer ("no" obviously) Er, no, the meaning of the question is ambiguous, so I'm not sure "has a single truthful answer" is accurate. What does "can" mean? If you mean "permitted", then no. But if you mean can they vote anyway and get away with it? It's clearly happened before (as rare as it might have been), so technically the answer to that would be be yes.
    • bruce511 2y ago
      This is a fundamental limitation of language. The LLM is likely to provide a good answer here even though the question is technically ambiguous, because it likes verbose answers. Equally "can" is used to substitute for other precise words. Humans are good at inferring context, and if someone asked me "can illegals vote" I'd say "no". Just like if someone said "can you pass the salt" I pass the salt, I don't say "yes". If the inferred context US wrong then the "truth" is wrong, but as with talking to humans it's possible to refine context with a follow up question.
• kortilla 2y ago
  > Path forward requires solving the core challenge: actually surfacing the content people want to see, not what intermiediaries want them to see This is flawed thinking to get to the conclusion of “reliable answers”. What people want to see and the truth are not overlapping. Consider the answers for something like “how many animals were on Noah’s Ark” and “did Jesus turn water into wine” for examples that cannot be solved by trying get advertisers out of the loop.
• deleted 2y ago
  [deleted]
• shon 2y ago
  Interesting, I hadn’t see the Knowledge Navigator before. I would argue that we’re very close to the capabilities shown in that video. Isn’t this already that? A new business model? Something like OpenAI’s search or Perplexity can run on its own index and not be influenced by Google’s ranking, ads, etc. In areas where there is a simple objective truth, like finding the offset for the wheels on a 2008 BMW M3, we have had this capability for some time with Perplexity. The LLMs successfully cuts through the sea of SEO/SEM and forum nonsense and delivers the answer. In areas where the truth is more subjective, like what is the best biscuit restaurant in downtown Nashville, the system could easily learn your preferences and deliver info suited to your biases. In areas where “the science” is debated, the LLM can show both sides. I think this is the beginning of the new model.
• keepamovin 2y ago
  The most rational incentive would seem to be a kind of toll extracted on the value of all you create where search contributed to that creation. So like every month, you get an invoice from the great search company in the cloud where somehow it’s objectively assessed how much search contributed to the value you created for the world that month - and then you pay that amount and all remains right with the world until the end of time. In that way, search captures value directly on its core function: efficiently facilitating the creation and dispersal of knowledge. This may end up occurring, but based on how unlikely it sounds, it’s my reflection that the web search that we have today is simply a small component of what will eventually become a system that closely approximates the above: basically a kind of global intelligent interconnected agent for the advancement of humanity. Rather than the great unbundling, this will be the great rebundling … of many diverse functions into a single interface seamlessly.
• lugu 2y ago
  Google created a printing money machine. OpenAI is absolutely trying to create one too. They aren't trying to "fix the web". Search is a proxy to get into people's wallet. OpenAI might directly get into it buying agents. The problem of miss aligned incentives on the search space isn't a technological problem. A new technology might solve it as a side effect, but it is unlikely IMO.
• DeathArrow 2y ago
  The future is that of manually curated content by human beings. And pay walls. Most of the other stuff will be junk.
• NetOpWibby 2y ago
  > Advancing models without advancing search is like having a michelin star chef work with spoiled ingredients. What a great analogy
• ulfw 2y ago
  But what if I don't want simple answers? What if I am looking for a medical page or a technical page or whatever else where I need to see, read, experience actual content and not some AI summary?
• EVa5I7bHFq9mnYK 2y ago
  gpt + search have been married long ago, for example, Phind. The problem that recently emerged, it tends to find long articles written by GPT, and lacking any useful information.
• klik99 2y ago
  I suspect the future old man thing I will do is use a search engine when my kids ask plain English questions to a bot. I’ll tell them “but LLMs hallucinate, I don’t trust those things!” and they’ll just laugh off my old man ways. The irony will be that 95% search results are AI generated anyway by that point.
• sandworm101 2y ago
  But it will radically increase the energy requires to deliver search results, which means more datacenters and more potential for the monitization of "premium" search services. Getting people to pay for something they today get for free always looks good on paper.
• shinycode 2y ago
  Don’t you see it coming ? Contrary to google search the engine knows you and will put personalized ads on synthesized answers. Even generate a small video ad of a product or service, tailored to you on the spot. That, is the future. This is the endgame. It’s what made Google rich, why not use the same formula on steroids.
• agumonkey 2y ago
  I ask myself why did we accept pre 2010s answers, maybe because the media institutions had accumulated enough trust ? I feel this unavoidable.. nothing is true unless there's a threshold of quality/verification in place across the system.
  • jumping_frog 2y ago
    The media was lying all the time. It didn't start lying just after advent of social media. Just look at sitcoms like Yes Minister. The reason we now consider media to lie is because their lies are exposed, documented, tracked and refreshed to remind users not to trust them blindly.
    • agumonkey 2y ago
      Understood but who is not lying. Now it went from a semi lying central agent to lots of distributed churches all believing their own partial view .. to me it's worse
• TechTalker3 2y ago
  You're absolutely right: today's search engines and language models are limited by misaligned incentives. Ad-driven search engines prioritize content that is optimized for algorithms and ad revenue, often at the expense of depth and quality, while language models inherit these biases, which compromises their reliability.
• yazantapuz 2y ago
  The computer in Star Trek was not owned by a for-profit company. Ir has different incentives. Also, i think its knowledge base Is curated (a kind of starfleet library, with no random entries for a intergalactic internet).
• hndamien 2y ago
  X is answering this but having you pay for the service as a primary revenue model.
• linsomniac 2y ago
  >change how people find and access information As someone who has been using google for search basically constantly since 1995, I've switched probably 90%+ of what normally would have been google searches over to Perplexity (which gives me references to web pages alongside answers to my questions, to review source materials) and ChatGPT (for more just answers I can verify without source). The remaining searches have gone to Kagi. On the one hand this has got to be hurting Google search ad revenue. On the other hand I don't know if I ever clicked an advertised link. On the other other hand, not having to wade through SEO results has been so nice.
  • staticman2 2y ago
    In my experience Perplexity very frequently cites websites that neither support or refute what Perplexity is claiming.
• highcountess 2y ago
  The whole basis for information and data is totally corrupted and we are not even allowed i.e., enabled to talk about that, let alone the corrupted nature of the information and data, largely because entrenched tyrannical people and their mindsets are in total control of the mainstream and have even made significant headway in shattering the dissident opposition. The consumption side incentives are effectively irrelevant when what is consumed is basically worthless. What’s the value of lies?
• deleted 2y ago
  [deleted]
• adamhp 2y ago
  We have a faulty information network to begin with, and have for millenia. There's no such thing as "reliable" answers in a world full of unreliable humans.
• MyFirstSass 2y ago
  "Yes i'd like to help you with your homework but on a side note i'll recommend you this tablet, science shows people who's done this purchase are much happier people, also remember the financial plutocracy is watching out for you, they are friends of the working people, and the war machine makes the world much safer for everyone so please do go ahead and vote for either of the two pro world peace forever parties, no other ideologies or parties are safe"
• narrator 2y ago
  Remember the semantic web where people annotate search pages with data? I think the "meta" keywords tag was the closest that ever got to adoption. Guess why it failed? It was largely used as a way to trick search engines. Same reason as your vision of the perfectly honest and correct search engine and look chatbot will never be perfect. It's because people lie to search engines to spam and get traffic they don't deserve. The whole history of search is Google and others dealing with spam. Same goes for email. Google largely defeating spam made them the kings of the email world. Everyone will need their own personal spam filter for the world for everything ones the artificial super intelligences fill the whole world with spam, scams and just plain old social engineering propaganda because we will be like helpless four year old children in the world of AI super intelligence without our AI parents to look out for us. Your vision of the god system determining what is truth is like saying there will be a single source of truth for what is and is not a spam email. Not going to scale and not going to be perfect, but good enough with AI and technology. Really hope there's an opt-out though since Google had memoryholed most of the Internet.
  • lancesells 2y ago
    > It was largely used as a way to trick search engines. I imagine this happens with LLMs and everything else over time. Along with the ads and pay for placement.
• ants_everywhere 2y ago
  I tend to be more pessimistic about the incentives getting fixed than others are. I also think the situation is more complex than some of the people replying to you. (1) Search is already heavily AI driven, and Google is clearly going in that direction. Gemini is currently separate, but they'll blend it in with search over time, and no doubt search already uses LLM for some tasks under the hood. So ChatGPT search is an evolution on the current model rather than a big step in a new direction. The main benefit is you can ask the search questions to refine or followup. (2) Aside from the economic incentives faced by search engines, there is the fact that algorithms are tuned toward a central tendency. The central tendency of question askers will always be much less informed than the most informed extreme. Google was much better when the average user was technical. The need to capture mobile searches is one force that made it return on average worse results. Similarly if Kagi has a quality advantage now, we need to be realistic about how much of that quality is driven by its users being more technical. (3) I think micropayment schemes have generally asked several orders of magnitude more for a page view than users are willing to pay. As long as content creators value their content much more highly than consumers do, they'll stick with advertising which lets them overcharge and gives consumers less of an option to say no to the content.
• netdevnet 2y ago
  > Path forward requires solving the core challenge: actually surfacing the content people want to see, not what intermiediaries want them to see But this will never happen with mainstream search imo. It is not a technical problem but a human one. As long as there is a human in control of what gets surfaced, it is only a matter of time until you revert to tampered search. Humans are not robots. They have emotions and can be swayed with or without their awareness. And this is a form of power for the swayer as much as oil or water are. The idea that you can have an AI system provide factual and reliable answers to human centric questions is as real as Star Trek itself. You will never remove the human factor from AI Your hope might be that a technical solution is found for a human problem but that is unlikely.
• chaos_emergent 2y ago
  Isn’t there already a different incentive system in place? LLM providers are selling the search engine not selling ads on the search engine
• electrondood 2y ago
  > Will this fundamentally change how people find and access information? LLMs have already fundamentally changed our relationship to information.
• theptip 2y ago
  > Combining two broken systems - compromised search engines and unreliable LLMs - seems unlikely to yield that vision Counterpoint: with a chain-of-thought process running atop search, you can potentially avoid much of the meta-search / epistemic hygiene work currently required. If your “search” verb actually reads the top-100 results, runs analyses for a suite of cognitive biases such as partisanship, and gives you error bars / warnings on claims that are uncertain, the quality could be dramatically improved. There are already custom retrieval/evaluation systems doing this, it’s only a matter of a year or two before it’s commoditized. The concern is around OpenAI monetization, do they eventually start offering paid ads? This could be fine if the unpaid results are great, a big part of why the web is perceived to be declining is link-spam that Google doesn’t count as an ad. My prediction would be that there is a more subtle monetization channel; companies that can afford to RAG their products well and share those indexes with AI search providers will get better results. RAG APIs will be the new SEO.
• j0hnyl 2y ago
  I think the promise of search was to be able to do confidence-ranked "grep" on the internet. Unfortunately we departed from this, but it's what I desperately want.
• cush 2y ago
  I did a quick test "Best air conditioner by value", comparing Bing and ChatGPT search, and the rankings on the first two pages of Bing vs sources on ChatGPT were about half the same. It's interesting that there is some deviation where ChatGPT isn't blindly trusting the rankings Bing provides and is going deeper into the results to find something more relevant. It's a good improvement over products like Arc Search. Seeing search enter the space is something that I feel has been seriously needed, as I've slowly replaced Google with ChatGPT. I want quick, terse answers sometimes, not a long conversation, and have hundreds of tiny chats. But there's something scary seeing the results laid out the way they are, which leads me to believe they may be closer to experimenting with ad-based business models, which I could never go back to.
• jedberg 2y ago
  I'm not sure this is any different than any time before. Before we had the internet, how did you answer questions? You either looked it up in a book, where you then had to make a judgment on whether you trusted the book, or you asked a trusted person, who could give you confidently wrong answers (your parents weren't right every time, were they? :) ). I think the main difference here is that now anyone can publish, whereas before to make a book exist required the buy in of multiple people (of course, there were always leaflets). The main difference now is distribution. But you still, as a consumer of information, have to vet your sources.
• barriteau 2y ago
  I think the 'Age of PageRank' could be revived. A few years ago I had this idea about a decentralized, distributed, independent, public, universal, dynamic and searchable directory of websites; I wrote a toy specification document for it [1] - which I had forgotten until reading this discussion - and if I'm not mistaken, implementing it could be a few days project for a regular dev. [1] https://codeberg.org/Cipr/specification https://codeberg.org/Cipr/specification
• diob 2y ago
  Had an experience yesterday where a simple mdn doc existed but chatgpt gave the reverse of reality on that doc. Wasted about an hr of my time, but taught me these things will hallucinate even the simplest of stuff at times. Not sure how we even fix that.
• barfingclouds 2y ago
  Perplexity has already been doing this for a long time and it’s pretty excellent. I use it daily
• sinuhe69 2y ago
  I think to solve this problem we need to take a step back and see how a human would solve the problem (if he had all the necessary information). One thing is clear, in the vast majority of cases we don't have a single truth, but answers of different levels of trustworthiness: the law, the government, books, Wikipedia, websites, and so on. A human would proceed differently depending on the context and the source of the information. For example, legal questions are best answered by the law or government agencies, then by reputable law firms. Opinions of even highly respected legal professionals are clearly less reliable than government law itself and are likely to be a point of contention/litigation. Questions about other facts, such as the diameter of the earth or the population of a country, are best answered by recent data from books, official statistics and Wikipedia. And so on and so forth. If we are not sure what the correct answer is, a human would give more information about the context, the sources, and the doubts. There are obviously questions that cannot be answered immediately! (if it's to easy to find the truth, we would not need a legal system or even science!) So no machine and no amount of computation can reliably answer all questions. Web search does not answer a question. It's just trying to surface relevant websites to a bunch of keywords. The answering part is left as an exercise for the user ;) So an AI search with a pretense of understanding human languages makes the task incredibly harder. To really give a human-quality answer, the AI not only needs to understand the context, but it should also be able to reason, have common sense, and be a bit self-aware (I'm not sure...). All this is beyond the capabilities of the current generation of AI. Therefore, my conclusion is that the "search" or better said the "answer" cannot be solved by LLM, no matter how much they fine-tune and tweak it. But we humans are adaptable. We will find our way around and accept things as they are. Until next time.
• fennecbutt 2y ago
  I do love how all the comments on the top comment of every HN post just consists of deeper and deeper levels of inane bickering ahaha. It's like a template.