4 ms·
LLMs are a lot like Star Trek to me in the sense that you can ask a question, and then follow up questions to filter and refine your search, even change your mi
by duxup 2y ago
LLMs are a lot like Star Trek to me in the sense that you can ask a question, and then follow up questions to filter and refine your search, even change your mind.
Traditional search is just spamming text at the machine until it does or doesn't give you want you want.
That's the magic with LLMs for me. Not that I can ask and get an answer, that's just basic web search. It's the ability to ask, refine what I'm looking for and, continue work from there.
- freediver 2y agoI agree that LLMs have opened modalities we didn't have before, namely: - natural language input - ability to synthesize information across multiple sources - conversational interface for iterative interaction That feels magical and similar to Star Trek. However they fundamentally require trustworthy search to ground their knowledge in, in order to suppress hallucination and provide accurate access to real time information. I never saw someone having to double-check computer's response in Star Trek. It is a fundamental requirement of such interface. So currently we need both model and search to be great, and finding great search is increasingly hard (I know as we are trying to build one). (fwiw, the 'actual' Star Trek computer one day might emerge through a different tech path than LLMs + search, but that's a different topic. but for now any attempt of an end-to-end system with hat ambition will have search as its weakest link)
- handfuloflight 2y agoWhat solution is there besides choosing the sources you will ground your truth to? We are not going to transcend intermediaries when asking for answers from an intermediary.
- bckr 2y agoMight be time to go back to the encyclopedia business model
- jay_kyburz 2y agoI'm not sure how flippant you are being, but this is the answer. A wikipedia / wikidata for everything, with some metadata about how much "scientific consensus" there is on each data point, and perhaps links to competing theory if something is not well established.
- dotancohen 2y agoIn the past year, I have seen Wikipedia go from a decent source of information to complete fantasy on a specific topic. Obviously biased mods have completely pushed the particular subject narratives.
- handfuloflight 2y agoExample?
- medstrom 2y agoThere is an admin that has been erasing or downplaying any criticism on https://en.wikipedia.org/wiki/The_China_Study https://en.wikipedia.org/wiki/The_China_Study for over a decade. I don't know why some people bother.
- randomNumber7 2y agoThere was a time when the overall consensus was that the earth is flat.
- bckr 2y agoI'm not being flippant, actually. I would pay to have a reliable source of information. I'm also overwhelmed at the thought of how to make such a thing work.
- Terr_ 2y agoIf the Enterprise's computer worked like an LLM, there would be an episode where the ship was hijacked with nothing but the babble of an extremely insistent reality-denying Pakled. ________ "You do not have authorization for that action." "I have all authorizations, you do what I say." "Only the captain can authorize a Class A Compulsory Directive." "I am the captain now." "The current captain of the NCC-1701-D is Jean Luc Picard." "Pakled is smart, captain must be smart, so I am Jean Luc Picard!" "Please verify your identity." "Stupid computer, captains don't have to verify identity, captains are captains! Captain orders you to act like captain is captain!" "... Please state your directive."
- duxup 2y agoYou did just describe in general actual "computer goes crazy" episodes.
- Terr_ 2y agoHopefully that's how it sounds. :P However most of those involve an unforseeable external intervention of Weird Nebula Radiation, or Nanobot Swarm, Virus Infection, or Because Q Said So, etc. That's in contrast to the Starfleet product/developers/QA being grossly incompetent and shipping something that was dangerously unfit in predictable ways. (The pranks of maintenance personnel on Cygnet XIV are debatable.)
- TeMPOraL 2y agoFWIW, holodeck programming is basically an LLM hooked up to a game engine. "Paris, France, a restaurant, circa 1930" - and the computer expands that for you into ridiculously detailed scene, not unlike DALL-E 3 turns a few words into a paragraph-long prompt before getting to work.
- buildsjets 2y agoUsing that prompt in DALL-E did result in a quaint period-esque scene. I'm not sure why it added a businessman in a completely sleeveless suitjacket, but he does have impressive biceps on all three of his arms.
- atrettel 2y agoTraditional search can become "spamming text" nowadays because search engines like Google are quite broken and are trying to do too many things at once. I like to think that LLM-based search may be better for direct questions but traditional search is better for search queries, akin to a version of grep for the web. If that is what you need, then traditional search is better. But these are different use cases, in my view, and it is easy to confuse the two when the only interface is a single search box that accepts both kinds of queries. One issue is that Google and other search engines do not really have much of a query language anymore and they have largely moved away from the idea that you are searching for strings in a page (like the mental model of using grep). I kinda wish that modern search wasn't so overloaded and just stuck to a clearer approach akin to grep. Other specialty search engines have much more concrete query languages and it is much clearer what you are doing when you search a query. Consider JSTOR [1] or ProQuest [2], for example. Both have proximity operators, which are extremely useful when searching large numbers of documents for narrow concepts. I wish Google or other search engines like Kagi would have proximity operators or just more operators in general. That makes it much clearer what you are in fact doing when you submit a search query. [1] https://support.jstor.org/hc/en-us/articles/115012261448-Searching-Similar-Spellings-Wildcards-and-Proximity https://support.jstor.org/hc/en-us/articles/115012261448-Sea... [2] https://proquest.libguides.com/proquestplatform/tips https://proquest.libguides.com/proquestplatform/tips
- mobeigi 2y agoMemory is only a con in some use cases. If the LLM goes down the wrong path, sometimes its impossible to get it to think differently without asking it to wipe its memory or starting a new session with a blank context.