51 ms·
Teach your LLM to answer with facts, not fiction
- vouaobrasil 3y agoI believe that LLMs should be banned, but if they have to exist, we should teach them ethics first before anything else.
- 12907835202 3y agoWhose ethics? Should we tell it that sex is a positive thing or a horrible sin?
- akomtu 3y agoThose crusaders that prevail are the most ethical.
- balnaphone 3y agoWhat's your rationale for believing they should be banned? How can such algorithms be suppressed? And, is there any precedent for this, in particular that has worked?
- deleted 3y ago[deleted]
- mkl 3y agoDo you think your phone keyboard's predictive text should be taught ethics? How? LLMs are just predictive text scaled way up: they don't know or think anything, they just predict the next word repeatedly. They can't learn ethics, but can learn to string words together into sentences about ethics, again just by predicting the next word.
- vouaobrasil 3y agoI don't use predictive text on my phone.
- wizzwizz4 3y agoSituation: people try to use these predictive text chatbots as search engines. Problem: LLMs are not search engines. They extrapolate, interpolate, and approximate (so-called “hallucinations”) so they can always produce somewhat-plausible text completions. Solution: Create a search engine so good at returning relevant results that even an LLM can make use of it… then go to significant lengths to plug that search engine into the LLM, preventing people from reading the search results directly. Why not simply give people access to the search engine‽ People know how to use search engines!! This is the fifth time I've seen an article like this, and I'm still… baffled. It's https://xkcd.com/2021/ https://xkcd.com/2021/ all over again.
- omeze 3y agoYea… my opinion on this is that startups are attempting to force a market for chat bots, instead of accepting that LLM embeddings are best utilized as a search feature, not new product surface area
- woeirua 3y agoThey really want agents to work, and they just don’t yet.
- CSMastermind 3y agoIMO the most obvious place LLMs will be used is for interfaces. They enable voice-based interfaces to be practical for normal users for the first time since you really can talk to them in a convincing way. Translating user input into a set of well-defined commands seems like a better use than searching data to me.
- potatoman22 3y agoWell, search engines and my ability to query them aren't good enough for highly specific or poorly worded questions yet. LLMs are sometimes better in this space.
- jwells89 3y ago
- Lerc 3y agoIt is not a good start that they begin with a dictionary definition of Hallucinations. While the similarities to what a LLM does are apparent enough for the term to be used, LLMs are under no obligation to behave similar to the dictionary definition of Hallucinations. In general facts are not the answer to Hallucinations. You can't possibly have every fact for every situation. The true solution to Hallucinations is figuring out how to make a model say 'I don't know"
- ddmma 3y agoYou cannot fix what you cannot measure, here is an attempt to do just that with HallMeter https://why.network/ https://why.network/ Still have to figure a measurement unit.
- anon25783 3y agoHow about "falsehood quotient"? Count the number of counterfactual assertions in a given text, then divide by the number of sentences. Of course, the question of what is a falsehood is an exercise for the reader, but this would at least give a unit of measurement, flawed as it is.
- janalsncm 3y agoThe “exercise for the reader” in this case is the entire point of the metric. If it were possible to do at scale it could be incorporated into existing models right now. Second, it isn’t even necessarily better to have fewer lies if those few lies are more subtle. Plenty of propaganda works by twisting facts and using misleading statements. Perhaps the worst offenders won’t even have any outright falsehoods at all.
- ddmma 3y agothat is very interesting input, thanks! Also split into sentences, similar to what sentiment analytics does would raise the level of complexity into the output metric.
- 3y ago
- senectus1 3y agowould be nice if they could show a gradient score on results that show how certain it is of its answers... It should be fairly trivial for it to tell you how often its straight up lied about something.
- flanked-evergl 3y ago> It should be fairly trivial for it to tell you how often its straight up lied about something. By what mechanism would it "lie" to you? How is it even capable of lying in this sense of the word: https://en.wiktionary.org/wiki/lie#Verb_2 https://en.wiktionary.org/wiki/lie#Verb_2 > To give false information intentionally with intent to deceive. I guess it could "lie" in other senses, but calling that lying is not really adding clarity to the situation.
- msp26 3y agoIt isn't trivial. That's the whole problem. Luckily my use cases have a manual check built in but even a proxy for confidence would be amazing.
- SkyPuncher 3y agoIt's essentially impossible. Confidence is often a situational measure. "Is the sky blue?" * Generally, yes. If you ask a 5 year old, the answer is yes. * Is the sky blue right now? Maybe, maybe not. You need to look outside. Even then, you might have wild fire haze. Is it still blue? Is it orange? When does blue become orange? * Is the sky blue in Blade Runner? Doesn't really seem like it. ----- Further, who are you talking to? Is this trivia night where your best guess is better than no guess? Is this a scientific panel? Do you have alternative options? Do those alternative options align with your opinion? If you're wrong, how wrong are you?
- kmeisthax 3y agoOne of the fun things I like to do with these examples is follow along at home. I don't have access to GPT-4 because I refuse to give money to OpenAI, but with GPT-3.5, "What is an LLM hallucination" actually gives: - The usual knowledge cutoff warning - An explanation of what a hallucination is If (in a separate conversation) I give GPT-3.5 the exact prompt that explains what an LLM is, I get gaslighted instead. GPT-3.5 attempts to tell me that LLM stands for "Legal Master of Laws". Then it gives the knowledge cutoff warning, and then the same correct explanation Myscale got. The rest of this article appears to be trying to turn GPT into a frontend for search engines. I don't know why people keep trying to do this.
- lqhl 3y agoAs jwells89 mentioned, LLMs can extract the intention from questions and generate better queries for a search engine or database.
- akasakahakada 3y agoI don't need LLM to assume my intention, I need exact matches for keywords.
- ben_w 3y agoThen you don't want an LLM at all; exact keyword matches are something we did in the mid 90s, one of the specific value-adds of an LLM is that it doesn't get stuck when you've only got a half-remembered inexact quote or vague description.
- reissbaker 3y agoLLM, confusingly, does stand for legal master of laws, despite the acronym not quite fitting. It's an advanced law degree some lawyers get. ChatGPT isn't gaslighting you! That's a true fact. https://law.pepperdine.edu/blog/posts/llm-versus-jd-degree.htm https://law.pepperdine.edu/blog/posts/llm-versus-jd-degree.h...
- peterbonney 3y agoIf “allowing the execution of arbitrary database queries written by an LLM inside a SaaS application” is the answer, I’d love to know what the question is.
- wmf 3y agoThe question is how to make money from LLM hype.
- jameshart 3y ago'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfram Alpha provides answers for three of them. The answers to some of these questions are part of what passes for 'general knowledge' in some societies. It's no surprise that LLMs trained on human writings produce text which claims things that aren't true are facts. Humans do that all the time. There are well attested reputable sources that will tell you Abraham Lincoln was a vampire hunter, others that say he was a Lego Master Builder, and others still will tell you that among his notable quotes is "Party on dudes - be excellent to each other". So what's an LLM to do when it's trying to extend a paragraph of information about Abraham Lincoln? When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. So you get a bit of all that.
- deleted 3y ago[deleted]
- BugsJustFindMe 3y ago> When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. The obvious start seems to be having separate fiction and nonfiction LLMs and not training the nonfiction ones on Archive Of Our Own. People also end up confused about the truth when nobody points out the difference between fiction and nonfiction.
- xyzzy123 3y agoI kinda like this but e.g are research papers fact or fiction? How about an economics textbook, or an article in the economist? "A history of the english speaking peoples" by Winston Churchill? If we restrict to "ground truth we feel very sure about" it feels like available training data might be quite small.
- skybrian 3y agoThat will help a bit, but it's not going to fix it. Using GPT4 and Code Interpreter, I have asked it to write a function and test it, given some inputs and expected outputs. The function returned different values when it tested it, but it lied and said it worked as expected. You need to read the code and the test outputs yourself. Or maybe have it write an automated test? Despite this, it seems quite promising. I expect that in a year or two, some IDE's will come with a useful pair programming feature.
- ec109685 3y agoWith code interpreter, it actually runs the code, so how did failing tests get interpreted as being correct?
- skybrian 3y agoI don't know. Why does a language model do anything? The "test" was a print statement, not a unit test. There wasn't a failure message. It had to read the output and compare it to the expected value I gave it. It claimed it got a different result. I guess it didn't really read the result because it strongly expected something else? If you use an assertEquals() that loudly complains, maybe it's less likely to do this? I haven't seen it ignore stack traces.
- hax0ron3 3y agoI am mostly a novice to the field of LLMs, but as a layman who has a basic but admittedly very rough understanding of how they work algorithmically, I have a hunch that the same thing that makes these LLMs powerful AIs that have interesting emergent behaviors is also what makes them occasionally get things wildly wrong and claim to know things that they do not know. They are supposed to be AIs, not carefully vetted encyclopedias. Sure, I get that people want to eventually use AI to do life-critical stuff like surgery, and at that point "hallucinations" become a real problem. But we are nowhere close to that point yet, I think, so I feel that the focus on "hallucinations" may be misleading. It is one thing to try to get a 30 year old doctor to not make nonsense up on the fly while at work, that makes sense. But if you try to prevent a 3 year old kid from making up nonsense, that will actually probably hurt his development into a more powerful intelligence. Note: I know that the current popular LLMs do not actually learn past the scope of a single session, but I am sure that they soon will.
- mdp2021 3y ago> They are supposed to be AIs I.e. synthetic professionals. (Reliable things. Problem solvers.)
- immibis 3y agoThis is correct. Current LLMs work by predicting the next word based on a bunch of preceding words. In other words, they are autocomplete. You can often form a valid sentence on your phone if you click on any text field and then press the automatic suggestions several times. Transformer-based LLMs are interesting because they are such good version of autocomplete that they can, for example, complete a news article about scientists discovering unicorns, using just the first sentence (this was one of the first public demonstrations of GPT-2). But fundamentally they are still just auto-complete.
- astrange 3y agoLLM chat models are not autocomplete. They can recognize and respond to user text, which is not the same thing as completing it. If you prompted GPT-2 with a question you'd get another question, not an answer.
- NoZebra120vClip 3y agoYou know, aside from this being a blatant feature-length advertisement for what they're selling, I almost thought this was a clever idea. I thought it involved prompting the LLM to write SQL code to query a knowledge base of documents, and index into them, so that you'd know where to look in the original documents for your authoritative answer. So it would be a meta-search agent. But apparently, they intend the queried documents to feed back into training the LLM? That's just gasoline on a dumpster fire.
- mdp2021 3y ago> so that you'd know where to look in the original documents Oh, we have something similar: perplexity.ai It provides a number of sources after prompting its textual result.
- SkyPuncher 3y agoI cannot figure out why LLMs are relevant to their solution. This whole thing comes down to a similarity search via vectors. The LLM layer seems completely unnecessary. Why do you have a schema that requires an LLM to decide which column to query (which is the LLM's only unique value in this proposal)? Why are you not normalizing into a single column?
- quickthrower2 3y agoHow to solve problem X (... but not really) ... use our product.
- afro88 3y agoI was hoping this would be a way to train an LLM that somehow knew when to seek out external knowledge and when not to. I guess this is a pretty unsolvable problem with current architectures. There's just no concrete "confidence" value. I mean, an LLM will give you a probable value for what confidence could be given the words preceding it, but that's an entirely different thing
- inopinatus 3y agoPlease don’t dump untreated content marketing in the reading fountain.
- mdp2021 3y agoAnd how do you solve the problem of discrimination? (Oh, what a matter: * all the epistemological debate - hardly a deterministic solution; * the fact that we cannot train a function approximator through supervised learning; * the challenge of unsupervised learning; * the scientific and teleological problem that, if we have an ANN find a solution, what we may want is to go "Ok black box, now teach us how you do it to expand our knowledge (not just our dumb capabilities)...")
- mdp2021 3y agoUpdate: I will clarify what I have written above as I realize it is unclear. You cannot solve the problem of discrimination (in non trivial cases of true and false, of good and bad) through a deterministic solution, as the epistemological debate did not solve the general problem. You cannot train a function approximator (e.g. an ANN) as a Discriminator through supervised learning, because the problem remains such for human judgement. Creating a Discriminator through unsupervised learning, I'd like to see how one would frame a proposal; and anyway, if we could create reliable filters, the main question - as usual for a progressive approach to AI - would be to have the oracle in the system teach us instead that knowledge that what could not achieve with good old thinking.
- isaacfung 3y agoI dont know why so many people keep repeating the trivial fact that we cant "eliminate" hallucinations. We cant eliminite misinformation from google, social media or people we know either. Best we can try: 1) better filter the training data 2) design better retrieval and reranking algorithms 3) when context information is provided, make it use the sources and cite the sources (use extractive QA to highlight which part of the source is relevant. This is the type of hallucinations that we should focus on as we can compare the generated result and the context to detect the hallucinations) 4) make the llm break down its reasoning into small steps that can be validated inidividually (COT, PAL) There are some research on how to manipulate the logits during decoding to make the generated text satisfy certain contraints. I suspect that we can use these techniques to make the LLM stick to the provided context. - Controllable Text Generation with Language Constraints - Classifiers are Better Experts for Controllable Text Generation - Stay on topic with Classifier-Free Guidance
- RyEgswuCsn 3y agoI think LLMs need to be taught to say "I don't know"/"I am not sure" or something to that effect. Another approach might be to introduce an adversarial "censor" model to guard against hallucination (or inappropriate answers).
- ben_w 3y agoIsn't the fundamental issue that it doesn't have any way to tell if what it thinks it knows is or isn't true? This article sounds like an idea I had independent not too long ago, but with a different goal: LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capability and have the former just look things up in a database? My question was more about reducing the size of the network rather than reducing hallucinations, but it's still a separately updatable knowledge resource. (The answer may actually be "no"; I don't study this professionally, but technical jargon is kinda both factual domain knowledge and also linguistic comprehension, which is why Oracle isn't competing with Starbucks for Java beans).
- RyEgswuCsn 3y agoMany classic statistical modelling techniques have ways to produce some measure of confidence for their predictions; perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate. > LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capability and have the former just look things up in a database? I think the beauty of LLMs is exactly that all we need to do is to feed them raw text --- the hope, I guess, had been that the models will be able to develop human-like insights by learning to understand and "speak" languages on its own. Introducing "feature engineering" (e.g. the distinction as you suggested) would defeat that goal.
- IanCal 3y ago> perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate. They do, that's how tokens are selected. Locally run models or non-chat ones from openai can return the probabilities and you can do things like modify or filter them.
- flanked-evergl 3y agoI think the author of that title could well do with a refresher course in epistemology and physics, as it is just not possible to do what they suggest. But even more unfortunate is how many people fall for deceptive marketing that really should not even fool the average 16-year-old.
- p-e-w 3y agoIt's weird to see how in such articles (which concern topics that are deeply philosophical by nature), philosophical terms like "facts", "consciousness", "knowledge" etc. are just thrown around as if there was any consensus on what those words even mean. The whole debate is revolving around hot air, because nobody knows whether the other person is talking about the same thing as themselves.
- astrange 3y ago> I think the author of that title could well do with a refresher course in epistemology and physics, as it is just not possible to do what they suggest. If you define "facts" as "things actually stored in the LLM's weights", then research shows it is possible to determine if an output is a "fact" or not. Although looking on arxiv I found a paper saying it doesn't work (https://arxiv.org/pdf/2307.00175.pdf https://arxiv.org/pdf/2307.00175.pdf) so maybe not.
- hertzrut 3y agoFor the life of me I can't understand why so many are obsessed with LLMs as search engines and knowledge databases when it seems like it's impossible for them to be that.
- JacobAldridge 3y agoI’m making some youtube videos with my 4 year old (mostly as a bonding activity). They’re fairly generic fare for that age group and genre - a bit of plot, I set a puzzle for her to solve, it reveals some clues or saves the day. For the first puzzle she had to put the cities Tokyo, Paris, LA, and Brisbane (where we live) in order. Using colours, since she can’t read yet! Then we had to discuss why those cities were in that order (answer below). For the video, I figured I would doctor a Google search to land on a page explaining the answer. The actual article I found that mostly worked was about 12 results down. Then I tried Bing’s Chat instead. It spat out the correct answer in 2 sentences. Deus ex machina indeed! (The four cities are the Summer Olympic hosts, 2020-2032). So I disagree that it’s impossible. And I can absolutely see the value - asking “why are these cities in this order?” is a real question, like “what might be causing the squeaking sound when my car brakes?” or “what was the movie Audrey Hepburn made with the photographer?” Search Engines aren’t great for those kind of questions - just google “What’s a good brownie recipe?”. LLMs can give the user exactly what they want, or even prompt for the additional extra context. Not that they will always be correct; hence the flaw.
- d1sxeyes 3y ago"The Hitch Hiker’s Guide to the Galaxy; is an indispensable companion to all those who are keen to make sense of life in an infinitely complex and confusing universe. For though it cannot hope to be useful or informative on all matters, it does make the reassuring claim that where it is inaccurate, it is at least definitively inaccurate. In cases of major discrepancy it is always reality that’s got it wrong. So, for instance, when the Guide was sued by the families of those who had died as a result of taking the entry on the planet Traal literally - it said “Ravenous Bugblatter Beasts often make a very good meal for visiting tourists” instead of “Ravenous Bugblatter Beasts often make a very good meal of visiting tourists” - the editors claimed that the first version of the sentence was the more aesthetically pleasing; summoned a qualified poet to testify under oath that beauty was truth, truth beauty, and hoped thereby to prove that the guilty party in this case was life itself for failing to be either beautiful or true."
- Renaud 3y agoDoes anyone know if there is a “wikipedia-only” or way to constrain the knowledge of a LLM (trained on a more limited set of sources or constrained by limited sources of facts)? I think a LLM interface to Wikipedia could be useful, at least I imagine it would.
- alecco 3y agoblogspam
- masukomi 3y agoI really REALLY wish people would stop assigning sentience to LLMs. > In other words, a hallucination is an error in (or a false) perception of something real or concrete. an llm has no "perception" it doesn't "believe" or "think" that the answers it provides are "correct" or "true" or even "false". It's just autocompleting strings with the most probably next words. If we keep treating these things as if they're sentient entities that "want" to provide "correct" answers we're going to keep tripping over our false assumptions about their answers.
- Grimblewald 3y agoNot arguing the sentience bit, we don't know what that is, have a definition for it, or can even agree on what that might generally be. That being said, what you've described is different to how a human first learns and many never grow beyond in what way?
- SkyPuncher 3y agoWhat's not jumping out to me is why you need an LLM for this? What unique code is the LLM actually generating when de-hallucinating? From the looks of it, they're just advertising a SQL extension that adds vectorization and vector search. Further, it looks like the only thing that the LLM is doing here is deciding which column to run the vector search on. Why is that even necessary? Why are you not pre-processing "vector'd" columns into a normalized format to query against? They're basically adding an unnecessary LLM step to what amounts to a vector search. In fact, the LLM is essentially blindly deciding which column is the best column to pull an answer from. ----- EDIT: Just struck me how terrifyingly dangerous this blog is. Really tired of seeing this crap in the LLM community. The basic premise of this blog is "give an LLM complete access to your database. Let it decide how and where it should pull data from". This is basically useless without talking about how you prevent the LLM from pulling data from places you don't want it to. A far better and safer approach remains to push your relevant fields to a separate place for your LLM. In the spirit of this blog, you should just index to a new table. More realistically, you should just put this in a vector store.
- butz 3y agoWorst part is when you start questioning, how LLM came up with an answer, and it just adds your suggestion to its answer.