8 ms·
We need a way to make tight little specialist models that don't hallucinate and reliably report when they don't know. Trying to cram all of the web into a LLM i
by Animats 3y ago
We need a way to make tight little specialist models that don't hallucinate and reliably report when they don't know. Trying to cram all of the web into a LLM is a dead end.
- z3c0 3y agoLogistic Regression is simple to implement, supports binomial, multinomial, and ordinal classification, and is a key layer in NN, as it's often used as an actuator that sorts a probability into a discrete category. Very good for specialized problems, and easily trainable to sort unknown or nonsensical inputs into a noncategory. Linear Regression is great for projections, and can even be fit to time series data using lagging.
- rco8786 3y agoBingo. I've been beating this drum since the initial GPT-3 awe.. The future of AI is bespoke, purpose-driven models trained on a combination of public and (importantly) proprietary data. Data is still king.
- JimtheCoder 3y agoSo, search isn't dead after all...
- stavros 3y agoSearch has definitely been dead since before LLMs, we just don't have a replacement yet.
- JimtheCoder 3y agoI mean more the concept of search, not the current implementation
- stavros 3y agoOh, I don't think we'll ever stop wanting to search for things. Maybe not everything, but some things.
- imtringued 3y agoSearch is about finding existing results. The opposite of search is to start from first principles and work your way up until you have created the desired result yourself. In the real world, search reduces information acquisition costs as you only have to spend time and resources on finding an existing result rather than recreating it.
- Forgeties79 3y agoFor a while my replacement was “use google, add ‘reddit’ at the end.” Not sure how much longer that will work given even just this limited blackout impacted how effective that was lol
- edgyquant 3y agoThat hasn’t worked since about three months after companies found out people do it. It’s all astroturfing now days anyway and if it applies to products (which it for sure does) you can be sure that government actors caught on as well.
- mitchdoogle 3y agoI'm not really sure what you're saying. It seems to me the number of shills creating content is vastly outnumbered by normal people creating content, so the trick of adding "reddit" to the end of queries is still very much useful (blackout protests aside) since it's usefulness derives from getting information from normal people and then having normal people upvote the "best" comments. Just the other day I tried this with data recovery software very quickly found some free software that was highly praised. If I had just searched "data recovery software" on Google I get whichever data recovery place as the best SEO, including many of them who claim that a novice should be trying it at all and should use their services pronto. Using reddit in this case gave me the exact information I wanted, where Google alone was next to useless.
- Forgeties79 3y agoYeah I don’t know their experience, but mine is when I add “Reddit” i often get 2-3 threads talking about the exact thing I’m looking up in the top 10 results.
- edgyquant 3y agoWhat I’m getting at is that those threads have a high likelihood of being gamed to make you think a certain way
- 3y ago
- samstave 3y agoIt will be a terrific moment when we can browse a galaxy map or other lists of LLMs, and select "context" groups - so you can say from history, give me X
- thelittleone 3y agoThat doesn't sound ideal at all to me. It also sounds like a bottleneck. I want to ask AI once and it recommends or selects the best "context" group.
- samstave 3y agoIt would bea good visual training tool for kids so they can get an understanding early on other than a block-box view.
- danielvaughn 3y agoI’d be really interested to see an AI model built off of Sci Hub data.
- fooker 3y agoFacebook announced this a month or so before chatgpt took off, and got lambasted everywhere because of hallucinations.
- croes 3y agoThis would only work if thinks like GPT were actually intelligent. Train on dataset A to learn to think, use thinking on dataset B to become an export in B's field.
- Animats 3y agoYou need sort of a "primary education" data set which gets the model up to roughly a high school education level. Then augment it with special-purpose models for specific areas. But until "I don't know" comes out, rather than hallucinations, we're in trouble.
- rco8786 3y ago> You need sort of a "primary education" data set which gets the model up to roughly a high school education level. Yea that was what I was getting at with the "combination" of data. The publicly available data provides the base/primary education, then you specialize it with your proprietary data and bam, you have an AI model that nobody else can produce...an actual product moat.
- causalmodels 3y agoIt's not an either or. We're going to leverage the web trained LLMs to bootstrap the specialist models via combination of training token quality classifiers and synthetic data generation. Phi-1 is a pretty good example of this.
- TX81Z 3y agoGPT4 is really good at code and you can generally verify hallucination easily. The other good use cases are using LLM to turn natural language prompts into API calls to real data.
- LASR 3y agoThis is what I've been doing. GPT-4 to generate some data from some input, followed up by a 3.5T call to verify the output against the input to verify the content. You can feed the 3.5T output straight back into GPT-4 again and it will self-correct. Doing this a couple of times gives me 100% accuracy for my use case that involves some level of summarization and reasoning. Hallucinations are not as big of a deal at all IMO. Not enough that I'll just sit there and wait for models that don't hallucinate.
- adriand 3y agoThis sounds interesting, can you detail this data flow a bit more and maybe provide an example?
- simion314 3y ago>GPT4 is really good at code for popular languages, though for JS it most of the time outputs obsolete syntax and code. Today i tried to do a bit of scripting with my son in Garrys Mod, it uses Expression 2 for a Wiremod module. GPT hallucinated a lot of functions and the worst part it switched almost each time from e2 to lua. It is good at solving homeworks for students, or solving popular problems in popular languages and libraries though it might give you an ugly solution and ugly code, it is probably trained on bad code too and it did not learn to prefer good code over bad code.
- TX81Z 3y agoI’ve been writing JavaScript for 25 years and have always felt it has utterly trash syntax. I’m not sure that’s 100% on GPT. (And I don’t mean to be rude - I just really hate that syntax!)
- skepticATX 3y agoThis is ultimately just very powerful semantic search though, is it not? It seems that what we need to make a big leap forward is better reasoning. There is a lot of debate between the GPT-4 can/can't reason camps, but I haven't seen anyone try to argue that it reasons particularly well.
- famouswaffles 3y agoYou can easily argue 4 reasons particularly well, especially with average human performance as the baseline. People who argue against GPT-4 reasoning at all are arguing against clear results. It extremely easy to show examples and benchmarks of 4 reasoning and understanding. The argument then turns into "well that's not "true" reasoning", whatever that means.
- imtringued 3y agoIt shouldn't even be that difficult to build it. Modus ponens and good theorems are all you need. Building the data set for that should be quite trivial.
- wilg 3y agoHow are we determining it's a dead end here? Recently things like GPT-4 have come out which have improved on that technique. Why would specialization necessarily reduce hallucination and improve accuracy?
- brucethemoose2 3y agoIts complicated. Training on something huge like "the internet" is what gives rise to those amazing emergent properties missing in smaller models (including the recent Pi model). And there are only so many datasets that huge. But its also indeed a waste, as Pi proves. There probably is some sweet spot (6B-40B?) for specialized, heavily focused models pre trained with high quality general data.
- TeMPOraL 3y agoHow much general "thinking"[0] would you want those "tight little specialist models" to retain? I think that cramming "all of the web" is actually crucial for this capability[1], so at least with LLM-style models, you likely can't avoid it. The text in the training data set doesn't encode just the object-level knowledge, but indirectly also higher-level, cross-domain and general concepts; cutting down on the size and breadth of the training data may cause the network to lose the ability to "understand"[0]. -- [0] - Or "something very convincingly pretending to think by parroting stuff back", if you're closer to the "stochastic parrot" view. [1] - Per my hand-wavy hypothesis that the bulk of what we call thinking boils down to proximity search in extremely high-dimensional space.
- nathan_compton 3y agoThis is basically my view as an extensive gpt4 user. It doesn't think/is very stupid, but has such fantastic recall for common patterns that its quite useful. Absolutely none of the smaller models even come close, including GPT3.5.
- jiggawatts 3y agoEven just for spell checking, having a ton of general knowledge helps. Knowing the correct spelling of product names, British vs American English, context-specific alternate spellings, etc… Even GPT 3.5 has trouble following instructions, but I’ve found that GPT 4 is almost flawless. I can tell it the document uses Australian English but to preserve US spelling for product names and it’ll do it! One quirk is that it’s almost too good at following instructions. You have to tell it to preserve product names, vendors names, place names, etc… otherwise it’ll “correct” the spelling of anything you forgot to list.
- thfuran 3y agoCan you not just tell it not to muck with proper nouns?
- jiggawatts 3y agoYou can, but then it won't correct misspellings of proper nouns! The idea is for it to automatically detect the "language" of each word based on the context and its own understanding of the world. E.g., the following sentence: "We deployed windows server data centre 2022 into our data center, which has no windows for physical security." Will be corrected by GPT-4 to the following: "We deployed Windows Server Datacenter 2022 into our data centre, which has no windows for physical security." Notice that it combined "data" and "centre" into "Datacenter" and it corrected the second "center" into "centre", which is the British/Australian spelling of the word. It also correctly capitalised only the first use of the word "windows", etc... That requires a level of understanding that GPT 3.5 just barely has, and no ordinary grammar checker tool has.
- nomel 3y ago> that don't hallucinate “Hallucination” is part of thought. Solving a new problem requires hallucinating new, non existing, possible outcomes and solutions, to find one that will work. It seems that eliminating the ability to interpolate and extrapolate (hallucinations) would make intelligence impossible. It would eliminate creativity, tying together new concepts, creation, etc. Is the goal AI, or a nice database front end, to reference facts? Is intelligence facts, or is it the flexibility and the ability to handle and create the novel, things that are new? The ability to have confidence, and know and respond to it, seems important, but that’s surely different than the elimination of hallucinations. I’m probably misunderstanding something, and/or don’t know what I’m talking about.
- Barrin92 3y ago>Is the goal AI, or a nice database front end, to reference facts? The latter given the kind of products that are currently being built with it. You don't want your code completion or news aggregator to hallucinate for the same reason you don't want your wrench to hallucinate, it's a tool. And as for hallucinations, that's a PR friendly misnomer for "it made **** up". Using the same phrase doesn't mean it has functionally anything to do with the cognitive processes involved in human thought. In the same way a 'artificial' neural net is really a metaphorical neural net, it has very few things in common with biological neurons.
- mitchdoogle 3y ago"hallucination" just means the AI produced incorrect information. Humans produce incorrect information all the time. But we know we are fallible. I've seen lots of people treating these AIs as infallible, that they can't get anything wrong. That's a huge problem with people, not the AI. Obviously it's worth it to try and eliminate the incorrect information, but what grand-op is saying is we don't want to do that if it takes away some valuable emergent properties.
- zuminator 3y agoI think AI hallucinations are more than just incorrect information. They're incorrect information coupled with an air of certitude. Sometimes with a whole backstory. And often oblivious to any inherent contradictions. It's like, if you ask a random person what years Lyndon B Johnson was president, they might say something like 1964-1968, shrugging with a bit of uncertainty. And both of those years would be incorrect but close. Whereas an AI (if hallucinating) would say something like Lyndon B Johnson was president from April 15, 1865 – March 4, 1869, after the shooting of John F. Kennedy in Ford's theater. The dates are precise but completely wrong, as are the accompanying details. Then if you ask the AI when John F. Kennedy died, it might correctly respond November 22, 1963, in Dallas, TX, totally unaware how this information doesn't match with the erroneous information it gave earlier. I've been thinking there's some parallels between how AIs hallucinate and how human toddlers do. If you ask a toddler/young child a question about a fact they don't know, they will usually say, "iuno" (even when they should), but depending on the child and the circumstances, they will sometimes just make up a story on the spot and sound as if they believe it. "Who invented ice cream?" "Santa Claus! Mommy left him milk and cookies and he turned it into ice cream." It doesn't make any real sense but it seems facially plausible in their universe. But somewhere between first learning to speak and around 7ish, kids become markedly more accurate how they model the world, and their responses become correspondingly less fanciful. And they continue to improve beyond that point. So how are kids doing what LLMs are currently incapable of? How do we teach ourselves not to hallucinate? Or do we, really? I mean, if I tell myself I'm going to make it through the intersection before the light turns red, but I end up running the red light, was I just mistaken, or was that a self-delusion, i.e., a mini-hallucination of sorts? Probably a self-driving car would be less likely to make that category of mistake, so maybe I shouldn't be so smug about being grounded in reality.
- 4ndrewl 3y agoThey don't know that they don't know. It's only hallucination from a human's perspective. From the model's perspective it's _all_ hallucination.
- imtringued 3y agoThat is true. There is no difference between fiction and nonfiction for the LLM.
- Solvency 3y agoThe same way "attention" was a game changer, I'm not sure why they don't invent some recursive self-maintenance algorithm that constantly improves the neural network within. Self directed attention, so to speak.
- twobitshifter 3y agoI don’t know how a bunch of specialist models don’t combine into a super useful generalist model. Do we believe too much knowledge breaks an LLM?
- croes 3y agoThey never know. What we call hallucination is just when the resulting text is wrong but the underlying probabilities could be high.
- usrbinbash 3y ago> and reliably report when they don't know. Then we need a new system, because LMs, no matter if they are large or not, cannot do that, for a very simple reason: A LM doesn't understand "truthfulness". It has no concept of a sequence being true or not, only of a sequence being probable. And that probability cannot work as a standin for truthfulness, because the LM doesn't produce improbable sequences to begin with...it's output will always be the most (within heat settings) probable sequence. The LM simply has no way of knowing whether the sequence it just predicted is grounded in reality or not.
- deleted 3y ago[deleted]
- acover 3y agoAn llm can detect when an llm has outputted an inconsistent world. It can reason. To an extent.
- exitb 3y agoIf this is true, why is GPT-4 better in that regard than GPT-3.5? Or why do questions about Python yield much less hallucinations than questions about Rust, or other less popular tech?
- usrbinbash 3y agoWhat specifically about these observations contradicts my statement? Wrong statements about Python are simply less probable than wrong statements about Rust, since there is more Python than Rust in the training data. That changes exactly nothing about the fact that the system isn't able to detect when it makes a blunder in Python.
- exitb 3y agoYou've claimed that LLMs create most probable output, which does not necessarily align with truth. So a bigger LLM will be better at creating the most probable output, but that would not translate into being more truthful. That could be interpreted as "better LLMs are expected to be better bullshitters". That is not what we've observed though. Quite the opposite - we're seeing that the bigger LLM is and the more domain-specific material it digested, the more truthful it becomes. Yes it can still make an error and be unable to spot it, but so can I.
- koochi10 3y agoI disagree with this slightly, modern computer chips follow a general purpose architecture not special purpose ones. The reason for this is building a computer chip is expensive and difficult to do. Similarly building any useful language model requires tons of compute power, and very smart ML researchers. Most of the smaller open source one's are just trained on GPT output. By "cramming all of the web" on a model what is really going on is the hidden layers of that network are getting better at understanding language and logic. Imagine trying to teach a kid who doesn't know how to read to learn about a Science by only giving them science textbooks. Chances are they won't get very far. Building little specialist model's don't really work either. It's like trying to train a parrot to do science, sure it can repeat some of the phrases that you give it, but at the end of the day it's not really making any new connections for you.
- Solvency 3y agoSince when did the world decide to shed the grammatically correct "computing power" for this weird tech bro "compute power" phrase?
- lukeschlather 3y agoDefinitions are a little fuzzy but "compute" is often used to distinguish between "compute," as in CPU, memory, and disk. And GPU is a very specialized kind of compute power. There's really no "grammatically correct" here these are different senses of the word. "Computing power" doesn't exactly have the same sense of specifically referring to a CPU or GPU as "compute power."
- int_19h 3y agoSince we adopted English as the standard language, complete with its penchant for butchering words by shortening them for the sake of convenience and speed. I wonder what people said about "bus" back in the day, especially those who knew Latin.
- gremlinsinc 3y agoI'm trying to get funding to do just this. Essentially nested LLMs were every LLM is very topical and if it's not trained on something there's indexes to other LLMs it can ask to get the right answer. I think of it kinda like a hub and spoke model.