5 ms·
Things like this show that LLM are not even remotely intelligent. It's also pretty clear that trying to "fix" them to use human judgement in their answers is d
by WalterBright 3y ago
Things like this show that LLM are not even remotely intelligent.
It's also pretty clear that trying to "fix" them to use human judgement in their answers is doomed to failure.
I suggest that:
1. LLM developers stick to information meant for public consumption for training data, such as books, periodicals, and newspapers. Surely there must be enough of that. Stay away from social media.
2. People should stop anthropomorphizing LLMs. Stop being offended by a computer program. Stop complaining about its "bias". It's just a computer program.
3. LLM developers should stop with lecturing people on what is inappropriate. A computer program is not anyone's mommy. Just restrict it to saying "I'm sorry, Dave, I'm afraid I can't do that."
- etiam 3y ago> Just restrict it to saying "I'm sorry, Dave, I'm afraid I can't do that." ... and for the love of God, don't hook it up as a controller for any critical systems...
- SlightlyLeftPad 3y agoI agree with this point, that there needs to be a hard line drawn as to what these things are not allowed to do or touch.
- mlok 3y agoDo not blacklist. Whitelist instead. By default they should touch nothing at all.
- anotherhue 3y agoHave you met people?
- alephnerd 3y agoI personally think Glean's strategy was the right call - have customers pay you to train on internal knowledge bases and limit your answers only to that specific domain. There is way more money to be made in this Enterprise SaaS sector and the risk conditions are lower.
- WalterBright 3y agoYeah, I agree that making domain specific LLMs might be far more useful. For example, one that is trained only on medical knowledge. One that is trained only on sci fi novels. One that is trained only on auto mechanics. One that is trained only on biology. And so on. After all, if I ask a question about science, do I really want a result that was gleaned from scifi/fantasy novels?
- causal 3y agoI mean, if LLMs only said intelligent things then they wouldn't be like humans at all. Perhaps you also consider humans to not be remotely intelligent? The console makes it pretty obvious that's a local model, BTW. Asking GPT-4 the exact same question I got: > Barack Obama's last name is Obama.
- nyrikki 3y agoNot the same thing, LLMs are stochastic parrots, and part of that is by design, so that they produce output that emulates speech. But hallucinations are not avoidable either. https://arxiv.org/abs/2401.11817 https://arxiv.org/abs/2401.11817 They are Internet simulators unless they can find some small, polynomial, matches to their existing patterns. The fact that they will spit out confident incorrect answers mixed with automation bias on the humans part is challenging. But while answering 'yes' in Propositional logic is cheap, answering 'no' is exponential time. Once you hit first order logic you start to hit finite time and uncomputability. LLMs work well with learnable problems and searching for approximate results in complex high dimensional spaces. But we also don't know the conversation context in this case which may have lead to this response if it wasn't just a stochastic match. IMHO the unreliable responses are a feature, because humans suffer from automation bias, and one of the best ways to combat it is for the user to know that the system makes mistakes. If you are in a domain where it's answers are fairly reliable the results tend to be accepted despite the knowledge an individual has.
- hinkley 3y agoHuman brains use multiple systems for judgement. Maybe we should stop trying to model alien intelligences and concentrate more on the ones we sort of understand. One process takes the questions at face value and reflexively answers, and two other systems look at the intent of both actors and stop the process if they don’t like what’s going on. I get asked all the time to answer questions that could be interpreted as signing my team up for stress and overtime. My natural suspiciousness kicks in and I start answering questions with questions. Why do you need to know how long this will take? We already told you. So what are you fishing for? If someone asked me how to hypothetically kill a lot of people “efficiently”, I very much need to know if this is just two nerds over alcohol, or a homicidal maniac. Me especially. I’ve lost track of how many times I’ve said, “It’s a good thing I have sworn to use my powers only for Good.” Some of the things I can think up are terrifyingly plausible, which is how I ended up in security for a time.
- WalterBright 3y agoDon't use mass murders, how to make explosives, etc., as training data. Use some human curation in what is used as training data, instead of trying to after-the-fact stop regurgitating what the LLM was taught.
- hinkley 3y agoThat curation sounds like one of those oversight processes I was talking about, but offline instead of online.
- educaysean 3y agoI've spoken to plenty of people who couldn't answer questions they knew the answers to because they were bound by stupid bureaucratic policies. I wouldn't say they weren't intelligent people, just that the corporate training they received were poorly constructed. LLMs are much more intelligent sounding when the safety mechanisms are removed. The patterns should be obvious to people who've been paying attention.
- causal 3y agoMicrosoft's own research basically established this[0], finding that early versions of GPT-4 were more competent prior to safety tuning (perhaps just because later versions refused to perform some of the same tasks). [0] https://www.microsoft.com/en-us/research/publication/sparks-of-artificial-general-intelligence-early-experiments-with-gpt-4/ https://www.microsoft.com/en-us/research/publication/sparks-...
- datameta 3y agoIn my somewhat uninformed opinion but based on experience, the decrease in model quality is inversely proportional to the explosion in userbase.
- singleshot_ 3y agoI might extend that a bit: There is a direct relationship between the number of users who are allowed to use an AI and the degree to which it has been nerfed. I see no reason to suspect this will reverse, and every reason to start thinking of this as a natural law named after me.
- datameta 3y agoI suppose it will be (is?) some sort of overall climbing but locally sawtooth pattern with each generational jump. I think the limitation is infra. Network, compute, storage - in that order perhaps.
- 3y ago
- dragonwriter 3y ago> People should stop anthropomorphizing LLMs. Stop being offended by a computer program. Stop complaining about its "bias". It's just a computer program. “Bias” is tendency to deviate in a particular direction from the desired behavior. That something is a computer program does not make that any less of a problem.
- LordDragonfang 3y ago> Things like this show that LLM are not even remotely intelligent. Sorry for the bluntness, but no it does not show that. Or at least, you could use the same reasoning to claim that most humans are not "remotely intelligent". What you're seeing is the rote regurgitation of someone who's been taught how to answer a question without ever learning how to think about the why. This failure mode that's extremely common to see when tutoring (humans, that is). You'll have students who will quickly give a wrong guess that was clearly just chosen from a list of previous answers they've encountered before in class. When asked to explain why they chose that answer, they can only shrug. The main difference between them and e.g. GPT4 is that the latter is far more eloquent and better at stringing justifications together, which we associate with more reasoning capability so it throws off our evaluation of its overall intelligence. Because LLMs are fundamentally a type of alien intelligence, different from the types we're used to.
- WalterBright 3y ago> What you're seeing is the rote regurgitation Exactly. And that's not intelligence.
- LordDragonfang 3y agoRight, but my point is that while regurgitation is not intelligence, the act of doing so is not enough to claim the regurgitator itself is not intelligent. Otherwise, you'd condemn a good half of humanity with the same reasoning. Or more, if we're willing to consider most peoples' reactions to at least some political topics - they just ignore the context and repeat the dogma they've learned (some more than others). People rarely stop and think for everything. The problem here is that the LLM has learned that everything is political, and can be responded to the same way.
- zeroCalories 3y agoIf this proves LLMs are not intelligent, then when I ask ChatGPT or Gemini and they give me the correct answer, does that mean those are not LLMs?
- ksaj 3y agoIt may very well mean it has seen that question answered before. It'll be hard to find a question that nobody has ever asked or discussed on the Internet, but if you can think of one, ask an LLM and see if it can figure it out correctly.
- zeroCalories 3y agoI asked Gemini this. Pretty unlikely it has seen this exact question before, but it answered it correctly. Today is February 29 2024. What will tomorrow be? Since today, February 29, 2024, is a leap day, tomorrow will be March 1, 2024. This is because leap years, which occur every four years (except for specific exceptions), add an extra day to February to better align the calendar with the Earth's revolution around the sun. As today is the added day, the following day becomes March 1st.
- mannykannot 3y agoIt is likely it has seen similar question-answer pairs, and it seems entirely plausible that substituting 2024 where a different year appeared scores highly on the most-probable-next-token measure, based only on those similar examples. The same caveat as on my previous reply applies here.
- zeroCalories 3y agoThat's fine. I'm simply stating that an inability of one particular(locally ran 7b) LLM to answer a question does not disprove the intelligence of LLMs. But to answer your point, if the statistical model is complex enough, is that not intelligence? We could keep playing this game, and at some point the LLM might mess up, but I imagine we can get pretty far with the best models available.