4 ms·

The hard part is that for all the things that the author says disprove LLMs are intelligent are failings for humans too. * Humans tell you how they think, but

The hard part is that for all the things that the author says disprove LLMs are intelligent are failings for humans too. * Humans tell you how they think, but it seemingly is not how they really think. * Humans tell you repeatedly they used a tool, but they did it another way. * Humans tell you facts they believe to be true but are false. * Humans often need to be verified by another human and should not be trusted. * Humans are extraordinarily hard to align. While I am sympathetic to the argument, and I agree that machines aligned on their own goals over a longer timeframe is still science fiction, I think this particular argument fails. GPT o3 is a better writer than most high school students at the time of graduation. GPT o3 is a better researcher than most high school students at the time of graduation. GPT o3 is a better lots of things than any high school student at the time of graduation. It is a better coder than the vast majority of first semester computer science students. The original Turing test has been shattered. We're building progressively harder standards to get to what is human intelligence and as we find another one, we are quickly achieving it. The gap is elsewhere: look at Devin as to the limitation. Its ability to follow its own goal plans is the next frontier and maybe we don't want to solve that problem yet. What if we just decide not to solve that particular problem and lean further into the cyborg model? We don't need them to replace humans - we need them to integrate with humans.

13years 1y ago
> GPT o3 is a better writer than most high school students at the time of graduation. All of these claims, based on benchmarks, don't hold up in the real world on real world tasks. Which is strongly supportive of the statistical model. It will be capable of answering patterns extensively trained on. But is quickly breaks down when you step outside that distribution. o3 is also a significant hallucinator. I spent quite a bit of time with it last weekend and found it to be probably far worse than any of the other top models. The catch is that it its hallucinations are quite sophisticated. Unless you are using it on material for which you are extremely knowledgeable, you won't know. LLMs are probability machines. Which means they will mostly produce content that aligns to the common distribution of data. They don't analyze what is correct, but only what is probable completions for your text by common word distributions. But when scaled to incomprehensible scales of combinatorial patterns, it does create a convincing mimic of intelligence and it does have its uses. But importantly, it diverges from the behaviors we would see in true intelligence in ways that make it inadequate for solving many of the kinds of tasks we are hoping to apply them to. The being namely the significant unpredictable behaviors. There is just no way to know what type of query/prompt will result in operating over concepts outside the training set.
- KTibow 1y ago
  Were you using it with search enabled?
- benlivengood 1y ago
  > o3 is also a significant hallucinator. I spent quite a bit of time with it last weekend and found it to be probably far worse than any of the other top models. The catch is that it its hallucinations are quite sophisticated. Unless you are using it on material for which you are extremely knowledgeable, you won't know. At least 3/4 of humans identify with a religion which at best can be considered a confabulation or hallucination in the rigorous terms you're using to judge LLMs. Dogma is almost identical to the doubling-down on hallucinations that LLMs produce. I think what this shows about intelligence in general is that without grounding in physical reality it tends to hallucinate from some statistical model of reality and confabulate further ungrounded statements without strong and active efforts to ground each statement in reality. LLMs have the disadvantage of having no real-time grounding in most instantiations; Gato and related robotics projects exempted. This is not so much a problem with transformers as it is with the lack of feedback tokens in most LLMs. Pretraining on ground truth texts can give an excellent prior probability of next tokens and I think feedback either in the weights (continuous fine-tuning) or real-world feedback as tokens in response to outputs can get transformers to hallucinate less in the long run (e.g. after responding to feedback when OOD)
  - ary 1y ago
    Arguing that many humans are stupid or ignorant does not support the idea that an LLM is intelligent. This argument is reductive in that it ignores the many, many diverse signals influencing the part of the brain that controls speech. Comparing a statistical word predictor and the human brain isn’t useful.
    - benlivengood 1y ago
      I'm arguing that it's natural for intelligent beings to hallucinate/confabulate in the case where ground truth can't be established. Stupidity does not apply to e.g. Isaac Newton or Kepler who were very religious, and any ignorance wasn't due to a fault in their intelligence per se. We as humans make our best guesses for what reality is even in the cases where it can't be grounded, e.g. string theory or M-theory if you want a non-religious example. Comparing humans to transformers is actually an instance of the phenomenon; we have an incomplete model of "intelligence" and we posit that humans have it but our model is only partially grounded. We assume humans ~100% have intelligence, are unsure of which animals might be intelligent, and are arguing about whether it's even well-typed to talk about transformer/LLM intelligence.
Yeask 1y ago
How many books or software wrote by recently graduated students have you read/use? And by LLMs?
TimTheTinker 1y ago
I think TA's argument fundamentally rests on two premises (quoting): (a) If we were on the path toward intelligence, the amount of training data and power requirements would both be reducing, not increasing. (b) [LLMs are] data bound and will always be unreliable as edge cases outside common data are infinite. The most important observed consequences of (b) are model collapse when repeatedly fed LLM output in further training iterations; and increasing hallucination when the LLM is asked for something truly novel (i.e. arising from understanding of first principles but not already enumerated or directly implicated in its training data). Yes, humans are capable of failing (and very often do) in the same ways: we can be extraordinarily inefficient with our thoughts and behaviors, we can fail to think critically, and we can get stuck in our own heads. But we are capable of rising above those failings through a commitment to truths (or principles, if you like) outside of ourselves, community (including thoughtful, even vulnerable conversations with other humans), self-discipline, intentionality, doing hard things, etc... There's a reason that considering the principles at play, sitting alone with your own thoughts, mulling over a problem for a long time, talking with others and listening carefully, testing ideas, and taking thoughtful action can create incredibly valuable results. LLMs alone won't ever achieve that.
GuB-42 1y ago
Humans can be deceptive, but it is usually deliberate. We can also honestly make things up and present them as fact but it is not that common, we usually say that we don't know. And generally, lying is harder for us than telling the truth, in the sense that making a consistent but false narrative requires effort. For LLMs, making stuff up is the default, one can argue that it is all they do, it just happens to be the truth most of the times. And AFAIK, what I would call the "real" Turing test hasn't been shattered by far. The idea is that the interrogator and the human subject are both experts and collaborate against the computer. They can't cheat by exchanging secrets, but anything else is fair game. I think it is important because the Turing test has already been "won" by primitive algorithms acting clueless to interrogators who were not aware of the trick. For me, this is not really a measure of computer intelligence, more like a measure of how clever the chatbot designers were at tricking unsuspecting people.
- 13years 1y ago
  > we usually say that we don't know I think this is one of the distinguishing attributes of human failures. Human failures have some degree of predictability. We know when we aren't good at something, we then devise processes to close that gap. Which can be consultations, training, process reviews, use of tools etc. The failures we see in LLMs are distinctly of a different nature. They often appear far more nonsensical and have more of a degree of randomness. The LLMs as a tool would be far more useful if they could indicate what they are good at, but since they cannot self reflect over their knowledge, it is not possible. So they are equally confident in everything regardless of its correctness.
  - ragequittah 1y ago
    I think the last few years are a good example of how this isn't really true. Covid came around and everyone became an epidemiologist and public health expert. The people in charge of the US government right now are also a perfect example. RFK Jr. is going to get to the bottom of autism. Trump is ruining the world economy seemingly by himself. Hegseth is in charge of the most powerful military in the world. Humans pretending they know what they're doing is a giant problem.
nopelynopington 1y ago
> * Humans tell you how they think, but it seemingly is not how they really think. > * Humans tell you facts they believe to be true but are false. Think and believe are key words here. I'm not trying to be spiritual but LLMs do not think or believe a thing, they only predict the next word. > * Humans often need to be verified by another human and should not be trusted. You're talking about trusting another human to do that though, so you trust the human that is verifying.
- deleted 1y ago
  [deleted]
goatlover 1y ago
But there are places where humans do follow reasoning steps, such as arithmetic and logic. The fact that we need to add RLHF to models to make them more useful humans is also evidence that statistical reasoning is not enough for a general intelligence.
minraws 1y ago
LLMs aren't as good as average humans, most software folks like us like to believe rest of the world is dumb but it isn't. My grand-dad who only ever farmed land and had no education at all could talk, calculate, and manage his farm land. Was he not as good as me at anything academic yes. But I will never be as good at him at understanding how to farm stuff. Most people in who think LLMs are smart seem to conflate ignorance/lack of knowledge to being dumb. This is a rather reductive take, and no I don't believe that's how human intelligence works. Your dumb uncle on thanks giving who might even have a lot of bad traits isn't dumb likely just ignorant. All human IQ studies and movies have distorted our perception of intelligence. A more intelligent person doesn't necessarily need to have more or better quality knowledge. Or measuring them with academic abilities like writing and maths is the dumber/irresponsible take. And yes please feel free to call me dumb in response.

13years 1y ago

> GPT o3 is a better writer than most high school students at the time of graduation. All of these claims, based on benchmarks, don't hold up in the real world on real world tasks. Which is strongly supportive of the statistical model. It will be capable of answering patterns extensively trained on. But is quickly breaks down when you step outside that distribution. o3 is also a significant hallucinator. I spent quite a bit of time with it last weekend and found it to be probably far worse than any of the other top models. The catch is that it its hallucinations are quite sophisticated. Unless you are using it on material for which you are extremely knowledgeable, you won't know. LLMs are probability machines. Which means they will mostly produce content that aligns to the common distribution of data. They don't analyze what is correct, but only what is probable completions for your text by common word distributions. But when scaled to incomprehensible scales of combinatorial patterns, it does create a convincing mimic of intelligence and it does have its uses. But importantly, it diverges from the behaviors we would see in true intelligence in ways that make it inadequate for solving many of the kinds of tasks we are hoping to apply them to. The being namely the significant unpredictable behaviors. There is just no way to know what type of query/prompt will result in operating over concepts outside the training set.

KTibow 1y ago
Were you using it with search enabled?
benlivengood 1y ago
> o3 is also a significant hallucinator. I spent quite a bit of time with it last weekend and found it to be probably far worse than any of the other top models. The catch is that it its hallucinations are quite sophisticated. Unless you are using it on material for which you are extremely knowledgeable, you won't know. At least 3/4 of humans identify with a religion which at best can be considered a confabulation or hallucination in the rigorous terms you're using to judge LLMs. Dogma is almost identical to the doubling-down on hallucinations that LLMs produce. I think what this shows about intelligence in general is that without grounding in physical reality it tends to hallucinate from some statistical model of reality and confabulate further ungrounded statements without strong and active efforts to ground each statement in reality. LLMs have the disadvantage of having no real-time grounding in most instantiations; Gato and related robotics projects exempted. This is not so much a problem with transformers as it is with the lack of feedback tokens in most LLMs. Pretraining on ground truth texts can give an excellent prior probability of next tokens and I think feedback either in the weights (continuous fine-tuning) or real-world feedback as tokens in response to outputs can get transformers to hallucinate less in the long run (e.g. after responding to feedback when OOD)
- ary 1y ago
  Arguing that many humans are stupid or ignorant does not support the idea that an LLM is intelligent. This argument is reductive in that it ignores the many, many diverse signals influencing the part of the brain that controls speech. Comparing a statistical word predictor and the human brain isn’t useful.
  - benlivengood 1y ago
    I'm arguing that it's natural for intelligent beings to hallucinate/confabulate in the case where ground truth can't be established. Stupidity does not apply to e.g. Isaac Newton or Kepler who were very religious, and any ignorance wasn't due to a fault in their intelligence per se. We as humans make our best guesses for what reality is even in the cases where it can't be grounded, e.g. string theory or M-theory if you want a non-religious example. Comparing humans to transformers is actually an instance of the phenomenon; we have an incomplete model of "intelligence" and we posit that humans have it but our model is only partially grounded. We assume humans ~100% have intelligence, are unsure of which animals might be intelligent, and are arguing about whether it's even well-typed to talk about transformer/LLM intelligence.

Yeask 1y ago

How many books or software wrote by recently graduated students have you read/use? And by LLMs?

TimTheTinker 1y ago

I think TA's argument fundamentally rests on two premises (quoting): (a) If we were on the path toward intelligence, the amount of training data and power requirements would both be reducing, not increasing. (b) [LLMs are] data bound and will always be unreliable as edge cases outside common data are infinite. The most important observed consequences of (b) are model collapse when repeatedly fed LLM output in further training iterations; and increasing hallucination when the LLM is asked for something truly novel (i.e. arising from understanding of first principles but not already enumerated or directly implicated in its training data). Yes, humans are capable of failing (and very often do) in the same ways: we can be extraordinarily inefficient with our thoughts and behaviors, we can fail to think critically, and we can get stuck in our own heads. But we are capable of rising above those failings through a commitment to truths (or principles, if you like) outside of ourselves, community (including thoughtful, even vulnerable conversations with other humans), self-discipline, intentionality, doing hard things, etc... There's a reason that considering the principles at play, sitting alone with your own thoughts, mulling over a problem for a long time, talking with others and listening carefully, testing ideas, and taking thoughtful action can create incredibly valuable results. LLMs alone won't ever achieve that.

GuB-42 1y ago

Humans can be deceptive, but it is usually deliberate. We can also honestly make things up and present them as fact but it is not that common, we usually say that we don't know. And generally, lying is harder for us than telling the truth, in the sense that making a consistent but false narrative requires effort. For LLMs, making stuff up is the default, one can argue that it is all they do, it just happens to be the truth most of the times. And AFAIK, what I would call the "real" Turing test hasn't been shattered by far. The idea is that the interrogator and the human subject are both experts and collaborate against the computer. They can't cheat by exchanging secrets, but anything else is fair game. I think it is important because the Turing test has already been "won" by primitive algorithms acting clueless to interrogators who were not aware of the trick. For me, this is not really a measure of computer intelligence, more like a measure of how clever the chatbot designers were at tricking unsuspecting people.

13years 1y ago
> we usually say that we don't know I think this is one of the distinguishing attributes of human failures. Human failures have some degree of predictability. We know when we aren't good at something, we then devise processes to close that gap. Which can be consultations, training, process reviews, use of tools etc. The failures we see in LLMs are distinctly of a different nature. They often appear far more nonsensical and have more of a degree of randomness. The LLMs as a tool would be far more useful if they could indicate what they are good at, but since they cannot self reflect over their knowledge, it is not possible. So they are equally confident in everything regardless of its correctness.
- ragequittah 1y ago
  I think the last few years are a good example of how this isn't really true. Covid came around and everyone became an epidemiologist and public health expert. The people in charge of the US government right now are also a perfect example. RFK Jr. is going to get to the bottom of autism. Trump is ruining the world economy seemingly by himself. Hegseth is in charge of the most powerful military in the world. Humans pretending they know what they're doing is a giant problem.

nopelynopington 1y ago

> * Humans tell you how they think, but it seemingly is not how they really think. > * Humans tell you facts they believe to be true but are false. Think and believe are key words here. I'm not trying to be spiritual but LLMs do not think or believe a thing, they only predict the next word. > * Humans often need to be verified by another human and should not be trusted. You're talking about trusting another human to do that though, so you trust the human that is verifying.

deleted 1y ago
[deleted]

goatlover 1y ago

But there are places where humans do follow reasoning steps, such as arithmetic and logic. The fact that we need to add RLHF to models to make them more useful humans is also evidence that statistical reasoning is not enough for a general intelligence.

minraws 1y ago

LLMs aren't as good as average humans, most software folks like us like to believe rest of the world is dumb but it isn't. My grand-dad who only ever farmed land and had no education at all could talk, calculate, and manage his farm land. Was he not as good as me at anything academic yes. But I will never be as good at him at understanding how to farm stuff. Most people in who think LLMs are smart seem to conflate ignorance/lack of knowledge to being dumb. This is a rather reductive take, and no I don't believe that's how human intelligence works. Your dumb uncle on thanks giving who might even have a lot of bad traits isn't dumb likely just ignorant. All human IQ studies and movies have distorted our perception of intelligence. A more intelligent person doesn't necessarily need to have more or better quality knowledge. Or measuring them with academic abilities like writing and maths is the dumber/irresponsible take. And yes please feel free to call me dumb in response.