6 ms·

AI capability isn't humanness

52 points by mdahardy 10mo ago

somewhereoutth 10mo ago
Unfortunately a lot of the hype around LLMs is that their capability is humanness, specifically that they are (much) cheaper humans for replacing your expensive and annoying current humans.
- bitwize 10mo ago
  What I think you mean to say is that AI is promoted as fungible with humans at a lower price point.
  - skydhash 10mo ago
    I think that's the first time the C suite is so interested in having their employees using a tool regardless of the result. It likes prescribing that you have to send an email twice a day using outlook. And make sure to use attachment both time.
- cloflaw 10mo ago
  > that they are (much) cheaper humans This is literally their inhumanness.
yannyu 10mo ago
One thing I don't understand in these conversations is why we're treating LLMs as if they are completely interchangeable with chatbots/assistants. A car is not just an engine, it's a drivetrain, a transmission, wheels, steering, all of which affect the end-product and its usability. LLMs are no different, and focusing on alignment without even addressing all the scaffolding that intermediates the exchange between the user and the LLM in an assistant use case seems disingenuous.
ForceBru 10mo ago
> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs. I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given model saw during training. We know exactly how many images or video frames were used to train it, and so on. Can we count the amount of input humans receive? I can look at my coffee mug from any angle I want, I can feel it in my hands, I can sniff it, lick it and fiddle with it as much as I want. What happens if I move it away from me? Can I turn it this way, can I lift it up? What does it feel like to drink from this cup? What does it feel like when someone else drinks from my cup? The LLM has no idea because it doesn't have access to sensory data and it can't manipulate real-life objects (yet).
- cortesoft 10mo ago
  Not only that, but humans also have access to all of the "training data" of hundreds of millions of years of evolution baked into our brains.
  - ACCount37 10mo ago
    Which must be doing some heavy lifting. Humans ship with all the priors evolution has managed to cram into them. LLMs have to rediscover all of it from scratch just by looking at an awful lot of data.
    - hathawsh 10mo ago
      OTOH, all that data is built on patterns that evolved from many years of evolution, so I think the LLM benefits from that evolution also.
      - ACCount37 10mo ago
        Sure, but LLMs are trying to build the algorithms of the human mind backwards, converge on similar functionality based on just some of the inputs and outputs. This isn't an efficient or a lossless process. The fact that they can pull it off to this extent was a very surprising finding.
zkmon 10mo ago
I think there might be a slight bias in this blog article in favor of their product/service. Their human verification service probably needs AI to have less humanness. But as we saw over the course of recent months or years, AI outputs are becoming more indistinguishable for human output.
- mdahardy 10mo ago
  Our main argument is that outputs will become increasingly indistinguishable, but the processes won't. E.g. in 5 years if you watch an AI book a flight it will do it in a very non-human way, even if it gets the same flight you yourself would book.
  - layer8 10mo ago
    If the observable behavior (output) becomes indistinguishable (which I’m doubtful of), what does it matter that the internal process is different? Surely only to the extent that the behavior still exhibits differences after all?
  - erichocean 10mo ago
    > in 5 years if you watch an AI book a flight it will do it in a very non-human way I would bet completely against this, models are becoming more human-like, not less, over time. What's more likely to change (that would cause a difference) is the work itself changing to adapt to areas where models are already super-human, such as being able to read entire novels in seconds with full attention.
  - aisdijoa 10mo ago
    I was reading about your company and really liked your post about benchmarking bot detection systems (https://research.roundtable.ai/bot-benchmarking/ https://research.roundtable.ai/bot-benchmarking/). I find myself wondering why Google isn't doing as good a job as you guys. In particular, I'm wondering if you think they've been kind of lazy about the problem because they don't see captcha detection as faring particularly poorly, and perhaps once AI agents really start taking off and becoming prevalent, then Google will buckle down, at which point they'll be able to hone their troves of data to build a newer captcha even better than yours. Or is there an additional secret sauce you guys have? That kind of leads me to my other question-I assume your secret sauce is the cognitive science, stroop-esque approach to bot detection (which I think is brilliant in the abstract). But I'm curious how that scales. Like, do you go one-by-one for each website you have a partnership with and figure out how a human would likely use the site, or do you use general techniques (besides the obvious ones like typing speed and mouse movements, which almost certainly Google could implement too if it decided to buckle down). Like, can you scale some broad stroop task across websites? By the way, asking all of this out of admiration. You guys are doing really clever work!
gmuslera 10mo ago
LLMs are language models. We interact with them using language, all of that, but also only that. That doesn't mean that they have "common sense", context, same motivations, agency, or even reasoning like us. But as we interact with other people using mostly language, and since the start of internet a lot of those interactions happen in way similar to how we interact with AI, the difference is not so obvious. We are falling into the Turing test in this, mostly because that test is more about language than about intelligence.
- ACCount37 10mo ago
  "Language" is just the interface. What happens on the inside of LLMs is a lot weirder than that.
  - freejazz 10mo ago
    And?
  - lawlessone 10mo ago
    I feel like the interface in this case has caused us to fool ourselves into thinking there's more there than there is. Before 2022 (most of history), if you had a long seemingly sensible conversation with something, you could safely assume this other party was a real thinking human mind. it's like a duck call. edit, i want to add because this is neural net that's trained to output sensible text, language isn't just the interface. unlike a website there's no separation between anything, with LLM's the back and front end are all one blob. edit2: seems I have upset the ducks that think the duck call is a real duck .
  - gmuslera 10mo ago
    What matters is what happen in the outside. We don't know what happen in our inside (or the inside of others, at least), we know the language and how it is used, event the meanings don't have to be the same as long as it is consistent. And you get that by construction. Does that mean intelligence, self consciousness, soul or whatever? We only know that it walk like a duck and quacks like a duck.
acituan 10mo ago
Language is not humanness either; it is a disembodied artifact of our extended cognition, it is a way of transferring the contents of our consciousness to others or to ourselves over time. This is precisely what LLMs piggyback on and therefore are exceedingly good at simulating, which is why the accuracy of "is this human" tools are stuck at %60-70's (%50 is a coin flip), and are going to be bounded for a foreseeable future. And I am sorry to be negative but there is so much bad cognitive science in this article that I couldn't take the product seriously. > LLMs can be scaled almost arbitrarily in ways biological brains cannot: more parameters, more training compute, more depth. - Capacity of raw compute is irrelevant without mentioning the complexity of computation task at hand. LLM's can scale - not infinitely - but they solve for O(n^2) tasks. It is also amiss to think human compute = a singular human's head. Language itself is both a tool and protocol of distributed compute among humans. You borrow a lot of your symbolic preprocessing from culture! Like said, this is exactly what LLM's piggyback on. > We are constantly hit with a large, continuous stream of sensory input, but we cannot process or store more than a very small part of it. - This is called relevance, and we are so frigging good at it! The fact that machine has to deal with a lot more unprioritized data in a relatively flat O(n^2) problem formulation is a shortcoming, not a feature. Visual cortex is such an opinionated accelerator of processing all that massive data that only the relevant bits need to make to your consciousness. And this architecture was trained for hundreds of millions of years, over trillions of experiment arms - that were in parallel experimenting on everything else too. > Humans often have to act quickly. Deliberation is slow, so many decisions rely on fast, heuristic processing. In many situations (danger, social interaction, physical movement), waiting for more evidence simply isn't an option. - Again a lot of this equivocates conscious processing to entire cognition. Anyone who plays sports or music knows to respect the implicit, embodied cognition that goes on to achieve complex motor tasks. We are yet to see a non-massively-fast-forwarded household robot do a mundane kitchen cleaning task, and go play table tennis with the same motor "cortex". Motor planning and articulation is a fantastically complex computation; just because it doesn't make it to our consciousness or instrumented exclusively through language doesn't mean it is not. > Human thinking works in a slow, step-by-step way. We pay attention to only a few things at a time, and our memory is limited. - Thinking, Fast and Slow by Kahneman is a fantastic way of getting into how much more complex the mechanism is. The key point here is as limited in their recall, how good humans are at relevance, because it matters, because it is existential. Therefore when you are using a tool to extend your recall, it is important to see its limitations. Google search having indexed billions of pages is not a feature if it can't bring the top results well. If it gets the capability to sell me whatever it brought up was relevant, that still doesn't mean the results are actually relevant. And this is exactly the degradation of relevance we are seeing in our culture. I don't care if the language terminal is a human or a machine, if the human was convinced by the low relevance crap of the machine it just a legitimacy laundering scheme. Therefore this is not a tech problem, it is a problem of culture; we need to be simultaneously cultivating epistemic humility, including quitting the Cartesian tyranny of worshipping explicit verbal cognition that is assumed to be locked up in a brain; we have to accept that we are also embodied and social beings that depend on a lot of distributed compute to solve for agency.
xg15 10mo ago
> LLMs process information very differently. They look at everything in parallel, all at once, and can use the whole context in one shot. Their “memory” is stored across billions of tiny weights, and they retrieve information by matching patterns, not by searching through memories like we do. Researchers have shown that LLMs automatically learn specific little algorithms (like copying patterns or doing simple lookups), all powered by huge matrix multiplications running in parallel rather than slow, step-by-step reasoning. I think this is incorrect on two accounts: Yes, transformers and individual layers are parallel, but the entire network is not. On a first level, it's obviously sequential over generated tokens - but even generation of a single token is sequential in the number of layers that the information travels through. Both those constraints are comparable to the way humans think I believe. (The human brain doesn't have neatly organized layers, but it does have "pathways" where certain brain regions project into other brain regions)
skybrian 10mo ago
I like "ghosts" as a simple metaphor for what you chat with when you chat with AI. Usually we chat with Casper the Friendly Ghost, but there are a lot of other ghosts that can be conjured up. Some people are obsessed with chatting with ghosts. It seems like a rational adult couldn't be seriously harmed by chatting with a ghost, but there are news reports showing that some people get possessed. It's a better metaphor than parrots, anyway. more: https://karpathy.bearblog.dev/animals-vs-ghosts/ https://karpathy.bearblog.dev/animals-vs-ghosts/
jeisc 10mo ago
The human condition is not ascertained through language; it is only expressed through language so truly understandable only to someone who also shares the human condition which certainly is not the case of any LLM.

52 points by mdahardy 10mo ago

Unfortunately a lot of the hype around LLMs is that their capability is humanness, specifically that they are (much) cheaper humans for replacing your expensive and annoying current humans.

bitwize 10mo ago
What I think you mean to say is that AI is promoted as fungible with humans at a lower price point.
- skydhash 10mo ago
  I think that's the first time the C suite is so interested in having their employees using a tool regardless of the result. It likes prescribing that you have to send an email twice a day using outlook. And make sure to use attachment both time.
cloflaw 10mo ago
> that they are (much) cheaper humans This is literally their inhumanness.

yannyu 10mo ago

One thing I don't understand in these conversations is why we're treating LLMs as if they are completely interchangeable with chatbots/assistants. A car is not just an engine, it's a drivetrain, a transmission, wheels, steering, all of which affect the end-product and its usability. LLMs are no different, and focusing on alignment without even addressing all the scaffolding that intermediates the exchange between the user and the LLM in an assistant use case seems disingenuous.

ForceBru 10mo ago

> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs. I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given model saw during training. We know exactly how many images or video frames were used to train it, and so on. Can we count the amount of input humans receive? I can look at my coffee mug from any angle I want, I can feel it in my hands, I can sniff it, lick it and fiddle with it as much as I want. What happens if I move it away from me? Can I turn it this way, can I lift it up? What does it feel like to drink from this cup? What does it feel like when someone else drinks from my cup? The LLM has no idea because it doesn't have access to sensory data and it can't manipulate real-life objects (yet).

cortesoft 10mo ago
Not only that, but humans also have access to all of the "training data" of hundreds of millions of years of evolution baked into our brains.
- ACCount37 10mo ago
  Which must be doing some heavy lifting. Humans ship with all the priors evolution has managed to cram into them. LLMs have to rediscover all of it from scratch just by looking at an awful lot of data.
  - hathawsh 10mo ago
    OTOH, all that data is built on patterns that evolved from many years of evolution, so I think the LLM benefits from that evolution also.
    - ACCount37 10mo ago
      Sure, but LLMs are trying to build the algorithms of the human mind backwards, converge on similar functionality based on just some of the inputs and outputs. This isn't an efficient or a lossless process. The fact that they can pull it off to this extent was a very surprising finding.

zkmon 10mo ago

I think there might be a slight bias in this blog article in favor of their product/service. Their human verification service probably needs AI to have less humanness. But as we saw over the course of recent months or years, AI outputs are becoming more indistinguishable for human output.

mdahardy 10mo ago
Our main argument is that outputs will become increasingly indistinguishable, but the processes won't. E.g. in 5 years if you watch an AI book a flight it will do it in a very non-human way, even if it gets the same flight you yourself would book.
- layer8 10mo ago
  If the observable behavior (output) becomes indistinguishable (which I’m doubtful of), what does it matter that the internal process is different? Surely only to the extent that the behavior still exhibits differences after all?
- erichocean 10mo ago
  > in 5 years if you watch an AI book a flight it will do it in a very non-human way I would bet completely against this, models are becoming more human-like, not less, over time. What's more likely to change (that would cause a difference) is the work itself changing to adapt to areas where models are already super-human, such as being able to read entire novels in seconds with full attention.
- aisdijoa 10mo ago
  I was reading about your company and really liked your post about benchmarking bot detection systems (https://research.roundtable.ai/bot-benchmarking/ https://research.roundtable.ai/bot-benchmarking/). I find myself wondering why Google isn't doing as good a job as you guys. In particular, I'm wondering if you think they've been kind of lazy about the problem because they don't see captcha detection as faring particularly poorly, and perhaps once AI agents really start taking off and becoming prevalent, then Google will buckle down, at which point they'll be able to hone their troves of data to build a newer captcha even better than yours. Or is there an additional secret sauce you guys have? That kind of leads me to my other question-I assume your secret sauce is the cognitive science, stroop-esque approach to bot detection (which I think is brilliant in the abstract). But I'm curious how that scales. Like, do you go one-by-one for each website you have a partnership with and figure out how a human would likely use the site, or do you use general techniques (besides the obvious ones like typing speed and mouse movements, which almost certainly Google could implement too if it decided to buckle down). Like, can you scale some broad stroop task across websites? By the way, asking all of this out of admiration. You guys are doing really clever work!

gmuslera 10mo ago

LLMs are language models. We interact with them using language, all of that, but also only that. That doesn't mean that they have "common sense", context, same motivations, agency, or even reasoning like us. But as we interact with other people using mostly language, and since the start of internet a lot of those interactions happen in way similar to how we interact with AI, the difference is not so obvious. We are falling into the Turing test in this, mostly because that test is more about language than about intelligence.

ACCount37 10mo ago
"Language" is just the interface. What happens on the inside of LLMs is a lot weirder than that.
- freejazz 10mo ago
  And?
- lawlessone 10mo ago
  I feel like the interface in this case has caused us to fool ourselves into thinking there's more there than there is. Before 2022 (most of history), if you had a long seemingly sensible conversation with something, you could safely assume this other party was a real thinking human mind. it's like a duck call. edit, i want to add because this is neural net that's trained to output sensible text, language isn't just the interface. unlike a website there's no separation between anything, with LLM's the back and front end are all one blob. edit2: seems I have upset the ducks that think the duck call is a real duck .
- gmuslera 10mo ago
  What matters is what happen in the outside. We don't know what happen in our inside (or the inside of others, at least), we know the language and how it is used, event the meanings don't have to be the same as long as it is consistent. And you get that by construction. Does that mean intelligence, self consciousness, soul or whatever? We only know that it walk like a duck and quacks like a duck.

acituan 10mo ago

Language is not humanness either; it is a disembodied artifact of our extended cognition, it is a way of transferring the contents of our consciousness to others or to ourselves over time. This is precisely what LLMs piggyback on and therefore are exceedingly good at simulating, which is why the accuracy of "is this human" tools are stuck at %60-70's (%50 is a coin flip), and are going to be bounded for a foreseeable future. And I am sorry to be negative but there is so much bad cognitive science in this article that I couldn't take the product seriously. > LLMs can be scaled almost arbitrarily in ways biological brains cannot: more parameters, more training compute, more depth. - Capacity of raw compute is irrelevant without mentioning the complexity of computation task at hand. LLM's can scale - not infinitely - but they solve for O(n^2) tasks. It is also amiss to think human compute = a singular human's head. Language itself is both a tool and protocol of distributed compute among humans. You borrow a lot of your symbolic preprocessing from culture! Like said, this is exactly what LLM's piggyback on. > We are constantly hit with a large, continuous stream of sensory input, but we cannot process or store more than a very small part of it. - This is called relevance, and we are so frigging good at it! The fact that machine has to deal with a lot more unprioritized data in a relatively flat O(n^2) problem formulation is a shortcoming, not a feature. Visual cortex is such an opinionated accelerator of processing all that massive data that only the relevant bits need to make to your consciousness. And this architecture was trained for hundreds of millions of years, over trillions of experiment arms - that were in parallel experimenting on everything else too. > Humans often have to act quickly. Deliberation is slow, so many decisions rely on fast, heuristic processing. In many situations (danger, social interaction, physical movement), waiting for more evidence simply isn't an option. - Again a lot of this equivocates conscious processing to entire cognition. Anyone who plays sports or music knows to respect the implicit, embodied cognition that goes on to achieve complex motor tasks. We are yet to see a non-massively-fast-forwarded household robot do a mundane kitchen cleaning task, and go play table tennis with the same motor "cortex". Motor planning and articulation is a fantastically complex computation; just because it doesn't make it to our consciousness or instrumented exclusively through language doesn't mean it is not. > Human thinking works in a slow, step-by-step way. We pay attention to only a few things at a time, and our memory is limited. - Thinking, Fast and Slow by Kahneman is a fantastic way of getting into how much more complex the mechanism is. The key point here is as limited in their recall, how good humans are at relevance, because it matters, because it is existential. Therefore when you are using a tool to extend your recall, it is important to see its limitations. Google search having indexed billions of pages is not a feature if it can't bring the top results well. If it gets the capability to sell me whatever it brought up was relevant, that still doesn't mean the results are actually relevant. And this is exactly the degradation of relevance we are seeing in our culture. I don't care if the language terminal is a human or a machine, if the human was convinced by the low relevance crap of the machine it just a legitimacy laundering scheme. Therefore this is not a tech problem, it is a problem of culture; we need to be simultaneously cultivating epistemic humility, including quitting the Cartesian tyranny of worshipping explicit verbal cognition that is assumed to be locked up in a brain; we have to accept that we are also embodied and social beings that depend on a lot of distributed compute to solve for agency.

xg15 10mo ago

> LLMs process information very differently. They look at everything in parallel, all at once, and can use the whole context in one shot. Their “memory” is stored across billions of tiny weights, and they retrieve information by matching patterns, not by searching through memories like we do. Researchers have shown that LLMs automatically learn specific little algorithms (like copying patterns or doing simple lookups), all powered by huge matrix multiplications running in parallel rather than slow, step-by-step reasoning. I think this is incorrect on two accounts: Yes, transformers and individual layers are parallel, but the entire network is not. On a first level, it's obviously sequential over generated tokens - but even generation of a single token is sequential in the number of layers that the information travels through. Both those constraints are comparable to the way humans think I believe. (The human brain doesn't have neatly organized layers, but it does have "pathways" where certain brain regions project into other brain regions)

skybrian 10mo ago

I like "ghosts" as a simple metaphor for what you chat with when you chat with AI. Usually we chat with Casper the Friendly Ghost, but there are a lot of other ghosts that can be conjured up. Some people are obsessed with chatting with ghosts. It seems like a rational adult couldn't be seriously harmed by chatting with a ghost, but there are news reports showing that some people get possessed. It's a better metaphor than parrots, anyway. more: https://karpathy.bearblog.dev/animals-vs-ghosts/ https://karpathy.bearblog.dev/animals-vs-ghosts/

jeisc 10mo ago

The human condition is not ascertained through language; it is only expressed through language so truly understandable only to someone who also shares the human condition which certainly is not the case of any LLM.