7 ms·

In my view 'understand' is a folk psychology term that does not have a technical meaning. Like 'intelligent', 'beautiful', and 'interesting'. It usefully labels

In my view 'understand' is a folk psychology term that does not have a technical meaning. Like 'intelligent', 'beautiful', and 'interesting'. It usefully labels a basket of behaviors we see in others, and that is all it does. In this view, if a machine performs a task as well as a human, it understands it exactly as much as a human. There's no problem of how to do understanding, only how to do tasks. The 'problem' melts away when you take this stance. Just my opinion, but my professional opinion from thirty-plus years in AI.

compass_copium 1y ago
Nonsense. A QC operator may be able to carry out a test with as much accuracy (or perhaps better accuracy, with enough practice) than the PhD quality chemist who developed it. They could plausibly do so with a high school education and not be able to explain the test in any detail. They do not understand the test in the same way as the chemist. If 'understand' is a meaningless term to someone who's spent 30 years in AI research, I understand why LLMs are being sold and hyped in the way they are.
- robotresearcher 1y ago
  > They do not understand the test in the same way as the chemist. Can you explain precisely what 'understand' means here, without using the word 'understand'? I don't think anyone can.
  - throw4847285 1y ago
    There are a number of competing models. The SEP page is probably a good place to start. https://plato.stanford.edu/entries/understanding https://plato.stanford.edu/entries/understanding
  - bandrami 1y ago
    Not to be flippant but have you considered that that question is an entire branch of philosophy with a several-millennias long history which people in some cases spend their entire life studying?
    - robotresearcher 1y ago
      I have. It robustly has the folk-psychological meaning I mentioned in my first sentence. Call it ‘philosophical’ instead of ‘folk-psychological’ if you like. It’s a useful concept. But the concept doesn’t require AI engineers to do anything. It certainly doesn’t give any hints about AI engineers what they should actually do. “Make it understand.” “How? What does that look like?” “… But it needs to understand…” “It answers your questions.” “But it doesn’t understand.” “Ok. Get back to me when that entails anything.”
      - mommys_little 1y ago
        I would say it understands if given many variations of a problem statement, it always gives correct answer without fail. I have this complicated mirror question that only Deepseek and qwen3-max got right every time, still they only answered it correctly about a dozen times, so we're left with high probability, I guess.
        deleted 1y ago
        [deleted]
        godelski 1y ago
        I disagree with robotresearcher but I think this is also an absurd definition. By that definition there is no human, nor creature, that understands anything. Not just by nature of humans making mistakes, including experts, but I'd say this is even impossible. You need infinite precision and infinite variation here. It turns "understanding" into a binary condition. Robotresearcher's does too, but I'm sure they would refine by saying that the level of understanding is directly proportional to task performance. But I still don't know how they'll address the issue of coverage, as ensuring tests have complete coverage is far from trivial (even harder when you want to differentiate from the training set, differentiating memorization). I think you're right in trying to differentiate memorization from generalization, but your way to measure this is not robust enough. A fundamental characteristic of where I disagree from them is that memorization is not the same as understanding.
        1y ago
- godelski 1y ago
  > If 'understand' is a meaningless term to someone who's spent 30 years in AI research, I understand why LLMs are being sold and hyped in the way they are. I don't have quite as much time as robotresearcher, but I've heard their sentiment frequently. I've been to conferences, talked with people at the top of the field (I'm "junior", but published and have a PhD) where when asking deeper questions I'll get a frequent response "I just care if it works." As if that also wasn't the motivation for my questions too. But I'll also tell you that there are plenty of us who don't ascribe to those beliefs. There's a wide breadth of opinions, even if one set is large and loud. (We are getting louder though) I do think we can get to AGI and I do think we can figure out what words like "understand" truly mean (with both accuracy and precision, the latter being what's more lacking). But it is also hard to navigate because we're discouraged from this work and little funding flows our way (I hope as we get louder we'll be able to explore more, but I fear we may switch from one railroad to the next). The weirdest part to me has been that it seems that even in the research space, talking to peers, that discussing flaws or limits is treated as dismissal. I thought our whole job was to find the limits, explore them, and find ways to resolve them. The way I see it now is that the field uses the duck test. If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck. The problem is people are replacing "probably" with "is". The duck test is great, and right now we don't have anything much better. But the part that is insane is to call it perfect. Certainly as someone who isn't an ornithologist, I'm not going to be able to tell a sophisticated artificial duck from a real one. But it's ability to fool me doesn't make it real. And that's exactly why it would be foolish to s/probably/is. So while I think you're understanding correctly, I just want to caution throwing the baby out with the bathwater. The majority of us dissenting from the hype train and "scale is all you need" don't believe humans are magic and operating outside the laws of physics. Unless this is a false assumption, artificial life is certainly possible. The question is just about when and how. I think we still have a ways to go. I think we should be exploring a wide breadth of ideas. I just don't think we should put all our eggs in one basket, especially if there's clear holes in it. [Side note]: An interesting relationship I've noticed is that the hype train people tend to have a full CS pedigree while dissenters have mixed (and typically start in something like math or physics and make their way to CS). It's a weak correlation, but I've found it interesting.
  - 1y ago
- pennaMan 1y ago
  so your definition of "understand" is "able to develop the QC test (or explain tests already developed)" I hate to break it to you, but the LLMs can already do all 3 tasks you outlined It can be argued for all 3 actors in this example (the QC operator, the PhD chemist and the LLM) that they don't really "understand" anything and are iterating on pre-learned patterns in order to complete the tasks. Even the ground-breaking chemist researcher developing a new test can be reduced to iterating on the memorized fundamentals of chemistry using a lot of compute (of the meat kind). The mythical Understanding is just a form of "no true Scotsman"
lelandbatey 1y ago
> if a machine performs a task as well as a human, it understands it exactly as much as a human. I think you're right, except that the ones judging "as well as a human" are in fact humans, and humans have expectations that expand beyond the specs. From the narrow perspective of engineering specifications or profit generated, a robot/AI may very well be exactly as understanding as a human. For the people which interact with those systems outside the money/specs/speeds & feeds, the AI/robot will always feel at least different compared to a person. And as long as it's different, there will always be room to un-falsifiably claim "this robot is worse in my opinion due to X/Y/Z difference."
- subjectivationx 1y ago
  This is all nonsense. It is like saying the airplane understands how to fly. "You disagree? Well lets see you fly! You are saying the airplane doesn't understand how to fly and you can't even fly yourself?" This would be confusing the fact humans built the flying machine and the flying machine doesn't understand anything.
  - robotresearcher 1y ago
    Right. A flying machine doesn’t need to understand anything to fly. It’s not even clear what it would mean for it to do so, or how it would fly any differently if it did. Same with the AI machines. Understanding is not something that any machine or person does. Understanding is a compact label applied to people’s behavior by an observer that allows the observer to predict future behavior. It’s not a process in itself. And yes, we apply this label to ourselves. Much of what we do is only available to consciousness post-hoc, and is available to be described just the same as the behavior of someone else.
    - godelski 1y ago
      > Understanding is not something that any machine or person does. Yet I can write down many equations necessary to build and design that plane. I can model the wind and air flow across the surface and design airfoils. I can interpret the mathematical symbols into real physical meaning. I can adapt these equations to novel settings or even fictitious ones. I can analyze them counterfactually; not just making predictions but also telling you why those predictions are accurate, what their inaccuracies are (such as which variables and measurements are more precise), and I can tell you what all those things mean. I can describe and derive the limits of the equations and models, discussing where they do and don't work. Including in the fictional settings. I can do this at an emergent macroscopic level and I can do it at a fine grain molecular or even atomic level. I can even derive the emergent macroscopic behavior from the more fine grain analysis and tell you the limits of each model. I can also respond that Bernoulli's equation is not an accurate description of why an airfoil works, even when prompted with those words[0]. These are characteristics that lead people to believe I understand the physics of fluid mechanics and flight. They correlate strongly with the ability to recall information from textbooks, but the actions aren't strictly the ability to recall and search over a memory database. Do these things prove that I understand? No, but we deal with what we got even if it is imperfect. It is not just the ability to perform a task, it includes the ability to explain it. The more depth I am able to the greater understanding people attribute. While this correlates with task performance it is not the same. Even Ramanujan had to work hard to understand even if he was somehow able to divine great equations without it. You're right that these descriptions are not the thing itself either. No one is claiming the map is the territory here. That's not the argument being made. Understanding the map is a very different thing than conflating the map and the territory. It is also a different thing than just being able to read it. [0] https://x.com/BethMayBarnes/status/1953504663531388985 https://x.com/BethMayBarnes/status/1953504663531388985
godelski 1y ago
> that does not have a technical meaning I don't think the definition is very refined, but I think we should be careful to differentiate that from useless or meaningless. I would say most definitions are accurate, but not precise. It's a hard problem, but we are making progress on it. We will probably get there, but it's going to end up being very nuanced and already it is important to recognize that the word means different things in vernacular and in even differing research domains. Words are overloaded and I think we need to recognize this divergence and that we are gravely miscommunicating by assuming the definitions are obvious. I'm not sure why we don't do more to work together on this. In our field we seem to think we got it all covered and don't need others. I don't get that. > In this view, if a machine performs a task as well as a human, it understands it exactly as much as a human. And I do not think this is accurate at all. I would not say my calculator understands math despite it being able to do it better than me. I can say the same thing about a lot of different things which we don't attribute intelligence to. I'm sorry, but the logic doesn't hold. Okay, you might take an out by saying the calculator can't do abstract math like I can, right? Well we're going to run into that same problem. You can't test your way out of it. We've known this in hard sciences like physics for centuries. It's why physicists do much more than just experiments. There's the classic story of Freeman Dyson speaking to Fermi, which is why so many know about the 4 parameter elephant[0], but it is also just repeated through our history of physics. Guess what? Dyson's experiments worked. They fit the model. They were accurate and made accurate predictions! Yet they were not correct. People didn't reject Galileo just because the church, there were serious problems with his work too. Geocentricism made accurate predictions, including ones that Galileo's version of Heliocentrism couldn't. These historical misunderstandings are quite common, including things like how the average person understands things like Schrodinger's Cat. The cat isn't in a parallel universe of both dead and alive lol. It's just that we, outside the box can't determine which. Oh, no, information is lossy, there's injective functions, the universe could then still be deterministic yet we wouldn't be able to determine that (and my name comes into play). So idk, it seems like you're just oversimplifying as a means to sidestep the hard problem[1]. The lack of a good technical definition of understanding should tell us we need to determine one. It's obviously a hard thing to do since, well... we don't have one and people have been trying to solve it for thousands of years lol. > Just my opinion, but my professional opinion from thirty-plus years in AI. Maybe I don't have as many years as you, but I do have a PhD in CS (thesis on neural networks) and a degree in physics. I think it certainly qualifies as a professional opinion. But at the end of the day it isn't our pedigree that makes us right or wrong. [0] https://www.youtube.com/watch?v=hV41QEKiMlM https://www.youtube.com/watch?v=hV41QEKiMlM [1] I'm perfectly fine tabling a hard problem and focusing on what's more approachable right now, but that's a different thing. We may follow a similar trajectory but I'm not going to say the path we didn't take is just an illusion. I'm not going to discourage others from trying to navigate it either. I'm just prioritizing. If they prove you right, then that's a nice feather in your hat, but I doubt it since people have tried that definition from the get go.
dullcrisp 1y ago
So my toaster understands toast and I don’t understand toast? Then why am I operating the toaster and not the other way around?
- simondotau 1y ago
  A toaster cannot perform the task of making toast any more than an Allen key can perform the task of assembling flat pack furniture.
  - dullcrisp 1y ago
    This is contrary to my experience with toasters, but it doesn’t seem worth arguing about.
    - jrflowers 1y ago
      How does your toaster get the bread on its own?
      - recursive 1y ago
        How do you get bread? Don't tell me you got it at the market. That's just paying someone else to get it for you.
        godelski 1y ago
        > That's just paying someone else to get it for you. We can automate that too![0] [0] https://news.ycombinator.com/item?id=45623154 https://news.ycombinator.com/item?id=45623154 (Your name is quite serendipitous to this conversation)
      - dullcrisp 1y ago
        It’s only responsible for the toasting part. The bread machine makes the bread.
ssivark 1y ago
> In this view, if a machine performs a task as well as a human, it understands it exactly as much as a human. There's no problem of how to do understanding, only how to do tasks. Yes, but you also gloss over what a "task" is or what a "benchmark" is (which has to do with the meaning of generalization). Suppose an AI or human answers 7 questions correctly out of 10 on an ICPC problem set, what are we able infer from that? 1. Is the task equal to answering these 10 questions well, with a uniform measure of importance? 2. Is the task be good at competitive programming problems? 3. Is the task be good at coding? 4. Is the task be good at problem solving? 5. Is the task not just to be effective under a uniform measure of importance, but an adversarial measure? (i.e. you can probably figure out all kinds of competitive programming questions, if you had more time / etc... but roughly not needing "exponentially more resources") These are very different levels of abstraction, and literally the same benchmark result can be interpreted to mean very different things. And that imputation of generality is not objective unless we know the mechanism by which it happens. "Understanding" is short-hand for saying that performance generalizes at one of the higher levels of abstraction (3--5), rather than narrow success -- because that is what we expect of a human.
- simianwords 1y ago
  How do you quantify generality? If we have a benchmark that can quantify it and that benchmark reliably tells us that the LLM is within human levels of generalisation then the llm is not distinguishable from a human. While it’s a good point that we need to benchmark generalisation ability, you have in fact agreed that it is not important to understand underlying mechanics.
  - godelski 1y ago
    That's kinda their point The difference though is they understand that you can't just benchmark your way into proofs. Just like you can't unit test your way into showing code is error free. Benchmarks and unit tests are great tools that provide a lot of help, but just because a hammer is useful doesn't make everything a nail.
deleted 1y ago
[deleted]

compass_copium 1y ago

Nonsense. A QC operator may be able to carry out a test with as much accuracy (or perhaps better accuracy, with enough practice) than the PhD quality chemist who developed it. They could plausibly do so with a high school education and not be able to explain the test in any detail. They do not understand the test in the same way as the chemist. If 'understand' is a meaningless term to someone who's spent 30 years in AI research, I understand why LLMs are being sold and hyped in the way they are.

robotresearcher 1y ago
> They do not understand the test in the same way as the chemist. Can you explain precisely what 'understand' means here, without using the word 'understand'? I don't think anyone can.
- throw4847285 1y ago
  There are a number of competing models. The SEP page is probably a good place to start. https://plato.stanford.edu/entries/understanding https://plato.stanford.edu/entries/understanding
- bandrami 1y ago
  Not to be flippant but have you considered that that question is an entire branch of philosophy with a several-millennias long history which people in some cases spend their entire life studying?
  - robotresearcher 1y ago
    I have. It robustly has the folk-psychological meaning I mentioned in my first sentence. Call it ‘philosophical’ instead of ‘folk-psychological’ if you like. It’s a useful concept. But the concept doesn’t require AI engineers to do anything. It certainly doesn’t give any hints about AI engineers what they should actually do. “Make it understand.” “How? What does that look like?” “… But it needs to understand…” “It answers your questions.” “But it doesn’t understand.” “Ok. Get back to me when that entails anything.”
    - mommys_little 1y ago
      I would say it understands if given many variations of a problem statement, it always gives correct answer without fail. I have this complicated mirror question that only Deepseek and qwen3-max got right every time, still they only answered it correctly about a dozen times, so we're left with high probability, I guess.
      - deleted 1y ago
        [deleted]
      - godelski 1y ago
        I disagree with robotresearcher but I think this is also an absurd definition. By that definition there is no human, nor creature, that understands anything. Not just by nature of humans making mistakes, including experts, but I'd say this is even impossible. You need infinite precision and infinite variation here. It turns "understanding" into a binary condition. Robotresearcher's does too, but I'm sure they would refine by saying that the level of understanding is directly proportional to task performance. But I still don't know how they'll address the issue of coverage, as ensuring tests have complete coverage is far from trivial (even harder when you want to differentiate from the training set, differentiating memorization). I think you're right in trying to differentiate memorization from generalization, but your way to measure this is not robust enough. A fundamental characteristic of where I disagree from them is that memorization is not the same as understanding.
      - 1y ago
godelski 1y ago
> If 'understand' is a meaningless term to someone who's spent 30 years in AI research, I understand why LLMs are being sold and hyped in the way they are. I don't have quite as much time as robotresearcher, but I've heard their sentiment frequently. I've been to conferences, talked with people at the top of the field (I'm "junior", but published and have a PhD) where when asking deeper questions I'll get a frequent response "I just care if it works." As if that also wasn't the motivation for my questions too. But I'll also tell you that there are plenty of us who don't ascribe to those beliefs. There's a wide breadth of opinions, even if one set is large and loud. (We are getting louder though) I do think we can get to AGI and I do think we can figure out what words like "understand" truly mean (with both accuracy and precision, the latter being what's more lacking). But it is also hard to navigate because we're discouraged from this work and little funding flows our way (I hope as we get louder we'll be able to explore more, but I fear we may switch from one railroad to the next). The weirdest part to me has been that it seems that even in the research space, talking to peers, that discussing flaws or limits is treated as dismissal. I thought our whole job was to find the limits, explore them, and find ways to resolve them. The way I see it now is that the field uses the duck test. If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck. The problem is people are replacing "probably" with "is". The duck test is great, and right now we don't have anything much better. But the part that is insane is to call it perfect. Certainly as someone who isn't an ornithologist, I'm not going to be able to tell a sophisticated artificial duck from a real one. But it's ability to fool me doesn't make it real. And that's exactly why it would be foolish to s/probably/is. So while I think you're understanding correctly, I just want to caution throwing the baby out with the bathwater. The majority of us dissenting from the hype train and "scale is all you need" don't believe humans are magic and operating outside the laws of physics. Unless this is a false assumption, artificial life is certainly possible. The question is just about when and how. I think we still have a ways to go. I think we should be exploring a wide breadth of ideas. I just don't think we should put all our eggs in one basket, especially if there's clear holes in it. [Side note]: An interesting relationship I've noticed is that the hype train people tend to have a full CS pedigree while dissenters have mixed (and typically start in something like math or physics and make their way to CS). It's a weak correlation, but I've found it interesting.
- 1y ago
pennaMan 1y ago
so your definition of "understand" is "able to develop the QC test (or explain tests already developed)" I hate to break it to you, but the LLMs can already do all 3 tasks you outlined It can be argued for all 3 actors in this example (the QC operator, the PhD chemist and the LLM) that they don't really "understand" anything and are iterating on pre-learned patterns in order to complete the tasks. Even the ground-breaking chemist researcher developing a new test can be reduced to iterating on the memorized fundamentals of chemistry using a lot of compute (of the meat kind). The mythical Understanding is just a form of "no true Scotsman"

lelandbatey 1y ago

> if a machine performs a task as well as a human, it understands it exactly as much as a human. I think you're right, except that the ones judging "as well as a human" are in fact humans, and humans have expectations that expand beyond the specs. From the narrow perspective of engineering specifications or profit generated, a robot/AI may very well be exactly as understanding as a human. For the people which interact with those systems outside the money/specs/speeds & feeds, the AI/robot will always feel at least different compared to a person. And as long as it's different, there will always be room to un-falsifiably claim "this robot is worse in my opinion due to X/Y/Z difference."

subjectivationx 1y ago
This is all nonsense. It is like saying the airplane understands how to fly. "You disagree? Well lets see you fly! You are saying the airplane doesn't understand how to fly and you can't even fly yourself?" This would be confusing the fact humans built the flying machine and the flying machine doesn't understand anything.
- robotresearcher 1y ago
  Right. A flying machine doesn’t need to understand anything to fly. It’s not even clear what it would mean for it to do so, or how it would fly any differently if it did. Same with the AI machines. Understanding is not something that any machine or person does. Understanding is a compact label applied to people’s behavior by an observer that allows the observer to predict future behavior. It’s not a process in itself. And yes, we apply this label to ourselves. Much of what we do is only available to consciousness post-hoc, and is available to be described just the same as the behavior of someone else.
  - godelski 1y ago
    > Understanding is not something that any machine or person does. Yet I can write down many equations necessary to build and design that plane. I can model the wind and air flow across the surface and design airfoils. I can interpret the mathematical symbols into real physical meaning. I can adapt these equations to novel settings or even fictitious ones. I can analyze them counterfactually; not just making predictions but also telling you why those predictions are accurate, what their inaccuracies are (such as which variables and measurements are more precise), and I can tell you what all those things mean. I can describe and derive the limits of the equations and models, discussing where they do and don't work. Including in the fictional settings. I can do this at an emergent macroscopic level and I can do it at a fine grain molecular or even atomic level. I can even derive the emergent macroscopic behavior from the more fine grain analysis and tell you the limits of each model. I can also respond that Bernoulli's equation is not an accurate description of why an airfoil works, even when prompted with those words[0]. These are characteristics that lead people to believe I understand the physics of fluid mechanics and flight. They correlate strongly with the ability to recall information from textbooks, but the actions aren't strictly the ability to recall and search over a memory database. Do these things prove that I understand? No, but we deal with what we got even if it is imperfect. It is not just the ability to perform a task, it includes the ability to explain it. The more depth I am able to the greater understanding people attribute. While this correlates with task performance it is not the same. Even Ramanujan had to work hard to understand even if he was somehow able to divine great equations without it. You're right that these descriptions are not the thing itself either. No one is claiming the map is the territory here. That's not the argument being made. Understanding the map is a very different thing than conflating the map and the territory. It is also a different thing than just being able to read it. [0] https://x.com/BethMayBarnes/status/1953504663531388985 https://x.com/BethMayBarnes/status/1953504663531388985

godelski 1y ago

> that does not have a technical meaning I don't think the definition is very refined, but I think we should be careful to differentiate that from useless or meaningless. I would say most definitions are accurate, but not precise. It's a hard problem, but we are making progress on it. We will probably get there, but it's going to end up being very nuanced and already it is important to recognize that the word means different things in vernacular and in even differing research domains. Words are overloaded and I think we need to recognize this divergence and that we are gravely miscommunicating by assuming the definitions are obvious. I'm not sure why we don't do more to work together on this. In our field we seem to think we got it all covered and don't need others. I don't get that. > In this view, if a machine performs a task as well as a human, it understands it exactly as much as a human. And I do not think this is accurate at all. I would not say my calculator understands math despite it being able to do it better than me. I can say the same thing about a lot of different things which we don't attribute intelligence to. I'm sorry, but the logic doesn't hold. Okay, you might take an out by saying the calculator can't do abstract math like I can, right? Well we're going to run into that same problem. You can't test your way out of it. We've known this in hard sciences like physics for centuries. It's why physicists do much more than just experiments. There's the classic story of Freeman Dyson speaking to Fermi, which is why so many know about the 4 parameter elephant[0], but it is also just repeated through our history of physics. Guess what? Dyson's experiments worked. They fit the model. They were accurate and made accurate predictions! Yet they were not correct. People didn't reject Galileo just because the church, there were serious problems with his work too. Geocentricism made accurate predictions, including ones that Galileo's version of Heliocentrism couldn't. These historical misunderstandings are quite common, including things like how the average person understands things like Schrodinger's Cat. The cat isn't in a parallel universe of both dead and alive lol. It's just that we, outside the box can't determine which. Oh, no, information is lossy, there's injective functions, the universe could then still be deterministic yet we wouldn't be able to determine that (and my name comes into play). So idk, it seems like you're just oversimplifying as a means to sidestep the hard problem[1]. The lack of a good technical definition of understanding should tell us we need to determine one. It's obviously a hard thing to do since, well... we don't have one and people have been trying to solve it for thousands of years lol. > Just my opinion, but my professional opinion from thirty-plus years in AI. Maybe I don't have as many years as you, but I do have a PhD in CS (thesis on neural networks) and a degree in physics. I think it certainly qualifies as a professional opinion. But at the end of the day it isn't our pedigree that makes us right or wrong. [0] https://www.youtube.com/watch?v=hV41QEKiMlM https://www.youtube.com/watch?v=hV41QEKiMlM [1] I'm perfectly fine tabling a hard problem and focusing on what's more approachable right now, but that's a different thing. We may follow a similar trajectory but I'm not going to say the path we didn't take is just an illusion. I'm not going to discourage others from trying to navigate it either. I'm just prioritizing. If they prove you right, then that's a nice feather in your hat, but I doubt it since people have tried that definition from the get go.

dullcrisp 1y ago

So my toaster understands toast and I don’t understand toast? Then why am I operating the toaster and not the other way around?

simondotau 1y ago
A toaster cannot perform the task of making toast any more than an Allen key can perform the task of assembling flat pack furniture.
- dullcrisp 1y ago
  This is contrary to my experience with toasters, but it doesn’t seem worth arguing about.
  - jrflowers 1y ago
    How does your toaster get the bread on its own?
    - recursive 1y ago
      How do you get bread? Don't tell me you got it at the market. That's just paying someone else to get it for you.
      - godelski 1y ago
        > That's just paying someone else to get it for you. We can automate that too![0] [0] https://news.ycombinator.com/item?id=45623154 https://news.ycombinator.com/item?id=45623154 (Your name is quite serendipitous to this conversation)
    - dullcrisp 1y ago
      It’s only responsible for the toasting part. The bread machine makes the bread.

ssivark 1y ago

> In this view, if a machine performs a task as well as a human, it understands it exactly as much as a human. There's no problem of how to do understanding, only how to do tasks. Yes, but you also gloss over what a "task" is or what a "benchmark" is (which has to do with the meaning of generalization). Suppose an AI or human answers 7 questions correctly out of 10 on an ICPC problem set, what are we able infer from that? 1. Is the task equal to answering these 10 questions well, with a uniform measure of importance? 2. Is the task be good at competitive programming problems? 3. Is the task be good at coding? 4. Is the task be good at problem solving? 5. Is the task not just to be effective under a uniform measure of importance, but an adversarial measure? (i.e. you can probably figure out all kinds of competitive programming questions, if you had more time / etc... but roughly not needing "exponentially more resources") These are very different levels of abstraction, and literally the same benchmark result can be interpreted to mean very different things. And that imputation of generality is not objective unless we know the mechanism by which it happens. "Understanding" is short-hand for saying that performance generalizes at one of the higher levels of abstraction (3--5), rather than narrow success -- because that is what we expect of a human.

simianwords 1y ago
How do you quantify generality? If we have a benchmark that can quantify it and that benchmark reliably tells us that the LLM is within human levels of generalisation then the llm is not distinguishable from a human. While it’s a good point that we need to benchmark generalisation ability, you have in fact agreed that it is not important to understand underlying mechanics.
- godelski 1y ago
  That's kinda their point The difference though is they understand that you can't just benchmark your way into proofs. Just like you can't unit test your way into showing code is error free. Benchmarks and unit tests are great tools that provide a lot of help, but just because a hammer is useful doesn't make everything a nail.

deleted 1y ago

[deleted]