4 ms·

Don’t humans fit that definition? We’ve managed okay for 1000s of years under those conditions.

by onethought 3y ago

bena 3y ago
They do not. A human can verify its own reliability.
- ryanklee 3y ago
  A human cannot do this self-sufficiently. This is why we work so hard to implement risk mitigation measures and checks and balances on changes.
  - bena 3y ago
    Yes, not all humans at all tasks all the time. And some things are important enough to implement checks regardless. However, there's a lot we can just throw humans at and trust that the thing will get complete and be correct. And even with the checks and balances, we can have the human perform those checks and balances. A human is a pretty autonomous unit on average. So far, AI can't really say "let me double check that for you" for instance. You ask it a thing, it does a thing, and that's it. If it's wrong, you have to tell it to do the thing again, but differently. In all the rush to paint these LLMs as "pretty much human", we've instead taken to severely downplaying just how adaptable and clever most sentient beings can be.
    - ryanklee 3y ago
      AIs can double check. Agent to agent confirmation and validation is a thing. In any case, the point is that we have learned techniques to compensate for human fallibility. We will learn techniques to compensate for gen AI fallibility, as well. The objection that AIs can be wrong is far less a barrier to the rise of their utility than is often supposed.
      - bena 3y ago
        You see how that's worse, right? The original argument put forth was that "an inherently unreliable tool cannot gauge its own reliability". Someone responded that humans fit that description as well. I said we don't. We can and do verify our own reliability. We can essentially test our assumptions against the real world and fix them. You then claimed we couldn't do that "self-sufficiently". I responded that while that is true for some tasks, for a lot of tasks, we can. That an AI can't check itself and won't even try. And now you're telling me that they can check against each other. But if you can't trust the originals, asking them if they trust each other is kind of pointless. You're not really doing anything more than adding another layer. For example: If I put my pants on backwards, I correct that myself. Without the need to check and/or balance against any other person. I am self-correcting to a large degree. The AI would not even know to check its pants until someone told it it was wrong. The objection isn't that "AIs can be wrong", the objection is that AIs can't really tell the difference between correct and incorrect. So everything has to be checked, often with as much effort as it would take to do the thing in the first place.
        ryanklee 3y ago
        This is a very narrow view of how LLMs can interact to improve inference accuracy. Different LLMs have different capabilities. They can be used in coordination to improve results. Your objections seem to rely on a restricted view that says "we can't do better" but with no evidence. Whereas we have plenty of evidence of massive, continual improvement in the very areas you are holding up as problematic.
- cornel_io 3y ago
  If that was even close to true then I would have had to fire far fewer people over the years.
ryanklee 3y ago
They do fit this definition. One result of the rise of generative AI is exposing just how severely and commonly people misperceive their own capabilities and the functioning of their cognitive powers.
trefoiled 3y ago
I hear this argument applied often when people bring up the deficiencies of AI, and I don't find it convincing. Compare an AI coding assistant to reaching out to another engineer on my team as an example. If I know this engineer, I will likely have an idea of their relative skill level, their familiarity with the problem at hand, their propensity to suggest one type of solution over another, etc. People are pretty good at developing this kind of sense because we work with other people constantly. The AI assistant, on the other hand, is very much not like a human. I have a limited capacity to understand its "thought process," and I consider myself far more technical than the average person. This makes a verification step troublesome, because I don't know what to expect. This difference is even more stark when it comes to driving assistants. Video compilations of Teslas with FSD behaving erratically and most importantly, unpredictably, are all over the place. Experienced Tesla drivers seem to have some limited ability to predict the weaknesses of the FSD package, but the issue is that the driving assistant is so unlike a human. I've seen multiple examples of people saying "well, humans cause car crashes too," but the key difference is that I have to sit behind the wheel and deal with the fact that my driving assistant may or may not suddenly swerve into oncoming traffic. The reasons for it doing so are likely obscure to me, and this is a real problem.
abeppu 3y ago
Humans are unreliable, but we are also under normal circumstances thoroughly and continually grounded in an external world whose mechanics we interact with, make predictions about, and correct our beliefs about. The specific way we're training coding assistants for next-token-prediction would also be an incredibly difficult context for humans to produce code. Suppose you were dropped off in an society of aliens whose perceptual, cultural and cognitive universe is meaningfully different from our own; you don't have a grounding in concepts of what they're trying to _do_ with their programs. You receive a giant dump of reams and reams of source code, in their unfamiliar script, where none of the names initially mean anything to you. In the pile of training material handed to you, you might find some documentation about their programming language, but it's written in their (foreign, weird to you) natural language, and is mixed with everythign else. You never get a teacher who can answer questions, never get access to a IDE/repl/interpreter/debugger/compiler, never get to _run_ a program on different inputs to see its outputs, never get to add a log line to peek at the program's internal state, etc. After a _lot_ of training, you can often predict the next symbol in a program text. But shouldn't we _expect_ you to be "unreliable"? You don't have the ability to run checks against the code you produce! You don't get a warning if you use a variable that doesn't exist! You just produce _tokens_, and get no feedback. To the degree humans are reliable at coding, it's because we can simulate what program execution will do, with a level of abstraction which we vary in a task dependent way. You can mentally step through every line in a program carefully if you need to. But you can also mentally choose to trust some abstraction and skip steps which you infer cannot be related to some attribute or condition of interest if that abstraction is upheld. The most important parts of your attention are on _what the program does_. This is fully hidden in the next-token-prediction scenario, which is totally focused on _what tokens are used to write the program_.

Don’t humans fit that definition? We’ve managed okay for 1000s of years under those conditions.

bena 3y ago

They do not. A human can verify its own reliability.

ryanklee 3y ago
A human cannot do this self-sufficiently. This is why we work so hard to implement risk mitigation measures and checks and balances on changes.
- bena 3y ago
  Yes, not all humans at all tasks all the time. And some things are important enough to implement checks regardless. However, there's a lot we can just throw humans at and trust that the thing will get complete and be correct. And even with the checks and balances, we can have the human perform those checks and balances. A human is a pretty autonomous unit on average. So far, AI can't really say "let me double check that for you" for instance. You ask it a thing, it does a thing, and that's it. If it's wrong, you have to tell it to do the thing again, but differently. In all the rush to paint these LLMs as "pretty much human", we've instead taken to severely downplaying just how adaptable and clever most sentient beings can be.
  - ryanklee 3y ago
    AIs can double check. Agent to agent confirmation and validation is a thing. In any case, the point is that we have learned techniques to compensate for human fallibility. We will learn techniques to compensate for gen AI fallibility, as well. The objection that AIs can be wrong is far less a barrier to the rise of their utility than is often supposed.
    - bena 3y ago
      You see how that's worse, right? The original argument put forth was that "an inherently unreliable tool cannot gauge its own reliability". Someone responded that humans fit that description as well. I said we don't. We can and do verify our own reliability. We can essentially test our assumptions against the real world and fix them. You then claimed we couldn't do that "self-sufficiently". I responded that while that is true for some tasks, for a lot of tasks, we can. That an AI can't check itself and won't even try. And now you're telling me that they can check against each other. But if you can't trust the originals, asking them if they trust each other is kind of pointless. You're not really doing anything more than adding another layer. For example: If I put my pants on backwards, I correct that myself. Without the need to check and/or balance against any other person. I am self-correcting to a large degree. The AI would not even know to check its pants until someone told it it was wrong. The objection isn't that "AIs can be wrong", the objection is that AIs can't really tell the difference between correct and incorrect. So everything has to be checked, often with as much effort as it would take to do the thing in the first place.
      - ryanklee 3y ago
        This is a very narrow view of how LLMs can interact to improve inference accuracy. Different LLMs have different capabilities. They can be used in coordination to improve results. Your objections seem to rely on a restricted view that says "we can't do better" but with no evidence. Whereas we have plenty of evidence of massive, continual improvement in the very areas you are holding up as problematic.
cornel_io 3y ago
If that was even close to true then I would have had to fire far fewer people over the years.

ryanklee 3y ago

They do fit this definition. One result of the rise of generative AI is exposing just how severely and commonly people misperceive their own capabilities and the functioning of their cognitive powers.

trefoiled 3y ago

I hear this argument applied often when people bring up the deficiencies of AI, and I don't find it convincing. Compare an AI coding assistant to reaching out to another engineer on my team as an example. If I know this engineer, I will likely have an idea of their relative skill level, their familiarity with the problem at hand, their propensity to suggest one type of solution over another, etc. People are pretty good at developing this kind of sense because we work with other people constantly. The AI assistant, on the other hand, is very much not like a human. I have a limited capacity to understand its "thought process," and I consider myself far more technical than the average person. This makes a verification step troublesome, because I don't know what to expect. This difference is even more stark when it comes to driving assistants. Video compilations of Teslas with FSD behaving erratically and most importantly, unpredictably, are all over the place. Experienced Tesla drivers seem to have some limited ability to predict the weaknesses of the FSD package, but the issue is that the driving assistant is so unlike a human. I've seen multiple examples of people saying "well, humans cause car crashes too," but the key difference is that I have to sit behind the wheel and deal with the fact that my driving assistant may or may not suddenly swerve into oncoming traffic. The reasons for it doing so are likely obscure to me, and this is a real problem.

abeppu 3y ago

Humans are unreliable, but we are also under normal circumstances thoroughly and continually grounded in an external world whose mechanics we interact with, make predictions about, and correct our beliefs about. The specific way we're training coding assistants for next-token-prediction would also be an incredibly difficult context for humans to produce code. Suppose you were dropped off in an society of aliens whose perceptual, cultural and cognitive universe is meaningfully different from our own; you don't have a grounding in concepts of what they're trying to _do_ with their programs. You receive a giant dump of reams and reams of source code, in their unfamiliar script, where none of the names initially mean anything to you. In the pile of training material handed to you, you might find some documentation about their programming language, but it's written in their (foreign, weird to you) natural language, and is mixed with everythign else. You never get a teacher who can answer questions, never get access to a IDE/repl/interpreter/debugger/compiler, never get to _run_ a program on different inputs to see its outputs, never get to add a log line to peek at the program's internal state, etc. After a _lot_ of training, you can often predict the next symbol in a program text. But shouldn't we _expect_ you to be "unreliable"? You don't have the ability to run checks against the code you produce! You don't get a warning if you use a variable that doesn't exist! You just produce _tokens_, and get no feedback. To the degree humans are reliable at coding, it's because we can simulate what program execution will do, with a level of abstraction which we vary in a task dependent way. You can mentally step through every line in a program carefully if you need to. But you can also mentally choose to trust some abstraction and skip steps which you infer cannot be related to some attribute or condition of interest if that abstraction is upheld. The most important parts of your attention are on _what the program does_. This is fully hidden in the next-token-prediction scenario, which is totally focused on _what tokens are used to write the program_.