5 ms·

My impression is that the vast majority of people who are currently engaged in the “discourse” surrounding AI do not have the technical background to make this

My impression is that the vast majority of people who are currently engaged in the “discourse” surrounding AI do not have the technical background to make this distinction. In my opinion, there’s an element of “the man behind the curtain” which is being unethically leveraged by AI companies to drive sales and hype.

shanebellone 3y ago
Plus, there seems to be a chronic conflation of "capable of configuring" and "fundamentally understanding the technology".
yowlingcat 3y ago
I agree and there is an enormous moral hazard to the latter. Just looking at what Hinton has been discussing post G, and you see ideological support for a narrow model, shameless anthropomorphization of LLM behavior, and downplaying of "hallucinations" (which I'd rather simply call what it is, which is lying). The moral hazard (for sure not new) is that those most equipped to call out the overplaying of the technology's abilities are also those most personally incented (via fame for their creation), or commercially incented (as you mentioned) to drive sales and hype.
- HybridCurve 3y ago
  > which I'd rather simply call what it is, which is lying I like the term "bullshitting" as it implies it acts like it knows, but doesn't. I can't stand it when they call it "hallucinating", as it instead might imply the problem is with the interpretation of the sensory input. This term might be more appropriate to describe adversarial attacks executed through information input.
  - shanebellone 3y ago
    "which I'd rather simply call what it is, which is lying" "I like the term "bullshitting" as it implies it acts like it knows, but doesn't." It's not "lying" or "bullshitting". It's working as designed. It's not a repository of truth; just long-form autofill.
    - HybridCurve 3y ago
      It's not being sold as long-form autofill. It's being sold as something which will respond to queries, and there is an expectation that the information it replies with should have some accuracy. I am not in disagreement with your generalization of how it functions or that it lacks empirical knowledge. Some responses are accurate, some are not. It will always give a response but does not always produce a disclaimer saying that the response might have been inaccurate. So, in criticism of the existing nomenclature used to describe this phenomena (hallucination), I simply suggested something more appropriate.
      - shanebellone 3y ago
        I agree that it's being positioned deceptively. Today, it is just a novelty.
  - kaycey2022 3y ago
    I think “malfunctioning” is the correct term you are looking for. Lying and Bullshitting and Hallucinating anthropomorphises the software. It is simply not functioning as advertised (not intended).
    - danaris 3y ago
      "Malfunctioning" isn't really accurate, either—because the common usage of that term isn't "not functioning as advertised", not in this world where salespeople will say literally anything to get someone to buy their product. It's "not functioning as designed". "Hallucinations" are 100% an LLM functioning as designed. They are every bit as valid as any other output. LLMs have no concept of truth, no model of the world, no way even in theory to determine the validity of any particular statement they output. The problem isn't the term "hallucinations", so much as it is the idea that anything they produce is not a hallucination.
- sfpotter 3y ago
  I totally agree with this. I like the term "criti-hype". To me, the NYT interview with Hinton seems so obviously calculated for this kind of effect. What is much more worrisome to me than any of the supposed abilities of recent ML models is the fact that the average person just... doesn't seem to pick up on this kind of thing. Although, I wouldn't personally say that it's "lying", "bullshitting", or anything else. That's just more anthropomorphization. It's "wrong" or "incorrect". A mathematical model, a clock, a map: these can all be "wrong" or "incorrect". You might say they were lying to you, but it would be for comedic effect. In this case, there are (again!) enough people who lack the ability to make the technical distinction that I think it's important we avoid any kind of anthropomorphization of any ML model to avoid confusion.
  - yowlingcat 3y ago
    Maybe a more precise (albeit informal) term would be "freestyling" or "improvising" :)
JustBreath 3y ago
Same thing happened with VMs, "the cloud" and containers... Hapless CIOs and senior technical advisors around the world were convinced all their problems would be solved if they'd just sign a contract with <arbitrary vendor>. And their on the ground technical people would tell them "this ain't it" but would be shushed to tell a story to CEOs and shareholders.
- George83728 3y ago
  > Same thing happened with VMs, "the cloud" and containers... Hapless CIOs and senior technical advisors around the world were convinced all their problems would be solved if they'd just sign a contract with <arbitrary vendor>. Pinning this on the phrase "the cloud" is a mistake, it's part of a much broader trend of using specialist contractors for things instead of doing everything in house. If you could somehow magically go back in time and prevent "the cloud" or any other equivalent phrase from existing, the same trend of companies hiring other companies to do the work they don't care to specialize in would still exist. From the perspective of... an insurance company.. why shouldn't they contract another company to handle the servers? They already hire another company to hire janitorial work, building construction and maintenance, human resources, running their cafeteria, call centers, etc, etc... The computer stuff seems special to us because we're tech workers and this is a tech forum, but really it isn't special and it happens in virtually all aspects of doing business. The trend doesn't exist because of the buzzword, the buzzword exists because of the trend. If the buzzword didn't exist, a new one would be invented immediately, because language is a tool for conveying ideas and people make new tools on the fly whenever we find ourselves needing one. Ideas don't come from words; words come from ideas.

shanebellone 3y ago

Plus, there seems to be a chronic conflation of "capable of configuring" and "fundamentally understanding the technology".

yowlingcat 3y ago

I agree and there is an enormous moral hazard to the latter. Just looking at what Hinton has been discussing post G, and you see ideological support for a narrow model, shameless anthropomorphization of LLM behavior, and downplaying of "hallucinations" (which I'd rather simply call what it is, which is lying). The moral hazard (for sure not new) is that those most equipped to call out the overplaying of the technology's abilities are also those most personally incented (via fame for their creation), or commercially incented (as you mentioned) to drive sales and hype.

HybridCurve 3y ago
> which I'd rather simply call what it is, which is lying I like the term "bullshitting" as it implies it acts like it knows, but doesn't. I can't stand it when they call it "hallucinating", as it instead might imply the problem is with the interpretation of the sensory input. This term might be more appropriate to describe adversarial attacks executed through information input.
- shanebellone 3y ago
  "which I'd rather simply call what it is, which is lying" "I like the term "bullshitting" as it implies it acts like it knows, but doesn't." It's not "lying" or "bullshitting". It's working as designed. It's not a repository of truth; just long-form autofill.
  - HybridCurve 3y ago
    It's not being sold as long-form autofill. It's being sold as something which will respond to queries, and there is an expectation that the information it replies with should have some accuracy. I am not in disagreement with your generalization of how it functions or that it lacks empirical knowledge. Some responses are accurate, some are not. It will always give a response but does not always produce a disclaimer saying that the response might have been inaccurate. So, in criticism of the existing nomenclature used to describe this phenomena (hallucination), I simply suggested something more appropriate.
    - shanebellone 3y ago
      I agree that it's being positioned deceptively. Today, it is just a novelty.
- kaycey2022 3y ago
  I think “malfunctioning” is the correct term you are looking for. Lying and Bullshitting and Hallucinating anthropomorphises the software. It is simply not functioning as advertised (not intended).
  - danaris 3y ago
    "Malfunctioning" isn't really accurate, either—because the common usage of that term isn't "not functioning as advertised", not in this world where salespeople will say literally anything to get someone to buy their product. It's "not functioning as designed". "Hallucinations" are 100% an LLM functioning as designed. They are every bit as valid as any other output. LLMs have no concept of truth, no model of the world, no way even in theory to determine the validity of any particular statement they output. The problem isn't the term "hallucinations", so much as it is the idea that anything they produce is not a hallucination.
sfpotter 3y ago
I totally agree with this. I like the term "criti-hype". To me, the NYT interview with Hinton seems so obviously calculated for this kind of effect. What is much more worrisome to me than any of the supposed abilities of recent ML models is the fact that the average person just... doesn't seem to pick up on this kind of thing. Although, I wouldn't personally say that it's "lying", "bullshitting", or anything else. That's just more anthropomorphization. It's "wrong" or "incorrect". A mathematical model, a clock, a map: these can all be "wrong" or "incorrect". You might say they were lying to you, but it would be for comedic effect. In this case, there are (again!) enough people who lack the ability to make the technical distinction that I think it's important we avoid any kind of anthropomorphization of any ML model to avoid confusion.
- yowlingcat 3y ago
  Maybe a more precise (albeit informal) term would be "freestyling" or "improvising" :)

JustBreath 3y ago

Same thing happened with VMs, "the cloud" and containers... Hapless CIOs and senior technical advisors around the world were convinced all their problems would be solved if they'd just sign a contract with <arbitrary vendor>. And their on the ground technical people would tell them "this ain't it" but would be shushed to tell a story to CEOs and shareholders.

George83728 3y ago
> Same thing happened with VMs, "the cloud" and containers... Hapless CIOs and senior technical advisors around the world were convinced all their problems would be solved if they'd just sign a contract with <arbitrary vendor>. Pinning this on the phrase "the cloud" is a mistake, it's part of a much broader trend of using specialist contractors for things instead of doing everything in house. If you could somehow magically go back in time and prevent "the cloud" or any other equivalent phrase from existing, the same trend of companies hiring other companies to do the work they don't care to specialize in would still exist. From the perspective of... an insurance company.. why shouldn't they contract another company to handle the servers? They already hire another company to hire janitorial work, building construction and maintenance, human resources, running their cafeteria, call centers, etc, etc... The computer stuff seems special to us because we're tech workers and this is a tech forum, but really it isn't special and it happens in virtually all aspects of doing business. The trend doesn't exist because of the buzzword, the buzzword exists because of the trend. If the buzzword didn't exist, a new one would be invented immediately, because language is a tool for conveying ideas and people make new tools on the fly whenever we find ourselves needing one. Ideas don't come from words; words come from ideas.