8 ms·

Nonsense, the industry giants are just trying to scare the law makers to license the technology. Effectively, cutting out everyone else. Remember the Google no

by b3nji 3y ago

Nonsense, the industry giants are just trying to scare the law makers to license the technology. Effectively, cutting out everyone else. Remember the Google note circulating saying "they have no moat", this is their moat. They have to protect their investment, we don't want people running this willy nilly for next to no cost on their own devices, God forbid!

sanderjd 3y ago
I would definitely find it more credible if the most capable models that are safe to grandfather in to being unregulated didn't just happen to be the already successful products from all the people leading these safety efforts. It also just happens to be the case that making proprietary models - like the current incumbents make - is the only safe way to do it.
aceon48 3y ago
That moat document was published by a single software engineer, not some exec or product leader. Humans dont really grasp exponential improvements. You wont have much time to regulate something that is improving exponentially.
- jvanderbot 3y ago
  Single software engineers writing influential papers is often enough how a exec or product leader draws conclusions, I expect. It worked that way in everywhere I've worked.
- aero-deck 3y ago
  It doesn't matter who wrote it, it got picked up, had a good argument and affected market opinion. The execs now need to respond to it. Humans also don't grasp that things can improve exponentially until they stop improving exponentially. This belief that AGI is just over the hill is sugar-water for extracting more hours from developers. The nuclear bomb was also supposed to change everything. But in the end nothing changed, we just got more of the same.
  - api 3y ago
    It's too early to say definitively but it's possible that the atomic bomb dramatically reduced the number of people killed in war by making great power conflicts too damaging to undertake: https://kagi.com/proxy/battle_deaths_chart.png?c=qmSKsRSwhgA0fDlF-W9QwvufnjsMl2SNOSuOrrpcCh51CkY0ib1Ce8jvK6POgw2GfCE4A6wA5b7KJzIosB9x2oLw58ggVXoyTHpCMp7FZ9w%3D https://kagi.com/proxy/battle_deaths_chart.png?c=qmSKsRSwhgA... The USA and USSR would almost certainly have fought a conventional WWIII without the bomb. Can you imagine the casualty rates for that...
    - aero-deck 3y ago
      cool - so AI is gonna dramatically reduce the number of emails that get misunderstood... still gonna still be sending those emails tho.
    - wrycoder 3y ago
      So far.
    - TheCaptain4815 3y ago
      I'd actually guess those casualties would be quite less than WW2. As tech advanced, more sophisticated targeting systems also advanced. No need to waste shells and missiles on civilian buildings, plus food and healthcare tech would continue to advance. Meanwhile, a single nuclear bomb hitting a major city could cause more casualties' than all American deaths in ww2 (400k).
      - snickerbockers 3y ago
        That's really only true for the Americans, the Russians still don't seem to care about limiting collateral damage and undoubtedly the Americans wouldn't either if their cities were getting carpet bombed by soviet aircraft.
  - kalkin 3y ago
    "nuclear weapons are no big deal actually" is just a wild place to get as a result of arguing against AI risk. Although I guess Eliezer Yudkowsky would agree! (On grounds that nukes won't kill literally everyone while AI will, but still.)
- deleted 3y ago
  [deleted]
layer8 3y ago
At least now if it turns out they are right they can’t claim anymore that they didn’t know.
kalkin 3y ago
This could be Google's motivation (although note that Google is not actually the market leader right now) but the risk could still be real. Most of the signatories are academics, for one thing, including two who won Turing awards for ML work and another who is the co-author of the standard AI textbook (at least when I was in school). You can be cynical about corporate motives and still worried. I personally am worried about AI partly because I am very cynical about how corporations will use it, and I don't really want my atoms to be ground up to add storage bits for the number that once represented Microsoft's market cap or whatever. But even cynicism doesn't seem to me to give much reason to worry about regulation of "next to no cost" open source models, though. There's only any chance of regulation being practical if models stay very expensive to make, requiring specialized hardware with a supply chain chokepoint. If personal devices do catch up to the state of the art, then for better or worse regulation is not going to prevent people from using them.
- hammock 3y ago
  >Most of the signatories are academics, for one thing Serious question, who funds their research? And do any of them ever plan to work or consult in industry? My econ professor was an “academic” and drew a modest salary while he made millions at the same time providing expert testimony for giant monopolies in antitrust disputes
  - vpastore 3y ago
    They are also interested in cutting out other researchers that can compete with them in accademic output and prestige.
  - holmesworcester 3y ago
    That's a good question, but at least some of the academics on this list are independent. Bruce Schneier, for example.
    - moffkalast 3y ago
      So some are naive and the rest are self interested?
nico 3y ago
> scare the law makers to license the technology You mean scare the public so they can do business with the lawmakers without people asking too many questions
arisAlexis 3y ago
All academia and researchers say X. Random redditor/HN lurker declares nonesense I know better! This is how we should bet our future.
- yeck 3y ago
  I find it so troubling that the most common HN response to this isn't to engage with the ideas or logic behind the concerns, but simply to speculate on the unknowable intentions of those that signed the letter. We can base our arguments on the unprovable, some specific person's secret intentions, or we engage the their ideas. One is lazy and meaningless, the other actually takes effort. Then there is this lazy and false equivalency between corporations being interested in market capture and AI risks being exaggerated.
jiggawatts 3y ago
Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI. This is what they’re terrified of. They’ve invested near a billion dollars and need billions in revenue to enrich their shareholders. But if the data leaks? They can’t stop random companies or moneyed individuals running the models on their own kit. My prediction is that there will be copyright enforcement mandated by law in all GPUs. If you upload weights from the big AI companies then the driver will block it and phone home. Or report you to the authorities for violations of corporate profits… err… “AI Safety”. I guarantee something like this will happen within months because the clock is ticking. It takes just one employee to deliberately or accidentally leak the weights…
- moneywoes 3y ago
  > Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI Sorry I don’t understand what would be the impact? Aren’t the results not deterministic What are the weights here
- BeFlatXIII 3y ago
  Time to jailbreak GPUs, then. That or import "international" editions grey market from Taiwan.
- okdood64 3y ago
  > My prediction is that there will be copyright enforcement mandated by law in all GPUs. Oh this is a nifty idea. Hadn't thought of regulation in this manner. Seems like it would be pretty effective too.

by b3nji 3y ago

sanderjd 3y ago

I would definitely find it more credible if the most capable models that are safe to grandfather in to being unregulated didn't just happen to be the already successful products from all the people leading these safety efforts. It also just happens to be the case that making proprietary models - like the current incumbents make - is the only safe way to do it.

aceon48 3y ago

That moat document was published by a single software engineer, not some exec or product leader. Humans dont really grasp exponential improvements. You wont have much time to regulate something that is improving exponentially.

jvanderbot 3y ago
Single software engineers writing influential papers is often enough how a exec or product leader draws conclusions, I expect. It worked that way in everywhere I've worked.
aero-deck 3y ago
It doesn't matter who wrote it, it got picked up, had a good argument and affected market opinion. The execs now need to respond to it. Humans also don't grasp that things can improve exponentially until they stop improving exponentially. This belief that AGI is just over the hill is sugar-water for extracting more hours from developers. The nuclear bomb was also supposed to change everything. But in the end nothing changed, we just got more of the same.
- api 3y ago
  It's too early to say definitively but it's possible that the atomic bomb dramatically reduced the number of people killed in war by making great power conflicts too damaging to undertake: https://kagi.com/proxy/battle_deaths_chart.png?c=qmSKsRSwhgA0fDlF-W9QwvufnjsMl2SNOSuOrrpcCh51CkY0ib1Ce8jvK6POgw2GfCE4A6wA5b7KJzIosB9x2oLw58ggVXoyTHpCMp7FZ9w%3D https://kagi.com/proxy/battle_deaths_chart.png?c=qmSKsRSwhgA... The USA and USSR would almost certainly have fought a conventional WWIII without the bomb. Can you imagine the casualty rates for that...
  - aero-deck 3y ago
    cool - so AI is gonna dramatically reduce the number of emails that get misunderstood... still gonna still be sending those emails tho.
  - wrycoder 3y ago
    So far.
  - TheCaptain4815 3y ago
    I'd actually guess those casualties would be quite less than WW2. As tech advanced, more sophisticated targeting systems also advanced. No need to waste shells and missiles on civilian buildings, plus food and healthcare tech would continue to advance. Meanwhile, a single nuclear bomb hitting a major city could cause more casualties' than all American deaths in ww2 (400k).
    - snickerbockers 3y ago
      That's really only true for the Americans, the Russians still don't seem to care about limiting collateral damage and undoubtedly the Americans wouldn't either if their cities were getting carpet bombed by soviet aircraft.
- kalkin 3y ago
  "nuclear weapons are no big deal actually" is just a wild place to get as a result of arguing against AI risk. Although I guess Eliezer Yudkowsky would agree! (On grounds that nukes won't kill literally everyone while AI will, but still.)
deleted 3y ago
[deleted]

layer8 3y ago

At least now if it turns out they are right they can’t claim anymore that they didn’t know.

kalkin 3y ago

This could be Google's motivation (although note that Google is not actually the market leader right now) but the risk could still be real. Most of the signatories are academics, for one thing, including two who won Turing awards for ML work and another who is the co-author of the standard AI textbook (at least when I was in school). You can be cynical about corporate motives and still worried. I personally am worried about AI partly because I am very cynical about how corporations will use it, and I don't really want my atoms to be ground up to add storage bits for the number that once represented Microsoft's market cap or whatever. But even cynicism doesn't seem to me to give much reason to worry about regulation of "next to no cost" open source models, though. There's only any chance of regulation being practical if models stay very expensive to make, requiring specialized hardware with a supply chain chokepoint. If personal devices do catch up to the state of the art, then for better or worse regulation is not going to prevent people from using them.

hammock 3y ago
>Most of the signatories are academics, for one thing Serious question, who funds their research? And do any of them ever plan to work or consult in industry? My econ professor was an “academic” and drew a modest salary while he made millions at the same time providing expert testimony for giant monopolies in antitrust disputes
- vpastore 3y ago
  They are also interested in cutting out other researchers that can compete with them in accademic output and prestige.
- holmesworcester 3y ago
  That's a good question, but at least some of the academics on this list are independent. Bruce Schneier, for example.
  - moffkalast 3y ago
    So some are naive and the rest are self interested?

nico 3y ago

> scare the law makers to license the technology You mean scare the public so they can do business with the lawmakers without people asking too many questions

arisAlexis 3y ago

All academia and researchers say X. Random redditor/HN lurker declares nonesense I know better! This is how we should bet our future.

yeck 3y ago
I find it so troubling that the most common HN response to this isn't to engage with the ideas or logic behind the concerns, but simply to speculate on the unknowable intentions of those that signed the letter. We can base our arguments on the unprovable, some specific person's secret intentions, or we engage the their ideas. One is lazy and meaningless, the other actually takes effort. Then there is this lazy and false equivalency between corporations being interested in market capture and AI risks being exaggerated.

jiggawatts 3y ago

Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI. This is what they’re terrified of. They’ve invested near a billion dollars and need billions in revenue to enrich their shareholders. But if the data leaks? They can’t stop random companies or moneyed individuals running the models on their own kit. My prediction is that there will be copyright enforcement mandated by law in all GPUs. If you upload weights from the big AI companies then the driver will block it and phone home. Or report you to the authorities for violations of corporate profits… err… “AI Safety”. I guarantee something like this will happen within months because the clock is ticking. It takes just one employee to deliberately or accidentally leak the weights…

moneywoes 3y ago
> Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI Sorry I don’t understand what would be the impact? Aren’t the results not deterministic What are the weights here
BeFlatXIII 3y ago
Time to jailbreak GPUs, then. That or import "international" editions grey market from Taiwan.
okdood64 3y ago
> My prediction is that there will be copyright enforcement mandated by law in all GPUs. Oh this is a nifty idea. Hadn't thought of regulation in this manner. Seems like it would be pretty effective too.