10 ms·

I'm really disappointed to see the HN reaction of straight up scoffing at this like it's irreversible and so let's just step on the gas. It's some weird combina

I'm really disappointed to see the HN reaction of straight up scoffing at this like it's irreversible and so let's just step on the gas. It's some weird combination of techno-utopianism and nihilism and good old elitism.

sp527 4y ago
Almost everyone here will be fine in a world in which only certain non-renewable scarce resources have any value, combined with impending massive currency deflation. Even a rank and file tier 2/3 engineer will manage to live a princely life in that new reality assuming (s)he saved up enough. Hardly anyone in the tech community is capable of truly empathizing with the vast majority of people, who are born with none of the advantages that they had (genetic, parental, cultural, geographic, etc).
- davesque 4y ago
  > Hardly anyone in the tech community is capable of truly empathizing with the vast majority of people This is a gross exaggeration.
  - JohnFen 4y ago
    It is a gross exaggeration -- but it is also the image that SV has been giving off for a long time now.
    - ChatGTP 4y ago
      These data used to train these models are tagged by people getting paid hardly anything, so why wouldn't it? The same people SV people then turn around and say that these systems will help them get access to free medical care? lol
  - sp527 4y ago
    Well that's not a counterargument, but you're also missing the point completely, which is that you have to have a very low capacity for empathy in order to push ahead towards AGI when you know society is not prepared for this and that it's going to induce considerable pain. Americans (let alone people elsewhere in the world) are already struggling. Recent reporting suggests a great many have to work multiple jobs. Almost all of us work an absurd number of hours per week. Many if not most can't afford homes. Plenty are ending up on the streets. Healthcare can literally bankrupt people. A vacation out of the country is an impossible luxury for most. The majority of Americans still don't send their children to college, usually because of affordability. And I haven't even touched on what life is like in most of Africa or Asia. This is the world we're bringing AI into. You have to be something adjacent to a sociopath to be okay with that. So long as our system is predicated on capitalism, AI may very well induce more downstream suffering than anything else humans have ever conceived.
    - pdonis 4y ago
      People aren't "strugging" because there is no work for them to do. They are strugging because the powers that be have jiggered our economic system in order to hamper, instead of facilitate, free market cooperation, specialization, and trade. Governments micromanage everything. That is what needs to stop. If "AI" ends up making governments think they can continue to micromanage and get away with it, yes, that will cause more suffering. But that's not the failure mode any critics of "AI" are trumpeting about.
    - davesque 4y ago
      > Americans (let alone people elsewhere in the world) are already struggling I agree. And I agree with your overall sentiment about the risks of pursuing AGI. I'm as cynical as anyone about the likelihood that the average person will really be any happier in a world with AGI (controlled by tech billionaires no less). That being said, to claim that hardly anyone in the tech community is capable of empathizing with the average person is a wild overstatement that brings nothing to the discussion. Just adds to the noise.
      - sp527 4y ago
        Late reply here but I wanted to point out that you still don’t get it. True empathy in the tech community would be e.g. having the courage to say that building HLAI of the kind we’re now approaching is guaranteed to cause tremendous amounts of suffering for ordinary people (who will not be able to respond elastically to so abrupt a tectonic shift), and therefore the whole enterprise is fundamentally evil. Let’s get real concrete about what’s going to happen: people will lose their jobs, then their homes, they’ll become destitute, they’ll experience divorces, some will commit suicide, they will suffer desperately in myriad other ways due to economic disenfranchisement, kids will be deprived of a comfortable upbringing, etc. How many in the tech industry are genuinely discussing the very real consequences of nonlinear degrees of automation for the kinds of ordinary people they barely interact with? How many are pretending that there isn’t something disgustingly immoral about having some of the most affluent and economically insulated people devise and inflict this reality upon countless millions? I will maintain that this industry is morally bankrupt and nearly entirely devoid of empathy. These are not the people who should be in charge of our future.
- gordian-mind 4y ago
  [flagged]
- pdonis 4y ago
  > a world in which only certain non-renewable scarce resources have any value If you think this is the world "AI" is leading us toward, I think you need to think again. There are a huge range of skills that only humans can provide, and that will continue to be true. GPT-3 can't fix your plumbing or your HVAC when it breaks. It can't fix your car. It can't build houses. It can't build roads. It can't build computers. And on and on and on... Nor are AI enthusiasts trying to get "AI" to do any of these things. Humans are real-world creatures with real-world needs that we supply to each other by free markets, specialization, and trade. The "tech community" might take all this for granted, but that doesn't mean it just happens by magic. It happens because real humans do real work. "AI" is not going to take that over any time soon.
  - sp527 4y ago
    What do you think happens when AI compresses the economy and leaves us almost exclusively with manual labor jobs, most of which are already low-paying? And you may not be aware of this having never done such a job, but physical labor is often brutally taxing on the body, putting people at risk of a wide array of potential health complications. I've seen this abjectly atrocious argument about manual labor made by a number of people on HN and it's astounding how poorly thought through it is.
    - pdonis 4y ago
      > when AI compresses the economy How? How will it do this? Basically, people seem to assume that "AI" has some kind of magical power to do whatever bad thing they can imagine, and then they extrapolate from there. I don't see it. > physical labor If you think the jobs I described are purely "physical labor", or that the physical labor involved is so highly taxing that people can only do those jobs for a short time before they wear out, you definitely need to think again. (You also definitely need to think again if you think those jobs are low paying. Plenty of people make quite a healthy living doing them.)
      - sp527 4y ago
        > You also definitely need to think again if you think those jobs are low paying. Plenty of people make quite a healthy living doing them. Will they still make a healthy living when there's an influx of laborers fleeing more automated parts of the economy? GS just put out a report today estimating the impact of AI: https://www.businessinsider.com/generative-ai-chatpgt-300-million-full-time-jobs-goldman-sachs-2023-3 https://www.businessinsider.com/generative-ai-chatpgt-300-mi.... I wonder sometimes if these accounts on HN making insane arguments that generative AI somehow won't be economically calamitous are bots. In fact, if I was at OpenAI and the goal was to avert scrutiny long enough to get to AGI, unleashing a torrent of AI shill bots might be near the top of the agenda.
kelseyfrog 4y ago
AGI is a break-capitalism lottery ticket I'm willing to entertain. If the marginal cost of labor collapses to zero then either we'll get robot police or finally be free to pursue lives of leisure.
- cmsonger 4y ago
  Except that it's going to be robot police. What in the current world order suggests labor wins. Who controls LLMs? Who has the compute to produce AGI?
  - bilbo0s 4y ago
    To be fair, there is always someone on the bottom. It just historically hasn't been people we cared about. That really says more about us than it says about AI. The time to complain about these power asymmetries was prior to them affecting us. Now it just sounds like garden variety sour grape hypocrisy. Not saying there aren't good points. Only pointing out that many, many people can all view the consequences of this dynamic differently.
    - noobermin 4y ago
      The only people who view this favorably are people at the top. People near the bottom perhaps feel disdain for people in the middle but that is misdirected because people at the top always were the ones in control. AI hasn't changed that dynamic.
      - anigbrowl 4y ago
        By 'this' do you mean AI or the petition?
        noobermin 4y ago
        AI. I am replying to the parent comment, but I too get that this petition is being signed onto by people at openAI specifically because they already made it with GPT-4. They definitely would not sign on to it if their (arbitrary) threshold for what to stop doing AI research with was a tool at a GPT-3 or GPT-3.5 level or higher or it included GPT-4 in their threshold, and not merely beyond it.
twblalock 4y ago
It really is irreversible. The methods and compute needed to build a GPT-type AI are widely available.
- versteegen 4y ago
  Yes, but superintelligence hasn't been created yet. That's what the petition is ultimately about, not chat assistants. Transformers unmodified aren't going to scale up to that, they're already nearly scaled maximally.
  - famouswaffles 4y ago
    You don't know what transformers unmodified scale up to. Nearly scaled maximally ? Based on what exactly ?
    - versteegen 4y ago
      The degree to which transformers scale, as measured by loss (training objective) is known remarkably well! See [1]. There's a formula in there for the minimum loss you possibly achieve even with infinite compute and training data, and it's hardly less than Chinchilla's loss. The recent GPT-4 paper further reinforces that these scaling laws are real, because they predicted with high accuracy the loss the model would have based on data+compute used. Admittedly, the link between improvement in loss and improvement of capabilities may break down or be misleading. However, there's just not much training data on the internet left unused. Maybe an order of magnitude. All books ever published (in English?) are a smaller dataset than the corpora already used for training. See [2] (which includes an easy summarisation of much of [1]). And the scaling laws show training data is already the bottleneck rather than compute. [1] DeepMind, 2022, Training Compute-Optimal Large Language Models https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556 [2] Chinchilla's wild implications https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla-s-wild-implications https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla...
      - deleted 4y ago
        [deleted]
andrewmutz 4y ago
You can accuse everyone of *isms, but there are a lot of us who have just never been persuaded by the many articles written to scare us about the effects of AI. The concerns about all the jobs going away have been persistent but the labor market continues to be extremely strong. As long as entrepreneurship is creating new business opportunities and new ways to turn labor into value, it's completely possible that the effects of automation are drowned out by new applications of labor. In fact, this is what we've seen throughout the industrial revolution: constant anxiety about automation that fails to materialize in to mass unemployment. The concerns about polluting the information space are still hypothetical as well. The current information space is a complete garbage fire and it's not due to generative AI. Might it get worse if bots start spamming people with GPT generated text on social media? Yeah it might. But social media is already terrible and we need to do something about it regardless of that.
- jimrandomh 4y ago
  There are two, mostly disjoint groups warning about AI. There are the people worried about comparatively mundane effects from comparatively mundane systems: job loss, spam, disinformation, maybe an occasional unfair loan-application rejection. These concerns don't have nothing going for them, but in all but the worst-case-scenario versions, these just aren't bad enough to make AI not be worth it. Then there's the people looking ahead, foreseeing a future where superintelligent AIs are more powerful than humanity, and worried that most possible variations of those superintelligences are incentivized to destroy us. I think this open letter puts much too much emphasis on the petty stuff, I think because they're trying to appeal to people who are allergic to anything that requires extrapolating more than a little bit into the future. But buying more time for alignment research, before we tackle superintelligence, does meaningfully improve humanity's odds of survival, so I hope this happens anyways.
- creamyhorror 4y ago
  The key issue in automation transitions is the transition of affected individuals to other sources of income. In previous technological revolutions, affected workers were hurt by their loss of income, and some no doubt fell into poverty without ever recovering. Not everyone can be retrained for new types of jobs immediately - (1) they may not have the needed foundational knowledge or the cognitive flexibility/ability, and (2) there might not be enough of the new types of jobs emerging quickly enough for them. Not every displaced miner can become a coder, or be competitive for junior dev jobs. (Why should the state provide for these workers? Well, primarily for humaneness, and also social stability.) The rewards of automation (cost savings as well as profits) are reaped by (1) the capital owners of the automation technology companies (and their higher-paid employees), as well as by (2) the companies and consumers using the new automation; therefore those owners and beneficiaries could be asked to bear at least part of the costs of supporting, retraining, and placing in jobs the workers they displaced. In a nutshell: Redistribution during structural unemployment caused by technological transitions. A humane policy would provide the above types of support for workers displaced by automation. Ideally it would already be handled by existing unemployment policy, but in many countries such support is limited or minimal. Corporate taxation might need some rethinking along the lines of job-displacement effects of companies (a tricky question, I admit - I've come across one or two proposals for assessing the automation level of companies for taxation purposes). The cross-border dynamics add further complexity, given that automation will displace many jobs outsourced across borders. Given that the current AI revolution looks like it will be causing even larger and faster changes than previous revolutions, such policies are imo needed as a prerequisite (one of several) for allowing the development of powerful job-displacing AI.
fwlr 4y ago
“Sir Roger L'Estrange tell us a Story in his Collection of Fables, of the Cock [Rooster] and the Horses. The Cock was gotten to Roost in the Stable, among the Horses, and there being no Racks, or other Conveniencies for him, it seems, he was forc'd to roost upon the Ground; the Horses jostling about for room, and putting the Cock in danger of his Life, he gives them this grave Advice; Pray Gentlefolks let us stand still, for fear we should tread upon one another.” HN is scoffing because the timing of this petition is suspicious. To borrow your metaphor of “step on the gas”, this is as if the race had already started, several competitors are out ahead of the rest of the field, and now someone is yelling “step on the brakes!”.
- reasonableklout 4y ago
  Sam Altman has publicly talked about wanting more regulation. Demis Hassabis stated that AI safety was one of the _motivations_ for founding DeepMind. I don't think any of the industry noise around safety and regulation is new, I think the timing of the petition is relevant _because_ the race has started in an especially messy way (with Llama's weights leaking, Bing/Sydney threatening a journalist based on tweets it found on the internet, various OpenAI security mishaps, etc).
versteegen 4y ago
It's striking how on AGI, the common opinion here on HN is the complete opposite to the one in the AI alignment field. There, people are angry with OpenAI for being too open, here people are angry with it not being open enough. For example, Eliezer Yudkowsky: > Ryan: OpenAI [...] It's kind of a completely open-source AI project? > Eliezer: That was the original suicidal vision, yes. But... > Ryan: And now they're bent on commercializing the technology, is that right? > Eliezer: That's an improvement, but not enough of one, because they're still generating lots of noise and hype and directing more resources into the field, and storming ahead with the safety that they have instead of the safety that they need, and setting bad examples. > [...] I would change the name to "ClosedAI". Because right now, they're making it look like being "closed" is hypocrisy. They're, like, being "closed" while keeping the name "OpenAI", and that itself makes it looks like closure is like not this thing that you do cooperatively so that humanity will not die, but instead this sleazy profit-making thing that you do while keeping the name “OpenAI”. https://www.lesswrong.com/posts/e4pYaNt89mottpkWZ/yudkowsky-on-agi-risk-on-the-bankless-podcast https://www.lesswrong.com/posts/e4pYaNt89mottpkWZ/yudkowsky-...
BobbyJo 4y ago
I personally scoff at this kind of stuff because it's asking for something no one has the power to give. It's like asking to stop the development of nukes in the 40s and 50s, it's just not gonna happen.
- jimrandomh 4y ago
  Preventing the development of nukes entirely was obviously not going to happen. But delaying the first detonations by a few years, and moving the Partial Nuclear Test Ban treaty up a few years, was quite achievable. Whether delaying AI development a little matters depends on whether you think the success of AI alignment, applied to future superintelligence, is overdetermined to succeed, overdetermined to fail, or close to borderline. Personally I think it looks borderline, so I'm glad to see things like this.
  - BobbyJo 4y ago
    I'm firmly in the camp that delaying it's development could make a difference, I just don't see how that's possible. These models are relatively simple and the equipment necessary to develop them is public (and relatively cheap if we're talking about corporate or national scales). At least with nukes, there was a raw material bottleneck, but there really isn't a limiting factor here that any "good guys" could choke point. It's out there and it's going to get worked on, and the only people the "good guys" can limit are themselves.
- paxys 4y ago
  And during that period, and later during the cold war, the decision to make (or stop making) nukes was in the hands of maybe 5 people total. Today there are thousands of companies and tens/hundreds of thousands of people who can legitimately compete in the space. Best of luck trying to resolve a prisoner's dilemma between all of them.
- G_z9 4y ago
  It actually very easily could have happened and almost did happen but the Russians decided to go back on their effort to do it. It really did almost happen. People act like it’s hard. Stopping a huge asteroid is hard. There might not be enough physical resources to do it. Stopping AI or nukes is definitely easy.
anigbrowl 4y ago
I'm very cognizant of the risks, but the biggest risk seems to me that wealthy people hold it back to get an uncrossable political-economic moat in order to permanently secure their own position.
- kvetching 4y ago
  exactly. Specifically China. They've said for years, they want to be the global AI leader. They have more data than anyone. Now that they know LLMs work as good as they do, they are going all in. They achieve AGI first, that's bad. Communist AGI is a greater existential risk than us creating AGI. We must create it first so we can use it to harden our systems.
Veen 4y ago
I think the "good old elitism" is exactly what the alignment folks are expressing. They think only they and large corporations should have these tools, while we, the common folk, should not.
- ChatGTP 4y ago
  Hanlon's Razor...Max Tegmark has been saying for a long time that his biggest concern would be black box models we don't understand gaining more intelligence without a solid understand about how they work ...this is long before ChatGPT4 existed so I don't believe it has anything to do with control and keeping people out of anything. Even Sam Altman has said similar things recently that he believes we're probably going to have to slow down and think about what we're doing. I think a more advanced civilization would slow down and continue AI research on another planet (Mars?)...not here on Earth.
CatWChainsaw 4y ago
Eh, just typical HN superiority complex on show. Personally when we don't slow down and we undermine civilization to a shadow of its current self, I'm going to make sure my last web post is here, and all it's going to read is "Told you. :) "

sp527 4y ago

Almost everyone here will be fine in a world in which only certain non-renewable scarce resources have any value, combined with impending massive currency deflation. Even a rank and file tier 2/3 engineer will manage to live a princely life in that new reality assuming (s)he saved up enough. Hardly anyone in the tech community is capable of truly empathizing with the vast majority of people, who are born with none of the advantages that they had (genetic, parental, cultural, geographic, etc).

davesque 4y ago
> Hardly anyone in the tech community is capable of truly empathizing with the vast majority of people This is a gross exaggeration.
- JohnFen 4y ago
  It is a gross exaggeration -- but it is also the image that SV has been giving off for a long time now.
  - ChatGTP 4y ago
    These data used to train these models are tagged by people getting paid hardly anything, so why wouldn't it? The same people SV people then turn around and say that these systems will help them get access to free medical care? lol
- sp527 4y ago
  Well that's not a counterargument, but you're also missing the point completely, which is that you have to have a very low capacity for empathy in order to push ahead towards AGI when you know society is not prepared for this and that it's going to induce considerable pain. Americans (let alone people elsewhere in the world) are already struggling. Recent reporting suggests a great many have to work multiple jobs. Almost all of us work an absurd number of hours per week. Many if not most can't afford homes. Plenty are ending up on the streets. Healthcare can literally bankrupt people. A vacation out of the country is an impossible luxury for most. The majority of Americans still don't send their children to college, usually because of affordability. And I haven't even touched on what life is like in most of Africa or Asia. This is the world we're bringing AI into. You have to be something adjacent to a sociopath to be okay with that. So long as our system is predicated on capitalism, AI may very well induce more downstream suffering than anything else humans have ever conceived.
  - pdonis 4y ago
    People aren't "strugging" because there is no work for them to do. They are strugging because the powers that be have jiggered our economic system in order to hamper, instead of facilitate, free market cooperation, specialization, and trade. Governments micromanage everything. That is what needs to stop. If "AI" ends up making governments think they can continue to micromanage and get away with it, yes, that will cause more suffering. But that's not the failure mode any critics of "AI" are trumpeting about.
  - davesque 4y ago
    > Americans (let alone people elsewhere in the world) are already struggling I agree. And I agree with your overall sentiment about the risks of pursuing AGI. I'm as cynical as anyone about the likelihood that the average person will really be any happier in a world with AGI (controlled by tech billionaires no less). That being said, to claim that hardly anyone in the tech community is capable of empathizing with the average person is a wild overstatement that brings nothing to the discussion. Just adds to the noise.
    - sp527 4y ago
      Late reply here but I wanted to point out that you still don’t get it. True empathy in the tech community would be e.g. having the courage to say that building HLAI of the kind we’re now approaching is guaranteed to cause tremendous amounts of suffering for ordinary people (who will not be able to respond elastically to so abrupt a tectonic shift), and therefore the whole enterprise is fundamentally evil. Let’s get real concrete about what’s going to happen: people will lose their jobs, then their homes, they’ll become destitute, they’ll experience divorces, some will commit suicide, they will suffer desperately in myriad other ways due to economic disenfranchisement, kids will be deprived of a comfortable upbringing, etc. How many in the tech industry are genuinely discussing the very real consequences of nonlinear degrees of automation for the kinds of ordinary people they barely interact with? How many are pretending that there isn’t something disgustingly immoral about having some of the most affluent and economically insulated people devise and inflict this reality upon countless millions? I will maintain that this industry is morally bankrupt and nearly entirely devoid of empathy. These are not the people who should be in charge of our future.
gordian-mind 4y ago
[flagged]
pdonis 4y ago
> a world in which only certain non-renewable scarce resources have any value If you think this is the world "AI" is leading us toward, I think you need to think again. There are a huge range of skills that only humans can provide, and that will continue to be true. GPT-3 can't fix your plumbing or your HVAC when it breaks. It can't fix your car. It can't build houses. It can't build roads. It can't build computers. And on and on and on... Nor are AI enthusiasts trying to get "AI" to do any of these things. Humans are real-world creatures with real-world needs that we supply to each other by free markets, specialization, and trade. The "tech community" might take all this for granted, but that doesn't mean it just happens by magic. It happens because real humans do real work. "AI" is not going to take that over any time soon.
- sp527 4y ago
  What do you think happens when AI compresses the economy and leaves us almost exclusively with manual labor jobs, most of which are already low-paying? And you may not be aware of this having never done such a job, but physical labor is often brutally taxing on the body, putting people at risk of a wide array of potential health complications. I've seen this abjectly atrocious argument about manual labor made by a number of people on HN and it's astounding how poorly thought through it is.
  - pdonis 4y ago
    > when AI compresses the economy How? How will it do this? Basically, people seem to assume that "AI" has some kind of magical power to do whatever bad thing they can imagine, and then they extrapolate from there. I don't see it. > physical labor If you think the jobs I described are purely "physical labor", or that the physical labor involved is so highly taxing that people can only do those jobs for a short time before they wear out, you definitely need to think again. (You also definitely need to think again if you think those jobs are low paying. Plenty of people make quite a healthy living doing them.)
    - sp527 4y ago
      > You also definitely need to think again if you think those jobs are low paying. Plenty of people make quite a healthy living doing them. Will they still make a healthy living when there's an influx of laborers fleeing more automated parts of the economy? GS just put out a report today estimating the impact of AI: https://www.businessinsider.com/generative-ai-chatpgt-300-million-full-time-jobs-goldman-sachs-2023-3 https://www.businessinsider.com/generative-ai-chatpgt-300-mi.... I wonder sometimes if these accounts on HN making insane arguments that generative AI somehow won't be economically calamitous are bots. In fact, if I was at OpenAI and the goal was to avert scrutiny long enough to get to AGI, unleashing a torrent of AI shill bots might be near the top of the agenda.

kelseyfrog 4y ago

AGI is a break-capitalism lottery ticket I'm willing to entertain. If the marginal cost of labor collapses to zero then either we'll get robot police or finally be free to pursue lives of leisure.

cmsonger 4y ago
Except that it's going to be robot police. What in the current world order suggests labor wins. Who controls LLMs? Who has the compute to produce AGI?
- bilbo0s 4y ago
  To be fair, there is always someone on the bottom. It just historically hasn't been people we cared about. That really says more about us than it says about AI. The time to complain about these power asymmetries was prior to them affecting us. Now it just sounds like garden variety sour grape hypocrisy. Not saying there aren't good points. Only pointing out that many, many people can all view the consequences of this dynamic differently.
  - noobermin 4y ago
    The only people who view this favorably are people at the top. People near the bottom perhaps feel disdain for people in the middle but that is misdirected because people at the top always were the ones in control. AI hasn't changed that dynamic.
    - anigbrowl 4y ago
      By 'this' do you mean AI or the petition?
      - noobermin 4y ago
        AI. I am replying to the parent comment, but I too get that this petition is being signed onto by people at openAI specifically because they already made it with GPT-4. They definitely would not sign on to it if their (arbitrary) threshold for what to stop doing AI research with was a tool at a GPT-3 or GPT-3.5 level or higher or it included GPT-4 in their threshold, and not merely beyond it.

twblalock 4y ago

It really is irreversible. The methods and compute needed to build a GPT-type AI are widely available.

versteegen 4y ago
Yes, but superintelligence hasn't been created yet. That's what the petition is ultimately about, not chat assistants. Transformers unmodified aren't going to scale up to that, they're already nearly scaled maximally.
- famouswaffles 4y ago
  You don't know what transformers unmodified scale up to. Nearly scaled maximally ? Based on what exactly ?
  - versteegen 4y ago
    The degree to which transformers scale, as measured by loss (training objective) is known remarkably well! See [1]. There's a formula in there for the minimum loss you possibly achieve even with infinite compute and training data, and it's hardly less than Chinchilla's loss. The recent GPT-4 paper further reinforces that these scaling laws are real, because they predicted with high accuracy the loss the model would have based on data+compute used. Admittedly, the link between improvement in loss and improvement of capabilities may break down or be misleading. However, there's just not much training data on the internet left unused. Maybe an order of magnitude. All books ever published (in English?) are a smaller dataset than the corpora already used for training. See [2] (which includes an easy summarisation of much of [1]). And the scaling laws show training data is already the bottleneck rather than compute. [1] DeepMind, 2022, Training Compute-Optimal Large Language Models https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556 [2] Chinchilla's wild implications https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla-s-wild-implications https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla...
    - deleted 4y ago
      [deleted]

andrewmutz 4y ago

You can accuse everyone of *isms, but there are a lot of us who have just never been persuaded by the many articles written to scare us about the effects of AI. The concerns about all the jobs going away have been persistent but the labor market continues to be extremely strong. As long as entrepreneurship is creating new business opportunities and new ways to turn labor into value, it's completely possible that the effects of automation are drowned out by new applications of labor. In fact, this is what we've seen throughout the industrial revolution: constant anxiety about automation that fails to materialize in to mass unemployment. The concerns about polluting the information space are still hypothetical as well. The current information space is a complete garbage fire and it's not due to generative AI. Might it get worse if bots start spamming people with GPT generated text on social media? Yeah it might. But social media is already terrible and we need to do something about it regardless of that.

jimrandomh 4y ago
There are two, mostly disjoint groups warning about AI. There are the people worried about comparatively mundane effects from comparatively mundane systems: job loss, spam, disinformation, maybe an occasional unfair loan-application rejection. These concerns don't have nothing going for them, but in all but the worst-case-scenario versions, these just aren't bad enough to make AI not be worth it. Then there's the people looking ahead, foreseeing a future where superintelligent AIs are more powerful than humanity, and worried that most possible variations of those superintelligences are incentivized to destroy us. I think this open letter puts much too much emphasis on the petty stuff, I think because they're trying to appeal to people who are allergic to anything that requires extrapolating more than a little bit into the future. But buying more time for alignment research, before we tackle superintelligence, does meaningfully improve humanity's odds of survival, so I hope this happens anyways.
creamyhorror 4y ago
The key issue in automation transitions is the transition of affected individuals to other sources of income. In previous technological revolutions, affected workers were hurt by their loss of income, and some no doubt fell into poverty without ever recovering. Not everyone can be retrained for new types of jobs immediately - (1) they may not have the needed foundational knowledge or the cognitive flexibility/ability, and (2) there might not be enough of the new types of jobs emerging quickly enough for them. Not every displaced miner can become a coder, or be competitive for junior dev jobs. (Why should the state provide for these workers? Well, primarily for humaneness, and also social stability.) The rewards of automation (cost savings as well as profits) are reaped by (1) the capital owners of the automation technology companies (and their higher-paid employees), as well as by (2) the companies and consumers using the new automation; therefore those owners and beneficiaries could be asked to bear at least part of the costs of supporting, retraining, and placing in jobs the workers they displaced. In a nutshell: Redistribution during structural unemployment caused by technological transitions. A humane policy would provide the above types of support for workers displaced by automation. Ideally it would already be handled by existing unemployment policy, but in many countries such support is limited or minimal. Corporate taxation might need some rethinking along the lines of job-displacement effects of companies (a tricky question, I admit - I've come across one or two proposals for assessing the automation level of companies for taxation purposes). The cross-border dynamics add further complexity, given that automation will displace many jobs outsourced across borders. Given that the current AI revolution looks like it will be causing even larger and faster changes than previous revolutions, such policies are imo needed as a prerequisite (one of several) for allowing the development of powerful job-displacing AI.

fwlr 4y ago

“Sir Roger L'Estrange tell us a Story in his Collection of Fables, of the Cock [Rooster] and the Horses. The Cock was gotten to Roost in the Stable, among the Horses, and there being no Racks, or other Conveniencies for him, it seems, he was forc'd to roost upon the Ground; the Horses jostling about for room, and putting the Cock in danger of his Life, he gives them this grave Advice; Pray Gentlefolks let us stand still, for fear we should tread upon one another.” HN is scoffing because the timing of this petition is suspicious. To borrow your metaphor of “step on the gas”, this is as if the race had already started, several competitors are out ahead of the rest of the field, and now someone is yelling “step on the brakes!”.

reasonableklout 4y ago
Sam Altman has publicly talked about wanting more regulation. Demis Hassabis stated that AI safety was one of the _motivations_ for founding DeepMind. I don't think any of the industry noise around safety and regulation is new, I think the timing of the petition is relevant _because_ the race has started in an especially messy way (with Llama's weights leaking, Bing/Sydney threatening a journalist based on tweets it found on the internet, various OpenAI security mishaps, etc).

versteegen 4y ago

It's striking how on AGI, the common opinion here on HN is the complete opposite to the one in the AI alignment field. There, people are angry with OpenAI for being too open, here people are angry with it not being open enough. For example, Eliezer Yudkowsky: > Ryan: OpenAI [...] It's kind of a completely open-source AI project? > Eliezer: That was the original suicidal vision, yes. But... > Ryan: And now they're bent on commercializing the technology, is that right? > Eliezer: That's an improvement, but not enough of one, because they're still generating lots of noise and hype and directing more resources into the field, and storming ahead with the safety that they have instead of the safety that they need, and setting bad examples. > [...] I would change the name to "ClosedAI". Because right now, they're making it look like being "closed" is hypocrisy. They're, like, being "closed" while keeping the name "OpenAI", and that itself makes it looks like closure is like not this thing that you do cooperatively so that humanity will not die, but instead this sleazy profit-making thing that you do while keeping the name “OpenAI”. https://www.lesswrong.com/posts/e4pYaNt89mottpkWZ/yudkowsky-on-agi-risk-on-the-bankless-podcast https://www.lesswrong.com/posts/e4pYaNt89mottpkWZ/yudkowsky-...

BobbyJo 4y ago

I personally scoff at this kind of stuff because it's asking for something no one has the power to give. It's like asking to stop the development of nukes in the 40s and 50s, it's just not gonna happen.

jimrandomh 4y ago
Preventing the development of nukes entirely was obviously not going to happen. But delaying the first detonations by a few years, and moving the Partial Nuclear Test Ban treaty up a few years, was quite achievable. Whether delaying AI development a little matters depends on whether you think the success of AI alignment, applied to future superintelligence, is overdetermined to succeed, overdetermined to fail, or close to borderline. Personally I think it looks borderline, so I'm glad to see things like this.
- BobbyJo 4y ago
  I'm firmly in the camp that delaying it's development could make a difference, I just don't see how that's possible. These models are relatively simple and the equipment necessary to develop them is public (and relatively cheap if we're talking about corporate or national scales). At least with nukes, there was a raw material bottleneck, but there really isn't a limiting factor here that any "good guys" could choke point. It's out there and it's going to get worked on, and the only people the "good guys" can limit are themselves.
paxys 4y ago
And during that period, and later during the cold war, the decision to make (or stop making) nukes was in the hands of maybe 5 people total. Today there are thousands of companies and tens/hundreds of thousands of people who can legitimately compete in the space. Best of luck trying to resolve a prisoner's dilemma between all of them.
G_z9 4y ago
It actually very easily could have happened and almost did happen but the Russians decided to go back on their effort to do it. It really did almost happen. People act like it’s hard. Stopping a huge asteroid is hard. There might not be enough physical resources to do it. Stopping AI or nukes is definitely easy.

anigbrowl 4y ago

I'm very cognizant of the risks, but the biggest risk seems to me that wealthy people hold it back to get an uncrossable political-economic moat in order to permanently secure their own position.

kvetching 4y ago
exactly. Specifically China. They've said for years, they want to be the global AI leader. They have more data than anyone. Now that they know LLMs work as good as they do, they are going all in. They achieve AGI first, that's bad. Communist AGI is a greater existential risk than us creating AGI. We must create it first so we can use it to harden our systems.

Veen 4y ago

I think the "good old elitism" is exactly what the alignment folks are expressing. They think only they and large corporations should have these tools, while we, the common folk, should not.

ChatGTP 4y ago
Hanlon's Razor...Max Tegmark has been saying for a long time that his biggest concern would be black box models we don't understand gaining more intelligence without a solid understand about how they work ...this is long before ChatGPT4 existed so I don't believe it has anything to do with control and keeping people out of anything. Even Sam Altman has said similar things recently that he believes we're probably going to have to slow down and think about what we're doing. I think a more advanced civilization would slow down and continue AI research on another planet (Mars?)...not here on Earth.

CatWChainsaw 4y ago

Eh, just typical HN superiority complex on show. Personally when we don't slow down and we undermine civilization to a shadow of its current self, I'm going to make sure my last web post is here, and all it's going to read is "Told you. :) "