4 ms·

No, my position is that you can not extrapolate on something that does not exist and that’s “category” ever never existed. We don’t know any significant other i

by i5heu 3y ago

No, my position is that you can not extrapolate on something that does not exist and that’s “category” ever never existed. We don’t know any significant other intelligent then humans, not even aliens, so we have just no trace of direction that is rooted in reality rather then fantasy. This is especially obvious since opinion differ so much about the effects of AGI on humanity and since you cannot prove them right or wrong, every opinion is equally realistic as each other. Like I could say that every AGI that becomes self aware will kill itself instantly because of boringness, and you cannot prove me wrong. We just don’t know yet.

mitthrowaway2 3y ago
It still seems to me that you're saying that we won't possibly be able to declare AGI an existential threat to humanity until after it has already been built. At that point, we can presumably settle the question by seeing if humanity goes extinct. This poses something of a paradox to those of us who prefer existential threats to humanity not get built in the first place.
- i5heu 3y ago
  We will not know if something is an existential threat until we can extrapolate on at least some data that is relevant. Feeling uncomfortable with this, will not change this sad reality.
  - mitthrowaway2 3y ago
    Given that you don't think the CEO of Anthropic is qualified to comment on this matter, what kind of "data that is relevant" do you imagine we might be able to have, prior to an AGI existing? I am indeed uncomfortable with someone saying that we can't worry about a gas leak until after someone strikes a match and notices an explosion or the lack thereof. If the sad reality is that we can't detect the gas leak any other way, I would argue that the precautionary principle suggests we ban the striking of matches until such sensors have been developed. But here we are in a room of people grinding up saltpetre with their mortars and pestles.
    - i5heu 3y ago
      I really wonder why there is such a fixation with these CEOs. I have proven to you logically that they cannot have more knowledge then anyone else, but you continue to present their authority as knowledge. Pls prove to me that they know more about the effects of AGI on humanity or that they have some magical knowledge we normal ppl are unable to comprehend. If you cannot do this, then why are you bringing up these ppl?
      - mitthrowaway2 3y ago
        I wasn't the one who brought that person up. I only kept with that example because you didn't answer my question when I asked you what would qualify someone to have an opinion, or what evidence you imagined could exist, and I didn't want to put words in your mouth by assuming that you also think that eg. Yoshua Bengio or Geoff Hinton or Max Tegmark (or take your pick of anybody) don't have informed opinions either. Or that they have well-reasoned arguments, rather than authoritative opinions, if you think that reason is applicable. I'm only asking for a standard that falls slightly short of "already having built Schrodinger's doomsday weapon and observed whether humanity dies as a result". You can set the bar wherever else you like, please!
      - Gud 3y ago
        This is not about “having knowledge “, but about being able to anticipate future problems. Considering how rapidly the technology is developing, and considering how clearly a threat a superior intelligence will be to the human race, it is absolutely the time to have this discussion NOW
      - ben_w 3y ago
        > I have proven to you logically that they cannot have more knowledge then anyone else, but you continue to present their authority as knowledge. You've asserted it, not proven it. Given their entire research field is "how to make the thing", it's a fair bet that they know more than me — even if what they know is, to paraphrase Thomas Edison, "1000 ways to not make an AGI", that's still more than me, and I follow some of the research and read some of the papers.
- ToucanLoucan 3y ago
  We don't even know if AGI is possible. Let's not mince words here: nothing, and I do mean nothing, not a single, solitary model on offer by anyone, anywhere, right now has a prayer of becoming AGI. That's just... not how it's going to be done. What we have right now are fantastically powerful, interesting, and neato to play with pattern recognition programs, that can take their understanding of patterns, and reproduce more of them given prompts. That's it. That is not general intelligence of any sort, it's not even really creative, it's analogous to creative output but it isn't outputting to say something, it's simply taking samples of all the things it's seen previously and making something new with as few "errors" as possible, whatever that means in context. This is not intelligence of any sort, period, paragraph. I don't know to what extent OpenAI and their compatriots are actually trying to bring forth artificial life (or at least, consciousness) forward, versus how much they're just banking on how cool AI is as a term in order to funnel even more money to themselves chasing the pipe dream of building it, and at this point, I don't care. Even the products they have made do not hold a candle to the things they claim to be trying to make, but they're more than happy to talk them up like there's a chance. And I suppose there is a chance, but I really struggle to see ChatGPT turning into skynet.
  - mitthrowaway2 3y ago
    > We don't even know if AGI is possible. My response to a very similar statement is here: https://news.ycombinator.com/item?id=39612641 https://news.ycombinator.com/item?id=39612641 > That's just... not how it's going to be done. I agree. I think there are some critical ingredients missing. Obviously the weights need to be able to update to new data in a semi-online fashion, for example. But I think there are less ingredients missing than there were a decade ago, by a much larger factor than I expected at the time, and a decade from now, the number of missing ingredients might be zero. My uncertainty on this is not because I think GPT is more than it seems,but because it's unclear how much of the amazing cognitive capability that humans have is, under the hood, much simpler than it seems. I really want humanity to exist even ten decades from now; I want my kids to have grandkids. So I care about this even if I don't expect AGI from a 2025 Q1 product launch. And I don't think it's too early to worry about it, just like I don't think 1980 was too early to worry about climate change.
- parineum 3y ago
  I think you're interpretating the statement as "nobody can have an opinion" from "AI programming doesn't give someone a more qualified opinion on AI". Plenty of people have interesting or valuable opinions and insights on AGI, programming LLMs doesn't have any impact into that though. It'd be like expecting the developers of friendster to have a good idea of what the impact of social media was going to be in 2020. AI isn't AGI.

by i5heu 3y ago

mitthrowaway2 3y ago

It still seems to me that you're saying that we won't possibly be able to declare AGI an existential threat to humanity until after it has already been built. At that point, we can presumably settle the question by seeing if humanity goes extinct. This poses something of a paradox to those of us who prefer existential threats to humanity not get built in the first place.

i5heu 3y ago
We will not know if something is an existential threat until we can extrapolate on at least some data that is relevant. Feeling uncomfortable with this, will not change this sad reality.
- mitthrowaway2 3y ago
  Given that you don't think the CEO of Anthropic is qualified to comment on this matter, what kind of "data that is relevant" do you imagine we might be able to have, prior to an AGI existing? I am indeed uncomfortable with someone saying that we can't worry about a gas leak until after someone strikes a match and notices an explosion or the lack thereof. If the sad reality is that we can't detect the gas leak any other way, I would argue that the precautionary principle suggests we ban the striking of matches until such sensors have been developed. But here we are in a room of people grinding up saltpetre with their mortars and pestles.
  - i5heu 3y ago
    I really wonder why there is such a fixation with these CEOs. I have proven to you logically that they cannot have more knowledge then anyone else, but you continue to present their authority as knowledge. Pls prove to me that they know more about the effects of AGI on humanity or that they have some magical knowledge we normal ppl are unable to comprehend. If you cannot do this, then why are you bringing up these ppl?
    - mitthrowaway2 3y ago
      I wasn't the one who brought that person up. I only kept with that example because you didn't answer my question when I asked you what would qualify someone to have an opinion, or what evidence you imagined could exist, and I didn't want to put words in your mouth by assuming that you also think that eg. Yoshua Bengio or Geoff Hinton or Max Tegmark (or take your pick of anybody) don't have informed opinions either. Or that they have well-reasoned arguments, rather than authoritative opinions, if you think that reason is applicable. I'm only asking for a standard that falls slightly short of "already having built Schrodinger's doomsday weapon and observed whether humanity dies as a result". You can set the bar wherever else you like, please!
    - Gud 3y ago
      This is not about “having knowledge “, but about being able to anticipate future problems. Considering how rapidly the technology is developing, and considering how clearly a threat a superior intelligence will be to the human race, it is absolutely the time to have this discussion NOW
    - ben_w 3y ago
      > I have proven to you logically that they cannot have more knowledge then anyone else, but you continue to present their authority as knowledge. You've asserted it, not proven it. Given their entire research field is "how to make the thing", it's a fair bet that they know more than me — even if what they know is, to paraphrase Thomas Edison, "1000 ways to not make an AGI", that's still more than me, and I follow some of the research and read some of the papers.
ToucanLoucan 3y ago
We don't even know if AGI is possible. Let's not mince words here: nothing, and I do mean nothing, not a single, solitary model on offer by anyone, anywhere, right now has a prayer of becoming AGI. That's just... not how it's going to be done. What we have right now are fantastically powerful, interesting, and neato to play with pattern recognition programs, that can take their understanding of patterns, and reproduce more of them given prompts. That's it. That is not general intelligence of any sort, it's not even really creative, it's analogous to creative output but it isn't outputting to say something, it's simply taking samples of all the things it's seen previously and making something new with as few "errors" as possible, whatever that means in context. This is not intelligence of any sort, period, paragraph. I don't know to what extent OpenAI and their compatriots are actually trying to bring forth artificial life (or at least, consciousness) forward, versus how much they're just banking on how cool AI is as a term in order to funnel even more money to themselves chasing the pipe dream of building it, and at this point, I don't care. Even the products they have made do not hold a candle to the things they claim to be trying to make, but they're more than happy to talk them up like there's a chance. And I suppose there is a chance, but I really struggle to see ChatGPT turning into skynet.
- mitthrowaway2 3y ago
  > We don't even know if AGI is possible. My response to a very similar statement is here: https://news.ycombinator.com/item?id=39612641 https://news.ycombinator.com/item?id=39612641 > That's just... not how it's going to be done. I agree. I think there are some critical ingredients missing. Obviously the weights need to be able to update to new data in a semi-online fashion, for example. But I think there are less ingredients missing than there were a decade ago, by a much larger factor than I expected at the time, and a decade from now, the number of missing ingredients might be zero. My uncertainty on this is not because I think GPT is more than it seems,but because it's unclear how much of the amazing cognitive capability that humans have is, under the hood, much simpler than it seems. I really want humanity to exist even ten decades from now; I want my kids to have grandkids. So I care about this even if I don't expect AGI from a 2025 Q1 product launch. And I don't think it's too early to worry about it, just like I don't think 1980 was too early to worry about climate change.
parineum 3y ago
I think you're interpretating the statement as "nobody can have an opinion" from "AI programming doesn't give someone a more qualified opinion on AI". Plenty of people have interesting or valuable opinions and insights on AGI, programming LLMs doesn't have any impact into that though. It'd be like expecting the developers of friendster to have a good idea of what the impact of social media was going to be in 2020. AI isn't AGI.