4 ms·

I am disappointed with increasing frequency to learn how certain individuals whom I previously believed to be serious scholars fall for the cargo cult of AI saf

I am disappointed with increasing frequency to learn how certain individuals whom I previously believed to be serious scholars fall for the cargo cult of AI safety. Personally I find it difficult to believe they are willfully participating in such conartistry, and makes me question their views on other subjects. Although I suppose intelligence and gullibility can coexist and some forms of delusion are better pitied than scorned.

sebzim4500 3y ago
Why is your prior belief that AI safety is a cult so strong that even multiple people you would rather assume that multiple people who you previously respected are now delusional than to consider that there might be arguments worth considering?
- bmc7505 3y ago
  It’s a LARP at best and a scam at worst. I shared some of my thoughts on the matter in a prior thread: https://news.ycombinator.com/item?id=35145189#35147288 https://news.ycombinator.com/item?id=35145189#35147288
  - sebzim4500 3y ago
    I don't understand the relevance of the prior thread, except that you made the same claim with similarly scant evidence. What exactly is it that makes you so confident?
    - bmc7505 3y ago
      The problems that AI will manifest are the result of human ambition and failings, no different as any other technology that empowers individuals. Yes, individuals and organizations will misuse AI for immoral purposes, but the popular belief that AI is inherently antihumanist launders accountability by pretending to remove human agency from the equation. How we use or misuse AI technology is entirely on us. Furthermore, I would argue there are strong complexity-theoretic bottlenecks to computational processes which limit the expressive power of neural networks, even if they could harness galactic amounts of energy. Physical Turing machines have bottlenecks that upper-bound the power of oracles.
      - sebzim4500 3y ago
        > The problems that AI will manifest are the result of human ambition and failings, no different as any other technology that empowers individuals. Yes, individuals and organizations will misuse AI for immoral purposes, but the popular belief that AI is inherently antihumanist launders accountability by pretending to remove human agency from the equation. How we use or misuse AI technology is entirely on us. So what? If it kills us, we're still dead. > I would argue there are strong complexity-theoretic bottlenecks to computational processes which limit the expressive power of neural networks Of course there are physics/CS limits to how intelligent something can be in a given volume, but those limits are vastly higher than our own so I don't think they are particularly relevant. For instance, a system which could simulate the brains of a thousand scientists as smart as Einstein a billion times faster than realtime would not violate any rules of physics, even though it is far beyond our current capabilities.
        bmc7505 3y ago
        Although I think their hearts are in the right place, AI safety researchers are primarily driven by irrational instincts and misjudge the promise and perils of artificial intelligence. If humanity chooses to turn away from God and sacrifice each other worshipping false idols, that will be our fault alone, whether or not the technology exists to hasten our demise. We do have thousands of Einsteins today wielding untold resources, but the slowing pace of scientific progress suggests there are limits to scaling intelligence. Even with a hundredfold increase in scientists, I am unconvinced that would lead to a meaningful increase in social progress and have come to believe the bottlenecks we face are not due to a lack of intelligence, but a lack of other virtues (e.g., kindness, curiosity, courage, compassion, perseverance).
        sebzim4500 3y ago
        For the record, if you had told me earlier in the conversation that your faith in your convictions arise from your religious faith you would have saved us both a fair amount of time. Obviously, your arguments are never going to be convincing to someone who does not share your beliefs.

sebzim4500 3y ago

Why is your prior belief that AI safety is a cult so strong that even multiple people you would rather assume that multiple people who you previously respected are now delusional than to consider that there might be arguments worth considering?

bmc7505 3y ago
It’s a LARP at best and a scam at worst. I shared some of my thoughts on the matter in a prior thread: https://news.ycombinator.com/item?id=35145189#35147288 https://news.ycombinator.com/item?id=35145189#35147288
- sebzim4500 3y ago
  I don't understand the relevance of the prior thread, except that you made the same claim with similarly scant evidence. What exactly is it that makes you so confident?
  - bmc7505 3y ago
    The problems that AI will manifest are the result of human ambition and failings, no different as any other technology that empowers individuals. Yes, individuals and organizations will misuse AI for immoral purposes, but the popular belief that AI is inherently antihumanist launders accountability by pretending to remove human agency from the equation. How we use or misuse AI technology is entirely on us. Furthermore, I would argue there are strong complexity-theoretic bottlenecks to computational processes which limit the expressive power of neural networks, even if they could harness galactic amounts of energy. Physical Turing machines have bottlenecks that upper-bound the power of oracles.
    - sebzim4500 3y ago
      > The problems that AI will manifest are the result of human ambition and failings, no different as any other technology that empowers individuals. Yes, individuals and organizations will misuse AI for immoral purposes, but the popular belief that AI is inherently antihumanist launders accountability by pretending to remove human agency from the equation. How we use or misuse AI technology is entirely on us. So what? If it kills us, we're still dead. > I would argue there are strong complexity-theoretic bottlenecks to computational processes which limit the expressive power of neural networks Of course there are physics/CS limits to how intelligent something can be in a given volume, but those limits are vastly higher than our own so I don't think they are particularly relevant. For instance, a system which could simulate the brains of a thousand scientists as smart as Einstein a billion times faster than realtime would not violate any rules of physics, even though it is far beyond our current capabilities.
      - bmc7505 3y ago
        Although I think their hearts are in the right place, AI safety researchers are primarily driven by irrational instincts and misjudge the promise and perils of artificial intelligence. If humanity chooses to turn away from God and sacrifice each other worshipping false idols, that will be our fault alone, whether or not the technology exists to hasten our demise. We do have thousands of Einsteins today wielding untold resources, but the slowing pace of scientific progress suggests there are limits to scaling intelligence. Even with a hundredfold increase in scientists, I am unconvinced that would lead to a meaningful increase in social progress and have come to believe the bottlenecks we face are not due to a lack of intelligence, but a lack of other virtues (e.g., kindness, curiosity, courage, compassion, perseverance).
        sebzim4500 3y ago
        For the record, if you had told me earlier in the conversation that your faith in your convictions arise from your religious faith you would have saved us both a fair amount of time. Obviously, your arguments are never going to be convincing to someone who does not share your beliefs.