7 ms·

This "pulling balls from an urn" analogy at the beginning is very misguided. Where is the agency in this analogy? If you build tools with the intention of them

This "pulling balls from an urn" analogy at the beginning is very misguided. Where is the agency in this analogy? If you build tools with the intention of them directly being useful to people, they're likely to be useful. If you build tools to be parasitic (i.e. useful to people who are trying to benefit at the expense of others), they're likely to be parasitic. It's quite simple. If, on the other hand, you try to "innovate" with no clear goal in mind (like much of the current "AGI" research), who knows. But there is no law of nature that says that innovation that is not explicitly designed to be beneficial will magically be beneficial. This is actually a common myth in silicon valley: that technology is somehow beneficial by default. But again, there is no law of nature that makes this myth true. First of all, technology is not one unified thing but an organic process of individual people building individual things. Second of all, the common explanation people give for this myth ("would you rather live 200 years ago?") is in no way a proof of technology's by-default beneficialness. Even if it is true that it's better to live today than 200 years ago (which is unclear), it's still possible that that is so because early innovators were focused on building useful things (and perhaps less money-motivated), whereas now we may or may not be as focused on building useful things. Bostrom's brand of fear mongering does not appear to be based in science, logic, or basic common sense. I mean, a lot of world issues are obvious to the naked eye: overpopulation, consumption, housing prices, destruction of natural habitats. Filtering to just "AI"-related world issues: addictive and distracting digital products powered in-part by ML or people losing jobs due to ML-based automation. But things that Bostrom speaks of, like superintelligence, are, at the very least, not nearly as obvious as the aforementioned concerns, which makes me question his sincerity. Furthermore, it's precisely fear-mongering that drives the development of potentially dangerous technologies like weapons.

deleted 4y ago
[deleted]
trashtester 4y ago
> This "pulling balls from an urn" analogy at the beginning is very misguided. Where is the agency in this analogy? If you build tools with the intention of them directly being useful to people, they're likely to be useful. If you build tools to be parasitic (i.e. useful to people who are trying to benefit at the expense of others), they're likely to be parasitic. It's quite simple. I find this very naiive, as this at best only applies on a short timescale and for the immediate effect. History is full of inventions that were created to help people, that is then quickly turned into something harmful, and vice versa. Also, a lot of "progress" has come as side effects of pure curiosity, espcially within science. I don't think anyone would be able to build modern electronic computers, if it wasn't for some other people a few decades earlier doing basic research in fields such as electromagnetism, quantum physics or chemistry.
- iroh2727 4y ago
  I agree with your point about scientific breakthroughs coming via pure curiosity. I’m just saying that we can’t forget the human agency at play here. We didn’t have to build the atom bomb, and it wasn’t exactly simple to turn insights from physics into the atom bomb (it took a team of the best scientists and technologists many years). Also, we can influence the direction of how technology is used. Even setting up the atom bomb to be used involves a ton of decisions and implementation that can be questioned and molded by individual people. Essentially I believe that fear of powerful technology comes from an irrational distrust in people as incompetent or malicious. Same with desire for powerful technology to control or replace people (e.g. algocracy). It seems to reflect a lot of the rhetoric and motivations of the eugenics movement. But the problem is, it can perhaps be a self-fulfilling prophecy. People are capable of being incredibly competent if encouraged to do so. So I think we should go in that direction, of empowerment and freedom rather than control or replacement. And if our governing structures or institutions or culture are making it difficult to go in that direction, we should fight those.
  - trashtester 4y ago
    The Manhatten project was started after Einstein (a pacifist) urged Roosevelt that Hitler could build it first. https://www.osti.gov/opennet/manhattan-project-history/Events/1939-1942/einstein_letter.htm#:~:text=Einstein%20had%20written%20to%20inform,extremely%20powerful%20bombs%22%20was%20conceivable https://www.osti.gov/opennet/manhattan-project-history/Event.... At the time Hiroshima and Nagasaki were bombed, about 50000 Chinese people died every week due to Japanese occupation. If the alternative were an invasion, those two bombs probably saved millions of lives. https://worldpopulationreview.com/country-rankings/world-war-two-casualties-by-country https://worldpopulationreview.com/country-rankings/world-war... Von Neumann lobbied to make sure that USA/USSR would end up with a situation of Mutually Assured Destruction. He believed that would be the best way to avoid large wars between the Soviets and NATO. In all these cases the development/use/threat of nukes were done with, arguably, the best of intentions. In their minds, they were employing agency to help protect the world. I would argue that the eugenic movement was the same. There may have been some malice, but there was definitely many eugenicists that truely believed that eugenics was the only way to prevent dysgengics, followed by a malthusian collapse. One could even argue that Hitler saw himself as the hero of Germany, the one that would save Germany from enemy "tribes", foreign and domestic (jews). If so, he may have seen the Holocaust as a heoric and brave act, and it was certainly executed with extreme competency. Mao's cultural revolution, Pol Pot Cambodian genocide, and the Hutu's genocide vs the Tutsies were all very similar to the Holocaust, even if less known. In all cases, a group of Others were seen as a threat, and eliminated. In most of these cases, I think the perpetrators saw themselves as the heroes, almost like the heroes of Minas Tirith fighting the hordes of Mordor. It is extremely hard to come up with a way to live in the world that minimizes harm. And, I think, it is those who has the strongest conviction that they are the Good (TM) struggeling against Evil, that has the greatest tendency to cause harm, as they are convinced that the end justifies the means.
wildmanx 4y ago
> Bostrom's brand of fear mongering does not appear to be based in science, logic, or basic common sense. I mean, a lot of world issues are obvious to the naked eye: overpopulation, consumption, housing prices, destruction of natural habitats. I really wonder whether people commenting here have actually read the article. It's quite involved, I've already spent a few hours and am far from through. It's fair to assume that most people have spent less time. Your comment is not very much related to the type of events that he is describing in the article. How are housing prices even remotely a technological development leading to devastation of civilization? If anything they are just an artifact of a particular economic incentive structure.

deleted 4y ago

[deleted]

trashtester 4y ago

> This "pulling balls from an urn" analogy at the beginning is very misguided. Where is the agency in this analogy? If you build tools with the intention of them directly being useful to people, they're likely to be useful. If you build tools to be parasitic (i.e. useful to people who are trying to benefit at the expense of others), they're likely to be parasitic. It's quite simple. I find this very naiive, as this at best only applies on a short timescale and for the immediate effect. History is full of inventions that were created to help people, that is then quickly turned into something harmful, and vice versa. Also, a lot of "progress" has come as side effects of pure curiosity, espcially within science. I don't think anyone would be able to build modern electronic computers, if it wasn't for some other people a few decades earlier doing basic research in fields such as electromagnetism, quantum physics or chemistry.

iroh2727 4y ago
I agree with your point about scientific breakthroughs coming via pure curiosity. I’m just saying that we can’t forget the human agency at play here. We didn’t have to build the atom bomb, and it wasn’t exactly simple to turn insights from physics into the atom bomb (it took a team of the best scientists and technologists many years). Also, we can influence the direction of how technology is used. Even setting up the atom bomb to be used involves a ton of decisions and implementation that can be questioned and molded by individual people. Essentially I believe that fear of powerful technology comes from an irrational distrust in people as incompetent or malicious. Same with desire for powerful technology to control or replace people (e.g. algocracy). It seems to reflect a lot of the rhetoric and motivations of the eugenics movement. But the problem is, it can perhaps be a self-fulfilling prophecy. People are capable of being incredibly competent if encouraged to do so. So I think we should go in that direction, of empowerment and freedom rather than control or replacement. And if our governing structures or institutions or culture are making it difficult to go in that direction, we should fight those.
- trashtester 4y ago
  The Manhatten project was started after Einstein (a pacifist) urged Roosevelt that Hitler could build it first. https://www.osti.gov/opennet/manhattan-project-history/Events/1939-1942/einstein_letter.htm#:~:text=Einstein%20had%20written%20to%20inform,extremely%20powerful%20bombs%22%20was%20conceivable https://www.osti.gov/opennet/manhattan-project-history/Event.... At the time Hiroshima and Nagasaki were bombed, about 50000 Chinese people died every week due to Japanese occupation. If the alternative were an invasion, those two bombs probably saved millions of lives. https://worldpopulationreview.com/country-rankings/world-war-two-casualties-by-country https://worldpopulationreview.com/country-rankings/world-war... Von Neumann lobbied to make sure that USA/USSR would end up with a situation of Mutually Assured Destruction. He believed that would be the best way to avoid large wars between the Soviets and NATO. In all these cases the development/use/threat of nukes were done with, arguably, the best of intentions. In their minds, they were employing agency to help protect the world. I would argue that the eugenic movement was the same. There may have been some malice, but there was definitely many eugenicists that truely believed that eugenics was the only way to prevent dysgengics, followed by a malthusian collapse. One could even argue that Hitler saw himself as the hero of Germany, the one that would save Germany from enemy "tribes", foreign and domestic (jews). If so, he may have seen the Holocaust as a heoric and brave act, and it was certainly executed with extreme competency. Mao's cultural revolution, Pol Pot Cambodian genocide, and the Hutu's genocide vs the Tutsies were all very similar to the Holocaust, even if less known. In all cases, a group of Others were seen as a threat, and eliminated. In most of these cases, I think the perpetrators saw themselves as the heroes, almost like the heroes of Minas Tirith fighting the hordes of Mordor. It is extremely hard to come up with a way to live in the world that minimizes harm. And, I think, it is those who has the strongest conviction that they are the Good (TM) struggeling against Evil, that has the greatest tendency to cause harm, as they are convinced that the end justifies the means.

wildmanx 4y ago

> Bostrom's brand of fear mongering does not appear to be based in science, logic, or basic common sense. I mean, a lot of world issues are obvious to the naked eye: overpopulation, consumption, housing prices, destruction of natural habitats. I really wonder whether people commenting here have actually read the article. It's quite involved, I've already spent a few hours and am far from through. It's fair to assume that most people have spent less time. Your comment is not very much related to the type of events that he is describing in the article. How are housing prices even remotely a technological development leading to devastation of civilization? If anything they are just an artifact of a particular economic incentive structure.