5 ms·

Seems like the authors do not consider how safe such a system would be. Yet it seems to me that a machine whose main goals would be to learn things about the w

Seems like the authors do not consider how safe such a system would be. Yet it seems to me that a machine whose main goals would be to learn things about the world so that it can communicate with humans and answer their questions, is possibly very dangerous. While reading the first section I kept thinking about this blog entry from Paul Tyma, "How Artificial Intelligence Will Really Kill Us All": http://paultyma.blogspot.fr/2015/10/how-artificial-intelligence-will-really.html http://paultyma.blogspot.fr/2015/10/how-artificial-intellige...

morgante 11y ago
Interesting entry, but I don't find it very credible. We already have superpowered weapons which make it possible for a few powerful individuals to obliterate the rest of humanity, yet thus far we've stayed alive.
- grondilu 11y ago
  Recently Max Tegmark talked at the UN (1) about the potential danger of AI, and he asked what would have happened if making a nuclear bomb was as simple as say baking sand in a micro-wave. He argues that if that was the case most of us would probably be dead. I guess the point is that in these matters, the past is not always a good indicator of the future. 1. http://www.youtube.com/watch?v=W9N_Fsbngh8&t=1h55m31s http://www.youtube.com/watch?v=W9N_Fsbngh8&t=1h55m31s
  - morgante 11y ago
    Interesting analogy, but I think it's rather debatable. We have no evidence that making a strong AI is as simple as baking sand in a microwave—in fact, we have quite a bit of evidence that it's actually extremely difficult. It's unlikely to think that such a powerful and challenging piece of software would have any wider distribution than nuclear weapons. I think this is affirmed by the fact that most state-of-the-art AI depends primarily on massive volumes of data rather than clever algorithms. Massive datasets are inherently concentrated in the hands of the few and powerful, so it seems unlikely that a rogue actor could acquire suitable data.
    - grondilu 11y ago
      You take the analogy too literally. The contingency is not on whether or not making AI is easy, it's about whether or not it is harmful. And yes, it is debatable, but as Tegmark puts it: relying on luck is probably not a good strategy.
      - robotresearcher 11y ago
        What is the point of the sand-microwave analogy? I really don't get it. It can't be about microprocessors, since we've had those for decades and don't have murderous AI yet.
        JoshTriplett 11y ago
        > What is the point of the sand-microwave analogy? I really don't get it. The point is to suggest something that nobody would do by accident, but almost anyone can do intentionally, and nobody can stop others from doing. It's a thought experiment: suppose that taking some common everyday substance, and using it with a common appliance many people have access to, would destroy the world. Right now, a very small handful of people have the ability to existentially threaten humanity. Fortunately, those people seem moderately unlikely to do so, and the process for becoming one of those people selects at least somewhat against the type of craziness that would hit the button just to see what happens, or because they genuinely want the world to end. Out of seven billion people, based on existing examples of human behavior, it seems exceedingly likely that there exist people who would intentionally choose to to destroy the world.
        robotresearcher 11y ago
        Thanks. I am enlightened.
        argonaut 11y ago
        Strong AI is not something that almost anyone can do intentionally, given that it's currently impossible. And any predictions on it becoming possible within a definite timeframe are total bullshit [1]. I think it's rather telling that the only people who make those kinds of optimistic predictions about neural networks do not have extensive research experience in / are experts in artificial neural networks. [1] I certainly think it will become possible, but the timeframe is indefinite. If we get really lucky and breakthrough after breakthrough is achieved, it could be within 10 years. But it could be 20 years, it could be 100 years.
  - RockyMcNuts 11y ago
    "Moore's Law of Mad Science: Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." - Eliezer Yudkowsky On a long enough timeline, we're probably screwed by Fermi's Great Filter regardless, but a self-improving machine learning singularity can certainly accelerate the process!
    - Moshe_Silnorin 11y ago
      AI seems to be at odds with the great filter. One would expect it to expand out into the universe and be visible.
      - deleted 11y ago
        [deleted]
      - varelse 11y ago
        Why... So... Visible?
        varelse 11y ago
        Really, you guys believe technology sufficiently advanced to be indistinguishable from magic (h/t to Arthur C. Clarke) will be festooned with the alien tech equivalent of pop-up ads and banners easily detectable by puny earthlings? Allow me to disagree.
emiliobumachar 11y ago
An interesting assessment of safety considerations is: http://intelligence.org/files/AIPosNegFactor.pdf http://intelligence.org/files/AIPosNegFactor.pdf I've submitted it for HN discussion: https://news.ycombinator.com/item?id=10637207 https://news.ycombinator.com/item?id=10637207
vdaniuk 11y ago
Mutually assured destruction doctrine is possibly very dangerous, too. Also: Biological weapons, destruction of rainforest ecosystems, ocean acidification, etc etc. Why these discussions don't consider opportunity costs of NOT developing AI as fast as possible? Most people live in appalling, inhuman conditions: $100 monthly income puts a person in the top 52.40% earners worldwide. [0] $300 monthly income puts a person in the top 28.35% earners worldwide. [0] Human intelligence is obviously suboptimal for solving global problems, perhaps AI will do better. [0] http://www.globalrichlist.com/ http://www.globalrichlist.com/
- logingone 11y ago
  $100 monthly income puts a person in the top 52.40% earners worldwide. That's misleading without considering numerous other variables - cost of things? It seems to me to be the kind of statement aimed at deceitfully triggering emotions, by the likes of charities. Charities are like police - there for good reason, but along with the good intentions they have a slightly seedy side. Of the countries I've lived in, the one where I was paid least had the highest standard of living.
  - vdaniuk 11y ago
    Actually, it's other way around. Your comment is misleading and your intentions have not-so-slightly seedy side. Are you really attacking a concept of charity organization in a capitalistic world? Calculation methodology is available on the linked page: "For currency conversion we use Purchasing Power Parity Dollars (PPP$) in order to take into account the difference in cost of living between countries; PPP$ are also less susceptible to short term fluctuations."
    - logingone 11y ago
      "there for good reason"

morgante 11y ago

Interesting entry, but I don't find it very credible. We already have superpowered weapons which make it possible for a few powerful individuals to obliterate the rest of humanity, yet thus far we've stayed alive.

grondilu 11y ago
Recently Max Tegmark talked at the UN (1) about the potential danger of AI, and he asked what would have happened if making a nuclear bomb was as simple as say baking sand in a micro-wave. He argues that if that was the case most of us would probably be dead. I guess the point is that in these matters, the past is not always a good indicator of the future. 1. http://www.youtube.com/watch?v=W9N_Fsbngh8&t=1h55m31s http://www.youtube.com/watch?v=W9N_Fsbngh8&t=1h55m31s
- morgante 11y ago
  Interesting analogy, but I think it's rather debatable. We have no evidence that making a strong AI is as simple as baking sand in a microwave—in fact, we have quite a bit of evidence that it's actually extremely difficult. It's unlikely to think that such a powerful and challenging piece of software would have any wider distribution than nuclear weapons. I think this is affirmed by the fact that most state-of-the-art AI depends primarily on massive volumes of data rather than clever algorithms. Massive datasets are inherently concentrated in the hands of the few and powerful, so it seems unlikely that a rogue actor could acquire suitable data.
  - grondilu 11y ago
    You take the analogy too literally. The contingency is not on whether or not making AI is easy, it's about whether or not it is harmful. And yes, it is debatable, but as Tegmark puts it: relying on luck is probably not a good strategy.
    - robotresearcher 11y ago
      What is the point of the sand-microwave analogy? I really don't get it. It can't be about microprocessors, since we've had those for decades and don't have murderous AI yet.
      - JoshTriplett 11y ago
        > What is the point of the sand-microwave analogy? I really don't get it. The point is to suggest something that nobody would do by accident, but almost anyone can do intentionally, and nobody can stop others from doing. It's a thought experiment: suppose that taking some common everyday substance, and using it with a common appliance many people have access to, would destroy the world. Right now, a very small handful of people have the ability to existentially threaten humanity. Fortunately, those people seem moderately unlikely to do so, and the process for becoming one of those people selects at least somewhat against the type of craziness that would hit the button just to see what happens, or because they genuinely want the world to end. Out of seven billion people, based on existing examples of human behavior, it seems exceedingly likely that there exist people who would intentionally choose to to destroy the world.
        robotresearcher 11y ago
        Thanks. I am enlightened.
        argonaut 11y ago
        Strong AI is not something that almost anyone can do intentionally, given that it's currently impossible. And any predictions on it becoming possible within a definite timeframe are total bullshit [1]. I think it's rather telling that the only people who make those kinds of optimistic predictions about neural networks do not have extensive research experience in / are experts in artificial neural networks. [1] I certainly think it will become possible, but the timeframe is indefinite. If we get really lucky and breakthrough after breakthrough is achieved, it could be within 10 years. But it could be 20 years, it could be 100 years.
- RockyMcNuts 11y ago
  "Moore's Law of Mad Science: Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." - Eliezer Yudkowsky On a long enough timeline, we're probably screwed by Fermi's Great Filter regardless, but a self-improving machine learning singularity can certainly accelerate the process!
  - Moshe_Silnorin 11y ago
    AI seems to be at odds with the great filter. One would expect it to expand out into the universe and be visible.
    - deleted 11y ago
      [deleted]
    - varelse 11y ago
      Why... So... Visible?
      - varelse 11y ago
        Really, you guys believe technology sufficiently advanced to be indistinguishable from magic (h/t to Arthur C. Clarke) will be festooned with the alien tech equivalent of pop-up ads and banners easily detectable by puny earthlings? Allow me to disagree.

emiliobumachar 11y ago

An interesting assessment of safety considerations is: http://intelligence.org/files/AIPosNegFactor.pdf http://intelligence.org/files/AIPosNegFactor.pdf I've submitted it for HN discussion: https://news.ycombinator.com/item?id=10637207 https://news.ycombinator.com/item?id=10637207

vdaniuk 11y ago

Mutually assured destruction doctrine is possibly very dangerous, too. Also: Biological weapons, destruction of rainforest ecosystems, ocean acidification, etc etc. Why these discussions don't consider opportunity costs of NOT developing AI as fast as possible? Most people live in appalling, inhuman conditions: $100 monthly income puts a person in the top 52.40% earners worldwide. [0] $300 monthly income puts a person in the top 28.35% earners worldwide. [0] Human intelligence is obviously suboptimal for solving global problems, perhaps AI will do better. [0] http://www.globalrichlist.com/ http://www.globalrichlist.com/

logingone 11y ago
$100 monthly income puts a person in the top 52.40% earners worldwide. That's misleading without considering numerous other variables - cost of things? It seems to me to be the kind of statement aimed at deceitfully triggering emotions, by the likes of charities. Charities are like police - there for good reason, but along with the good intentions they have a slightly seedy side. Of the countries I've lived in, the one where I was paid least had the highest standard of living.
- vdaniuk 11y ago
  Actually, it's other way around. Your comment is misleading and your intentions have not-so-slightly seedy side. Are you really attacking a concept of charity organization in a capitalistic world? Calculation methodology is available on the linked page: "For currency conversion we use Purchasing Power Parity Dollars (PPP$) in order to take into account the difference in cost of living between countries; PPP$ are also less susceptible to short term fluctuations."
  - logingone 11y ago
    "there for good reason"