6 ms·

Overall I am sympathetic to the authors argument that fear of super ai is overblown. But I do take issue with some of his arguments. > Even if the smartest phy

by bko 10y ago

Overall I am sympathetic to the authors argument that fear of super ai is overblown. But I do take issue with some of his arguments. > Even if the smartest physicists were 1,000 times smarter than they are now, without a Collider, they will know nothing new. I'm not a historian but I have read that some scientific discoveries are made through pure logic. Einstein and relativity come to mind as he was not an empiricist. So perhaps there is some hope that ai can lead to scientific discoveries without experimentation >So the question is, where is the limit of intelligence? We tend to believe that the limit is way beyond us, way “above” us, as we are “above” an ant. Setting aside the recurring problem of a single dimension, what evidence do we have that the limit is not us? Why can’t we be at the maximum? Or maybe the limits are only a short distance away from us? Why do we believe that intelligence is something that can continue to expand forever? The idea that humans could, just by chance, be pushing the limits of intelligence strikes me as silly

Retric 10y ago
Einstein's work was based on a lot of evidence that seemed very strange. Light somehow had a fixed speed in all reference frames WTF?
- mfav 10y ago
  It's a problem perfectly suited to a logical AI, really. Take in the experimental observation data, and produce a solution that fits the parameters. Human intuition was the major thing getting in the way, and even Einstein's intuition got in the way of his acceptance of quantum mechanics.
- lazaroclapp 10y ago
  Sure. Question is: how many other discoveries await today such that we, collectively, as a species, already have all the puzzle pieces, but haven't yet put them together? An AI as smart as your average grad student, but which somehow could retain in mind at the same time all of our accumulated scientific knowledge, might be able to quickly draw all sorts of seemingly brilliant conclusions. Now imagine its reasoning process works 6 to 7 orders of magnitude faster than ours, even if not qualitatively different in its logic or biases. Dunno, I don't really believe we are that close to building that sort of AI, but it doesn't seem fundamentally impossible, and it does seem like it could do things that to us would look as "more intelligent" than us. It might in the end be better at creating scientific knowledge in the way current computers are better than us at solving arithmetic: faster and capable of holding more in memory, rather than following any qualitatively different process. But even that would be enough.
  - Retric 10y ago
    IMO, few major ones. We have billions of man years of research and things are better than before that period, but diminishing returns are real. Don't get me wrong I think it would be useful, just that the gap from human level AI to 1,000x human AI is simply not that huge. Let's say you moved someone from 2006 to 2016 or even from 1996. Yea sure there is real progress, but not really. We have better drugs in terms of AIDS for example, but we are worse off in terms of antibiotics. Game graphics have improved, but quake is from 1996 so we already had real 3D first person shooters and gameplay is not that different. Hell FTP is arguably worse. Further that's 20 years so we are talking literally millions-man years of effort and trillions of dollars worth of R&D for not much. In terms of machines the SR-71 is still the fastest manned aircraft, no supersonic passenger aircraft. Tallest building is much taller, but lacks much space on the top floors making it more monument than utility as the sears tower has more useable space and a smaller footprint.
    - TheOtherHobbes 10y ago
      Invention proceeds because of game-changing insights. Maxwell's equations were a game changer. So were Newton's laws. So were Relativity and QM. Church-Turing was also a game changer. But I don't think there's been anything equivalent in computing since. There's been a lot of application, but no game-changing deep theoretical insights. Quantum computing may - should? - eventually throw up something new. It's going to have to. If it doesn't I think we're going to be stuck with much less progress than we're expecting.
      - jacquesm 10y ago
        > Church-Turing was also a game changer. But I don't think there's been anything equivalent in computing since. Quantity when the difference is large enough becomes quality. The 9 orders of magnitude or so that computers have gone through in storage capacity and speed definitely count as a game-changer.
      - adrianN 10y ago
        The insight that all problems in NP are basically the same was a game changer in complexity theory (it basically started the whole field).
- dnautics 10y ago
  There's tangherlini relativity which doesn't make these assumptions and is consistent with existing observations! The math is crazy hard.
canes123456 10y ago
Also, the limit of one agent's intelligence is kind of missing the point. Humans are powerful because of societies of people loosely working together. AI agents will be able to communicate and interact at speeds that are we cannot imagine. A network of near human IQ AI agents would be much more powerful than a single super smart AI agent.
- aetherson 10y ago
  I think that there is very compelling evidence (but not formal proof) that there is exponential falloff in terms of the ability of parallel networks of intelligent agents to have heightened overall intelligence. Imagining that there will be near-human IQ AIs that think orders of magnitude more quickly than humans puts the cart before the horse, as well.
  - zeeveener 10y ago
    Would it not appear as though it was more intelligent simply based on the speed at which it makes decisions? The Actual Intelligence may be the same (or a little higher) considering the Average Intelligence, but the effect the higher speed would likely be a measurable difference.
    - aetherson 10y ago
      Yes, that's what I'm saying. If you can make an AI that's roughly human IQ (even like 80 IQ), but thinks 100x faster than a human, then that's something very much like, if not identical to, "a superhuman AI." So when you say, "Here's how we'll get superhuman AI: We'll network together bunches of 80 IQ AI's that think 100x faster than a human," it's kind of assuming its own solution.
      - rtx 10y ago
        Why would you network them? Just add more ram.
        10y ago
parenthephobia 10y ago
Without further experimentation the best one can say is that you have a theory which covers that which has already been observed. This is not a useful theory, however: useful theories make predictions about future observations. Really useful ones make correct predictions. An AI can certainly produce a theory which matches observations, but no matter how powerful it is it won't be able to magically produce only correct useful theories any more than a human can. We had over 200 years from Newton's laws of motion to produce relativity from pure logic alone, but we didn't manage it. We had to reason from the evidence, and even then we didn't accept relativity as "true" until it had been experimentally verified.
BaronSamedi 10y ago
I agree that the fear of "super AI" is overblown, vastly in my opinion. Discussion on this topic is tantamount to arguing over the attributes of imaginary entities. Simple artificial life of some kind seems much more likely in the foreseeable future. I've often wondered why no-one has written a virus or worm that is capable of mutation or even cross-over (sexual reproduction of a sort). Under selection pressure it might evolve into something resembling life. The Tierra experiments were a fascinating experiment in this area.
ThomPete 10y ago
A tiger is dangerous even if its not selfaware.
mcguire 10y ago
A mathematical theory (á la Einstein) without experimental confirmation is just a theory. Or perhaps economics. And the idea that all forms of computation, like Turing machines, lambda calculus, etc., etc., also seems pretty darn silly at first glance, too.
epx 10y ago
Relativity stems from the Michelson-Morley experiment.
- oldandtired 10y ago
  The ideas of relativity predate Einstein - his particular view/model still has serious flaws and in the modern rendering leads to non-reality conclusions. In a very funny way, Einstein's two theories of relativity have become religious dogma. Try to discuss this and the usual conduct of "true-believers" is NOT to show where your thinking is wrong, but to simply call you an idiot. This approach does nothing to convince anyone who believes differently that they are wrong, all it does do is convince them that the "true-believers" are themselves deluded. This does nothing to advance the understanding of society of the universe around them.
  - Ono-Sendai 10y ago
    I mostly agree with you, but what are the 'non-reality' conclusions?
    - oldandtired 10y ago
      Gravitational singularities. If one puts thought into it, one will recognise the shell game played when describing how something that requires infinite time to form as defined by the model, forms in finite time in the real universe. One then recognises that any two such entities will take infinite time to merge in the real finite timed universe. I am also looking at a paper that purports to say that Einstein's GR which is used to calculate the precession of Mercury uses a couple of simplifications, which if not used, gives GR calculating the precession of Mercury at twice (or thereabouts what is measured). To test the veracity of the paper, I'll have to do some serious brushing up of my engineering mathematics. Another example is related to the speed of light c. This value is inversely proportion to the square root of the (electric permittivity of the environment times the magnetic permeability of the environment). The normal assumption is that outside of the atmosphere of the earth, the environment is a perfect vacuum. It is assumed that the speed of light is therefore a constant in a perfect vacuum. For many areas, this simplifying assumption is adequate, but this assumption may not be true. It is only so assumed. It may turn out that due to variations in both the permittivity and the permeability of the extraterrestrial environments, that this assumption is not inadequate but grossly inadequate. When doing any experimental work, one should keep in mind and have written down all the base assumptions, otherwise, one may garden path oneself.
jacquesm 10y ago
> The idea that humans could, just by chance, be pushing the limits of intelligence strikes me as silly It's the wrong argument entirely. It has nothing to do with intelligence but everything to do with knowledge. Smarts are useless if you have no knowledge. And that's exactly where computers have the edge. Even a 'regular' human level intelligence with all the knowledge that currently exists at its beck and call would be a formidable adversary. Remember: Knowledge is power.
ankurdhama 10y ago
> I'm not a historian but I have read that some scientific discoveries are made through pure logic What you are talking about is scientific hypothesis/theory which obviously needs to be tested using experiments to make sure it works. Without experiment there is nothing much you can do in real world.

by bko 10y ago

Retric 10y ago

Einstein's work was based on a lot of evidence that seemed very strange. Light somehow had a fixed speed in all reference frames WTF?

mfav 10y ago
It's a problem perfectly suited to a logical AI, really. Take in the experimental observation data, and produce a solution that fits the parameters. Human intuition was the major thing getting in the way, and even Einstein's intuition got in the way of his acceptance of quantum mechanics.
lazaroclapp 10y ago
Sure. Question is: how many other discoveries await today such that we, collectively, as a species, already have all the puzzle pieces, but haven't yet put them together? An AI as smart as your average grad student, but which somehow could retain in mind at the same time all of our accumulated scientific knowledge, might be able to quickly draw all sorts of seemingly brilliant conclusions. Now imagine its reasoning process works 6 to 7 orders of magnitude faster than ours, even if not qualitatively different in its logic or biases. Dunno, I don't really believe we are that close to building that sort of AI, but it doesn't seem fundamentally impossible, and it does seem like it could do things that to us would look as "more intelligent" than us. It might in the end be better at creating scientific knowledge in the way current computers are better than us at solving arithmetic: faster and capable of holding more in memory, rather than following any qualitatively different process. But even that would be enough.
- Retric 10y ago
  IMO, few major ones. We have billions of man years of research and things are better than before that period, but diminishing returns are real. Don't get me wrong I think it would be useful, just that the gap from human level AI to 1,000x human AI is simply not that huge. Let's say you moved someone from 2006 to 2016 or even from 1996. Yea sure there is real progress, but not really. We have better drugs in terms of AIDS for example, but we are worse off in terms of antibiotics. Game graphics have improved, but quake is from 1996 so we already had real 3D first person shooters and gameplay is not that different. Hell FTP is arguably worse. Further that's 20 years so we are talking literally millions-man years of effort and trillions of dollars worth of R&D for not much. In terms of machines the SR-71 is still the fastest manned aircraft, no supersonic passenger aircraft. Tallest building is much taller, but lacks much space on the top floors making it more monument than utility as the sears tower has more useable space and a smaller footprint.
  - TheOtherHobbes 10y ago
    Invention proceeds because of game-changing insights. Maxwell's equations were a game changer. So were Newton's laws. So were Relativity and QM. Church-Turing was also a game changer. But I don't think there's been anything equivalent in computing since. There's been a lot of application, but no game-changing deep theoretical insights. Quantum computing may - should? - eventually throw up something new. It's going to have to. If it doesn't I think we're going to be stuck with much less progress than we're expecting.
    - jacquesm 10y ago
      > Church-Turing was also a game changer. But I don't think there's been anything equivalent in computing since. Quantity when the difference is large enough becomes quality. The 9 orders of magnitude or so that computers have gone through in storage capacity and speed definitely count as a game-changer.
    - adrianN 10y ago
      The insight that all problems in NP are basically the same was a game changer in complexity theory (it basically started the whole field).
dnautics 10y ago
There's tangherlini relativity which doesn't make these assumptions and is consistent with existing observations! The math is crazy hard.

canes123456 10y ago

Also, the limit of one agent's intelligence is kind of missing the point. Humans are powerful because of societies of people loosely working together. AI agents will be able to communicate and interact at speeds that are we cannot imagine. A network of near human IQ AI agents would be much more powerful than a single super smart AI agent.

aetherson 10y ago
I think that there is very compelling evidence (but not formal proof) that there is exponential falloff in terms of the ability of parallel networks of intelligent agents to have heightened overall intelligence. Imagining that there will be near-human IQ AIs that think orders of magnitude more quickly than humans puts the cart before the horse, as well.
- zeeveener 10y ago
  Would it not appear as though it was more intelligent simply based on the speed at which it makes decisions? The Actual Intelligence may be the same (or a little higher) considering the Average Intelligence, but the effect the higher speed would likely be a measurable difference.
  - aetherson 10y ago
    Yes, that's what I'm saying. If you can make an AI that's roughly human IQ (even like 80 IQ), but thinks 100x faster than a human, then that's something very much like, if not identical to, "a superhuman AI." So when you say, "Here's how we'll get superhuman AI: We'll network together bunches of 80 IQ AI's that think 100x faster than a human," it's kind of assuming its own solution.
    - rtx 10y ago
      Why would you network them? Just add more ram.
      - 10y ago

parenthephobia 10y ago

Without further experimentation the best one can say is that you have a theory which covers that which has already been observed. This is not a useful theory, however: useful theories make predictions about future observations. Really useful ones make correct predictions. An AI can certainly produce a theory which matches observations, but no matter how powerful it is it won't be able to magically produce only correct useful theories any more than a human can. We had over 200 years from Newton's laws of motion to produce relativity from pure logic alone, but we didn't manage it. We had to reason from the evidence, and even then we didn't accept relativity as "true" until it had been experimentally verified.

BaronSamedi 10y ago

I agree that the fear of "super AI" is overblown, vastly in my opinion. Discussion on this topic is tantamount to arguing over the attributes of imaginary entities. Simple artificial life of some kind seems much more likely in the foreseeable future. I've often wondered why no-one has written a virus or worm that is capable of mutation or even cross-over (sexual reproduction of a sort). Under selection pressure it might evolve into something resembling life. The Tierra experiments were a fascinating experiment in this area.

ThomPete 10y ago

A tiger is dangerous even if its not selfaware.

mcguire 10y ago

A mathematical theory (á la Einstein) without experimental confirmation is just a theory. Or perhaps economics. And the idea that all forms of computation, like Turing machines, lambda calculus, etc., etc., also seems pretty darn silly at first glance, too.

epx 10y ago

Relativity stems from the Michelson-Morley experiment.

oldandtired 10y ago
The ideas of relativity predate Einstein - his particular view/model still has serious flaws and in the modern rendering leads to non-reality conclusions. In a very funny way, Einstein's two theories of relativity have become religious dogma. Try to discuss this and the usual conduct of "true-believers" is NOT to show where your thinking is wrong, but to simply call you an idiot. This approach does nothing to convince anyone who believes differently that they are wrong, all it does do is convince them that the "true-believers" are themselves deluded. This does nothing to advance the understanding of society of the universe around them.
- Ono-Sendai 10y ago
  I mostly agree with you, but what are the 'non-reality' conclusions?
  - oldandtired 10y ago
    Gravitational singularities. If one puts thought into it, one will recognise the shell game played when describing how something that requires infinite time to form as defined by the model, forms in finite time in the real universe. One then recognises that any two such entities will take infinite time to merge in the real finite timed universe. I am also looking at a paper that purports to say that Einstein's GR which is used to calculate the precession of Mercury uses a couple of simplifications, which if not used, gives GR calculating the precession of Mercury at twice (or thereabouts what is measured). To test the veracity of the paper, I'll have to do some serious brushing up of my engineering mathematics. Another example is related to the speed of light c. This value is inversely proportion to the square root of the (electric permittivity of the environment times the magnetic permeability of the environment). The normal assumption is that outside of the atmosphere of the earth, the environment is a perfect vacuum. It is assumed that the speed of light is therefore a constant in a perfect vacuum. For many areas, this simplifying assumption is adequate, but this assumption may not be true. It is only so assumed. It may turn out that due to variations in both the permittivity and the permeability of the extraterrestrial environments, that this assumption is not inadequate but grossly inadequate. When doing any experimental work, one should keep in mind and have written down all the base assumptions, otherwise, one may garden path oneself.

jacquesm 10y ago

> The idea that humans could, just by chance, be pushing the limits of intelligence strikes me as silly It's the wrong argument entirely. It has nothing to do with intelligence but everything to do with knowledge. Smarts are useless if you have no knowledge. And that's exactly where computers have the edge. Even a 'regular' human level intelligence with all the knowledge that currently exists at its beck and call would be a formidable adversary. Remember: Knowledge is power.

ankurdhama 10y ago

> I'm not a historian but I have read that some scientific discoveries are made through pure logic What you are talking about is scientific hypothesis/theory which obviously needs to be tested using experiments to make sure it works. Without experiment there is nothing much you can do in real world.