8 ms·

Darwin, Machines, and the Future: A conversation with George Dyson

71 points by domrdy 6y ago

Johnjonjoan 6y ago
This is how I imagine 'mad scientists' justify their work.
- throwanem 6y ago
  I feel like calling VCs "mad scientists" is unduly ennobling.
  - polynomial 6y ago
    "mad engineers"? /s
    - TeMPOraL 6y ago
      "Mad millionaires". Which is closer to the mark and invokes correct Hollywood clichés.
      - klipt 6y ago
        Rich people can't be mad, they're just eccentric.
    - mrec 6y ago
      Sad truth: http://cowbirdsinlove.com/comics/46/engineer.png http://cowbirdsinlove.com/comics/46/engineer.png
      - throwanem 6y ago
        Having seen how scientists build things vs. how engineers build things, I don't see how that can be true. If it were, I'd expect the take-over-the-world failure rate to be somewhat less than the current 100%.
polynomial 6y ago
Am I the only one who thinks it's weird Dyson rips off the title from Samuel Butler's seminal masterpiece? It strikes me as extremely disingenuous; certainly unbecoming his intellectual aspirations.
xamuel 6y ago
From the article: "In terms of a distributed artificial intelligence, Dyson is a believer. Because with a distributed system, you have an opportunity for evolution, for it to find itself and to learn on its own." I published a paper in this year's AGI conference that would certainly support this line of reasoning. In my paper, "AGI and the Knight-Darwin Law", I argue that if an AGI single-handedly creates a child AGI with no outside assistance, than the child is necessarily less intelligent than the parent. Thus if machines are going to create more intelligent machines, it's necessary for the creating machines to collaborate in order to do so. This closely parallels a law proposed by Darwin, called the Knight-Darwin Law, a cornerstone of his Origin of Species. The KDL states that it's impossible for one organism to asexually produce another, which asexually produces another, and so on forever; sexual reproduction is necessary or the line must terminate. (Darwin was of course well aware of seemingly-asexual species. His motivation for postulating the KDL was the observation that seemingly-asexual species do rarely sexually reproduce, e.g. if a rainstorm damages the part of a flower that would otherwise isolate its stamen, etc.) Here is the paper: https://philpapers.org/archive/ALEAAT-10.pdf https://philpapers.org/archive/ALEAAT-10.pdf
- at_a_remove 6y ago
  What about the trivial case of the AGI simply self-replicating? Then you have a child, it is identical, and is therefore of identical intelligence (however you measure that), and therefore it is not necessarily less intelligent. What if the AGI is intelligent enough to randomly mutate offspring here and there? Then one might be more intelligent, and again it is not necessarily less intelligent. What if the AGI is intelligent enough to look at its own code, identify some kind of bias in its programming (not unlike human introspection), and produce a child without that bias? That child would then be better able to model reality -- a core element of intelligence -- and again you've got a smarter child. I have a difficult time buying the "necessarily" part.
  - Animats 6y ago
    I have a difficult time buying the "necessarily" part. Yes. That's borrowed from an old theological argument. Especially since much of the progress in machine learning is just putting more hardware and data on the same algorithms.
amb23 6y ago
The lineage of technological progress does not follow the same path as commercial development of a pre-existing technology applied to a pre-existing market. We need to be careful not to equate the two. As much as a D2C brand or SaaS startup may brilliantly bring a new market innovation to the fore, they rarely bring new technological innovations--and the potential to expand the pie rather than extract market value--with them. Technological progress is a halting, stop-and-start process, and the market isn't always welcoming to it even in cases when the economics of the innovation make sense. So, yes--if you're working on truly progressive technology, take this sense of purpose the author speaks of to heart. But tech as an industry shouldn't appropriate this purpose when it cannot follow through on it.
- zippy5 6y ago
  I respectfully disagree. New technologies don't intrinsically expand the pie, lowering the price/performance ratio does. D2C and SAAS lower the cost of distribution which historically been a significant markup on goods. The consumer pays less and thus have more money to spend on other goods and services. Therefore the pie expands. Examples of technologies failing to change price/performance: 1) Self driving cars: can you replace people working for less than minimum wage with the expensive hardware and liability related costs? Eventually yes but even airplanes are struggling and that is a far easier problem. The counter example might be closer to Tesla's auto pilot. 2) DC electrical grid (as implemented by Edison): expensive to transmit over distance relative to AC. 3) Fuel cell cars: the platinum catalysts drove up the price. (There were other factors as well). Examples of technologies succeeding to change price/performance: 1) Light bulb: factories could be productive even at night. Since most of the costs were fixed, getting more productivity was invaluable. 2) CPU's: Moore's law 3) Solar: Manufacturing costs were the real obstacle, and improved manufacturing and economies of scale led to an economically viable product. I don't claim to be perfectly right about all these cases but I think they illustrate the gist of what I'm saying.
hitech_dude 6y ago
Has any body been anlemto secure funding from the NFX FAST seed program ? It was supposed to be an online only application with a clear timetable.
bumby 6y ago
> As Dyson says, we’re in the middle of a Black Swan It’s interesting to note that Nassim Taleb who coined the term “black swan” disagrees that the current pandemic qualifies. As he points out, not only was this predictable, it was also inevitable. We have literally made movies portraying it
- op03 6y ago
  Also DARPA - https://www.washingtonpost.com/national-security/how-a-secretive-pentagon-agency-seeded-the-ground-for-a-rapid-coronavirus-cure/2020/07/30/ad1853c4-c778-11ea-a9d3-74640f25b953_story.html https://www.washingtonpost.com/national-security/how-a-secre... There were comments in HN predictions threads for 2020/for the next decade etc talking about pandemics. The hive mind belches out some interesting stuff once in while.
- deleted 6y ago
  [deleted]
- spaetzleesser 6y ago
  Agreed. Same with big fires or earthquakes in California. We know they are coming. We are just hoping they will come a little later.
- Barrin92 6y ago
  Not to mention Gates who warned of this barely five years ago https://www.ted.com/talks/bill_gates_the_next_outbreak_we_re_not_ready https://www.ted.com/talks/bill_gates_the_next_outbreak_we_re...
jecel 6y ago
I have no idea what George Dyson means when he says Turing had a 1D memory and Von Neumann changed it to a 2D memory. I consider both to be equally 1D with Turing having sequential access and Von Neumann random access. What I call 2D memory is a segmented addressing scheme. The implementation in the form of Williams Tubes (used in Von Neumann's Princeton machine) is indeed 2D, as is a modern DRAM or SRAM. But it is wrapped up in a 1D interface to the rest of the computer (a little less obviously so with RAS and CAS in DRAMs).

71 points by domrdy 6y ago

Johnjonjoan 6y ago

This is how I imagine 'mad scientists' justify their work.

throwanem 6y ago
I feel like calling VCs "mad scientists" is unduly ennobling.
- polynomial 6y ago
  "mad engineers"? /s
  - TeMPOraL 6y ago
    "Mad millionaires". Which is closer to the mark and invokes correct Hollywood clichés.
    - klipt 6y ago
      Rich people can't be mad, they're just eccentric.
  - mrec 6y ago
    Sad truth: http://cowbirdsinlove.com/comics/46/engineer.png http://cowbirdsinlove.com/comics/46/engineer.png
    - throwanem 6y ago
      Having seen how scientists build things vs. how engineers build things, I don't see how that can be true. If it were, I'd expect the take-over-the-world failure rate to be somewhat less than the current 100%.

polynomial 6y ago

Am I the only one who thinks it's weird Dyson rips off the title from Samuel Butler's seminal masterpiece? It strikes me as extremely disingenuous; certainly unbecoming his intellectual aspirations.

xamuel 6y ago

From the article: "In terms of a distributed artificial intelligence, Dyson is a believer. Because with a distributed system, you have an opportunity for evolution, for it to find itself and to learn on its own." I published a paper in this year's AGI conference that would certainly support this line of reasoning. In my paper, "AGI and the Knight-Darwin Law", I argue that if an AGI single-handedly creates a child AGI with no outside assistance, than the child is necessarily less intelligent than the parent. Thus if machines are going to create more intelligent machines, it's necessary for the creating machines to collaborate in order to do so. This closely parallels a law proposed by Darwin, called the Knight-Darwin Law, a cornerstone of his Origin of Species. The KDL states that it's impossible for one organism to asexually produce another, which asexually produces another, and so on forever; sexual reproduction is necessary or the line must terminate. (Darwin was of course well aware of seemingly-asexual species. His motivation for postulating the KDL was the observation that seemingly-asexual species do rarely sexually reproduce, e.g. if a rainstorm damages the part of a flower that would otherwise isolate its stamen, etc.) Here is the paper: https://philpapers.org/archive/ALEAAT-10.pdf https://philpapers.org/archive/ALEAAT-10.pdf

at_a_remove 6y ago
What about the trivial case of the AGI simply self-replicating? Then you have a child, it is identical, and is therefore of identical intelligence (however you measure that), and therefore it is not necessarily less intelligent. What if the AGI is intelligent enough to randomly mutate offspring here and there? Then one might be more intelligent, and again it is not necessarily less intelligent. What if the AGI is intelligent enough to look at its own code, identify some kind of bias in its programming (not unlike human introspection), and produce a child without that bias? That child would then be better able to model reality -- a core element of intelligence -- and again you've got a smarter child. I have a difficult time buying the "necessarily" part.
- Animats 6y ago
  I have a difficult time buying the "necessarily" part. Yes. That's borrowed from an old theological argument. Especially since much of the progress in machine learning is just putting more hardware and data on the same algorithms.

amb23 6y ago

The lineage of technological progress does not follow the same path as commercial development of a pre-existing technology applied to a pre-existing market. We need to be careful not to equate the two. As much as a D2C brand or SaaS startup may brilliantly bring a new market innovation to the fore, they rarely bring new technological innovations--and the potential to expand the pie rather than extract market value--with them. Technological progress is a halting, stop-and-start process, and the market isn't always welcoming to it even in cases when the economics of the innovation make sense. So, yes--if you're working on truly progressive technology, take this sense of purpose the author speaks of to heart. But tech as an industry shouldn't appropriate this purpose when it cannot follow through on it.

zippy5 6y ago
I respectfully disagree. New technologies don't intrinsically expand the pie, lowering the price/performance ratio does. D2C and SAAS lower the cost of distribution which historically been a significant markup on goods. The consumer pays less and thus have more money to spend on other goods and services. Therefore the pie expands. Examples of technologies failing to change price/performance: 1) Self driving cars: can you replace people working for less than minimum wage with the expensive hardware and liability related costs? Eventually yes but even airplanes are struggling and that is a far easier problem. The counter example might be closer to Tesla's auto pilot. 2) DC electrical grid (as implemented by Edison): expensive to transmit over distance relative to AC. 3) Fuel cell cars: the platinum catalysts drove up the price. (There were other factors as well). Examples of technologies succeeding to change price/performance: 1) Light bulb: factories could be productive even at night. Since most of the costs were fixed, getting more productivity was invaluable. 2) CPU's: Moore's law 3) Solar: Manufacturing costs were the real obstacle, and improved manufacturing and economies of scale led to an economically viable product. I don't claim to be perfectly right about all these cases but I think they illustrate the gist of what I'm saying.

hitech_dude 6y ago

Has any body been anlemto secure funding from the NFX FAST seed program ? It was supposed to be an online only application with a clear timetable.

bumby 6y ago

> As Dyson says, we’re in the middle of a Black Swan It’s interesting to note that Nassim Taleb who coined the term “black swan” disagrees that the current pandemic qualifies. As he points out, not only was this predictable, it was also inevitable. We have literally made movies portraying it

op03 6y ago
Also DARPA - https://www.washingtonpost.com/national-security/how-a-secretive-pentagon-agency-seeded-the-ground-for-a-rapid-coronavirus-cure/2020/07/30/ad1853c4-c778-11ea-a9d3-74640f25b953_story.html https://www.washingtonpost.com/national-security/how-a-secre... There were comments in HN predictions threads for 2020/for the next decade etc talking about pandemics. The hive mind belches out some interesting stuff once in while.
deleted 6y ago
[deleted]
spaetzleesser 6y ago
Agreed. Same with big fires or earthquakes in California. We know they are coming. We are just hoping they will come a little later.
Barrin92 6y ago
Not to mention Gates who warned of this barely five years ago https://www.ted.com/talks/bill_gates_the_next_outbreak_we_re_not_ready https://www.ted.com/talks/bill_gates_the_next_outbreak_we_re...

jecel 6y ago

I have no idea what George Dyson means when he says Turing had a 1D memory and Von Neumann changed it to a 2D memory. I consider both to be equally 1D with Turing having sequential access and Von Neumann random access. What I call 2D memory is a segmented addressing scheme. The implementation in the form of Williams Tubes (used in Von Neumann's Princeton machine) is indeed 2D, as is a modern DRAM or SRAM. But it is wrapped up in a 1D interface to the rest of the computer (a little less obviously so with RAS and CAS in DRAMs).