5 ms·

Superconducting Optoelectronic Neurons I: General Principles

56 points by indescions_2018 8y ago

philipkglass 8y ago
This is part one of five. Here are the other 4: Superconducting Optoelectronic Neurons II: Receiver Circuits: https://arxiv.org/abs/1805.02599 https://arxiv.org/abs/1805.02599 Superconducting Optoelectronic Neurons III: Synaptic Plasticity: https://arxiv.org/abs/1805.01937 https://arxiv.org/abs/1805.01937 Superconducting Optoelectronic Neurons IV: Transmitter Circuits: https://arxiv.org/abs/1805.01941 https://arxiv.org/abs/1805.01941 Superconducting Optoelectronic Neurons V: Networks and Scaling: https://arxiv.org/abs/1805.01942 https://arxiv.org/abs/1805.01942 It's so ambitious and so questionable at the same time. I'd classify it as hard science fiction, like a really good Orion's Arm entry, if the authors weren't from NIST. Final two discussion paragraphs from paper V: While it may be difficult to build systems larger than 10 billion neurons in the near term, such a system is not physically limited. Like the brain, such limits will be incurred due to the velocity of signal propagation. From Fig. 6(c) we know that networks as large as data centers can sustain coherent oscillations at 1 MHz. Such a facility would house 10^8 300 mm wafers if they were stacked 100 deep. This would result in 100 trillion neurons per data center across modules interconnected with another power law distribution. Networks need not oscillate at 1 MHz, and if they supported system-wide activity at 1 kHz—faster than any oscillation of the human brain—the neuronal pool could occupy a significant fraction of the earth’s surface and employ quintillions of neurons. We do not wish to cover earth in such devices, but asteroids provide ample, uncontroversial real estate. The materials for this hardware are abundant on M-type and S-type asteroids [76–80]. It appears possible for an asteroid belt to form the nodes and light to form the edges of a solar-system scale intelligent network. Asteroids can be separated by billions of meters, so light-speed communication delays may be several seconds or longer. For cognitive systems oscillating up to 20 MHz, such delays would cause individual modules to operate as separate cognitive systems, much like a society of humans. Apart from the breathtaking scale of speculation -- which one could admittedly also find to good effect in older papers about e.g. nuclear power -- there is a more concrete question. What's the all-in energy cost-per-operation vs. conventional hardware, CMOS devices operating above room temperature? Operating at liquid helium temperatures dramatically shrinks the on-chip power demand, and then the cryocooler dramatically re-inflates it. Lab scale production of liquid helium takes ~570 watts of wall-plug power to produce 1 watt of cooling near 4.2 K [1]. At the wall plug, cryogenic cooling systems included, how does this design compare to existing hardware on power and speed for neural network training or inference? AFAICT, the authors do not attempt to estimate such a figure of merit. [1] Basics of low-temperature refrigeration: https://cds.cern.ch/record/1974048/files/arXiv:1501.07392.pdf https://cds.cern.ch/record/1974048/files/arXiv:1501.07392.pd...
- adrianN 8y ago
  Cooling to liquid helium temperatures might be easier on asteroids.
  - MrEldritch 8y ago
    Not even remotely. For one thing, it's not even that cold out there; average temperature out at the main belt is only around -75 C, based on radiative equilibrium with the incoming sunlight. (Obviously, the near-earth asteroids are warmer.) And then you've got the problem that you're surrounded by vacuum, so the only way you have to dump heat for further cooling is through thermal radiation, which is terribly inefficient. Not to mention, thirdly, the fact that there's no natural helium available on asteroids, so you'd have to ship it up constantly to replace your coolant as it boiled off. Meanwhile, here on Earth, we have billions of cubic feet of helium gas underground with infrastructure for liquefying it, we have access to conductive and convective cooling for our heat pumps with the whole thermal mass of the Earth to sink heat into, and we have all the industrial infrastructure and construction and maintenance infrastructure right here. Oh, and if your datacenter is in Palo Alto instead of on Pallas, you have the added bonus that you don't have to worry about networking with a 15-minute ping. tl;dr space isn't cold, vacuum isn't a great cooling medium, and shipping is expensive.
    - adrianN 8y ago
      You'd have your radiators in the shade, where it's at least a hundred degrees cooler and you have the thermal mass of the asteroid where you can dump heat temporarily. Sourcing Helium might be difficult, but maybe you can get it from one of the gas giants?
      - jjaredsimpson 8y ago
        But now you are lifting helium out the potential well of a gas giant instead of Earth's.
      - MrEldritch 8y ago
        That'd be even less efficient; the gas giants have much deeper gravity wells than Earth, and it's correspondingly much harder to bring material up from them. Jupiter's exhaust velocity is 60 km/s, Saturn's is 35 - compared to a measly 11 km/s for Earth. And as the rocket equation tells us, the fuel required goes up exponentially the greater the change in velocity you need.

This is part one of five. Here are the other 4: Superconducting Optoelectronic Neurons II: Receiver Circuits: https://arxiv.org/abs/1805.02599 https://arxiv.org/abs/1805.02599 Superconducting Optoelectronic Neurons III: Synaptic Plasticity: https://arxiv.org/abs/1805.01937 https://arxiv.org/abs/1805.01937 Superconducting Optoelectronic Neurons IV: Transmitter Circuits: https://arxiv.org/abs/1805.01941 https://arxiv.org/abs/1805.01941 Superconducting Optoelectronic Neurons V: Networks and Scaling: https://arxiv.org/abs/1805.01942 https://arxiv.org/abs/1805.01942 It's so ambitious and so questionable at the same time. I'd classify it as hard science fiction, like a really good Orion's Arm entry, if the authors weren't from NIST. Final two discussion paragraphs from paper V: While it may be difficult to build systems larger than 10 billion neurons in the near term, such a system is not physically limited. Like the brain, such limits will be incurred due to the velocity of signal propagation. From Fig. 6(c) we know that networks as large as data centers can sustain coherent oscillations at 1 MHz. Such a facility would house 10^8 300 mm wafers if they were stacked 100 deep. This would result in 100 trillion neurons per data center across modules interconnected with another power law distribution. Networks need not oscillate at 1 MHz, and if they supported system-wide activity at 1 kHz—faster than any oscillation of the human brain—the neuronal pool could occupy a significant fraction of the earth’s surface and employ quintillions of neurons. We do not wish to cover earth in such devices, but asteroids provide ample, uncontroversial real estate. The materials for this hardware are abundant on M-type and S-type asteroids [76–80]. It appears possible for an asteroid belt to form the nodes and light to form the edges of a solar-system scale intelligent network. Asteroids can be separated by billions of meters, so light-speed communication delays may be several seconds or longer. For cognitive systems oscillating up to 20 MHz, such delays would cause individual modules to operate as separate cognitive systems, much like a society of humans. Apart from the breathtaking scale of speculation -- which one could admittedly also find to good effect in older papers about e.g. nuclear power -- there is a more concrete question. What's the all-in energy cost-per-operation vs. conventional hardware, CMOS devices operating above room temperature? Operating at liquid helium temperatures dramatically shrinks the on-chip power demand, and then the cryocooler dramatically re-inflates it. Lab scale production of liquid helium takes ~570 watts of wall-plug power to produce 1 watt of cooling near 4.2 K [1]. At the wall plug, cryogenic cooling systems included, how does this design compare to existing hardware on power and speed for neural network training or inference? AFAICT, the authors do not attempt to estimate such a figure of merit. [1] Basics of low-temperature refrigeration: https://cds.cern.ch/record/1974048/files/arXiv:1501.07392.pdf https://cds.cern.ch/record/1974048/files/arXiv:1501.07392.pd...

adrianN 8y ago
Cooling to liquid helium temperatures might be easier on asteroids.
- MrEldritch 8y ago
  Not even remotely. For one thing, it's not even that cold out there; average temperature out at the main belt is only around -75 C, based on radiative equilibrium with the incoming sunlight. (Obviously, the near-earth asteroids are warmer.) And then you've got the problem that you're surrounded by vacuum, so the only way you have to dump heat for further cooling is through thermal radiation, which is terribly inefficient. Not to mention, thirdly, the fact that there's no natural helium available on asteroids, so you'd have to ship it up constantly to replace your coolant as it boiled off. Meanwhile, here on Earth, we have billions of cubic feet of helium gas underground with infrastructure for liquefying it, we have access to conductive and convective cooling for our heat pumps with the whole thermal mass of the Earth to sink heat into, and we have all the industrial infrastructure and construction and maintenance infrastructure right here. Oh, and if your datacenter is in Palo Alto instead of on Pallas, you have the added bonus that you don't have to worry about networking with a 15-minute ping. tl;dr space isn't cold, vacuum isn't a great cooling medium, and shipping is expensive.
  - adrianN 8y ago
    You'd have your radiators in the shade, where it's at least a hundred degrees cooler and you have the thermal mass of the asteroid where you can dump heat temporarily. Sourcing Helium might be difficult, but maybe you can get it from one of the gas giants?
    - jjaredsimpson 8y ago
      But now you are lifting helium out the potential well of a gas giant instead of Earth's.
    - MrEldritch 8y ago
      That'd be even less efficient; the gas giants have much deeper gravity wells than Earth, and it's correspondingly much harder to bring material up from them. Jupiter's exhaust velocity is 60 km/s, Saturn's is 35 - compared to a measly 11 km/s for Earth. And as the rocket equation tells us, the fuel required goes up exponentially the greater the change in velocity you need.