5 ms·

It’s worth noting that black holes were also independently described by Laplace in 1796. Quite a coincidence! https://researchonline.jcu.edu.au/9892/1/Microso

It’s worth noting that black holes were also independently described by Laplace in 1796. Quite a coincidence! https://researchonline.jcu.edu.au/9892/1/Microsoft_Word_-_Paper__Black_Hole_Concept_Final_.pdf https://researchonline.jcu.edu.au/9892/1/Microsoft_Word_-_Pa...

ioblomov 3y ago
Happens more often than you’d think: * Newton and Leibniz with calculus * Darwin and Wallace with evolution * all those shared Nobel Prizes by independent scientists (eg: Feynman and Tomonaga/Schwinger with quantum electrodynamics, et al) Presumably because knowledge converges and the field in question becomes ripe for discovery.
- roenxi 3y ago
  I'd bet economics. There are magic $/experiment thresholds where it becomes possible for scientists to try something, and one the threshold is crossed a bunch of people explore the same area at the same time. Take neural nets. FLOPS/$ crosses some magic number so that universities can afford the hardware, and bam; AI everywhere. No obvious theoretical breakthroughs since the 70s, but suddenly the field gets intense due to economics.
  - Gh0stRAT 3y ago
    IIRC backpropagation was a very significant breakthrough on how to update weights when training neural nets.
    - roenxi 3y ago
      Do you have a citation for that? It is undergrad level maths and I struggle to believe the technique is news to the AI people. The mathematicians would have known about it in theory for centuries.
      - _a_a_a_ 3y ago
        Maybe, maybe not https://en.wikipedia.org/wiki/Backpropagation#History https://en.wikipedia.org/wiki/Backpropagation#History It's odd how blatantly simple stuff can be so difficult for even experts to see sometimes.
        roenxi 3y ago
        We might note that it wouldn't have been useful if it was discovered much earlier. There is a minimum amount of computation required to get good results out of neural nets that we've only been crossing relatively recently. From some perspectives the technique could be argued as the most computationally intensive approach to problem solving humans have employed to date.
      - peterfirefly 3y ago
        https://www.reddit.com/r/MachineLearning/comments/2xcyrl/i_am_j%C3%BCrgen_schmidhuber_ama/cp7adxn/ https://www.reddit.com/r/MachineLearning/comments/2xcyrl/i_a... 1) Schmidhuber does NOT claim to have invented it. He even provides lots of really old references. You know it's old when he didn't invent it, at least in his own mind. 2) even with his generous attributions, "the first application of backpropagation to neural networks" is from 1980. 3) "LeCun et al. (1989) applied backpropagation to Fukushima’s convolutional architecture (1979)". In other words, the chain rule is really old but figuring out how to use that to adjust weights in neural nets was surprisingly unobvious. It was even more unobvious that that was a good way of adjusting weights.
        roenxi 3y ago
        I've glanced through that material, and I still think it is all obvious. It just wouldn't have worked any earlier than when the results started coming out. If they'd tried these techniques in the 50s it'd have been computationally impossible. If they try them in 2020 they're computationally trivial. These results would have all started to happen at about the time the cost of computation was within reach of the researcher's budgets. The "theoretical" breakthroughs are of the form "we can implement this technique from the 60s and get good results". Which is impressive, but it does not represent breakthroughs of knowledge as much as incremental improvements in hardware crossing key thresholds. The breakthrough is detecting that hardware can make something work now.
  - tsimionescu 3y ago
    The discovery of blackholes was not done by any kind of experiment. Leibniz and Newton's invention of calculus was likewise not a matter of experiment. Evolution is more debatable, it seems that Darwin's voyages of discovery were certainly an important part of his thinking. Still, it happened ~200 years after European ocean expeditions really started taking off, so doesn't seem like it was cutting edge experimentation. The evolution of ideas seems like a much more plausible explanation.
    - roenxi 3y ago
      Newton's discovery clearly was a matter of experiment. We don't call them "Newton's Laws" because the person who discovered them liked Newton. The man was a physicist. He was interested in problems like mapping the motion of things like planets. That sort of hobby requires a certain amount of data to be gathered. Leibniz I'm not sure about, but given the timing? There was definitely a lot of interest in basic physics at the time and I bet that was related to his interests, somehow. It isn't a hard set of coincidences to imagine. That was all being fueled by improvements in measuring technology. Evolution of ideas is plausible, but it suggests a remarkably homogeneous intellectual environment in a time where communication was expensive. Economic forces are a lot more likely to me, they scale. What we see in maths is usually one genius can discover a huge amount of stuff very quickly, they don't need to wait for other people to have ideas for them. They just need a little push in what to direct their attention to.
      - tsimionescu 3y ago
        Lots of physics experiments are actually thought experiments, they don't require money. The motions of the planets and stars had been largely mapped for more than 50 years or more by the time Newton came up with his novel mathematical description of them. > Evolution of ideas is plausible, but it suggests a remarkably homogeneous intellectual environment in a time where communication was expensive. Science doesn't often advance alone, it requires communication between scientists. And that same communication spreads ideas among those scientists, until they build a common understanding.

ioblomov 3y ago

Happens more often than you’d think: * Newton and Leibniz with calculus * Darwin and Wallace with evolution * all those shared Nobel Prizes by independent scientists (eg: Feynman and Tomonaga/Schwinger with quantum electrodynamics, et al) Presumably because knowledge converges and the field in question becomes ripe for discovery.

roenxi 3y ago
I'd bet economics. There are magic $/experiment thresholds where it becomes possible for scientists to try something, and one the threshold is crossed a bunch of people explore the same area at the same time. Take neural nets. FLOPS/$ crosses some magic number so that universities can afford the hardware, and bam; AI everywhere. No obvious theoretical breakthroughs since the 70s, but suddenly the field gets intense due to economics.
- Gh0stRAT 3y ago
  IIRC backpropagation was a very significant breakthrough on how to update weights when training neural nets.
  - roenxi 3y ago
    Do you have a citation for that? It is undergrad level maths and I struggle to believe the technique is news to the AI people. The mathematicians would have known about it in theory for centuries.
    - _a_a_a_ 3y ago
      Maybe, maybe not https://en.wikipedia.org/wiki/Backpropagation#History https://en.wikipedia.org/wiki/Backpropagation#History It's odd how blatantly simple stuff can be so difficult for even experts to see sometimes.
      - roenxi 3y ago
        We might note that it wouldn't have been useful if it was discovered much earlier. There is a minimum amount of computation required to get good results out of neural nets that we've only been crossing relatively recently. From some perspectives the technique could be argued as the most computationally intensive approach to problem solving humans have employed to date.
    - peterfirefly 3y ago
      https://www.reddit.com/r/MachineLearning/comments/2xcyrl/i_am_j%C3%BCrgen_schmidhuber_ama/cp7adxn/ https://www.reddit.com/r/MachineLearning/comments/2xcyrl/i_a... 1) Schmidhuber does NOT claim to have invented it. He even provides lots of really old references. You know it's old when he didn't invent it, at least in his own mind. 2) even with his generous attributions, "the first application of backpropagation to neural networks" is from 1980. 3) "LeCun et al. (1989) applied backpropagation to Fukushima’s convolutional architecture (1979)". In other words, the chain rule is really old but figuring out how to use that to adjust weights in neural nets was surprisingly unobvious. It was even more unobvious that that was a good way of adjusting weights.
      - roenxi 3y ago
        I've glanced through that material, and I still think it is all obvious. It just wouldn't have worked any earlier than when the results started coming out. If they'd tried these techniques in the 50s it'd have been computationally impossible. If they try them in 2020 they're computationally trivial. These results would have all started to happen at about the time the cost of computation was within reach of the researcher's budgets. The "theoretical" breakthroughs are of the form "we can implement this technique from the 60s and get good results". Which is impressive, but it does not represent breakthroughs of knowledge as much as incremental improvements in hardware crossing key thresholds. The breakthrough is detecting that hardware can make something work now.
- tsimionescu 3y ago
  The discovery of blackholes was not done by any kind of experiment. Leibniz and Newton's invention of calculus was likewise not a matter of experiment. Evolution is more debatable, it seems that Darwin's voyages of discovery were certainly an important part of his thinking. Still, it happened ~200 years after European ocean expeditions really started taking off, so doesn't seem like it was cutting edge experimentation. The evolution of ideas seems like a much more plausible explanation.
  - roenxi 3y ago
    Newton's discovery clearly was a matter of experiment. We don't call them "Newton's Laws" because the person who discovered them liked Newton. The man was a physicist. He was interested in problems like mapping the motion of things like planets. That sort of hobby requires a certain amount of data to be gathered. Leibniz I'm not sure about, but given the timing? There was definitely a lot of interest in basic physics at the time and I bet that was related to his interests, somehow. It isn't a hard set of coincidences to imagine. That was all being fueled by improvements in measuring technology. Evolution of ideas is plausible, but it suggests a remarkably homogeneous intellectual environment in a time where communication was expensive. Economic forces are a lot more likely to me, they scale. What we see in maths is usually one genius can discover a huge amount of stuff very quickly, they don't need to wait for other people to have ideas for them. They just need a little push in what to direct their attention to.
    - tsimionescu 3y ago
      Lots of physics experiments are actually thought experiments, they don't require money. The motions of the planets and stars had been largely mapped for more than 50 years or more by the time Newton came up with his novel mathematical description of them. > Evolution of ideas is plausible, but it suggests a remarkably homogeneous intellectual environment in a time where communication was expensive. Science doesn't often advance alone, it requires communication between scientists. And that same communication spreads ideas among those scientists, until they build a common understanding.