4 ms·

nah, they're arbitrary function approximators that caught a lucky break. CNNs rose to prominence because natural scene statistics are translation invariant and

nah, they're arbitrary function approximators that caught a lucky break. CNNs rose to prominence because natural scene statistics are translation invariant and convolutions can be efficiently computed on GPUs. and now that we have whole warehouses of GPUs, the current mood in DL is to stop building the symmetries of your dataset into the model (which is insane btw) and use brute force. the tenuous connection DL once had to neuroscience (perceptrons) is a distant memory

fastball 3y ago
A fabricated re-telling of the past, given that we didn't start using GPUs for this type of compute until the turn of the millenium.
- andbberger 3y ago
  AlexNet was the turning point for DL.
  - fastball 3y ago
    Why do you say that? Deep Learning was accelerating well before that (I would argue it has been accelerating for its entire existence). AlexNet was a state-of-the-art image recognition net for a (relatively) brief amount of time. It wasn't the first CNN to use GPU acceleration, and it was quickly eclipsed in terms of ImageNet performance. Regardless, I think bringing up AlexNet kinda invalidates your initial point. Although yes, it turns out that the two were a great match, CNNs and modern GPUs were clearly developed independently of each other, as evidenced by the many, many iterations of both before they were combined.
    - andbberger 3y ago
      is this schmidhuber's alt? sure they existed before AlexNet was where it really took off. just look at the number of citations. right paper, right time. CNNs were uniquely suited to the hardware at the time. because of their efficiency due to symmetry and suitability to GPGPU computing. not because of their history.
- __loam 3y ago
  If you want to talk about history, these things were invented using a 1950's understanding of neuroscience then promptly discarded until the ml people figured out how to make them useful.

fastball 3y ago

A fabricated re-telling of the past, given that we didn't start using GPUs for this type of compute until the turn of the millenium.

andbberger 3y ago
AlexNet was the turning point for DL.
- fastball 3y ago
  Why do you say that? Deep Learning was accelerating well before that (I would argue it has been accelerating for its entire existence). AlexNet was a state-of-the-art image recognition net for a (relatively) brief amount of time. It wasn't the first CNN to use GPU acceleration, and it was quickly eclipsed in terms of ImageNet performance. Regardless, I think bringing up AlexNet kinda invalidates your initial point. Although yes, it turns out that the two were a great match, CNNs and modern GPUs were clearly developed independently of each other, as evidenced by the many, many iterations of both before they were combined.
  - andbberger 3y ago
    is this schmidhuber's alt? sure they existed before AlexNet was where it really took off. just look at the number of citations. right paper, right time. CNNs were uniquely suited to the hardware at the time. because of their efficiency due to symmetry and suitability to GPGPU computing. not because of their history.
__loam 3y ago
If you want to talk about history, these things were invented using a 1950's understanding of neuroscience then promptly discarded until the ml people figured out how to make them useful.