5 ms·

People are too obsessed with more compute. Definitely having more clusters, ghz, flops is a good thing, but in my experience, experiments that take embarrassing

by imranq 4y ago

People are too obsessed with more compute. Definitely having more clusters, ghz, flops is a good thing, but in my experience, experiments that take embarrassing amounts of computational resources are extremely low yield. Algorithmic advances are the real MVPs and remain underrated. They are usually the prerequisites for scientific breakthroughs. E.g. AlphaFold2, MapReduce, Blockchain, etc.

988747 4y ago
Adding compute is easy. Coming up with algorithmic advances is anywhere from hard to impossible.
Certhas 4y ago
... correct me if I am wrong but I believe that AlphaFold2 required massive computational resources beyond the scope of what most university groups have access to. And Blockchain and MapReduce have absolutely 0 to do with enabling science. Which is a pity because I generally believe your statement the algorithms matter is on the money, it's just that the examples you give merely replace one uninformed hype with another uninformed hype that's even less relevant...
- imranq 4y ago
  Ah you're absolutely right. Those examples are more about shifting paradigms over science. I just gave a few examples that came into my head at the moment. Some example of algorithm advances actually furthering science: Compressive sensing - improves MRI reconstruction above Nyquist frequency Union Quick Find - solving percolation problems way faster KD trees - makes particle simulation a tractable problem Sources Princeton Algorithms I and II https://www.cs.princeton.edu/courses/archive/fall18/cos226/assignments.php https://www.cs.princeton.edu/courses/archive/fall18/cos226/a... MIT 6.854 advanced algorithms http://people.csail.mit.edu/moitra/854.html http://people.csail.mit.edu/moitra/854.html
- imranq 4y ago
  As for AlphaFold2 and the other DeepMind algorithms, they definitely require lots of compute but not so much that they are blocked by advances in supercomputing. This is evidenced by other teams with fewer resources starting to replicate their results.
- dekhn 4y ago
  Mapreduce, at the time, was actually a huge step forward for many scientific codes even if few adopted it. An enormous amount of computing can be done with the Map-Shuffle-Combine-Reduce paradigm (and the better replacements, such as Apache Beam/Google Flume). I used it (while working at Google) to reimplement a number of processing pipelines with the result of better performance and reliability compared to standard genomics codes.

by imranq 4y ago

988747 4y ago

Adding compute is easy. Coming up with algorithmic advances is anywhere from hard to impossible.

Certhas 4y ago

... correct me if I am wrong but I believe that AlphaFold2 required massive computational resources beyond the scope of what most university groups have access to. And Blockchain and MapReduce have absolutely 0 to do with enabling science. Which is a pity because I generally believe your statement the algorithms matter is on the money, it's just that the examples you give merely replace one uninformed hype with another uninformed hype that's even less relevant...

imranq 4y ago
Ah you're absolutely right. Those examples are more about shifting paradigms over science. I just gave a few examples that came into my head at the moment. Some example of algorithm advances actually furthering science: Compressive sensing - improves MRI reconstruction above Nyquist frequency Union Quick Find - solving percolation problems way faster KD trees - makes particle simulation a tractable problem Sources Princeton Algorithms I and II https://www.cs.princeton.edu/courses/archive/fall18/cos226/assignments.php https://www.cs.princeton.edu/courses/archive/fall18/cos226/a... MIT 6.854 advanced algorithms http://people.csail.mit.edu/moitra/854.html http://people.csail.mit.edu/moitra/854.html
imranq 4y ago
As for AlphaFold2 and the other DeepMind algorithms, they definitely require lots of compute but not so much that they are blocked by advances in supercomputing. This is evidenced by other teams with fewer resources starting to replicate their results.
dekhn 4y ago
Mapreduce, at the time, was actually a huge step forward for many scientific codes even if few adopted it. An enormous amount of computing can be done with the Map-Shuffle-Combine-Reduce paradigm (and the better replacements, such as Apache Beam/Google Flume). I used it (while working at Google) to reimplement a number of processing pipelines with the result of better performance and reliability compared to standard genomics codes.