3 ms·

Is distributed computing dying, or just fading into the backdrop?

13 points by dangle1 3y ago

nonameiguess 3y ago
This article seems to be using a nonstandard definition of distributed computing. All of the cited projects, SETI, protein folding studies, whatever, still use distributed computing. What changed was they no longer attempt to distribute some of the computation to machines owned by interested but unaffiliated people donating time from their home PCs. Whether the reason is unreliable connections to the machines that may not even always be on, home users no longer wanting remote software executing continuously on their networks, or compute power in the data centers just got bigger and cheaper and it is no longer necessary to try and find even more free compute, that is separate from it now being possible to do something like SETI on a single really big computer. Heck, it might even be possible, but considering the sensor data is inherently distributed, keeping some compute near the sensors is pretty desirable rather than needing to send it off to one common place for all processing to be done.
- WorldMaker 3y ago
  One other consideration is how much of that is simply "mission accomplished". From my understanding, there's a sense SETI's raw dataset also just simply "shrunk" in the last several years. In the furthest distributed among general users period they were analyzing a lot of raw Arecibo data collected for other science needs and not already in the best shape for SETI experiments so a lot of it was "tedious grunt work". Between the shutdown of the Arecibo antenna and the stand up of dedicated SETI mission hardware (the Allen Telescope Array) built directly for their experiments a lot of the "tedious grunt work" just disappeared (and yes, as mentioned, doing more of the signal processing directly on site as the data is collected as part of the array design). Relatedly, from how I've heard it, protein folding hit a number of key milestones and solved a lot of (most of?) the most common/most generic proteins and their various folds. The biggest remaining problems are supposedly just ad hoc specialty projects for specific experiments/grant sponsors (in which case they can include computation resources in the experiment design/grant sponsorship) or specific rare events (the massive distribution asked for during the COVID-19 processing period). I think it is worth celebrating that apparently there was a good amount of "mission accomplished" science and that "use all the computation available for free from home users" period has mostly ended.
pjmlp 3y ago
If anything, it is more relevant than ever, at a scale we were hardly doing it 50 years ago.
kobalsky 3y ago
> An incalculable amount of PC cycles and electricity wasted for nothing. was it really wasted as in “the research data was not used”? or is the author just calling it a waste because we didn’t find aliens and they are disappointed?

13 points by dangle1 3y ago

This article seems to be using a nonstandard definition of distributed computing. All of the cited projects, SETI, protein folding studies, whatever, still use distributed computing. What changed was they no longer attempt to distribute some of the computation to machines owned by interested but unaffiliated people donating time from their home PCs. Whether the reason is unreliable connections to the machines that may not even always be on, home users no longer wanting remote software executing continuously on their networks, or compute power in the data centers just got bigger and cheaper and it is no longer necessary to try and find even more free compute, that is separate from it now being possible to do something like SETI on a single really big computer. Heck, it might even be possible, but considering the sensor data is inherently distributed, keeping some compute near the sensors is pretty desirable rather than needing to send it off to one common place for all processing to be done.

WorldMaker 3y ago
One other consideration is how much of that is simply "mission accomplished". From my understanding, there's a sense SETI's raw dataset also just simply "shrunk" in the last several years. In the furthest distributed among general users period they were analyzing a lot of raw Arecibo data collected for other science needs and not already in the best shape for SETI experiments so a lot of it was "tedious grunt work". Between the shutdown of the Arecibo antenna and the stand up of dedicated SETI mission hardware (the Allen Telescope Array) built directly for their experiments a lot of the "tedious grunt work" just disappeared (and yes, as mentioned, doing more of the signal processing directly on site as the data is collected as part of the array design). Relatedly, from how I've heard it, protein folding hit a number of key milestones and solved a lot of (most of?) the most common/most generic proteins and their various folds. The biggest remaining problems are supposedly just ad hoc specialty projects for specific experiments/grant sponsors (in which case they can include computation resources in the experiment design/grant sponsorship) or specific rare events (the massive distribution asked for during the COVID-19 processing period). I think it is worth celebrating that apparently there was a good amount of "mission accomplished" science and that "use all the computation available for free from home users" period has mostly ended.

pjmlp 3y ago

If anything, it is more relevant than ever, at a scale we were hardly doing it 50 years ago.

kobalsky 3y ago

> An incalculable amount of PC cycles and electricity wasted for nothing. was it really wasted as in “the research data was not used”? or is the author just calling it a waste because we didn’t find aliens and they are disappointed?