4 ms·

> What about Tannenbaum et al's Amoeba OS? Unlike the contemporary Sprite, Amoeba does not support process migration. https://en.wikipedia.org/wiki/Amoeba_(ope

by sigil 9y ago

> What about Tannenbaum et al's Amoeba OS? Unlike the contemporary Sprite, Amoeba does not support process migration. https://en.wikipedia.org/wiki/Amoeba_(operating_system) https://en.wikipedia.org/wiki/Amoeba_(operating_system) > Globe toolkit Got a link to this one? > Or the grid computing platforms that have been around a long time? There's been steady work in this area with some of it used in production in both CompSci and industry. Curious if any of these grid computing platforms support process migration (aka "don't make me think about machine boundaries"), and whether any of the more recent ones have been informed by CAP / designed to withstand unreliable networks. LASP is interesting because it has been. As I understand it, process state is a CRDT, so computations should survive network partitions and state should reconverge afterwards. The biggest questions for me are around the performance characteristics of this approach.

nickpsecurity 9y ago
"Unlike the contemporary Sprite, Amoeba does not support process migration" Oops. My bad. It did a lot as a distributed system except for that apparently. You also found the one that did. "Globe toolkit Got a link to this one?" It's an object-based system. The old site is dead I think with me having to constantly dig it up. Here's a nice summary: https://cds.cern.ch/record/400321/files/p117.pdf https://cds.cern.ch/record/400321/files/p117.pdf They also mapped the WWW protocols to it to some degree to show we could reuse those legacy systems benefiting from Globe's capabilities. Or start the transition that way. "Curious if any of these grid computing platforms support process migration " The first I saw was MOSIX: https://en.wikipedia.org/wiki/MOSIX https://en.wikipedia.org/wiki/MOSIX Modern, smaller experiment that's similar in Linux: http://citeseerx.ist.psu.edu/viewdoc/download?rep=rep1&type=pdf&doi=10.1.1.103.417 http://citeseerx.ist.psu.edu/viewdoc/download?rep=rep1&type=... I found a nice survey paper that includes references to a number of systems from 2006: https://www.cse.buffalo.edu/~vipin/book_Chapters/2006/2006_1.pdf https://www.cse.buffalo.edu/~vipin/book_Chapters/2006/2006_1... I think most of the research is going into "VM" rather than "process" migrations given most computing is going in that direction. If you research that, you will get many more results whose techniques might apply to process migration. Quick example: http://mvapich.cse.ohio-state.edu/static/media/publications/abstract/huang-cluster07.pdf http://mvapich.cse.ohio-state.edu/static/media/publications/... The high-performance part of CompSci has mostly shifted in that direction since most industry and FOSS have shifted toward containers/VM's. The point of my original comment, though, was to say that what you say needs to be done has been happening for a while. It's just a small few in academia who dig in that deep to develop those mechanisms. And resulting in something worth finding with my quick searches. :)
- sigil 9y ago
  All good stuff, thanks. > I think most of the research is going into "VM" rather than "process" migrations given most computing is going in that direction. The VM-migration-over-RDMA paper you cite is interesting, at the very least, because it gives current upper bounds on the wait time for that operation. They do mention there's application downtime. This might not matter at all for scientific / HPC computing (the paper chooses CFD as a benchmark), but it definitely matters to people building highly available services, which I'd guess is most of HN. Application downtime on a single migrating VM isn't a dealbreaker, but it does assume you've either cleverly replicated state across VM instances, or you've got some other load-balancing infrastructure that's routing traffic in front of a mostly stateless app tier. In either case, you have to be careful with state, and careful operationally as you bring these things up and down. Wouldn't it be nice if the OS just did this for you? That's all I'm saying. I can't remember the last time I migrated a running process from CPU0 to CPU1 by hand. Or even wrote a script to do that. > The high-performance part of CompSci has mostly shifted in that direction since most industry and FOSS have shifted toward containers/VM's. It most certainly has. We might be at a local optimum though, and one determined by increasingly irrelevant computing history. I think about that every time I rent a VM to stand up a Unix box to run a container to run what's supposed to be a totally stateless application. Seriously, at that point, why aren't we running side-effect-free FP code as close to the metal as possible? We've got all these PCs laying around, we've got the peace dividends of the smartphone wars kicking in, we've got almost 50 years of Unix, we don't talk about the Von Neumann architecture anymore because everything is Von Neumann, etc. From my perspective it sure looks like we're surrounded by hammers. If we happen to be living in the Nail Universe, there's nothing wrong with that. > The point of my original comment, though, was to say that what you say needs to be done has been happening for a while. I'm very interested in the history of distributed computing -- tons to learn there. "Relearn" might be more accurate. At the same time, I don't trust anything pre-CAP-theorem to work at datacenter-scales. We simply didn't understand the scale problems well enough before the early 2000s.
- sigil 9y ago
  Addendum: MOSIX looks pretty interesting for small-scale things.

by sigil 9y ago

nickpsecurity 9y ago

"Unlike the contemporary Sprite, Amoeba does not support process migration" Oops. My bad. It did a lot as a distributed system except for that apparently. You also found the one that did. "Globe toolkit Got a link to this one?" It's an object-based system. The old site is dead I think with me having to constantly dig it up. Here's a nice summary: https://cds.cern.ch/record/400321/files/p117.pdf https://cds.cern.ch/record/400321/files/p117.pdf They also mapped the WWW protocols to it to some degree to show we could reuse those legacy systems benefiting from Globe's capabilities. Or start the transition that way. "Curious if any of these grid computing platforms support process migration " The first I saw was MOSIX: https://en.wikipedia.org/wiki/MOSIX https://en.wikipedia.org/wiki/MOSIX Modern, smaller experiment that's similar in Linux: http://citeseerx.ist.psu.edu/viewdoc/download?rep=rep1&type=pdf&doi=10.1.1.103.417 http://citeseerx.ist.psu.edu/viewdoc/download?rep=rep1&type=... I found a nice survey paper that includes references to a number of systems from 2006: https://www.cse.buffalo.edu/~vipin/book_Chapters/2006/2006_1.pdf https://www.cse.buffalo.edu/~vipin/book_Chapters/2006/2006_1... I think most of the research is going into "VM" rather than "process" migrations given most computing is going in that direction. If you research that, you will get many more results whose techniques might apply to process migration. Quick example: http://mvapich.cse.ohio-state.edu/static/media/publications/abstract/huang-cluster07.pdf http://mvapich.cse.ohio-state.edu/static/media/publications/... The high-performance part of CompSci has mostly shifted in that direction since most industry and FOSS have shifted toward containers/VM's. The point of my original comment, though, was to say that what you say needs to be done has been happening for a while. It's just a small few in academia who dig in that deep to develop those mechanisms. And resulting in something worth finding with my quick searches. :)

sigil 9y ago
All good stuff, thanks. > I think most of the research is going into "VM" rather than "process" migrations given most computing is going in that direction. The VM-migration-over-RDMA paper you cite is interesting, at the very least, because it gives current upper bounds on the wait time for that operation. They do mention there's application downtime. This might not matter at all for scientific / HPC computing (the paper chooses CFD as a benchmark), but it definitely matters to people building highly available services, which I'd guess is most of HN. Application downtime on a single migrating VM isn't a dealbreaker, but it does assume you've either cleverly replicated state across VM instances, or you've got some other load-balancing infrastructure that's routing traffic in front of a mostly stateless app tier. In either case, you have to be careful with state, and careful operationally as you bring these things up and down. Wouldn't it be nice if the OS just did this for you? That's all I'm saying. I can't remember the last time I migrated a running process from CPU0 to CPU1 by hand. Or even wrote a script to do that. > The high-performance part of CompSci has mostly shifted in that direction since most industry and FOSS have shifted toward containers/VM's. It most certainly has. We might be at a local optimum though, and one determined by increasingly irrelevant computing history. I think about that every time I rent a VM to stand up a Unix box to run a container to run what's supposed to be a totally stateless application. Seriously, at that point, why aren't we running side-effect-free FP code as close to the metal as possible? We've got all these PCs laying around, we've got the peace dividends of the smartphone wars kicking in, we've got almost 50 years of Unix, we don't talk about the Von Neumann architecture anymore because everything is Von Neumann, etc. From my perspective it sure looks like we're surrounded by hammers. If we happen to be living in the Nail Universe, there's nothing wrong with that. > The point of my original comment, though, was to say that what you say needs to be done has been happening for a while. I'm very interested in the history of distributed computing -- tons to learn there. "Relearn" might be more accurate. At the same time, I don't trust anything pre-CAP-theorem to work at datacenter-scales. We simply didn't understand the scale problems well enough before the early 2000s.
sigil 9y ago
Addendum: MOSIX looks pretty interesting for small-scale things.