4 ms·
Condor, a distributed computing environment, has done IO remoting (where all calls to IO on the target machine get sent back to the source) for several decades.
by dekhn 7y ago
Condor, a distributed computing environment, has done IO remoting (where all calls to IO on the target machine get sent back to the source) for several decades. The origin of Linux Containers was process migration.
I believe people have found other ways to do this, personally I think the ECS model (like k8s, but the cloud provider hosts the k8s environment) where the user packages up all the dependencies and clearly specifies the IO mechanisms through late biniding, makes a lot more sense for distributed computing.
- vidarh 7y agoI clicked through to mention Condor too... I first came across it in the 90's, and it seems like one of those obvious hacks that keeps being reinvented.
- dekhn 7y agoI was actually channeling the creator of Condor, Miron Livny, who has a history of going to talks about distributed computing and pointing out that "Condor already does that" for nearly everything that people try to tout as new and cool. He's not wrong, but few people use Condor.
- iaresee 7y agoFew people outside academia, maybe? But inside it still seems to dominate in areas like physics computation. CERN uses a world wide Condor grid. LIGO too. It’s excellent for sharing cycles for those slow-burn, highly parallel, massive data scale problems. I spent more than a decade bringing Condor to semi-conductor, financial and biomedical institutions. It was always a fight to show them there was a better way to utilize their massive server farms that didn’t require paying the LSF Tax. Without a shiny sales or marketing department, Condor was hard to pitch to IT departments. Still, to this day, I see people doing things with “modern” platforms like Kubernetes and such and I chuckle. Had that in Condor 15 years ago in many cases. :)
- 0xdeadbeefbabe 7y ago> Still, to this day, I see people doing things with “modern” platforms like Kubernetes and such and I chuckle. Had that in Condor 15 years ago in many cases. :) I'm reading the docs and it seems used mostly for solving long running math problems like protein folding or seti at home? Can it be used for scaling a website too? I think that's k8s "killer" feature heh.
- iaresee 7y agoYes, it can scale based on metrics. And metrics can be anything. What it’s missing is all the discovery and network plumbing to tie running instances together with load balancing and inter-service comms. Googles old Borg paper mentions Condor as a thing they considered and cribbed features from. Honestly, serving a website is not as different from batch processing problems as you’d think. There are differences but they’re subtle, not mountainous.
- yummypaint 7y agoI've used condor for ~5 years now, mostly for running simulations and processing data. Everything i've done with it has been trivially parallelizable (divide data into chunks based on time, etc), and in those applications it has been a superb tool that just works. It should be possible to run a scalable website with it, but then you don't get "infinite" scalability like cloud services offer, since you're limited by the size of the compute pool. It would probably have its pitfalls. That being said, coming in without knowledge of either, i found it much easier to learn and get started doing things with condor than kubernetes. I had all kinds of issues just getting simple things like LaTeX compilation as part of gitlab CI to work reliably. Clearly the experts know how to make things go, but condor is lower barrier to entry. For use cases where condor CAN work, especially data processing, i always recommend that.
- dekhn 7y agoMost systems like Condor have a concept that a job or task is something that "comes up, runs for a while, writes to log files, and then exits on its own, or after a deadline". I've talked to the various batch queue providers and I don't think they really consider "services" (like a webserver, app server, rpc server, or whatever) in their purview. In fact, that was what struck me the most when I started at Google (a looong time ago): at first I thought of borg like a batch queue system, but really it's more of a "Service-keeper-upper" that does resource management and batch jobs are just sort one type of job (webserving, etc are examples of "Service jobs") laid on top of the RM system. Over time I've really come to prefer the google approach, for example when a batch job is running, it still listens on a web port, serves up a status page, and numerous other introspection pages that are great for debugging. TBH I haven't read the Condor, PBS, LSF manuals in a while so it's very well possible they handle service jobs and the associated problems like dynamic port management, task discovery, RPC balancing, etc.
- mempko 7y agoWe use Condor. Oh and did you know it can run docker containers too? And it's constantly being updated an improved. What is lacking is in my opinion is a cool GUI to monitor and spawn things.
- iaresee 7y agoPre-MS acquisition you used to be able to use Cycle Cloud's software for ~free without support. Unfortunately it went all-Azure-only post-acquisition.
- gmfawcett 7y agoFor those like me who haven't heard of Condor before, it's called HTCondor now: https://research.cs.wisc.edu/htcondor/description.html https://research.cs.wisc.edu/htcondor/description.html