3 ms·
Cool. This is essentially about creating a single image cluster that runs python. I suppose if one can accept the need for python and a python module be install
by e12e 9y ago
Cool. This is essentially about creating a single image cluster that runs python. I suppose if one can accept the need for python and a python module be installed on all "nodes" - this would start to look a lot like parallel python:
http://www.parallelpython.com/ http://www.parallelpython.com/
From an architectural point of view, an si cluster could be viewed as code+communication (+data, but the data could be "wrapped" in code). Ssh and shell provides this - shell basically wraps libc and binary blobs as callable code (built-in + assumed-available code) - and if the cluster isn't too heterogeneous can be augmented with anything from c/go code (compiler assumed available) to machine code (scp /usr/bin/<someprog>...).
And it is of course possible to build this on top of python. But the result can be much more than "just" for infrastructure - if it really works it becomes a transparent cluster platform.
- _wmd 9y agoA zero byte Python script can grow up to be anything in the absence of constraints, the infrastructure label makes it simple to say no when thinking about including a feature, and to ease selecting one implementation strategy over another. The nerd within would love to implement subtree detachment ( https://github.com/dw/mitogen/blob/master/docs/images/disconnected-subtree.png https://github.com/dw/mitogen/blob/master/docs/images/discon... ): this is basically the beginnings of an Internet worm, but it has zero practical use for infrastructure, so it's complexity I'd prefer to avoid. Commercially, the killer feature for me in this library will be its Python 2.4 support. I need that for consulting work, but that constraint stands in direct contrast to a general purpose framework for use in application development. Developers want shiny new Python 3 and suchlike, commercially I've never needed that and will be unlikely to need it for a few more Debian/RHEL/Ubuntu releases (I'm guessing, probably 2019) I don't want to rewrite Pyro4 or end up with a design as complex as CORBA (with all the pain transparent object proxies entail).. it should have just enough complexity to make it ideal for the target use case.