4 ms·
Honest question: what kind of work is that? The more I think of it, the more I think that "high concurrency with local state on one machine" is a use case that
by chubot 15y ago
Honest question: what kind of work is that?
The more I think of it, the more I think that "high concurrency with local state on one machine" is a use case that is not going to get more popular in the future.
I believe a big part of the uptick in interest in concurrency is an uptick in the use of distributed systems. With distributed systems, all your authoritative state is stored outside the process anyway (at least if you want it to be fault tolerant, which you do if you need it to be of any significant size).
And of course once you have more than one machine you must deal with serialization. Also, I'm not sure what the kind of work there is where "serialization is prohibitively expensive" but Python is still a natural choice.
- fijal 15y agowell. it's not entirely true, if distributed systems were the future, we would never see multi-core systems at all, there are still tasks that are much easier to do if you have them done on a single shared-memory system then distributed. I don't think they're necesarilly less complex, they just don't deal with any sort of "big data". pypy's translation toolchain is one of those problems, but I can see a lot of situations where a large mostly-read-only set is necessary to be accessible all the time. regarding "Python is still a natural choice" - don't confuse python as a language and python (CPython) as interpreter. You would not use CPython for any performance-critical tasks probably, at least not without spending significant time rewriting pieces to C, but PyPy is quite usable in some scenarios and the list is only to grow.
- chubot 15y ago> there are still tasks that are much easier to do if you have them done on a single shared-memory system then distributed. Such as...? A lot of the big users of cray-type systems were for scientific uses. AFAIK a lot of them are seriously looking at cloud or commodity-type clusters as the problems get bigger. Anyway I am curious if PyPy is aiming at some specific problem that I don't know about. For "web stuff", I think what is proposed is perhaps overly complicated. If you want a large read-only set of data to fit on one machine and need high performance, Java, Python and the like aren't great choices because they don't give you much control over the memory layout. Python is probably better because you could write a C extension. But PyPy itself is not optimized for memory size (in fact I think it uses more memory to get speed).
- hythloday 15y agoGame servers are never going to be anything other than "high concurrency with local state on one machine", due to their intersecting requirements of low latency, authority, high computational demands, and relative indifference to uptime. Relevantly, CCP say the idea of EVE is "incontheivable" without Stackless Python[0], which doesn't have STM, but does go in a similar direction (co-operative multitasking) to solve the same problem. [0] http://www.slideshare.net/Arbow/stackless-python-in-eve http://www.slideshare.net/Arbow/stackless-python-in-eve
- est 15y agosessions, push notifications, XMPP
- FooBarWidget 15y agoWell, imagine the database server itself. You want it to be able to use multiple cores. If the database server is multi-process, you want to be able to access the entire data set regardless of which process is handling your client connection. Of course this problem becomes moot if the database supports auto-sharding over multiple machines, but SQL databases generally have bad support for sharding.