6 ms·

I actually experimented a while ago by running a long-running Twisted-based daemon on top of PyPy to see if I could squeeze more speed out. PyPy did indeed vast

by kbuck 13y ago

I actually experimented a while ago by running a long-running Twisted-based daemon on top of PyPy to see if I could squeeze more speed out. PyPy did indeed vastly increase the speed versus the plain Python version, but once I discovered that Twisted was using select/poll by default and switched it to epoll, my performance issues with the original CPython version were gone (and PyPy couldn't use Twisted's epoll at the time). Another major issue was that running the daemon under PyPy used about 5 times the memory that the CPython version did. This was a really old version of PyPy, though, so they have probably fixed some of this memory greediness.

alexk 13y ago
What version of twisted and os you were using? I'm asking because all latest Twisted releases are using epoll by default.
- kingkilr 13y ago
  It's worth noting that PyPy also supports epoll (and kqueue), and has for a few versions.
  - kbuck 13y ago
    I remember looking at that, but Twisted's epoll reactor was a C extension at the time. It looks like Twisted 12.1.0 switched to using the epoll provided by the Python base library, but that was released about a year after I was originally installing this daemon (and I was installing everything from apt, so add another year to the age of the packages I got).
- willvarfar 13y ago
  A couple of years back when I was deploying twisted using Debian packages, it was not using epoll by default.
  - kbuck 13y ago
    Yep, this was on a Debian 6.0.2 install, with packages from apt.
- kbuck 13y ago
  This was with Twisted 10.1.0 and Python 2.6.6. I remember being in extreme disbelief when I found that it was using select/poll instead of epoll (who does that, especially when they already have epoll support?). I ended up writing this: for reactor in ["epoll", "kq"]: try: rn = reactor + "reactor" getattr(__import__("twisted.internet", fromlist=[rn], rn).install() print "Auto-selecting reactor: " + reactor break except ImportError: pass

by kbuck 13y ago

alexk 13y ago

What version of twisted and os you were using? I'm asking because all latest Twisted releases are using epoll by default.

kingkilr 13y ago
It's worth noting that PyPy also supports epoll (and kqueue), and has for a few versions.
- kbuck 13y ago
  I remember looking at that, but Twisted's epoll reactor was a C extension at the time. It looks like Twisted 12.1.0 switched to using the epoll provided by the Python base library, but that was released about a year after I was originally installing this daemon (and I was installing everything from apt, so add another year to the age of the packages I got).
willvarfar 13y ago
A couple of years back when I was deploying twisted using Debian packages, it was not using epoll by default.
- kbuck 13y ago
  Yep, this was on a Debian 6.0.2 install, with packages from apt.
kbuck 13y ago
This was with Twisted 10.1.0 and Python 2.6.6. I remember being in extreme disbelief when I found that it was using select/poll instead of epoll (who does that, especially when they already have epoll support?). I ended up writing this: for reactor in ["epoll", "kq"]: try: rn = reactor + "reactor" getattr(__import__("twisted.internet", fromlist=[rn], rn).install() print "Auto-selecting reactor: " + reactor break except ImportError: pass