4 ms·

Fast portable non-blocking network programming with Libevent

50 points by silentbicycle 17y ago

jklghjkygu 17y ago
========FOR WOMEN Christan Audigier bikini $25 Ed Hardy Bikini $25 Smful short_t-shirt_woman $15 ed hardy short_tank_woman $16 Sandal $32 christian louboutin $39 Sunglass $16 COACH_Necklace $27 handbag $36 AF tank woman $17 -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com ============FOR MAN -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com POLO new t-shirt man $16 ed hardy short t-shirt $17 POLO short $35 Ed Hardy board short man $33 jordan men $34 Sunglass $16 ed hardy cap $18 -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com
RoboTeddy 17y ago
Libevent is rock solid -- it handles the event loop for memcache, among other things. This guide looks like a great improvement/accompaniment to libevent's docs. One problem with using an event dispatcher like libevent is that libraries that make their own blocking calls won't cooperate. That means you'll often be stuck with limited capabilties, unless you write your own libraries or hack up existing ones. There's a Python project called gevent (http://www.gevent.org/ http://www.gevent.org/) that works atop libevent. It monkey-patches the python socket module with a version that automatically cooperates with libevent's loop. It uses greenlets to make code written sychronously run asynchronously behind the scenes. If, for example, you want to use an Amazon S3 library that makes use of python's httplib (which would normally block and choke your event loop), it'll automatically work with gevent/libevent. If you need to handle lots of concurrent connections and don't need to be working in C, check it out.
- deleted 17y ago
  [deleted]
- houseabsolute 17y ago
  It's too bad that no standard method for a asynchronous events has been developed for Unix in general. Where I work, there's one single library for concurrency, and every program or toolkit uses it. We do a lot of network programming, so it would be unthinkable for any performance intensive application to do blocking calls exclusively.
  - silentbicycle 17y ago
    libevent tries to provide a common interface to select, poll, kselect (on BSD), epoll (on Linux), and comparable calls on Windows. It's not a standard part of Unix, of course, but it seems reasonably portable.
    - houseabsolute 17y ago
      Sure, but libraries underneath you have to cooperate for you to use it to the fullest potential. That no common means have been invented for that is what I'm sad about.
      - silentbicycle 17y ago
        Right. Once you set up a non-blocking event loop, it's much simpler when you can go async all the way down.
- mcav 17y ago
  Note that gevent is a fork of Eventlet (http://www.eventlet.org/ http://www.eventlet.org/), and there have been talks of reincorporating changes that went to gevent back into eventlet.
- sophacles 17y ago
  Also for python is pyevent, which works pretty well. I have used it to much success -- some people don't like the minimalist style. Also, the maintainer only updates sporadically, but I've never had real problems with it. http://code.google.com/p/pyevent/ http://code.google.com/p/pyevent/
ajross 17y ago
OK, I gotta ask: what's with the sudden interest in event-based I/O paradigms? This is hardly a new idea, in fact it's very well-traveled ground. It was used heavily in the 90's when threaded multiplexing was still new (and threads were expensive), and it works well enough. But it's difficult and error-prone. It splits sequential algorithms up into multiple callbacks with manually managed shared state. It makes "what happens next" a really hard problem. And it leads to some nasty state bugs, because conditions that are obvious in sequential code ("parse failed? Return ERROR_PARSE_FAIL to the top level request handler") become subtle (e.g.: parse failed, stash a failure result somewhere to be returned when the socket-is-closed callback happens, but oops! forget to actually close the socket so it lives as a zombie.) Really, I'm curious. What exactly is it that made this thing the snake oil of the week?
- RoboTeddy 17y ago
  If I had to speculate: - epoll, which was added to the linux kernel ~2002, makes it possible to hang on to potentially hundreds of thousands of concurrent connections on one machine. Just can't do that with threads/processes. - the desire for real time web applications, which have to hold large numbers of concurrent requests (e.g. friendfeed) - a focus on lowering web application latency, which can necessitate making data requests in parallel - advances in software (such as gevent, mentioned below) that reduce the complexity of writing async software - increasing use of remote services / slow requests over the internet (if you're working in a threading/process model, you'll need a process open for each concurrent request, which can become prohibitive) - web applications are simply serving more requests than they used to be, and probably have more to gain from being asynchronous Without async I/O operating a large modern web application would be a lot harder. Lots of huge sites rely on memcache, for example, which would suck if it didn't use event-based IO.
  - ajross 17y ago
    I don't think all of those items are really arguments about I/O paradigms, though. Again, event stuff has been here forever, and clearly is appropriate for some tasks. But it's a tradeoff: you have performance needs you can't get from a simple implementation so you resort to the fancy tricks. So if you're writing the client handler for a database, or something like memcached, you need this technique (whether you need an abstraction library is another argument...). But most apps don't fall into those categories, and I'm not getting any of that vibe from this. It seems that a lot of people sincerely believe that this stuff is "easier" or "better" just because it has the fancy new Spicy Event Sauce. And the truth isn't like that at all, and in fact these are truths that we've all known for years now...

50 points by silentbicycle 17y ago

========FOR WOMEN Christan Audigier bikini $25 Ed Hardy Bikini $25 Smful short_t-shirt_woman $15 ed hardy short_tank_woman $16 Sandal $32 christian louboutin $39 Sunglass $16 COACH_Necklace $27 handbag $36 AF tank woman $17 -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com ============FOR MAN -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com POLO new t-shirt man $16 ed hardy short t-shirt $17 POLO short $35 Ed Hardy board short man $33 jordan men $34 Sunglass $16 ed hardy cap $18 -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com -------------------------- http://www.uspopularbiz.com http://www.uspopularbiz.com

RoboTeddy 17y ago

Libevent is rock solid -- it handles the event loop for memcache, among other things. This guide looks like a great improvement/accompaniment to libevent's docs. One problem with using an event dispatcher like libevent is that libraries that make their own blocking calls won't cooperate. That means you'll often be stuck with limited capabilties, unless you write your own libraries or hack up existing ones. There's a Python project called gevent (http://www.gevent.org/ http://www.gevent.org/) that works atop libevent. It monkey-patches the python socket module with a version that automatically cooperates with libevent's loop. It uses greenlets to make code written sychronously run asynchronously behind the scenes. If, for example, you want to use an Amazon S3 library that makes use of python's httplib (which would normally block and choke your event loop), it'll automatically work with gevent/libevent. If you need to handle lots of concurrent connections and don't need to be working in C, check it out.

deleted 17y ago
[deleted]
houseabsolute 17y ago
It's too bad that no standard method for a asynchronous events has been developed for Unix in general. Where I work, there's one single library for concurrency, and every program or toolkit uses it. We do a lot of network programming, so it would be unthinkable for any performance intensive application to do blocking calls exclusively.
- silentbicycle 17y ago
  libevent tries to provide a common interface to select, poll, kselect (on BSD), epoll (on Linux), and comparable calls on Windows. It's not a standard part of Unix, of course, but it seems reasonably portable.
  - houseabsolute 17y ago
    Sure, but libraries underneath you have to cooperate for you to use it to the fullest potential. That no common means have been invented for that is what I'm sad about.
    - silentbicycle 17y ago
      Right. Once you set up a non-blocking event loop, it's much simpler when you can go async all the way down.
mcav 17y ago
Note that gevent is a fork of Eventlet (http://www.eventlet.org/ http://www.eventlet.org/), and there have been talks of reincorporating changes that went to gevent back into eventlet.
sophacles 17y ago
Also for python is pyevent, which works pretty well. I have used it to much success -- some people don't like the minimalist style. Also, the maintainer only updates sporadically, but I've never had real problems with it. http://code.google.com/p/pyevent/ http://code.google.com/p/pyevent/

ajross 17y ago

OK, I gotta ask: what's with the sudden interest in event-based I/O paradigms? This is hardly a new idea, in fact it's very well-traveled ground. It was used heavily in the 90's when threaded multiplexing was still new (and threads were expensive), and it works well enough. But it's difficult and error-prone. It splits sequential algorithms up into multiple callbacks with manually managed shared state. It makes "what happens next" a really hard problem. And it leads to some nasty state bugs, because conditions that are obvious in sequential code ("parse failed? Return ERROR_PARSE_FAIL to the top level request handler") become subtle (e.g.: parse failed, stash a failure result somewhere to be returned when the socket-is-closed callback happens, but oops! forget to actually close the socket so it lives as a zombie.) Really, I'm curious. What exactly is it that made this thing the snake oil of the week?

RoboTeddy 17y ago
If I had to speculate: - epoll, which was added to the linux kernel ~2002, makes it possible to hang on to potentially hundreds of thousands of concurrent connections on one machine. Just can't do that with threads/processes. - the desire for real time web applications, which have to hold large numbers of concurrent requests (e.g. friendfeed) - a focus on lowering web application latency, which can necessitate making data requests in parallel - advances in software (such as gevent, mentioned below) that reduce the complexity of writing async software - increasing use of remote services / slow requests over the internet (if you're working in a threading/process model, you'll need a process open for each concurrent request, which can become prohibitive) - web applications are simply serving more requests than they used to be, and probably have more to gain from being asynchronous Without async I/O operating a large modern web application would be a lot harder. Lots of huge sites rely on memcache, for example, which would suck if it didn't use event-based IO.
- ajross 17y ago
  I don't think all of those items are really arguments about I/O paradigms, though. Again, event stuff has been here forever, and clearly is appropriate for some tasks. But it's a tradeoff: you have performance needs you can't get from a simple implementation so you resort to the fancy tricks. So if you're writing the client handler for a database, or something like memcached, you need this technique (whether you need an abstraction library is another argument...). But most apps don't fall into those categories, and I'm not getting any of that vibe from this. It seems that a lot of people sincerely believe that this stuff is "easier" or "better" just because it has the fancy new Spicy Event Sauce. And the truth isn't like that at all, and in fact these are truths that we've all known for years now...