5 ms·

Bjoern: A screamingly fast Python WSGI server written in C.

52 points by jgalvez 16y ago

jcw 16y ago
I've been doing some research on small web servers, trying to understand didiwiki's code (a web server and wiki engine in ~2k lines of C): http://c2.com/cgi/wiki?DidiWiki http://c2.com/cgi/wiki?DidiWiki This is a nice introduction to writing a minimal web server: http://www.ibm.com/developerworks/systems/library/es-nweb/index.html http://www.ibm.com/developerworks/systems/library/es-nweb/in...
- abecedarius 16y ago
  Thanks for pointing that out -- I'll want to look didiwiki over since I also wrote a webserver/wiki in <2k lines of C, long ago: http://wry.me/~darius/hacks/ikiwiki.tar.gz http://wry.me/~darius/hacks/ikiwiki.tar.gz There's usually something to learn from anyone else tackling the same problem.
daeken 16y ago
This looks like a really cool project, but the comparisons of other webservers are uh, a bit on the negative side. No code is perfect, even if it's better than the rest. While I find myself falling into the same thinking, it's rarely a good idea, and it certainly doesn't help your project's image.
- beoba 16y ago
  I'm wondering why it says that it has "no threads, coroutines or other crap", and then complains about difficulty with "tr[ying] to patch Fapws so that it would support threading".
- jgalvez 16y ago
  I feel the same way. His arrogance is strangely reassuring though heh.
- scrod 16y ago
  >..."Fapws done right" I know if I were the author of Fapws I would probably find that section a little irritating.
  - mfukar 16y ago
    Only if he wouldn't concede that his software (like everyone else's) has deficiencies.
yashh 16y ago
Interesting. I think I am gonna benchmark this against gunicorn with a simple flask app.
- yashh 16y ago
  Ok took 30 minutes to run some benchmarks on a AWS instance. I used a small instance run a simple Flask Hello World app with gunicorn (1 worker and 4 workers) and bjoern. Frankly I was not able to fund any speed difference between both the servers. Here is the gist of the benchmark results https://gist.github.com/753998 https://gist.github.com/753998
  - njharman 16y ago
    > n. Frankly I was not able to fund any speed difference between both the servers. That is because of this http://news.ycombinator.com/item?id=2037060 http://news.ycombinator.com/item?id=2037060
  - jokull 16y ago
    What about memory consumption?
  - jgalvez 16y ago
    Is anyone else confused by "1 worker and 4 workers"?
    - yashh 16y ago
      Well I run gunicorn in production with 4 workers so just wanted to give it a try. Also I chose to add network latency by doing it on amazon instance.
    - thegoleffect 16y ago
      My guess is he meant 1 master & 4 workers.
    - oozcitak 16y ago
      He did 3 benchmarks: gunicorn with 1 worker process, gunicorn with 4 worker processes, bjoern.
wildmXranat 16y ago
So he started off a bit aggressively, so what. I looked at the code and it's very spartan and light. It's quite the opposite of the competition and it works for him. Good job on what looks like to be an advanced coders Hello World: web server.
jrockway 16y ago
Is the name a reference to the main character of Peggle? (Also, how does this compare speed-wise to Mongrel2? It easily handles the 4000 requests/second that my PSGI handler can do. I am sure it could do more if the backend was faster.)
- StavrosK 16y ago
  Björn is hardly an uncommon name, as I understand it.
kennu 16y ago
Any experiences with Bjoern's Django compatibility?
- est 16y ago
  Bjoern is a WSGI server, so in theory you just have to run Django using WSGI http://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/ http://docs.djangoproject.com/en/dev/howto/deployment/modwsg... http://code.djangoproject.com/wiki/django_apache_and_mod_wsgi http://code.djangoproject.com/wiki/django_apache_and_mod_wsg...
FooBarWidget 16y ago
I think all emphasis on server software speed is overrated. In pretty much everything but hello world, the vast majority of the time is spent inside the web app, not in the server. Yes, lowering the web server overhead is a good thing. But I think in most cases it's already so low that further reducing this overhead results in no noticeable impact in real-life scenarios. At some point you'll just be benchmarking how fast the kernel is at doing connect(), read() and write() - in other words how fast your computer can do nothing. For example, let's consider this thought experiment: Someone here mentioned Mongrel2 getting 4000 req/sec. Let's replace the name "Mongrel2" with "Server A" because this thought experiment is not limited to Mongrel2, but all servers. I assume he's benchmarking a hello world app on his laptop. Suppose that a hypothetical Server B gets "only" 2000 req/sec. One might now (mistakenly) conclude that: - Server B is a lot slower. - One should use Server A instead of Server B in high-traffic production environments. Now put Server A behind HAProxy. HAproxy is known as a high-performance HTTP proxy server with minimal overhead. Benchmark this setup, and watch req/sec drop to about 2000-3000 (when benchmarked on a typical dual core laptop). What just happened? Server B appears to be very slow. But the reality is that both Server A and Server B are so fast that doing even a minimum amount of extra work will have a significant effect on the req/sec number. In this case, the overhead of an extra context switch and a read()/write() call to the kernel is already enough to make the req/sec number drop by half. Any reasonably complex web app logic will make the number drop so much that the performance difference between the different servers become negligible.
- sliverstorm 16y ago
  Why on earth should developers ever say "Ok, it's fast enough. We're done." I mean, maybe temporarily done so that you can patch up some higher priority (more broken?) stuff, but DONE? Pushing the envelope is how you progress, and in this case also how you can learn.
  - njharman 16y ago
    Because increasing benchmark speed provides no business value (assuming your business isn't self-promotion, getting into flamewars, etc). There's a extremely well known saying that speaks to this issue, goes something like "Preoptimiztion is root of all evil."
verysimple 16y ago
Thanks for this. One of my goals for 2011 is to get into network programming. At less than 1000 lines of code, I think I may just have to go through the source.

52 points by jgalvez 16y ago

jcw 16y ago

I've been doing some research on small web servers, trying to understand didiwiki's code (a web server and wiki engine in ~2k lines of C): http://c2.com/cgi/wiki?DidiWiki http://c2.com/cgi/wiki?DidiWiki This is a nice introduction to writing a minimal web server: http://www.ibm.com/developerworks/systems/library/es-nweb/index.html http://www.ibm.com/developerworks/systems/library/es-nweb/in...

abecedarius 16y ago
Thanks for pointing that out -- I'll want to look didiwiki over since I also wrote a webserver/wiki in <2k lines of C, long ago: http://wry.me/~darius/hacks/ikiwiki.tar.gz http://wry.me/~darius/hacks/ikiwiki.tar.gz There's usually something to learn from anyone else tackling the same problem.

daeken 16y ago

This looks like a really cool project, but the comparisons of other webservers are uh, a bit on the negative side. No code is perfect, even if it's better than the rest. While I find myself falling into the same thinking, it's rarely a good idea, and it certainly doesn't help your project's image.

beoba 16y ago
I'm wondering why it says that it has "no threads, coroutines or other crap", and then complains about difficulty with "tr[ying] to patch Fapws so that it would support threading".
jgalvez 16y ago
I feel the same way. His arrogance is strangely reassuring though heh.
scrod 16y ago
>..."Fapws done right" I know if I were the author of Fapws I would probably find that section a little irritating.
- mfukar 16y ago
  Only if he wouldn't concede that his software (like everyone else's) has deficiencies.

yashh 16y ago

Interesting. I think I am gonna benchmark this against gunicorn with a simple flask app.

yashh 16y ago
Ok took 30 minutes to run some benchmarks on a AWS instance. I used a small instance run a simple Flask Hello World app with gunicorn (1 worker and 4 workers) and bjoern. Frankly I was not able to fund any speed difference between both the servers. Here is the gist of the benchmark results https://gist.github.com/753998 https://gist.github.com/753998
- njharman 16y ago
  > n. Frankly I was not able to fund any speed difference between both the servers. That is because of this http://news.ycombinator.com/item?id=2037060 http://news.ycombinator.com/item?id=2037060
- jokull 16y ago
  What about memory consumption?
- jgalvez 16y ago
  Is anyone else confused by "1 worker and 4 workers"?
  - yashh 16y ago
    Well I run gunicorn in production with 4 workers so just wanted to give it a try. Also I chose to add network latency by doing it on amazon instance.
  - thegoleffect 16y ago
    My guess is he meant 1 master & 4 workers.
  - oozcitak 16y ago
    He did 3 benchmarks: gunicorn with 1 worker process, gunicorn with 4 worker processes, bjoern.

wildmXranat 16y ago

So he started off a bit aggressively, so what. I looked at the code and it's very spartan and light. It's quite the opposite of the competition and it works for him. Good job on what looks like to be an advanced coders Hello World: web server.

jrockway 16y ago

Is the name a reference to the main character of Peggle? (Also, how does this compare speed-wise to Mongrel2? It easily handles the 4000 requests/second that my PSGI handler can do. I am sure it could do more if the backend was faster.)

StavrosK 16y ago
Björn is hardly an uncommon name, as I understand it.

kennu 16y ago

Any experiences with Bjoern's Django compatibility?

est 16y ago
Bjoern is a WSGI server, so in theory you just have to run Django using WSGI http://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/ http://docs.djangoproject.com/en/dev/howto/deployment/modwsg... http://code.djangoproject.com/wiki/django_apache_and_mod_wsgi http://code.djangoproject.com/wiki/django_apache_and_mod_wsg...

FooBarWidget 16y ago

I think all emphasis on server software speed is overrated. In pretty much everything but hello world, the vast majority of the time is spent inside the web app, not in the server. Yes, lowering the web server overhead is a good thing. But I think in most cases it's already so low that further reducing this overhead results in no noticeable impact in real-life scenarios. At some point you'll just be benchmarking how fast the kernel is at doing connect(), read() and write() - in other words how fast your computer can do nothing. For example, let's consider this thought experiment: Someone here mentioned Mongrel2 getting 4000 req/sec. Let's replace the name "Mongrel2" with "Server A" because this thought experiment is not limited to Mongrel2, but all servers. I assume he's benchmarking a hello world app on his laptop. Suppose that a hypothetical Server B gets "only" 2000 req/sec. One might now (mistakenly) conclude that: - Server B is a lot slower. - One should use Server A instead of Server B in high-traffic production environments. Now put Server A behind HAProxy. HAproxy is known as a high-performance HTTP proxy server with minimal overhead. Benchmark this setup, and watch req/sec drop to about 2000-3000 (when benchmarked on a typical dual core laptop). What just happened? Server B appears to be very slow. But the reality is that both Server A and Server B are so fast that doing even a minimum amount of extra work will have a significant effect on the req/sec number. In this case, the overhead of an extra context switch and a read()/write() call to the kernel is already enough to make the req/sec number drop by half. Any reasonably complex web app logic will make the number drop so much that the performance difference between the different servers become negligible.

sliverstorm 16y ago
Why on earth should developers ever say "Ok, it's fast enough. We're done." I mean, maybe temporarily done so that you can patch up some higher priority (more broken?) stuff, but DONE? Pushing the envelope is how you progress, and in this case also how you can learn.
- njharman 16y ago
  Because increasing benchmark speed provides no business value (assuming your business isn't self-promotion, getting into flamewars, etc). There's a extremely well known saying that speaks to this issue, goes something like "Preoptimiztion is root of all evil."

verysimple 16y ago

Thanks for this. One of my goals for 2011 is to get into network programming. At less than 1000 lines of code, I think I may just have to go through the source.