6 ms·

Alex Gaynor - Thoughts on HipHop PHP

71 points by twampss 17y ago

barrkel 17y ago
I wonder where this myth that "[websites] on the internet [are] I/O bound, not CPU bound" seemly implying that optimizing CPU usage is a waste of time, comes from. Since the I/O bound generally comes from latency, rather than total throughput, the number of concurrent connections a single webserver can handle is often proportional to how much memory and CPU resources each connection uses. If you have an asynchronous design for your server, concurrent connections don't cost physical threads - they just cost the bookkeeping overhead in the kernel. The faster you can switch between those connections, and get finished with work on them when I/Os complete, the fewer physical machines you need to serve a website to a given number of users. Or to put it another way, the saturation point of processing asynchronous I/Os is CPU-bound, even when speeding up individual requests is I/O-bound.
- wmf 17y ago
  When people say I/O-bound, I think they mean disk-bound. Using less CPU really won't help you there. Whether Web servers really are disk-bound is a different issue. If you have an asynchronous design for your server... This is PHP we're talking about.
  - barrkel 17y ago
    Asynchronous design just removes some of the memory pressure and a chunk of CPU loss from context switching. If your thread stacks are short and you have plentiful memory, you're still CPU-bound. But yes, if you have to hit a serialized disk somewhere for each and every request, then you really are screwed. But I don't think that's really often the case, is it? If it were, it would be a miracle that most web pages ever come up in under a second.
- jrockway 17y ago
  Yeah, exactly. The reality is that most web applications don't handle IO sanely, and have a 100-400MB process sit idle while waiting for the results of a database query (or for a slow client to suck down the rendered page). Using a proper threading system (and I mean lightweight threads, not OS threads) makes any application CPU intensive instead of IO-intensive. If the app doesn't use all the CPU, it's performing as well as possible. If it does use all the CPU, then making it run faster is a net gain for your users. This is PHP, though, and you would have to jump through a lot of hoops to remove the IO overhead. At least HipHop uses libevent, so at least apps won't be blocked by slow clients anymore.
  - mscarborough 17y ago
    >This is PHP, though, and you would have to jump through a lot of hoops to remove the IO overhead. At least HipHop uses libevent, so at least apps won't be blocked by slow clients anymore. Really, in PHP you don't have to jump through many hoops at all to minimize disk I/O. Employ APC opcode and user variable caching, build PHP's Memcached (with the 'd' on the end, not the old 'Memcache' lib) into your data layer, and you're good to go. If you're letting any process sit idle, much less a '100-400 MB' one, you're doing it wrong--in any language.
- tlack 17y ago
  I never understood that either. For years I've been hearing people say that web apps shouldn't worry about CPU speed, but in 100% of cases over the past ten years I've found servers with faster processors to be much less sluggish when serving both dynamic and static requests. In fact I've often perceived a bigger benefit from a faster processor rather than a marginally faster disk. I must be crazy I suppose..
- wanderr 17y ago
  Yeah, I pretty much gave up on reading the article at that line. It would be pretty awesome if we lived in a magical world where the performance of code running on a web server didn't matter, but that's just not the case. To know that, one needs simply run top on a busy web server.
- ubernostrum 17y ago
  Well... In a dynamic web application, at first, you will nearly always be database bound. Faster algorithms and faster programming-language implementations on web servers will do nothing whatsoever for this (and increasing the concurrent load of requests the web server can handle will in fact only overload the DB even more). That's when you start doing caching, and that's why caching has such dramatic effects. Once you've got your caching going nicely and your DB humming along, you will end up bandwidth bound. Not by your own pipe, but by clients; you may have a nice fat line running out of the data center, but your users may be on anything down to mobile phones or even dialup, and you'll only be able to push responses to them at the speed they can handle. This is the spoon-feeding problem, and once again algorithms and language implementations on the server can't do anything at all to help it. That's when you start putting fast, light, highly-concurrent reverse proxies (nginx appears to be winning the market share battle) in front of your actual web servers, and once again you will see a drastic effect. Or you combine caching and proxy into one component and do Varnish. Once you've done this, you might finally start to reach a point where you're genuinely I/O or CPU bound on a server that's actually running your application code. Or you might not; there are other roadblocks you might run into first. At any rate, optimizing CPU usage is, for the vast majority of websites, a waste of time at least until you've been through the phases I've outlined above. And, generally, I think you'll find that's the advice (the "myth") you've been hearing: fiddling with programming languages and algorithms is literally a net loss of performance until you've dealt with quite a few other (and more important, performance-wise) things.
  - barrkel 17y ago
    > Once you've got your caching going nicely and your DB humming along, you will end up bandwidth bound. Not by your own pipe, but by clients [...] This is the spoon-feeding problem, and once again algorithms and language implementations on the server can't do anything at all to help it. Here. Here is where you made the mistake in your assertions. > That's when you start putting fast, light, highly-concurrent reverse proxies (nginx appears to be winning the market share battle) in front of your actual web servers My point is that the number of web servers you need in this spoon-feeding case is inversely proportional to how CPU-optimized your servers are. The spoon-feeding problem, as you put it, is just bookkeeping in the kernel (keeping track of open sockets) and iteratively processing I/Os when they come due, and is CPU-bound, unless you actually have approaching 64K open sockets, in which case a reverse proxy won't do, you'll need DNS tricks etc.
jrockway 17y ago
My thoughts exactly; this is really not too exciting. (I will disagree that web apps are never CPU-bound. That has always been the bottleneck for me.) I haven't seen the source code yet, but I think Chicken Scheme did everything HipHop does but many years ago. And, there are some hard problems to solve when implementing Scheme, notably handling continuations. PHP is much simpler in comparison. I do see good things coming out of this project, though. As people are enticed by the biggest temptress in computing, speed, they'll learn to run and deploy web applications that aren't as simple as "ftp this HTML file to a directory". Once PHP's only advantage over other languages is gone, and developers realize that it wasn't that big of an advantage, they'll switch to better programming languages. Then we can forget PHP ever happened, and the field can move on!
- pvg 17y ago
  You think PHP is more likely to decline in popularity because there's now a way to run it faster? I suspect you are in for a very long surprise.
scott_s 17y ago
Because of differences like this ... I believe that the work done on HipHop represents a fundamentally smaller challenge than that taken on by the teams working to improve the implementations of languages like Python, Ruby, or Javascript. I think this is true. On the other hand, it's not Facebook's job to do research. If they can benefit equally from solving a simpler problem, then that's what they should do. I agree with the author's points, I just want to make explicit that Facebook didn't make wrong choices. It's just that their work is probably of limited value to everyone else.
- pvg 17y ago
  There are many more people working on PHP apps than there are people working on dynamic language runtimes. Facebook's work is of limited value to everyone else for a limited value of 'everyone else'.
  - scott_s 17y ago
    I'm taking the author's arguments at face value, that their project is of limited value to others because most PHP apps aren't CPU bound, and that they only handle a subset of the language.
ericb 17y ago
Almost every single website on the internet is I/O bound, not CPU bound This claim is patently false. Under load, a surprisingly large percentage of applications are CPU bound. Disclaimer: I make my living doing web application load tests on a consultative basis--I don't have a study to cite offhand, just experience. I should publish a study, but that's a different issue.
- ehsanul 17y ago
  That's interesting. I'm sure some people here, like me, believe the opposite, since I/O is so much slower than processors now. Can you give a bit more of an explanation as to what kind of applications are CPU bound and why?
  - wmf 17y ago
    I/O is slow if you actually do any, but RAM is so cheap that many Web sites shouldn't need much I/O.
    - nixy 17y ago
      I think that by I/O in this case people mean network I/O, not disk I/O. Edit: Sorry, maybe I was jumping to conclusions. Databases and files are of course not always cached in-memory.
  - ericb 17y ago
    What wmf said sums it up. In terms of who is CPU bound, it often points to a need for caching or indexing. All it really takes to be CPU bound is to use up more CPU than your hardware can offer. Once you are CPU bound, you're unlikely to push traffic through fast enough to be network or disk I/O bound. If you're caught on something else, like CPU, the speed of those things won't matter much.
- loganfrederick 17y ago
  It seems like you're both right. You qualified your statement with "under load". Are a majority of websites/web applications under a significant load? I'd guess the real answer is that, for most websites, they're I/O bound due to the process of sending data between the client and server. However, heavier applications and sites with significant traffic then require more CPU processing, thus becoming CPU bound. I don't have any real expertise, but from everything I've read and seen discussed, this is my interpretation of the issue.
coffeemug 17y ago
Firstly, there's the question of what problem HipHop solves. It reduces infrastructure expenses of the top 100 web properties so much that the pain of building it, testing it, rolling it out, administering it, and maintaining it is worth it.
leej 17y ago
he may meant "it is not as CPU-bound as in the case of rendering or en/decoding video".

71 points by twampss 17y ago

I wonder where this myth that "[websites] on the internet [are] I/O bound, not CPU bound" seemly implying that optimizing CPU usage is a waste of time, comes from. Since the I/O bound generally comes from latency, rather than total throughput, the number of concurrent connections a single webserver can handle is often proportional to how much memory and CPU resources each connection uses. If you have an asynchronous design for your server, concurrent connections don't cost physical threads - they just cost the bookkeeping overhead in the kernel. The faster you can switch between those connections, and get finished with work on them when I/Os complete, the fewer physical machines you need to serve a website to a given number of users. Or to put it another way, the saturation point of processing asynchronous I/Os is CPU-bound, even when speeding up individual requests is I/O-bound.

wmf 17y ago
When people say I/O-bound, I think they mean disk-bound. Using less CPU really won't help you there. Whether Web servers really are disk-bound is a different issue. If you have an asynchronous design for your server... This is PHP we're talking about.
- barrkel 17y ago
  Asynchronous design just removes some of the memory pressure and a chunk of CPU loss from context switching. If your thread stacks are short and you have plentiful memory, you're still CPU-bound. But yes, if you have to hit a serialized disk somewhere for each and every request, then you really are screwed. But I don't think that's really often the case, is it? If it were, it would be a miracle that most web pages ever come up in under a second.
jrockway 17y ago
Yeah, exactly. The reality is that most web applications don't handle IO sanely, and have a 100-400MB process sit idle while waiting for the results of a database query (or for a slow client to suck down the rendered page). Using a proper threading system (and I mean lightweight threads, not OS threads) makes any application CPU intensive instead of IO-intensive. If the app doesn't use all the CPU, it's performing as well as possible. If it does use all the CPU, then making it run faster is a net gain for your users. This is PHP, though, and you would have to jump through a lot of hoops to remove the IO overhead. At least HipHop uses libevent, so at least apps won't be blocked by slow clients anymore.
- mscarborough 17y ago
  >This is PHP, though, and you would have to jump through a lot of hoops to remove the IO overhead. At least HipHop uses libevent, so at least apps won't be blocked by slow clients anymore. Really, in PHP you don't have to jump through many hoops at all to minimize disk I/O. Employ APC opcode and user variable caching, build PHP's Memcached (with the 'd' on the end, not the old 'Memcache' lib) into your data layer, and you're good to go. If you're letting any process sit idle, much less a '100-400 MB' one, you're doing it wrong--in any language.
tlack 17y ago
I never understood that either. For years I've been hearing people say that web apps shouldn't worry about CPU speed, but in 100% of cases over the past ten years I've found servers with faster processors to be much less sluggish when serving both dynamic and static requests. In fact I've often perceived a bigger benefit from a faster processor rather than a marginally faster disk. I must be crazy I suppose..
wanderr 17y ago
Yeah, I pretty much gave up on reading the article at that line. It would be pretty awesome if we lived in a magical world where the performance of code running on a web server didn't matter, but that's just not the case. To know that, one needs simply run top on a busy web server.
ubernostrum 17y ago
Well... In a dynamic web application, at first, you will nearly always be database bound. Faster algorithms and faster programming-language implementations on web servers will do nothing whatsoever for this (and increasing the concurrent load of requests the web server can handle will in fact only overload the DB even more). That's when you start doing caching, and that's why caching has such dramatic effects. Once you've got your caching going nicely and your DB humming along, you will end up bandwidth bound. Not by your own pipe, but by clients; you may have a nice fat line running out of the data center, but your users may be on anything down to mobile phones or even dialup, and you'll only be able to push responses to them at the speed they can handle. This is the spoon-feeding problem, and once again algorithms and language implementations on the server can't do anything at all to help it. That's when you start putting fast, light, highly-concurrent reverse proxies (nginx appears to be winning the market share battle) in front of your actual web servers, and once again you will see a drastic effect. Or you combine caching and proxy into one component and do Varnish. Once you've done this, you might finally start to reach a point where you're genuinely I/O or CPU bound on a server that's actually running your application code. Or you might not; there are other roadblocks you might run into first. At any rate, optimizing CPU usage is, for the vast majority of websites, a waste of time at least until you've been through the phases I've outlined above. And, generally, I think you'll find that's the advice (the "myth") you've been hearing: fiddling with programming languages and algorithms is literally a net loss of performance until you've dealt with quite a few other (and more important, performance-wise) things.
- barrkel 17y ago
  > Once you've got your caching going nicely and your DB humming along, you will end up bandwidth bound. Not by your own pipe, but by clients [...] This is the spoon-feeding problem, and once again algorithms and language implementations on the server can't do anything at all to help it. Here. Here is where you made the mistake in your assertions. > That's when you start putting fast, light, highly-concurrent reverse proxies (nginx appears to be winning the market share battle) in front of your actual web servers My point is that the number of web servers you need in this spoon-feeding case is inversely proportional to how CPU-optimized your servers are. The spoon-feeding problem, as you put it, is just bookkeeping in the kernel (keeping track of open sockets) and iteratively processing I/Os when they come due, and is CPU-bound, unless you actually have approaching 64K open sockets, in which case a reverse proxy won't do, you'll need DNS tricks etc.

jrockway 17y ago

My thoughts exactly; this is really not too exciting. (I will disagree that web apps are never CPU-bound. That has always been the bottleneck for me.) I haven't seen the source code yet, but I think Chicken Scheme did everything HipHop does but many years ago. And, there are some hard problems to solve when implementing Scheme, notably handling continuations. PHP is much simpler in comparison. I do see good things coming out of this project, though. As people are enticed by the biggest temptress in computing, speed, they'll learn to run and deploy web applications that aren't as simple as "ftp this HTML file to a directory". Once PHP's only advantage over other languages is gone, and developers realize that it wasn't that big of an advantage, they'll switch to better programming languages. Then we can forget PHP ever happened, and the field can move on!

pvg 17y ago
You think PHP is more likely to decline in popularity because there's now a way to run it faster? I suspect you are in for a very long surprise.

scott_s 17y ago

Because of differences like this ... I believe that the work done on HipHop represents a fundamentally smaller challenge than that taken on by the teams working to improve the implementations of languages like Python, Ruby, or Javascript. I think this is true. On the other hand, it's not Facebook's job to do research. If they can benefit equally from solving a simpler problem, then that's what they should do. I agree with the author's points, I just want to make explicit that Facebook didn't make wrong choices. It's just that their work is probably of limited value to everyone else.

pvg 17y ago
There are many more people working on PHP apps than there are people working on dynamic language runtimes. Facebook's work is of limited value to everyone else for a limited value of 'everyone else'.
- scott_s 17y ago
  I'm taking the author's arguments at face value, that their project is of limited value to others because most PHP apps aren't CPU bound, and that they only handle a subset of the language.

ericb 17y ago

Almost every single website on the internet is I/O bound, not CPU bound This claim is patently false. Under load, a surprisingly large percentage of applications are CPU bound. Disclaimer: I make my living doing web application load tests on a consultative basis--I don't have a study to cite offhand, just experience. I should publish a study, but that's a different issue.

ehsanul 17y ago
That's interesting. I'm sure some people here, like me, believe the opposite, since I/O is so much slower than processors now. Can you give a bit more of an explanation as to what kind of applications are CPU bound and why?
- wmf 17y ago
  I/O is slow if you actually do any, but RAM is so cheap that many Web sites shouldn't need much I/O.
  - nixy 17y ago
    I think that by I/O in this case people mean network I/O, not disk I/O. Edit: Sorry, maybe I was jumping to conclusions. Databases and files are of course not always cached in-memory.
- ericb 17y ago
  What wmf said sums it up. In terms of who is CPU bound, it often points to a need for caching or indexing. All it really takes to be CPU bound is to use up more CPU than your hardware can offer. Once you are CPU bound, you're unlikely to push traffic through fast enough to be network or disk I/O bound. If you're caught on something else, like CPU, the speed of those things won't matter much.
loganfrederick 17y ago
It seems like you're both right. You qualified your statement with "under load". Are a majority of websites/web applications under a significant load? I'd guess the real answer is that, for most websites, they're I/O bound due to the process of sending data between the client and server. However, heavier applications and sites with significant traffic then require more CPU processing, thus becoming CPU bound. I don't have any real expertise, but from everything I've read and seen discussed, this is my interpretation of the issue.

coffeemug 17y ago

Firstly, there's the question of what problem HipHop solves. It reduces infrastructure expenses of the top 100 web properties so much that the pain of building it, testing it, rolling it out, administering it, and maintaining it is worth it.

leej 17y ago

he may meant "it is not as CPU-bound as in the case of rendering or en/decoding video".