9 ms·

6 Line EventMachine Bugfix = 2x faster GC, 13x requests/sec

107 points by EvilTrout 17y ago

cperciva 17y ago
This is why I hate garbage collection -- it tends to break horribly as soon as someone does anything unexpected. Using 800 kB of stack is an incredibly dumb thing to do (not as bad on a 64-bit system as it is on a 32-bit system, admittedly), but it still shouldn't cause a huge drop in performance.
- staticshock 17y ago
  That's not an argument specific to garbage collection.
  - abstractbill 17y ago
    Right, the same argument has been made for building on top of anything other than the bare metal. At some point you have to accept that the productivity gains are worth trusting the substrate you're building on top of.
- briansmith 17y ago
  "It tends to break horribly as soon as someone does anything unexpected" is usually the argument used against manual memory management.
  - cperciva 17y ago
    No. Using malloc/free will break if your code is broken -- but you look at your code and prove that you're not using memory after it has been freed, not accessing memory locations outside of the allocated regions, and not allocating unbounded amounts of memory, then you have a guarantee that memory management won't cause new problems for you. With garbage collection it's possible to have perfectly correct code which nonetheless breaks horribly as a result of the garbage collector being "smart" in very stupid ways.
    - carterschonwald 17y ago
      the tricky bit is that pen and paper proofs break down when managing software level of complexity of that sort, and mechanized theorem proving is tough even with toy examples
      - dinkumthinkum 17y ago
        I don't think he's talking about a formal mathematical proof, rather inspecting the code and understanding it thoroughly such that you can show, without the Pumping Lemma or anything like that, that memory access is being done properly.
        nostrademons 17y ago
        ...which is also exceptionally difficult. Yeah, there're tricks you can use to make it easier, like RAII and thoroughly documenting ownership of parameters. But once you get past a few tens of thousands of lines, written by more than one programmer, it becomes essentially impossible to understand it thoroughly enough that you can tell by inspection whether it'll leak memory.
    - briansmith 17y ago
      You make that proof sound easy. In reality, proving that code is correct is one of the most difficult things that a programmer can be tasked with. And, even if you could program is correct, somebody could make a small change--even a single character--and invalidate your whole proof. That is the primary reason garbage collection exists.
      - psranga 17y ago
        If we interpret what cperciva said to mean 100% correct proofs you're correct. But you can do a lot of things in C++ to avoid memory leaks/reading from freed memory etc. Some of them are: 1) are auto-freeing templatized heap object class (the compiler will automatically call destructor when objects goes out of scope, call free in the destructor). 2) Overload free/delete to fill up freed memory with 0xdeadbeef etc so that code that reads from there is likely to crash due to invalid data. 3) Use one of the non-libc memory managers which will track memory allocs and frees and tell at exit if all memory was freed. Some will even have tricks to tell you which allocation was not freed. 4) If all else fails, use Valgrind for the really tough ones. Coupled with a good regression/unit test suite, memory leak tracking will be an insignificant part of your development effort.
  - biohacker42 17y ago
    You're both right! Obviously manual memory management is very dangerous and straight up not for the average programmer. But if you look thorough GC code by great hackers. You'll notice a style that looks almost like free/malloc in that they take deliberate care to make life easy for the GC. And on the other hand, programmers who are completely oblivious to how memory works, often write code which can confuse even the best GC so much that you essentially end up with a big memory leak. Strictly speaking the memory isn't leaking, the GC just can't quite dare clean it up. The lesson is, even the best VM and GC won't make bad programmers into good ones.
- patio11 17y ago
  This strikes me a lot like noting "Michael Jordan should never miss free throws. Heck, I can make free throws. Well, guess you can't trust NBA players at basketball." All programmers cause memory management issues, even the folks who write garbage collectors. Those folks have typically forgotten more about memory management than you or I will ever know. If you do not have a compelling reason to believe that you will be better than them quite literally all of the time, then you might want to learn to love the GC. (See also: trying to out-optimize both gcc and the processor pipelining which you don't really understand anyhow, reimplementing core features of your web stack to "do them right this time", etc etc.)
  - dinkumthinkum 17y ago
    Gotta love logoc. :)
- bcl 17y ago
  Garbage collection, like any other feature, is a tradeoff. For me the benefits of using a GC language like Python or Ruby far outweigh the edge cases where things go wonky. I certainly wouldn't reject a language just because it uses GC.
- jteo 17y ago
  sometimes, a Mark and Sweep GC is good enough. In this case, it wasn't.
- Daniel_Newby 17y ago
  "Using 800 kB of stack is an incredibly dumb thing to do ..., but it still shouldn't cause a huge drop in performance." Dunno about you, but if I accidentally use 800 kB of stack, I want the computer to emit fire every time the function is called.
abstractbill 17y ago
The real question is, why does Ruby still use a mark-and-sweep GC?
- wingo 17y ago
  That is not the issue at hand. The issue is conservative scanning of the stack.
rcoder 17y ago
And this, kids and kid-ettes, is why having someone who knows their way around your C compiler, assembler, and debugger on your team can be worthwhile, even if you're working in a high-level language like Ruby.
- judofyr 17y ago
  Even if you're working in a high-level language like a C extension to Ruby?
  - jey 17y ago
    They had to patch the C extension to make their Ruby code go faster. They weren't hacking the C extension to begin with.
    - scott_s 17y ago
      It certainly looks like a C++ module is part of their project: http://github.com/eventmachine/eventmachine/tree/master http://github.com/eventmachine/eventmachine/tree/master
- yan 17y ago
  I think in this case, knowing how a GC works was more important than knowing your way around the tools they used. Don't get me wrong, knowing what a compiler does, how to use gdb well, and the semantics of the stack well is important, but I think, in this scenario, understanding the garbage collection algorithm was more important than the tools they used.
- alexgartrell 17y ago
  As a representative of kids and kid-ettes, that stuff is actually still taught. In addition to writing malloc and a proxy, we (I'm a TA) have labs that are designed to force students to disassemble and reverse engineer compiled code. Further, this is actually a required course for Computer Science majors and Electrical and Computer Engineers at my school. We're keepin' it real http://www.cs.cmu.edu/~213/assignments.html http://www.cs.cmu.edu/~213/assignments.html
dinkumthinkum 17y ago
This was a very interesting discussion.
jacktang 17y ago
Well, one of my friends drop me one formula. Let x to be the rate of GC time out of total running time, and we can get (1-2x) * 14 = 1- x => x = 13/27 which means now ruby spends 13/27 time on GC, fast enough? (before patch, it spends 26/27 time on GC, really bad)
- dfranke 17y ago
  I don't see how you're getting this formula, but if it spends 13/27 on GC after a 2x GC speedup, then it was 26/40 before, not 26/27.
gojomo 17y ago
The local fix is nice, but I would be tempted to offer a solution for all code that does the same thing -- put a big chunk of GC-opaque data on the stack. It might take the form of informing the garbage collector: skip this range.

107 points by EvilTrout 17y ago

This is why I hate garbage collection -- it tends to break horribly as soon as someone does anything unexpected. Using 800 kB of stack is an incredibly dumb thing to do (not as bad on a 64-bit system as it is on a 32-bit system, admittedly), but it still shouldn't cause a huge drop in performance.

staticshock 17y ago
That's not an argument specific to garbage collection.
- abstractbill 17y ago
  Right, the same argument has been made for building on top of anything other than the bare metal. At some point you have to accept that the productivity gains are worth trusting the substrate you're building on top of.
briansmith 17y ago
"It tends to break horribly as soon as someone does anything unexpected" is usually the argument used against manual memory management.
- cperciva 17y ago
  No. Using malloc/free will break if your code is broken -- but you look at your code and prove that you're not using memory after it has been freed, not accessing memory locations outside of the allocated regions, and not allocating unbounded amounts of memory, then you have a guarantee that memory management won't cause new problems for you. With garbage collection it's possible to have perfectly correct code which nonetheless breaks horribly as a result of the garbage collector being "smart" in very stupid ways.
  - carterschonwald 17y ago
    the tricky bit is that pen and paper proofs break down when managing software level of complexity of that sort, and mechanized theorem proving is tough even with toy examples
    - dinkumthinkum 17y ago
      I don't think he's talking about a formal mathematical proof, rather inspecting the code and understanding it thoroughly such that you can show, without the Pumping Lemma or anything like that, that memory access is being done properly.
      - nostrademons 17y ago
        ...which is also exceptionally difficult. Yeah, there're tricks you can use to make it easier, like RAII and thoroughly documenting ownership of parameters. But once you get past a few tens of thousands of lines, written by more than one programmer, it becomes essentially impossible to understand it thoroughly enough that you can tell by inspection whether it'll leak memory.
  - briansmith 17y ago
    You make that proof sound easy. In reality, proving that code is correct is one of the most difficult things that a programmer can be tasked with. And, even if you could program is correct, somebody could make a small change--even a single character--and invalidate your whole proof. That is the primary reason garbage collection exists.
    - psranga 17y ago
      If we interpret what cperciva said to mean 100% correct proofs you're correct. But you can do a lot of things in C++ to avoid memory leaks/reading from freed memory etc. Some of them are: 1) are auto-freeing templatized heap object class (the compiler will automatically call destructor when objects goes out of scope, call free in the destructor). 2) Overload free/delete to fill up freed memory with 0xdeadbeef etc so that code that reads from there is likely to crash due to invalid data. 3) Use one of the non-libc memory managers which will track memory allocs and frees and tell at exit if all memory was freed. Some will even have tricks to tell you which allocation was not freed. 4) If all else fails, use Valgrind for the really tough ones. Coupled with a good regression/unit test suite, memory leak tracking will be an insignificant part of your development effort.
- biohacker42 17y ago
  You're both right! Obviously manual memory management is very dangerous and straight up not for the average programmer. But if you look thorough GC code by great hackers. You'll notice a style that looks almost like free/malloc in that they take deliberate care to make life easy for the GC. And on the other hand, programmers who are completely oblivious to how memory works, often write code which can confuse even the best GC so much that you essentially end up with a big memory leak. Strictly speaking the memory isn't leaking, the GC just can't quite dare clean it up. The lesson is, even the best VM and GC won't make bad programmers into good ones.
patio11 17y ago
This strikes me a lot like noting "Michael Jordan should never miss free throws. Heck, I can make free throws. Well, guess you can't trust NBA players at basketball." All programmers cause memory management issues, even the folks who write garbage collectors. Those folks have typically forgotten more about memory management than you or I will ever know. If you do not have a compelling reason to believe that you will be better than them quite literally all of the time, then you might want to learn to love the GC. (See also: trying to out-optimize both gcc and the processor pipelining which you don't really understand anyhow, reimplementing core features of your web stack to "do them right this time", etc etc.)
- dinkumthinkum 17y ago
  Gotta love logoc. :)
bcl 17y ago
Garbage collection, like any other feature, is a tradeoff. For me the benefits of using a GC language like Python or Ruby far outweigh the edge cases where things go wonky. I certainly wouldn't reject a language just because it uses GC.
jteo 17y ago
sometimes, a Mark and Sweep GC is good enough. In this case, it wasn't.
Daniel_Newby 17y ago
"Using 800 kB of stack is an incredibly dumb thing to do ..., but it still shouldn't cause a huge drop in performance." Dunno about you, but if I accidentally use 800 kB of stack, I want the computer to emit fire every time the function is called.

abstractbill 17y ago

The real question is, why does Ruby still use a mark-and-sweep GC?

wingo 17y ago
That is not the issue at hand. The issue is conservative scanning of the stack.

rcoder 17y ago

And this, kids and kid-ettes, is why having someone who knows their way around your C compiler, assembler, and debugger on your team can be worthwhile, even if you're working in a high-level language like Ruby.

judofyr 17y ago
Even if you're working in a high-level language like a C extension to Ruby?
- jey 17y ago
  They had to patch the C extension to make their Ruby code go faster. They weren't hacking the C extension to begin with.
  - scott_s 17y ago
    It certainly looks like a C++ module is part of their project: http://github.com/eventmachine/eventmachine/tree/master http://github.com/eventmachine/eventmachine/tree/master
yan 17y ago
I think in this case, knowing how a GC works was more important than knowing your way around the tools they used. Don't get me wrong, knowing what a compiler does, how to use gdb well, and the semantics of the stack well is important, but I think, in this scenario, understanding the garbage collection algorithm was more important than the tools they used.
alexgartrell 17y ago
As a representative of kids and kid-ettes, that stuff is actually still taught. In addition to writing malloc and a proxy, we (I'm a TA) have labs that are designed to force students to disassemble and reverse engineer compiled code. Further, this is actually a required course for Computer Science majors and Electrical and Computer Engineers at my school. We're keepin' it real http://www.cs.cmu.edu/~213/assignments.html http://www.cs.cmu.edu/~213/assignments.html

dinkumthinkum 17y ago

This was a very interesting discussion.

jacktang 17y ago

Well, one of my friends drop me one formula. Let x to be the rate of GC time out of total running time, and we can get (1-2x) * 14 = 1- x => x = 13/27 which means now ruby spends 13/27 time on GC, fast enough? (before patch, it spends 26/27 time on GC, really bad)

dfranke 17y ago
I don't see how you're getting this formula, but if it spends 13/27 on GC after a 2x GC speedup, then it was 26/40 before, not 26/27.

gojomo 17y ago

The local fix is nice, but I would be tempted to offer a solution for all code that does the same thing -- put a big chunk of GC-opaque data on the stack. It might take the form of informing the garbage collector: skip this range.