7 ms·

Ruby

51 points by frostmatthew 12y ago

TheLoneWolfling 12y ago
Read: "The overhead of abstraction in languages that do optimizing compilation versus languages that don't".
- haberman 12y ago
  Also: CPython and MRI don't do optimizing compilation, which is not entirely obvious considering they both do compile to bytecode and optimize slightly.
  - TheLoneWolfling 12y ago
    Don't know about MRI. With Python (as it's the reference implementation, I don't see the need to call it by another name) all, IIRC, compilation does with optimization enabled is remove assert and if __debug__: statements. Oh, and docstrings. So yeah... slightly. To put it mildly. (Note that with this "benchmark", PyPy has (almost) no difference in runtime between the abstracted and non-abstracted versions.)
TillE 12y ago
The "C with classes" code looks so weird. I'm a little surprised you got away with int main() not explicitly returning 0, didn't realize that was still valid C++. Anyway, C doesn't really do abstractions to any great extent, so it would make more sense to focus only on C++ and delve a little deeper. Observe what happens when you make a method virtual, for example.
- jeorgun 12y ago
  V-tables aren't a very expensive abstraction. They prevent inlining, which isn't exactly ideal (hell, that's the main point of the post), but nowadays their cost over a normal function call is basically negligible.
  - coherentpony 12y ago
    > V-tables aren't a very expensive abstraction. They prevent inlining, which isn't exactly ideal (hell, that's the main point of the post), but nowadays their cost over a normal function call is basically negligible. This is not true. Sweeping general statements like this are almost never true. It is really problem dependent whether a particular type of abstraction is 'too expensive' or not. I have to actively avoid vtbl lookups in my domain, and this is because if I don't I lose about 15% in execute time. You might think 15% "isn't very long", but it is when your application uses several thousand MPI processes.
    - jasode 12y ago
      > I lose about 15% in execute time. Can you paste a representative snippet of your code that demonstrates this 15% penalty? In my experience with tight loops executing a million iterations, the extra indirection of a vtable lookup is never more than 1% slower. Others also observe similar minor differences.[1] [1]http://stackoverflow.com/a/667680 http://stackoverflow.com/a/667680
      - haberman 12y ago
        I have definitely observed penalties much greater than 1% (closer to 15%). This paper is old, but measures the direct cost of virtual table lookups (not taking into account indirect costs arising from the inability to inline) as 5% in real C++ programs. When they converted the C++ programs to use all virtual functions, the overhead rose to 13.7%: http://www.cs.ucsb.edu/~urs/oocsb/papers/oopsla96.pdf http://www.cs.ucsb.edu/~urs/oocsb/papers/oopsla96.pdf
        jasode 12y ago
        I didn't make it clear but I wasn't comparing inline to virtual.[1] That type of drastic difference can be greater than 10% which is understandable. I was comparing non-inline function to virtual function and I haven't been able to recreate a 15% slowdown even with zero compiler optimization switches. The stackoverflow observation and mine were on more modern cpus. Maybe that has something to do with the conflicting anecdotes. If someone has a small snippet of code that shows non-inlined function calls running 15% faster than vtable calls, I'd like to study it. [1]the context in grandparent post was already constrained to (non-inlined) normal function calls: ", but nowadays their cost over a normal function call is basically negligible." https://news.ycombinator.com/item?id=8476208 https://news.ycombinator.com/item?id=8476208
- StephanTLavavej 12y ago
  C++ did that first. C++98 and C99 implicitly "return 0;" when falling off the end of main(). It was C89 that required "return 0;" here.
mackwic 12y ago
Even though the question is good, the article answer is disappointing. First, I really don't like the angle taken, then the question of abstraction (why we do it, how) and choices made by languages designers (and variations in their idioms) are so vast that you really can't treat the question through a <1000 signs blog post. Some quick points: - You don't code for the machine, you code to be read by another human (possibly you) in the future. I insist, you will be read regularly and frequently. Thus, your code needs to be clear, precise, and concise. This must be the first thing in mind when coding: program what need to be done in a way that a fellow stranger could understand. - Abstraction is a way to keep a structure of code clear when the interactions become complex and/or abundant. If you can avoid them when still being crystal clear in your code, do it. Direct speaking is always better than convolution. - The main requirements when you code are often one or two of: quick to develop, easy to maintain, extensible, efficient (you control your big-O and _WORST_ exec times), correct (no bug. at. all.), real time (when X happens, Y is done between n µs and m µs during p ns) - So, know when performance is a goal, and know when it's not. Choose your language, your techs, your team considering these goals. - And yeah, C89, C99 and the C++es have a very high overhead of abstraction: clarity, concision, and sometimes performance (all abstractions can't be inlined). Think of it.
- haberman 12y ago
  You are speaking in vague platitudes, most of which don't contradict what I wrote in the article at all. I was speaking of a specific context in which I was working (writing C or C++ extensions for Python and Ruby), and addressing a specific design question (how much of the library should be in C or C++ vs. Python or Ruby). I guess I didn't specifically say this, but I thought I made it pretty obvious that I'm working in a situation where performance is a factor. My observations were also specific to the question of the relative cost of creating layering abstractions in Python/Ruby vs. C/C++. > And yeah, C89, C99 and the C++es have a very high overhead of abstraction: clarity, concision Now you're just being silly. Abstractions are sometimes good, making things clearer and more concise. The wrong abstractions can make code less clear and concise. Here is my solution to that problem: if an abstraction is not making your code clearer and more concise, don't use it. Saying that abstractions in C and C++ are inherently less clear and concise just shows anti-C/C++ bias.
jerven 12y ago
Maybe, this person should understand what they are timing. The time command is a wonderful tool, but comparing the execution time of a 4 line C program with the start up cost of a VM and interpreter is inane. One needs to measure the program doing the real work. And in this case for the C program you might find that bash forking is the actual cost not the C code running through the loop. JRuby and PyPy are starting to do things with their specialisation tricks that mean that the overhead of abstraction is 1000 loops before it goes away. A performance benchmark running for less than a few minutes is a joke. Startup variance alone will dominate the results.
- Chris911 12y ago
  Except he's not comparing the C program and the Python one. He's comparing the time difference (or ratio) for each language with and without abstraction.
- haberman 12y ago
  > comparing the execution time of a 4 line C program with the start up cost of a VM and interpreter is inane. That's not what I compared. Maybe you should read the article again? > And in this case for the C program you might find that bash forking is the actual cost not the C code running through the loop. I didn't publish any benchmark numbers of any C programs, because I didn't need to: the two C++ programs I was comparing compiled to exactly the same machine code, making empirical observations of their execution time immaterial.
theflubba 12y ago
This article conflates abstraction with OO and language features, IMO. Putting some functionality into an object is not as interesting as running that functionality as a monadic command. Would be more interesting to run some benchmarks in Haskell using functional abstractions rather than OO machinery.
jaimebuelta 12y ago
The problem with this article is that is doing something extremely simple, and calling that an abstraction. The power of Python or Ruby is not demonstrated doing a for loop, is with something like this: print sum(int(n) for n in open('/a/file/with/numbers')) We don't need to care about a lot of stuff that we would in C/C++ (memory management, iterate through a file, the format of stuff, terminating the loop properly) Are C/C++ interesting languages? Of course, sometimes you need that extra level of control about what's going on. But I don't agree that the cost of abstraction is higher in Python/Ruby. You can create code that does A LOT in a few lines, that most of the times will have a good enough speed. In C/C++ you need a lot of work to create a good abstraction, that's not guaranteed to perform well.
- vinkelhake 12y ago
  It's not quite that bad. To sum the numbers in a file you've got a line to open the file and another to call std::accumulate with istream iterators. Memory management? Doesn't come up. Terminating the loop? What loop?
- ajtulloch 12y ago
  FWIW, with folly::gen [1], this is just: using namespace folly::gen; byLine('/a/file/with/numbers') | eachTo<int> | sum; https://github.com/facebook/folly/tree/master/folly/gen https://github.com/facebook/folly/tree/master/folly/gen
  - seri 12y ago
    Thank you for this. Got me really interested in folly now.
  - nly 12y ago
    It's only 2 lines even restricting yourself to the standard library #include <fstream> #include <iterator> #include <numeric> using namespace std; int main() { ifstream nf ("numbers.txt"); return accumulate (istream_iterator<int>(nf), istream_iterator<int>(), 0); }
- 12y ago
zem 12y ago
i was expecting just the opposite - the overhead in terms of programmer time and effort involved in abstracting something out versus just copy/pasting the code.
tokenrove 12y ago
It would be interesting to see the same comparison with a whole-program optimizing compiler like MLton, especially with the abstraction split across source file boundaries (an area where C compilers have gotten much stronger lately).
chrisseaton 12y ago
A while ago I wrote a blog post about how compilers for Ruby can remove abstraction. Features that you might think of as being expensive abstractions such as using #send to call methods instead of calling them directly, need not have any overhead at all with a good enough compiler http://www.chrisseaton.com/rubytruffle/pushing-pixels/ http://www.chrisseaton.com/rubytruffle/pushing-pixels/
sbov 12y ago
The overhead of abstraction is slightly more than 1.5 addition operations? Sign me up.
astrobe_ 12y ago
I think that this observation is not as relevant to the task at hand that it seems. The author is in the same position as the implementor of a VM, who has to decide which operation will be a primitive and which will not be. What is actually more important than anything else if you care about performance is which operations are likely to be called from hot loops. It's classic optimization wisdom except that you can only work on expected scenarios rather than real-world uses (until you release your stuff and other people use it). So a 100x slower abstraction is ok(-ish) as long as it's not somthing to be used in performance critical sections.
- haberman 12y ago
  Your comment amounts to: "this performance-related observation doesn't apply in situations where performance is not an issue." Yes, but I am working in a situation where performance is an issue. So yes, the observation is relevant to the task at hand.

51 points by frostmatthew 12y ago

TheLoneWolfling 12y ago

Read: "The overhead of abstraction in languages that do optimizing compilation versus languages that don't".

haberman 12y ago
Also: CPython and MRI don't do optimizing compilation, which is not entirely obvious considering they both do compile to bytecode and optimize slightly.
- TheLoneWolfling 12y ago
  Don't know about MRI. With Python (as it's the reference implementation, I don't see the need to call it by another name) all, IIRC, compilation does with optimization enabled is remove assert and if __debug__: statements. Oh, and docstrings. So yeah... slightly. To put it mildly. (Note that with this "benchmark", PyPy has (almost) no difference in runtime between the abstracted and non-abstracted versions.)

TillE 12y ago

The "C with classes" code looks so weird. I'm a little surprised you got away with int main() not explicitly returning 0, didn't realize that was still valid C++. Anyway, C doesn't really do abstractions to any great extent, so it would make more sense to focus only on C++ and delve a little deeper. Observe what happens when you make a method virtual, for example.

jeorgun 12y ago
V-tables aren't a very expensive abstraction. They prevent inlining, which isn't exactly ideal (hell, that's the main point of the post), but nowadays their cost over a normal function call is basically negligible.
- coherentpony 12y ago
  > V-tables aren't a very expensive abstraction. They prevent inlining, which isn't exactly ideal (hell, that's the main point of the post), but nowadays their cost over a normal function call is basically negligible. This is not true. Sweeping general statements like this are almost never true. It is really problem dependent whether a particular type of abstraction is 'too expensive' or not. I have to actively avoid vtbl lookups in my domain, and this is because if I don't I lose about 15% in execute time. You might think 15% "isn't very long", but it is when your application uses several thousand MPI processes.
  - jasode 12y ago
    > I lose about 15% in execute time. Can you paste a representative snippet of your code that demonstrates this 15% penalty? In my experience with tight loops executing a million iterations, the extra indirection of a vtable lookup is never more than 1% slower. Others also observe similar minor differences.[1] [1]http://stackoverflow.com/a/667680 http://stackoverflow.com/a/667680
    - haberman 12y ago
      I have definitely observed penalties much greater than 1% (closer to 15%). This paper is old, but measures the direct cost of virtual table lookups (not taking into account indirect costs arising from the inability to inline) as 5% in real C++ programs. When they converted the C++ programs to use all virtual functions, the overhead rose to 13.7%: http://www.cs.ucsb.edu/~urs/oocsb/papers/oopsla96.pdf http://www.cs.ucsb.edu/~urs/oocsb/papers/oopsla96.pdf
      - jasode 12y ago
        I didn't make it clear but I wasn't comparing inline to virtual.[1] That type of drastic difference can be greater than 10% which is understandable. I was comparing non-inline function to virtual function and I haven't been able to recreate a 15% slowdown even with zero compiler optimization switches. The stackoverflow observation and mine were on more modern cpus. Maybe that has something to do with the conflicting anecdotes. If someone has a small snippet of code that shows non-inlined function calls running 15% faster than vtable calls, I'd like to study it. [1]the context in grandparent post was already constrained to (non-inlined) normal function calls: ", but nowadays their cost over a normal function call is basically negligible." https://news.ycombinator.com/item?id=8476208 https://news.ycombinator.com/item?id=8476208
StephanTLavavej 12y ago
C++ did that first. C++98 and C99 implicitly "return 0;" when falling off the end of main(). It was C89 that required "return 0;" here.

mackwic 12y ago

Even though the question is good, the article answer is disappointing. First, I really don't like the angle taken, then the question of abstraction (why we do it, how) and choices made by languages designers (and variations in their idioms) are so vast that you really can't treat the question through a <1000 signs blog post. Some quick points: - You don't code for the machine, you code to be read by another human (possibly you) in the future. I insist, you will be read regularly and frequently. Thus, your code needs to be clear, precise, and concise. This must be the first thing in mind when coding: program what need to be done in a way that a fellow stranger could understand. - Abstraction is a way to keep a structure of code clear when the interactions become complex and/or abundant. If you can avoid them when still being crystal clear in your code, do it. Direct speaking is always better than convolution. - The main requirements when you code are often one or two of: quick to develop, easy to maintain, extensible, efficient (you control your big-O and _WORST_ exec times), correct (no bug. at. all.), real time (when X happens, Y is done between n µs and m µs during p ns) - So, know when performance is a goal, and know when it's not. Choose your language, your techs, your team considering these goals. - And yeah, C89, C99 and the C++es have a very high overhead of abstraction: clarity, concision, and sometimes performance (all abstractions can't be inlined). Think of it.

haberman 12y ago
You are speaking in vague platitudes, most of which don't contradict what I wrote in the article at all. I was speaking of a specific context in which I was working (writing C or C++ extensions for Python and Ruby), and addressing a specific design question (how much of the library should be in C or C++ vs. Python or Ruby). I guess I didn't specifically say this, but I thought I made it pretty obvious that I'm working in a situation where performance is a factor. My observations were also specific to the question of the relative cost of creating layering abstractions in Python/Ruby vs. C/C++. > And yeah, C89, C99 and the C++es have a very high overhead of abstraction: clarity, concision Now you're just being silly. Abstractions are sometimes good, making things clearer and more concise. The wrong abstractions can make code less clear and concise. Here is my solution to that problem: if an abstraction is not making your code clearer and more concise, don't use it. Saying that abstractions in C and C++ are inherently less clear and concise just shows anti-C/C++ bias.

jerven 12y ago

Maybe, this person should understand what they are timing. The time command is a wonderful tool, but comparing the execution time of a 4 line C program with the start up cost of a VM and interpreter is inane. One needs to measure the program doing the real work. And in this case for the C program you might find that bash forking is the actual cost not the C code running through the loop. JRuby and PyPy are starting to do things with their specialisation tricks that mean that the overhead of abstraction is 1000 loops before it goes away. A performance benchmark running for less than a few minutes is a joke. Startup variance alone will dominate the results.

Chris911 12y ago
Except he's not comparing the C program and the Python one. He's comparing the time difference (or ratio) for each language with and without abstraction.
haberman 12y ago
> comparing the execution time of a 4 line C program with the start up cost of a VM and interpreter is inane. That's not what I compared. Maybe you should read the article again? > And in this case for the C program you might find that bash forking is the actual cost not the C code running through the loop. I didn't publish any benchmark numbers of any C programs, because I didn't need to: the two C++ programs I was comparing compiled to exactly the same machine code, making empirical observations of their execution time immaterial.

theflubba 12y ago

This article conflates abstraction with OO and language features, IMO. Putting some functionality into an object is not as interesting as running that functionality as a monadic command. Would be more interesting to run some benchmarks in Haskell using functional abstractions rather than OO machinery.

jaimebuelta 12y ago

The problem with this article is that is doing something extremely simple, and calling that an abstraction. The power of Python or Ruby is not demonstrated doing a for loop, is with something like this: print sum(int(n) for n in open('/a/file/with/numbers')) We don't need to care about a lot of stuff that we would in C/C++ (memory management, iterate through a file, the format of stuff, terminating the loop properly) Are C/C++ interesting languages? Of course, sometimes you need that extra level of control about what's going on. But I don't agree that the cost of abstraction is higher in Python/Ruby. You can create code that does A LOT in a few lines, that most of the times will have a good enough speed. In C/C++ you need a lot of work to create a good abstraction, that's not guaranteed to perform well.

vinkelhake 12y ago
It's not quite that bad. To sum the numbers in a file you've got a line to open the file and another to call std::accumulate with istream iterators. Memory management? Doesn't come up. Terminating the loop? What loop?
ajtulloch 12y ago
FWIW, with folly::gen [1], this is just: using namespace folly::gen; byLine('/a/file/with/numbers') | eachTo<int> | sum; https://github.com/facebook/folly/tree/master/folly/gen https://github.com/facebook/folly/tree/master/folly/gen
- seri 12y ago
  Thank you for this. Got me really interested in folly now.
- nly 12y ago
  It's only 2 lines even restricting yourself to the standard library #include <fstream> #include <iterator> #include <numeric> using namespace std; int main() { ifstream nf ("numbers.txt"); return accumulate (istream_iterator<int>(nf), istream_iterator<int>(), 0); }
12y ago

zem 12y ago

i was expecting just the opposite - the overhead in terms of programmer time and effort involved in abstracting something out versus just copy/pasting the code.

tokenrove 12y ago

It would be interesting to see the same comparison with a whole-program optimizing compiler like MLton, especially with the abstraction split across source file boundaries (an area where C compilers have gotten much stronger lately).

chrisseaton 12y ago

A while ago I wrote a blog post about how compilers for Ruby can remove abstraction. Features that you might think of as being expensive abstractions such as using #send to call methods instead of calling them directly, need not have any overhead at all with a good enough compiler http://www.chrisseaton.com/rubytruffle/pushing-pixels/ http://www.chrisseaton.com/rubytruffle/pushing-pixels/

sbov 12y ago

The overhead of abstraction is slightly more than 1.5 addition operations? Sign me up.

astrobe_ 12y ago

I think that this observation is not as relevant to the task at hand that it seems. The author is in the same position as the implementor of a VM, who has to decide which operation will be a primitive and which will not be. What is actually more important than anything else if you care about performance is which operations are likely to be called from hot loops. It's classic optimization wisdom except that you can only work on expected scenarios rather than real-world uses (until you release your stuff and other people use it). So a 100x slower abstraction is ok(-ish) as long as it's not somthing to be used in performance critical sections.

haberman 12y ago
Your comment amounts to: "this performance-related observation doesn't apply in situations where performance is not an issue." Yes, but I am working in a situation where performance is an issue. So yes, the observation is relevant to the task at hand.