6 ms·

> Interestingly, C/C++ beats Fortran on all but two of the benchmarks, although they are fairly close on most. Unless I missed something, this directly contrad

> Interestingly, C/C++ beats Fortran on all but two of the benchmarks, although they are fairly close on most. Unless I missed something, this directly contradicts the quotes benchmark source[1], where C beat Fortran on all tests except one (where they were tied), and usually by a wide margin. Plus I believe gcc usually produces slower code than Intel's compilers (at least for C), so it's not clear that C-gcc and Fortran-Intel is a fair comparison. But beyond that, I'm always confused as to why there is a significant difference. I would expect two compiled languages with reasonable compilers to perform almost exactly the same when they're doing nothing but arithmetic operations, since those are almost entirely determined by the CPU, not the cleverness or speed of the standard library algorithms. Can anyone enlighten me? [1] https://benchmarksgame.alioth.debian.org/u64q/fortran.html https://benchmarksgame.alioth.debian.org/u64q/fortran.html

santaclaus 10y ago
> I would expect two compiled languages with reasonable compilers to perform almost exactly the same when they're doing nothing but arithmetic operations Auto-vectorization? These implementations vary wildly between GCC and Intel.
jstewartmobile 10y ago
Lies, damn lies, and benchmarks. I know as a long-time C programmer, there are small changes I can make to source that will result in big-time performance gains. Could hinge on how tight their FORTRAN code is vs their C code... Could also depend on how much love their respective machine code translations have been getting.
- dfgasfjhjhsdojj 10y ago
  Fortran hasn't been all caps for nearly 30 years now... It is very easy for a physical scientist to write Fortran that performs well - the syntax looks very similar to the maths that is being modeled. It is a lot harder for a scientist to write tight C code.
  - jstewartmobile 10y ago
    Not bashing Fortran here, I'm defending it.
- igouy 10y ago
  > small changes … result in big-time performance gains. Look and you might see measurements of several different C or C++ or … programs doing the same task with different performance. https://benchmarksgame.alioth.debian.org/u64q/performance.php?test=nbody https://benchmarksgame.alioth.debian.org/u64q/performance.ph...
throwaway7645 10y ago
C can be faster than Fortan if you optimize it so. Fortran is built to run at very close to the max C speed for these things out of the box (as I understand it after doing some extensive googling a while back).
amenghra 10y ago
> I would expect two compiled languages [...] to perform almost exactly the same when they're doing nothing but arithmetic operations [...] Different languages are going to have different memory models which is going to result is vastly different performance. E.g. a GC vs reference counting vs linear types. If you are doing arithmetic on something that needs to be managed (e.g. an array), you'll see differences. Some language features close the door for optimizations. E.g. the ability to dynamically add or remove methods to an object/class can make method dispatch more or less expensive. Finally, some languages have undefined behavior (e.g. what happens when there's an overflow or a division by zero) while other languages want specific behavior for the edge cases. This all affects the performance of the resulting code, even for basic arithmetic operations.
- supergarfield 10y ago
  These are all good points. In this case though, C doesn't have garbage collection or OO features, and (I believe) has unspecified behavior for arithmetic exceptions. I assume Fortran does the same (since it's comparably fast), so I guess I still don't really see where people expect the difference to come from.
  - greglindahl 10y ago
    C and Fortran both obey IEEE 754 and have well-defined arithmetic exceptions on most platforms -- but programmers usually instruct the compilers to ignore all that in the interests of faster code. As for what the differences are, try talking to a compiler engineer. The better ones love how Fortran features like the argument aliasing rule for 100% of code makes generating fast code easier. In fact that's one of my tests for compiler engineers in interviews; if they've worked on a C/C++/Fortran compiler and can't say anything good about the challenges and opportunities of Fortran optimization relative to C/C++, then they haven't really drilled down on performance issues!
CJefferson 10y ago
One big difference is the rules on pointers. In C, if you write code like this: void dostufftotwoarrays(int* a, int* b); Then maybe b is equal to a + 3, so your compiled code has to assume whenever it writes to an element of a, it might have changed an element of b, and so reload b from memory. Alternatively, it can do a bunch of tests at the start of the function, and compile multiple copies of the function for different situations. In FORTRAN, this isn't possible (you don't tend to pass around raw pointers), so it is much easier for the compiler to reason about what objects might be in the same place, and which aren't. The restrict keyword goes some way to fixing this in C, but is tricky yo use correctly, and also doesn't solve all problems.
- bmh100 10y ago
  What makes it tricky to use, and where does it fall short?
  - CJefferson 10y ago
    There are some weird rules, like: mypair* restrict position = &object->pair; float* restrict position_x = &position->x; Is undefined behaviour, as you can't take a restricted pointer to a child of a restricted pointer in the same scope (the rule isn't quite that simple, but that's part of it). In general, rules about pointers to pointers get complicated, and I've had trouble with them in the past. In general, it's a good idea to try to only use restrict for "leaf" pointers, not intermediate objects, but that does block some optimisations. There is also the obvious problem of people getting carried away, and marking something 'restrict' where they shouldn't have. While 'restrict' is very easy to use on simple structures (like just int*), if you look around you'll find very little, if any, material about using it for complicated nested data structures.
    - Someone 10y ago
      There's also the fact that restrict is only a promise to the compiler "this function will not be called with aliased arguments", not also a command "do not allow people to call this function with aliased arguments". As far as I know Fortran, at least in its original form, enforced the latter (that was fairly easy to do, as it didn't have dynamic memory allocation, pointers or recursion (the compiler determined the location in memory of all variables))
bmh100 10y ago
There are some important things to note about these benchmarks. On the Mandelbrot set, I noticed that the program does not use arrays despite the obvious 2D-array structure of the bitmap. Two of the four cores were also underutilized. Next, the k-nucleotide, reverse-complement, and fasta programs only utilized a single core, compared to the C programs utilizing multiple cores. It is not clear to me why this should be the case, but it leads me to cast doubt on the tests I listed.
- igouy 10y ago
  Simple -- those are the programs that have been contributed. https://benchmarksgame.alioth.debian.org/play.html https://benchmarksgame.alioth.debian.org/play.html
  - emmelaich 10y ago
    Isaac, thank you for your (almost :-) indefatigable work for and defence of the benchmarks game.
    - igouy 10y ago
      You're welcome. I have cut back: x64 single core & x86 quad core are no longer measured, x86 single core updates are not frequent, and x64 quad core updates are less frequent than they used to be (although more frequent than the chart dates might suggest). I've kind-of been hoping the hardware would fail -- but someone would probably gift me a new machine :-) From my perspective "defence of the benchmarks game" is a matter of saying "Keep it in perspective" and "Keep it in context" until we do ;-)
  - bmh100 10y ago
    > You will probably come across people saying that the programs are not idiomatic ("enough"). This seems to describe me. I am tempted to rewrite those programs more idiomatically. Thank you for pointing this out.
    - igouy 10y ago
guitarbill 10y ago
Like you observe, the compilers matter. Historically, C compilers just couldn't do such a good job at optimising, because C is pretty complex and has loads of undefined behaviour. Also, Fortran compilers have always been optimised for numerical computation. Recently * , a lot of effort has gone into making C compilers produce very optimised code. In general, this is a good idea because humans are bad at optimising and also remembering every damned Intel vectorisation instruction. So historically, Fortran was much faster, recently * with compiler advances in e.g. reordering and AMD64, it's possible C compilers beat Fortran compilers. * edit: adjust your definition of recently to encompass the entire timeframe of Fortran's existance
- flamedoge 10y ago
  Compilers are afraid to optimize around possible UB, even though they can.
  - greglindahl 10y ago
    Having worked with key people from the SGI compiler team, and then competed with Intel / PGI / Sun / gcc on x86, I totally disagree.
    - flamedoge 10y ago
      I guess I should've looked when I said "Compilers". I only skimmed two and one of them is LLVM.
    - flamedoge 10y ago
      I totally see UB as a blessing in disguise for optimizer.
      - greglindahl 10y ago
        Usually, yeah. The art is making it debuggable, when the user accidentally strays into UB and needs help not doing that.

santaclaus 10y ago

> I would expect two compiled languages with reasonable compilers to perform almost exactly the same when they're doing nothing but arithmetic operations Auto-vectorization? These implementations vary wildly between GCC and Intel.

jstewartmobile 10y ago

Lies, damn lies, and benchmarks. I know as a long-time C programmer, there are small changes I can make to source that will result in big-time performance gains. Could hinge on how tight their FORTRAN code is vs their C code... Could also depend on how much love their respective machine code translations have been getting.

dfgasfjhjhsdojj 10y ago
Fortran hasn't been all caps for nearly 30 years now... It is very easy for a physical scientist to write Fortran that performs well - the syntax looks very similar to the maths that is being modeled. It is a lot harder for a scientist to write tight C code.
- jstewartmobile 10y ago
  Not bashing Fortran here, I'm defending it.
igouy 10y ago
> small changes … result in big-time performance gains. Look and you might see measurements of several different C or C++ or … programs doing the same task with different performance. https://benchmarksgame.alioth.debian.org/u64q/performance.php?test=nbody https://benchmarksgame.alioth.debian.org/u64q/performance.ph...

throwaway7645 10y ago

C can be faster than Fortan if you optimize it so. Fortran is built to run at very close to the max C speed for these things out of the box (as I understand it after doing some extensive googling a while back).

amenghra 10y ago

> I would expect two compiled languages [...] to perform almost exactly the same when they're doing nothing but arithmetic operations [...] Different languages are going to have different memory models which is going to result is vastly different performance. E.g. a GC vs reference counting vs linear types. If you are doing arithmetic on something that needs to be managed (e.g. an array), you'll see differences. Some language features close the door for optimizations. E.g. the ability to dynamically add or remove methods to an object/class can make method dispatch more or less expensive. Finally, some languages have undefined behavior (e.g. what happens when there's an overflow or a division by zero) while other languages want specific behavior for the edge cases. This all affects the performance of the resulting code, even for basic arithmetic operations.

supergarfield 10y ago
These are all good points. In this case though, C doesn't have garbage collection or OO features, and (I believe) has unspecified behavior for arithmetic exceptions. I assume Fortran does the same (since it's comparably fast), so I guess I still don't really see where people expect the difference to come from.
- greglindahl 10y ago
  C and Fortran both obey IEEE 754 and have well-defined arithmetic exceptions on most platforms -- but programmers usually instruct the compilers to ignore all that in the interests of faster code. As for what the differences are, try talking to a compiler engineer. The better ones love how Fortran features like the argument aliasing rule for 100% of code makes generating fast code easier. In fact that's one of my tests for compiler engineers in interviews; if they've worked on a C/C++/Fortran compiler and can't say anything good about the challenges and opportunities of Fortran optimization relative to C/C++, then they haven't really drilled down on performance issues!

CJefferson 10y ago

One big difference is the rules on pointers. In C, if you write code like this: void dostufftotwoarrays(int* a, int* b); Then maybe b is equal to a + 3, so your compiled code has to assume whenever it writes to an element of a, it might have changed an element of b, and so reload b from memory. Alternatively, it can do a bunch of tests at the start of the function, and compile multiple copies of the function for different situations. In FORTRAN, this isn't possible (you don't tend to pass around raw pointers), so it is much easier for the compiler to reason about what objects might be in the same place, and which aren't. The restrict keyword goes some way to fixing this in C, but is tricky yo use correctly, and also doesn't solve all problems.

bmh100 10y ago
What makes it tricky to use, and where does it fall short?
- CJefferson 10y ago
  There are some weird rules, like: mypair* restrict position = &object->pair; float* restrict position_x = &position->x; Is undefined behaviour, as you can't take a restricted pointer to a child of a restricted pointer in the same scope (the rule isn't quite that simple, but that's part of it). In general, rules about pointers to pointers get complicated, and I've had trouble with them in the past. In general, it's a good idea to try to only use restrict for "leaf" pointers, not intermediate objects, but that does block some optimisations. There is also the obvious problem of people getting carried away, and marking something 'restrict' where they shouldn't have. While 'restrict' is very easy to use on simple structures (like just int*), if you look around you'll find very little, if any, material about using it for complicated nested data structures.
  - Someone 10y ago
    There's also the fact that restrict is only a promise to the compiler "this function will not be called with aliased arguments", not also a command "do not allow people to call this function with aliased arguments". As far as I know Fortran, at least in its original form, enforced the latter (that was fairly easy to do, as it didn't have dynamic memory allocation, pointers or recursion (the compiler determined the location in memory of all variables))

bmh100 10y ago

There are some important things to note about these benchmarks. On the Mandelbrot set, I noticed that the program does not use arrays despite the obvious 2D-array structure of the bitmap. Two of the four cores were also underutilized. Next, the k-nucleotide, reverse-complement, and fasta programs only utilized a single core, compared to the C programs utilizing multiple cores. It is not clear to me why this should be the case, but it leads me to cast doubt on the tests I listed.

igouy 10y ago
Simple -- those are the programs that have been contributed. https://benchmarksgame.alioth.debian.org/play.html https://benchmarksgame.alioth.debian.org/play.html
- emmelaich 10y ago
  Isaac, thank you for your (almost :-) indefatigable work for and defence of the benchmarks game.
  - igouy 10y ago
    You're welcome. I have cut back: x64 single core & x86 quad core are no longer measured, x86 single core updates are not frequent, and x64 quad core updates are less frequent than they used to be (although more frequent than the chart dates might suggest). I've kind-of been hoping the hardware would fail -- but someone would probably gift me a new machine :-) From my perspective "defence of the benchmarks game" is a matter of saying "Keep it in perspective" and "Keep it in context" until we do ;-)
- bmh100 10y ago
  > You will probably come across people saying that the programs are not idiomatic ("enough"). This seems to describe me. I am tempted to rewrite those programs more idiomatically. Thank you for pointing this out.
  - igouy 10y ago

guitarbill 10y ago

Like you observe, the compilers matter. Historically, C compilers just couldn't do such a good job at optimising, because C is pretty complex and has loads of undefined behaviour. Also, Fortran compilers have always been optimised for numerical computation. Recently * , a lot of effort has gone into making C compilers produce very optimised code. In general, this is a good idea because humans are bad at optimising and also remembering every damned Intel vectorisation instruction. So historically, Fortran was much faster, recently * with compiler advances in e.g. reordering and AMD64, it's possible C compilers beat Fortran compilers. * edit: adjust your definition of recently to encompass the entire timeframe of Fortran's existance

flamedoge 10y ago
Compilers are afraid to optimize around possible UB, even though they can.
- greglindahl 10y ago
  Having worked with key people from the SGI compiler team, and then competed with Intel / PGI / Sun / gcc on x86, I totally disagree.
  - flamedoge 10y ago
    I guess I should've looked when I said "Compilers". I only skimmed two and one of them is LLVM.
  - flamedoge 10y ago
    I totally see UB as a blessing in disguise for optimizer.
    - greglindahl 10y ago
      Usually, yeah. The art is making it debuggable, when the user accidentally strays into UB and needs help not doing that.