9 ms·

Except that there are fine high-level matrix libraries, optimization libraries, etc. for C/C++. I needed an parameter estimator for maximum entropy models that

Except that there are fine high-level matrix libraries, optimization libraries, etc. for C/C++. I needed an parameter estimator for maximum entropy models that was efficient for training rankers (e.g. for parse disambiguation, fluency ranking, etc.), that also had good support for feature selection. I just used an off-the-shelf optimizer (liblbfgs), implemented calculation of the objective and gradients, plus various feature selection methods. Within no-time, we had an extremely fast estimator, that could also help answering my research question (what are the most discriminative features in our models). Let me push this point somewhat further: C is a very simple language. It's something that someone with a certain level of intelligence can fairly quickly pick up. There is also a wealth of excellent libraries available. The problem with 'use high-level, the compiler/interpreter will take care of it' is that those languages are either a lot more complex (Haskell, OCaml) or a lot slower (Python, Perl, Ruby). In the latter case, you will end up implementing modules in C anyway.

squidfood 15y ago
Yes, it is in fact so "simple" that you have to bang it on the head and shout at it to get it to actually do anything. For the audience of this article, a good test is to compare a copy of Numerical Recipes in c versus the fortran edition, where the whole purpose of the book is to present numerical code to scientists. c (and c++) have to spend ages developing up notation and workarounds for the simplest matrix to correct c's deficiencies; the pointer manipulations (array pointers, function pointers, pointer pointers) make the code nigh-unreadable compared to the straightforward and understandable fortran. Edit: I agree with your point with respect to interpreted languages, especially those with weak typing; I don't need to control where my number lives in memory, but I do need to control the bits of precision from the beginning.
- danieldk 15y ago
  Your definition of unreadable must then differ from mine: http://eigen.tuxfamily.org/dox/QuickRefPage.html http://eigen.tuxfamily.org/dox/QuickRefPage.html
  - archgoon 15y ago
    And working code breaks when moving to new computers or upgrading the operating system, meaning you have to dive into the template library to figure out how the array alignment is working, and how Mac OS X function calls affect the stack. Been there. Not pleasant.
    - danieldk 15y ago
      Well, arguably, Apple upgraded Xcode, sorry your favorite interpreter/compiler broke, is even less fun.
- derleth 15y ago
  > interpreted languages, especially those with weak typing Dynamic and weak are not the same thing. Weak typing means you're allowed to break abstractions; C has weak typing, C++ is marginally less weak, and Python has strong typing for its built-in types.
MichaelSalib 15y ago
C is a very simple language. It is? Out of curiosity, did you know about all the undefined behaviors described at http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-... and did you understand how your C compiler takes advantage of them before you wrote that statement?
- danieldk 15y ago
  It is? Out of curiosity, did you know about all the undefined behaviors described at Yes, those are familiar and described in every decent C introduction. Of course, that doesn't mean people do not make those mistakes. Let the compiler be pedantic. did you understand how your C compiler takes advantage of them before you wrote that statement? At the very least, is fairly easy to understand how the compiler optimizes correct code. In say, Haskell (which I do like a lot), you often have to resort to reading ghc's Cmm output to see why something is not optimized.
- rcfox 15y ago
  Yes, that's what makes C simple. If all of those edge cases were covered, the language would be much more complex. Simple != Easy to use
  - MichaelSalib 15y ago
    So when people say that "C is simple", you think they really mean "C is simple to implement" or maybe "C is simple to specify"? I don't think the grandparent meant either of those things and I don't think either of them are true anyway.
    - danieldk 15y ago
      Simple in the sense that it is a small language. There are not a lot of constructs, abstractions, etc. C lets you allocate blocks of memory, and perform operations on those blocks of memory. That's mostly it. In my experience, this very much maps to the problem domain (number crunching). If you want to do performant number crunching with most other languages, you have to not only grasp the language, but also the underlying virtual machine or compiler. The more layers you pile on top, the harder it becomes to identify bottlenecks. Sure, you can use fast implementations of common algorithms in a high-level language. But they are often written in C or C++. So, if you want to modify or extend such algorithms (which is likely, or you could just use an off-the-shelf program), you will end up in C-land anyway.
yummyfajitas 15y ago
Except that there are fine high-level matrix libraries, optimization libraries, etc. for C/C++. You'd be surprised how much of it is actually fortran at the bottom. Lapack and FFTpack are two of the more important examples.
- jbooth 15y ago
  True, but the reasons for that are historical, not performance-based. Lapack depends on a BLAS implementation for the primitive matrix and vector operations (the actual number crunching), and while the reference impl is in Fortran, the high-performance ATLAS implementation is written in C, and I believe Intel MKL is as well. These implementations parallelize the matrix and vector ops, leaving the single-threaded fortran code in more of a "controller" role. There's also work being done to parallelize the LAPACK operations at a higher level than delegating to parallel BLAS, it's shown promise but isn't done yet, see the PLASMA package released by some oak ridge national labs people.
  - ArbitraryLimits 15y ago
    Until C changes so that the compiler can tell that a function's pointer arguments are actually arrays (a drastic change, so it will never happen), Fortran will always have a place in numeric work..
    - jbooth 15y ago
      I'm not enough of a compiler buff to be able to say for sure, but I think for problems like BLAS or LAPACK where you have a very well specified problem and the motivation to dedicate absurd amounts of time to micro-optimization, C's got an edge there. Now, if you're writing from scratch and if the Fortran compiler is capable of the right optimizations, then yeah, it's probably more worth it to use fortran. Especially if you can code directly to your problem domain. But I'm pretty sure ATLAS and MKL beat the standard compiled fortran, otherwise they wouldn't exist. In short, just because the C compiler can't parallelize your code for you doesn't mean that you can't do that yourself. It's just a lot of work.

squidfood 15y ago

Yes, it is in fact so "simple" that you have to bang it on the head and shout at it to get it to actually do anything. For the audience of this article, a good test is to compare a copy of Numerical Recipes in c versus the fortran edition, where the whole purpose of the book is to present numerical code to scientists. c (and c++) have to spend ages developing up notation and workarounds for the simplest matrix to correct c's deficiencies; the pointer manipulations (array pointers, function pointers, pointer pointers) make the code nigh-unreadable compared to the straightforward and understandable fortran. Edit: I agree with your point with respect to interpreted languages, especially those with weak typing; I don't need to control where my number lives in memory, but I do need to control the bits of precision from the beginning.

danieldk 15y ago
Your definition of unreadable must then differ from mine: http://eigen.tuxfamily.org/dox/QuickRefPage.html http://eigen.tuxfamily.org/dox/QuickRefPage.html
- archgoon 15y ago
  And working code breaks when moving to new computers or upgrading the operating system, meaning you have to dive into the template library to figure out how the array alignment is working, and how Mac OS X function calls affect the stack. Been there. Not pleasant.
  - danieldk 15y ago
    Well, arguably, Apple upgraded Xcode, sorry your favorite interpreter/compiler broke, is even less fun.
derleth 15y ago
> interpreted languages, especially those with weak typing Dynamic and weak are not the same thing. Weak typing means you're allowed to break abstractions; C has weak typing, C++ is marginally less weak, and Python has strong typing for its built-in types.

MichaelSalib 15y ago

C is a very simple language. It is? Out of curiosity, did you know about all the undefined behaviors described at http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-... and did you understand how your C compiler takes advantage of them before you wrote that statement?

danieldk 15y ago
It is? Out of curiosity, did you know about all the undefined behaviors described at Yes, those are familiar and described in every decent C introduction. Of course, that doesn't mean people do not make those mistakes. Let the compiler be pedantic. did you understand how your C compiler takes advantage of them before you wrote that statement? At the very least, is fairly easy to understand how the compiler optimizes correct code. In say, Haskell (which I do like a lot), you often have to resort to reading ghc's Cmm output to see why something is not optimized.
rcfox 15y ago
Yes, that's what makes C simple. If all of those edge cases were covered, the language would be much more complex. Simple != Easy to use
- MichaelSalib 15y ago
  So when people say that "C is simple", you think they really mean "C is simple to implement" or maybe "C is simple to specify"? I don't think the grandparent meant either of those things and I don't think either of them are true anyway.
  - danieldk 15y ago
    Simple in the sense that it is a small language. There are not a lot of constructs, abstractions, etc. C lets you allocate blocks of memory, and perform operations on those blocks of memory. That's mostly it. In my experience, this very much maps to the problem domain (number crunching). If you want to do performant number crunching with most other languages, you have to not only grasp the language, but also the underlying virtual machine or compiler. The more layers you pile on top, the harder it becomes to identify bottlenecks. Sure, you can use fast implementations of common algorithms in a high-level language. But they are often written in C or C++. So, if you want to modify or extend such algorithms (which is likely, or you could just use an off-the-shelf program), you will end up in C-land anyway.

yummyfajitas 15y ago

Except that there are fine high-level matrix libraries, optimization libraries, etc. for C/C++. You'd be surprised how much of it is actually fortran at the bottom. Lapack and FFTpack are two of the more important examples.

jbooth 15y ago
True, but the reasons for that are historical, not performance-based. Lapack depends on a BLAS implementation for the primitive matrix and vector operations (the actual number crunching), and while the reference impl is in Fortran, the high-performance ATLAS implementation is written in C, and I believe Intel MKL is as well. These implementations parallelize the matrix and vector ops, leaving the single-threaded fortran code in more of a "controller" role. There's also work being done to parallelize the LAPACK operations at a higher level than delegating to parallel BLAS, it's shown promise but isn't done yet, see the PLASMA package released by some oak ridge national labs people.
- ArbitraryLimits 15y ago
  Until C changes so that the compiler can tell that a function's pointer arguments are actually arrays (a drastic change, so it will never happen), Fortran will always have a place in numeric work..
  - jbooth 15y ago
    I'm not enough of a compiler buff to be able to say for sure, but I think for problems like BLAS or LAPACK where you have a very well specified problem and the motivation to dedicate absurd amounts of time to micro-optimization, C's got an edge there. Now, if you're writing from scratch and if the Fortran compiler is capable of the right optimizations, then yeah, it's probably more worth it to use fortran. Especially if you can code directly to your problem domain. But I'm pretty sure ATLAS and MKL beat the standard compiled fortran, otherwise they wouldn't exist. In short, just because the C compiler can't parallelize your code for you doesn't mean that you can't do that yourself. It's just a lot of work.