6 ms·

> I'm convinced most programs easily waste 90% of their CPU time. Hah, in my experience that's hopelessly optimistic. The C I write is probably 10-100x slower

> I'm convinced most programs easily waste 90% of their CPU time. Hah, in my experience that's hopelessly optimistic. The C I write is probably 10-100x slower than it should be (leaving SSE and threading aside entirely), but the Python I write most days is a hundred times slower still.

Radim 10y ago
One data point to support your claim: * pure Python: 1.0x (baseline) * NumPy (numeric python): 120x * Google-optimized pure C: 2,500x * optimized Python + Cython + BLAS: 8,700x This is optimizing the same numeric algorithm (word2vec) using different languages and tricks, from naive pure-Python down to compiled Cython with optimized CPU-specific assembly/Fortran (BLAS) for the hotspots [0]. [0] http://rare-technologies.com/word2vec-in-python-part-two-optimizing/ http://rare-technologies.com/word2vec-in-python-part-two-opt...
- flamedoge 10y ago
  and I'm sure pure Fortran would be even faster
  - eru 10y ago
    That's mostly because of Fortran doesn't have the same aliasing trouble as C?
    - flamedoge 10y ago
      Pointer indirection is expensive tradeoff for flexibility
  - deleted 10y ago
    [deleted]
  - alfalfasprout 10y ago
    Eh, doubtful. Sure, it'll be faster than unoptimized C. But when you observe strict aliasing rules in C a modern compiler will generally produce similar quality assembly for both languages. The reason BLAS is so fast is that every single line of code in a hot loop has been hand-optimized and tons of CPU-specific optimizations exist Fortran yields some of the fastest code out of the box, yes, but C/C++ can yield equivalent performance by adhering to certain basic rules.
    - flamedoge 10y ago
      Even with strict aliasing, the only way C/C++ standard effectively lets one type to be cast to another is incurring a copy. Tell me what can be done better for performance in C that can't be done in Fortran or "another static language without pointer indirection".
jeremiep 10y ago
You can get a lot of performance back just by taking branch predictions and cache misses into your designs. I did just that in C# last years and got at least two orders of magnitude of performance gain from it. Still, I constantly wished I was working in C instead :)

Radim 10y ago

One data point to support your claim: * pure Python: 1.0x (baseline) * NumPy (numeric python): 120x * Google-optimized pure C: 2,500x * optimized Python + Cython + BLAS: 8,700x This is optimizing the same numeric algorithm (word2vec) using different languages and tricks, from naive pure-Python down to compiled Cython with optimized CPU-specific assembly/Fortran (BLAS) for the hotspots [0]. [0] http://rare-technologies.com/word2vec-in-python-part-two-optimizing/ http://rare-technologies.com/word2vec-in-python-part-two-opt...

flamedoge 10y ago
and I'm sure pure Fortran would be even faster
- eru 10y ago
  That's mostly because of Fortran doesn't have the same aliasing trouble as C?
  - flamedoge 10y ago
    Pointer indirection is expensive tradeoff for flexibility
- deleted 10y ago
  [deleted]
- alfalfasprout 10y ago
  Eh, doubtful. Sure, it'll be faster than unoptimized C. But when you observe strict aliasing rules in C a modern compiler will generally produce similar quality assembly for both languages. The reason BLAS is so fast is that every single line of code in a hot loop has been hand-optimized and tons of CPU-specific optimizations exist Fortran yields some of the fastest code out of the box, yes, but C/C++ can yield equivalent performance by adhering to certain basic rules.
  - flamedoge 10y ago
    Even with strict aliasing, the only way C/C++ standard effectively lets one type to be cast to another is incurring a copy. Tell me what can be done better for performance in C that can't be done in Fortran or "another static language without pointer indirection".

jeremiep 10y ago

You can get a lot of performance back just by taking branch predictions and cache misses into your designs. I did just that in C# last years and got at least two orders of magnitude of performance gain from it. Still, I constantly wished I was working in C instead :)