3 ms·

Fair enough. My complaint is that the C code isn't given the same chance. He even calls out that reading data with getc is a known performance problem, but th

Fair enough. My complaint is that the C code isn't given the same chance. He even calls out that reading data with getc is a known performance problem, but then does it anyway. Any book on learning C will point out that getc is slow for reading lots of data, and fscanf or fread should be used instead.

alayne 14y ago
I would never encourage use of the f* I/O functions from the standard C library for high performance code. They're all buffered which means there's an extra copy happening. You should use read and other POSIX I/O functions instead.
- jacquesm 14y ago
  This is not 100% true. f* calls are buffered this is true. But depending on your workload this may be a blessing in disguise because if you're parsing a very long stream of unstructured data you'll have to re-implement most of that buffering logic in your own code to deal with the inevitable corner cases where your data overlaps the borders of the buffersize chosen. The obvious optimization here is to pre-allocate a buffer of the right size and read in the data in one go but for a stream of unknown and possibly infinite length (such as used in a filter like this) that is not possible. So f* function have their place, even in optimized code, but should be used with care. If you're reading data with fixed size blocks (such as in this example) then yes, a simple 'read' call will likely be faster, because as you note the read call will place the data directly into a user accessible buffer without the need to go through more library calls to get at the data. (fread is a wrapper function around the read system call).
gnuvince 14y ago
From what I understood from his post, using a buffered input function would a) make the code differ from the specification, b) require more refactoring than the Haskell code needed. b) seems particularly important when the program is not <100LOC, but tens or hundreds of lines of code.
- cube13 14y ago
  The problem is that his implementation is essentially a test of how he's using libc versus how Haskell is using it. The pointer arithmetic he's using shouldn't need to be optimized, since the page sizes he's malloc'ing and pulling from memory are small enough that they should stay in the L1/L2 cache for the entire run(he's using 1k blocks of data, most processors use 4k pages). There's almost no optimization to be done there. The biggest performance hit is actually the single character puts versus a block read or write. Haskell is probably implemented to read a large block of the file(or perhaps the entire file) into memory, then parse after. That would be a minimal number of system fread calls over the entire run. Versus the C code, where 1 block's parse could be 1024 getc and putc calls.
  - dllthomas 14y ago
    It's not about system calls, it's about locks. The getc function is buffered (by default) - that's what's going on behind the scenes in that FILE structure. What is slow about calling getc over and over is synchronization around that FILE object (hence the existence of functions getc_unlocked, &c).

alayne 14y ago

I would never encourage use of the f* I/O functions from the standard C library for high performance code. They're all buffered which means there's an extra copy happening. You should use read and other POSIX I/O functions instead.

jacquesm 14y ago
This is not 100% true. f* calls are buffered this is true. But depending on your workload this may be a blessing in disguise because if you're parsing a very long stream of unstructured data you'll have to re-implement most of that buffering logic in your own code to deal with the inevitable corner cases where your data overlaps the borders of the buffersize chosen. The obvious optimization here is to pre-allocate a buffer of the right size and read in the data in one go but for a stream of unknown and possibly infinite length (such as used in a filter like this) that is not possible. So f* function have their place, even in optimized code, but should be used with care. If you're reading data with fixed size blocks (such as in this example) then yes, a simple 'read' call will likely be faster, because as you note the read call will place the data directly into a user accessible buffer without the need to go through more library calls to get at the data. (fread is a wrapper function around the read system call).

gnuvince 14y ago

From what I understood from his post, using a buffered input function would a) make the code differ from the specification, b) require more refactoring than the Haskell code needed. b) seems particularly important when the program is not <100LOC, but tens or hundreds of lines of code.

cube13 14y ago
The problem is that his implementation is essentially a test of how he's using libc versus how Haskell is using it. The pointer arithmetic he's using shouldn't need to be optimized, since the page sizes he's malloc'ing and pulling from memory are small enough that they should stay in the L1/L2 cache for the entire run(he's using 1k blocks of data, most processors use 4k pages). There's almost no optimization to be done there. The biggest performance hit is actually the single character puts versus a block read or write. Haskell is probably implemented to read a large block of the file(or perhaps the entire file) into memory, then parse after. That would be a minimal number of system fread calls over the entire run. Versus the C code, where 1 block's parse could be 1024 getc and putc calls.
- dllthomas 14y ago
  It's not about system calls, it's about locks. The getc function is buffered (by default) - that's what's going on behind the scenes in that FILE structure. What is slow about calling getc over and over is synchronization around that FILE object (hence the existence of functions getc_unlocked, &c).