12 ms·

I was suprised at the C++ outcome, so I started testing myself. I was very amazed to find a 4~5x speedup when reading directly from file (std::ifstream) instead

I was suprised at the C++ outcome, so I started testing myself. I was very amazed to find a 4~5x speedup when reading directly from file (std::ifstream) instead of reading stdin. This was on a 10.5.7 MBP 2014 [edit] more like 9~10x actually... time { ./build/Release/countwords_ifstream; } | tail ... real 0m1.337s user 0m1.332s sys 0m0.022s time { ./build/Release/countwords_original < kjvbible_x10.txt; } | tail ... real 0m12.184s user 0m12.138s sys 0m0.044s changes between these two: 6a7 > #include <fstream> 8a10 > std::ifstream inp( "/Users/macgyverismo/Desktop/test/kjvbible_x10.txt" ); 13c15 < while (std::cin >> word) { --- > while (inp >> word) {

nerdponx 6y ago
Is that due to something at the OS or filesystem driver level?
c-cube 6y ago
Could be a lock on stdin that is acquired in a loop? Rust has such a lock and typically one should lock stdin at the beginning and read from within the critical section.
FAANG_dream 6y ago
Try this maybe? https://codeforces.com/blog/entry/6251 https://codeforces.com/blog/entry/6251
- macgyverismo 6y ago
  Asuming you mean 'std::ios::sync_with_stdio(false);', that was already present in the original code.
usefulcat 6y ago
I tried that on Linux and got at best a 5% improvement.
- benhoyt 6y ago
  Yeah, I'm on Linux too and tried that with no noticeable difference in results. There's something off about macOS's approach.
cornstalks 6y ago
I dug into this once because reading from stdin on my Mac was so slow, but only when using std::cin. Using C APIs to read stdin was fast. Turns out libc++ on mac uses `getc` to read from stdin one character at a time. Madness.
- darwingr 6y ago
  Huh, well that makes a lot of sense. Three years ago when I was in University practicing for competitive coding we came across a limit in the performance (ms) we could get on even simple problems. The professor leading the practice poked away at his own solution to some problem for days because he couldn't get a better time than the top score online. Eventually he implemented his own versions of cin and cout. Needless to say he was pretty frustrated but there's just some problems that are hard to let go of without a solution.
w-m 6y ago
Similar here, reading the file from stdin on a 2020 MBP on macOS 11.2.2 is incredibly slow. int main() { std::ios::sync_with_stdio(false); std::string str((std::istreambuf_iterator<char>(std::cin)), std::istreambuf_iterator<char>()); } takes around 8 seconds to run, no counting at all. int main() { std::ifstream f("kjvbible_x10.txt"); std::string str((std::istreambuf_iterator<char>(f)), std::istreambuf_iterator<char>()); } runs in ~270 miliseconds. Factor 30x, yikes.

nerdponx 6y ago

Is that due to something at the OS or filesystem driver level?

c-cube 6y ago

Could be a lock on stdin that is acquired in a loop? Rust has such a lock and typically one should lock stdin at the beginning and read from within the critical section.

FAANG_dream 6y ago

Try this maybe? https://codeforces.com/blog/entry/6251 https://codeforces.com/blog/entry/6251

macgyverismo 6y ago
Asuming you mean 'std::ios::sync_with_stdio(false);', that was already present in the original code.

usefulcat 6y ago

I tried that on Linux and got at best a 5% improvement.

benhoyt 6y ago
Yeah, I'm on Linux too and tried that with no noticeable difference in results. There's something off about macOS's approach.

cornstalks 6y ago

I dug into this once because reading from stdin on my Mac was so slow, but only when using std::cin. Using C APIs to read stdin was fast. Turns out libc++ on mac uses `getc` to read from stdin one character at a time. Madness.

darwingr 6y ago
Huh, well that makes a lot of sense. Three years ago when I was in University practicing for competitive coding we came across a limit in the performance (ms) we could get on even simple problems. The professor leading the practice poked away at his own solution to some problem for days because he couldn't get a better time than the top score online. Eventually he implemented his own versions of cin and cout. Needless to say he was pretty frustrated but there's just some problems that are hard to let go of without a solution.

w-m 6y ago

Similar here, reading the file from stdin on a 2020 MBP on macOS 11.2.2 is incredibly slow. int main() { std::ios::sync_with_stdio(false); std::string str((std::istreambuf_iterator<char>(std::cin)), std::istreambuf_iterator<char>()); } takes around 8 seconds to run, no counting at all. int main() { std::ifstream f("kjvbible_x10.txt"); std::string str((std::istreambuf_iterator<char>(f)), std::istreambuf_iterator<char>()); } runs in ~270 miliseconds. Factor 30x, yikes.