Though the results may have changed since, I once had to parse a very, very large file (about 5 GB) line-by-line. I tested a number of different methods, and ul

Though the results may have changed since, I once had to parse a very, very large file (about 5 GB) line-by-line. I tested a number of different methods, and ultimately discovered that reading the entire file using .readlines() was faster by a significant margin. I vaguely recall it was more than four times faster, actually - and that was a really big win when you're talking about a data set that large. Granted, our machine had a perhaps-unusual design in that it had multiple RAID 6 data banks, a very powerful dedicated IO controller, and about 128 GB of RAM. So your mileage may vary. Unfortunately I don't have access to that machine any more so I can't perform the same benchmark, but I have a nagging feeling that if I ran the tests again with Python 3.2 I'd get different results.