The problem with CSV files are the endless edge cases. What I'll typically do for larger ones is that I'll write a program that does the reading and the rudime

The problem with CSV files are the endless edge cases. What I'll typically do for larger ones is that I'll write a program that does the reading and the rudimentary processing in C, and then saves the data as a binary .npy file in /dev/shm (or ImDisk if I'm on windows) and load that up in Python. Massage the data more if necessary and finally do the plotting. I found that this helps, hope it helps you too