If you can do what ever you want to do by analyzing one line at a time I would recommend reading the file as a stream for processing. If you can't do whatever

If you can do what ever you want to do by analyzing one line at a time I would recommend reading the file as a stream for processing. If you can't do whatever you'd like one line at a time, read the file as a stream and write it to disk with some database(e.g sqlite, MySQL, Mongo). This will be slower but might be easier to play with the data.