Unfortunately my data isn't all numbers. It has text too. The sparse examples in that link only show this for reading in numbers. Do you know off hand if it tra

Unfortunately my data isn't all numbers. It has text too. The sparse examples in that link only show this for reading in numbers. Do you know off hand if it translates well? There is a dtype parameter, but it'll take me a few days to get back to this code, so I figured I'd check beforehand. Your second paragraph is essentially what I want. I'm willing to wait a day for code that may run in 1 hour from memory, so time isn't entirely an issue unless it's starting to bleed into weeks. The read_csv function in pandas has a parameter called memory_map, but when I tried using it on a smaller 7GB dataset, it read the whole thing into memory (32GB instance) even when I set it to True. SQLite is definitely not my best option here. It was the only server-less implementation I could find, so I tried to use it and it didn't work. However, a database like implementation will be helpful because each operation I need to do will require data that satisfies certain timestamp and arithmetic conditions. I figured it'd be best to load the whole thing into a db and query it for every operation to train my model.