4 ms·
When your data doesn't fit in memory: SQLite
by ak39 7y ago
When your data doesn't fit in memory: SQLite
- lytefm 7y agoYeah for me this was also the obvious solution when I had to extract information from ~80 GB of tabular data. It's not the most trendy tool, but it did the job extremely well and with little effort. Just be sure to only create an index after having inserted all the data...
- ak39 7y agoCorrect. Drop indexes for inserts and recreate after bulk inserts. Also, another hint: process all inserts in the bulk in a single transaction. I was amazed at the speed that SQLite ingests data! Speed demon! (Don’t forget PRAGMA synchronous OFF too)