I agree that the database will be the bottleneck, and would suggest: (1) Look at bunching your insertions into appropriately sized transactions. You don't want

I agree that the database will be the bottleneck, and would suggest: (1) Look at bunching your insertions into appropriately sized transactions. You don't want transactions to be too small (you spend most of your time waiting for tiny writes) or too big (buffering gets inefficient and you spill over to temp files) (2) Tweak your database config. Can you turn off "safety" features like fsyncing post-transaction? Can you configure the db server to accept only a few connections, each with loads of working memory? (3) Writing out SQL to disk then having a database client read it back in is wasted time. The fastest solution would probably be to have your parsing software talk direct to the database. Piping generated SQL straight into the standard database client will get you most of that speed with zero effort. I'm a postgresql guy myself. Some databases already have widely used bulk loader solutions, and all of the transaction/config stuff may not apply to you as much as it does to me.