So you hang onto your allocated strings in a record array and wonder why you run out of memory? You should be saving the strings to the database so the GC has

So you hang onto your allocated strings in a record array and wonder why you run out of memory? You should be saving the strings to the database so the GC has something to reclaim. It doesn't work when you keep fields 5, 7, 8 around in your records array. Bulk insert should probably be used and bulk write should be called periodically. You should pause processing while it saves so you keep total active allocations nice and compact. Otherwise the process will slow to a crawl when you get near your memory limit and all the time is spent in the GC trying to find ways to reclaim memory that you are still hanging onto. You see massive swings in processing time by chunking the data because you are handing memory very poorly. This should be bounded by database writes, not your language. Sigh.