I had a similar problem with a gawk (yes, gawk!) program I was writing. I had to accumulate 10,000,000 32-character strings to produce a 320,000,000 (three hund

I had a similar problem with a gawk (yes, gawk!) program I was writing. I had to accumulate 10,000,000 32-character strings to produce a 320,000,000 (three hundred and twenty million!) character string. It was taking forever. I eventually realized that this string reallocation that was being done 10,000,000 times was the problem. To solve this, I did a two-level accumulation (perhaps three levels would have been better, but two was enough). I first accumulated 3,000 of the 32-character strings (3,000 because that was about the square root of 10,000,000). I then accumulated the (about) 3,000 of these (about) 100,000 character strings. The result took about 30 seconds, which was good enough for what I needed to do.