Thank you for your comment. I hear you when you say postgres would probably be faster. However, I imagine there would be more work if I chose postgres and the w

Thank you for your comment. I hear you when you say postgres would probably be faster. However, I imagine there would be more work if I chose postgres and the whole development and testing paradigm would change. First, I would need to figure the right schema and populate the database. Second, it would need some creative SQL acrobatics that I would probably not be comfortable with. Third, it would probably be hard to perform quick tests at a small scale that I can perform easily with text files. Fourth, the solution would probably be hard to port elsewhere where postgres is not available. Most Unix systems have awk available. Fifth, programming the postgres db in a higher level language would require connector api libs which would be additional effort. Note that this is not a production work -- it started simply as a hack to see how far I can go without getting into serious rabbitholes and giving up. Surprisingly, with Awk I went really far and never fell into any rabbithole so to speak.