"But isn't transient log storage a popular use case, and if so why don't any services optimize for it?" Well I could start with "define transient"! I also not

"But isn't transient log storage a popular use case, and if so why don't any services optimize for it?" Well I could start with "define transient"! I also note your unhappiness with third party logging facilities. I also note you went very DIY. A three node ES cluster with a Graylog on the front can ingest quite a lot of logs for a reasonable time outlay in learning how it hangs together. A bit of tuning might be required. I suspect I'm not your target market.