4 ms·

Very helpful article! It jives with the evolution of my own understanding of autovacuum tuning over a decade and a half of writing high-performance Postgres-bac

Very helpful article! It jives with the evolution of my own understanding of autovacuum tuning over a decade and a half of writing high-performance Postgres-backed VoIP routing systems. I would add to this that if you truly have a high-churn table, perhaps a traditional RDBM is not the best choice. Ask me how I learned that lesson the hard way. There are many things that, in hindsight, really should have gone into ELK or Splunk or Redis or something more suited to ephemeral or short-lived data sets that also turn over constantly. But, sometimes there is no choice, or the churn is high but not quite high enough to rule out the use of an RDBM altogether. For those scenarios, the tips in this article really shine.

sophacles 3y ago
How'd you learn that lesson the hard way?
- abalashov 3y ago
  I made the mistake of writing a searchable syslog archiver where the parsing was done in stored functions (very basic string-splitting) and the log entries were stored in a heavily indexed Postgres table. At the time it was created, the company was in its infancy, there wasn't much throughput, the service emitting the logs was kind of obscure, and the solution was a quick, dirty stopgap meant to be easily digestible and consumable to the customer, compatible with the rest of their in-house knowledge and the rest of their backend stack. However, as happens too often, the company grew meteorically, the volume got to be insane, many more default-on log entries were added to the standard event loop, and the system was put straight into production without any interest in redesign; after all, it worked fine so far. As you might guess, it got to the point where we were seeing insanely degraded performance and cascade failures after only a few days' log activity, and increasing time and resources were devoted to the endless care and feeding of this beast. It took a surprisingly long time for the rubber band to snap and for the customer to consider ELK, despite the fact that I advocated for this almost from the very beginning.

sophacles 3y ago

How'd you learn that lesson the hard way?

abalashov 3y ago
I made the mistake of writing a searchable syslog archiver where the parsing was done in stored functions (very basic string-splitting) and the log entries were stored in a heavily indexed Postgres table. At the time it was created, the company was in its infancy, there wasn't much throughput, the service emitting the logs was kind of obscure, and the solution was a quick, dirty stopgap meant to be easily digestible and consumable to the customer, compatible with the rest of their in-house knowledge and the rest of their backend stack. However, as happens too often, the company grew meteorically, the volume got to be insane, many more default-on log entries were added to the standard event loop, and the system was put straight into production without any interest in redesign; after all, it worked fine so far. As you might guess, it got to the point where we were seeing insanely degraded performance and cascade failures after only a few days' log activity, and increasing time and resources were devoted to the endless care and feeding of this beast. It took a surprisingly long time for the rubber band to snap and for the customer to consider ELK, despite the fact that I advocated for this almost from the very beginning.