4 ms·
Geat post!! At Rudder (open-source segment), we used Postgres as our streaming layer in a similar fashion. It has been super stable in production, very easy to
by soumyadeb 7y ago
Geat post!! At Rudder (open-source segment), we used Postgres as our streaming layer in a similar fashion. It has been super stable in production, very easy to setup and we easily get to > 10K events/sec insertion performance.
The code is open-sourced here in case anyone wants to re-use
https://github.com/rudderlabs/rudder-server/blob/master/jobsdb/jobsdb.go https://github.com/rudderlabs/rudder-server/blob/master/jobs...
We had to built additional logic to clean-up old jobs (similar to Level merges in similar queing systems)