3 ms·
> This is all working fine Are there any specific reasons that you think Postgres won't be able to scale? I don't have personal experience solving the problem
by sixhobbits 7y ago
> This is all working fine
Are there any specific reasons that you think Postgres won't be able to scale?
I don't have personal experience solving the problem you're describing but if you do choose to use something other than Postgres I think Apache Kafka might be interesting for some parts of your infrastructure at least.
Segment [0] has written quite a lot about their tech in a similar space.
[0] https://segment.com/blog/the-10m-engineering-problem/ https://segment.com/blog/the-10m-engineering-problem/
- AdriaanvRossum 7y agoI think you are right, I'm kind of looking for a technical challenge where I actually should just improve the current setup. Thanks for the link!
- deleted 7y ago[deleted]
- hodgesrm 7y agoCloudflare documented their PostgreSQL to ClickHouse journey here: https://blog.cloudflare.com/http-analytics-for-6m-requests-per-second-using-clickhouse/ https://blog.cloudflare.com/http-analytics-for-6m-requests-p... It covers a lot of the scaling bottlenecks in their original PG-based pipeline and how they eliminated them. (Disclaimer: I work on ClickHouse.)