4 ms·
This seems to be (with coincidental timing) the cause of Mandrill’s current outage [1]: > Mandrill uses a sharded Postgres setup as one of our main datastores.
by amarshall 8y ago
This seems to be (with coincidental timing) the cause of Mandrill’s current outage [1]:
> Mandrill uses a sharded Postgres setup as one of our main datastores. On Sunday, February 3, at 10:30pm EST, 1 of our 5 physical Postgres instances saw a significant spike in writes. The spike in writes triggered a Transaction ID Wraparound issue. When this occurs, database activity is completely halted. The database sets itself in read-only mode until offline maintenance (known as vacuuming) can occur.
> The database is large—running the vacuum process takes a significant amount of time and resources, and there’s no clear way to track progress.
[1] https://news.ycombinator.com/item?id=19084525 https://news.ycombinator.com/item?id=19084525
- ldng 8y agoIt is probably too late, for later reference, when this talk is up, check it out, it has some answers. https://fosdem.org/2019/schedule/event/breaking_postgresql_on_scale/ https://fosdem.org/2019/schedule/event/breaking_postgresql_o...