2 ms·

There is a list of issues in https://about.gitlab.com/2017/02/10/postmortem-of-database-outage-of-january-31/ https://about.gitlab.com/2017/02/10/postmortem-of-

by a3_nm 10y ago

There is a list of issues in https://about.gitlab.com/2017/02/10/postmortem-of-database-outage-of-january-31/ https://about.gitlab.com/2017/02/10/postmortem-of-database-o... -- also in https://docs.google.com/document/d/1GCK53YDcBWQveod9kfzW-VCxIABGiryG7_z_6jHdVik/pub https://docs.google.com/document/d/1GCK53YDcBWQveod9kfzW-VCx..., see Recovery, 3, l. I think it's great that they are being completely transparent about this. That said, it's true that it's been almost two months and it seems that the some important issues there are still open and don't look especially active.

sytse 10y ago
The follow up was pretty extensive and we'll be working on it for months to come. Some issues that have been done: 1. Update PS1 across all hosts to more clearly differentiate between hosts and environments https://gitlab.com/gitlab-com/infrastructure/issues/1094 https://gitlab.com/gitlab-com/infrastructure/issues/1094 2. Set PostgreSQL's max_connections to a sane value https://gitlab.com/gitlab-com/infrastructure/issues/1096 https://gitlab.com/gitlab-com/infrastructure/issues/1096 3. Move staging to the ARM environment https://gitlab.com/gitlab-com/infrastructure/issues/1100 https://gitlab.com/gitlab-com/infrastructure/issues/1100 4. Improve PostgreSQL replication documentation/runbooks https://gitlab.com/gitlab-com/infrastructure/issues/1103 https://gitlab.com/gitlab-com/infrastructure/issues/1103 5. Build Streaming Database Backup https://gitlab.com/gitlab-com/infrastructure/issues/1152 https://gitlab.com/gitlab-com/infrastructure/issues/1152 6. Assign an owner for data durability https://gitlab.com/gitlab-com/infrastructure/issues/1163 https://gitlab.com/gitlab-com/infrastructure/issues/1163