4 ms·

We need more details on 6. This is the hard part, like you swap connection from A to B, but if B is not synced properly and you write to it then you start havin

by Thaxll 8mo ago

We need more details on 6. This is the hard part, like you swap connection from A to B, but if B is not synced properly and you write to it then you start having diff between the two and there is no way back. Like B is slightly out of date ( replication wise ) the service modify something, then A comes with change that modify the same data that you just wrote. How do you ensure that B is up to date without stopping write to A ( no downtime ).

Kaliboy 8mo ago
Not sure how they do it, but I would do it like so: Have old database be master. Let new be a slave. Load in latest db dump, may take as long as it wants. Then start replication and catch up on the delay. You would need, depending on the db type, a load balancer/failover manager. PgBouncer and PgPoolII come to mind, but MySQL has some as well. Let that connect to the master and slave, connect the application to the database through that layer. Then trigger a failover. That should be it.
- Snelius 8mo ago
  > Load in latest db dump, may take as long as it wants. 400TB its about a week+ ? > Then start replication and catch up on the delay. Then u have a changes in the delay about +- 1TB. It means a changes syncing about few days more while changes still coming. They said "current requests are buffered" which is impossible, especial for long distributed (optional) transactions which in a progress (it can spend a hours, days (for analitycs)). Overwall this article is a BS or some super custom case which irrelevant for common systems. You can't migrate w/o downtime, it's a physical impossible.
  - freakynit 8mo ago
    Feels the same to me as well. "Take snapshot and begin streaming replication"... like to where? The snapshot isn't even prepared fully yet and definitely hasn't reached the target. Where are you dumping/keeping those replication logs for the time being? Secondly, how are you managing database state changes due to realtime update queries? They are definitely going in source table at this point. I don't get this. Im still stuck on point 1... have read it twice already.
    - mattlord 8mo ago
      It's open source. If you want to understand exactly how, you certainly can! :-) https://github.com/vitessio/vitess https://github.com/vitessio/vitess https://vitess.io/docs/reference/vreplication/ https://vitess.io/docs/reference/vreplication/ https://vitess.io/docs/reference/features/vtgate-buffering/ https://vitess.io/docs/reference/features/vtgate-buffering/
      - Snelius 8mo ago
        He can't. It's not a reference, just a bunch of CLI examples. Please learn what is the reference. Even docs is a BS, wonderful product. Overall this article is a typical advertising and clickbait..
        joshuamorton 8mo ago
        The code is open source though, you can read it. The cli examples point you towards the relevant bits of the actual database code to read. For my own sake, I'm not sure what is so surprising here. "Turn up a hot second replica and fail over to it intentionally behind a global load balancer." Is pretty well trodden ground.
        Snelius 8mo ago
        > The code is open source though, you can read it Thank you! :D > Is pretty well trodden ground. YES!! But the article point us to it's a 400TB+ w/o downtime migration. This is impossible. That why is looks like clickbait and advertising of a product.
        joshuamorton 8mo ago
        I will simply point out that I'm aware of larger zero down-time migrations, like this one: https://www.youtube.com/watch?v=ih97gwNmkRA https://www.youtube.com/watch?v=ih97gwNmkRA
  - mattlord 8mo ago
    So you don't understand how something works. That's fine. But to then say the article and/or tech are BS is... a choice. This work has been and is being used by some of the largest sites / apps in the world including Uber, Slack, GitHub, Square... But sure, "it's BS, super custom, and irrelevant". Gee, yer super smart! Thank you for the amazing insights. 5 stars.
mattlord 8mo ago
It's open source. You can get as many details as you like :) https://github.com/vitessio/vitess https://github.com/vitessio/vitess https://vitess.io/docs/reference/vreplication/ https://vitess.io/docs/reference/vreplication/ https://vitess.io/docs/reference/features/vtgate-buffering/ https://vitess.io/docs/reference/features/vtgate-buffering/

by Thaxll 8mo ago

Kaliboy 8mo ago

Not sure how they do it, but I would do it like so: Have old database be master. Let new be a slave. Load in latest db dump, may take as long as it wants. Then start replication and catch up on the delay. You would need, depending on the db type, a load balancer/failover manager. PgBouncer and PgPoolII come to mind, but MySQL has some as well. Let that connect to the master and slave, connect the application to the database through that layer. Then trigger a failover. That should be it.

Snelius 8mo ago
> Load in latest db dump, may take as long as it wants. 400TB its about a week+ ? > Then start replication and catch up on the delay. Then u have a changes in the delay about +- 1TB. It means a changes syncing about few days more while changes still coming. They said "current requests are buffered" which is impossible, especial for long distributed (optional) transactions which in a progress (it can spend a hours, days (for analitycs)). Overwall this article is a BS or some super custom case which irrelevant for common systems. You can't migrate w/o downtime, it's a physical impossible.
- freakynit 8mo ago
  Feels the same to me as well. "Take snapshot and begin streaming replication"... like to where? The snapshot isn't even prepared fully yet and definitely hasn't reached the target. Where are you dumping/keeping those replication logs for the time being? Secondly, how are you managing database state changes due to realtime update queries? They are definitely going in source table at this point. I don't get this. Im still stuck on point 1... have read it twice already.
  - mattlord 8mo ago
    It's open source. If you want to understand exactly how, you certainly can! :-) https://github.com/vitessio/vitess https://github.com/vitessio/vitess https://vitess.io/docs/reference/vreplication/ https://vitess.io/docs/reference/vreplication/ https://vitess.io/docs/reference/features/vtgate-buffering/ https://vitess.io/docs/reference/features/vtgate-buffering/
    - Snelius 8mo ago
      He can't. It's not a reference, just a bunch of CLI examples. Please learn what is the reference. Even docs is a BS, wonderful product. Overall this article is a typical advertising and clickbait..
      - joshuamorton 8mo ago
        The code is open source though, you can read it. The cli examples point you towards the relevant bits of the actual database code to read. For my own sake, I'm not sure what is so surprising here. "Turn up a hot second replica and fail over to it intentionally behind a global load balancer." Is pretty well trodden ground.
        Snelius 8mo ago
        > The code is open source though, you can read it Thank you! :D > Is pretty well trodden ground. YES!! But the article point us to it's a 400TB+ w/o downtime migration. This is impossible. That why is looks like clickbait and advertising of a product.
        joshuamorton 8mo ago
        I will simply point out that I'm aware of larger zero down-time migrations, like this one: https://www.youtube.com/watch?v=ih97gwNmkRA https://www.youtube.com/watch?v=ih97gwNmkRA
- mattlord 8mo ago
  So you don't understand how something works. That's fine. But to then say the article and/or tech are BS is... a choice. This work has been and is being used by some of the largest sites / apps in the world including Uber, Slack, GitHub, Square... But sure, "it's BS, super custom, and irrelevant". Gee, yer super smart! Thank you for the amazing insights. 5 stars.

mattlord 8mo ago

It's open source. You can get as many details as you like :) https://github.com/vitessio/vitess https://github.com/vitessio/vitess https://vitess.io/docs/reference/vreplication/ https://vitess.io/docs/reference/vreplication/ https://vitess.io/docs/reference/features/vtgate-buffering/ https://vitess.io/docs/reference/features/vtgate-buffering/