3 ms·

They needed the Percona toolkit (previously Maatkit). http://www.percona.com/doc/percona-toolkit/2.0/pt-table-sync.html http://www.percona.com/doc/percona-toolk

They needed the Percona toolkit (previously Maatkit). http://www.percona.com/doc/percona-toolkit/2.0/pt-table-sync.html http://www.percona.com/doc/percona-toolkit/2.0/pt-table-sync...

kfool 15y ago
The toolkit documentation reports limitations. What they really needed is ChronicDB http://chronicdb.com http://chronicdb.com
- waitwhat 15y ago
  ChronicDB is a hosted solution. That isn't even remotely what DeviantArt were looking for.
  - kfool 15y ago
    ChronicDB is not only a hosted solution. It also runs locally.
    - waitwhat 15y ago
      Not according to your own website. ChronicDB is a combination of a software engineer and a hosted database: You run your database in our datacenters -- http://chronicdb.com/how_it_works http://chronicdb.com/how_it_works Your database will be hosted in our datacenters -- http://chronicdb.com/features http://chronicdb.com/features
- newman314 15y ago
  If you are going to flog ChronicDB, please at least disclose if you have some sort of relationship with said company. Most of your previous submissions pertain to ChronicDB.
  - kfool 15y ago
    Sorry for not disclosing I work on ChronicDB. The author of the 6b article kindly provided more information about the time it took for the migration: http://news.ycombinator.com/item?id=3621561 http://news.ycombinator.com/item?id=3621561 A couple of limitations of the Percona tool are related to changing data in tables with ON DELETE and ON UPDATE foreign key constraints and possible lock-ups of tables. It is a bi-directional replication tool, so it has to deal with the master-master replication case and as such does not guarantee data consistency. Still the tool is helpful in many cases and congrats to Percona for developing it. Replication safety is a non-trivial problem in general.
    - waitwhat 15y ago
      They weren't doing a master-master sync.
banksyw 15y ago
Disclaimer: I wrote that article. Percona toolkit has some useful stuff. Thanks for the link. In this case our solution ended up being pretty much the same as pt-table-sync except that it made use of our distributed task queuing system so was MUCH faster than if we had used a single threaded perl script. (Up to a few hundred chunks of rows being checksummed/synced in parallel.) I didn't talk about this aspect directly in the article to keep it focused. Not sure if it would be possible to use pt-table-sync in that way, certainly would have taken a lot more work to get a third-party perl script to distribute the work through our job queue.

They needed the Percona toolkit (previously Maatkit). http://www.percona.com/doc/percona-toolkit/2.0/pt-table-sync.html http://www.percona.com/doc/percona-toolkit/2.0/pt-table-sync...

kfool 15y ago

The toolkit documentation reports limitations. What they really needed is ChronicDB http://chronicdb.com http://chronicdb.com

waitwhat 15y ago
ChronicDB is a hosted solution. That isn't even remotely what DeviantArt were looking for.
- kfool 15y ago
  ChronicDB is not only a hosted solution. It also runs locally.
  - waitwhat 15y ago
    Not according to your own website. ChronicDB is a combination of a software engineer and a hosted database: You run your database in our datacenters -- http://chronicdb.com/how_it_works http://chronicdb.com/how_it_works Your database will be hosted in our datacenters -- http://chronicdb.com/features http://chronicdb.com/features
newman314 15y ago
If you are going to flog ChronicDB, please at least disclose if you have some sort of relationship with said company. Most of your previous submissions pertain to ChronicDB.
- kfool 15y ago
  Sorry for not disclosing I work on ChronicDB. The author of the 6b article kindly provided more information about the time it took for the migration: http://news.ycombinator.com/item?id=3621561 http://news.ycombinator.com/item?id=3621561 A couple of limitations of the Percona tool are related to changing data in tables with ON DELETE and ON UPDATE foreign key constraints and possible lock-ups of tables. It is a bi-directional replication tool, so it has to deal with the master-master replication case and as such does not guarantee data consistency. Still the tool is helpful in many cases and congrats to Percona for developing it. Replication safety is a non-trivial problem in general.
  - waitwhat 15y ago
    They weren't doing a master-master sync.

banksyw 15y ago

Disclaimer: I wrote that article. Percona toolkit has some useful stuff. Thanks for the link. In this case our solution ended up being pretty much the same as pt-table-sync except that it made use of our distributed task queuing system so was MUCH faster than if we had used a single threaded perl script. (Up to a few hundred chunks of rows being checksummed/synced in parallel.) I didn't talk about this aspect directly in the article to keep it focused. Not sure if it would be possible to use pt-table-sync in that way, certainly would have taken a lot more work to get a third-party perl script to distribute the work through our job queue.