17 ms·

Pretty similar to that, we used pigz and netcat to bring up new MySQL read replicas in a chain at line speeds. I recall learning the technique from Tumblr's en

• evanelias 4y ago
  I wrote that Tumblr eng blog post, glad to see it's still making the rounds! I later joined FB's mysql team a few years after that, although I can't quite remember if FB was still using pigz by that time. (also, hi Eric!) Separately, at Tumblr I vaguely remember examining some alternative to pigz that was consistently faster at the time (11 years ago) because pigz couldn't parallelize decompression. Can't quite remember the name of the alternative, but it had licensing restrictions which made it less attractive than pigz. Edit: the old fast alternative I was thinking of is qpress, formerly hosted at http://www.quicklz.com/ http://www.quicklz.com/ but that's no longer online. Googling it now, there are some mirrors and also looks like Percona tools used/bundled it. Not sure if they still do or if they've since switched to zstd.
  • Xorlev 4y ago
    Small world! Thanks for writing that, it was a really clever way to do it and saved me a bunch of time. :)