3 ms·

Amazon EC2 offers 68GB RAM and it's hardly an obscure offering. But they don't offer SSDs. SSD might be a solution, but we don't have any proof either way, and

by ntoshev 15y ago

Amazon EC2 offers 68GB RAM and it's hardly an obscure offering. But they don't offer SSDs. SSD might be a solution, but we don't have any proof either way, and we also don't understand why is this problem happening.

dmk23 15y ago
Quoting from the article: ============================= What to do? I wish I could say you should use Percona Server with XtraDB. If we were using SSD as storage, then I would recommend it. Vanilla MySQL performs equally bad on SSD and HDD, while for SSD in Percona Server we have “innodb_adaptive_flushing_method = keep_average”. Unfortunately on spinning disks (as in this case), Percona Server may not show significant improvement. I am going to followup with results on Percona Server. ============================= SSD is the solution because unlike rotational disks it allows for lots of random accesses. While they are too expensive to use as "archiving" storage, SSD excels as a layer between traditional disks and RAM. Put your high I/O workload on SSD and keep your historic data on spinning disks. If you want to see some SSD benchmarks, here is your resource: http://www.ssdperformanceblog.com/ http://www.ssdperformanceblog.com/ As far as cloud storage, yes, Amazon EC2 offers 68GB RAM, however if you need hardware of that size it is not hard to justify setting up your own servers. It would be cheaper too and you'll be able to fully optimize your storage.
spudlyo 15y ago
We do understand why this problem is happening. The algorithm that takes the nice fast (but limited in size) sequential log writes and tries to intelligently write them (mostly randomly) in the background to the data/index blocks where they live can't keep up, and eventually stalls out as the amount of free log space dwindles. SSDs really help this process happen faster because they have way better random write performance than hard disks.
alnayyir 15y ago
I get random writes on the order of 5-10 mb/s at best on a typical server we use for our databases. My macbook pro I upgraded with a gaming grade SSD gets 100-200 mb/s random write. Just upgrading to SSDs is more viable than hacking up a database with a huge community behind it that you didn't originally create yourself.

by ntoshev 15y ago

dmk23 15y ago

Quoting from the article: ============================= What to do? I wish I could say you should use Percona Server with XtraDB. If we were using SSD as storage, then I would recommend it. Vanilla MySQL performs equally bad on SSD and HDD, while for SSD in Percona Server we have “innodb_adaptive_flushing_method = keep_average”. Unfortunately on spinning disks (as in this case), Percona Server may not show significant improvement. I am going to followup with results on Percona Server. ============================= SSD is the solution because unlike rotational disks it allows for lots of random accesses. While they are too expensive to use as "archiving" storage, SSD excels as a layer between traditional disks and RAM. Put your high I/O workload on SSD and keep your historic data on spinning disks. If you want to see some SSD benchmarks, here is your resource: http://www.ssdperformanceblog.com/ http://www.ssdperformanceblog.com/ As far as cloud storage, yes, Amazon EC2 offers 68GB RAM, however if you need hardware of that size it is not hard to justify setting up your own servers. It would be cheaper too and you'll be able to fully optimize your storage.

spudlyo 15y ago

We do understand why this problem is happening. The algorithm that takes the nice fast (but limited in size) sequential log writes and tries to intelligently write them (mostly randomly) in the background to the data/index blocks where they live can't keep up, and eventually stalls out as the amount of free log space dwindles. SSDs really help this process happen faster because they have way better random write performance than hard disks.

alnayyir 15y ago

I get random writes on the order of 5-10 mb/s at best on a typical server we use for our databases. My macbook pro I upgraded with a gaming grade SSD gets 100-200 mb/s random write. Just upgrading to SSDs is more viable than hacking up a database with a huge community behind it that you didn't originally create yourself.