4 ms·

The issue is x86/x64 being a turd. Throughput of just about everything drops when you go over the 36Gb boundary for some reason. I have no idea why. It can d

The issue is x86/x64 being a turd. Throughput of just about everything drops when you go over the 36Gb boundary for some reason. I have no idea why. It can drop by as much as 30%. This is from experience operating VMware on the physically the same type of kit (except with FC SAN / NetApp). I'd personally rather buy high end SPARC64 kit which has linear performance scalability but we all know what happened to Sun, plus it doesn't run Windows anyway :( whimpers a little

trafficlight 15y ago
I haven't heard/experienced the 36 GB thing since I don't run anything that big right now. Are you aware of any websites or blog posts that talk about it?
- pointyhat 15y ago
  Some bits buried in here: http://frankdenneman.nl/2010/02/sizing-vms-and-numa-nodes/ http://frankdenneman.nl/2010/02/sizing-vms-and-numa-nodes/
zwischenzug 15y ago
I've seen DBs run on 128G machines on x6 and have no experience of such problems. Do you have a link or hard evidence of this? I have seen throughput issues on VMWare machines (and false reporting on standard linux utilities, eg "time" output and various other issues that make me distrust it for performance-sensitive software), but don't know if this is to do with 128G machines; usually it's to do with contention on one network card handling all traffic to disks.
jeremyw 15y ago
Be careful of your NUMA boundaries. 20-30% is exactly the cross-bank access cost.
- pointyhat 15y ago
  That's what I was thinking. Need some analysis on it really. We throw money at the problem at the moment and it goes away :)
- spudlyo 15y ago
  That's what I thought too. There are a lot of weird problems that can happen due to NUMA architecture issues. Reading the numactl for me really drives home just how much I don't understand it. http://www.linuxmanpages.com/man8/numactl.8.php http://www.linuxmanpages.com/man8/numactl.8.php

trafficlight 15y ago

I haven't heard/experienced the 36 GB thing since I don't run anything that big right now. Are you aware of any websites or blog posts that talk about it?

pointyhat 15y ago
Some bits buried in here: http://frankdenneman.nl/2010/02/sizing-vms-and-numa-nodes/ http://frankdenneman.nl/2010/02/sizing-vms-and-numa-nodes/

zwischenzug 15y ago

I've seen DBs run on 128G machines on x6 and have no experience of such problems. Do you have a link or hard evidence of this? I have seen throughput issues on VMWare machines (and false reporting on standard linux utilities, eg "time" output and various other issues that make me distrust it for performance-sensitive software), but don't know if this is to do with 128G machines; usually it's to do with contention on one network card handling all traffic to disks.

jeremyw 15y ago

Be careful of your NUMA boundaries. 20-30% is exactly the cross-bank access cost.

pointyhat 15y ago
That's what I was thinking. Need some analysis on it really. We throw money at the problem at the moment and it goes away :)
spudlyo 15y ago
That's what I thought too. There are a lot of weird problems that can happen due to NUMA architecture issues. Reading the numactl for me really drives home just how much I don't understand it. http://www.linuxmanpages.com/man8/numactl.8.php http://www.linuxmanpages.com/man8/numactl.8.php