4 ms·

Compare with SoftLayer which has decided to charge through the nose for their RAM upgrades: going from 16 to 64GB RAM on their E5 servers is an additional $1200

by Erwin 14y ago

Compare with SoftLayer which has decided to charge through the nose for their RAM upgrades: going from 16 to 64GB RAM on their E5 servers is an additional $1200/month. They're far more expensive than e.g. Rackspace there for some reason.

Smerity 14y ago
I haven't looked at SoftLayer's offerings re: RAM but the most important thing to note with Hetzner is that only some of their dedicated machines have error-correcting code (ECC) memory. Hetzner's 16GB machine[1] (89 Euros/month) has ECC memory for example. Whilst this might not sound like an enormous issue, DRAM errors are surprisingly common in real deployments[2] and can result in scary things happening to your data. If you're using a machine without ECC memory make sure you're prepared to deal with any possible issues that might arise, especially if it will impact your core business. [1]: http://www.hetzner.de/en/hosting/produkte_rootserver/ex8 http://www.hetzner.de/en/hosting/produkte_rootserver/ex8 [2] DRAM Errors in the Wild: A Large-Scale Field Study: http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
- jules 14y ago
  How common DRAM errors are is very unclear. The numbers in that paper are astronomically high, but other sources have published numbers that are 30x or 100,000x or literally 10,000,000x lower. See this Stanford ee380 talk: http://stanford-online.stanford.edu/courses/ee380/100922-ee380-300.asx http://stanford-online.stanford.edu/courses/ee380/100922-ee3... The part about DRAM errors is about 57 minutes in. Abstract here: http://www.stanford.edu/class/ee380/Abstracts/100922.html http://www.stanford.edu/class/ee380/Abstracts/100922.html Also, AFAIK most hosting providers don't even have ECC ram as an option for servers, e.g. Amazon.
  - justincormack 14y ago
    Do you have a source on Amazon not using ECC?
    - jules 14y ago
      I was not able to find Amazon declaring that they don't, but neither do they say anywhere that they do. For example they describe the hardware specs here: http://aws.amazon.com/ec2/instance-types/ http://aws.amazon.com/ec2/instance-types/ If they did have ECC RAM it is unlikely that they would keep it secret, especially given that ECC RAM can be twice as expensive.
    - EwanToo 14y ago
      I don't think there's a confirmed source, but there's certainly indications, for example, an AWS post that they use ECC memory for the GPUs in their cluster GPU instance http://aws.typepad.com/aws/2010/11/new-ec2-instance-type-the-cluster-gpu-instance.html http://aws.typepad.com/aws/2010/11/new-ec2-instance-type-the... Given that they list both the standard memory and GPU memory next to each other, but only put ECC next to the GPU memory, it seems relatively likely to me that the standard server memory is not ECC.
      - justincormack 14y ago
        But Nvidia only sell Tesla's with ECC. I don't really read that implication into this. On the other hand, the fact that https://forums.aws.amazon.com/message.jspa?messageID=203167 https://forums.aws.amazon.com/message.jspa?messageID=203167 has never been answered suggests they do not, as they would answer affirmatively if it was true surely. Of course they may use a mixture. EDIT: Interesting that James Hamilton is on their team and thinks they should use it http://perspectives.mvdirona.com/2009/10/07/YouReallyDONeedECCMemory.aspx http://perspectives.mvdirona.com/2009/10/07/YouReallyDONeedE...
        EwanToo 14y ago
        I can well imagine that newer machines have it, and older ones don't, and slowly over time they'll end up with everyone using ECC. The costs involved were a lot higher 3-5 years ago than they are today.
  - wmf 14y ago
    Amazon uses Xeons and Opterons where ECC is pretty much mandatory. They don't have an option for ECC because all their servers have it.
- nirvana 14y ago
  I think it is a good idea to architect your system under the assumption that once a year, some theif is going to sneak into the data center and make off with an entire server. Just one, but the whole server will then die horribly in a fire after a shoot-out with the police. Lots of things can happen, you should have a higher level of replication such that you can handle a whole server going poof, not just a single bit going poof. The cost of ECC at Hetzner-- the cheapest provider out there- is about half an additional server. So, buy three servers without ECC for the price of two servers with ECC, and replicate your data three times (and triple your bandwidth, horsepower, etc.) This is not hard with platforms like Riak which are distributed homogenous clusters of nodes. And if your service isn't built like that, then really it should be. (IMNSHO, of course.)
  - AdamGibbins 14y ago
    Replication isn't going to help if your data is being silently corrupted, you'll just replicate the corruption.
    - nirvana 14y ago
      ECC Does nothing to prevent your server from dying in a fire after that police chase, either. However, replication is an effective solution to the problem you describe: Whenever data is read, you read from more than one replica, and then compare them. If one of them has been corrupted, its hash won't match and you'll know it. You can then write out the correct data to the node with the error. This is very easy in systems like Riak.
stevencorona 14y ago
SoftLayer charges an insane amount for memory

by Erwin 14y ago

Smerity 14y ago

I haven't looked at SoftLayer's offerings re: RAM but the most important thing to note with Hetzner is that only some of their dedicated machines have error-correcting code (ECC) memory. Hetzner's 16GB machine[1] (89 Euros/month) has ECC memory for example. Whilst this might not sound like an enormous issue, DRAM errors are surprisingly common in real deployments[2] and can result in scary things happening to your data. If you're using a machine without ECC memory make sure you're prepared to deal with any possible issues that might arise, especially if it will impact your core business. [1]: http://www.hetzner.de/en/hosting/produkte_rootserver/ex8 http://www.hetzner.de/en/hosting/produkte_rootserver/ex8 [2] DRAM Errors in the Wild: A Large-Scale Field Study: http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf

jules 14y ago
How common DRAM errors are is very unclear. The numbers in that paper are astronomically high, but other sources have published numbers that are 30x or 100,000x or literally 10,000,000x lower. See this Stanford ee380 talk: http://stanford-online.stanford.edu/courses/ee380/100922-ee380-300.asx http://stanford-online.stanford.edu/courses/ee380/100922-ee3... The part about DRAM errors is about 57 minutes in. Abstract here: http://www.stanford.edu/class/ee380/Abstracts/100922.html http://www.stanford.edu/class/ee380/Abstracts/100922.html Also, AFAIK most hosting providers don't even have ECC ram as an option for servers, e.g. Amazon.
- justincormack 14y ago
  Do you have a source on Amazon not using ECC?
  - jules 14y ago
    I was not able to find Amazon declaring that they don't, but neither do they say anywhere that they do. For example they describe the hardware specs here: http://aws.amazon.com/ec2/instance-types/ http://aws.amazon.com/ec2/instance-types/ If they did have ECC RAM it is unlikely that they would keep it secret, especially given that ECC RAM can be twice as expensive.
  - EwanToo 14y ago
    I don't think there's a confirmed source, but there's certainly indications, for example, an AWS post that they use ECC memory for the GPUs in their cluster GPU instance http://aws.typepad.com/aws/2010/11/new-ec2-instance-type-the-cluster-gpu-instance.html http://aws.typepad.com/aws/2010/11/new-ec2-instance-type-the... Given that they list both the standard memory and GPU memory next to each other, but only put ECC next to the GPU memory, it seems relatively likely to me that the standard server memory is not ECC.
    - justincormack 14y ago
      But Nvidia only sell Tesla's with ECC. I don't really read that implication into this. On the other hand, the fact that https://forums.aws.amazon.com/message.jspa?messageID=203167 https://forums.aws.amazon.com/message.jspa?messageID=203167 has never been answered suggests they do not, as they would answer affirmatively if it was true surely. Of course they may use a mixture. EDIT: Interesting that James Hamilton is on their team and thinks they should use it http://perspectives.mvdirona.com/2009/10/07/YouReallyDONeedECCMemory.aspx http://perspectives.mvdirona.com/2009/10/07/YouReallyDONeedE...
      - EwanToo 14y ago
        I can well imagine that newer machines have it, and older ones don't, and slowly over time they'll end up with everyone using ECC. The costs involved were a lot higher 3-5 years ago than they are today.
- wmf 14y ago
  Amazon uses Xeons and Opterons where ECC is pretty much mandatory. They don't have an option for ECC because all their servers have it.
nirvana 14y ago
I think it is a good idea to architect your system under the assumption that once a year, some theif is going to sneak into the data center and make off with an entire server. Just one, but the whole server will then die horribly in a fire after a shoot-out with the police. Lots of things can happen, you should have a higher level of replication such that you can handle a whole server going poof, not just a single bit going poof. The cost of ECC at Hetzner-- the cheapest provider out there- is about half an additional server. So, buy three servers without ECC for the price of two servers with ECC, and replicate your data three times (and triple your bandwidth, horsepower, etc.) This is not hard with platforms like Riak which are distributed homogenous clusters of nodes. And if your service isn't built like that, then really it should be. (IMNSHO, of course.)
- AdamGibbins 14y ago
  Replication isn't going to help if your data is being silently corrupted, you'll just replicate the corruption.
  - nirvana 14y ago
    ECC Does nothing to prevent your server from dying in a fire after that police chase, either. However, replication is an effective solution to the problem you describe: Whenever data is read, you read from more than one replica, and then compare them. If one of them has been corrupted, its hash won't match and you'll know it. You can then write out the correct data to the node with the error. This is very easy in systems like Riak.

stevencorona 14y ago

SoftLayer charges an insane amount for memory