4 ms·

I would rethink that if I were you. What if you started to provide virtual private servers on one of these boxes instead of offering colocation? I think this in

I would rethink that if I were you. What if you started to provide virtual private servers on one of these boxes instead of offering colocation? I think this innovation makes VPS solutions even more efficient than they are today.

lsc 16y ago
ECC is a huge thing in that case. The thing about being a VPS provider is that if your hardware chokes, your customers notice, and they get pissed. Sure, if you are running some kind of web farm you set it up so that things keep running without interruption if you lose a server. But if you provide VPS hosting and you lose a server? People notice, and they will be pissed. If you are a VPS provider and you loose customer data? they will parade your severed head around town on the end of a pike. Amazon has successfully tackled this problem by training their customers that they can shoot a server at any time. This is good for them, but it's not how VPSs have traditionally operated. customers expect the thing to stay up and for their data to stay safe.
- bnoordhuis 16y ago
  I've seen ECC mentioned a few times in this thread but how relevant is it really? At my old job we used both regular COTS hardware from Dell and HP and high-performance gear from SUN, often at four times the price. The latter had ECC memory and, if memory serves, shielded CPUs but I never noticed much difference in stability or performance when compared to regular hardware. If someone has some numbers on this, I'd love to see them.
  - lsc 16y ago
    if your dell and HP kit was server grade, it almost certainly had ECC ram. before nehalam, it was almost impossible to get dual-cpu servers that would support non-ecc ram. Even today, now that unbuffered ram is supported (buffering and ECC are not the same thing, but it's unusual to see registered or buffered ram that is not also ECC) nearly all server hardware you get from the likes of dell or HP is going to default to using ECC ram. The big deal with ECC is that with non-ecc ram, you don't know when in-ram data was corrupted. If you have a bad bit of ram that is being used for our journaling/disk caching, it's pretty easy to corrupt your data. If you've got ECC, it corrects single bit errors, and logs that there was a problem, and it can be configured to crash on double bit errors, rather than just silently corrupting your data. If you use non-ecc ram, not only will bad ram silently corrupt your data, and sometimes crash your server, but there will be no indicator (other than random crashes) that the problem is in fact bad ram. You can get away with not having ecc in large server farms if you have lots of checksums on generated data... but in a VPS type situation where people get pissed if you loose a single server, it's sure to end in tears.
  - ableal 16y ago
    James Hamilton (the Amazon VP linked to in another comment) also mentions lack of ECC in his appreciation of SeaMicro; he links to his own take last year, referring to studies with numbers: "You Really do Need ECC Memory in Servers" http://perspectives.mvdirona.com/2009/10/07/YouReallyDONeedECCMemory.aspx http://perspectives.mvdirona.com/2009/10/07/YouReallyDONeedE...
lsc 16y ago
hm. the price data would show it's not more efficient in terms of capital. $140K, right? before disk? for 1024GiB ram and 512 CPUs. with my current setup, I end up paying about $1900 for 32GiB ram and 8 (/much/ more powerful) CPUs before disk. So, I'd need 32 of those puppies, at a much more reasonable $60,800 - it would give me half as many CPUs, but, uh, from my experience a atom core is less than half a opteron core. how much power would that eat? in my setup, we'd be talking about 3840 watts, each one of those single socket 8 core boxes eats about one amp of 120v, give or take. they say 4w per server, so that'd be 2048 watts for their solution; so yeah, it will take quite a while to make up the price difference. Now there are some reasons why having your own physical server is better than having a virtual slice, but many of those reasons have to do with I/O, and as these boxes don't come with local storage... so at least for my workloads, they still can't beat the nice, standard opteron G34 systems I throw together in my workshop.
- hga 16y ago
  You can put up to 64 hard disks/SSDs in the front. If your power consumption figures are right, it would take 46 years to make up the difference at 10 cents/kW h. As I recall, you'd divide that by half for the cost of cooling, and then you'd have to figure in the lower capital costs of requiring 1/2 the cooling. Also possibly subtract some network hardware. The box provides "Up to 64 1 Gbps or 16 10 Gbps uplinks". And it may be more "server grade" than your setup, it integrates load balancing and server management; at the very least, the hotswap granularity is smaller. That still sounds off, except for companies that are running out of machine room space (the type of premium you pay for a laptop). How sure are you of your alternative's power consumption? I.e. have you hooked one up to a Kill-A-Watt meter? They claim 1/4 of the power of "today's best in class volume server". I think we're obsessing too much on the "$139,000" figure, if you look at http://www.seamicro.com/?q=node/38 http://www.seamicro.com/?q=node/38 you'll see there are so many options (1 or 2 GB per CPU, Ethernet and storage cards, possibly/likely some cost for the optional hot swap disk physical infrastructure, etc.). To make a good comparison you'd need to figure out and price a system including load balancing and network hardware, measure it for real and contact SeaMicro to find out the real price for a comparable system from them. Plus until they work through their backlog they won't be too eager to discount....

lsc 16y ago

ECC is a huge thing in that case. The thing about being a VPS provider is that if your hardware chokes, your customers notice, and they get pissed. Sure, if you are running some kind of web farm you set it up so that things keep running without interruption if you lose a server. But if you provide VPS hosting and you lose a server? People notice, and they will be pissed. If you are a VPS provider and you loose customer data? they will parade your severed head around town on the end of a pike. Amazon has successfully tackled this problem by training their customers that they can shoot a server at any time. This is good for them, but it's not how VPSs have traditionally operated. customers expect the thing to stay up and for their data to stay safe.

bnoordhuis 16y ago
I've seen ECC mentioned a few times in this thread but how relevant is it really? At my old job we used both regular COTS hardware from Dell and HP and high-performance gear from SUN, often at four times the price. The latter had ECC memory and, if memory serves, shielded CPUs but I never noticed much difference in stability or performance when compared to regular hardware. If someone has some numbers on this, I'd love to see them.
- lsc 16y ago
  if your dell and HP kit was server grade, it almost certainly had ECC ram. before nehalam, it was almost impossible to get dual-cpu servers that would support non-ecc ram. Even today, now that unbuffered ram is supported (buffering and ECC are not the same thing, but it's unusual to see registered or buffered ram that is not also ECC) nearly all server hardware you get from the likes of dell or HP is going to default to using ECC ram. The big deal with ECC is that with non-ecc ram, you don't know when in-ram data was corrupted. If you have a bad bit of ram that is being used for our journaling/disk caching, it's pretty easy to corrupt your data. If you've got ECC, it corrects single bit errors, and logs that there was a problem, and it can be configured to crash on double bit errors, rather than just silently corrupting your data. If you use non-ecc ram, not only will bad ram silently corrupt your data, and sometimes crash your server, but there will be no indicator (other than random crashes) that the problem is in fact bad ram. You can get away with not having ecc in large server farms if you have lots of checksums on generated data... but in a VPS type situation where people get pissed if you loose a single server, it's sure to end in tears.
- ableal 16y ago
  James Hamilton (the Amazon VP linked to in another comment) also mentions lack of ECC in his appreciation of SeaMicro; he links to his own take last year, referring to studies with numbers: "You Really do Need ECC Memory in Servers" http://perspectives.mvdirona.com/2009/10/07/YouReallyDONeedECCMemory.aspx http://perspectives.mvdirona.com/2009/10/07/YouReallyDONeedE...

lsc 16y ago

hm. the price data would show it's not more efficient in terms of capital. $140K, right? before disk? for 1024GiB ram and 512 CPUs. with my current setup, I end up paying about $1900 for 32GiB ram and 8 (/much/ more powerful) CPUs before disk. So, I'd need 32 of those puppies, at a much more reasonable $60,800 - it would give me half as many CPUs, but, uh, from my experience a atom core is less than half a opteron core. how much power would that eat? in my setup, we'd be talking about 3840 watts, each one of those single socket 8 core boxes eats about one amp of 120v, give or take. they say 4w per server, so that'd be 2048 watts for their solution; so yeah, it will take quite a while to make up the price difference. Now there are some reasons why having your own physical server is better than having a virtual slice, but many of those reasons have to do with I/O, and as these boxes don't come with local storage... so at least for my workloads, they still can't beat the nice, standard opteron G34 systems I throw together in my workshop.

hga 16y ago
You can put up to 64 hard disks/SSDs in the front. If your power consumption figures are right, it would take 46 years to make up the difference at 10 cents/kW h. As I recall, you'd divide that by half for the cost of cooling, and then you'd have to figure in the lower capital costs of requiring 1/2 the cooling. Also possibly subtract some network hardware. The box provides "Up to 64 1 Gbps or 16 10 Gbps uplinks". And it may be more "server grade" than your setup, it integrates load balancing and server management; at the very least, the hotswap granularity is smaller. That still sounds off, except for companies that are running out of machine room space (the type of premium you pay for a laptop). How sure are you of your alternative's power consumption? I.e. have you hooked one up to a Kill-A-Watt meter? They claim 1/4 of the power of "today's best in class volume server". I think we're obsessing too much on the "$139,000" figure, if you look at http://www.seamicro.com/?q=node/38 http://www.seamicro.com/?q=node/38 you'll see there are so many options (1 or 2 GB per CPU, Ethernet and storage cards, possibly/likely some cost for the optional hot swap disk physical infrastructure, etc.). To make a good comparison you'd need to figure out and price a system including load balancing and network hardware, measure it for real and contact SeaMicro to find out the real price for a comparable system from them. Plus until they work through their backlog they won't be too eager to discount....