5 ms·
I know it's not exactly in vogue these days to tout the merits of bare hardware, but.. after all the VPS hubbub over the last couple of years, the best progress
by jamwt 16y ago
I know it's not exactly in vogue these days to tout the merits of bare hardware, but.. after all the VPS hubbub over the last couple of years, the best progression for your website still seems to be:
1. No traction? Just put it anywhere, 'cause frankly, it doesn't matter. Cheapest reputable VPS possible. Let's say, Linode.
2. Scaling out, high concurrency and rapid growth? DEDICATED hardware from a QUALITY service provider--use rackspace, softlayer et al. Have them rack the servers for you and you'll still get ~3 hour turnarounds on new server orders. That's plenty fast for most kinds of growth. No inventory to deal with, and with deployment automation you're really not doing much "sysadmin-y" work or requiring full timers that know what Cisco switch to buy.
3. Technology megacorp, top-100 site? Staff up on hardcore net admin and sysadmin types, colocate first, and eventually, take control of/design the entire datacenter.
I simply don't understand why so many of these high-traffic services continue to rely on VPSes for phase 2 instead of managed or unmanaged dedicated hosting. The price/concurrent user is competitive or cheaper for bare metal. Most critically, it's insanely hard to predictably scale out database systems with high write loads when you have unpredictable virtualized (or even networked) I/O performance on your nodes.
- jedberg 16y agoreddit actually is a top 100 site, but we don't have nearly the need to host our own datacenter or co-locate. If we do make a move, it will be to #2. I don't want to hire people to be hands on -- I'd rather outsource that and let someone else pay to have spare capacity laying around.
- jamwt 16y agoFair enough; 100 was sort of arbitrary, but there is some metric when you're sort of at google scale and cutting down on power costs or achieving absolute minimum latencies between datacenters has a meaningful impact on the bottom line. Draw a line somewhere, and yeah, Reddit is probably on the other side.
- phire 16y agoReddit has always amazed me with what they can do with extremely limited resources. But it looks like that attitude has finally caught up with them, especially since they are down to just 3 technical staff (from 5 last week), and two of them are brand new.
- jedberg 16y agoHave some faith. We'll pull through. :)
- lsc 16y agowhat kind of scale are you at? I mean, about how many 32GiB ram/8 core servers would you need if you were using real hardware?
- jedberg 16y agoWe have ~130 servers at Amazon right now. We could probably do it with 50-75 or less, depending on how big the boxes are.
- cagenut 16y agoconde would very easily do this for you, they built an entire datacenter in deleware just for their child-companies to use like this.
- jedberg 16y agoYes, they could. It is an option.
- brk 16y agoGenerally with you, except for the Rackspace recommendation, been burned and pissed off by Rackspace too many times to ever use or recommend them again. I tend to try to find at least 1 good local/regional datacenter for a good portion of a server stack. There is huge benefits (IMO) in being able to drive to your servers and have a face to face meeting if there are issues and/or take your toys and go someplace else if there is a massive outage. If you're in Green Bay, options might be limited, but in any semi-major metropolitan area there are usually enough datacenter options that you have multiple choices.
- reiddraper 16y agoAnother thought. If availability is your goal, with the trend toward 'operations as code', I think a small development team can build a system on top of AWS that can automatically respond to arbitrary node/resource/data-center failure. Netflix seems to do this to an extent, with their Chaos Monkey. That being said, there are situations where you may truly need single-node performance that isn't available on AWS.
- jedberg 16y agoChaos Monkey causes chaos, it does not fix it. :) But yes, you are right. The goal is to have a system that scales itself. Not an easy task for sure.