6 ms·
> If you are using a compute optimized AWS c4.4xlarge instance: you will be able to purchase the Intel Xeon D-1587 system for about the same price as the “Parti
by matt_wulfeck 11y ago
> If you are using a compute optimized AWS c4.4xlarge instance: you will be able to purchase the Intel Xeon D-1587 system for about the same price as the “Partial Upfront” up front AWS fee, then colocate the box saving over $1500/ year per instance while getting better performance.
I hear this type of argument a lot, but it's so important to include the cost in engineering and talent necessary to keep your datacenter humming.
If you're a small shop with a small future growth expectation then sure, forget Amazon and start racking your own boxes. Just be ready to hire ops staff competent to run your operation. You need to be realistic about both aspects of cost.
- Sanddancer 11y agoOne can always do a hybrid approach, where baseline load is handled on purchased machines, but with renting AWS time on an as-needed basis. Also, ops staff is useful, and often has skillsets that augment development. Letting someone else worry about building daemons, tuning parameters, etc, means that your devs can spend more time actually writing code. Interrupting your devs because nginx has a memory leak can be a much bigger waste of time.
- voltagex_ 11y agoI wish I had an excuse to play with one of these - I wonder what their colo costs are. I suspect you could run quite a few build boxes for all sorts of FOSS projects with a system like this.
- jjn2009 11y agoIts unfortunate they didn't give an more details on the per year costs in colo. I'm pretty interested in that metric as well and how it compares to the other intel xeon chips.
- zaroth 11y agoColo cost varies based on the facility, uplink, power density, and various other factors. So it doesn't quite make sense to talk about the "colo cost" of a specific device, or comparing colo cost of one chip versus another... unless what you're really just comparing is power efficiency. Let's say you're renting half a rack with 100Mbps unmetered bandwidth and 20amps of 120V for $500 per month. So you have 16 amps / 1920 watts of continuous power you can draw. These Xeon D's are very lower power, although strangely Pat doesn't report the actual idle and full load draw as measured by a Kill-o-Watt meter. With the 10GBE I can't imagine budgeting much less than 150 watts per machine. If you can put let's say a dozen of these in that half rack, in my contrived example, the "colo cost" would be $41.66 / month or just $2.60 / core.
- jjn2009 11y agoThe Xeon D-1540 is running 73 watts under load according to anandtech http://www.anandtech.com/show/9185/intel-xeon-d-review-performance-per-watt-server-soc-champion/16 http://www.anandtech.com/show/9185/intel-xeon-d-review-perfo... Across all the SKUs in OP the TDP is 35-65 watts. 20 More watts for the new 1587 SKU compared to the benchmarked 1540 in the anandtech article. So a rough estimate would be maybe ~95 watts at under load? Assuming the board is the same and only the CPU has changed that should be a pretty good estimate. You're right it mostly doesn't make sense to compare machines in that way due to all of the variables surrounding colo costs, max load wattage is the metric I should be interested in.
- Patrick-STH 11y agoHi zaroth - We do not use Kill-o-Watt meters since we use calibrated Extech TrueRMS units for our "desktop" testing. We also have a few racks in Sunnyvale, California that we use APC metered by outlet PDUs for and that we actually calibrated by testing with the Extech units and found two of over a dozen that gave us consistent readings across PDUs and ports. This unit is in a SC515 chassis with redundant PSUs. Even with 4x 7200 rpm Seagate 2.5" drives, a 64GB mSATA drive and a Samsung SM951 m.2 SSD it is still pulling 0.24A on 208V (so 0.48A on 208V). If you were to gain a bit of efficiency running a single PSU, you could easily fit these in 1A on 120V power envelope. What we have seen several folks do with similar D-1540 machines is actually use 1U 1A/ 120V hosting (which is very inexpensive) to deliver cheap local PoPs for their applications. We have been strongly considering decommissioning our Las Vegas 1/2 cab DR site and moving to this sort of distributed model as there are a lot of benefits with this. We published more complete benchmark results ( http://www.servethehome.com/intel-xeon-d-1587-benchmarks-16-core-xeon-d-is-here/ http://www.servethehome.com/intel-xeon-d-1587-benchmarks-16-... )of the D-1587 yesterday, expect more in terms of reviewing the overall platform next week. The Xeon D is very platform dependent for power and the particular unit we have does have the additional LSI/ Avago/ Broadcom SAS 2116 (16 port SAS2) HBA onboard.
- zaroth 11y agoThanks Pat! Long-time reader of STH, I really find your articles incredibly useful and informative! I assume you meant to write "(so .48A on 120V)". 50W idle with 4 spinning drives, an M.2, and the redundant PSU really is really very impressive. As you say, keeping full load under 120 watts to be able to fit in a 1U/1amp colo gives an incredible value. The hardware's not even that expensive! And for $50 - $60 / month in colo cost [1], the reliability of systems now with SSDs, and the amazing orchestration tools that are available, the likes of AWS, DigitalOcean, etc. start to look really expensive. [1] http://www.webhostingtalk.com/forumdisplay.php?f=131 http://www.webhostingtalk.com/forumdisplay.php?f=131
- detaro 11y agoThey talk about their colo stuff quite a bit, with prices here and there inbetween: http://www.servethehome.com/going-colo-series-part-1-deciding-colocation/ http://www.servethehome.com/going-colo-series-part-1-decidin... http://www.servethehome.com/colo-series-part-2-picking-colocation-site/ http://www.servethehome.com/colo-series-part-2-picking-coloc... http://www.servethehome.com/colocation-years-learning-experience/ http://www.servethehome.com/colocation-years-learning-experi...
- ChuckMcM 11y agoHaving done this computation a number of times, with these machines it would flip over to 'colo' at about 150 machines. The really interesting thing is that amazon costs scale linearly with size while owned infrastructure scale fractionally. So the advantage just keeps growing and growing. Bottom line, if you can host your app and resources on less than a 150 machines in Amazon you win, more than that and you're leaving money on the table. Once you get to the point where you are deploying new datacenters to support your customers you get a huge boost in operational efficiency.
- ciupicri 11y agoYet Netflix migrated to Amazon and I think that it has way more than 150 machines.
- rhizome 11y agoI have to think they're getting a deal.
- kevcampb 11y agoWhich should be expected when buying in bulk.
- BorisMelnik 11y agothat would be fun to calculate
- vosper 11y agoPlus, they're the number one poster-child for AWS. Netflix running on AWS makes AWS safe for CTOs everywhere. So, yeah, they're definitely getting a deal.
- ChuckMcM 11y agoYes they did. And that is a really great exercise in thinking about what their profitability would be like if they were structured more like YouTube in terms of their server infrastructure. But to the point of many other folks here, if you're all cloud you can really shrink fast if you need too. Lose a millions customers? No problem, just drop some machines. Any business that can price into the monthly customer fee the marginal monthly cost of AWS to support that customer, can keep their costs flat when their customer counts vary. Someone with dedicated hardware is still paying the bills for it, even when their customers leave.
- seanwilson 11y ago> I hear this type of argument a lot, but it's so important to include the cost in engineering and talent necessary to keep your datacenter humming. I'm surprised how often I see arguments like this as well. If it takes even a few days of time for your employees to manage the hardware and you're only saving a few $100 a year buying the hardware yourself, you're already making a loss.
- kayoone 11y agoImo it is not that easy. Amazon AWS does not manage itself, so you most probably need a Dev Ops guy anyway to not waste the time of your developers with configuring environments and keeping them running. I would say that the hardware part of devops is way less when you rent boxes and basically run them stateless.
- seanwilson 11y agoI'm not saying it's easy but it's easier. Different cloud services let someone else take care of replacing faulty hardware, upgrading hardware, adding new servers, software updates, backups etc. It's very difficult and time consuming to replicate all this reliably with a couple of dev ops guys.
- kayoone 11y agoIf you rent dedicated boxes you don't have to worry about replacing hardware or adding new servers. Also the cost savings are so huge, as someone else stated a dedicated box could be 1/7 the monthly price of an equally specced AWS instance you can easily have a 1-2 devops guys on call and still save a lot of money. I don't see where buying/colo makes sense though, maybe if you have a really huge operation going on. Hetzner offers a i7 Skylake QuadCore 6700, 64GB Ram, 512GB SSD Raid1 for 60 EUR/month, an equally specced AWS instance (or multiple smaller ones) would easily be >400 EUR even when using upfront payment and reserved instances. That difference is huge if we are talking about 10-50 of these machines. The only benefit i see is when you can scale your app up and down a lot during the day to save costs, but i'd still like to see a real world case where that actually saves money. I am pretty sure most companies use AWS for the convienence and just don't care about the cost because it's minor in the grand scheme of things, like super popular startups with huge funding and years to figure out and optimize profitability.