15 ms·

Except that none of these compute resources are free and the cost of cloud-based super computing is a complete rip-off. It's actually thousands of dollars cheap

Except that none of these compute resources are free and the cost of cloud-based super computing is a complete rip-off. It's actually thousands of dollars cheaper to build your own super computer from used rack mount servers than it is to use AWS, Azure, or Google cloud for lengthy computations. As an example: you can buy a Dell PowerEdge M915 from ebay with 128 cores for ~$500 USD and a rack costs around the same. Five of them is 640 cores for a total cost of just 3k USD. That's 640 cores that you now own forever. Guess how much it would cost to use that many cores for a month on AWS? Well over 10k... and next month its another 10k... and so is next month... With this option you only pay for power and still have the resources for all future experiments. I think the m1000e rack can fit 8 blades in total so you could upgrade to a max 1024 cores in the future! The downside with this particular rack is that it's very, very loud. But I've run the numbers here and it's hard to beat high-core, used rack servers on a $/Ghz basis.

bpodgursky 6y ago
None of this math is correct. You absolutely cannot get 128 cores for $500. Please link to a single valid-looking eBay offer at that amount (you may have seen 128GB and thought it was 128 cores). I see 32 cores for $500, at best. Second, 640 cores on an m4.large for 3 years upfront (the equivalent purchase) is $8755/mo, not well over 10k. Third, you really underestimate electricity costs if you're running a server 24/7 at all cores. Last, and most importantly, most people simply don't run servers 24/7. They run batch computations for five hours a week, or a day, and then spend a week or two crunching the numbers. There's some case to make that for certain institutions with very particular compute needs, running on-premise might make short-term financial sense (let's not even get started on labor costs). But it's really inaccurate to call cloud computing a "rip-off" for anything but the niche-est customers.
- sigstoat 6y ago
  > Third, you really underestimate electricity costs if you're running a server 24/7 at all cores. not to mention the cooling, or the per square foot cost of renting the space for this hardware. anything close to what's being described here is going to need some sound deadening (be it distance or material) from humans.
  - EB66 6y ago
    Frankly, I think many people here are overestimating the cost of electricity and cooling. My company leases several 42U cabinets with 208V 30A circuits in a modern SOC 2 Type 2 audited datacenter. We pay $600 per month per cabinet. Excludes IP transit costs. Granted this is in the Pacific Northwest where we have plentiful cheap hydro power.
    - rumanator 6y ago
      > We pay $600 per month per cabinet. Still a world of difference between AWS's 10k/month cost to access that many cores.
      - maccard 6y ago
        Comparing a Dell r740 (56 cores, 192GB ram, 1.6 kw psu), you need 11 of these for 640 cores (for a comparison with [0]). They're roughly 2500 each, so your up front hardware costs are 27,500. Assuming the $600 covers power, rack costs and cooling, your data center bill is 6600/month. For three years, the AWS bill is $315k, and the self hosted option is $265,100 (best case). I'm not up to speed with the pricing trends to say whether the gap will change over those 3 years based on data center changes or aws changes, but the difference is ~50k for three years, or roughly 3 months of an engineers salary over the cost of three months. [0] https://news.ycombinator.com/item?id=23514698 https://news.ycombinator.com/item?id=23514698
        namibj 6y ago
        Did you consider that AWS counts threads, not cores?
        maccard 6y ago
        No, I did napkin math that compared an unverified number farther up the thread to the first server I found on Google. If you're considering buying hardware you should do your own numbers, I was merely pointing out that difference isn't 10k/month vs 27k up front
- carlosf 6y ago
  On a side note, you would expect people to be super excited to explore possibilities of new infrastructure models. Instead, most posts related to lambda, etc... always come with this sort of reply. For my use case (infrequent, easily separable jobs) FaaS stuff is a godsend. In the last year I've been doing things that would be orders of magnitude harder and more expensive thanks to it.
- nullc 6y ago
  > I see 32 cores for $500, at best. Second, 640 cores on an m4.large for 3 years upfront (the equivalent purchase) is $8755/mo, You can beat that cores per dollar handedly that if you're willing to use sufficiently odd hardware. E.g. my Intel cluster here is mostly I have systems with 4x E7-8880v4 (22c@2.2GHz). I paid $150/chip + $450 host (thanks ebay) excluding ram. (I have 8 such hosts now, so 704 cores) So essentially one months cost for your AWS price w/ a 3year commit pays for an equivalent core count in systems for me... Throw in another month for the ram. Round up to another another to count for the (quite) non-trivial power usage. You do not have to be anywhere near 100% utilization for AWS to be an extremely poor value large computing jobs compared to thrifty spending on the surplus market.
  - what-the-grump 6y ago
    how? >I paid $150/chip + $450 host (thanks ebay) excluding ram. E7-8880v4 are $600 used each. Quad core mobos are not cheap, ranked ram is not cheap. Even if you pulled these from a vendor going out of business the bit coin mining crews would suck these up in heartbeat. Where are you getting your prices?
    - nullc 6y ago
      Ebay. Hunt harder (and best offer harder). :) E7 CPUs sell absurdly cheap at times because it can be a bit hard to find compatible systems. (similarly, quad socket systems with slow or no cpus go cheap from people that don't know much about them) > ranked ram is not cheap. Yes, ram for quad socket hosts it adds up, since you need at least 2-8 dimms per socket for full memory bandwidth (which was why I explicitly pointed out that I left it out). On the plus side, those systems tend to have a lot of dimm sockets so you can use lower capacity dimms. > bit coin mining crews nah, not useful for that.
deleted 6y ago
[deleted]
cortesoft 6y ago
Where are you going to store this thing? Pay for the power? Cool it? Replace failing hardware? As someone who works for a company with over 150 data centers around the world, I know space and power is always one of our biggest expenses.
- threeseed 6y ago
  You can just store it at the datacenter. Many of them offer a service which allows you to drop off your hardware and they will manage the power, cooling, even repairs and replacement for a monthly fee.
  - chrisseaton 6y ago
    > You can just store it at the datacenter. What do you think is ‘the’ data centre? There’s no single universal data centre. And most people don’t have a data centre, or access to one.
    - threeseed 6y ago
      Actually everyone has access to managed data centre services. It’s called co-location where you buy rack space in one of the core data centre hubs in each city. Been available for many years and it’s very affordable e.g. a few hundred a month.
      - chrisseaton 6y ago
        Not sure why you said 'the data centre' rather than 'a data centre' though. Not sure if you think there's one massive data centre for the whole world?
        vladvasiliu 6y ago
        I think GP said "the datacenter" rather than "a datacenter" in the same way they would say they have a computer at "the office" and not at "an office". Even though there's no "massive office for the whole world".
jungturk 6y ago
Are any compute resources free? It's telling that you've neglected your own labor in this assessment, and that of the other specialists required to get such ad-hoc compute working at scale (dynamic software networks, on-demand compute, provisioning/billing/monitoring apparatuses, etc...) What plenty have realized in cloud-native approaches is that the TCO is pretty compelling, sometimes because its easier to distribute procurement/provisioning/billing to teams for their own resource needs, or just in sidestepping the inevitable bureaucracy when you try to centralize that decision-making. Of course its different if you're talking hobby kit setups in our own homes, but if you're aiming at real scale for absurd parallelisms then that ship has sailed.
- joshuaellinger 6y ago
  I'm in the midst of returning from the cloud to coloc. It's looking like for about $50K in hardware I get x10 the ram/compute/storage/(internal) network I get from the cloud at around $10K/mo. However, it is taking me about $25K in labor to setup. Hosting costs about $1K for a rack and a decent pipe to the server. So more than a hobby but not approaching real scale. The main barrier to using a lambda model is if you have anything that smacks of a DB. It would take me $100K+ to transition away from SQL. If you can do lambda w/o a big retool cost, then it is probably viable. If you are just running a bunch of VMs on the cloud, it is pretty expensive. ... Interestingly, my main client (a very large company) just went the other direction and moved all its compute to a system that is Spark underneath running on Azure. They are trying to decommission some expensive TerraData instances. So far, it is a mixed bag -- it is a big step forward (for them) on anything that is 'batch-oriented' but 'interactive' performance is dismal. Price appears to be a big motivation. I always forget that most large enterprises run on exotic stuff with crazy service contracts that makes the cloud look cheap.
  - user5994461 6y ago
    > Price appears to be a big motivation. I always forget that most large enterprises run on exotic stuff with crazy service contracts that makes the cloud look cheap. Don't forget the second factor of (not having) the workforce. Physical servers require a bunch of suckers who know physical infrastructure and accept to be oncall 24/7 and deal with DELL/HP/colo full time on top of periodic travels to the datacenter.
p1necone 6y ago
You can't really compare value just by looking at core count, I imagine an old 128 core server off of eBay is going to perform much differently from something running on a more modern architecture.
- rumanator 6y ago
  However AWS lambda stuff runs on a pile of abstraction layers that have an impact on performance, not to mention their need to do cold starts.
jariel 6y ago
" It's actually thousands of dollars cheaper to build your own super computer from used rack mount servers than it is to use AWS, Azure, or Google cloud for lengthy computations." The issue is 'total cost of ownership' - not 'unit cost of equipment'. The operational cost and overhead of running your own gear can be prohibitive. AWS is successful because it is in fact much cheaper than the alternative in many enterprise situations.
- rightbyte 6y ago
  AWS is sucsessfull becouse there is no capital tied to hardware for the leasers. For some reason it is what executives prefer. A 1000USD server is in practice impossible for me to order at work but I can use multiple 1000s a year in leased servers...
  - jariel 6y ago
    "AWS is sucsessfull becouse there is no capital tied to hardware for the leasers. For some reason it is what executives prefer." Your execs are probably right. Capital outlays are expensive and risky and imply a structural lock-in - but that's just the tip of the iceberg. "A 1000USD server is in practice impossible for me to order at work" And what about hosting? Networking gear? Support? Repairs and upgrades? Networking security? And how will those servers integrate with the rest of your outlay? AWS is ridiculously cheap compared to the alternatives in most scenarios and that's why it's so successful. AWS/Cloud enables so much more, far more dynamically - the value is considerable. In some situations, if you have a fairly big need for computing, and it's predictable over long time-frames, and those services don't need to be tightly integrated with other cloud services, and you have the internal know-how to keep them running, the cost obviously savings can be achieved, but this is an optimization. Put on your Eng/Ops/Business Hat for a minute and consider why services like AWS are exploding and growing to be one of the biggest segments in tech? Because 'stupid executives'? No - it's because the value add is fairly immense.
    - vidarh 6y ago
      > AWS is ridiculously cheap compared to the alternatives in most scenarios and that's why it's so successful. It's so successful because a lot of engineers selecting what to rent never see the invoices and don't understand the costs of colo or managed servers. You see it all over this discussion e.g. with people assuming a rack or two requires full time staff when you can get sufficient on-call resources on retainer for a few hundred a month or so depending on complexity of the setup most places. Having ordered, configured, set up, hired staff for and run both colo setups, managed servers, and AWS setups many times I've yet to see AWS be remotely competitive on price ever, to the point that when I did contracting I used to offer clients to transition their systems to managed hosting (whether managed or colo comes out cheapest depends on scale and location - e.g. I'm in London and real-estate prices here are too high for colo to typically beat managed servers if you can deal with ~8ms round-trip to providers in Germany or France; for others managed is necessary to be close to customers) for a percentage of their first 3-6 months of reduced cost. That was costs including their devops contracts and monitoring etc. I use AWS. It has lots of great features, and sometimes those features are worth the cost, but it's the expensive luxury option of hosting. > Put on your Eng/Ops/Business Hat for a minute and consider why services like AWS are exploding and growing to be one of the biggest segments in tech? Because 'stupid executives'? No - it's because the value add is fairly immense. I've done this for 25 years, including private cloud setups from before AWS was a thing, and I've had to yell at execs that wanted to triple our monthly costs because AWS sales had whispered buzzwords into their ears. That was the all in costs. It took a massive effort to explain it to them even with the numbers in black and white in front of their faces. So, yeah, a lot of the time (not always), it is "stupid" executives. Sometimes talked into it by engineers working around planning processes that makes paying by use an easy end-run around budgeting. The biggest achievement of AWS is to sell the idea that it is cheap because Amazon.
jayd16 6y ago
The way I look at this is you still need 3 people minimum on call to have around the clock ops for this rack. Three IT salaries is more than $10k a month. Eventually you'll find the balancing point but its higher than you think.
- afwaller 6y ago
  Yes. This is the reason cloud is popular. You don’t have to pay for computer janitors.
  - aea65 6y ago
    We have a title and don’t carry mops, my dear condescending developer who ostensibly doesn’t respect the entire ecosystem of people holding the Internet, the public cloud, and your entire CI/CD pipeline for whatever bullshit you’re cooking in NPM together. Do you have any idea how many “janitors,” most basically volunteer and most not employed by Amazon, are involved in the pedestrian aspect of merely making amazon.com land at the right place? Yes, this is a good future: who would ever want to touch the computers they operate? Better to rent computers and position the core competency of your business with a third party, right? Because you can shed some of those pesky janitorial salaries? I’ll be waiting by my phone when you’re on the verge of bankruptcy once the clouds have their teeth in your books, and suddenly I get promoted from “janitor” to “computer operator who has my interests in mind, even though I not-so-subtly malign his existence”. All you’ve done with this mindset is drive the same janitors to work for the clouds instead and contributed to the downfall of computing as a discipline that any player has any semblance of agency within, as the people who have actually touched datacenter equipment all work for them now or sit around in horror watching as a generation sympathetic to FLOSS arguments willingly hands over the reins of owning a computer to massive corporations.
    - louwrentius 6y ago
      Reading the comments in this thread already shows how much of 'those' 'software janitors'[1] don't understand what is required (and how little) to run on metal. It's not that cloud is not the right answer. But people have started to forget that running your own metal is still an option. Or with current prices and performance: even a more viable option as it was in the past, because you can do so much more with so much less. [1]: cranking out useless features nobody ask for while looking down on those (dev)ops people.
coldtea 6y ago
>Except that none of these compute resources are free and the cost of cloud-based super computing is a complete rip-off. It's actually thousands of dollars cheaper to build your own super computer Only if your time is worthless
csdreamer7 6y ago
> you can buy a Dell PowerEdge M915 from ebay with 128 cores for ~$500 USD Where? Was not able to find one. Found a 32 core for $750. It was a pre-Epyc AMD server which has it's own problems. There were only 9 options listed for the exact term you gave 'Dell PowerEdge M915' for the USA before I got to pricey international sellers. The few options make me feel skeptical about this. This is of course before I even consider security and performance issues of used Intel servers.
- cambalache 6y ago
  Every time someone says on the Internet "You can buy x thing in Ebay for Y price" multiply the price by 3.
louwrentius 6y ago
You are right. And if you buy metal and rent some space in a DC, it is still more cost-effective. You need a few people who understand a little bit of hardware and IPMI and you are set. That's not a big deal. The cloud gaslighting is nuts. Tons of companies now exist to give you insight in your cloud bill because that's the last thing cloud provides want you to have. So in the end you must decide what kind of pain you want. Cost pain, tech pain, and that all depends on your company's particular circumstances.
6gvONxR4sf7o 6y ago
You also have to account for burstiness. If you have nightly/weekly workflows, you might see some wild swings in resource usage, where an average usage and max usage differ by a lot. It’s nice to pay for average needs rather than max needs (especially when you can pay for on demand pricing).
garethmcc 6y ago
You forget a few things: 1. AWS and other cloud vendors don't have just one data centre per region, they usually have three redundant locations for availability reasons connected via a direct fibre link. 2. Cost of time; time to find hardware, prep hardware, maintain hardware 3. Cost of your agility. If you need more compute capacity, it will take you weeks to get the hardware required and installed, otherwise you need to have unused hardware sitting around "just in case". 4. Cost of your availability. What if you have a sudden spike of traffic within minutes and the current available hardware cannot service it? At least in the cloud you can spin up short lived resources to manage that load before it throttles back down again. 5. Permanent running costs of fixed hardware. A lot of implementations do not need permanent hardware running and can throttle down to a base of almost nothing so the average running cost on a monthly basis is actually very low.

bpodgursky 6y ago

None of this math is correct. You absolutely cannot get 128 cores for $500. Please link to a single valid-looking eBay offer at that amount (you may have seen 128GB and thought it was 128 cores). I see 32 cores for $500, at best. Second, 640 cores on an m4.large for 3 years upfront (the equivalent purchase) is $8755/mo, not well over 10k. Third, you really underestimate electricity costs if you're running a server 24/7 at all cores. Last, and most importantly, most people simply don't run servers 24/7. They run batch computations for five hours a week, or a day, and then spend a week or two crunching the numbers. There's some case to make that for certain institutions with very particular compute needs, running on-premise might make short-term financial sense (let's not even get started on labor costs). But it's really inaccurate to call cloud computing a "rip-off" for anything but the niche-est customers.

sigstoat 6y ago
> Third, you really underestimate electricity costs if you're running a server 24/7 at all cores. not to mention the cooling, or the per square foot cost of renting the space for this hardware. anything close to what's being described here is going to need some sound deadening (be it distance or material) from humans.
- EB66 6y ago
  Frankly, I think many people here are overestimating the cost of electricity and cooling. My company leases several 42U cabinets with 208V 30A circuits in a modern SOC 2 Type 2 audited datacenter. We pay $600 per month per cabinet. Excludes IP transit costs. Granted this is in the Pacific Northwest where we have plentiful cheap hydro power.
  - rumanator 6y ago
    > We pay $600 per month per cabinet. Still a world of difference between AWS's 10k/month cost to access that many cores.
    - maccard 6y ago
      Comparing a Dell r740 (56 cores, 192GB ram, 1.6 kw psu), you need 11 of these for 640 cores (for a comparison with [0]). They're roughly 2500 each, so your up front hardware costs are 27,500. Assuming the $600 covers power, rack costs and cooling, your data center bill is 6600/month. For three years, the AWS bill is $315k, and the self hosted option is $265,100 (best case). I'm not up to speed with the pricing trends to say whether the gap will change over those 3 years based on data center changes or aws changes, but the difference is ~50k for three years, or roughly 3 months of an engineers salary over the cost of three months. [0] https://news.ycombinator.com/item?id=23514698 https://news.ycombinator.com/item?id=23514698
      - namibj 6y ago
        Did you consider that AWS counts threads, not cores?
        maccard 6y ago
        No, I did napkin math that compared an unverified number farther up the thread to the first server I found on Google. If you're considering buying hardware you should do your own numbers, I was merely pointing out that difference isn't 10k/month vs 27k up front
carlosf 6y ago
On a side note, you would expect people to be super excited to explore possibilities of new infrastructure models. Instead, most posts related to lambda, etc... always come with this sort of reply. For my use case (infrequent, easily separable jobs) FaaS stuff is a godsend. In the last year I've been doing things that would be orders of magnitude harder and more expensive thanks to it.
nullc 6y ago
> I see 32 cores for $500, at best. Second, 640 cores on an m4.large for 3 years upfront (the equivalent purchase) is $8755/mo, You can beat that cores per dollar handedly that if you're willing to use sufficiently odd hardware. E.g. my Intel cluster here is mostly I have systems with 4x E7-8880v4 (22c@2.2GHz). I paid $150/chip + $450 host (thanks ebay) excluding ram. (I have 8 such hosts now, so 704 cores) So essentially one months cost for your AWS price w/ a 3year commit pays for an equivalent core count in systems for me... Throw in another month for the ram. Round up to another another to count for the (quite) non-trivial power usage. You do not have to be anywhere near 100% utilization for AWS to be an extremely poor value large computing jobs compared to thrifty spending on the surplus market.
- what-the-grump 6y ago
  how? >I paid $150/chip + $450 host (thanks ebay) excluding ram. E7-8880v4 are $600 used each. Quad core mobos are not cheap, ranked ram is not cheap. Even if you pulled these from a vendor going out of business the bit coin mining crews would suck these up in heartbeat. Where are you getting your prices?
  - nullc 6y ago
    Ebay. Hunt harder (and best offer harder). :) E7 CPUs sell absurdly cheap at times because it can be a bit hard to find compatible systems. (similarly, quad socket systems with slow or no cpus go cheap from people that don't know much about them) > ranked ram is not cheap. Yes, ram for quad socket hosts it adds up, since you need at least 2-8 dimms per socket for full memory bandwidth (which was why I explicitly pointed out that I left it out). On the plus side, those systems tend to have a lot of dimm sockets so you can use lower capacity dimms. > bit coin mining crews nah, not useful for that.

deleted 6y ago

[deleted]

cortesoft 6y ago

Where are you going to store this thing? Pay for the power? Cool it? Replace failing hardware? As someone who works for a company with over 150 data centers around the world, I know space and power is always one of our biggest expenses.

threeseed 6y ago
You can just store it at the datacenter. Many of them offer a service which allows you to drop off your hardware and they will manage the power, cooling, even repairs and replacement for a monthly fee.
- chrisseaton 6y ago
  > You can just store it at the datacenter. What do you think is ‘the’ data centre? There’s no single universal data centre. And most people don’t have a data centre, or access to one.
  - threeseed 6y ago
    Actually everyone has access to managed data centre services. It’s called co-location where you buy rack space in one of the core data centre hubs in each city. Been available for many years and it’s very affordable e.g. a few hundred a month.
    - chrisseaton 6y ago
      Not sure why you said 'the data centre' rather than 'a data centre' though. Not sure if you think there's one massive data centre for the whole world?
      - vladvasiliu 6y ago
        I think GP said "the datacenter" rather than "a datacenter" in the same way they would say they have a computer at "the office" and not at "an office". Even though there's no "massive office for the whole world".

jungturk 6y ago

Are any compute resources free? It's telling that you've neglected your own labor in this assessment, and that of the other specialists required to get such ad-hoc compute working at scale (dynamic software networks, on-demand compute, provisioning/billing/monitoring apparatuses, etc...) What plenty have realized in cloud-native approaches is that the TCO is pretty compelling, sometimes because its easier to distribute procurement/provisioning/billing to teams for their own resource needs, or just in sidestepping the inevitable bureaucracy when you try to centralize that decision-making. Of course its different if you're talking hobby kit setups in our own homes, but if you're aiming at real scale for absurd parallelisms then that ship has sailed.

joshuaellinger 6y ago
I'm in the midst of returning from the cloud to coloc. It's looking like for about $50K in hardware I get x10 the ram/compute/storage/(internal) network I get from the cloud at around $10K/mo. However, it is taking me about $25K in labor to setup. Hosting costs about $1K for a rack and a decent pipe to the server. So more than a hobby but not approaching real scale. The main barrier to using a lambda model is if you have anything that smacks of a DB. It would take me $100K+ to transition away from SQL. If you can do lambda w/o a big retool cost, then it is probably viable. If you are just running a bunch of VMs on the cloud, it is pretty expensive. ... Interestingly, my main client (a very large company) just went the other direction and moved all its compute to a system that is Spark underneath running on Azure. They are trying to decommission some expensive TerraData instances. So far, it is a mixed bag -- it is a big step forward (for them) on anything that is 'batch-oriented' but 'interactive' performance is dismal. Price appears to be a big motivation. I always forget that most large enterprises run on exotic stuff with crazy service contracts that makes the cloud look cheap.
- user5994461 6y ago
  > Price appears to be a big motivation. I always forget that most large enterprises run on exotic stuff with crazy service contracts that makes the cloud look cheap. Don't forget the second factor of (not having) the workforce. Physical servers require a bunch of suckers who know physical infrastructure and accept to be oncall 24/7 and deal with DELL/HP/colo full time on top of periodic travels to the datacenter.

p1necone 6y ago

You can't really compare value just by looking at core count, I imagine an old 128 core server off of eBay is going to perform much differently from something running on a more modern architecture.

rumanator 6y ago
However AWS lambda stuff runs on a pile of abstraction layers that have an impact on performance, not to mention their need to do cold starts.

jariel 6y ago

" It's actually thousands of dollars cheaper to build your own super computer from used rack mount servers than it is to use AWS, Azure, or Google cloud for lengthy computations." The issue is 'total cost of ownership' - not 'unit cost of equipment'. The operational cost and overhead of running your own gear can be prohibitive. AWS is successful because it is in fact much cheaper than the alternative in many enterprise situations.

rightbyte 6y ago
AWS is sucsessfull becouse there is no capital tied to hardware for the leasers. For some reason it is what executives prefer. A 1000USD server is in practice impossible for me to order at work but I can use multiple 1000s a year in leased servers...
- jariel 6y ago
  "AWS is sucsessfull becouse there is no capital tied to hardware for the leasers. For some reason it is what executives prefer." Your execs are probably right. Capital outlays are expensive and risky and imply a structural lock-in - but that's just the tip of the iceberg. "A 1000USD server is in practice impossible for me to order at work" And what about hosting? Networking gear? Support? Repairs and upgrades? Networking security? And how will those servers integrate with the rest of your outlay? AWS is ridiculously cheap compared to the alternatives in most scenarios and that's why it's so successful. AWS/Cloud enables so much more, far more dynamically - the value is considerable. In some situations, if you have a fairly big need for computing, and it's predictable over long time-frames, and those services don't need to be tightly integrated with other cloud services, and you have the internal know-how to keep them running, the cost obviously savings can be achieved, but this is an optimization. Put on your Eng/Ops/Business Hat for a minute and consider why services like AWS are exploding and growing to be one of the biggest segments in tech? Because 'stupid executives'? No - it's because the value add is fairly immense.
  - vidarh 6y ago
    > AWS is ridiculously cheap compared to the alternatives in most scenarios and that's why it's so successful. It's so successful because a lot of engineers selecting what to rent never see the invoices and don't understand the costs of colo or managed servers. You see it all over this discussion e.g. with people assuming a rack or two requires full time staff when you can get sufficient on-call resources on retainer for a few hundred a month or so depending on complexity of the setup most places. Having ordered, configured, set up, hired staff for and run both colo setups, managed servers, and AWS setups many times I've yet to see AWS be remotely competitive on price ever, to the point that when I did contracting I used to offer clients to transition their systems to managed hosting (whether managed or colo comes out cheapest depends on scale and location - e.g. I'm in London and real-estate prices here are too high for colo to typically beat managed servers if you can deal with ~8ms round-trip to providers in Germany or France; for others managed is necessary to be close to customers) for a percentage of their first 3-6 months of reduced cost. That was costs including their devops contracts and monitoring etc. I use AWS. It has lots of great features, and sometimes those features are worth the cost, but it's the expensive luxury option of hosting. > Put on your Eng/Ops/Business Hat for a minute and consider why services like AWS are exploding and growing to be one of the biggest segments in tech? Because 'stupid executives'? No - it's because the value add is fairly immense. I've done this for 25 years, including private cloud setups from before AWS was a thing, and I've had to yell at execs that wanted to triple our monthly costs because AWS sales had whispered buzzwords into their ears. That was the all in costs. It took a massive effort to explain it to them even with the numbers in black and white in front of their faces. So, yeah, a lot of the time (not always), it is "stupid" executives. Sometimes talked into it by engineers working around planning processes that makes paying by use an easy end-run around budgeting. The biggest achievement of AWS is to sell the idea that it is cheap because Amazon.

jayd16 6y ago

The way I look at this is you still need 3 people minimum on call to have around the clock ops for this rack. Three IT salaries is more than $10k a month. Eventually you'll find the balancing point but its higher than you think.

afwaller 6y ago
Yes. This is the reason cloud is popular. You don’t have to pay for computer janitors.
- aea65 6y ago
  We have a title and don’t carry mops, my dear condescending developer who ostensibly doesn’t respect the entire ecosystem of people holding the Internet, the public cloud, and your entire CI/CD pipeline for whatever bullshit you’re cooking in NPM together. Do you have any idea how many “janitors,” most basically volunteer and most not employed by Amazon, are involved in the pedestrian aspect of merely making amazon.com land at the right place? Yes, this is a good future: who would ever want to touch the computers they operate? Better to rent computers and position the core competency of your business with a third party, right? Because you can shed some of those pesky janitorial salaries? I’ll be waiting by my phone when you’re on the verge of bankruptcy once the clouds have their teeth in your books, and suddenly I get promoted from “janitor” to “computer operator who has my interests in mind, even though I not-so-subtly malign his existence”. All you’ve done with this mindset is drive the same janitors to work for the clouds instead and contributed to the downfall of computing as a discipline that any player has any semblance of agency within, as the people who have actually touched datacenter equipment all work for them now or sit around in horror watching as a generation sympathetic to FLOSS arguments willingly hands over the reins of owning a computer to massive corporations.
  - louwrentius 6y ago
    Reading the comments in this thread already shows how much of 'those' 'software janitors'[1] don't understand what is required (and how little) to run on metal. It's not that cloud is not the right answer. But people have started to forget that running your own metal is still an option. Or with current prices and performance: even a more viable option as it was in the past, because you can do so much more with so much less. [1]: cranking out useless features nobody ask for while looking down on those (dev)ops people.

coldtea 6y ago

>Except that none of these compute resources are free and the cost of cloud-based super computing is a complete rip-off. It's actually thousands of dollars cheaper to build your own super computer Only if your time is worthless

csdreamer7 6y ago

> you can buy a Dell PowerEdge M915 from ebay with 128 cores for ~$500 USD Where? Was not able to find one. Found a 32 core for $750. It was a pre-Epyc AMD server which has it's own problems. There were only 9 options listed for the exact term you gave 'Dell PowerEdge M915' for the USA before I got to pricey international sellers. The few options make me feel skeptical about this. This is of course before I even consider security and performance issues of used Intel servers.

cambalache 6y ago
Every time someone says on the Internet "You can buy x thing in Ebay for Y price" multiply the price by 3.

louwrentius 6y ago

You are right. And if you buy metal and rent some space in a DC, it is still more cost-effective. You need a few people who understand a little bit of hardware and IPMI and you are set. That's not a big deal. The cloud gaslighting is nuts. Tons of companies now exist to give you insight in your cloud bill because that's the last thing cloud provides want you to have. So in the end you must decide what kind of pain you want. Cost pain, tech pain, and that all depends on your company's particular circumstances.

6gvONxR4sf7o 6y ago

You also have to account for burstiness. If you have nightly/weekly workflows, you might see some wild swings in resource usage, where an average usage and max usage differ by a lot. It’s nice to pay for average needs rather than max needs (especially when you can pay for on demand pricing).

garethmcc 6y ago

You forget a few things: 1. AWS and other cloud vendors don't have just one data centre per region, they usually have three redundant locations for availability reasons connected via a direct fibre link. 2. Cost of time; time to find hardware, prep hardware, maintain hardware 3. Cost of your agility. If you need more compute capacity, it will take you weeks to get the hardware required and installed, otherwise you need to have unused hardware sitting around "just in case". 4. Cost of your availability. What if you have a sudden spike of traffic within minutes and the current available hardware cannot service it? At least in the cloud you can spin up short lived resources to manage that load before it throttles back down again. 5. Permanent running costs of fixed hardware. A lot of implementations do not need permanent hardware running and can throttle down to a base of almost nothing so the average running cost on a monthly basis is actually very low.