11 ms·

A friend of mine was annoyed that a small service he liked was shutting down. He contacted the developer who said that they were shutting it down because the s

A friend of mine was annoyed that a small service he liked was shutting down. He contacted the developer who said that they were shutting it down because the server costs were higher than the money they were making. They were spending 5k a month on AWS crap and claimed it was impossible to get any lower. He helped them consolidate everything onto a single rented dedicated server costing 400 a month. Now the service is profitable, and will stay up. It runs way faster on the single server. It also has required less maintenance after the move too. This kind of shit is everywhere. At this point simply not using AWS is a competitive advantage.

throwawaysed 10y ago
A lot of people are in denial that there isn't using some magic efficiency to cloud services that other datacenters don't have. Primary cost savings on cloud VM's is from overprovisioning. The more abstracted away the service is from the hardware, the more they can overprovision without customers noticing. The 50%+ profit margins have to be coming from somewhere. AWS is not made of magic, it's made from largely the same PC parts you buy on newegg
- user5994461 10y ago
  > The 50%+ profit margins Charge more than what it costs you. That's how to make money.
  - deleted 10y ago
    [deleted]
- Shivetya 10y ago
  well to be honest its because it sounds cool, its in the news, and so how can such proposals be wrong. Seriously the number of times I heard "the cloud" bandied about by people who have continuously failed or were "thwarted" by people who knew better is getting silly. I am sure it happens in other places but too many think it makes them look smart to suggest it. if anything its a diversion from fixing what is broke or even admitting to it within some organizations
- illumin8 10y ago
  This is just factually incorrect. AWS hard partitions all instance types, except T2 (which are overcommitted, and clearly advertised as such: "burstable"). So, if you provision an x1.32xlarge with 128 vCPU and 1.92TB of RAM, you get a single, dedicated host with that much CPU and memory. Nobody else gets it - it's dedicated to you 100%. The profit AWS is making is purely due to datacenter efficiency and being able to automate their operations at scale.
  - patrickg_zill 10y ago
    The instant you wrote "128vCPU" you undermined your own argument. What is a vCPU?
    - brianwawok 10y ago
      It's early defined in the docs. The machine also has a physical core count that is yours.
      - patrickg_zill 10y ago
        There is nothing stopping Amazon from having a machine that is double that config, and renting you half of it, however. As soon as 32GB DIMMs drop enough in price, I would expect them to do exactly that. You can configure a Dell R930 with e7 Haswell CPUs online, if you want to double-check it. Not sure if the R930 can go to 8 CPUs (which the e7 Haswells CPUs support).
        dexterdog 10y ago
        But as soon as they double the size of the machine there is no reason for them to not have an instance type that is the size of the machine minus the host layer.
  - deleted 10y ago
    [deleted]
  - hueving 10y ago
user5994461 10y ago
You shouldn't be on AWS in the first place if everything you do can fit on a single server. Use the right tool for the job.
- lossolo 10y ago
  I have services that do not fit on one server and still I don't need AWS. Distributing load/dividing services on x-xx servers is not rocket science, especially with tools we have available at the moment.
  - sdflkd 10y ago
    How does one begin to learn about these things? Minimizing cost of running services sounds super interesting but as a student I've never had to deal with it and am basically starting with 0 knowledge.
    - sillysaurus3 10y ago
      I worked through the labs at https://pdos.csail.mit.edu/6.824/ https://pdos.csail.mit.edu/6.824/ for fun. It's more along the lines of "How can we write a distributed fault-tolerant database?" but you might like it anyway. Lab 4 is a beast with more lives than <insert metaphor>. The moment you think you've finally written your distributed system correctly, the unit tests will prove your service fails during XYZ network partition topologies. It's very worthwhile to be forced to think about issues like that and to design distributed systems for correctness. But to address your question more directly, it's generally just a matter of scaling your service as much as possible on a single server. The server has finite resources (CPU, Memory, network, disk) so you either know how your system consumes those resources (because e.g. you wrote the service, and you know it uses O(n) memory w.r.t. the workload) or you graph your usage over time and try to predict when you'll exhaust your available resources. At that point you can usually think of some straightforward optimizations, which keeps everything on a single server. But eventually you might run out of resources with no obvious path to optimize it, so what then? It depends entirely on your service, but typically you can just use off-the-shelf software to scale to multiple servers. For example you could set up three servers, each running Redis, and have Redis keep a list of "work to be done." Then your central process just farms out the workload to each of the three servers round-robin style, for example. But at that point your service becomes a lot more brittle, e.g. you'll need to set up a failover solution so that your cluster can survive partitions and outages. (Redis uses Sentinel for that.) So it's worth keeping everything on a single server for as long as possible, if you can work out the optimizations to do so, since it's so much simpler with only one server. (HN is still running on a single core and serving 350k daily uniques, I believe, which shows just how effective it can be to keep your architecture as simple as possible.)
jaredklewis 10y ago
Well, it seems obvious that if you want scalable, redundant, no downtime infrastructure, it will cost more than non-redundant, non-scalable infrastructure. While it was obviously a good trade off for this company, it seems silly to then extrapolate that all companies would benefit by getting rid of redundancy and scalability.
walrus01 10y ago
A million dollars buys a shitload of fast 1U dual socket xeon servers each with 512GB of RAM.
- user5994461 10y ago
  -looks at this datacenter- Well, actually it doesn't. RAM and Xeons are expensive.
  - gkop 10y ago
    So how many does it buy?
    - Aeolun 10y ago
      Approximately 4?
    - tiplus 10y ago
      I would estimate the cost for three years like this: - 1U single socket Supermicro Xeon E5v4-2630 10core+1GTX1070 GPU = 1500EUR - operation costs (power+cooling) over 3 years = 1500EUR - cost of 2 hardware admins 160k/year - renting local rack space 15EUR/U This equals about 145 servers for 3 years for us for 1 Million. The setup is highly optimized for _one_ set of data crunching tasks, not DB. We do not need much ram, or network but require a specific CPU/GPU ratio.
      - geezerjay 10y ago
        >cost of 2 hardware admins 160k/year This is by far the most expensive item in your estimate, and I'm sure that you don't suddenly need to hire another pair of admins when you buy a single Xeon blade. Once you factor this fact in your estimates, you'll realise that the unit cost of each server is, unlike your estimate, negligible.
    - user5994461 10y ago
      Let's say 1 Rack of 42 * 1U servers. That's about the upfront costs of servers ONLY. Ignoring cooling, all personal costs, electricity, networking gear, network storage and future maintenance.
that_colo_guy 10y ago
anon account for obvious reasons. Came here to share my story, which is very close to what you describe. tl;dr: Moved from AWS to colo in desperate attempt to make balance sheet more attractive. It worked. Before I took over the engineering bits at this plateau/failing startup, they were all in on AWS. Monthly costs were running at around £3k. Which is not much, but since the revenue was tiny, the costs of hosting was "considerable". After a monumental effort on my part to try to make things more efficient and streamlined, I eventually run out of motivation (but that's another story) so I expressed my wish to leave to the founder and he finally agreed to my years old suggestion of going lean-and-mean and in one last attempt to make the company more attractive to one last investment round (E by then). Make it or break it, as the say goes. On that same week I came across a HPE "Buy One Get One Free" offer (similar to this one https://www.serversdirect.co.uk/pdf/BOGOF-Gen9-Servers.pdf https://www.serversdirect.co.uk/pdf/BOGOF-Gen9-Servers.pdf). Then found guy who was re-selling colo space at a under-used datacentre in London. Since I (sysadmin/DevOps/InfoSec/Backend engineer Jack-of-all-trades master of none) was the person who would end up managing the new metal, I picked a location within an hours drive from home. And so it happened. 2x 1U HPE proliant servers with decent CPU and 32G of RAM. And 6x Samsung 850 SSDs off Amazon. Total bill for the hardware (BOGOF refund included): £2600 ish. Post migration, monthly hosting costs: £50 for the colo; £50-ish AWS S3+CloudFront (10M assets would increase the SSD storage costs too much); £40 in taxi fares when I visit the DC once a month for kernel updates. And just like that, we broke even the very next month after the last "big" AWS bill. Our product wasn't all that exciting, but 2 months later we found a buyer who was happy to snatch this "non loss making" operation. 3 months after the acquisition, they had already migrated the whole thing back to AWS and the HPE proliant servers were gathering dust in their office.
- empath75 10y ago
  Of course, because how do you scale up when you're building everything yourself? Suddenly you have to hire a whole bunch of hardware guys, storage guys, db guys etc, etc.
  - brianwawok 10y ago
    Why do you need a DB guy if you host it on your server or AWSs? You have a DB either way. You can have a guy if you want or not... Physical location means nothing.
lossolo 10y ago
Exactly this. I am using dedicated servers everywhere, I would pay easily 10-20x more for the same if I would use AWS instead.
yegle 10y ago
So, how about backup, availability and a potential additional employee to make sure the server runs fine? If they can indeed fit their footprint in one single server, then it should be fisible to just host the server at someone's home with enough upload speed, no?
rbranson 10y ago
Totally. If the growth rate is reasonable (<50% y/y) and haven't really broken beyond what can be done on a single machine, using AWS or GCE or Azure is a ludicrous decision, short of a "serverless" style architecture. It's pretty easy to get sucked into the "best practices" and overengineer something that costs a lot of money.
voidlogic 10y ago
I used to do data-center and virtualization consulting- we always designed our enterprise client's systems to handle the baseload in-house and burst to public cloud. Not rock science. We even automatically live-migrated load from and shutdown onsite hypervisors at night when the system had surplus capacity... I think the real issue is that people assume all workloads are public cloud workloads. The bigger and less dynamic your workload the less that is true.
- klodolph 10y ago
  This is a great approach. A company I worked at was running mostly on dedicated hardware, and due to the pain of moving jobs around and getting new hardware, we focused hard on optimizing our code to run within our existing capacity whenever we ran into problems. After a couple years of customer growth, I'm sure we would have been running on at least 10x as many machines if we had never done that. We then set up cloud spot instances for when we just needed extra short-term capacity.
  - user5994461 10y ago
    > I'm sure we would have been running on at least 10x as many machines Seeing the opposite trend. Because your own machines are so expensive and annoying to manage, you only get 500GB memory 40 cores servers. You don't optimize and you run stuff randomly on whatever seem available. Whereas in the cloud, you put VMs per role, with appropriate sizing. And when someone asks for 5 machines with 16 cores, you can be like, WTF are you running with all that power???
    - klodolph 10y ago
      I would believe that, but we were hitting the limits of our huge servers. There's only so much QPS you can throw at a single box. These servers were also "pets" as opposed to "cattle", and a big part of the prep for cloud was treating them as cattle.
amelius 10y ago
I guess you could go even lower by not renting, but buying a server instead.
- qeternity 10y ago
  Not necessarily true. We run a decent amount of gear with OVH, and because if their scale and bulk discounts from OEM, our TCO is actually lower than it otherwise would be. I'm sure this isn't true at much larger scale, but I would bet it's true for most people.
QuinnyPig 10y ago
> At this point simply not using AWS is a competitive advantage. Respectfully, I'm going to disagree. I consult full time on AWS cost optimization / reduction / understanding. If you blindly run things on AWS without an understanding of the costing model, that'll work for a time. As you scale, you start to realize "oh my god it runs on money." There are myriad ways around that, but a blanket "never use AWS" isn't going to address it constructively.
- qeternity 10y ago
  I am curious your thoughts on the following. It seems like there is a trough of benefit to AWS. In the mid-size world, especially where computing needs are relatively stable/predictable, you are much better leasing dedicated servers from the usual suspects. Below that size, the linear scaling allows small scrappy startups to really scale/burst with their needs. And at large scale, but below Google/Facebook et al scale, it's cheaper than building your own datacenter yourself. At least this has been my casual observations. And fwiw, the startup I have just launched fits into this trough perfectly. So hopefully I am not just biased. I love the cloud for the spot market. But not for our base load. Spot is the nat gas of the server world, and OVH is the coal.
  - QuinnyPig 10y ago
    I agree with a lot of what you're saying-- but that trough is a lot wider than most people think. At the single instance very small scale, it simply doesn't matter all that much. "Our business failed because our infrastructure bill was $400 instead of $150 every month" isn't that common of a story. Base load can be effectively migrated to AWS, depending upon how it's built / structured. It's hard to lose money on infrastructure by converting worker servers to spot instances or lambda functions, as a for-instance-- but it's VERY easy to lose months of engineering effort in doing so. Remember as well: done right you're not just buying services from the cloud, but flexibility as well. Resizing on the fly, bursting, duplicating your infrastructure for a test-- these things add value.
    - rhizome 10y ago
kbenson 10y ago
The company I work for is backed by a bunch of data in a MySQL database, scripts to aggregate and process more data into that database, and simple internal webapp to both manage inventory, track tasks and run lots of reporting on the aggregated data. We made do for quite a while on a single colocated database with 32 GB of RAM and a few fast SSDs (for the time, which was about 6 years ago), and a few multi-TB archival drives, but disk space is a concern since our compressed innodb tables are well over 100GB now. I set up a Google Cloud SQL instance (the largest possible at the time I believe, 104GB RAM, and enough disks to hit peak IOPS) a few months back in preparation of migrating the data store to it, and found it wholly inadequate for our needs prior to even getting it going. First, the second generation instances don't support being a replication slave to a non Cloud SQL instance, making migration a pain. Second, just running through the replication logs on a client and piping it to a connection to the cloud SQL instance showed that the cloud SQL instance was just barely able to keep up with the insert and update traffic from the replication log, even while handling no other queries. Catching up the the master would take over a week, even though the backup was from less than 48 hours prior. We signed up for an actual hardware server from Rackspace at about 2/3 the cost, but with 128 GB of RAM (which I can request expanded), and nice hardware RAID 10 of fast SSD drives, and tons of logical CPUs. It's appropriately faster compared to the original database server, to the degree you would expect from from stats quoted. It fits our needs, because really I just don't want to manage the hardware anymore (this the decision not to deploy more hardware in the local datacenter we are colocated at), and that's covered. On demand spin up/down of resources and per-minute billing is nice, and I've used it to good effect for non-database resources before, so I understand the versatility of it and how it can really enable some new types of usage and am a fan of that aspect, but I just wasn't prepared for how substandard the DB offerings are if you actually have any sort of load. At this point, the only good thing I have to say about the higher end database offerings is that they were quick to destroy so I didn't get billed anymore. :/
- illumin8 10y ago
  May I make a suggestion: Try Amazon Aurora. You'll get 500,000 read operations per second, and 100,000 writes per second. It supports up to 64TB of total capacity, and 244GB RAM on your primary master. You can also have up to 15 read replicas with a similar footprint. You should seriously try it. It's amazing.
whalesalad 10y ago
The problem is not AWS. The problem is poorly architected systems and software. We see a similar issue with the "we switched from PHP to Haskell on .NET in Azure and improved performance 3000x" -- the issue wasn't the fault of the language, it was the way it was used. You put bald tires that are over-inflated on a Porsche and you're gonna have a shitty experience. Sure, bare-metal is gonna have performance benefits over AWS and at times might be cheaper -- but that isn't the only thing an eng organization is looking for. tl;dr – you can build systems on AWS that perform well and are reasonably priced.
- ars 10y ago
  > You put bald tires that are over-inflated on a Porsche and you're gonna have a shitty experience. This is off topic, but bald tires have better traction than grooved ones in dry conditions. (But the slightest bit of rain and they will not hold.) Also tire inflation does not have a strong correlation with traction (there is some, but it's not large), so it's not so obvious that over-inflated means a bad experience.
  - SudhindraBhat 10y ago
    Bald tires also mean that the tire compound has degraded. So it is going to have worse traction. Slick tires on the other hand are new without any grooves.
patrickg_zill 10y ago
You are absolutely correct, and I have seen this sort of thing time and again... "cloud" doesn't mean that the server is magically faster - it usually means "a great deal slower" because it is a shared resource.
YZF 10y ago
If you have a more or less fixed workload then AWS ends up being a lot more expensive. If you have a highly variable workload and you have a bunch of servers sitting idle most of the time then it becomes more competitive. AWS lets a small startup go from one server to 10K servers without breaking sweat, at least in theory, but it's not for everyone. At the same time not everyone wants to build DCs with thousands of servers in different geographies all by their own. But yeah, AWS generally is expensive, which is part of why it's making so much money. It definitely makes business sense to try to avoid locking yourself in to AWS but at the same time it's a good place to ramp up or test your business ideas without needing to invest a lot of money in servers, real estate etc. [You can probably substitute any other cloud provider for AWS here]
- phamilton 10y ago
  The variable workload has been a little less important with reserved instances. They used to have a "light/heavy utilization" pricing structure but now it's basically cheaper to reserve the instance than it is to scale it up and down, unless you need the extra capacity less than 60% of the time.
  - dexterdog 10y ago
    That's going between 1 and 0 servers for an instance. Real scaling needs are met in a setup going from 3-50+ servers throughout the week.
    - phamilton 10y ago
      I think you misunderstand me. Doing 100 servers for 16 hours and scaling down to 30 servers for 8 hours a day is more expensive than running 100 reserved instances for 24 hours a day. (100 reserved instances == average of 60 on demand instances).
ilaksh 10y ago
> simply not using AWS is a competitive advantage. The way you wrote your comment about using a dedicated server implies that there are no other viable cloud providers besides AWS. That is the problem, people do not know or do not acknowledge the existence of a number of other cloud/VPS providers that, in most cases, would work perfectly fine for their startup, and be a massively better value. Such as Digital Ocean, Linode, or many small relatively less well-known providers that are perfectly fine like UltraVPS etc. Sure, some few people do actually need DynamoDB or some other ultra-scaling AWS service, but people who just need VPSs can get a much better deal elsewhere.
reacweb 10y ago
IMHO, AWS is fabulous when your needs are a fast moving target. When you have something stable and have time to optimize for performance and cost, dedicated servers are better (cheaper and/or faster). There is no silver bullet.

throwawaysed 10y ago

A lot of people are in denial that there isn't using some magic efficiency to cloud services that other datacenters don't have. Primary cost savings on cloud VM's is from overprovisioning. The more abstracted away the service is from the hardware, the more they can overprovision without customers noticing. The 50%+ profit margins have to be coming from somewhere. AWS is not made of magic, it's made from largely the same PC parts you buy on newegg

user5994461 10y ago
> The 50%+ profit margins Charge more than what it costs you. That's how to make money.
- deleted 10y ago
  [deleted]
Shivetya 10y ago
well to be honest its because it sounds cool, its in the news, and so how can such proposals be wrong. Seriously the number of times I heard "the cloud" bandied about by people who have continuously failed or were "thwarted" by people who knew better is getting silly. I am sure it happens in other places but too many think it makes them look smart to suggest it. if anything its a diversion from fixing what is broke or even admitting to it within some organizations
illumin8 10y ago
This is just factually incorrect. AWS hard partitions all instance types, except T2 (which are overcommitted, and clearly advertised as such: "burstable"). So, if you provision an x1.32xlarge with 128 vCPU and 1.92TB of RAM, you get a single, dedicated host with that much CPU and memory. Nobody else gets it - it's dedicated to you 100%. The profit AWS is making is purely due to datacenter efficiency and being able to automate their operations at scale.
- patrickg_zill 10y ago
  The instant you wrote "128vCPU" you undermined your own argument. What is a vCPU?
  - brianwawok 10y ago
    It's early defined in the docs. The machine also has a physical core count that is yours.
    - patrickg_zill 10y ago
      There is nothing stopping Amazon from having a machine that is double that config, and renting you half of it, however. As soon as 32GB DIMMs drop enough in price, I would expect them to do exactly that. You can configure a Dell R930 with e7 Haswell CPUs online, if you want to double-check it. Not sure if the R930 can go to 8 CPUs (which the e7 Haswells CPUs support).
      - dexterdog 10y ago
        But as soon as they double the size of the machine there is no reason for them to not have an instance type that is the size of the machine minus the host layer.
- deleted 10y ago
  [deleted]
- hueving 10y ago

user5994461 10y ago

You shouldn't be on AWS in the first place if everything you do can fit on a single server. Use the right tool for the job.

lossolo 10y ago
I have services that do not fit on one server and still I don't need AWS. Distributing load/dividing services on x-xx servers is not rocket science, especially with tools we have available at the moment.
- sdflkd 10y ago
  How does one begin to learn about these things? Minimizing cost of running services sounds super interesting but as a student I've never had to deal with it and am basically starting with 0 knowledge.
  - sillysaurus3 10y ago
    I worked through the labs at https://pdos.csail.mit.edu/6.824/ https://pdos.csail.mit.edu/6.824/ for fun. It's more along the lines of "How can we write a distributed fault-tolerant database?" but you might like it anyway. Lab 4 is a beast with more lives than <insert metaphor>. The moment you think you've finally written your distributed system correctly, the unit tests will prove your service fails during XYZ network partition topologies. It's very worthwhile to be forced to think about issues like that and to design distributed systems for correctness. But to address your question more directly, it's generally just a matter of scaling your service as much as possible on a single server. The server has finite resources (CPU, Memory, network, disk) so you either know how your system consumes those resources (because e.g. you wrote the service, and you know it uses O(n) memory w.r.t. the workload) or you graph your usage over time and try to predict when you'll exhaust your available resources. At that point you can usually think of some straightforward optimizations, which keeps everything on a single server. But eventually you might run out of resources with no obvious path to optimize it, so what then? It depends entirely on your service, but typically you can just use off-the-shelf software to scale to multiple servers. For example you could set up three servers, each running Redis, and have Redis keep a list of "work to be done." Then your central process just farms out the workload to each of the three servers round-robin style, for example. But at that point your service becomes a lot more brittle, e.g. you'll need to set up a failover solution so that your cluster can survive partitions and outages. (Redis uses Sentinel for that.) So it's worth keeping everything on a single server for as long as possible, if you can work out the optimizations to do so, since it's so much simpler with only one server. (HN is still running on a single core and serving 350k daily uniques, I believe, which shows just how effective it can be to keep your architecture as simple as possible.)

jaredklewis 10y ago

Well, it seems obvious that if you want scalable, redundant, no downtime infrastructure, it will cost more than non-redundant, non-scalable infrastructure. While it was obviously a good trade off for this company, it seems silly to then extrapolate that all companies would benefit by getting rid of redundancy and scalability.

walrus01 10y ago

A million dollars buys a shitload of fast 1U dual socket xeon servers each with 512GB of RAM.

user5994461 10y ago
-looks at this datacenter- Well, actually it doesn't. RAM and Xeons are expensive.
- gkop 10y ago
  So how many does it buy?
  - Aeolun 10y ago
    Approximately 4?
  - tiplus 10y ago
    I would estimate the cost for three years like this: - 1U single socket Supermicro Xeon E5v4-2630 10core+1GTX1070 GPU = 1500EUR - operation costs (power+cooling) over 3 years = 1500EUR - cost of 2 hardware admins 160k/year - renting local rack space 15EUR/U This equals about 145 servers for 3 years for us for 1 Million. The setup is highly optimized for _one_ set of data crunching tasks, not DB. We do not need much ram, or network but require a specific CPU/GPU ratio.
    - geezerjay 10y ago
      >cost of 2 hardware admins 160k/year This is by far the most expensive item in your estimate, and I'm sure that you don't suddenly need to hire another pair of admins when you buy a single Xeon blade. Once you factor this fact in your estimates, you'll realise that the unit cost of each server is, unlike your estimate, negligible.
  - user5994461 10y ago
    Let's say 1 Rack of 42 * 1U servers. That's about the upfront costs of servers ONLY. Ignoring cooling, all personal costs, electricity, networking gear, network storage and future maintenance.

that_colo_guy 10y ago

anon account for obvious reasons. Came here to share my story, which is very close to what you describe. tl;dr: Moved from AWS to colo in desperate attempt to make balance sheet more attractive. It worked. Before I took over the engineering bits at this plateau/failing startup, they were all in on AWS. Monthly costs were running at around £3k. Which is not much, but since the revenue was tiny, the costs of hosting was "considerable". After a monumental effort on my part to try to make things more efficient and streamlined, I eventually run out of motivation (but that's another story) so I expressed my wish to leave to the founder and he finally agreed to my years old suggestion of going lean-and-mean and in one last attempt to make the company more attractive to one last investment round (E by then). Make it or break it, as the say goes. On that same week I came across a HPE "Buy One Get One Free" offer (similar to this one https://www.serversdirect.co.uk/pdf/BOGOF-Gen9-Servers.pdf https://www.serversdirect.co.uk/pdf/BOGOF-Gen9-Servers.pdf). Then found guy who was re-selling colo space at a under-used datacentre in London. Since I (sysadmin/DevOps/InfoSec/Backend engineer Jack-of-all-trades master of none) was the person who would end up managing the new metal, I picked a location within an hours drive from home. And so it happened. 2x 1U HPE proliant servers with decent CPU and 32G of RAM. And 6x Samsung 850 SSDs off Amazon. Total bill for the hardware (BOGOF refund included): £2600 ish. Post migration, monthly hosting costs: £50 for the colo; £50-ish AWS S3+CloudFront (10M assets would increase the SSD storage costs too much); £40 in taxi fares when I visit the DC once a month for kernel updates. And just like that, we broke even the very next month after the last "big" AWS bill. Our product wasn't all that exciting, but 2 months later we found a buyer who was happy to snatch this "non loss making" operation. 3 months after the acquisition, they had already migrated the whole thing back to AWS and the HPE proliant servers were gathering dust in their office.

empath75 10y ago
Of course, because how do you scale up when you're building everything yourself? Suddenly you have to hire a whole bunch of hardware guys, storage guys, db guys etc, etc.
- brianwawok 10y ago
  Why do you need a DB guy if you host it on your server or AWSs? You have a DB either way. You can have a guy if you want or not... Physical location means nothing.

lossolo 10y ago

Exactly this. I am using dedicated servers everywhere, I would pay easily 10-20x more for the same if I would use AWS instead.

yegle 10y ago

So, how about backup, availability and a potential additional employee to make sure the server runs fine? If they can indeed fit their footprint in one single server, then it should be fisible to just host the server at someone's home with enough upload speed, no?

rbranson 10y ago

Totally. If the growth rate is reasonable (<50% y/y) and haven't really broken beyond what can be done on a single machine, using AWS or GCE or Azure is a ludicrous decision, short of a "serverless" style architecture. It's pretty easy to get sucked into the "best practices" and overengineer something that costs a lot of money.

voidlogic 10y ago

I used to do data-center and virtualization consulting- we always designed our enterprise client's systems to handle the baseload in-house and burst to public cloud. Not rock science. We even automatically live-migrated load from and shutdown onsite hypervisors at night when the system had surplus capacity... I think the real issue is that people assume all workloads are public cloud workloads. The bigger and less dynamic your workload the less that is true.

klodolph 10y ago
This is a great approach. A company I worked at was running mostly on dedicated hardware, and due to the pain of moving jobs around and getting new hardware, we focused hard on optimizing our code to run within our existing capacity whenever we ran into problems. After a couple years of customer growth, I'm sure we would have been running on at least 10x as many machines if we had never done that. We then set up cloud spot instances for when we just needed extra short-term capacity.
- user5994461 10y ago
  > I'm sure we would have been running on at least 10x as many machines Seeing the opposite trend. Because your own machines are so expensive and annoying to manage, you only get 500GB memory 40 cores servers. You don't optimize and you run stuff randomly on whatever seem available. Whereas in the cloud, you put VMs per role, with appropriate sizing. And when someone asks for 5 machines with 16 cores, you can be like, WTF are you running with all that power???
  - klodolph 10y ago
    I would believe that, but we were hitting the limits of our huge servers. There's only so much QPS you can throw at a single box. These servers were also "pets" as opposed to "cattle", and a big part of the prep for cloud was treating them as cattle.

amelius 10y ago

I guess you could go even lower by not renting, but buying a server instead.

qeternity 10y ago
Not necessarily true. We run a decent amount of gear with OVH, and because if their scale and bulk discounts from OEM, our TCO is actually lower than it otherwise would be. I'm sure this isn't true at much larger scale, but I would bet it's true for most people.

QuinnyPig 10y ago

> At this point simply not using AWS is a competitive advantage. Respectfully, I'm going to disagree. I consult full time on AWS cost optimization / reduction / understanding. If you blindly run things on AWS without an understanding of the costing model, that'll work for a time. As you scale, you start to realize "oh my god it runs on money." There are myriad ways around that, but a blanket "never use AWS" isn't going to address it constructively.

qeternity 10y ago
I am curious your thoughts on the following. It seems like there is a trough of benefit to AWS. In the mid-size world, especially where computing needs are relatively stable/predictable, you are much better leasing dedicated servers from the usual suspects. Below that size, the linear scaling allows small scrappy startups to really scale/burst with their needs. And at large scale, but below Google/Facebook et al scale, it's cheaper than building your own datacenter yourself. At least this has been my casual observations. And fwiw, the startup I have just launched fits into this trough perfectly. So hopefully I am not just biased. I love the cloud for the spot market. But not for our base load. Spot is the nat gas of the server world, and OVH is the coal.
- QuinnyPig 10y ago
  I agree with a lot of what you're saying-- but that trough is a lot wider than most people think. At the single instance very small scale, it simply doesn't matter all that much. "Our business failed because our infrastructure bill was $400 instead of $150 every month" isn't that common of a story. Base load can be effectively migrated to AWS, depending upon how it's built / structured. It's hard to lose money on infrastructure by converting worker servers to spot instances or lambda functions, as a for-instance-- but it's VERY easy to lose months of engineering effort in doing so. Remember as well: done right you're not just buying services from the cloud, but flexibility as well. Resizing on the fly, bursting, duplicating your infrastructure for a test-- these things add value.
  - rhizome 10y ago

kbenson 10y ago

The company I work for is backed by a bunch of data in a MySQL database, scripts to aggregate and process more data into that database, and simple internal webapp to both manage inventory, track tasks and run lots of reporting on the aggregated data. We made do for quite a while on a single colocated database with 32 GB of RAM and a few fast SSDs (for the time, which was about 6 years ago), and a few multi-TB archival drives, but disk space is a concern since our compressed innodb tables are well over 100GB now. I set up a Google Cloud SQL instance (the largest possible at the time I believe, 104GB RAM, and enough disks to hit peak IOPS) a few months back in preparation of migrating the data store to it, and found it wholly inadequate for our needs prior to even getting it going. First, the second generation instances don't support being a replication slave to a non Cloud SQL instance, making migration a pain. Second, just running through the replication logs on a client and piping it to a connection to the cloud SQL instance showed that the cloud SQL instance was just barely able to keep up with the insert and update traffic from the replication log, even while handling no other queries. Catching up the the master would take over a week, even though the backup was from less than 48 hours prior. We signed up for an actual hardware server from Rackspace at about 2/3 the cost, but with 128 GB of RAM (which I can request expanded), and nice hardware RAID 10 of fast SSD drives, and tons of logical CPUs. It's appropriately faster compared to the original database server, to the degree you would expect from from stats quoted. It fits our needs, because really I just don't want to manage the hardware anymore (this the decision not to deploy more hardware in the local datacenter we are colocated at), and that's covered. On demand spin up/down of resources and per-minute billing is nice, and I've used it to good effect for non-database resources before, so I understand the versatility of it and how it can really enable some new types of usage and am a fan of that aspect, but I just wasn't prepared for how substandard the DB offerings are if you actually have any sort of load. At this point, the only good thing I have to say about the higher end database offerings is that they were quick to destroy so I didn't get billed anymore. :/

illumin8 10y ago
May I make a suggestion: Try Amazon Aurora. You'll get 500,000 read operations per second, and 100,000 writes per second. It supports up to 64TB of total capacity, and 244GB RAM on your primary master. You can also have up to 15 read replicas with a similar footprint. You should seriously try it. It's amazing.

whalesalad 10y ago

The problem is not AWS. The problem is poorly architected systems and software. We see a similar issue with the "we switched from PHP to Haskell on .NET in Azure and improved performance 3000x" -- the issue wasn't the fault of the language, it was the way it was used. You put bald tires that are over-inflated on a Porsche and you're gonna have a shitty experience. Sure, bare-metal is gonna have performance benefits over AWS and at times might be cheaper -- but that isn't the only thing an eng organization is looking for. tl;dr – you can build systems on AWS that perform well and are reasonably priced.

ars 10y ago
> You put bald tires that are over-inflated on a Porsche and you're gonna have a shitty experience. This is off topic, but bald tires have better traction than grooved ones in dry conditions. (But the slightest bit of rain and they will not hold.) Also tire inflation does not have a strong correlation with traction (there is some, but it's not large), so it's not so obvious that over-inflated means a bad experience.
- SudhindraBhat 10y ago
  Bald tires also mean that the tire compound has degraded. So it is going to have worse traction. Slick tires on the other hand are new without any grooves.

patrickg_zill 10y ago

You are absolutely correct, and I have seen this sort of thing time and again... "cloud" doesn't mean that the server is magically faster - it usually means "a great deal slower" because it is a shared resource.

YZF 10y ago

If you have a more or less fixed workload then AWS ends up being a lot more expensive. If you have a highly variable workload and you have a bunch of servers sitting idle most of the time then it becomes more competitive. AWS lets a small startup go from one server to 10K servers without breaking sweat, at least in theory, but it's not for everyone. At the same time not everyone wants to build DCs with thousands of servers in different geographies all by their own. But yeah, AWS generally is expensive, which is part of why it's making so much money. It definitely makes business sense to try to avoid locking yourself in to AWS but at the same time it's a good place to ramp up or test your business ideas without needing to invest a lot of money in servers, real estate etc. [You can probably substitute any other cloud provider for AWS here]

phamilton 10y ago
The variable workload has been a little less important with reserved instances. They used to have a "light/heavy utilization" pricing structure but now it's basically cheaper to reserve the instance than it is to scale it up and down, unless you need the extra capacity less than 60% of the time.
- dexterdog 10y ago
  That's going between 1 and 0 servers for an instance. Real scaling needs are met in a setup going from 3-50+ servers throughout the week.
  - phamilton 10y ago
    I think you misunderstand me. Doing 100 servers for 16 hours and scaling down to 30 servers for 8 hours a day is more expensive than running 100 reserved instances for 24 hours a day. (100 reserved instances == average of 60 on demand instances).

ilaksh 10y ago

> simply not using AWS is a competitive advantage. The way you wrote your comment about using a dedicated server implies that there are no other viable cloud providers besides AWS. That is the problem, people do not know or do not acknowledge the existence of a number of other cloud/VPS providers that, in most cases, would work perfectly fine for their startup, and be a massively better value. Such as Digital Ocean, Linode, or many small relatively less well-known providers that are perfectly fine like UltraVPS etc. Sure, some few people do actually need DynamoDB or some other ultra-scaling AWS service, but people who just need VPSs can get a much better deal elsewhere.

reacweb 10y ago

IMHO, AWS is fabulous when your needs are a fast moving target. When you have something stable and have time to optimize for performance and cost, dedicated servers are better (cheaper and/or faster). There is no silver bullet.