6 ms·

This article is written in kind of a controversial way but it seems like the throughline of the argument is something like "use heroku until you have 100k users

This article is written in kind of a controversial way but it seems like the throughline of the argument is something like "use heroku until you have 100k users". This seems very reasonable to me. I thought it was going to be a pitch for on prem, which is also fine for certain scales. I think generally the scaling steps from startup to megacorp go: Heroku/Dokku > Public Cloud >Dedicated servers in someone else's DC > Custom Hardware in custom built data centers. Each makes sense at each scale. I find it to be more of a right tool for the job consideration than one being better than the other. With modern cloud tooling your infra can also look more or less logically the same once you grow past the heroku level.

dijit 4y ago
I really agree with you, what's weird though is how many mega-corps are going away from Custom Hardware in Custom Built DC towards Cloud. There's also something to be said for buying a VPS or a Colo machine, making sure it's backed up and dealing with the 9's that you get from that machine on it's own. I am routinely surprised by how far a single node machine will get you.
- mwcampbell 4y ago
  > how far a single node machine will get you. This. I took the wrong lesson from the DDoS attacks on Linode in late 2015 (particularly the one on Christmas Day), and the intermittent issues I encountered with DigitalOcean and Vultr in 2016 while both providers were still fairly young. A single dedicated server from a mature provider (ideally not during its hyper-growth phase) is pretty reliable.
- sofixa 4y ago
  > what's weird though is how many mega-corps are going away from Custom Hardware in Custom Built DC towards Cloud It costs a lot of money to run your own datacenters, and very very few companies are capable of doing it as good as AWS or even Scaleway/OVH can. By that I mean, waiting weeks/months to get through tickets, approvals, multiple different teams just to get a server deployed. Then waiting a few more weeks for monitoring/backups. Allowing developers and related to have hardware/software at a whim is a massive advantage.
  - dijit 4y ago
    100% Agree. I don't like the idea that the only way to get developers moving is to use cloud, but I agree that it's a solid replacement for really bad ops. What I've seen in many places is an abstraction over bare metal, some are better than others, openstack, Kubernetes on-prem, vmware etc; are all solutions that have differing amounts of adoption. Ubisoft had a lot of stuff in this area, as does Google. Ubisofts was pretty terrible though. If you need a physical machine to be deployed, you've hit a certain level of scale and your load is much more known: and even though it can take a few weeks, what you get back is quite competitive. But if you're waiting for hardware to get anything moving in the first place then that's obviously bad. What I've taken to doing is prototyping on Google Cloud and then planning to migrate things to on-prem once everything is reaching maturity.
  - dcoperator1 4y ago
    The price for colocating in a Tier 2 datacenter close enough for me to walk to was $75/amp. Which is about the same cost as AWS. You're really paying somewhere between the savings of putting a datacenter in Nowhere, Oregon and the cost to convince someone to live there.
    - nine_k 4y ago
      How can you directly compare $75 / amp to AWS? This very much depends on the kind if load you have. Also, AWS's egress likely costs much more than the datacenter's; again, possible savings very much depend on the kind of load you have.
    - wongarsu 4y ago
      I'd agree that unless you can really profit from having very specific hardware, you're better off renting dedicated servers than colocating servers you own. Have somebody else worry about having people on call to switch out failed hard drives. Comparing either to AWS will inevitably lead to a much more complex discussion about spot instances, traffic costs, ancillary services etc.
  - kqr 4y ago
    If you have a little spare capacity, developers can still get hardware/software on a whim, at just a (comparatively) small one-time expense. Spare capacity is much cheaper than people make it out to be. The upside is that it's much cheaper once you're at the scale where you no longer need to variableise your compute costs, but can tank the up-front fixed costs and do proper capacity planning.
- nine_k 4y ago
  These megacorps likely have IT which is not mega enough to justify owning a massive datacenter. The right scale is Amazon, Google, Facebook, Microsoft. Likely much fewer than a hundred companies in the entire world.
- PragmaticPulp 4y ago
  > what's weird though is how many mega-corps are going away from Custom Hardware in Custom Built DC towards Cloud. Why is it surprising? Building and maintaining custom data centers is a big, slow business initiative. It takes months to years of forecasting to get the data center buildout to match the business needs, as opposed to the extreme flexibility of using a cloud provider. > There's also something to be said for buying a VPS or a Colo machine, making sure it's backed up and dealing with the 9's that you get from that machine on it's own. I am routinely surprised by how far a single node machine will get you. For personal projects this is exactly what I do. It’s great until something goes wrong with that one machine or VPS. But it’s not really a good option for any business that needs consistent operations and uptime. Years ago I worked at a company that tried to self-host some of their collaboration tools on a VPS to save money over the cloud-hosted versions. When the server went down it stalled productivity for a day while the team restored a backup, with another week of confusion as we tried to find all of the things that were lost between the last backup and when the server went down. When someone did the rough estimations on how much it cost to pay everyone’s salaries for that day of lost productivity, the number was far higher than the trivial cost savings we got from self-hosting. We also had a constant background burden on someone internally to maintain and monitor the server, plus the burden of them being on call. Often, moving to cloud anything can be a huge load off the company’s back.
  - dijit 4y ago
    > as opposed to the extreme flexibility of using a cloud provider. I don't really buy this honestly. What you buy with cloud providers is quality tooling, not flexibility. If you're bin-packing with Kubernetes properly then capacity is capacity and it doesn't matter if the marketing department are using it or the developers are. You just buy a bunch of servers and when you see the load approaching 70% you buy more. It's a 2 person job. Is it harder? Yes. Definitely. Is it a panacea? No. Not at all. Is it universally cheaper? Also no. Definitely not. I feel like whenever I talk about the Cloud as an expensive thing that people get emotionally defensive. I'm not here to take your toys away. Services like cloud are just tools and tools always have pros and cons. If you can't reasonably discuss the con's without resorting to "I need to hire more staff" or "its a lot better than $strawman" then we're just cargo culting. > it’s not really a good option for any business that needs consistent operations and uptime. Most business cases for computers can just eat the downtime honestly. Your URL redirector doesn't need 5 9's. There's a grading scale of complexity and uptime, on one side you have a single hosted server that has profoundly strong uptime (especially with the redundancies in normal servers); then you start adding complexity to get HA, and weirdly: the complexity lowers the reliability. If you keep following the line of redundancies and HA complexity, eventually you can get to a point where the service is even more reliable than a single node. Which is what everyone assumes they will get straight away, but usually it's a lot of work to get there. > the trivial cost savings This will differ a lot. I made two games, one was hybrid-cloud and one was bare metal only; the cost savings were not trivial. If we had 100% clouded the hybrid deployment we would easily have paid 10x in the hosting costs which would have been enough money to pay for 250 contractors at a premium rate. That said: the toys were definitely shiny.
- PubliusMI 4y ago
  It's not weird. Many mega-corps are extremely bloated and dysfunctional. Their IT (Private) Cloud teams slower and less competent. With public cloud, a small team can be fully responsible for all their resources with crystal clear cost accounting.
- P5fRxh5kUvp2th 4y ago
  At a certain scale you have more negotiating power so it probably makes more sense from that perspective.
- Fendii 4y ago
  Every internal it department was always slower than any cloud offering. Or laged features. Or had underlying infra issues. With AWS in one Startup I was able to build and maintain infrastructure were you needed a small team just 10 years before.
- jacobr1 4y ago
  Data Residency. Scaling to dozens of global regions is not cost effective for running your own DCs.
  - dijit 4y ago
    Sorry, the next step after cloud isn't running your own DC's. It's renting space from someone who is doing that for you. The scale of cost savings/ownership as you scale kinda goes: FaaS -> PaaS -> Cloud -> VPS -> Rented Hardware -> Rented space in DC -> Own DC. Let's avoid conflating everything left of VPS with the most difficult form of it, because nobody is going there from nothing.
osigurdson 4y ago
I think you can be pretty big and still stay in Public Cloud. There is enough competition that pricing should trend toward commodity over time.
buzzdenver 4y ago
> "use heroku until you have 100k users" Thing is that for most/many startups 100k users is not a lot. Rejiggling your basic infra just as your growth is starting to accelerate is a non-trivial task, a risk, and something that doesn't fundamentally move the needle.
- hagbarth 4y ago
  This really depends on what you are building of course. Working in enterprise SaaS with only a few users per account? You'll be doing really well at 100k users.
- icedchai 4y ago
  Most B2B startups never get anywhere near 100K users.
- lelanthran 4y ago
  > Thing is that for most/many startups 100k users is not a lot. Depends; if you're a startup offering free or ad-supported services and the exit plan is "be bought out by existing entrenched competitor", then, yes, 100k users is not enough to hit your goals. If you're a startup offering B2B services, even 10k users is enough to be madly profitable.
cultofmetatron 4y ago
> "use heroku until you have 100k users" nowadays, i'd say use fly.io or render till you have 200k users.
sakisv 4y ago
> Heroku/Dokku > Public Cloud >Dedicated servers in someone else's DC > Custom Hardware in custom built data centers. I think stackoverflow and its siblings proved that having a handful of servers can go a very long way, even making cloud ops redundant. Of course this is function of what you're optimizing for, and whether you want to go down the "boring monolithic app" route.
- Nextgrid 4y ago
  > Of course this is function of what you're optimizing for, and whether you want to go down the "boring monolithic app" route. Microservices do add some overhead but it's not extreme - even a microservices-based app can run just fine on a decent bare-metal box if you run all the services on it. Of course, the talk of microservices brings the question of what problem you're actually trying to solve - are you aiming to build a technical solution to a business problem or are you aiming to create an engineering playground so there's endless busywork and justification for hiring lots of engineers? If it's the latter, then bare-metal is going to be a bad option anyway as it's not the kind of toy a typical startup engineer wants to play with.
blackoil 4y ago
I would recommend starting with DigitalOcean. It may have little overhead but a much more cost effective and you can stay on it for longer before migrating.
buffalobuffalo 4y ago
As a software engineer who doesn't really like devops and has been in this position multiple times, I'm a huge fan of buying à la carte services from different providers that specialize in managing a specific type of service (often since they are the developer/maintainer of said service). As long as you make sure they are all in the same datacenter, you still get great performance. And typically minimal configuration woes. For example: datacenter - aws: us-east-2 Dockerized Webservers/task servers: Render or Engineyard Postgres & Kafka: aiven or 84codes Redis: Redis labs Unified logging Elastic or Grafana I still end up using some underlying AWS services like S3 and lambda, but it's a lot less work than managing an entire AWS ecosystem with security groups/VPC/networking etc.
- nijave 4y ago
  That can work well but you have to be careful/mindful of egress charges and latency (if the server supports co-locating in the same cloud or even managing directly in your account, it can alleviate those issues)
Nextgrid 4y ago
I don't see the point of public cloud. In practice, it still requires a sysadmin (now called "DevOps engineers") so it's not any better than rented bare-metal in terms of maintenance overhead, while still being extremely expensive. Use a managed PaaS to begin with (you pay more but it does genuinely save you time as there is no management overhead), then when you're ready to do things yourself go straight to hosted bare-metal, and only use public cloud services for their managed services that you can't replicate yourself (think Redshift/Athena/Aurora/etc).
- tatersolid 4y ago
  > so it's not any better than rented bare-metal in terms of maintenance overhead In my experience the maintenance overhead of the cloud is much lower. My dayjob (B2B SaaS) spent about 75% of the infrastructure team’s time on things like patching switch firmware, balancing UPS loads, diagnosing flaky switch ports or transceivers, managing logging growth, etc. None of that made our products better from a customer perspective. Since our cloud move those same infra staff support many more services and apps with much faster turnaround for product teams. And we traded upcoming multi-million capex investments in servers/switches/appliances into a monthly cloud bill that scales much more closely with revenue. The public cloud is for businesses constrained by people; we simply could not afford to hire enough people to do the same stuff on-prem or in colo.
  - Nextgrid 4y ago
    > patching switch firmware, balancing UPS loads, diagnosing flaky switch ports or transceivers, managing logging growth I've never had to worry about messing with switches/cabling/UPSes with my Hetzner or OVH servers. I'm not sure why people always believe that bare-metal == self-managed colo. That's not the case and colo only really makes sense for very large companies who can actually save by buying & managing their own hardware or have specific requirements that dedicated server providers don't offer. Log growth can be addressed by using a managed service (including AWS Cloudwatch if you really wanted to, but you'd have to be a masochist). Frankly, you'd have the same issue if you were on EC2.
  - locutous 4y ago
avrionov 4y ago
I don't think this is a good advice in general and I haven't seen many companies to do it practice. The initial decisions which a company makes most of the time are hard to change. Moving a successful business with clients from one platform to another 3 or 4 times is very difficult to do or hardly impossible. It may work for a simple website, but for any more complicated project with web clients, mobile clients third party integrations, migrating from Heroku to cloud provider to on prem means refactoring big parts of the project. What is even bigger problem a migration like this is hard to do incrementally.

dijit 4y ago

I really agree with you, what's weird though is how many mega-corps are going away from Custom Hardware in Custom Built DC towards Cloud. There's also something to be said for buying a VPS or a Colo machine, making sure it's backed up and dealing with the 9's that you get from that machine on it's own. I am routinely surprised by how far a single node machine will get you.

mwcampbell 4y ago
> how far a single node machine will get you. This. I took the wrong lesson from the DDoS attacks on Linode in late 2015 (particularly the one on Christmas Day), and the intermittent issues I encountered with DigitalOcean and Vultr in 2016 while both providers were still fairly young. A single dedicated server from a mature provider (ideally not during its hyper-growth phase) is pretty reliable.
sofixa 4y ago
> what's weird though is how many mega-corps are going away from Custom Hardware in Custom Built DC towards Cloud It costs a lot of money to run your own datacenters, and very very few companies are capable of doing it as good as AWS or even Scaleway/OVH can. By that I mean, waiting weeks/months to get through tickets, approvals, multiple different teams just to get a server deployed. Then waiting a few more weeks for monitoring/backups. Allowing developers and related to have hardware/software at a whim is a massive advantage.
- dijit 4y ago
  100% Agree. I don't like the idea that the only way to get developers moving is to use cloud, but I agree that it's a solid replacement for really bad ops. What I've seen in many places is an abstraction over bare metal, some are better than others, openstack, Kubernetes on-prem, vmware etc; are all solutions that have differing amounts of adoption. Ubisoft had a lot of stuff in this area, as does Google. Ubisofts was pretty terrible though. If you need a physical machine to be deployed, you've hit a certain level of scale and your load is much more known: and even though it can take a few weeks, what you get back is quite competitive. But if you're waiting for hardware to get anything moving in the first place then that's obviously bad. What I've taken to doing is prototyping on Google Cloud and then planning to migrate things to on-prem once everything is reaching maturity.
- dcoperator1 4y ago
  The price for colocating in a Tier 2 datacenter close enough for me to walk to was $75/amp. Which is about the same cost as AWS. You're really paying somewhere between the savings of putting a datacenter in Nowhere, Oregon and the cost to convince someone to live there.
  - nine_k 4y ago
    How can you directly compare $75 / amp to AWS? This very much depends on the kind if load you have. Also, AWS's egress likely costs much more than the datacenter's; again, possible savings very much depend on the kind of load you have.
  - wongarsu 4y ago
    I'd agree that unless you can really profit from having very specific hardware, you're better off renting dedicated servers than colocating servers you own. Have somebody else worry about having people on call to switch out failed hard drives. Comparing either to AWS will inevitably lead to a much more complex discussion about spot instances, traffic costs, ancillary services etc.
- kqr 4y ago
  If you have a little spare capacity, developers can still get hardware/software on a whim, at just a (comparatively) small one-time expense. Spare capacity is much cheaper than people make it out to be. The upside is that it's much cheaper once you're at the scale where you no longer need to variableise your compute costs, but can tank the up-front fixed costs and do proper capacity planning.
nine_k 4y ago
These megacorps likely have IT which is not mega enough to justify owning a massive datacenter. The right scale is Amazon, Google, Facebook, Microsoft. Likely much fewer than a hundred companies in the entire world.
PragmaticPulp 4y ago
> what's weird though is how many mega-corps are going away from Custom Hardware in Custom Built DC towards Cloud. Why is it surprising? Building and maintaining custom data centers is a big, slow business initiative. It takes months to years of forecasting to get the data center buildout to match the business needs, as opposed to the extreme flexibility of using a cloud provider. > There's also something to be said for buying a VPS or a Colo machine, making sure it's backed up and dealing with the 9's that you get from that machine on it's own. I am routinely surprised by how far a single node machine will get you. For personal projects this is exactly what I do. It’s great until something goes wrong with that one machine or VPS. But it’s not really a good option for any business that needs consistent operations and uptime. Years ago I worked at a company that tried to self-host some of their collaboration tools on a VPS to save money over the cloud-hosted versions. When the server went down it stalled productivity for a day while the team restored a backup, with another week of confusion as we tried to find all of the things that were lost between the last backup and when the server went down. When someone did the rough estimations on how much it cost to pay everyone’s salaries for that day of lost productivity, the number was far higher than the trivial cost savings we got from self-hosting. We also had a constant background burden on someone internally to maintain and monitor the server, plus the burden of them being on call. Often, moving to cloud anything can be a huge load off the company’s back.
- dijit 4y ago
  > as opposed to the extreme flexibility of using a cloud provider. I don't really buy this honestly. What you buy with cloud providers is quality tooling, not flexibility. If you're bin-packing with Kubernetes properly then capacity is capacity and it doesn't matter if the marketing department are using it or the developers are. You just buy a bunch of servers and when you see the load approaching 70% you buy more. It's a 2 person job. Is it harder? Yes. Definitely. Is it a panacea? No. Not at all. Is it universally cheaper? Also no. Definitely not. I feel like whenever I talk about the Cloud as an expensive thing that people get emotionally defensive. I'm not here to take your toys away. Services like cloud are just tools and tools always have pros and cons. If you can't reasonably discuss the con's without resorting to "I need to hire more staff" or "its a lot better than $strawman" then we're just cargo culting. > it’s not really a good option for any business that needs consistent operations and uptime. Most business cases for computers can just eat the downtime honestly. Your URL redirector doesn't need 5 9's. There's a grading scale of complexity and uptime, on one side you have a single hosted server that has profoundly strong uptime (especially with the redundancies in normal servers); then you start adding complexity to get HA, and weirdly: the complexity lowers the reliability. If you keep following the line of redundancies and HA complexity, eventually you can get to a point where the service is even more reliable than a single node. Which is what everyone assumes they will get straight away, but usually it's a lot of work to get there. > the trivial cost savings This will differ a lot. I made two games, one was hybrid-cloud and one was bare metal only; the cost savings were not trivial. If we had 100% clouded the hybrid deployment we would easily have paid 10x in the hosting costs which would have been enough money to pay for 250 contractors at a premium rate. That said: the toys were definitely shiny.
PubliusMI 4y ago
It's not weird. Many mega-corps are extremely bloated and dysfunctional. Their IT (Private) Cloud teams slower and less competent. With public cloud, a small team can be fully responsible for all their resources with crystal clear cost accounting.
P5fRxh5kUvp2th 4y ago
At a certain scale you have more negotiating power so it probably makes more sense from that perspective.
Fendii 4y ago
Every internal it department was always slower than any cloud offering. Or laged features. Or had underlying infra issues. With AWS in one Startup I was able to build and maintain infrastructure were you needed a small team just 10 years before.
jacobr1 4y ago
Data Residency. Scaling to dozens of global regions is not cost effective for running your own DCs.
- dijit 4y ago
  Sorry, the next step after cloud isn't running your own DC's. It's renting space from someone who is doing that for you. The scale of cost savings/ownership as you scale kinda goes: FaaS -> PaaS -> Cloud -> VPS -> Rented Hardware -> Rented space in DC -> Own DC. Let's avoid conflating everything left of VPS with the most difficult form of it, because nobody is going there from nothing.

osigurdson 4y ago

I think you can be pretty big and still stay in Public Cloud. There is enough competition that pricing should trend toward commodity over time.

buzzdenver 4y ago

> "use heroku until you have 100k users" Thing is that for most/many startups 100k users is not a lot. Rejiggling your basic infra just as your growth is starting to accelerate is a non-trivial task, a risk, and something that doesn't fundamentally move the needle.

hagbarth 4y ago
This really depends on what you are building of course. Working in enterprise SaaS with only a few users per account? You'll be doing really well at 100k users.
icedchai 4y ago
Most B2B startups never get anywhere near 100K users.
lelanthran 4y ago
> Thing is that for most/many startups 100k users is not a lot. Depends; if you're a startup offering free or ad-supported services and the exit plan is "be bought out by existing entrenched competitor", then, yes, 100k users is not enough to hit your goals. If you're a startup offering B2B services, even 10k users is enough to be madly profitable.

cultofmetatron 4y ago

> "use heroku until you have 100k users" nowadays, i'd say use fly.io or render till you have 200k users.

sakisv 4y ago

> Heroku/Dokku > Public Cloud >Dedicated servers in someone else's DC > Custom Hardware in custom built data centers. I think stackoverflow and its siblings proved that having a handful of servers can go a very long way, even making cloud ops redundant. Of course this is function of what you're optimizing for, and whether you want to go down the "boring monolithic app" route.

Nextgrid 4y ago
> Of course this is function of what you're optimizing for, and whether you want to go down the "boring monolithic app" route. Microservices do add some overhead but it's not extreme - even a microservices-based app can run just fine on a decent bare-metal box if you run all the services on it. Of course, the talk of microservices brings the question of what problem you're actually trying to solve - are you aiming to build a technical solution to a business problem or are you aiming to create an engineering playground so there's endless busywork and justification for hiring lots of engineers? If it's the latter, then bare-metal is going to be a bad option anyway as it's not the kind of toy a typical startup engineer wants to play with.

blackoil 4y ago

I would recommend starting with DigitalOcean. It may have little overhead but a much more cost effective and you can stay on it for longer before migrating.

buffalobuffalo 4y ago

As a software engineer who doesn't really like devops and has been in this position multiple times, I'm a huge fan of buying à la carte services from different providers that specialize in managing a specific type of service (often since they are the developer/maintainer of said service). As long as you make sure they are all in the same datacenter, you still get great performance. And typically minimal configuration woes. For example: datacenter - aws: us-east-2 Dockerized Webservers/task servers: Render or Engineyard Postgres & Kafka: aiven or 84codes Redis: Redis labs Unified logging Elastic or Grafana I still end up using some underlying AWS services like S3 and lambda, but it's a lot less work than managing an entire AWS ecosystem with security groups/VPC/networking etc.

nijave 4y ago
That can work well but you have to be careful/mindful of egress charges and latency (if the server supports co-locating in the same cloud or even managing directly in your account, it can alleviate those issues)

Nextgrid 4y ago

I don't see the point of public cloud. In practice, it still requires a sysadmin (now called "DevOps engineers") so it's not any better than rented bare-metal in terms of maintenance overhead, while still being extremely expensive. Use a managed PaaS to begin with (you pay more but it does genuinely save you time as there is no management overhead), then when you're ready to do things yourself go straight to hosted bare-metal, and only use public cloud services for their managed services that you can't replicate yourself (think Redshift/Athena/Aurora/etc).

tatersolid 4y ago
> so it's not any better than rented bare-metal in terms of maintenance overhead In my experience the maintenance overhead of the cloud is much lower. My dayjob (B2B SaaS) spent about 75% of the infrastructure team’s time on things like patching switch firmware, balancing UPS loads, diagnosing flaky switch ports or transceivers, managing logging growth, etc. None of that made our products better from a customer perspective. Since our cloud move those same infra staff support many more services and apps with much faster turnaround for product teams. And we traded upcoming multi-million capex investments in servers/switches/appliances into a monthly cloud bill that scales much more closely with revenue. The public cloud is for businesses constrained by people; we simply could not afford to hire enough people to do the same stuff on-prem or in colo.
- Nextgrid 4y ago
  > patching switch firmware, balancing UPS loads, diagnosing flaky switch ports or transceivers, managing logging growth I've never had to worry about messing with switches/cabling/UPSes with my Hetzner or OVH servers. I'm not sure why people always believe that bare-metal == self-managed colo. That's not the case and colo only really makes sense for very large companies who can actually save by buying & managing their own hardware or have specific requirements that dedicated server providers don't offer. Log growth can be addressed by using a managed service (including AWS Cloudwatch if you really wanted to, but you'd have to be a masochist). Frankly, you'd have the same issue if you were on EC2.
- locutous 4y ago

avrionov 4y ago

I don't think this is a good advice in general and I haven't seen many companies to do it practice. The initial decisions which a company makes most of the time are hard to change. Moving a successful business with clients from one platform to another 3 or 4 times is very difficult to do or hardly impossible. It may work for a simple website, but for any more complicated project with web clients, mobile clients third party integrations, migrating from Heroku to cloud provider to on prem means refactoring big parts of the project. What is even bigger problem a migration like this is hard to do incrementally.