6 ms·

Figma spends $300k on AWS daily

57 points by handfuloflight 1y ago

javierluraschi 1y ago
Any chance we can get a high-level breakdown of the AWS services they are paying for?
- pgwhalen 1y ago
  No. The figure is from their S-1 filing and further detail is not provided.
jbverschoor 1y ago
On what jeeeeeeezus
Neywiny 1y ago
I don't understand how it takes 3 years to get off the cloud. I'm not a cloud developer, though. The most I've done is run code on free hosts or compute instances. Presumably there's something to the microservices and lambdas and distributed compute that makes this hard. I'm thinking if this was a monolith (like AWS themselves admit is cheaper), they could just run it locally? What a giant waste of money. I'm very glad to start seeing xAAS start to die out. At the end of the day it's just looking like more middle-men instead of how I've always assumed it was intended to be: economies of scale. However, and missing from this article + discussion so far, is their revenue. If they pay $4/day and make $2 in revenue, that's bad. They pay $300k/day but make ~ $2250k/day in revenue. I don't know what the ratio is supposed to be, but at first blush that doesn't actually seem too bad. I'll let the more qualified take over, I'm struggling to find out how big a % of their total expenses this is.
- haiku2077 1y ago
  The services are the "easy" part; moving data out of a cloud provider is slow and expensive. For a _really_ big dataset it can take months, sometimes years, just to complete the data transfer.
- nevon 1y ago
  A mistake I see commonly whenever someone says to "just move off of the cloud" is that they see the cloud as just a VM provider. If it was, then yeah, moving to another provider wouldn't be such a big deal. In reality, the cloud creeps into your systems in all sorts of ways. Your permissions use cloud identities, your firewalls are based on security group referencing, your cross-region connectivity relies on cloud networking products, you're managing secrets and secret rotation using cloud secrets management, your observability is based on cloud metrics, your partners have whitelisted static ip ranges that belong to the cloud provider, your database upgrades are automated by the cloud provider, your VM images are built specifically for your cloud provider, your auditing is based on the cloud provider's logs, half the items in your security compliance audit reference things that are solved by your cloud provider, your applications are running on a container scheduler managed by your cloud provider, your serverless systems are strongly coupled distributed monoliths dependent on events on cloud specific event buses, your disaster recovery plans depend on your cloud provider's backup or region failover capabilities, etc. Not to mention that when you have several hundred systems, you're not going to be moving them all at the same time. They still need to be able to communicate during the transition period (extra fun when your service-to-service authentication is dependent on your cloud) without any downtime. It's not just a matter of dropping a server binary onto a VM from a different provider. If I think about how long it would take my org to move fully off of _a_ cloud (just to a different cloud with somewhat similar capabilities), 3 years doesn't sound unrealistic.
  - Nextgrid 1y ago
    Which is exactly why you don't make the mistake of relying on all that. If you can't run it locally, don't use it unless you have absolutely no choice.
  - motorest 1y ago
    > A mistake I see commonly whenever someone says to "just move off of the cloud" is that they see the cloud as just a VM provider. If it was, then yeah, moving to another provider wouldn't be such a big deal. I think it can still be a big deal depending on what's the overall system architecture, where are all data stores, how many services you run, and what constraints you're dealing with. For example, when you are between two cloud providers, more often than not you will have to replace internal calls with external calls, at least within the migration stage. That as important impacts on reliability and performance. In some scenarios, this performance impact is not acceptable and might require architecting services.
- fifilura 1y ago
  > I've always assumed it was intended to be: economies of scale IMO, the value proposition these days is rather to avoid maintenance. I.e. help with up with all the latest patches on your infrastructure.
  - Neywiny 1y ago
    To me it's the same thing. You can pay somebody to care about that, but they might be underutilized for the majority of time so it's not worth it. If you have a service, instead of your security expert being used idk 1/x of full time, they can be y/x where y is the number of contracts. For me and my time we are just way too small to have somebody full-time dedicated. So that's how I think about it
    - fifilura 1y ago
      It is a reasonable point. But i think it is not exactly that. Having your organisation focus on maintenance is a certain type of opportunity cost. It is pretty often one of your most knowledgeable engineer that does this. And it also interrupts the flow of many of your other engineers.
    - vidarh 1y ago
      All of these services - be it in or out of the cloud - are trivially available as service contract with external providers.
  - tonyedgecombe 1y ago
    I'd love to see some real figures on that. My gut feeling is most companies spend as much on AWS experts as they previously did on people running in house facilities but I really don't know.
    - icedchai 1y ago
      Probably more. Folks that do IAC / Terraform are more expensive than old school sysadmins that would write shell scripts.
  - 1y ago
- undebuggable 1y ago
  > I don't understand how it takes 3 years to get off the cloud. Because in fact cloud is not just someone else's computer.
  - icedchai 1y ago
    When you code your entire app to proprietary APIs like Lambda or DynamoDB, it becomes complex to migrate.
- vidarh 1y ago
  People dig themselves in hard, with reliances on all kinds of proprietary services, and complex relationships. My experience in helping people do cloud migrations are that companies also often quickly lose oversight over which cloud services they are even still running. Sometimes systems that should've been shut down years ago are still hanging around, or parts of them anyway, like S3 buckets etc. Most companies that use cloud systems underprovision their devops because they think they don't need much for a cloud system (in fact, they typically need more to do it well). $300k/day for their revenue is very much crazy high.
  - icedchai 1y ago
    During good times, many companies simply don't care about which services or how much. I've worked at several startups where they got a large funding round and the word was "we don't care about cloud costs, just get it done fast / make it scale." Unfortunately, one bad mistake (like storing all data in a proprietary service like DynamoDB) can make this difficult to unwind when things get bad...
- PunchyHamster 1y ago
  If you use cloud brand queue system, you not only need to run some servers to run the queue, you also need to rewrite all parts of the system that relied on cloud brand queue system into something else. Repeat it by a bunch of cloud services your app use and you're in for major rewrite. Cloud offers many, many "good enough" solutions for common problems in development and it is oh so easy to use another cloud api for few cents more And that's not even touching a bunch of auxiliary services you now need to provide like monitoring, backups, metrics etc. Usually best strategy is to start move from most expensive parts but if that part is entangled in cloud services it could take forever
andrewstuart 1y ago
Utterly bananas. Seriously the CTO should be fired.
lazystar 1y ago
edit - i read it wrong, removing my comment.
- kgc 1y ago
  Two different companies. Figma is not 37Signals.
  - lazystar 1y ago
    oh, woops. thanks for correcting me, will edit my comment.
samcat116 1y ago
Am I the only one that finds that not that crazy?
vidarh 1y ago
It's insane spends like that which are the main reason that reducing cloud costs is the main money maker for my consulting business at the moment. It's extremely rare to come across people using cloud services where you can't cut their costs by at least 30%, often more than half, sometimes as much as 90%+, and usually it results in reduced devops spends as well. The interesting thing is that the hardest part of the sales process is to convince people that cloud isn't cheap, because there's a near religious belief that cloud providers must be cost effective in some companies.
- handfuloflight 1y ago
  What percentage of clients do you think don't even need cloud?
  - vidarh 1y ago
    For the vast majority of them, rented servers at a managed hosting provider would beat cloud in price hands down, but for some the difference might be small enough once they've actually optimised their cloud setup that it's a judgement call. I usually recommend clients plan for optimizing their cloud setup first, both to get a fair comparison and also because a lot of the assessments you need to do of your cloud environment to do that will also make it easier to figure out what you'd pay for a managed hosting setup as well (e.g. understanding base load vs. spikes, durability requirements for different subsets of storage etc.) For some of them, a colo facility would be cheaper, but that's highly dependent on where you want to host it (e.g. I'm in London - putting things in a colo in London is really hard to make cost effective vs. renting servers somewhere with lower land costs; data centre operators are real-estate plays) However, you can usually make managed hosting/colo even cheaper by sprinkling some cloud in. E.g. a "trick" that can work amazingly well is to set up the bare minimum to let you spin up what you need to handle traffic spikes in a cloud environment, and then set up monitoring for your load balancer so that you start scaling into the cloud environment once load hits a certain level, but use only the managed hosting below that level. That way, you almost never end up actually spinning up cloud instances, but you gain the ability to run the managed hosting environment far closer to the wire than you could otherwise safely do, and drive your cost per request down accordingly.
wg0 1y ago
They need cloud storage. Object store. For design files. Some web sockets with CRDT processes running. What else?
- fulafel 1y ago
  Bandwidth and networking stuff is famously expensive there
puff_pastry 1y ago
This is absolutely bonkers, $110mln/year?? How’re they spending this much??
yogi4D 1y ago
It’s wild how someone can invent an idea, force it on everyone something no one asked for then turn around and sell it to us, layering on FOMO to keep us hooked. Emotional manipulation at its finest. No wonder nothing feels special anymore. Everything’s automated, abstracted, owned by someone else. If you make things by hand, you have to prove their value meanwhile, platforms like AWS are printing money. They want us to own nothing. And if “heaven” is real, why aren’t they there already?
- PunchyHamster 1y ago
  Sir, this is Wendy's
mattbillenstein 1y ago
This is basically technical debt of some sort - they may be minting so much money it doesn't matter atm, but eventually, someone will get some wins cutting costs. Probably just the bandwidth is a lot of it.
rkj93 1y ago
I see lot of comments questioning why can't they move to another provider. When you start using any of the cloud providers there so many services you start using and then you start building integrations, email notifications, api gateways, and what not. This makes it very difficult to move all the established processes.
- vilanye 1y ago
  This is why I don't use AWS or any "cloud" provider. That is not 100% true. I have 110 GB or so of personal data backed up on Glacier. The bill is 6 cents a month, and they obviously don't charge my card. It may take longer to start, but owning all your servers is cheaper in the long run. At the very least, don't lock yourself into one provider. Spread out any services you use. More than one company has screwed itself over permanently by tying itself to a single service. At the end of the day, it is the extremely poor decision to follow the trend of locking oneself into a single company that dooms it. You would think people would have learned the lesson to avoid lock-in by now. It is absolutely Darwinian at this point.

javierluraschi 1y ago

Any chance we can get a high-level breakdown of the AWS services they are paying for?

pgwhalen 1y ago
No. The figure is from their S-1 filing and further detail is not provided.

jbverschoor 1y ago

On what jeeeeeeezus

Neywiny 1y ago

I don't understand how it takes 3 years to get off the cloud. I'm not a cloud developer, though. The most I've done is run code on free hosts or compute instances. Presumably there's something to the microservices and lambdas and distributed compute that makes this hard. I'm thinking if this was a monolith (like AWS themselves admit is cheaper), they could just run it locally? What a giant waste of money. I'm very glad to start seeing xAAS start to die out. At the end of the day it's just looking like more middle-men instead of how I've always assumed it was intended to be: economies of scale. However, and missing from this article + discussion so far, is their revenue. If they pay $4/day and make $2 in revenue, that's bad. They pay $300k/day but make ~ $2250k/day in revenue. I don't know what the ratio is supposed to be, but at first blush that doesn't actually seem too bad. I'll let the more qualified take over, I'm struggling to find out how big a % of their total expenses this is.

haiku2077 1y ago
The services are the "easy" part; moving data out of a cloud provider is slow and expensive. For a _really_ big dataset it can take months, sometimes years, just to complete the data transfer.
nevon 1y ago
A mistake I see commonly whenever someone says to "just move off of the cloud" is that they see the cloud as just a VM provider. If it was, then yeah, moving to another provider wouldn't be such a big deal. In reality, the cloud creeps into your systems in all sorts of ways. Your permissions use cloud identities, your firewalls are based on security group referencing, your cross-region connectivity relies on cloud networking products, you're managing secrets and secret rotation using cloud secrets management, your observability is based on cloud metrics, your partners have whitelisted static ip ranges that belong to the cloud provider, your database upgrades are automated by the cloud provider, your VM images are built specifically for your cloud provider, your auditing is based on the cloud provider's logs, half the items in your security compliance audit reference things that are solved by your cloud provider, your applications are running on a container scheduler managed by your cloud provider, your serverless systems are strongly coupled distributed monoliths dependent on events on cloud specific event buses, your disaster recovery plans depend on your cloud provider's backup or region failover capabilities, etc. Not to mention that when you have several hundred systems, you're not going to be moving them all at the same time. They still need to be able to communicate during the transition period (extra fun when your service-to-service authentication is dependent on your cloud) without any downtime. It's not just a matter of dropping a server binary onto a VM from a different provider. If I think about how long it would take my org to move fully off of _a_ cloud (just to a different cloud with somewhat similar capabilities), 3 years doesn't sound unrealistic.
- Nextgrid 1y ago
  Which is exactly why you don't make the mistake of relying on all that. If you can't run it locally, don't use it unless you have absolutely no choice.
- motorest 1y ago
  > A mistake I see commonly whenever someone says to "just move off of the cloud" is that they see the cloud as just a VM provider. If it was, then yeah, moving to another provider wouldn't be such a big deal. I think it can still be a big deal depending on what's the overall system architecture, where are all data stores, how many services you run, and what constraints you're dealing with. For example, when you are between two cloud providers, more often than not you will have to replace internal calls with external calls, at least within the migration stage. That as important impacts on reliability and performance. In some scenarios, this performance impact is not acceptable and might require architecting services.
fifilura 1y ago
> I've always assumed it was intended to be: economies of scale IMO, the value proposition these days is rather to avoid maintenance. I.e. help with up with all the latest patches on your infrastructure.
- Neywiny 1y ago
  To me it's the same thing. You can pay somebody to care about that, but they might be underutilized for the majority of time so it's not worth it. If you have a service, instead of your security expert being used idk 1/x of full time, they can be y/x where y is the number of contracts. For me and my time we are just way too small to have somebody full-time dedicated. So that's how I think about it
  - fifilura 1y ago
    It is a reasonable point. But i think it is not exactly that. Having your organisation focus on maintenance is a certain type of opportunity cost. It is pretty often one of your most knowledgeable engineer that does this. And it also interrupts the flow of many of your other engineers.
  - vidarh 1y ago
    All of these services - be it in or out of the cloud - are trivially available as service contract with external providers.
- tonyedgecombe 1y ago
  I'd love to see some real figures on that. My gut feeling is most companies spend as much on AWS experts as they previously did on people running in house facilities but I really don't know.
  - icedchai 1y ago
    Probably more. Folks that do IAC / Terraform are more expensive than old school sysadmins that would write shell scripts.
- 1y ago
undebuggable 1y ago
> I don't understand how it takes 3 years to get off the cloud. Because in fact cloud is not just someone else's computer.
- icedchai 1y ago
  When you code your entire app to proprietary APIs like Lambda or DynamoDB, it becomes complex to migrate.
vidarh 1y ago
People dig themselves in hard, with reliances on all kinds of proprietary services, and complex relationships. My experience in helping people do cloud migrations are that companies also often quickly lose oversight over which cloud services they are even still running. Sometimes systems that should've been shut down years ago are still hanging around, or parts of them anyway, like S3 buckets etc. Most companies that use cloud systems underprovision their devops because they think they don't need much for a cloud system (in fact, they typically need more to do it well). $300k/day for their revenue is very much crazy high.
- icedchai 1y ago
  During good times, many companies simply don't care about which services or how much. I've worked at several startups where they got a large funding round and the word was "we don't care about cloud costs, just get it done fast / make it scale." Unfortunately, one bad mistake (like storing all data in a proprietary service like DynamoDB) can make this difficult to unwind when things get bad...
PunchyHamster 1y ago
If you use cloud brand queue system, you not only need to run some servers to run the queue, you also need to rewrite all parts of the system that relied on cloud brand queue system into something else. Repeat it by a bunch of cloud services your app use and you're in for major rewrite. Cloud offers many, many "good enough" solutions for common problems in development and it is oh so easy to use another cloud api for few cents more And that's not even touching a bunch of auxiliary services you now need to provide like monitoring, backups, metrics etc. Usually best strategy is to start move from most expensive parts but if that part is entangled in cloud services it could take forever

andrewstuart 1y ago

Utterly bananas. Seriously the CTO should be fired.

lazystar 1y ago

edit - i read it wrong, removing my comment.

kgc 1y ago
Two different companies. Figma is not 37Signals.
- lazystar 1y ago
  oh, woops. thanks for correcting me, will edit my comment.

samcat116 1y ago

Am I the only one that finds that not that crazy?

vidarh 1y ago

It's insane spends like that which are the main reason that reducing cloud costs is the main money maker for my consulting business at the moment. It's extremely rare to come across people using cloud services where you can't cut their costs by at least 30%, often more than half, sometimes as much as 90%+, and usually it results in reduced devops spends as well. The interesting thing is that the hardest part of the sales process is to convince people that cloud isn't cheap, because there's a near religious belief that cloud providers must be cost effective in some companies.

handfuloflight 1y ago
What percentage of clients do you think don't even need cloud?
- vidarh 1y ago
  For the vast majority of them, rented servers at a managed hosting provider would beat cloud in price hands down, but for some the difference might be small enough once they've actually optimised their cloud setup that it's a judgement call. I usually recommend clients plan for optimizing their cloud setup first, both to get a fair comparison and also because a lot of the assessments you need to do of your cloud environment to do that will also make it easier to figure out what you'd pay for a managed hosting setup as well (e.g. understanding base load vs. spikes, durability requirements for different subsets of storage etc.) For some of them, a colo facility would be cheaper, but that's highly dependent on where you want to host it (e.g. I'm in London - putting things in a colo in London is really hard to make cost effective vs. renting servers somewhere with lower land costs; data centre operators are real-estate plays) However, you can usually make managed hosting/colo even cheaper by sprinkling some cloud in. E.g. a "trick" that can work amazingly well is to set up the bare minimum to let you spin up what you need to handle traffic spikes in a cloud environment, and then set up monitoring for your load balancer so that you start scaling into the cloud environment once load hits a certain level, but use only the managed hosting below that level. That way, you almost never end up actually spinning up cloud instances, but you gain the ability to run the managed hosting environment far closer to the wire than you could otherwise safely do, and drive your cost per request down accordingly.

wg0 1y ago

They need cloud storage. Object store. For design files. Some web sockets with CRDT processes running. What else?

fulafel 1y ago
Bandwidth and networking stuff is famously expensive there

puff_pastry 1y ago

This is absolutely bonkers, $110mln/year?? How’re they spending this much??

yogi4D 1y ago

It’s wild how someone can invent an idea, force it on everyone something no one asked for then turn around and sell it to us, layering on FOMO to keep us hooked. Emotional manipulation at its finest. No wonder nothing feels special anymore. Everything’s automated, abstracted, owned by someone else. If you make things by hand, you have to prove their value meanwhile, platforms like AWS are printing money. They want us to own nothing. And if “heaven” is real, why aren’t they there already?

PunchyHamster 1y ago
Sir, this is Wendy's

mattbillenstein 1y ago

This is basically technical debt of some sort - they may be minting so much money it doesn't matter atm, but eventually, someone will get some wins cutting costs. Probably just the bandwidth is a lot of it.

rkj93 1y ago

I see lot of comments questioning why can't they move to another provider. When you start using any of the cloud providers there so many services you start using and then you start building integrations, email notifications, api gateways, and what not. This makes it very difficult to move all the established processes.

vilanye 1y ago
This is why I don't use AWS or any "cloud" provider. That is not 100% true. I have 110 GB or so of personal data backed up on Glacier. The bill is 6 cents a month, and they obviously don't charge my card. It may take longer to start, but owning all your servers is cheaper in the long run. At the very least, don't lock yourself into one provider. Spread out any services you use. More than one company has screwed itself over permanently by tying itself to a single service. At the end of the day, it is the extremely poor decision to follow the trend of locking oneself into a single company that dooms it. You would think people would have learned the lesson to avoid lock-in by now. It is absolutely Darwinian at this point.