7 ms·

Of course you reduced 90% of the cost. Most of these costs don't come from the software, but from the people and automation maintaining it. With that cost redu

by jusomg 2y ago

Of course you reduced 90% of the cost. Most of these costs don't come from the software, but from the people and automation maintaining it. With that cost reduction you also removed monitoring of the platform, people oncall to fix issues that appear, upgrades, continuous improvements, etc. Who/What is going to be doing that on this new platform and how much does that cost? Now you need to maintain k8s, postgresql, elasticsearch, redis, secret managements, OSs, storage... These are complex systems that require people understanding how they internally work, how they scale and common pitfalls. Who is going to upgrade kubernetes when they release a new version that has breaking changes? What happens when Elasticsearch decides to splitbrain and your search stops working? When the DB goes down or you need to set up replication? What is monitoring replication lag? Or even simply things like disks being close to full? What is acting on that? I don't mean to say Heroku is fairly priced (I honestly have no idea) but this comparison is not apples to apples. You could have your team focused on your product before. Now you need people dedicated to work on this stuff.

matus_congrady 2y ago
Since DHH has been promoting the 'do-it-yourself' approach, many people have fallen for it. You're asking the right questions that only a few people know they need answers to. In my opinion, the closest thing to "reclaiming the stack" while still being a PaaS is to use a "deploy to your cloud account" PaaS provider. These services offer the convenience of a PaaS provider, yet allow you to "eject" to using the cloud provider on your own should your use case evolve. Example services include https://stacktape.com https://stacktape.com, https://flightcontrol.dev https://flightcontrol.dev, and https://www.withcoherence.com https://www.withcoherence.com. I'm also working on a PaaS comparison site at https://paascout.io https://paascout.io. Disclosure: I am a founder of Stacktape.
dzikimarian 2y ago
Sorry, but that's just ton of FUD. We run both private cloud and (for a few customers) AWS. Of course you have more maintenance on on-prem, but typical k8s update is maybe a few hours of work, when you know what you are doing. Also AWS is also, complex, also requires configuration and also generates alerts in the middle of the night. It's still a lot cheaper than managed service.
- filleokus 2y ago
  As with everything it's not black or white, but rather a spectrum. Sure, updating k8s is not that bad, but operating a distributed storage solution is no joke. Or really anything that requires persistence and clustering (like elastic). You can also trade operational complexity for cash via support contracts and/or enterprise solutions (like just throwing money at Hitachi for storage rather than trying to keep Ceph alive).
  - p_l 2y ago
    If you don't need something crazy you can just grab what a lot of enterprises already had done for years, which is drop a few big storage servers and call it a day, connecting over iSCSI/NFS/whatever
    - filleokus 2y ago
      If you are in Kubernetes land you probably want object storage and some kind of PVC provider. Not thaaat different from an old fashioned iSCSI/NFS setup to be honest, but in my experience different enough to cause friction in an enterprise setting. You really don't want a ticket-driven, manual, provisioning process of shares
      - p_l 2y ago
        a PVC provider is nice, sure, but depending on how much you need/want simplest cases can be "mount a subdirectory from common exported volume", and for many applications ticket-based provisioning will be enough. That said on my todo-list is some tooling to make simple cases with linux NFS or SMI-capable servers work as PVC providers.
- tinco 2y ago
  Sure, but it requires that your engineers are vertically capable. In my experience, about 1 in 5 developers has the required experience and does not flat out refuse to have vertical responsibility over their software stack. And that number might be high, in larger more established companies there might be more engineers who want to stick to their comfort bubble. So many developers reject the idea of writing SQL themselves instead of having the ORM do it, let alone know how to configure replication and failover. I'd maybe hire for the people who could and would, but the people advocating for just having the cloud take care of these things have a point. You might miss out on an excellent application engineer, if you reject them for not having any Linux skills.
Nextgrid 2y ago
This is FUD unless you're running a stock exchange or payment processor where every minute of downtime will cost you hundreds of thousands. For most businesses this is fear-mongering to keep the DevOps & cloud industry going and ensure continued careers in this field.
- The_Colonel 2y ago
  It's not just about downtime, but also about not getting your systems hacked, not losing your data if sh1t hits the fan, regulation compliance, flexibility (e.g. ability to quickly spin-out new test envs) etc. My preferred solution to this problem is different, though. For most businesses, apps, a monolith (maybe with a few extra services) + 1 relational DB is all you need. In such a simple setup, many of the problems faced either disappear or get much smaller.
  - packetlost 2y ago
    > also about not getting your systems hacked... The only systems I have ever seen get compromised firsthand were in public clouds and because they were in public clouds. Most of my career has been at shops that, for one reason or another, primarily own their own infrastructure, cloud represents a rather small fraction. It's far easier to secure a few servers behind a firewall than figure out the Rube Goldberg Machine that is cloud configuration. > not losing your data if sh1t hits the fan You can use off-site backup without using cloud systems, you know? Backblaze, AWS Glacier, etc. are all pretty reasonable solutions. Most of the time when I've seen the need to exercise the backup strategy it's because of some software fuckup, not something like a disk dying. Using a managed database isn't going to save you when the intern TRUNCATEs the prod database on accident (and if something like that happens, it means you fucked up elsewhere). > regulation compliance Most shops would be way better suited to paying a payment processor like Stripe, or other equivalent vendors for similarly protected data. Defense is a whole can of worms, "government clouds" are a scam that make you more vulnerable to an unauthorized export than less. > flexibility (e.g. ability to quickly spin-out new test envs) etc. You actually lose flexibility by buying into a particular cloud provider, not gain it. Some things become easier, but many things become harder. Also, IME the hard part of creating reasonable test envs is configuring your edge (ingress, logging infra) and data.
tucnak 2y ago
[flagged]
- jusomg 2y ago
  Not sure if this is going to help Heroku's people at all but I feel bad for them now! haha I'm not a Heroku employee. I don't even work in any sort of managed service / platform provider. This is indeed a new account but not a throwaway account! I intended to use it long term.
- almost 2y ago
  You really think that, incredibly lukewarm, argument for Heroku is so extreme that it could only have been written by some kind of undercover shill?
  - tucnak 2y ago
    Why, yes?
    - cpach 2y ago
      Please don’t do this. It’s against HN’s guidelines. Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
dbackeus 2y ago
Original creator and maintainer of Reclaim the Stack here. > you also removed monitoring of the platform No we did not: Monitoring: https://reclaim-the-stack.com/docs/platform-components/monitoring https://reclaim-the-stack.com/docs/platform-components/monit... Log aggregation: https://reclaim-the-stack.com/docs/platform-components/log-aggregation https://reclaim-the-stack.com/docs/platform-components/log-a... Observability is on the whole better than what we had at Heroku since we now have direct access to realtime resource consumption of all infrastructure parts. We also have infinite log retention which would have been prohibitively expensive using Heroku logging addons (though we cap retention at 12 months for GDPR reasons). > Who/What is going to be doing that on this new platform and how much does that cost? Me and my colleague who created the tool together manage infrastructure / OS upgrades and look into issues etc. So far we've been in production 1.5 years on this platform. On average we spent perhaps 3 days per month doing platform related work (mostly software upgrades). The rest we spend on full stack application development. The hypothesis for migrating to Kubernetes was that the available database operators would be robust enough to automate all common high availability / backup / disaster recovery issues. This has proven to be true, apart from the Redis operator which has been our only pain point from a software point of view so far. We are currently rolling out a replacement approach using our own Kubernetes templates instead of relying on an operator at all for Redis. > Now you need to maintain k8s, postgresql, elasticsearch, redis, secret managements, OSs, storage... These are complex systems that require people understanding how they internally work Thanks to Talos Linux (https://www.talos.dev/ https://www.talos.dev/), maintaining K8s has been a non issue. Running databases via operators has been a non issue, apart from Redis. Secret management via sealed secrets + CLI tooling has been a non issue (https://reclaim-the-stack.com/docs/platform-components/secrets-management https://reclaim-the-stack.com/docs/platform-components/secre...) OS management with Talos Linux has been a learning curve but not too bad. We built talos-manager to manage bootstrapping new nodes to our cluster straight forward (https://reclaim-the-stack.com/docs/talos-manager/introduction https://reclaim-the-stack.com/docs/talos-manager/introductio...). The only remaining OS related maintenance is OS upgrades, which requires rebooting servers, but that's about it. For storage we chose to go with simple local storage instead of complicated network based storage (https://reclaim-the-stack.com/docs/platform-components/persistent-storage https://reclaim-the-stack.com/docs/platform-components/persi...). Our servers come with datacenter grade NVMe drives. All our databases are replicated across multiple servers so we can gracefully deal with failures, should they occur. > Who is going to upgrade kubernetes when they release a new version that has breaking changes? Ugrading kubernetes in general can be done with 0 downtime and is handled by a single talosctl CLI command. Breaking changes in K8s implies changes to existing resource manifest schemas and are detected by tooling before upgrades occur. Given how stable Kubernetes resource schemas are and how averse the community is to push breaking changes I don't expect this to cause major issues going forward. But of course software upgrades will always require due diligence and can sometimes be time consuming, K8s is no exception. > What happens when ElasticSearch decides to splitbrain and your search stops working? ElasticSearch, since major version 7, should not enter split brain if correctly deployed across 3 or more nodes. That said, in case of a complete disaster we could either rebuild our index from source of truth (Postgres) or do disaster recovery from off site backups. It's not like using ElasticCloud protects against these things in any meaningfully different way. However, the feedback loop of contacting support would be slower. > When the DB goes down or you need to set up replication? Operators handle failovers. If we would lose all replicas in a major disaster event we would have to recover from off site backups. Same rules would apply for managed databases. > What is monitoring replication lag? For Postgres, which is our only critical data source. Replication lag monitoring + alerting is built into the operator. It should be straight forward to add this for Redis and ElasticSearch as well. > Or even simply things like disks being close to full? Disk space monitoring and alerting is built into our monitoring stack. At the end of the day I can only describe to you the facts of our experience. We have reduced costs to cover hiring about 4 full time DevOps people so far. But we have hired 0 new engineers and are managing fine with just a few days of additional platform maintenance per month. That said, we're not trying to make the point that EVERYONE should Reclaim the Stack. We documented our thoughts about it here: https://reclaim-the-stack.com/docs/kubernetes-platform/introduction#is-it-for-you https://reclaim-the-stack.com/docs/kubernetes-platform/intro...
almost 2y ago
The fact that HN seems to think this is "FUD" is absolutely wild. You just talked about (some of) the tradeoffs involved in running all this stuff yourself. Obviously for some people it'll be worth and for others not, but absolutely amazing that there are people who don't even seem to accept that those tradoffs exist!
- dzikimarian 2y ago
  I assume you reference my comment. The reason I think parent comment is FUD isn't because I don't acknowledge tradeoffs (they are very real). It's because parent comment implies that people behind "reclaim the stack" didn't account for the monitoring, people's cost etc. Obviously any reasonable person making that decision includes it into calculation. Obviously nobody sane throws entire monitoring out of the window for savings. Accounting for all of these it can be still viable and significantly cheaper to run own infra. Especially if you operate outside of the US and you're able to eat an initial investment.
  - almost 2y ago
    Not your comment specifically, you're one of many saying FUD. Honestly if you accept that the comment was talking about real tradeoffs then I'm a bit baffled that you though it was FUD. It seems like an important thing to be talking about when there's a post advocating moving away from PaaS and doing it all yourself. It's great if you already knew all about all that and didn't need to discuss it, but just stare into the abyss of the other comments and you'll see that others very much don't understand those tradeoffs at all.
mlinhares 2y ago
Anything you don't know about managing these systems can be learned asking chatgpt :P Whenever I see people doing something like this I remember I did the same when I was in 10 people startups and it required A LOT of work to keep all these things running (mostly because back then we didn't have all these cloud managed systems) and that time would have been better invested in the product instead of wasting time figuring out how these tools work. I see value in this kind of work if you're at the scale of something like Dropbox and moving from S3 will greatly improve your bottom line and you have a team that knows exactly what they're doing and will be assigned the maintenance of this work. If this is being done merely from a cost cutting perspective and you don't have the people that understand these systems, its a recipe for disaster and once shit is on fire the people that would be assigned to "fix" the problem will quickly disappear because the "on call schedule is insane".
- deleted 2y ago
  [deleted]
- re-thc 2y ago
  > and that time would have been better invested in the product instead of wasting time figuring out how these tools work It really depends on what you're doing. Back then a lot of non-VC startups worked better and the savings possibly helped. It also helps grow the team and have less reliance on the vendor. It's long term value. Is it really time wasted? People often go into resume building mode and do all kinds of wacky things regardless. Perhaps this just helps scratch that itch.
  - mlinhares 2y ago
    Definitely fine from a personal perspective and resume building, it's just not in the best interest of the business because as soon as the person doing resume building is finished they'll jump ship. I've definitely done this myself. But i don't see this being good from a pure business perspective.
    - re-thc 2y ago
ugh123 2y ago
> you also removed monitoring of the platform You don't think they have any monitoring within Kubernetes? I imagine they have more monitoring capabilities now than they did with Heroku.
johnnyanmac 2y ago
>Who/What is going to be doing that on this new platform and how much does that cost? If you're already a web platform with hired talent (and someone using Heroku for a SaaS probably already is), I'd be surprised if the marginal cost was 10x.that paid support is of course coming at a premium, and isn't too flexible on what level of support you need. And yeah, it isn't apples to apples. Maybe you are in a low CoL area and can find a decent DevOps for 80-100k. Maybe you're in SF and any extra dev will be 250k. It'll vary immensely on cost.
kmacdough 2y ago
Exactly. It all depends on your needs and — to be honest — the quality of your sysops engineering. You may not only need dedicated sysops, but you may incur higher incidental costs with lost productivity when your solution inevitably goes down (or just from extra dev load when things are harder to use). That said, at least in 2016 Heroku was way overpriced for high volume sites. My startup of 10 engineers w/ 1M monthly active users saved 300k+/yr switching off heroku. But we had Jerry. Jerry was a beast and did most of the migration work in a month, with some dead-simple AWS scaling. His solution lacked many of the features of Heroku, but it massively reduced costs for developers running full test stacks which, in turn increased internal productivity. And did I mention it was dead simple? It's hard to overstate how valuable this was for the rest of us, who could easily grok the inner workings and know the consequences of our decisions. Perhaps this stack will open that opportunity to less equipped startups, but I've found few open source "drop-in replacements" to be truly drop-in. And I've never found k3 to be dead simple.
cryptonector 2y ago
Who says they reduced costs by cutting staff? They could instead have scaled their staff better.

by jusomg 2y ago

matus_congrady 2y ago

Since DHH has been promoting the 'do-it-yourself' approach, many people have fallen for it. You're asking the right questions that only a few people know they need answers to. In my opinion, the closest thing to "reclaiming the stack" while still being a PaaS is to use a "deploy to your cloud account" PaaS provider. These services offer the convenience of a PaaS provider, yet allow you to "eject" to using the cloud provider on your own should your use case evolve. Example services include https://stacktape.com https://stacktape.com, https://flightcontrol.dev https://flightcontrol.dev, and https://www.withcoherence.com https://www.withcoherence.com. I'm also working on a PaaS comparison site at https://paascout.io https://paascout.io. Disclosure: I am a founder of Stacktape.

dzikimarian 2y ago

Sorry, but that's just ton of FUD. We run both private cloud and (for a few customers) AWS. Of course you have more maintenance on on-prem, but typical k8s update is maybe a few hours of work, when you know what you are doing. Also AWS is also, complex, also requires configuration and also generates alerts in the middle of the night. It's still a lot cheaper than managed service.

filleokus 2y ago
As with everything it's not black or white, but rather a spectrum. Sure, updating k8s is not that bad, but operating a distributed storage solution is no joke. Or really anything that requires persistence and clustering (like elastic). You can also trade operational complexity for cash via support contracts and/or enterprise solutions (like just throwing money at Hitachi for storage rather than trying to keep Ceph alive).
- p_l 2y ago
  If you don't need something crazy you can just grab what a lot of enterprises already had done for years, which is drop a few big storage servers and call it a day, connecting over iSCSI/NFS/whatever
  - filleokus 2y ago
    If you are in Kubernetes land you probably want object storage and some kind of PVC provider. Not thaaat different from an old fashioned iSCSI/NFS setup to be honest, but in my experience different enough to cause friction in an enterprise setting. You really don't want a ticket-driven, manual, provisioning process of shares
    - p_l 2y ago
      a PVC provider is nice, sure, but depending on how much you need/want simplest cases can be "mount a subdirectory from common exported volume", and for many applications ticket-based provisioning will be enough. That said on my todo-list is some tooling to make simple cases with linux NFS or SMI-capable servers work as PVC providers.
tinco 2y ago
Sure, but it requires that your engineers are vertically capable. In my experience, about 1 in 5 developers has the required experience and does not flat out refuse to have vertical responsibility over their software stack. And that number might be high, in larger more established companies there might be more engineers who want to stick to their comfort bubble. So many developers reject the idea of writing SQL themselves instead of having the ORM do it, let alone know how to configure replication and failover. I'd maybe hire for the people who could and would, but the people advocating for just having the cloud take care of these things have a point. You might miss out on an excellent application engineer, if you reject them for not having any Linux skills.

Nextgrid 2y ago

This is FUD unless you're running a stock exchange or payment processor where every minute of downtime will cost you hundreds of thousands. For most businesses this is fear-mongering to keep the DevOps & cloud industry going and ensure continued careers in this field.

The_Colonel 2y ago
It's not just about downtime, but also about not getting your systems hacked, not losing your data if sh1t hits the fan, regulation compliance, flexibility (e.g. ability to quickly spin-out new test envs) etc. My preferred solution to this problem is different, though. For most businesses, apps, a monolith (maybe with a few extra services) + 1 relational DB is all you need. In such a simple setup, many of the problems faced either disappear or get much smaller.
- packetlost 2y ago
  > also about not getting your systems hacked... The only systems I have ever seen get compromised firsthand were in public clouds and because they were in public clouds. Most of my career has been at shops that, for one reason or another, primarily own their own infrastructure, cloud represents a rather small fraction. It's far easier to secure a few servers behind a firewall than figure out the Rube Goldberg Machine that is cloud configuration. > not losing your data if sh1t hits the fan You can use off-site backup without using cloud systems, you know? Backblaze, AWS Glacier, etc. are all pretty reasonable solutions. Most of the time when I've seen the need to exercise the backup strategy it's because of some software fuckup, not something like a disk dying. Using a managed database isn't going to save you when the intern TRUNCATEs the prod database on accident (and if something like that happens, it means you fucked up elsewhere). > regulation compliance Most shops would be way better suited to paying a payment processor like Stripe, or other equivalent vendors for similarly protected data. Defense is a whole can of worms, "government clouds" are a scam that make you more vulnerable to an unauthorized export than less. > flexibility (e.g. ability to quickly spin-out new test envs) etc. You actually lose flexibility by buying into a particular cloud provider, not gain it. Some things become easier, but many things become harder. Also, IME the hard part of creating reasonable test envs is configuring your edge (ingress, logging infra) and data.

tucnak 2y ago

[flagged]

jusomg 2y ago
Not sure if this is going to help Heroku's people at all but I feel bad for them now! haha I'm not a Heroku employee. I don't even work in any sort of managed service / platform provider. This is indeed a new account but not a throwaway account! I intended to use it long term.
almost 2y ago
You really think that, incredibly lukewarm, argument for Heroku is so extreme that it could only have been written by some kind of undercover shill?
- tucnak 2y ago
  Why, yes?
  - cpach 2y ago
    Please don’t do this. It’s against HN’s guidelines. Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

dbackeus 2y ago

Original creator and maintainer of Reclaim the Stack here. > you also removed monitoring of the platform No we did not: Monitoring: https://reclaim-the-stack.com/docs/platform-components/monitoring https://reclaim-the-stack.com/docs/platform-components/monit... Log aggregation: https://reclaim-the-stack.com/docs/platform-components/log-aggregation https://reclaim-the-stack.com/docs/platform-components/log-a... Observability is on the whole better than what we had at Heroku since we now have direct access to realtime resource consumption of all infrastructure parts. We also have infinite log retention which would have been prohibitively expensive using Heroku logging addons (though we cap retention at 12 months for GDPR reasons). > Who/What is going to be doing that on this new platform and how much does that cost? Me and my colleague who created the tool together manage infrastructure / OS upgrades and look into issues etc. So far we've been in production 1.5 years on this platform. On average we spent perhaps 3 days per month doing platform related work (mostly software upgrades). The rest we spend on full stack application development. The hypothesis for migrating to Kubernetes was that the available database operators would be robust enough to automate all common high availability / backup / disaster recovery issues. This has proven to be true, apart from the Redis operator which has been our only pain point from a software point of view so far. We are currently rolling out a replacement approach using our own Kubernetes templates instead of relying on an operator at all for Redis. > Now you need to maintain k8s, postgresql, elasticsearch, redis, secret managements, OSs, storage... These are complex systems that require people understanding how they internally work Thanks to Talos Linux (https://www.talos.dev/ https://www.talos.dev/), maintaining K8s has been a non issue. Running databases via operators has been a non issue, apart from Redis. Secret management via sealed secrets + CLI tooling has been a non issue (https://reclaim-the-stack.com/docs/platform-components/secrets-management https://reclaim-the-stack.com/docs/platform-components/secre...) OS management with Talos Linux has been a learning curve but not too bad. We built talos-manager to manage bootstrapping new nodes to our cluster straight forward (https://reclaim-the-stack.com/docs/talos-manager/introduction https://reclaim-the-stack.com/docs/talos-manager/introductio...). The only remaining OS related maintenance is OS upgrades, which requires rebooting servers, but that's about it. For storage we chose to go with simple local storage instead of complicated network based storage (https://reclaim-the-stack.com/docs/platform-components/persistent-storage https://reclaim-the-stack.com/docs/platform-components/persi...). Our servers come with datacenter grade NVMe drives. All our databases are replicated across multiple servers so we can gracefully deal with failures, should they occur. > Who is going to upgrade kubernetes when they release a new version that has breaking changes? Ugrading kubernetes in general can be done with 0 downtime and is handled by a single talosctl CLI command. Breaking changes in K8s implies changes to existing resource manifest schemas and are detected by tooling before upgrades occur. Given how stable Kubernetes resource schemas are and how averse the community is to push breaking changes I don't expect this to cause major issues going forward. But of course software upgrades will always require due diligence and can sometimes be time consuming, K8s is no exception. > What happens when ElasticSearch decides to splitbrain and your search stops working? ElasticSearch, since major version 7, should not enter split brain if correctly deployed across 3 or more nodes. That said, in case of a complete disaster we could either rebuild our index from source of truth (Postgres) or do disaster recovery from off site backups. It's not like using ElasticCloud protects against these things in any meaningfully different way. However, the feedback loop of contacting support would be slower. > When the DB goes down or you need to set up replication? Operators handle failovers. If we would lose all replicas in a major disaster event we would have to recover from off site backups. Same rules would apply for managed databases. > What is monitoring replication lag? For Postgres, which is our only critical data source. Replication lag monitoring + alerting is built into the operator. It should be straight forward to add this for Redis and ElasticSearch as well. > Or even simply things like disks being close to full? Disk space monitoring and alerting is built into our monitoring stack. At the end of the day I can only describe to you the facts of our experience. We have reduced costs to cover hiring about 4 full time DevOps people so far. But we have hired 0 new engineers and are managing fine with just a few days of additional platform maintenance per month. That said, we're not trying to make the point that EVERYONE should Reclaim the Stack. We documented our thoughts about it here: https://reclaim-the-stack.com/docs/kubernetes-platform/introduction#is-it-for-you https://reclaim-the-stack.com/docs/kubernetes-platform/intro...

almost 2y ago

The fact that HN seems to think this is "FUD" is absolutely wild. You just talked about (some of) the tradeoffs involved in running all this stuff yourself. Obviously for some people it'll be worth and for others not, but absolutely amazing that there are people who don't even seem to accept that those tradoffs exist!

dzikimarian 2y ago
I assume you reference my comment. The reason I think parent comment is FUD isn't because I don't acknowledge tradeoffs (they are very real). It's because parent comment implies that people behind "reclaim the stack" didn't account for the monitoring, people's cost etc. Obviously any reasonable person making that decision includes it into calculation. Obviously nobody sane throws entire monitoring out of the window for savings. Accounting for all of these it can be still viable and significantly cheaper to run own infra. Especially if you operate outside of the US and you're able to eat an initial investment.
- almost 2y ago
  Not your comment specifically, you're one of many saying FUD. Honestly if you accept that the comment was talking about real tradeoffs then I'm a bit baffled that you though it was FUD. It seems like an important thing to be talking about when there's a post advocating moving away from PaaS and doing it all yourself. It's great if you already knew all about all that and didn't need to discuss it, but just stare into the abyss of the other comments and you'll see that others very much don't understand those tradeoffs at all.

mlinhares 2y ago

Anything you don't know about managing these systems can be learned asking chatgpt :P Whenever I see people doing something like this I remember I did the same when I was in 10 people startups and it required A LOT of work to keep all these things running (mostly because back then we didn't have all these cloud managed systems) and that time would have been better invested in the product instead of wasting time figuring out how these tools work. I see value in this kind of work if you're at the scale of something like Dropbox and moving from S3 will greatly improve your bottom line and you have a team that knows exactly what they're doing and will be assigned the maintenance of this work. If this is being done merely from a cost cutting perspective and you don't have the people that understand these systems, its a recipe for disaster and once shit is on fire the people that would be assigned to "fix" the problem will quickly disappear because the "on call schedule is insane".

deleted 2y ago
[deleted]
re-thc 2y ago
> and that time would have been better invested in the product instead of wasting time figuring out how these tools work It really depends on what you're doing. Back then a lot of non-VC startups worked better and the savings possibly helped. It also helps grow the team and have less reliance on the vendor. It's long term value. Is it really time wasted? People often go into resume building mode and do all kinds of wacky things regardless. Perhaps this just helps scratch that itch.
- mlinhares 2y ago
  Definitely fine from a personal perspective and resume building, it's just not in the best interest of the business because as soon as the person doing resume building is finished they'll jump ship. I've definitely done this myself. But i don't see this being good from a pure business perspective.
  - re-thc 2y ago

ugh123 2y ago

> you also removed monitoring of the platform You don't think they have any monitoring within Kubernetes? I imagine they have more monitoring capabilities now than they did with Heroku.

johnnyanmac 2y ago

>Who/What is going to be doing that on this new platform and how much does that cost? If you're already a web platform with hired talent (and someone using Heroku for a SaaS probably already is), I'd be surprised if the marginal cost was 10x.that paid support is of course coming at a premium, and isn't too flexible on what level of support you need. And yeah, it isn't apples to apples. Maybe you are in a low CoL area and can find a decent DevOps for 80-100k. Maybe you're in SF and any extra dev will be 250k. It'll vary immensely on cost.

kmacdough 2y ago

Exactly. It all depends on your needs and — to be honest — the quality of your sysops engineering. You may not only need dedicated sysops, but you may incur higher incidental costs with lost productivity when your solution inevitably goes down (or just from extra dev load when things are harder to use). That said, at least in 2016 Heroku was way overpriced for high volume sites. My startup of 10 engineers w/ 1M monthly active users saved 300k+/yr switching off heroku. But we had Jerry. Jerry was a beast and did most of the migration work in a month, with some dead-simple AWS scaling. His solution lacked many of the features of Heroku, but it massively reduced costs for developers running full test stacks which, in turn increased internal productivity. And did I mention it was dead simple? It's hard to overstate how valuable this was for the rest of us, who could easily grok the inner workings and know the consequences of our decisions. Perhaps this stack will open that opportunity to less equipped startups, but I've found few open source "drop-in replacements" to be truly drop-in. And I've never found k3 to be dead simple.

cryptonector 2y ago

Who says they reduced costs by cutting staff? They could instead have scaled their staff better.