7 ms·

This is what I call fake work. This company builds a team vacation tracker. As long as it's reliable, their customers could not give a rat's ass whether it run

This is what I call fake work. This company builds a team vacation tracker. As long as it's reliable, their customers could not give a rat's ass whether it runs in AWS, Google, or Joebob's House Of Ill Compute. Every engineer hour spent creating abstraction layers and docker containers and whiz-bang multiplatform plumbing is an hour that could have been spent working on something your users actually care about. Switching clouds is a cost optimization. You have to be an enormous company (or in a resource-intensive domain) before this is more productive than building features. It means you've moved out of the growth phase and into the "how do I milk what I've got" phase. Vendor lock-in is something that engineers care about because they like playing with tech. It's cool switching platforms or databases or whatnot because it really feels like hard, sophisticated work! But it doesn't move the needle.

dexen 8y ago
To push back a bit, without going into the abstract of portability: - performance is a feature - reliability is a feature - security is a feature - and lastly, cost-effectiveness is a feature If the app is to be used through wide range of regions, you want geographically close datacenters for low latencies. If the app is to be used for mission-critical purposes, and contain sensitive data, you want high assurance and privacy[0]. And lastly, the cheaper the app is to run and maintain - and that includes not just the infrastructure bill, but also availability of engineers who know the platform well[1] - the better you can develop the same app within the given budget & time. [0] there are some legal compliance requirements for handling medical data [1] one of the reason Windows apps are rather cheap and plentiful - there's wide availability of engineers who know the platform well. Likewise for certain popular web stacks.
- JamesBarney 8y ago
  But I feel like it's way easier to hire for Azure,AWS, or Google than all three, or the in-house cross provider solution.
- jimmychangas 8y ago
  Better yet, the things you listed are qualities. They encompass every feature in the product. I think parent is generalizing too much on his assumption that only mature companies should spend time in optimization. Early stage startups can sometimes make mistakes, and optimizing early on can help them save the precious resources needed for them to survive another six months.
  - dexen 8y ago
    >They encompass every feature in the product That's not a given. It's plenty common to have the most often used features (say, sub-pages, or key processes) optimized the best, with the rarely used ones lingering at low optimization. User behavior is non-linear; small changes around pivotal values can yield great results. Over-optimizing on the other hand yields diminishing returns. Making even just your home page snappy can greatly improve conversions (or whatever is your success metric). Other than that you are right that a bit of optimization here or there can save a lot of costs. I distinctly remember certain popular phpBB forum that, while quickly growing the userbase, somehow managed to still improve performance on rather measly hardware - and without much horizontal scaling! - getting in return genuine praise from the users, and also keeping the cost & complexity in check.
    - jimmychangas 8y ago
      You are right, thank you for your reply. The "user behavior is non-linear" and "optimization yealds diminishing returns" are powerful insights.
theknarf 8y ago
> Due to a change in how they report data usage, our monthly costs for Firebase, a SaaS provided by Google, has increased from $25 a month to what is now moving towards the $2,000 mark — with no changes to our actual data use. This change was made without warning. ref https://news.ycombinator.com/item?id=14356409 https://news.ycombinator.com/item?id=14356409 Vendor lock-in can make or break startups if you're not careful enough.
- JOnAgain 8y ago
  If $25/month breaks your startup ....
  - robrtsql 8y ago
    I think it's the other $1975 that is breaking their startup. Depending on your perspective, that might also be a small amount, but let's not misrepresent the situation.
    - maxxxxx 8y ago
      The question is how much you can shave off the 1975 by going to another vendor.
      - acct1771 8y ago
        The question should be how much you can save by not tying an anchor you don't control the weight of around your neck in the first place. The anchor in the metaphor is using large tech conglomerate without being able to quickly pivot away from them at a moment's notice.
        spookthesunset 8y ago
        > The question should be how much you can save by not tying an anchor you don't control the weight of around your neck in the first place. How much did it cost to engineer your product to not "tie an anchor"? My guess is, you didn't "save" anything trying to build out several layers of abstraction to deal with a hypothetical risk of getting screwed by "large tech conglomerates".
slothario 8y ago
It's fake work until your web application reaches a point where server loads take your web API down, and endpoints frequently take well over 10 seconds to respond. Yes, I worked at a place where both of those things were true. Switching some services out to a service (in this case, Azure functions) saved my company like $10k/mo, too.
- dexen 8y ago
  This is a fine point. And yet both FB and Twitter managed to go through long period of servers being overloaded and fail-prone, to the point of frequent outages - cue the Fail Whale[1] - relatively unscathed, and emerge victorious. In the end it's about taking calculated risks, and concentrating on where your effort and capital are most effective. Virtually nobody thinks of the Fail Whale as a real risk now a day. [1] https://pbs.twimg.com/media/DorDnbAU4AARTc3.jpg https://pbs.twimg.com/media/DorDnbAU4AARTc3.jpg
  - lugg 8y ago
    Survivorship bias. I'd also posit that there isn't much competition at the top.
schnable 8y ago
I agree somewhat, but vendor lock-in is also something often pushed by engineering and business leadership due to previous experiences of being badly burned by lock-in in the past. For FaaS in particular, if designed and factored correctly, I think the cost to move clouds can be minimized without building multi cloud capability upfront.
closeparen 8y ago
Vendor lock-in is something the business starts to care about very deeply once the vendor is in a position to exploit it with pricing. Think of it like a backup, only instead of a hard disk failure you are insuring against a vendor’s market power.
gambler 8y ago
>Vendor lock-in is something that engineers care about because they like playing with tech. Bad engineers care about vendor lock-in, because they want to pad their resumes with abstraction tools. Good engineers care about vendor lock-in, because all cloud providers have limits you do not control. Your architecture can become insanely expensive or simply stop working if something you do grows beyond one of those limits. In those cases you can either re-architect the whole thing or simply switch vendors. This is one of the hidden costs of "cloud" platforms everyone suffers from but no one talks about.
- scarface74 8y ago
  If we grow so large that we hit the limits of whst AWS can handle, I think we will have enough investor money flowing in to be able to afford rearchitecting.
  - regularfry 8y ago
    You might not be able to afford the time.
    - scarface74 8y ago
      So what are the chances that your business is going to grow large enough that AWS or Azure can handle the requirements? Is your business likely to grow large enough to need more from AWS than Netflix or even Apple?
      - gambler 8y ago
        >So what are the chances that your business is going to grow large enough that AWS or Azure can handle the requirements? You naively assume that hosting something at AWS or Azure means all of their resources are at your disposal and subject to reasonable pricing. That is not the case. Cloud services are not designed for you. They are designed to make money for whomever actually owns them. They have their own architecture, assumptions and target use cases. In reality, the chances that a large company will eventually hit some limitation of AWS or Azure are nearly 100%. The only question is whether it will be something that can be worked around. One problem with "the cloud" is that a lot of the limits are no-obvious, and if your use case significantly deviates from whatever the provider had in mind, there might not be an incremental solution. Another problem is that linear resource pricing is not always sustainable if you're experiencing exponential growth.
rorykoehler 8y ago
Drove me crazy in the past when I got pressured by CFO to look into cutting infra costs which weren't even that high and the company was facing much great technical challenges (which were pre-requisites to even thinking about optimising infra costs). Always wanting me to kick the tyres with this provider or that provider. Unbelievable waste of time and resources to try save a few grand here and there.
JMTQp8lwXL 8y ago
It can become a real problem if you pick a niche PaaS that say, gets bought out and closes up shop real quick. Probably wouldn't happen if you picked AWS, but I wouldn't consider this "fake work" in all contexts. You have evaluate the probability of a risk occurring, and its impact to your business. Small-time service providers get bought out and close shop left and right.
spookthesunset 8y ago
The funny thing about "vendor lock in" that some engineers never consider is by avoiding "vendor lock in", you run a very real risk of getting locked into your own pile of crap instead. I've seen so many examples of homebrew database abstraction layers, homebrew javascript frameworks, homebrew reporting systems, homebrew cloud stacks (if you are luckly... there is also the rack-and-stack crowd), homebrew build systems, etc. All of them have just as high, or higher, switching costs than any paid vendor. All of them are unsupported, largely crappy rip-offs of what you could buy. I've seen companies get far more burned by these homebrew monsters than I've ever seen them get burned by "vendor lock in".
- debt 8y ago
  Furthermore, in my experience the usually poorly documented homebrew stuff is almost always worse than whatever crappy legacy but well-documented vendor their locked into. If I see some custom framework it’s usually an instant facepalm.
cryptonector 8y ago
Managing cost is work, real work.

dexen 8y ago

To push back a bit, without going into the abstract of portability: - performance is a feature - reliability is a feature - security is a feature - and lastly, cost-effectiveness is a feature If the app is to be used through wide range of regions, you want geographically close datacenters for low latencies. If the app is to be used for mission-critical purposes, and contain sensitive data, you want high assurance and privacy[0]. And lastly, the cheaper the app is to run and maintain - and that includes not just the infrastructure bill, but also availability of engineers who know the platform well[1] - the better you can develop the same app within the given budget & time. [0] there are some legal compliance requirements for handling medical data [1] one of the reason Windows apps are rather cheap and plentiful - there's wide availability of engineers who know the platform well. Likewise for certain popular web stacks.

JamesBarney 8y ago
But I feel like it's way easier to hire for Azure,AWS, or Google than all three, or the in-house cross provider solution.
jimmychangas 8y ago
Better yet, the things you listed are qualities. They encompass every feature in the product. I think parent is generalizing too much on his assumption that only mature companies should spend time in optimization. Early stage startups can sometimes make mistakes, and optimizing early on can help them save the precious resources needed for them to survive another six months.
- dexen 8y ago
  >They encompass every feature in the product That's not a given. It's plenty common to have the most often used features (say, sub-pages, or key processes) optimized the best, with the rarely used ones lingering at low optimization. User behavior is non-linear; small changes around pivotal values can yield great results. Over-optimizing on the other hand yields diminishing returns. Making even just your home page snappy can greatly improve conversions (or whatever is your success metric). Other than that you are right that a bit of optimization here or there can save a lot of costs. I distinctly remember certain popular phpBB forum that, while quickly growing the userbase, somehow managed to still improve performance on rather measly hardware - and without much horizontal scaling! - getting in return genuine praise from the users, and also keeping the cost & complexity in check.
  - jimmychangas 8y ago
    You are right, thank you for your reply. The "user behavior is non-linear" and "optimization yealds diminishing returns" are powerful insights.

theknarf 8y ago

> Due to a change in how they report data usage, our monthly costs for Firebase, a SaaS provided by Google, has increased from $25 a month to what is now moving towards the $2,000 mark — with no changes to our actual data use. This change was made without warning. ref https://news.ycombinator.com/item?id=14356409 https://news.ycombinator.com/item?id=14356409 Vendor lock-in can make or break startups if you're not careful enough.

JOnAgain 8y ago
If $25/month breaks your startup ....
- robrtsql 8y ago
  I think it's the other $1975 that is breaking their startup. Depending on your perspective, that might also be a small amount, but let's not misrepresent the situation.
  - maxxxxx 8y ago
    The question is how much you can shave off the 1975 by going to another vendor.
    - acct1771 8y ago
      The question should be how much you can save by not tying an anchor you don't control the weight of around your neck in the first place. The anchor in the metaphor is using large tech conglomerate without being able to quickly pivot away from them at a moment's notice.
      - spookthesunset 8y ago
        > The question should be how much you can save by not tying an anchor you don't control the weight of around your neck in the first place. How much did it cost to engineer your product to not "tie an anchor"? My guess is, you didn't "save" anything trying to build out several layers of abstraction to deal with a hypothetical risk of getting screwed by "large tech conglomerates".

slothario 8y ago

It's fake work until your web application reaches a point where server loads take your web API down, and endpoints frequently take well over 10 seconds to respond. Yes, I worked at a place where both of those things were true. Switching some services out to a service (in this case, Azure functions) saved my company like $10k/mo, too.

dexen 8y ago
This is a fine point. And yet both FB and Twitter managed to go through long period of servers being overloaded and fail-prone, to the point of frequent outages - cue the Fail Whale[1] - relatively unscathed, and emerge victorious. In the end it's about taking calculated risks, and concentrating on where your effort and capital are most effective. Virtually nobody thinks of the Fail Whale as a real risk now a day. [1] https://pbs.twimg.com/media/DorDnbAU4AARTc3.jpg https://pbs.twimg.com/media/DorDnbAU4AARTc3.jpg
- lugg 8y ago
  Survivorship bias. I'd also posit that there isn't much competition at the top.

schnable 8y ago

I agree somewhat, but vendor lock-in is also something often pushed by engineering and business leadership due to previous experiences of being badly burned by lock-in in the past. For FaaS in particular, if designed and factored correctly, I think the cost to move clouds can be minimized without building multi cloud capability upfront.

closeparen 8y ago

Vendor lock-in is something the business starts to care about very deeply once the vendor is in a position to exploit it with pricing. Think of it like a backup, only instead of a hard disk failure you are insuring against a vendor’s market power.

gambler 8y ago

>Vendor lock-in is something that engineers care about because they like playing with tech. Bad engineers care about vendor lock-in, because they want to pad their resumes with abstraction tools. Good engineers care about vendor lock-in, because all cloud providers have limits you do not control. Your architecture can become insanely expensive or simply stop working if something you do grows beyond one of those limits. In those cases you can either re-architect the whole thing or simply switch vendors. This is one of the hidden costs of "cloud" platforms everyone suffers from but no one talks about.

scarface74 8y ago
If we grow so large that we hit the limits of whst AWS can handle, I think we will have enough investor money flowing in to be able to afford rearchitecting.
- regularfry 8y ago
  You might not be able to afford the time.
  - scarface74 8y ago
    So what are the chances that your business is going to grow large enough that AWS or Azure can handle the requirements? Is your business likely to grow large enough to need more from AWS than Netflix or even Apple?
    - gambler 8y ago
      >So what are the chances that your business is going to grow large enough that AWS or Azure can handle the requirements? You naively assume that hosting something at AWS or Azure means all of their resources are at your disposal and subject to reasonable pricing. That is not the case. Cloud services are not designed for you. They are designed to make money for whomever actually owns them. They have their own architecture, assumptions and target use cases. In reality, the chances that a large company will eventually hit some limitation of AWS or Azure are nearly 100%. The only question is whether it will be something that can be worked around. One problem with "the cloud" is that a lot of the limits are no-obvious, and if your use case significantly deviates from whatever the provider had in mind, there might not be an incremental solution. Another problem is that linear resource pricing is not always sustainable if you're experiencing exponential growth.

rorykoehler 8y ago

Drove me crazy in the past when I got pressured by CFO to look into cutting infra costs which weren't even that high and the company was facing much great technical challenges (which were pre-requisites to even thinking about optimising infra costs). Always wanting me to kick the tyres with this provider or that provider. Unbelievable waste of time and resources to try save a few grand here and there.

JMTQp8lwXL 8y ago

It can become a real problem if you pick a niche PaaS that say, gets bought out and closes up shop real quick. Probably wouldn't happen if you picked AWS, but I wouldn't consider this "fake work" in all contexts. You have evaluate the probability of a risk occurring, and its impact to your business. Small-time service providers get bought out and close shop left and right.

spookthesunset 8y ago

The funny thing about "vendor lock in" that some engineers never consider is by avoiding "vendor lock in", you run a very real risk of getting locked into your own pile of crap instead. I've seen so many examples of homebrew database abstraction layers, homebrew javascript frameworks, homebrew reporting systems, homebrew cloud stacks (if you are luckly... there is also the rack-and-stack crowd), homebrew build systems, etc. All of them have just as high, or higher, switching costs than any paid vendor. All of them are unsupported, largely crappy rip-offs of what you could buy. I've seen companies get far more burned by these homebrew monsters than I've ever seen them get burned by "vendor lock in".

debt 8y ago
Furthermore, in my experience the usually poorly documented homebrew stuff is almost always worse than whatever crappy legacy but well-documented vendor their locked into. If I see some custom framework it’s usually an instant facepalm.

cryptonector 8y ago

Managing cost is work, real work.