4 ms·

I care. It's easy to brush off scaling concerns as not important, but I've had personal experience where it's mattered, and if you want a high profile example,

I care. It's easy to brush off scaling concerns as not important, but I've had personal experience where it's mattered, and if you want a high profile example, look at twitter. Yes, premature optimization is a bad thing, and so is over engineering; but that's easy to say if you have the experience to make the right initial choices that mean you have a meaningful path forward to scale when you do need it. For example, lets say you build a typical business app and push something out quickly that doesn't say, log when it fails, or provide an auto-update mechanism, or have any remote access. Now you have it deployed at 50 locations and its 'not working' for some reason. Not only do you physically have to go out to see whats wrong, you have to organize a reinstall at 50 locations. Bad right? yes. It's very bad. (<---- Personal experience) Or, you do a similar ruby or python app when your domain is something that involves bulk processing massive loads of data. It works fine and you have a great 'platform' until you have 3 users, and then it starts to slow down for everyone; and it turns out, you need a dedicated server for each customer because doing your business logic in a slow language works when you only need to do 10 items a second, not 10000. Bad right? yes. Very. Bad. (<---- Personal experience) It's not premature optimization to not pick stupid technology choices for your domain, or ship prototypes. ...but sometimes you don't have someone on the team with the experience to realize that, and the push from management is to just get it out, and not worry about the details; but trust me, if you have someone who is sticking their neck out and go, hey wait, this isn't going to scale... Maybe you should listen to what they have to say, not quote platitudes. Ecommerce is probably one of those things where the domain is well enough known you can get away with it; heck, just throw away all your rubbish and use an off-the-shelf solution if you hit a problem; but I'm going to suggest that the majority of people aren't building that sort of platform, because its largely a solved problem.

std_throwaway 9y ago
I don't like opinionated pieces like the one presented here because while they are right about some things they miss other things and present half-truths as full-truths. In my experience (and I have little of that) it's important to know the upgrade path and adjust your planning accordingly. How many users can you serve with your solution? How big do you expect the market to be in that stage? What technology would be the next step? How do you get there? How much more work would it be? Always be one step ahead with technology, but not two. Most markets are surprisingly small. Most use cases scale surprisingly well. You can probably push your solution by an order of one magnitude if you need it quickly. In order to answer the questions you need people who know the product, the (potential) technologies and the market. When you start you probably won't know any of that. See the first prototype you deploy to the customers as a means of collecting data for the first production version. Your first product is not your first product. Your funding should respect that. Get it done quickly with the aim of answering the critical questions. Then go back and design the next version "good-enough" for the second scaling step with the upgrade path in mind.
- user5994461 9y ago
  > You can probably push your solution by an order of one magnitude if you need it quickly. You can always push a python/ruby app an order of magnitude by putting an order of magnitude more AWS instances. It will almost always bankrupt you in the medium term. The only place I've seen it sustainable is a place that was generating a $100 per user, and there weren't many active users either (thousands, not millions).
  - luord 9y ago
    > You can always push a python/ruby app an order of magnitude by putting an order of magnitude more AWS instances. Or a Java app or a Go app. Really, if one's working in a domain where the language would become the bottleneck, one deliberately screwed up by going against the grain because that language is little used in that domain. For almost everything else or with very specific exceptions, something else is the bottleneck. Everyone here is giving anecdotal "evidence" of their claims so I'll follow suit: In the first company I worked in, the backend was entirely in Java and the application was internal (in-house CMS), meaning only ten users tops; everything about it was horribly slow. It was a sea of poor code, there was no such thing as a deployment pipeline and the servers it was hosted in were inadequate. There was also no relational schema to speak of in the database (basically MySQL used as a dumb document store). The next place I worked at, my team's job was to build an actual customer-facing application. We did it in python and, while it only has a few hundred users so far, there haven't been complaints about poor performance that I know of. Really, for every Twitter replacing Ruby, there's a Facebook written in PHP. Don't understand why so many people use one side to support their claim but forget the other.
    - manquer 9y ago
      Langauge is rarely the first bottle neck . This also doesn't apply in the same away for data stores and when your app is stateful. Scaling rdbms is not the simple. Query fine tunning and performance optimisation for single slow queries cannot be solved by just more resources. Just from a database pov Handling replication , multi master caching , backup and fail over especially can fuck you over . In b2b a single data loss event can kill your business, same with security .
Trundle 9y ago
The article is about choosing a business idea where technical scale isn't important. Twitter would fall under a "media play" which is what the author is telling the reader not to do. They aren't saying "try to build twitter but skimp on the tech" they're saying "don't try to build twitter, build something where people pay you money to use it and you'll be raking in millions in profit by the time technical scaling is an issue". Terrible example imo.
- shadowmint 9y ago
  The article is about choosing a business idea where technical scale isn't important. Those sorts of business ideas don't exist, unless you don't have any customers. :)
- reledi 9y ago
  Twitter is not exempt. They focused on the product before scale. They were regularly overloaded in the early days.
deleted 9y ago
[deleted]
deleted 9y ago
[deleted]
flukus 9y ago
Wouldn't twitter be a counter-example? They had some issues but resolved them later when the had the resources to and they are now a multi-billion dollar company. That may not have happened had the devoted resources into scaling from the outset.

std_throwaway 9y ago

I don't like opinionated pieces like the one presented here because while they are right about some things they miss other things and present half-truths as full-truths. In my experience (and I have little of that) it's important to know the upgrade path and adjust your planning accordingly. How many users can you serve with your solution? How big do you expect the market to be in that stage? What technology would be the next step? How do you get there? How much more work would it be? Always be one step ahead with technology, but not two. Most markets are surprisingly small. Most use cases scale surprisingly well. You can probably push your solution by an order of one magnitude if you need it quickly. In order to answer the questions you need people who know the product, the (potential) technologies and the market. When you start you probably won't know any of that. See the first prototype you deploy to the customers as a means of collecting data for the first production version. Your first product is not your first product. Your funding should respect that. Get it done quickly with the aim of answering the critical questions. Then go back and design the next version "good-enough" for the second scaling step with the upgrade path in mind.

user5994461 9y ago
> You can probably push your solution by an order of one magnitude if you need it quickly. You can always push a python/ruby app an order of magnitude by putting an order of magnitude more AWS instances. It will almost always bankrupt you in the medium term. The only place I've seen it sustainable is a place that was generating a $100 per user, and there weren't many active users either (thousands, not millions).
- luord 9y ago
  > You can always push a python/ruby app an order of magnitude by putting an order of magnitude more AWS instances. Or a Java app or a Go app. Really, if one's working in a domain where the language would become the bottleneck, one deliberately screwed up by going against the grain because that language is little used in that domain. For almost everything else or with very specific exceptions, something else is the bottleneck. Everyone here is giving anecdotal "evidence" of their claims so I'll follow suit: In the first company I worked in, the backend was entirely in Java and the application was internal (in-house CMS), meaning only ten users tops; everything about it was horribly slow. It was a sea of poor code, there was no such thing as a deployment pipeline and the servers it was hosted in were inadequate. There was also no relational schema to speak of in the database (basically MySQL used as a dumb document store). The next place I worked at, my team's job was to build an actual customer-facing application. We did it in python and, while it only has a few hundred users so far, there haven't been complaints about poor performance that I know of. Really, for every Twitter replacing Ruby, there's a Facebook written in PHP. Don't understand why so many people use one side to support their claim but forget the other.
  - manquer 9y ago
    Langauge is rarely the first bottle neck . This also doesn't apply in the same away for data stores and when your app is stateful. Scaling rdbms is not the simple. Query fine tunning and performance optimisation for single slow queries cannot be solved by just more resources. Just from a database pov Handling replication , multi master caching , backup and fail over especially can fuck you over . In b2b a single data loss event can kill your business, same with security .

Trundle 9y ago

The article is about choosing a business idea where technical scale isn't important. Twitter would fall under a "media play" which is what the author is telling the reader not to do. They aren't saying "try to build twitter but skimp on the tech" they're saying "don't try to build twitter, build something where people pay you money to use it and you'll be raking in millions in profit by the time technical scaling is an issue". Terrible example imo.

shadowmint 9y ago
The article is about choosing a business idea where technical scale isn't important. Those sorts of business ideas don't exist, unless you don't have any customers. :)
reledi 9y ago
Twitter is not exempt. They focused on the product before scale. They were regularly overloaded in the early days.

deleted 9y ago

[deleted]

deleted 9y ago

[deleted]

flukus 9y ago

Wouldn't twitter be a counter-example? They had some issues but resolved them later when the had the resources to and they are now a multi-billion dollar company. That may not have happened had the devoted resources into scaling from the outset.