5 ms·

This article is wildly off base, particularly when you look at the examples the article cites. WhatsApp was able to scale to so many users while keeping the en

This article is wildly off base, particularly when you look at the examples the article cites. WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. (Does he think commodity coders know Erlang? Really!?) They heavily tuned their architecture and built efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code. Yes, we increasingly have tools which allow mediocre coders to get stuff done. (For example, Ruby on Rails.) But the inevitable byproduct of letting mediocre coders loose on a stack they only partially understand is a mess of spaghetti code which takes even more developers to maintain. This is already what you see with non-tech corporations who employ dozens/hundreds of developers to maintain Drupal/RoR/Java apps which are significantly simpler in scale & complexity than WhatsApp or Instagram. If you want to use tools well, you need to hire people who understand the tools. Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allows them to fail faster. I might be able to build a house with legos, but I expect to be spending a lot more time repairing it than if I'd just hired a carpenter in the first place. [1] http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...

mtanski 12y ago
If I remember correctly from a high scalability post about WhatsApp it was also on bare metal. So more complexity to manage then a simple AWS setup where you can tear up/down in seconds.
coldtea 12y ago
>WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. They heavily tuned their architecture and build efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code. The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough. As for the "near great" they would too be less needed, and commodity engineers would be the equivalent of burger flipers. >Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allow. The first yes, the second not really. A bad engineer can get enormously more done with modern tools than he could 10 or 30 years ago. Essentially a bad engineer today can build stuff that took high level senior engineers to build -- and do it in less time too. It might be a little crude, but it will work. E.g a bad engineer can automate a workflow that would have taken expert engineers, custom software and several man-years to do 20 years ago, and can do it in a day. And because it's so abstracted, building something like that needs little influence from his skills, so his "badness" doesn't show up much.
- dropit_sphere 12y ago
  There are always greater problems. Still don't have my flying car.
- harpastum 12y ago
  The problem with the article's conclusion (and title) is that they don't stop with 100,000 -> 1,000 engineers. "Tomorrow, that billion-dollar startup acquisition might not need an engineer at all." The article's claim is not that startups are going to keep having smaller and smaller teams of higher-value engineers, they're saying that they're going to have zero engineers. Which I don't see at all from the evidence provided.
  - coldtea 12y ago
    Well, they might technically have an engineer, but it could be some average joe, using a "21st century" equivalent of Hypercard -- building a whole startup with some composable cloud components and some logic. After all, the basic skills for a successful startup, even today, are not hardcore engineering skills, but idea, business, finances, etc. I mean, most startups bought for multi billion dollars work on BS trivial engineering problems (mostly: make a social web service scalable), compared to what talented engineers do elsewhere (in research, in space exploration, in robotics, in DSP, etc). Heck, wasn't Facebook started with crappy PHP code a 15 year old could write (and a 20 year old did wrote)? Or, for a smaller scale example, Derek Shivers (which I admire) sold his CDBaby startup for a nice sum, and that was also started and based on amateurish code at best that he wrote himself.
    - deleted 12y ago
      [deleted]
- morgante 12y ago
  > The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough. The premise of the article seems to be that we no longer need great engineers at all. Or that they only need to work at infrastructure companies (not application companies like WhatsApp). The very example he chose shows that's empirically untrue. The fact that a small team of amazing engineers can accomplish huge things (given some ops support) is not new. The original Macintosh team was also tiny. There's no evidence that we will ever stop needing great engineers, particularly as more and more applications require huge scale while maintaining agility.
mahyarm 12y ago
Whatsapp also makes design decisions that I would call 'engineer-easy'. Their iOS app is pretty much all default system widgets and behavior for example, which is significantly less code to maintain than something that was customized heavily. On the server side they don't retain conversations, have multi-session usage (many clients, one user), etc which simplifies what is required. I haven't used whatsapp in a while so things might of changed, but before they were acquired by Facebook, that was definitely their direction.
deleted 12y ago
[deleted]

mtanski 12y ago

If I remember correctly from a high scalability post about WhatsApp it was also on bare metal. So more complexity to manage then a simple AWS setup where you can tear up/down in seconds.

coldtea 12y ago

>WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. They heavily tuned their architecture and build efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code. The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough. As for the "near great" they would too be less needed, and commodity engineers would be the equivalent of burger flipers. >Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allow. The first yes, the second not really. A bad engineer can get enormously more done with modern tools than he could 10 or 30 years ago. Essentially a bad engineer today can build stuff that took high level senior engineers to build -- and do it in less time too. It might be a little crude, but it will work. E.g a bad engineer can automate a workflow that would have taken expert engineers, custom software and several man-years to do 20 years ago, and can do it in a day. And because it's so abstracted, building something like that needs little influence from his skills, so his "badness" doesn't show up much.

dropit_sphere 12y ago
There are always greater problems. Still don't have my flying car.
harpastum 12y ago
The problem with the article's conclusion (and title) is that they don't stop with 100,000 -> 1,000 engineers. "Tomorrow, that billion-dollar startup acquisition might not need an engineer at all." The article's claim is not that startups are going to keep having smaller and smaller teams of higher-value engineers, they're saying that they're going to have zero engineers. Which I don't see at all from the evidence provided.
- coldtea 12y ago
  Well, they might technically have an engineer, but it could be some average joe, using a "21st century" equivalent of Hypercard -- building a whole startup with some composable cloud components and some logic. After all, the basic skills for a successful startup, even today, are not hardcore engineering skills, but idea, business, finances, etc. I mean, most startups bought for multi billion dollars work on BS trivial engineering problems (mostly: make a social web service scalable), compared to what talented engineers do elsewhere (in research, in space exploration, in robotics, in DSP, etc). Heck, wasn't Facebook started with crappy PHP code a 15 year old could write (and a 20 year old did wrote)? Or, for a smaller scale example, Derek Shivers (which I admire) sold his CDBaby startup for a nice sum, and that was also started and based on amateurish code at best that he wrote himself.
  - deleted 12y ago
    [deleted]
morgante 12y ago
> The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough. The premise of the article seems to be that we no longer need great engineers at all. Or that they only need to work at infrastructure companies (not application companies like WhatsApp). The very example he chose shows that's empirically untrue. The fact that a small team of amazing engineers can accomplish huge things (given some ops support) is not new. The original Macintosh team was also tiny. There's no evidence that we will ever stop needing great engineers, particularly as more and more applications require huge scale while maintaining agility.

mahyarm 12y ago

Whatsapp also makes design decisions that I would call 'engineer-easy'. Their iOS app is pretty much all default system widgets and behavior for example, which is significantly less code to maintain than something that was customized heavily. On the server side they don't retain conversations, have multi-session usage (many clients, one user), etc which simplifies what is required. I haven't used whatsapp in a while so things might of changed, but before they were acquired by Facebook, that was definitely their direction.

deleted 12y ago

[deleted]