6 ms·

I've been doing data analysis and machine learning client work for quite some time now and for companies as small as a 3 person startup to advising a department

I've been doing data analysis and machine learning client work for quite some time now and for companies as small as a 3 person startup to advising a department of the Canadian government. Almost always numpy matrix math + cython or C or Java on a single machine is enough. Not always-always; but if you can relax requirements slighly say by accepting a 45 minute lag from new data impacting the total model, or by caching the results of the top 10k most likely queries, or by putting more effort into stripping out the garbage parts of the data, or, sometimes, just throwing a $10k a month server or mathematician at the problem (sure is cheaper than a bunch of cheap servers + larger infrastructure team). The times you need real scalability you know you need it. You'd laugh at how silly someone would be for trying to put it onto one machine. You're solving the travelling salesman problem for UPS (although I can think of some hacks here - I probably can't get it down to a single machine), or you're detecting logos in every Youtube video ever made, or you're working for the NSA. Even if you know for sure you're going to need scalability. I don't think it hurts to just do it on a single box at first. Iterating quickly on the product is more important and once you have something proven you can get money from the market or from VCs to distribute it.

brianwawok 10y ago
This is kind of the same argument as microservices. We could write 30 microservices deployed on 30 docker images with load balancing and FT and all that magic for a basic webapp... Or we could just write a pretty fast webserver and do it with 1 server. (Or if it is stateless, do it with a few for still a lot less work than a giant microservice cluster). I think in the last year or so microservices have become a little less cool, and people are more along the lines of "code cleanly so we can microservice if we need to down the road, but don't deploy it like that for 1.0"... seems similar for this.
- sigusr1 10y ago
  If you haven't seen stackoverflows server architecture posts you'd probably enjoy them: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... I still think containers are great, and I really like the abstractions kubernetes provides, but if I ever had enough traffic to worry about scaling, I envision running a small cluster of very powerful computers rather than 100s of weak ones.
  - Joeri 10y ago
    So basically, through careful engineering, they can run stackoverflow.com, one of the top-ten sites in the world, on a single web server and a single database server. Beefy machines for sure, but it kind of takes the air out of the web-scale hype balloon.
    - toomuchtodo 10y ago
      Physical servers they host themselves, mind you.
    - brianwawok 10y ago
      Ya I love this as a counterpoint to much scaling. Stackoverflow has pretty good uptime and pretty good performance, can't fault the end result.
    - mrep 10y ago
      > one of the top-ten sites in the world 49th actually: http://www.alexa.com/siteinfo/stackoverflow.com http://www.alexa.com/siteinfo/stackoverflow.com
  - Bromskloss 10y ago
    Huh? Why are they running Windows?
    - e12e 10y ago
      I don't know about the rest of the (initial) team, but Joel was lead on Excel as I recall. If you have a group if devs that know the nt kernel, SQL server and iis very well - why would you not use the ms stack? Don't get me wrong, I prefer Linux and Free software myself - but that doesn't mean the thousands of Man years sybase and ms have spent on their stack is wasted effort.
    - Joeri 10y ago
      Because they thought it was the better choice for them. You cannot argue with the result. https://www.quora.com/Why-does-StackExchange-StackOverflow-uses-ASP-NET-instead-of-some-opensource-frameworks https://www.quora.com/Why-does-StackExchange-StackOverflow-u...
- geofft 10y ago
  See also https://circleci.com/blog/its-the-future/ https://circleci.com/blog/its-the-future/, which is almost too true to be funny.
- ben_jones 10y ago
  People forget that the two methods of scaling are HORIZONTAL and VERTICAL. They think: "I can just put some micro-services behind HA proxy and boom, more capacity!". And then they forget if they had just modified that one query and tweaked that one for-loop they could've had that same capacity without launching six new servers with all kinds of potential for the wiring to go down and cause downtime. Plus the dev time to build the services.
  - aianus 10y ago
    Vertical scaling requires hiring good engineers instead of mediocre ones (additional cost of $100,000s per year across the team). Horizontal scaling in comparison is much, much cheaper for your average CRUD app.
    - brianwawok 10y ago
      Maybe, maybe not. For example choosing Java over Ruby would give you 2-10x better perf per server... and I am not sure that Java devs cost any more or less than Ruby devs. Now we can get into an argument about developer productivity and all that.. but form a purely "i want to run 10x more users per server"... something like Java / Ruby gets you a long ways.
      - joslin01 10y ago
        I talked to a CTO once that said he brought his RoR fleet down from 60 servers to 6-8 by switching to Scala.
        matthewmacleod 10y ago
        When he said "switching to Scala" I reckon he probably meant "rewriting our platform." It's very difficult to do a comparison like this in practice, because switching languages inherently involves a rewrite of a platform.
        10y ago
- tracker1 10y ago
  My solution was more about higher availability, and staggered deployments... but each of about 8 services (including web) would be deployed to 3 servers, identical config with dokku... then there would be a load balancing nginx instance that pointed to the app.foo on each of the three servers.. deployment would update/cycle nginx, deploy to first server, roll over, then deploy to the other two. That was the plan. (I won't go into the political bs at a company I no longer work with).
Bromskloss 10y ago
> throwing a $10k a month server or mathematician at the problem Do you have any examples of problems at which you have thrown a mathematician or could imagine doing so?

brianwawok 10y ago

This is kind of the same argument as microservices. We could write 30 microservices deployed on 30 docker images with load balancing and FT and all that magic for a basic webapp... Or we could just write a pretty fast webserver and do it with 1 server. (Or if it is stateless, do it with a few for still a lot less work than a giant microservice cluster). I think in the last year or so microservices have become a little less cool, and people are more along the lines of "code cleanly so we can microservice if we need to down the road, but don't deploy it like that for 1.0"... seems similar for this.

sigusr1 10y ago
If you haven't seen stackoverflows server architecture posts you'd probably enjoy them: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... I still think containers are great, and I really like the abstractions kubernetes provides, but if I ever had enough traffic to worry about scaling, I envision running a small cluster of very powerful computers rather than 100s of weak ones.
- Joeri 10y ago
  So basically, through careful engineering, they can run stackoverflow.com, one of the top-ten sites in the world, on a single web server and a single database server. Beefy machines for sure, but it kind of takes the air out of the web-scale hype balloon.
  - toomuchtodo 10y ago
    Physical servers they host themselves, mind you.
  - brianwawok 10y ago
    Ya I love this as a counterpoint to much scaling. Stackoverflow has pretty good uptime and pretty good performance, can't fault the end result.
  - mrep 10y ago
    > one of the top-ten sites in the world 49th actually: http://www.alexa.com/siteinfo/stackoverflow.com http://www.alexa.com/siteinfo/stackoverflow.com
- Bromskloss 10y ago
  Huh? Why are they running Windows?
  - e12e 10y ago
    I don't know about the rest of the (initial) team, but Joel was lead on Excel as I recall. If you have a group if devs that know the nt kernel, SQL server and iis very well - why would you not use the ms stack? Don't get me wrong, I prefer Linux and Free software myself - but that doesn't mean the thousands of Man years sybase and ms have spent on their stack is wasted effort.
  - Joeri 10y ago
    Because they thought it was the better choice for them. You cannot argue with the result. https://www.quora.com/Why-does-StackExchange-StackOverflow-uses-ASP-NET-instead-of-some-opensource-frameworks https://www.quora.com/Why-does-StackExchange-StackOverflow-u...
geofft 10y ago
See also https://circleci.com/blog/its-the-future/ https://circleci.com/blog/its-the-future/, which is almost too true to be funny.
ben_jones 10y ago
People forget that the two methods of scaling are HORIZONTAL and VERTICAL. They think: "I can just put some micro-services behind HA proxy and boom, more capacity!". And then they forget if they had just modified that one query and tweaked that one for-loop they could've had that same capacity without launching six new servers with all kinds of potential for the wiring to go down and cause downtime. Plus the dev time to build the services.
- aianus 10y ago
  Vertical scaling requires hiring good engineers instead of mediocre ones (additional cost of $100,000s per year across the team). Horizontal scaling in comparison is much, much cheaper for your average CRUD app.
  - brianwawok 10y ago
    Maybe, maybe not. For example choosing Java over Ruby would give you 2-10x better perf per server... and I am not sure that Java devs cost any more or less than Ruby devs. Now we can get into an argument about developer productivity and all that.. but form a purely "i want to run 10x more users per server"... something like Java / Ruby gets you a long ways.
    - joslin01 10y ago
      I talked to a CTO once that said he brought his RoR fleet down from 60 servers to 6-8 by switching to Scala.
      - matthewmacleod 10y ago
        When he said "switching to Scala" I reckon he probably meant "rewriting our platform." It's very difficult to do a comparison like this in practice, because switching languages inherently involves a rewrite of a platform.
        10y ago
tracker1 10y ago
My solution was more about higher availability, and staggered deployments... but each of about 8 services (including web) would be deployed to 3 servers, identical config with dokku... then there would be a load balancing nginx instance that pointed to the app.foo on each of the three servers.. deployment would update/cycle nginx, deploy to first server, roll over, then deploy to the other two. That was the plan. (I won't go into the political bs at a company I no longer work with).

Bromskloss 10y ago

> throwing a $10k a month server or mathematician at the problem Do you have any examples of problems at which you have thrown a mathematician or could imagine doing so?