3 ms·

You can always throw hardware at the performance problem. No, you can't. That's what scalability means. If you don't have scalability, then additional hardware

You can always throw hardware at the performance problem. No, you can't. That's what scalability means. If you don't have scalability, then additional hardware will only take you so far before you begin wasting it.

justin_vanw 17y ago
Well, you are both kindof taking the extreme view. If you build your app such that the webservers can act independently, then by using various methods of load balancing you can just throw hardware at the problem. However, this just usually leaves the database as the single point of contention among all webservers. There are various ways to reduce load on the database server by some constant factor (memcache, query caching), but in the end database load will be roughly linear with site load (requests/time). You can't just add SQL-based DB servers and get linear speedup like (properly thought out) webservers. If you think your site will get so big so fast you won't have time to hire engineers to figure out the scaling problems before they become unmanageable (for example, facebook got huge, but it took a few years, so they had time to raise money and adapt their stuff, although they were pretty slow for a long time), then use appengine datastore or simpledb or some other db that promises to scale really well (this is a really hard problem, which is why there aren't a lot of open source things that do this. Note: couchedb can't scale writes, so I don't count it). My personal thinking is that one big properly designed database can get you pretty far, and database clustering can get you far enough. If you find it pleasant, putting your app in appengine would make you pay the price of scalability a little at a time starting right away, instead of letting you push it off (but with a huge performance wall that you won't be able to get over without basically redesigning much of your apps internals).
rue 17y ago
Well, strictly speaking scalability means scaling both up and down. "Scripting languages" may fail on the former, and C certainly fails on the latter.
nostrademons 17y ago
Many dynamic-languages frameworks (notably PHP) are more scalable than writing a custom webapp in C. This is because C encourages you to store state in memory on the server, because it's really easy when everything is a single long-lived process. Good for performance, terrible for scalability. PHP (and Rails/Django, if you ignore their ORMs) encourage you to store state on some collection of backend servers and communicate via RPC to retrieve that. This is bad for performance, but good for scalability. The front-end is just a dumb intermediary that converts data into HTML, so if you need to scale, just add more of them. Scaling the backend is more challenging, but at least you can do it without dragging all the front-end code around.
- CodeMage 17y ago
  Absolutely! Just for the record -- since people seem to assume otherwise -- I'm not agreeing with the article. I'm just disagreeing with the idea of ignoring "scalability and performance" and then expecting to solve performance problems by "throwing hardware" at them.

justin_vanw 17y ago

Well, you are both kindof taking the extreme view. If you build your app such that the webservers can act independently, then by using various methods of load balancing you can just throw hardware at the problem. However, this just usually leaves the database as the single point of contention among all webservers. There are various ways to reduce load on the database server by some constant factor (memcache, query caching), but in the end database load will be roughly linear with site load (requests/time). You can't just add SQL-based DB servers and get linear speedup like (properly thought out) webservers. If you think your site will get so big so fast you won't have time to hire engineers to figure out the scaling problems before they become unmanageable (for example, facebook got huge, but it took a few years, so they had time to raise money and adapt their stuff, although they were pretty slow for a long time), then use appengine datastore or simpledb or some other db that promises to scale really well (this is a really hard problem, which is why there aren't a lot of open source things that do this. Note: couchedb can't scale writes, so I don't count it). My personal thinking is that one big properly designed database can get you pretty far, and database clustering can get you far enough. If you find it pleasant, putting your app in appengine would make you pay the price of scalability a little at a time starting right away, instead of letting you push it off (but with a huge performance wall that you won't be able to get over without basically redesigning much of your apps internals).

rue 17y ago

Well, strictly speaking scalability means scaling both up and down. "Scripting languages" may fail on the former, and C certainly fails on the latter.

nostrademons 17y ago

Many dynamic-languages frameworks (notably PHP) are more scalable than writing a custom webapp in C. This is because C encourages you to store state in memory on the server, because it's really easy when everything is a single long-lived process. Good for performance, terrible for scalability. PHP (and Rails/Django, if you ignore their ORMs) encourage you to store state on some collection of backend servers and communicate via RPC to retrieve that. This is bad for performance, but good for scalability. The front-end is just a dumb intermediary that converts data into HTML, so if you need to scale, just add more of them. Scaling the backend is more challenging, but at least you can do it without dragging all the front-end code around.

CodeMage 17y ago
Absolutely! Just for the record -- since people seem to assume otherwise -- I'm not agreeing with the article. I'm just disagreeing with the idea of ignoring "scalability and performance" and then expecting to solve performance problems by "throwing hardware" at them.