5 ms·
Is this the shape of things to come? Comparisons of apples to oranges... It's not about traffic, it's about what sort of website you have, how dynamic/static i
by points 16y ago
Is this the shape of things to come?
Comparisons of apples to oranges... It's not about traffic, it's about what sort of website you have, how dynamic/static it is, etc
A wiki likely has relatively few writes compared to reads, so caching should work very well.
That said, always nice to see people optimizing properly and using a sane number of servers.
- superjared 16y agoThat's kind of the point. Yesterday Spolsky asked why Digg had 500 servers and less traffic than StackOverflow, yet StackOverflow had 5 servers. False dichotomies huzzah!
- points 16y agoWell TBH 500 is ridiculous overkill. Even 100 would be pushing it.
- Goladus 16y agoWhether it's overkill or not depends entirely on what the servers are being used for. Without actually working there it's hard for one to say for sure.
- points 16y agoWe know what the service does. We have that much data - we can see what users are allowed to do on the website, and what they cannot do. And for the level of content, interactivity, media, etc etc, I can say that 500 is silly.
- Goladus 16y agoYou don't know what each server is doing. They might have 10 servers being used by an internal marketing analytics team, with 40 support servers for development, QA, testing, and disaster recovery of those specific services. What sort of computing resources do you think it took for Google to develop the autopilot car? Would you have been able to determine that by looking at their homepage and the services they provide? No. That's the point. I'm not suggesting that Digg is doing anything so interesting, but most of those 500 servers are almost certainly NOT being used to support their website directly, they're being used by the business for other things.
- Keyframe 16y agoYes, title is kind of a homage to that false dichotomy posted yesterday by Spolsky.
- ericflo 16y agoI liked this submission simply because it demonstrates the ridiculousness of such claims.
- citricsquid 16y agoThe title was supposed to be a serious representation of our traffic numbers (our amount of servers wasn't supposed to be part of the comparison, the poster here is a slippery snake) but we can pretend it was if you like, nobody likes joel so it fits in well.
- ojbyrne 16y agoThankfully it wasn't actually less traffic, it was only 4x (maybe 5x) the traffic, with 10x the servers.
- fauigerzigerk 16y agoSpolsky compared two rather more similar sites though. This one is a wiki where most of the content is static, served out of cache. Still, I agree that such comparisons are on a very weak footing. A single feature could easily kill any valid comparison.
- bad_user 16y agoYes but StackOverflow is very similar to Digg: you've got articles that have rating, you've comments to those articles that have rating, articles and comments are sorted according to rating, people earn karma from posting good comments / articles, everything has tags. So personally I don't see a false dichotomy; even if Digg is more dynamic / complex ... WTF are they doing with those 500 servers?
- qeorge 16y agoDigg is more like Facebook and Twitter than StackOverflow. Each of Digg's logged in users gets their own "News Feed" based on the users they follow/friended/are most similar to. StackOverflow on the other hand is much simpler - questions and responses, plus users and voting. AFAIK there's no collaborative filtering going on at SO, be it user or item based. I don't know why Digg needs so many boxen, but I did find Spolsky's comparison disingenuous.
- rythie 16y agoAlso, it's MediaWiki so all the optimisation work was done by someone other people, to make Wikipedia work.
- citricsquid 16y agoMediawiki is fantastic software, we'd be able to operate on a lot less hardware if it was just mediawiki, but because of the forum we had to boost everything up. For anyone who ever considers using phpbb for anything serious: please don't.
- hartror 16y agoSuggestions for good alternatives with good scaling?
- citricsquid 16y agoWe're looking at a couple of alternatives and I can report back with our findings when we know enough, right now we're looking at vbulletin which powers some of the larger forums, it is apparently very good if you're willing to strip out the poor parts (apparently search is terrible). It looks as if the best approach is to roll your own, phpbb seems to be designed with the smaller user in mind, so while routing every single file through file.php for easy processing might work well for Johnny and his friends forum, once you hit a large scale it becomes rather a pain. So yeah, no idea, we'll find out soon though, I'll report back when I know :-)
- dmak 16y agoWhere can I stay posted with your findings?
- sandGorgon 16y agoAny chance you could post your configuration files somewhere ? (you know PHP, varnish, etc.) It'd be great to grok those.
- jshen 16y agoI think there is an important lesson in this even if it's apples to oranges. Often in my career I've been in a debate with a non engineer (product person, ceo, etc) about why a certain features sounds good to them but a variation of it which provides most of what they want is so much better because I can keep the page mostly static. Sometimes I've won that debate and sometimes I lost. Being able to hold up examples, see such and such wiki serves x million pages with 3 servers because it's mostly static vs ...