4 ms·

No real reason to think that the costs scale linearly. Maybe you need 300 employees to maintain the site at that scale?

eropple 9y ago
> Maybe you need 300 employees to maintain the site at that scale? Should you? I mean, my day job is making it so you expressly don't. If your costs are scaling even linearly I would say you're doing something wrong. The point of scaling is to reduce costs--economies of scale are why you scale. And a user-editable encyclopedia and PHP application are not really good arguments for diseconomies.
- muninn_ 9y ago
  But you can't reach perfect efficiency. Scaling will of course reduce costs in general, but it's not a given. If I had an app that had 5 users, my costs would increase if my user number grew to 5,000,000,000. Without knowing more information about the financials, or how resources are allocated this is all conjecture. But a website that services 17bb page views/month is going to cost a lot of money to run. They could be spending their money very poorly, idk, but I also don't know whether or not what they are spending is an appropriate amount.
  - eropple 9y ago
    We aren't talking about "perfect efficiency". We're talking about not blowing up your costs. A website that serves seventeen billion pageviews per month of mostly cacheable and edge-serviced data is, while certainly a technical challenge, a very surmountable one. And a lot of the harder parts are blunted through Wikipedia's situation. Search, for example, is a difficult problem in those situations--but I'd bet money that most of those searches are coming from Google, which mitigates a large chunk of the demands on in-house search that a different kind of website might see. (I've used Wikipedia's internal search once this year, according to my browser history.) Point to the diseconomies of scale and we can talk about them, but everybody else has figured out how to leverage economies of scale when building out a large technical system.
  - PKop 9y ago
    > Scaling will of course reduce costs in general, but it's not a given. Not only does it seem costs haven't been reduced, their rate of increase has exceeded the growth in pages served, quite substantially. That's not the whole story I'm sure, but as a rough estimate that doesn't seem sustainable or healthy... or necessary simply in terms of general hosting cost declines over that period.
nitwit005 9y ago
I'd expect sublinear growth for the kind of hosting Wikipedia is doing. They're mostly serving cached text documents and images.
smsm42 9y ago
To just maintain the site - I mean, making sure every HTTP request (or reasonable part of them) is answered and Apache does not crash and logs are rotated and backups are performed - no, you don't need 300 people. If you froze Wikipedia in 2005 and never wanted to improve anything there, and never open new project, never make a local chapter, an editathon or support a new language, etc. etc. - you could probably do with 25-30 people. But that won't be a live project. That would be a fossil that would slowly wither and die, as it becomes less and less relevant and more and more inadequate with the needs of the current user. In 2005, iPhone didn't exist, now everybody has a smartphone. Should we somehow account for that or just ignore it? How about knowledge graphs and linked data and all AI developments - should all Wikipedia knowledge still be text-only and ignoring whole Linked Data universe? How about supporting thousands of existing languages - should we just dump them in their own domain, or should we help them with automatic translations, article templating, language-sensitive searches and so on? How about creating richer media like maps, diagrams, graphs, video and audio content - should we help this or should we be content with just inserting links to outside content? And that's only minuscule part of the questions we can ask about things changed and developed in the last decade. The point here is that Wiki universe is a a big and complex live active project (or set of projects), with very many facets, and reducing it to technical maintenance of one webserver site - even one that gets tons of traffic - is not a good idea. The goal of the movement, as I understand it, is not "make sure site en.wikipedia.org does not crash", it is "make all sum of human knowledge be available to everyone". It's a big mission, and it requires people to achieve it.

> Maybe you need 300 employees to maintain the site at that scale? Should you? I mean, my day job is making it so you expressly don't. If your costs are scaling even linearly I would say you're doing something wrong. The point of scaling is to reduce costs--economies of scale are why you scale. And a user-editable encyclopedia and PHP application are not really good arguments for diseconomies.

muninn_ 9y ago
But you can't reach perfect efficiency. Scaling will of course reduce costs in general, but it's not a given. If I had an app that had 5 users, my costs would increase if my user number grew to 5,000,000,000. Without knowing more information about the financials, or how resources are allocated this is all conjecture. But a website that services 17bb page views/month is going to cost a lot of money to run. They could be spending their money very poorly, idk, but I also don't know whether or not what they are spending is an appropriate amount.
- eropple 9y ago
  We aren't talking about "perfect efficiency". We're talking about not blowing up your costs. A website that serves seventeen billion pageviews per month of mostly cacheable and edge-serviced data is, while certainly a technical challenge, a very surmountable one. And a lot of the harder parts are blunted through Wikipedia's situation. Search, for example, is a difficult problem in those situations--but I'd bet money that most of those searches are coming from Google, which mitigates a large chunk of the demands on in-house search that a different kind of website might see. (I've used Wikipedia's internal search once this year, according to my browser history.) Point to the diseconomies of scale and we can talk about them, but everybody else has figured out how to leverage economies of scale when building out a large technical system.
- PKop 9y ago
  > Scaling will of course reduce costs in general, but it's not a given. Not only does it seem costs haven't been reduced, their rate of increase has exceeded the growth in pages served, quite substantially. That's not the whole story I'm sure, but as a rough estimate that doesn't seem sustainable or healthy... or necessary simply in terms of general hosting cost declines over that period.

nitwit005 9y ago

I'd expect sublinear growth for the kind of hosting Wikipedia is doing. They're mostly serving cached text documents and images.

smsm42 9y ago

To just maintain the site - I mean, making sure every HTTP request (or reasonable part of them) is answered and Apache does not crash and logs are rotated and backups are performed - no, you don't need 300 people. If you froze Wikipedia in 2005 and never wanted to improve anything there, and never open new project, never make a local chapter, an editathon or support a new language, etc. etc. - you could probably do with 25-30 people. But that won't be a live project. That would be a fossil that would slowly wither and die, as it becomes less and less relevant and more and more inadequate with the needs of the current user. In 2005, iPhone didn't exist, now everybody has a smartphone. Should we somehow account for that or just ignore it? How about knowledge graphs and linked data and all AI developments - should all Wikipedia knowledge still be text-only and ignoring whole Linked Data universe? How about supporting thousands of existing languages - should we just dump them in their own domain, or should we help them with automatic translations, article templating, language-sensitive searches and so on? How about creating richer media like maps, diagrams, graphs, video and audio content - should we help this or should we be content with just inserting links to outside content? And that's only minuscule part of the questions we can ask about things changed and developed in the last decade. The point here is that Wiki universe is a a big and complex live active project (or set of projects), with very many facets, and reducing it to technical maintenance of one webserver site - even one that gets tons of traffic - is not a good idea. The goal of the movement, as I understand it, is not "make sure site en.wikipedia.org does not crash", it is "make all sum of human knowledge be available to everyone". It's a big mission, and it requires people to achieve it.