7 ms·

6 million pages! A quick look at their distribution: 3.6M of these pages got less than 10 views during December. 4.3M pages got less than 100 views each. 2.3M

by fhoffa 7y ago

6 million pages! A quick look at their distribution: 3.6M of these pages got less than 10 views during December. 4.3M pages got less than 100 views each. 2.3M got less than 1000 views. That's the long tail. On the other side 680K pages got more than 1,000 views, 111K pages got more than 10k views, only 6k pages got more than 100k views, and 90 pages were able to gather more than 1M views (Star Wars, The Mandalorian, The Witcher...). In summary: The top 7.2% of Wikipedia pages earn 87% of all the monthly views. Yesterday I posted a deeper analysis on how these pages get their daily views: - https://towardsdatascience.com/interactive-the-top-2019-wikipedia-pages-d3b96335b6ae https://towardsdatascience.com/interactive-the-top-2019-wiki... Apparently the most popular pages are all related to movies (Avengers, Joker, ...), series (Ted Bundy, Chernobyl, Game of Thrones), and deaths.

Thorentis 7y ago
Wow that's fascinating, I'd never really thought about page distribution. I have good feeling that the majority of those views are due to Google providing the answer for almost anything that can be found on Wikipedia in an "instant search result" box. It has a wikipedia link beneath it to read more, and I guess many people do.
- deleted 7y ago
  [deleted]
- rapnie 7y ago
  Also makes me wonder how many pages are not viewed because instant search result was sufficiently informative.
- busymom0 7y ago
  The 80/20 rule applies to many these sort of things. In case of Wiki, it's 90/10 https://en.wikipedia.org/wiki/Pareto_principle https://en.wikipedia.org/wiki/Pareto_principle
balladeer 7y ago
Sometimes I worry about projects about Wikipedia and the Archive. When will they run out of storage space and money? When will it become too big to handle?
- ripdog 7y ago
  That will never happen for Wikipedia. Their content is tiny in disk space and not growing very fast. They are also drowning in money, despite what their donation begging messages claim. The Internet Archive, on the other hand, is drowning in data and pinching pennies. Still, they're doing okay, and the constant decrease in per-gigabyte HDD costs are allowing them to continue for the time being. They absolutely need and deserve your donations, however. If HDD price/gig stopped going down, however, they might be in a lot of trouble.
  - alanbernstein 7y ago
    You're saying they rely on the decreasing prices, rather than low prices? How's that?
    - skosch 7y ago
      The amount of data is growing superlinearly while their resources (probably) aren't, so a dropping price for storage is currently allowing them to keep up (if just barely so).
      - eru 7y ago
        As a mathematical curiosity: an exponential decreasing price means that the total expense to store a Gigabyte forever is finite. Eg a 10% decrease a year, means that storing forever costs approximately ten times the first year's storage cost. https://en.wikipedia.org/wiki/Geometric_series#Sum https://en.wikipedia.org/wiki/Geometric_series#Sum has the math.
        goodside 7y ago
bduerst 7y ago
That's really not too shabby for a long tail though. What's the % that had zero views in December? Do they account for bots?
Udik 7y ago
> 90 pages were able to gather more than 1M views (Star Wars, The Mandalorian, The Witcher...). So they could cut their network costs by a lot if they just removed pages about current popular culture, which have the highest view rate and a cultural value of exactly zero. Edit: I'm being sarcastic and a bit bitter, the comments below all make good points. However I find it sad, though unavoidable, that the highest page views are for highly marketed entertainment franchises that have a slim chance to withstand the test of time.
- dna_polymerase 7y ago
  Their network cost is negligible. Overall hosting cost them less then $3 million, actually they spend more on travel & conferences [0]. [0]: https://upload.wikimedia.org/wikipedia/foundation/3/31/Wikimedia_Foundation_Audit_Report_-_FY18-19.pdf https://upload.wikimedia.org/wikipedia/foundation/3/31/Wikim...
- tjoff 7y ago
  You don't think those views result in a stronger brand and more views on pages with a higher cultural value?
- govg 7y ago
  Why do they have a cultural value of exactly zero?
  - olooney 7y ago
    More to the point, what even is "cultural value?" After Googling it, I found that it doesn't even seem to be a well defined phrase, much less a quantifiable concept. Wikipedia uses notability[1] to decide which articles to include. Notability is defined as significant coverage in 3rd party sources. Popular movies are "notable" because other people write about them. [1]: https://en.wikipedia.org/wiki/Wikipedia:Notability https://en.wikipedia.org/wiki/Wikipedia:Notability
    - friendlybus 7y ago
      There's too much in art that relies on unmeasured effects like the impact on the audience's consciousness and the execution on structure and intention to be bothered with numbers beyond box office or critic reviews. You could measure something related to those concepts and you would be capturing a fourth of available judgement for cultural impact. Culture is a game we have been playing for thousands of years. Art pieces with an observable influence on other artists, the public audience or any human decision making along with an ability to survive the wear and tear of existing in the world as an artwork is a good place to start summarizing an artwork's cultural value. A grand cathedral like st peter's stands as an example of high quality baroque architecture, the value of the church as an institution. It remains intact after a large number of years, not having been physically deconstructed. It's influence on other artists is common to this day, recently used as a baroque example of maximalism on HN. The site attracts visitors and has yet to be rivalled in it's unique baroque grandeur. Those influences are quantifiable, and kind of comparable to a flash in a pan like a Hollywood film that gets most of its attention and influence in the first few months then sits in an institutional back pocket whilst other artists denigrate it so that new films can occupy the spotlight and the cycle continues. These films are sometimes used as reference for making new films and the industry puts out two similar styles of film at the same time reducing uniqueness. Generally otherwise left to sit in DVD/bluray jewel cases or disney vaults as they fall out of favour. You can rank these impacts and have conversations over which interpretive labels they fit into and conclude some are mor valuable than others.
tigerlily 7y ago
This is unremarkable since this is just Zipf's law: https://en.m.wikipedia.org/wiki/Zipf%27s_law https://en.m.wikipedia.org/wiki/Zipf%27s_law
jakear 7y ago
These numbers don’t add up... does the 4.3M pages that got less than 100 include the 3.6M that got less than 10? I should image so, otherwise that’s 7.9M pages. But then you say only 2.3M got less than 1000 views? That obviously can’t include the less than 100 or 10’s, but if you add them then you’re at 6.6M..??
- fhoffa 7y ago
  I was wondering the same! Not every page is an article, and I'm counting page views from their logs. https://en.m.wikipedia.org/wiki/Wikipedia:What_is_an_article%3F https://en.m.wikipedia.org/wiki/Wikipedia:What_is_an_article... Even then you can see in my query that I have removed most of their "special" pages.
- yread 7y ago
  I think its a typo 5.3M not 2.3M have less than 1000 views. Makes sense with 600k over 1000 vies and 6M total
H8crilA 7y ago
Ehm, isn't it super obvious that this, too, follows the Pareto distribution?
knbknb 7y ago
Do we have a power-law distribution here? (They are rare)
- moh_maya 7y ago
  Isn't Pareto a power law distribution?
DonCopal 7y ago
And here you can see the most visited pages for every week: https://en.wikipedia.org/w/index.php?title=Category:Top_25_Report&pagefrom=Top+25+20160220%0ATop+25+Report%2FFebruary+14+to+20%2C+2016#mw-pages https://en.wikipedia.org/w/index.php?title=Category:Top_25_R...
lr4444lr 7y ago
I wonder what the correlation is between inbound links from other articles and views, though.
vladak 7y ago
Whenever seeing the staggering number of Wikipedia articles it reminds me of my own recent experience with creating an article. To centralize some of the knowledge of now niche set of keyboards (manufactured by what used to be one of the FAANGs of its time) I set to create a page. After adding some - what I thought - useful and interesting data, the page got deleted as it was not "notable" enough, i.e. lacking sufficient number of external references. These keyboards did exist and are used until this day however are not as popular (and therefore not as commented/described) like Apple's keyboards (which do have their own page on Wikipedia) so the line was drawn there. In the end I found it funny and also interesting to see how Wikipedia works. If the editors were not so strictly bound to the set of rules there would be probably tens of millions of articles/pages now. Another experience I had was watching a video on YouTube from Japan where a guy was brewing coffee which reminded me that the brand of the coffee equipment used in the video is sometimes sold in coffee shops in Europe. I wanted to know a little bit more about the company so I went to Wikipedia - the English page got deleted because there was little information about the company on the English speaking part of the Internet. This does not say anything about the company (which seems to be hugely popular in Japan and certain social circles in Europe), rather it demonstrates the limit of the model Wikipedia uses for curating the articles.
- the_af 7y ago
  There is a struggle between "deletionist" and "inclusionist" factions within Wikipedia. One side tends to find fault (many times real faults, mind you) and err on the side of deletion; the other prefers to err on the side of inclusion in Wikipedia. All agree they want to keep some standard of quality in Wikipedia, they just disagree on the method and threshold for deletion. Just be aware Wikipedia is not a single hive-mind :)
  - Carpetsmoker 7y ago
    I find all of this really hilarious especially considering that a lot of Wikipedia is about trivial stuff like entertainment (films, sports, anime, whatnot), but whenever someone tries to a document something technical in the wrong way ... NO SOUP FOR YOU! I find it really really rude and disrespectful to just trample over other volunteers' work like this. I'm all for deleting crap, but outright deletion instead of offering suggestions first and then being super-defensive when asked "hey, what's up?" Yeah nah, that doesn't fly. It's like just closing people's PRs on GitHub without offering suggestions on how to fix the patch first with just a single line of "doesn't fit code standards", "doesn't work on Windows", or just a link to a lengthy page. Guess everyone is too busy fighting over capitalisation of movies titles or whether or not the Stanley Kubrick article should have an infobox to offer meaningful suggestions to new users shrug. On Wikipedia, the biggest asshole with the most persistence wins. They've deluded themselves in to thinking this is "meritocracy".

by fhoffa 7y ago

Thorentis 7y ago

Wow that's fascinating, I'd never really thought about page distribution. I have good feeling that the majority of those views are due to Google providing the answer for almost anything that can be found on Wikipedia in an "instant search result" box. It has a wikipedia link beneath it to read more, and I guess many people do.

deleted 7y ago
[deleted]
rapnie 7y ago
Also makes me wonder how many pages are not viewed because instant search result was sufficiently informative.
busymom0 7y ago
The 80/20 rule applies to many these sort of things. In case of Wiki, it's 90/10 https://en.wikipedia.org/wiki/Pareto_principle https://en.wikipedia.org/wiki/Pareto_principle

balladeer 7y ago

Sometimes I worry about projects about Wikipedia and the Archive. When will they run out of storage space and money? When will it become too big to handle?

ripdog 7y ago
That will never happen for Wikipedia. Their content is tiny in disk space and not growing very fast. They are also drowning in money, despite what their donation begging messages claim. The Internet Archive, on the other hand, is drowning in data and pinching pennies. Still, they're doing okay, and the constant decrease in per-gigabyte HDD costs are allowing them to continue for the time being. They absolutely need and deserve your donations, however. If HDD price/gig stopped going down, however, they might be in a lot of trouble.
- alanbernstein 7y ago
  You're saying they rely on the decreasing prices, rather than low prices? How's that?
  - skosch 7y ago
    The amount of data is growing superlinearly while their resources (probably) aren't, so a dropping price for storage is currently allowing them to keep up (if just barely so).
    - eru 7y ago
      As a mathematical curiosity: an exponential decreasing price means that the total expense to store a Gigabyte forever is finite. Eg a 10% decrease a year, means that storing forever costs approximately ten times the first year's storage cost. https://en.wikipedia.org/wiki/Geometric_series#Sum https://en.wikipedia.org/wiki/Geometric_series#Sum has the math.
      - goodside 7y ago

bduerst 7y ago

That's really not too shabby for a long tail though. What's the % that had zero views in December? Do they account for bots?

Udik 7y ago

> 90 pages were able to gather more than 1M views (Star Wars, The Mandalorian, The Witcher...). So they could cut their network costs by a lot if they just removed pages about current popular culture, which have the highest view rate and a cultural value of exactly zero. Edit: I'm being sarcastic and a bit bitter, the comments below all make good points. However I find it sad, though unavoidable, that the highest page views are for highly marketed entertainment franchises that have a slim chance to withstand the test of time.

dna_polymerase 7y ago
Their network cost is negligible. Overall hosting cost them less then $3 million, actually they spend more on travel & conferences [0]. [0]: https://upload.wikimedia.org/wikipedia/foundation/3/31/Wikimedia_Foundation_Audit_Report_-_FY18-19.pdf https://upload.wikimedia.org/wikipedia/foundation/3/31/Wikim...
tjoff 7y ago
You don't think those views result in a stronger brand and more views on pages with a higher cultural value?
govg 7y ago
Why do they have a cultural value of exactly zero?
- olooney 7y ago
  More to the point, what even is "cultural value?" After Googling it, I found that it doesn't even seem to be a well defined phrase, much less a quantifiable concept. Wikipedia uses notability[1] to decide which articles to include. Notability is defined as significant coverage in 3rd party sources. Popular movies are "notable" because other people write about them. [1]: https://en.wikipedia.org/wiki/Wikipedia:Notability https://en.wikipedia.org/wiki/Wikipedia:Notability
  - friendlybus 7y ago
    There's too much in art that relies on unmeasured effects like the impact on the audience's consciousness and the execution on structure and intention to be bothered with numbers beyond box office or critic reviews. You could measure something related to those concepts and you would be capturing a fourth of available judgement for cultural impact. Culture is a game we have been playing for thousands of years. Art pieces with an observable influence on other artists, the public audience or any human decision making along with an ability to survive the wear and tear of existing in the world as an artwork is a good place to start summarizing an artwork's cultural value. A grand cathedral like st peter's stands as an example of high quality baroque architecture, the value of the church as an institution. It remains intact after a large number of years, not having been physically deconstructed. It's influence on other artists is common to this day, recently used as a baroque example of maximalism on HN. The site attracts visitors and has yet to be rivalled in it's unique baroque grandeur. Those influences are quantifiable, and kind of comparable to a flash in a pan like a Hollywood film that gets most of its attention and influence in the first few months then sits in an institutional back pocket whilst other artists denigrate it so that new films can occupy the spotlight and the cycle continues. These films are sometimes used as reference for making new films and the industry puts out two similar styles of film at the same time reducing uniqueness. Generally otherwise left to sit in DVD/bluray jewel cases or disney vaults as they fall out of favour. You can rank these impacts and have conversations over which interpretive labels they fit into and conclude some are mor valuable than others.

tigerlily 7y ago

This is unremarkable since this is just Zipf's law: https://en.m.wikipedia.org/wiki/Zipf%27s_law https://en.m.wikipedia.org/wiki/Zipf%27s_law

jakear 7y ago

These numbers don’t add up... does the 4.3M pages that got less than 100 include the 3.6M that got less than 10? I should image so, otherwise that’s 7.9M pages. But then you say only 2.3M got less than 1000 views? That obviously can’t include the less than 100 or 10’s, but if you add them then you’re at 6.6M..??

fhoffa 7y ago
I was wondering the same! Not every page is an article, and I'm counting page views from their logs. https://en.m.wikipedia.org/wiki/Wikipedia:What_is_an_article%3F https://en.m.wikipedia.org/wiki/Wikipedia:What_is_an_article... Even then you can see in my query that I have removed most of their "special" pages.
yread 7y ago
I think its a typo 5.3M not 2.3M have less than 1000 views. Makes sense with 600k over 1000 vies and 6M total

H8crilA 7y ago

Ehm, isn't it super obvious that this, too, follows the Pareto distribution?

knbknb 7y ago

Do we have a power-law distribution here? (They are rare)

moh_maya 7y ago
Isn't Pareto a power law distribution?

DonCopal 7y ago

And here you can see the most visited pages for every week: https://en.wikipedia.org/w/index.php?title=Category:Top_25_Report&pagefrom=Top+25+20160220%0ATop+25+Report%2FFebruary+14+to+20%2C+2016#mw-pages https://en.wikipedia.org/w/index.php?title=Category:Top_25_R...

lr4444lr 7y ago

I wonder what the correlation is between inbound links from other articles and views, though.

vladak 7y ago

Whenever seeing the staggering number of Wikipedia articles it reminds me of my own recent experience with creating an article. To centralize some of the knowledge of now niche set of keyboards (manufactured by what used to be one of the FAANGs of its time) I set to create a page. After adding some - what I thought - useful and interesting data, the page got deleted as it was not "notable" enough, i.e. lacking sufficient number of external references. These keyboards did exist and are used until this day however are not as popular (and therefore not as commented/described) like Apple's keyboards (which do have their own page on Wikipedia) so the line was drawn there. In the end I found it funny and also interesting to see how Wikipedia works. If the editors were not so strictly bound to the set of rules there would be probably tens of millions of articles/pages now. Another experience I had was watching a video on YouTube from Japan where a guy was brewing coffee which reminded me that the brand of the coffee equipment used in the video is sometimes sold in coffee shops in Europe. I wanted to know a little bit more about the company so I went to Wikipedia - the English page got deleted because there was little information about the company on the English speaking part of the Internet. This does not say anything about the company (which seems to be hugely popular in Japan and certain social circles in Europe), rather it demonstrates the limit of the model Wikipedia uses for curating the articles.

the_af 7y ago
There is a struggle between "deletionist" and "inclusionist" factions within Wikipedia. One side tends to find fault (many times real faults, mind you) and err on the side of deletion; the other prefers to err on the side of inclusion in Wikipedia. All agree they want to keep some standard of quality in Wikipedia, they just disagree on the method and threshold for deletion. Just be aware Wikipedia is not a single hive-mind :)
- Carpetsmoker 7y ago
  I find all of this really hilarious especially considering that a lot of Wikipedia is about trivial stuff like entertainment (films, sports, anime, whatnot), but whenever someone tries to a document something technical in the wrong way ... NO SOUP FOR YOU! I find it really really rude and disrespectful to just trample over other volunteers' work like this. I'm all for deleting crap, but outright deletion instead of offering suggestions first and then being super-defensive when asked "hey, what's up?" Yeah nah, that doesn't fly. It's like just closing people's PRs on GitHub without offering suggestions on how to fix the patch first with just a single line of "doesn't fit code standards", "doesn't work on Windows", or just a link to a lengthy page. Guess everyone is too busy fighting over capitalisation of movies titles or whether or not the Stanley Kubrick article should have an infobox to offer meaningful suggestions to new users shrug. On Wikipedia, the biggest asshole with the most persistence wins. They've deluded themselves in to thinking this is "meritocracy".