4 ms·

How big is the Wikipedia database? Current revision only of just the text of the articles is 8.8GB compressed, 27GB uncompressed. All revisions, all pages (tal

by jarcane 11y ago

How big is the Wikipedia database? Current revision only of just the text of the articles is 8.8GB compressed, 27GB uncompressed. All revisions, all pages (talk etc. included) of text is around 5 terabytes. [...] Neither of those include media. [0] The biggest version of this device has 32GB of storage. The math is not hard to follow. This is the kind of transparently infeasible pipe dream I expect from Kickstarter projects. [0] https://www.quora.com/How-many-gigabytes-or-terabytes-of-information-is-Wikipedia https://www.quora.com/How-many-gigabytes-or-terabytes-of-inf...

placeybordeaux 11y ago
> made up of frequently accessed Wikipedia articles Wikipedia has a very long tail, no need to load articles such as Jabal al Utayfah or Poul Reumert or Screaming Life/Fopp.
- jawarner 11y ago
  Huh. What percent of pages would estimate get less than say 100 organic views a month?
  - placeybordeaux 11y ago
    The data appears to be avaliable here: http://dumps.wikimedia.org/other/pagecounts-raw/ http://dumps.wikimedia.org/other/pagecounts-raw/ Might be another way to answer that question, but I didn't find it here: https://en.wikipedia.org/wiki/Wikipedia:Statistics#Page_views https://en.wikipedia.org/wiki/Wikipedia:Statistics#Page_view...
  - rasz_pl 11y ago
    This is somewhat dangerous way of looking at things and will lead you to believe kittens are more important than math.
EmanueleAina 11y ago
> just the text of the articles is 8.8GB compressed Which seems to mean that storing only the version in the user's language should not pose a huge challenge, storage-wise. No?
- DanBC 11y ago
  I think that 8.8 GB is just EN wiki. https://en.wikipedia.org/wiki/English_Wikipedia https://en.wikipedia.org/wiki/English_Wikipedia > There are 5,049,507 articles on the site (live count).[4] In October 2015, the combined text of the English Wikipedia's articles totalled 11.5 gigabytes when compressed.[5]
wzdd 11y ago
I upvoted you. The pitch doesn't reflect reality. The computer actually consists of an offline reader for popular Wikipedia articles and some parts of Khan Academy. It also has some open-source music and a couple of games. This is only storing "as much of the Internet as it can" insofar as 32GB means storing very, very little.

by jarcane 11y ago

placeybordeaux 11y ago

> made up of frequently accessed Wikipedia articles Wikipedia has a very long tail, no need to load articles such as Jabal al Utayfah or Poul Reumert or Screaming Life/Fopp.

jawarner 11y ago
Huh. What percent of pages would estimate get less than say 100 organic views a month?
- placeybordeaux 11y ago
  The data appears to be avaliable here: http://dumps.wikimedia.org/other/pagecounts-raw/ http://dumps.wikimedia.org/other/pagecounts-raw/ Might be another way to answer that question, but I didn't find it here: https://en.wikipedia.org/wiki/Wikipedia:Statistics#Page_views https://en.wikipedia.org/wiki/Wikipedia:Statistics#Page_view...
- rasz_pl 11y ago
  This is somewhat dangerous way of looking at things and will lead you to believe kittens are more important than math.

EmanueleAina 11y ago

> just the text of the articles is 8.8GB compressed Which seems to mean that storing only the version in the user's language should not pose a huge challenge, storage-wise. No?

DanBC 11y ago
I think that 8.8 GB is just EN wiki. https://en.wikipedia.org/wiki/English_Wikipedia https://en.wikipedia.org/wiki/English_Wikipedia > There are 5,049,507 articles on the site (live count).[4] In October 2015, the combined text of the English Wikipedia's articles totalled 11.5 gigabytes when compressed.[5]

wzdd 11y ago

I upvoted you. The pitch doesn't reflect reality. The computer actually consists of an offline reader for popular Wikipedia articles and some parts of Khan Academy. It also has some open-source music and a couple of games. This is only storing "as much of the Internet as it can" insofar as 32GB means storing very, very little.