4 ms·
It's mostly composed of PDFs actually. Judging from my own library the average publication is about 2 MB, and I believe about 1e8 publications are currently ind
by radus 8y ago
It's mostly composed of PDFs actually. Judging from my own library the average publication is about 2 MB, and I believe about 1e8 publications are currently indexed, so it's on the order of hundreds of TB.
- emilfihlman 8y agoEither a) those pdfs contain images or other graphics that don't compress easily or b) the text is not compressed.
- radus 8y agoThere are definitely many images and graphics. I don't know if my sample is representative of most literature, but I don't think it's too far off..
- gpm 8y agoOr c) The text is terribly compressed. Probably a lot of the "text" is also a scan of text as well. Could be an interesting project to try and make a sci-hub optimized compression system.
- Quiark 8y agoDejaVu format
- Reelin 8y agoThis got me curious. For reference, my Zotero library is predominantly biomedical in nature. This device currently has 810 of the pdf files pulled down to the local drive, of which the vast majority are journal articles (a very few of the largest ones are presentation slide decks). Size Range Count [0, 0.5) 255 (31%) [0.5, 1) 179 (22%) [1, 1.5) 108 (13%) [1.5, 2) 69 (9%) [2, 3) 80 (10%) [3, 10) 95 (12%) > 10 24 (3%) That's ~50% < 1 MB and ~25% > 2 MB. I'm actually a bit surprised they're that small.
- agumonkey 8y agobtw anyone knows if there are PDF shrinkers ? some journals can weight a lot (scanned pages) even though there's potential to recompress images while keeping it readable
- harryposner 8y agoI've used Ghostscript to shrink scanned PDFs for emailing, although I don't know how lossy the compression is. [0] https://askubuntu.com/a/256449 https://askubuntu.com/a/256449
- agumonkey 8y agoI tried it but it was cpu intensive for very small improvements