12 ms·
It clearly hasn't occurred to you that private collections evaporate over time, just like public ones. > I would like something that not only archives pages I
by lutusp 12y ago
It clearly hasn't occurred to you that private collections evaporate over time, just like public ones.
> I would like something that not only archives pages I visit, but also versions them and tracks changes.
That's a huge storage requirement, you must realize this. If you're an avid Web browser, and if every archived page had to look as it originally looked (i.e. all the linked resources) you could accumulate several terabytes per week.
> This type of system would be a huge boost to something like the wayback machine.
Here's the road to madness. Someone, aware of the rapidly declining cost of storage, rebuilds the Wayback machine based on your scheme, with the intent of archiving every Web page in existence, including all required resources so the pages look just as they originally did. Then, as the project approaches completion, this genius says, "For the next phase, I need to archive the Wayback machine itself." At that point, as the implications of what he's said occur to him, a strange look crosses his face and his imagination begins writing checks his intellect can't cash.
- sroerick 12y agoI never said I was trying to save my entire web cache locally. Just bookmarks. I don't have Flash Player installed. Instead, I use youtube-dl for flash videos across many websites. I'm well aware of the storage implications of this kind of activity.
- idlewords 12y agoThe storage implications are really not so bad. You would have trouble ever breaking 100 GB unless you are some kind of bookmarking mutant.
- idlewords 12y agoSeveral terabytes per week is a huge overestimate. Right now I'm getting 12 Mbps download speed (sadly typical for US broadband). If I saturated the connection, and if no one throttled me, I could download 900 gigabytes in a week. That would be some intense web surfing. My practical experience is that you need about 1.5 MB per URL for storing large numbers of web pages, if you exclude video.
- Houshalter 12y agoFor compressed static html and maybe images, you could store your entire history on a flash drive nowdays. See this: http://memkite.com/blog/2014/04/01/technical-feasibility-of-building-hitchikers-guide-to-the-galaxy-i-e-offline-web-search-part-i/ http://memkite.com/blog/2014/04/01/technical-feasibility-of-...