6 ms·
Show HN: DiskerNet – Browse the Internet from Your Disk, Now Open Source
- Tomte 3y agoIt's working as a proxy, I suppose? Do you have some set-up information beyond building it?
- keepamovin 3y agoGood point re instructions. I've added an issue for that. IIRC you can basically just clone it and run it with node. Shouldn't be too hard. :) Not a proxy tho. More better! :)
- Tomte 3y agoI still don‘t understand how it tracks what I‘m surfing. Is it watching cache folders of Edge and Firefox?
- tenebrisalietum 3y agoThere is some project number that begins with the number 2 and is five digits (refers to a port number) - it is a Python app that uses Chrome debug features to save everything you are doing. I don't know how it would work with Firefox unless the debug functionality there is the same. Memory is hazy on this so I might have some details wrong.
- armitron 3y agoIt hooks browser requests/responses and saves responses as-is under a key based on the request. It can then replay the original response. Has been done multiple times before, the model is cumbersome / hard to manage, has plenty of annoying edge cases where it completely fails and doesn't work well with streaming audio/video.
- keepamovin 3y agoTrue that it completely fails with audio and video streaming (tho really that's probably achievable), but if you want that content there's many good tools. The advantage is that it just saves content to disk and revives it, so the browser seems as if it's online, when it may not be. It doesn't alter the content in anyway, and doesn't need to compress or rewrite anything in order to fit into a single file or strange archive format, or whatever. I'm sure there are uses for those, but this is not that. It just saves each resource to disk as it receives it. Actually, I think the edge cases of this "high fidelity" (but not necessarily "broadband", as it--for now--excludes audio/video streaming), are less than with archive formats where you need to rewrite or otherwise alter the content. But the web is vast, you probably have a different experience! :) As far as the browser is concerned, I think all the CORS, HTTPS-mandatory stuff just works (at least it used to!). BTW - I was not aware of this having been done before? Do you have any links? Very interested!
- mxuribe 3y agoIt sounds similar to ArchiveBox (https://archivebox.io/ https://archivebox.io/) - which is not a bad thing; if it is, then the more, the merrier!
- freedomben 3y agoSimilar in some ways, quite different in others. They can both be used to archive/snapshot web content, but they take very different approaches.
- chillbill 3y agoDoes it provide full text search? How does it handle content behind paywalls or content requiring auth?
- keepamovin 3y agoWell, it piggybacks on your browsing (obviously you are "opted in" because you are using it knowingly to archive what you are browsing!), so everything you can see, it can see. If you don't have access over the great paywall, neither does it. But a cool thing is, it can index the content you can auth to. The full text search is all local. This is not the "local client" of some service. It's just a local app, there is not server that anything is sent to (maybe one day there could be like a federated search, or a way to publish and share your archives, but there is nothing like that now). Full text search is provided by a couple of different libraries. Fuzzy search works.
- Alifatisk 3y agoImagine this + some sort of p2p network where all our local archive becomes available to each other.
- keepamovin 3y agoYeah I love that idea. Plus search. Imagine building your own Google, based on a specialist expert archive of the stuff you're really good at and sharing that out to whoever wants it...and federating it into a giant search engine. Like Yahoo
- Alifatisk 3y agoGood addition, in this way, we all would be a gigantic crawler. I think the trade-off here is the latency and data-loss? Or data-loss could be resolved by introducing some sort of redudancy. Like how seeding works in torrents.
- felipeqq2 3y agoYou should check out https://yacy.net https://yacy.net: a global, P2P web search engine, where each peer can build and share its own index, etc.
- Alifatisk 3y agoCool find, this seems to be a good option. SearX also came to my mind but isn't exactly p2p.
- password4321 3y agoYaCy discussed almost a year ago: https://news.ycombinator.com/item?id=32597309 https://news.ycombinator.com/item?id=32597309
- myself248 3y agoSounds like a perfect use case for NNCP. It works over any transport, from normal networks to radio links to sneakernet.
- rickcarlino 3y agoLooks interesting. It would be great if the README explained how it works. It sounds like it is some sort of proxy-like web service that does aggressive local caching?
- keepamovin 3y agoSorry, I should update that. Basically yes, but not a proxy.
- Alifatisk 3y agoFound this which seems a bit similar, archivebox.io
- hiatus 3y agoThis sounds cool but there are no instructions beyond how to download/install. After installing with npm, I tried to invoke it with `diskernet` and got some inscrutable error. https://pastebin.com/7kmcG9Fp https://pastebin.com/7kmcG9Fp
- andix 3y agoSounds like an awesome idea. To make it even more useful it should be a Http proxy that I can set up easily on all my devices (smartphone, tablet, pcs, ...). All my efforts in this direction suffered from the problem that I'm using a lot of different devices and platforms for browsing.
- mmcclure 3y agoThis makes me miss when browsers just had “offline mode” built right in. It certainly wasn’t foolproof, but it was often enough.
- Dylan16807 3y agoThese days just trying to save an image redownloads it most of the time. Browsers cache use has really regressed.
- beagle3 3y agoMuch of it, perhaps all of it, is a result of trackers/advertisers abusing caching for tracking. We can’t have nice things.
- Dylan16807 3y agoThere is the non-default paranoid option of disabling caches entirely. But in general the mitigation only requires isolation between sites, which doesn't have too much of an impact and shouldn't affect saving at all.
- beagle3 3y agoIsolation between sites is not sufficient - collaborating sites (and everyone colllaborates with e.g. Google analytics or the hundred other tracking sites) could have an ifrane that would abuse cache. You need isolation by primary site, and i don’t remember the exact issues, but there’s some obstacles over there that need addressing (and that browsers do address, but which result in the cache-hardly-ever-used situation we are in today).
- Dylan16807 3y agoRight, that's the isolation I was thinking of. Other than jquery or google font, what does that stop from caching? Most assets only get loaded from a single site.
- yewenjie 3y agoWas this not on GitHub for long time now?
- marcescence 3y agoI think this task is better accomplished by Zotero (at least the offline bookmarking part), but it's always nice to have alternatives.