4 ms·
I find SingleFile [0] to be a much more robust version of this. It strips out all the JavaScript too, but also packs everything into a single HTML file that is
by maxloh 4mo ago
I find SingleFile [0] to be a much more robust version of this.
It strips out all the JavaScript too, but also packs everything into a single HTML file that is easy to transfer. Binary assets (like web fonts and images) are packed as base64 strings.
They also offer a CLI powered by Puppeteer. [1]
[0]: https://github.com/gildas-lormeau/singlefile https://github.com/gildas-lormeau/singlefile
[1]: https://github.com/gildas-lormeau/single-file-cli https://github.com/gildas-lormeau/single-file-cli
- tamnd 4mo agoIt seems this repo only saves one web page? What I'm implementing here is mirroring a whole website, with all its subpages, so you can browse it all offline. For example, all essays from paulgraham.com.
- sdevonoes 4mo ago[flagged]
- sermah 4mo agoUm. Whose website are you on right now?
- ivangelion 4mo agoDon't come here to laugh but always great when it happens anyways.
- maxloh 4mo agoOh, I see. In that case, feature-wise, it is actually a modern alternative to HTTrack. I think the misunderstanding stems from the browser's "Save As" reference in the description. It is misleading. You use "Save As" to save a single page, not an entire website. Also, the description lacks a clear explanation of the project's purpose. It would be helpful to include a sentence explaining that the program downloads an entire website, not just a single page.
- sillysaurusx 4mo ago> For example, all essays from paulgraham.com Not the same thing, but I made a clone of pg’s website which can be used for exactly that: https://github.com/shawwn/pg https://github.com/shawwn/pg https://shawwn.github.io/pg/ https://shawwn.github.io/pg/ If you want to read all essays, just clone the repo and open any of the .html files. Or any of the .page files which generated them.
- nikisweeting 4mo agoSinglefile supports scoped recursive crawls too: https://github.com/gildas-lormeau/single-file-cli#:~:text=and%20crawl%20its-,internal%20links,-with%20the%20query https://github.com/gildas-lormeau/single-file-cli#:~:text=an... I highly recommend reading the singlefile source or https://archiveweb.page/ https://archiveweb.page/ to see how they handle closed shadow DOMs, cross-origin iframes, websockets, media urls, deduping large assets, etc.
- tamnd 4mo agoAnd thanks for the link. Let me implement this single HTML feature, it looks nice to have!
- maxloh 4mo agoYeah. An idea on top of that is to bundle an entire website into a single HTML page, with vendored JavaScript to enable client-side routing (all of the original pages' JS is still stripped out). That way, the page is self-contained as it is, but requires no bundled binary code to serve the site. It is actually safer security-wise. The vendored script can be as simple as this: const site = { "path-1": "<!DOCTYPE html><html> ... </html>", "path-2": "<!DOCTYPE html><html> ... </html>", // More paths } function attachListeners() { for (const [path, html] of Object.entries(site)) { document.querySelector(`a[href=${path}]`).onclick = () => { document.documentElement.outerHTML = html attachListeners() } } } document.addEventListeners("DOMContentLoaded", attachListeners)
- HelloUsername 4mo agoWhat's the difference with, any webbrowser on a computer, File -> Save as ?
- nmstoker 4mo agoThat's for a single page, this handles the whole site. Also the browser Save As options often work poorly.
- dmazzoni 4mo agoSave As works fine for simple websites with static content. Let's say you have a site that fetches content from a database. If you Save As, then at best you'll get a local copy of an HTML page with JS that loads the content from the same remote database. It might not work (since the local copy has a different origin), or if it does, it requires you to be online, which defeats half of the purpose. What this project, and SingleFile, both do is save a snapshot of what the rendered page actually looks like at that moment in time. The scripts are stripped out so it runs locally and has no external dependencies.
- deleted 4mo ago[deleted]
- wamatt 4mo agoLove love love SingleFile too. The FF extension works pretty well for a clean save. That said, Kage looks promising if OP can combine SingleFile reproduction quality with the HTTPTrack spidering approach. SPA's are kinda tricky with archiving and do wonder how well Kage would handle that
- initramfs 4mo agoI've seen the option in IE- .mhtml. For some reason it displays in IE better but I don't recall seeing this option in chrome of Firefox recently..
- arikrahman 4mo agoThis is what I first thought and it's a very elegant solution, and not needlessly overcomplicated.