5 ms·

FYI - there’s an official standard (MHTML) for doing this that has existed for 20+ years and exists natively in browsers. https://en.m.wikipedia.org/wiki/MHTML

FYI - there’s an official standard (MHTML) for doing this that has existed for 20+ years and exists natively in browsers. https://en.m.wikipedia.org/wiki/MHTML https://en.m.wikipedia.org/wiki/MHTML

toqy 5y ago
For anyone else that didn't read the README, MHTML is mentioned in the comparison section https://github.com/gildas-lormeau/SingleFile#file-format-comparison https://github.com/gildas-lormeau/SingleFile#file-format-com...
- dsl 5y ago
  Take the comparison with a grain of salt. Not including WARC is like excluding water from a comparison of beverages, it is the baseline standard.
geitir 5y ago
And it generally does not do a good job
- als0 5y ago
  What are the issues?
  - hulitu 5y ago
    From my experience, wrong layout,missing pictures.
  - geitir 5y ago
    The big one in my experience is it doesn't play well at all with JavaScript. Single file to my knowledge (I experimented with it briefly) allows all js to load on page and can then embed loaded media as base64. I think it also has heuristics to embed relevant js as well. It still only gets you 90% of the way there, and I came to the conclusion that unless you are doing web archive type work or need audio / video a composite image works well
ByThyGrace 5y ago
> MHTML, (...) is a web page archive format used to combine, in a single computer file, the HTML code and its companion resources (such as images, Flash animations, Java applets, (...) Well that goes to show its longevity I guess.
rpdillon 5y ago
The browser compatibility section suggests MHTML is unsupported in current versions of Firefox and Safari.
- tekknik 5y ago
  Safari supports webarchive, which does basically the same thing
  - gildas 5y ago
    The problem is that it is a proprietary format. The advantage of the format produced by SingleFile (HTML) is that as long as your browser is capable of interpreting HTML, you will be able to read your archives without worries.
    - tekknik 5y ago
      Not so proprietary. It's really just a plist file, which the format is known and even open sourced by Apple[1]. Really it's only proprietary in that no other platforms have implemented it. [1]: https://opensource.apple.com/source/CF/CF-550/CFBinaryPList.c https://opensource.apple.com/source/CF/CF-550/CFBinaryPList....
- mrspuratic 5y ago
  I don't think it was ever native in Firefox, there is/was the excellent unMHT extension that was broken by Quantum/WebExtensions and The Great XUL Silliness. Shame. I have Waterfox-Classic and unMHT (fished out of the Classic Addons Archive, just remember to turn off Waterfox's multiprocess feature) since I occasionally need to archive web pages - and more importantly, reopen them later. mhtml is just MIME, literally every discrete URL as a MIME part with its origin in a Content-Location header, all wrapped in a multipart container. I don't understand why it's not a default format.
  - Groxx 5y ago
    I can see WebExtensions breaking it (as it's a completely new set of APIs for extensions, and the losses do definitely still hurt)... but quantum/xul? How is that related, aside from "it happened around the same time"?
iKlsR 5y ago
Over a decade ago I had a laptop but no internet at home. This was one of the ways I taught myself programming (and also downloading dozens of manga) by using internet explorer at a cafe which had an option to save to mhtml which was one file and had everything self contained. Legit owe a portion of my success to this. I still have some of these files, old crusty hello world c++ tutorials etc.
- a9h74j 5y ago
  For people who cannot afford internet access now, and for perhaps more in the future if times get more difficult, I believe this is a very important use-case.
- falcolas 5y ago
  I have fantastic internet, and I still do something similar. Local docs just load so much faster, and if something happens (which it still does, even on Fiber in the US), I have docs and can program. Lemme see if I can pull up the command I use to mirror doc sites. wget \ --recursive \ --level=5 \ --convert-links \ --page-requisites \ --wait=1 \ --random-wait \ --timestamping \ --no-parent \ $1
setum 5y ago
IIRC, back in the day mhtml won’t save java applets.
- pstuart 5y ago
  Are any sites still using applets these days?
  - IYasha 5y ago
    80% of server IPMI Web control panels. But who whould want to save those anyway? :)
    - jabbany 5y ago
      A lot of those are getting HTML5/Canvas based implementations and most of the old AST BMCs can get it through upgraded firmware.
      - IYasha 5y ago
        None of my machines had any such upgrades and never will :(
admax88qqq 5y ago
Unfortunately mhtml is not widely supported.
bgro 5y ago
I’ve extensively looked into this as I can’t find a good light and easy backup options that isn’t extreme overkill. I thought MHTML was NOT standardized which is why it wasn’t across all browsers yet. From what I remember, every company was doing their own implementation of it. Maybe it’s gotten more standardized the last few years though.
- chungy 5y ago
  I've always thought the "M" stood for "Microsoft" -- wasn't even aware any browsers other than IE supported it.
  - chme 5y ago
    There is also CHM which is actually a Microsoft only file format for "Compiled HTML Help" files.
    - IYasha 5y ago
      I love this format. Very fast and compact. Entire Visual Studio help was in it once. Worked VERY well. And there's a KDE/Qt reader.
cookiengineer 5y ago
> FYI The alternative format (used by the Internet Archive and Wayback Machine) is WARC. It's also a single file, but it's preserving the HTTP headers as well; so its applications is specifically for archival purposes. [1] The "wget" tool which is co-maintained by the Web Archive people also has support for it via CLI flags. Though when it comes to mobile browser support I'd recommend to use MHTML, because webkit and chromium both have support for it upstream. [1] http://iipc.github.io/warc-specifications/ http://iipc.github.io/warc-specifications/ [2] https://www.gnu.org/software/wget/wget.html https://www.gnu.org/software/wget/wget.html
- londons_explore 5y ago
  Is there any objection to adding WARC support to webkit/chromium? Seems like a not-so-complex project...
  - cookiengineer 5y ago
    I know that WebKit relies on either libsoup [1] (on Linux/Unices) or curl [2] (legacy Windows and maybe WPE(?)) as a network adapter, so the header handling and parsing mechanisms would have to be implemented in there. Though, on MacOS, WebKit tries to migrate most APIs to the Core Foundation Framework, which makes it kind of impossible to implement as a non-Apple-employee because it's basically a dump-it-and-never-care Open Source approach. [3] Don't know about chromium (my knowledge is ~2012ish about their architecture, and pre-Blink). [1] https://github.com/WebKit/WebKit/tree/main/Source/WebKit/NetworkProcess/soup https://github.com/WebKit/WebKit/tree/main/Source/WebKit/Net... [2] https://github.com/WebKit/WebKit/tree/main/Source/WebKit/NetworkProcess/curl https://github.com/WebKit/WebKit/tree/main/Source/WebKit/Net... [3] https://github.com/opensource-apple/CF https://github.com/opensource-apple/CF
    - TingPing 5y ago
      GTK/WPE use libsoup. Playstation/Windows uses curl. And yes Apples networking is proprietary.
twapi 5y ago
I use this Chrome extension to save web pages as MHTML: https://chrome.google.com/webstore/detail/save-webpages-offline-as/nfbcfginnecenjncdjhaminfcienmehn https://chrome.google.com/webstore/detail/save-webpages-offl...
paulirish 5y ago
The Chrome engineer who maintains the MTHML work wrote up a comprehensive doc on the modifications on the MHTML spec (RFC 2557) that are implemented: https://docs.google.com/document/d/1FvmYUC0S0BkdkR7wZsg0hLdKc_qjGnGahBwwa0CdnHE/edit https://docs.google.com/document/d/1FvmYUC0S0BkdkR7wZsg0hLdK... Might be useful for you, gildas.
- gildas 5y ago
  Thank you Paul! I had read this document some time ago, especially to see how the shadow DOM was serialized.
rtsil 5y ago
I remember saving webpages in MHTML when I was using dial-up so that I could read them offline later. I would also download entire websites using a software which name I forgot, to read them offline. Back when websites held in a single floppy disk. Good times!
- TheFlyingFish 5y ago
  I remember using HTTrack for this a while back. Still have a few of those sites lying around, I think.
domador 5y ago
Does anyone else get two security warnings whenever you try to save an MHTML page using a Chrome extension? I have to click on one warning's button to confirm that I indeed want to save the "dangerous" file and another to confirm I'm really sure. It's gotten very annoying. I've looked all over for an option to disable this behavior but haven't been able to.
rplnt 5y ago
I was gonna say Opera (the old, good one) had this. When saving a page there were some options and one was a single file IIRC.

FYI - there’s an official standard (MHTML) for doing this that has existed for 20+ years and exists natively in browsers. https://en.m.wikipedia.org/wiki/MHTML https://en.m.wikipedia.org/wiki/MHTML

toqy 5y ago

For anyone else that didn't read the README, MHTML is mentioned in the comparison section https://github.com/gildas-lormeau/SingleFile#file-format-comparison https://github.com/gildas-lormeau/SingleFile#file-format-com...

dsl 5y ago
Take the comparison with a grain of salt. Not including WARC is like excluding water from a comparison of beverages, it is the baseline standard.

geitir 5y ago

And it generally does not do a good job

als0 5y ago
What are the issues?
- hulitu 5y ago
  From my experience, wrong layout,missing pictures.
- geitir 5y ago
  The big one in my experience is it doesn't play well at all with JavaScript. Single file to my knowledge (I experimented with it briefly) allows all js to load on page and can then embed loaded media as base64. I think it also has heuristics to embed relevant js as well. It still only gets you 90% of the way there, and I came to the conclusion that unless you are doing web archive type work or need audio / video a composite image works well

ByThyGrace 5y ago

> MHTML, (...) is a web page archive format used to combine, in a single computer file, the HTML code and its companion resources (such as images, Flash animations, Java applets, (...) Well that goes to show its longevity I guess.

rpdillon 5y ago

The browser compatibility section suggests MHTML is unsupported in current versions of Firefox and Safari.

tekknik 5y ago
Safari supports webarchive, which does basically the same thing
- gildas 5y ago
  The problem is that it is a proprietary format. The advantage of the format produced by SingleFile (HTML) is that as long as your browser is capable of interpreting HTML, you will be able to read your archives without worries.
  - tekknik 5y ago
    Not so proprietary. It's really just a plist file, which the format is known and even open sourced by Apple[1]. Really it's only proprietary in that no other platforms have implemented it. [1]: https://opensource.apple.com/source/CF/CF-550/CFBinaryPList.c https://opensource.apple.com/source/CF/CF-550/CFBinaryPList....
mrspuratic 5y ago
I don't think it was ever native in Firefox, there is/was the excellent unMHT extension that was broken by Quantum/WebExtensions and The Great XUL Silliness. Shame. I have Waterfox-Classic and unMHT (fished out of the Classic Addons Archive, just remember to turn off Waterfox's multiprocess feature) since I occasionally need to archive web pages - and more importantly, reopen them later. mhtml is just MIME, literally every discrete URL as a MIME part with its origin in a Content-Location header, all wrapped in a multipart container. I don't understand why it's not a default format.
- Groxx 5y ago
  I can see WebExtensions breaking it (as it's a completely new set of APIs for extensions, and the losses do definitely still hurt)... but quantum/xul? How is that related, aside from "it happened around the same time"?

iKlsR 5y ago

Over a decade ago I had a laptop but no internet at home. This was one of the ways I taught myself programming (and also downloading dozens of manga) by using internet explorer at a cafe which had an option to save to mhtml which was one file and had everything self contained. Legit owe a portion of my success to this. I still have some of these files, old crusty hello world c++ tutorials etc.

a9h74j 5y ago
For people who cannot afford internet access now, and for perhaps more in the future if times get more difficult, I believe this is a very important use-case.
falcolas 5y ago
I have fantastic internet, and I still do something similar. Local docs just load so much faster, and if something happens (which it still does, even on Fiber in the US), I have docs and can program. Lemme see if I can pull up the command I use to mirror doc sites. wget \ --recursive \ --level=5 \ --convert-links \ --page-requisites \ --wait=1 \ --random-wait \ --timestamping \ --no-parent \ $1

setum 5y ago

IIRC, back in the day mhtml won’t save java applets.

pstuart 5y ago
Are any sites still using applets these days?
- IYasha 5y ago
  80% of server IPMI Web control panels. But who whould want to save those anyway? :)
  - jabbany 5y ago
    A lot of those are getting HTML5/Canvas based implementations and most of the old AST BMCs can get it through upgraded firmware.
    - IYasha 5y ago
      None of my machines had any such upgrades and never will :(

admax88qqq 5y ago

Unfortunately mhtml is not widely supported.

bgro 5y ago

I’ve extensively looked into this as I can’t find a good light and easy backup options that isn’t extreme overkill. I thought MHTML was NOT standardized which is why it wasn’t across all browsers yet. From what I remember, every company was doing their own implementation of it. Maybe it’s gotten more standardized the last few years though.

chungy 5y ago
I've always thought the "M" stood for "Microsoft" -- wasn't even aware any browsers other than IE supported it.
- chme 5y ago
  There is also CHM which is actually a Microsoft only file format for "Compiled HTML Help" files.
  - IYasha 5y ago
    I love this format. Very fast and compact. Entire Visual Studio help was in it once. Worked VERY well. And there's a KDE/Qt reader.

cookiengineer 5y ago

> FYI The alternative format (used by the Internet Archive and Wayback Machine) is WARC. It's also a single file, but it's preserving the HTTP headers as well; so its applications is specifically for archival purposes. [1] The "wget" tool which is co-maintained by the Web Archive people also has support for it via CLI flags. Though when it comes to mobile browser support I'd recommend to use MHTML, because webkit and chromium both have support for it upstream. [1] http://iipc.github.io/warc-specifications/ http://iipc.github.io/warc-specifications/ [2] https://www.gnu.org/software/wget/wget.html https://www.gnu.org/software/wget/wget.html

londons_explore 5y ago
Is there any objection to adding WARC support to webkit/chromium? Seems like a not-so-complex project...
- cookiengineer 5y ago
  I know that WebKit relies on either libsoup [1] (on Linux/Unices) or curl [2] (legacy Windows and maybe WPE(?)) as a network adapter, so the header handling and parsing mechanisms would have to be implemented in there. Though, on MacOS, WebKit tries to migrate most APIs to the Core Foundation Framework, which makes it kind of impossible to implement as a non-Apple-employee because it's basically a dump-it-and-never-care Open Source approach. [3] Don't know about chromium (my knowledge is ~2012ish about their architecture, and pre-Blink). [1] https://github.com/WebKit/WebKit/tree/main/Source/WebKit/NetworkProcess/soup https://github.com/WebKit/WebKit/tree/main/Source/WebKit/Net... [2] https://github.com/WebKit/WebKit/tree/main/Source/WebKit/NetworkProcess/curl https://github.com/WebKit/WebKit/tree/main/Source/WebKit/Net... [3] https://github.com/opensource-apple/CF https://github.com/opensource-apple/CF
  - TingPing 5y ago
    GTK/WPE use libsoup. Playstation/Windows uses curl. And yes Apples networking is proprietary.

twapi 5y ago

I use this Chrome extension to save web pages as MHTML: https://chrome.google.com/webstore/detail/save-webpages-offline-as/nfbcfginnecenjncdjhaminfcienmehn https://chrome.google.com/webstore/detail/save-webpages-offl...

paulirish 5y ago

The Chrome engineer who maintains the MTHML work wrote up a comprehensive doc on the modifications on the MHTML spec (RFC 2557) that are implemented: https://docs.google.com/document/d/1FvmYUC0S0BkdkR7wZsg0hLdKc_qjGnGahBwwa0CdnHE/edit https://docs.google.com/document/d/1FvmYUC0S0BkdkR7wZsg0hLdK... Might be useful for you, gildas.

gildas 5y ago
Thank you Paul! I had read this document some time ago, especially to see how the shadow DOM was serialized.

rtsil 5y ago

I remember saving webpages in MHTML when I was using dial-up so that I could read them offline later. I would also download entire websites using a software which name I forgot, to read them offline. Back when websites held in a single floppy disk. Good times!

TheFlyingFish 5y ago
I remember using HTTrack for this a while back. Still have a few of those sites lying around, I think.

domador 5y ago

Does anyone else get two security warnings whenever you try to save an MHTML page using a Chrome extension? I have to click on one warning's button to confirm that I indeed want to save the "dangerous" file and another to confirm I'm really sure. It's gotten very annoying. I've looked all over for an option to disable this behavior but haven't been able to.

rplnt 5y ago

I was gonna say Opera (the old, good one) had this. When saving a page there were some options and one was a single file IIRC.