3 ms·

Why is it not feasible to store and later rebuild the page DOM and view state? This is an annoyance across all browsers; there's no built-in way to bottle up t

Why is it not feasible to store and later rebuild the page DOM and view state? This is an annoyance across all browsers; there's no built-in way to bottle up the current state except by printing to PDF, which then becomes unclickable. I spent time one summer playing around building a capturing system addon for Firefox which bottles a browser page DOM state, all assets, and necessary CSS styles inline into one single blob of HTML for long-term preservation. There were a lot of subtle gotchas to work out (which made for a stimulating exercise) but after some time it worked just fine, so it's 100% totally achievable. I'd like to understand why this functionality isn't provided with every tier-1 browser.

Salgat 4y ago
Saving state is probably pretty involved once you factor in Javascript.
- mixedCase 4y ago
  Given that they already have coroutine support... how hard could it really be? Honest question for people familiar with V8/complex JIT compilers .
  - fathyb 4y ago
    Saving the VM state is not hard at all, the feature already exists (v8 snapshot) and is used in Chrome and Node.js to initialize the JS standard library. The problem is native handles. The DOM is easy to restore, but how do you restore a WebSocket connection? An IndexedDB transaction? A `Blob`, a stream, or a WebAssembly module that JavaScript assumes to be initiated? There are solutions to each one of these, but there are still a lot of remaining pieces that are stateful and not easy to restore or gracefully close. It's possible, but it requires a lot of work.
    - vlovich123 4y ago
      I could be mistaken but I don’t believe that v8 snapshots support saving the vm in an arbitrary state. The snapshot is created after the script is compiled and that’s about all you can use to restore (it’s for speeding up loading the JS only).
kwhitefoot 4y ago
If you actually want to save the rendered HTML of a page you can use the SingleFile Firefox addon: https://addons.mozilla.org/en-US/firefox/addon/single-file/ https://addons.mozilla.org/en-US/firefox/addon/single-file/ I think it's available for Chrome as well. It is able to embed images in the file as well as data urls. Edit: I use this to capture social media web pages that I have contributed to so that I have a copy in case they get deleted.
zarzavat 4y ago
> Why is it not feasible to store and later rebuild the page DOM and view state? It is. It's called swap. They should just throttle the JS down to zero, and then let the OS deal with the memory. Each tab is a separate process. What they need is the opposite of mlock, a way to tell the OS to pretty please swap out this process's memory because it is unlikely to be used in the foreseeable future.
throwaway14356 4y ago
a while back on hn there was talk of an archive project or webscraper that "got it right"[sic] It didn't rewrite the html but served files by the original path. It sounds like the right approach. Maybe someone remembers what it was called?
krackers 4y ago
The mechanism for this already exists, and it saves the state of JS as well: https://web.dev/bfcache/ https://web.dev/bfcache/
- fathyb 4y ago
  This mechanism is not doing any serialization, it just disconnects the page from the renderer and puts it into a memory cache, keeping the v8 VM initiated, but paused. It cannot be used to save a tab into the disk. https://source.chromium.org/chromium/chromium/src/+/main:content/browser/renderer_host/stored_page.h;drc=e1591f7e48004bf610e6e53a4f213d032e221f23;l=36 https://source.chromium.org/chromium/chromium/src/+/main:con...
dspillett 4y ago
> Why is it not feasible to store and later rebuild the page DOM and view state? For the purposes of saving memory, which is what tab discarding is intended for, where should it be stored? * In RAM? It is already there and that is the problem. Maybe you could compress it when inactive, but that won't be effective for pages with much image or other media data. * To disk? At that point you might as well leave it in RAM but inactive and let the OS page it out as needed. Maybe marking it as inactive internally (so your own garbage collection sweeps don't make the OS think the data is active) and to the OS (assuming there is a concept of this-is-low-priority-so-page-it-out-before-other-stuff at that level). For storage, save-entire-page-to-disk which has existed in most browsers since early versions does most of the job well enough, with opening the stored result in the same browser being the rehydration method, that it might not be worth creating and maintaining something more complex for the remaining part of the job to cover all the cases that this doesn't. Though the balance may have changed, as even simple pages are often effectively SPAs these days there may be more edge cases that “save all” doesn't work with then cases where it is fine. > I'd like to understand why this functionality isn't provided with every tier-1 browser. Lack of demand. While it would be very useful for a minority of users, there aren't enough of them for the feature to affect market share, so the big browsers aren't going to bother. It is the sort of thing niche browsers/add-ons might have more interest in.

Salgat 4y ago

Saving state is probably pretty involved once you factor in Javascript.

mixedCase 4y ago
Given that they already have coroutine support... how hard could it really be? Honest question for people familiar with V8/complex JIT compilers .
- fathyb 4y ago
  Saving the VM state is not hard at all, the feature already exists (v8 snapshot) and is used in Chrome and Node.js to initialize the JS standard library. The problem is native handles. The DOM is easy to restore, but how do you restore a WebSocket connection? An IndexedDB transaction? A `Blob`, a stream, or a WebAssembly module that JavaScript assumes to be initiated? There are solutions to each one of these, but there are still a lot of remaining pieces that are stateful and not easy to restore or gracefully close. It's possible, but it requires a lot of work.
  - vlovich123 4y ago
    I could be mistaken but I don’t believe that v8 snapshots support saving the vm in an arbitrary state. The snapshot is created after the script is compiled and that’s about all you can use to restore (it’s for speeding up loading the JS only).

kwhitefoot 4y ago

If you actually want to save the rendered HTML of a page you can use the SingleFile Firefox addon: https://addons.mozilla.org/en-US/firefox/addon/single-file/ https://addons.mozilla.org/en-US/firefox/addon/single-file/ I think it's available for Chrome as well. It is able to embed images in the file as well as data urls. Edit: I use this to capture social media web pages that I have contributed to so that I have a copy in case they get deleted.

zarzavat 4y ago

> Why is it not feasible to store and later rebuild the page DOM and view state? It is. It's called swap. They should just throttle the JS down to zero, and then let the OS deal with the memory. Each tab is a separate process. What they need is the opposite of mlock, a way to tell the OS to pretty please swap out this process's memory because it is unlikely to be used in the foreseeable future.

throwaway14356 4y ago

a while back on hn there was talk of an archive project or webscraper that "got it right"[sic] It didn't rewrite the html but served files by the original path. It sounds like the right approach. Maybe someone remembers what it was called?

krackers 4y ago

The mechanism for this already exists, and it saves the state of JS as well: https://web.dev/bfcache/ https://web.dev/bfcache/

fathyb 4y ago
This mechanism is not doing any serialization, it just disconnects the page from the renderer and puts it into a memory cache, keeping the v8 VM initiated, but paused. It cannot be used to save a tab into the disk. https://source.chromium.org/chromium/chromium/src/+/main:content/browser/renderer_host/stored_page.h;drc=e1591f7e48004bf610e6e53a4f213d032e221f23;l=36 https://source.chromium.org/chromium/chromium/src/+/main:con...

dspillett 4y ago

> Why is it not feasible to store and later rebuild the page DOM and view state? For the purposes of saving memory, which is what tab discarding is intended for, where should it be stored? * In RAM? It is already there and that is the problem. Maybe you could compress it when inactive, but that won't be effective for pages with much image or other media data. * To disk? At that point you might as well leave it in RAM but inactive and let the OS page it out as needed. Maybe marking it as inactive internally (so your own garbage collection sweeps don't make the OS think the data is active) and to the OS (assuming there is a concept of this-is-low-priority-so-page-it-out-before-other-stuff at that level). For storage, save-entire-page-to-disk which has existed in most browsers since early versions does most of the job well enough, with opening the stored result in the same browser being the rehydration method, that it might not be worth creating and maintaining something more complex for the remaining part of the job to cover all the cases that this doesn't. Though the balance may have changed, as even simple pages are often effectively SPAs these days there may be more edge cases that “save all” doesn't work with then cases where it is fine. > I'd like to understand why this functionality isn't provided with every tier-1 browser. Lack of demand. While it would be very useful for a minority of users, there aren't enough of them for the feature to affect market share, so the big browsers aren't going to bother. It is the sort of thing niche browsers/add-ons might have more interest in.