4 ms·

Browsers evolve and break older pages. JS that requires requests as you interact with it will need an implementation of the server it uses (the subset and data

Browsers evolve and break older pages. JS that requires requests as you interact with it will need an implementation of the server it uses (the subset and data of it's endpoints) How do you preserve that?

deleted 5y ago
[deleted]
tinco 5y ago
Browsers don't usually break older pages. The only time this happens when you rely on unstandardized features. At least, I have not noticed any page breaking in the past 15 years except for the ones I built using unstandardized API's.
- mappu 5y ago
  The changing behaviour of browser autocomplete and the new disregard for autocomplete="off" really harmed multiple large CRM / ERP-style sites i worked on, as passwords would get "helpfully" autofilled into completely wrong fields, causing data loss. I actually still don't think there's a proper sanctioned solution to this, it seems to be cat-and-moused by web developers and browser developers every year or two.
- deleted 5y ago
  [deleted]
- Macha 5y ago
  The largest browser maker no longer believes so: https://www.theregister.com/2021/08/05/google_chrome_iframe/ https://www.theregister.com/2021/08/05/google_chrome_iframe/ https://mobile.twitter.com/estark37/status/1422694856544059396 https://mobile.twitter.com/estark37/status/14226948565440593...
- asddubs 5y ago
  -https sites embedding http images -SameSite:none cookies (with bonus breakage that makes it impossible to use accross older and newer browsers simultaneously without user agent sniffing) -the planned chrome alert/prompt changes already mentioned below browsers have been getting a lot more comfortable with the idea of breaking backwards compatibility as of late.
- spicybright 5y ago
  > except for the ones I built using unstandardized API's. Think you proved my point.
deleted 5y ago
[deleted]
programmarchy 5y ago
Maybe a good example is the original Nintendo (NES) emulators. New gaming consoles can't play those old cartridges, but we have a virtual layer that can. The same holds true for browsers, OSs, etc. It does create a pretty long chain of dependencies, though.
- spicybright 5y ago
  Ignoring the network point I made above, it'll be a monumental effort to get there. Best we can hope for is virtual machines, and archive files targeting a specific browser version on specific virtual machines.
tenebrisalietum 5y ago
I've had some success in the past using a WARC proxy - it will basically record everything that traverses the browser and can "play it back" on demand. So while it won't automatically download everything on a site the idea is that whatever you visit with and interact with in a session can be "played back" in the future at some point.
- genewitch 5y ago
  I actually built a machine to do this, and wanted to use l my "old" spindle drives as storage, but I was and am unhappy with the offerings of replicated or raid like storage solutions. I keep waiting for ceph or something to have a better NFS layer, but I might just have to have scripts that balance/replicate the warc however many times based on usage. For those interested I have 43TB, 210TB, 61-2TB, 212TB. The machine boots and sees all the spindles. ZFS would be ideal but cannot handle differing drive sizes I guess. JBOD misses the replication requirement. Just doing mdraid on all the similar drives and having different folders or JBOD "wastes" too many spindles (at least one per group, so 4-6 wasted spindles by size alone!). So I'll probably handroll something with whatever that triggered rsync program is, or cronjobs. lsyncd, that's the one.
tshaddox 5y ago
How is that any different than wanting to archive a CGI website from the 90s with a URL structure like http://example.com/?query=foo http://example.com/?query=foo? Unless there's an index page with links to all possible query values, or you can work out how to manually iterate all possible query values, there's not much you can do. This doesn't seem to have anything to do with JavaScript data visualizations specifically.
- titusjohnson 5y ago
  That URL structure is trivial for a crawler to walk and index. I'm not sure why you'd assume that there wouldn't be an index page, such a site would have all desired links in the DOM, the crawler just sniffs those out and visits in sequence. There's no need to think that the links would somehow be 'hidden' from the user and have to be randomly enumerated... Not only that, but a site of that era probably also has a sitemap.xml file which would enumerate all available public endpoints, specifically to make it easier for crawlers to index everything.
  - tshaddox 5y ago
    > I'm not sure why you'd assume that there wouldn't be an index page I’m not assuming either way. I’m just pointing out that either type of web site could choose to have an index page or choose not to have an index page.
selfhoster11 5y ago
If it's view-only, request-response replaying could be a viable option. Browser software can be emulated.

deleted 5y ago

[deleted]

tinco 5y ago

Browsers don't usually break older pages. The only time this happens when you rely on unstandardized features. At least, I have not noticed any page breaking in the past 15 years except for the ones I built using unstandardized API's.

mappu 5y ago
The changing behaviour of browser autocomplete and the new disregard for autocomplete="off" really harmed multiple large CRM / ERP-style sites i worked on, as passwords would get "helpfully" autofilled into completely wrong fields, causing data loss. I actually still don't think there's a proper sanctioned solution to this, it seems to be cat-and-moused by web developers and browser developers every year or two.
deleted 5y ago
[deleted]
Macha 5y ago
The largest browser maker no longer believes so: https://www.theregister.com/2021/08/05/google_chrome_iframe/ https://www.theregister.com/2021/08/05/google_chrome_iframe/ https://mobile.twitter.com/estark37/status/1422694856544059396 https://mobile.twitter.com/estark37/status/14226948565440593...
asddubs 5y ago
-https sites embedding http images -SameSite:none cookies (with bonus breakage that makes it impossible to use accross older and newer browsers simultaneously without user agent sniffing) -the planned chrome alert/prompt changes already mentioned below browsers have been getting a lot more comfortable with the idea of breaking backwards compatibility as of late.
spicybright 5y ago
> except for the ones I built using unstandardized API's. Think you proved my point.

deleted 5y ago

[deleted]

programmarchy 5y ago

Maybe a good example is the original Nintendo (NES) emulators. New gaming consoles can't play those old cartridges, but we have a virtual layer that can. The same holds true for browsers, OSs, etc. It does create a pretty long chain of dependencies, though.

spicybright 5y ago
Ignoring the network point I made above, it'll be a monumental effort to get there. Best we can hope for is virtual machines, and archive files targeting a specific browser version on specific virtual machines.

tenebrisalietum 5y ago

I've had some success in the past using a WARC proxy - it will basically record everything that traverses the browser and can "play it back" on demand. So while it won't automatically download everything on a site the idea is that whatever you visit with and interact with in a session can be "played back" in the future at some point.

genewitch 5y ago
I actually built a machine to do this, and wanted to use l my "old" spindle drives as storage, but I was and am unhappy with the offerings of replicated or raid like storage solutions. I keep waiting for ceph or something to have a better NFS layer, but I might just have to have scripts that balance/replicate the warc however many times based on usage. For those interested I have 43TB, 210TB, 61-2TB, 212TB. The machine boots and sees all the spindles. ZFS would be ideal but cannot handle differing drive sizes I guess. JBOD misses the replication requirement. Just doing mdraid on all the similar drives and having different folders or JBOD "wastes" too many spindles (at least one per group, so 4-6 wasted spindles by size alone!). So I'll probably handroll something with whatever that triggered rsync program is, or cronjobs. lsyncd, that's the one.

tshaddox 5y ago

How is that any different than wanting to archive a CGI website from the 90s with a URL structure like http://example.com/?query=foo http://example.com/?query=foo? Unless there's an index page with links to all possible query values, or you can work out how to manually iterate all possible query values, there's not much you can do. This doesn't seem to have anything to do with JavaScript data visualizations specifically.

titusjohnson 5y ago
That URL structure is trivial for a crawler to walk and index. I'm not sure why you'd assume that there wouldn't be an index page, such a site would have all desired links in the DOM, the crawler just sniffs those out and visits in sequence. There's no need to think that the links would somehow be 'hidden' from the user and have to be randomly enumerated... Not only that, but a site of that era probably also has a sitemap.xml file which would enumerate all available public endpoints, specifically to make it easier for crawlers to index everything.
- tshaddox 5y ago
  > I'm not sure why you'd assume that there wouldn't be an index page I’m not assuming either way. I’m just pointing out that either type of web site could choose to have an index page or choose not to have an index page.

selfhoster11 5y ago

If it's view-only, request-response replaying could be a viable option. Browser software can be emulated.