5 ms·

A new way to approach Wayback Machine

41 points by wabarc 5y ago

codetrotter 5y ago
I sometimes find that when I submit a link manually to Wayback or to archive.is, they may temporarily fail to save the link and I need to do it again. Other times they fail in a different way. In particular Wayback may say that a live version is not available even though it is. Does this tool handle resubmitting the link for the kinds of failures that are temporary? And what do you do about permanent failures?
- wabarc 5y ago
  archive.is may have enforced a strictly CAPTCHA policy, causing an exception to the request, and it is hard to resubmit currently.
bobsmooth 5y ago
I recommend the Wayback Machine add-on. Let's you quickly archive pages and automatically serves an archived page for non-working sites. https://addons.mozilla.org/en-CA/firefox/addon/wayback-machine_new/ https://addons.mozilla.org/en-CA/firefox/addon/wayback-machi...
- teddyh 5y ago
  Last updated 3 years ago (Apr 18, 2018)
  - codetrotter 5y ago
    That’s not important. What’s important is, does it still work in the most recent version of Firefox? (I’m on mobile rn so can’t check for myself.)
  - softblush 5y ago
    So? The extension is still being developed[1] but they haven't uploaded newer versions for Firefox recently. Check the Github Issues ([2],[3]) [1] https://github.com/internetarchive/wayback-machine-webextension https://github.com/internetarchive/wayback-machine-webextens... [2] https://github.com/internetarchive/wayback-machine-webextension/issues/689 https://github.com/internetarchive/wayback-machine-webextens... [3] https://github.com/internetarchive/wayback-machine-webextension/issues/740 https://github.com/internetarchive/wayback-machine-webextens...
    - teddyh 5y ago
      > still being developed This does not help anyone since you can’t sideload an unreleased extension in Firefox anymore (except by manually installing it every time you start Firefox, via a “Debug Add-ons” menu).
      - skinnymuch 5y ago
        Wow did not know it’s that hard to sideload. Essentially not possible practically speaking if you want to do it all the time. That sucks.
- smileforburrito 5y ago
  That extension asks for permissions for all tabs, and for all data on all websites. Try https://addons.mozilla.org/en-US/firefox/addon/one-click-wayback/ https://addons.mozilla.org/en-US/firefox/addon/one-click-way... which asks for no permissions. A bookmarklet could work, too.
  - pronoiac 5y ago
    I've been using a bookmarklet in Mobile Safari. Looking for its origin just now, it looks like archive.org uses a Wikipedia page as a reference: https://en.wikipedia.org/wiki/Help:Using_the_Wayback_Machine#JavaScript_bookmarklet https://en.wikipedia.org/wiki/Help:Using_the_Wayback_Machine...
generationP 5y ago
$ curl -sf https://gobinaries.com/wabarc/wayback/cmd/wayback https://gobinaries.com/wabarc/wayback/cmd/wayback | sh uname_os_check 'CYGWIN_NT-10.0' got converted to 'cygwin_nt-10.0' which is not a GOOS value. Please file bug at https://github.com/client9/shlib Hmm... let's add "cygwin*) os="windows" ;;" to the shell file. skraeling@moria ~ $ curl -sf https://gobinaries.com/wabarc/wayback/cmd/wayback https://gobinaries.com/wabarc/wayback/cmd/wayback > wayback skraeling@moria ~ $ nano ./wayback skraeling@moria ~ $ chmod 777 ./wayback skraeling@moria ~ $ ./wayback ==> Downloading github.com/wabarc/wayback/cmd/wayback@master ==> Resolved version master to v0.12.1 ==> Downloading binary for windows amd64 Error downloading, got 500 response from server Am I doing something wrong here?
- wabarc 5y ago
  I'm sorry about that install binary failure from https://gobinaries.com https://gobinaries.com. I've found that the GoBinaries's Go version still 1.15.x, but the wayback required 1.16.x, and it's waiting for the Pull requests https://github.com/tj/gobinaries/pull/39 https://github.com/tj/gobinaries/pull/39. Furthermore, I suggest that you could download it from the releases https://github.com/wabarc/wayback/releases/tag/v0.12.1 https://github.com/wabarc/wayback/releases/tag/v0.12.1.
kevincox 5y ago
I was curious about the IPFS archiving and it appears that it just inlines everything. This is a bit of a shame because with IPFS's content-addressing and deduplication it would be really cool if the assets were out-of-line so that they could be duplicated between webpages. Example of my blog. You can see that the images, CSS and JS are inlined: https://ipfs.io/ipfs/QmcEKGbtFsYrUoDLFMLv45yrZ1hP7ZyyX79x9Ts7TF5rkm https://ipfs.io/ipfs/QmcEKGbtFsYrUoDLFMLv45yrZ1hP7ZyyX79x9Ts... That being said there are problems with inlining IPFS URLs currently. Because few people support ipfs: URLs you essentially need to pick a gateway and embed that into the archived document which is unfortunate. That being said most people who run their own gateway intercept ipfs.io automatically so it is a sort of defacto IPFS scheme. You could probably post-process this yourself if you wanted to to "pop out" the inline blobs.
- wabarc 5y ago
  This sounds like a great idea about archiving assets out-of-line, and it will be one of the goals Wayback Archiver is pursuing.
kordlessagain 5y ago
Been messing about with this: https://mitta.us/https://github.com/wabarc/wayback https://mitta.us/https://github.com/wabarc/wayback

41 points by wabarc 5y ago

I sometimes find that when I submit a link manually to Wayback or to archive.is, they may temporarily fail to save the link and I need to do it again. Other times they fail in a different way. In particular Wayback may say that a live version is not available even though it is. Does this tool handle resubmitting the link for the kinds of failures that are temporary? And what do you do about permanent failures?

wabarc 5y ago
archive.is may have enforced a strictly CAPTCHA policy, causing an exception to the request, and it is hard to resubmit currently.

bobsmooth 5y ago

I recommend the Wayback Machine add-on. Let's you quickly archive pages and automatically serves an archived page for non-working sites. https://addons.mozilla.org/en-CA/firefox/addon/wayback-machine_new/ https://addons.mozilla.org/en-CA/firefox/addon/wayback-machi...

teddyh 5y ago
Last updated 3 years ago (Apr 18, 2018)
- codetrotter 5y ago
  That’s not important. What’s important is, does it still work in the most recent version of Firefox? (I’m on mobile rn so can’t check for myself.)
- softblush 5y ago
  So? The extension is still being developed[1] but they haven't uploaded newer versions for Firefox recently. Check the Github Issues ([2],[3]) [1] https://github.com/internetarchive/wayback-machine-webextension https://github.com/internetarchive/wayback-machine-webextens... [2] https://github.com/internetarchive/wayback-machine-webextension/issues/689 https://github.com/internetarchive/wayback-machine-webextens... [3] https://github.com/internetarchive/wayback-machine-webextension/issues/740 https://github.com/internetarchive/wayback-machine-webextens...
  - teddyh 5y ago
    > still being developed This does not help anyone since you can’t sideload an unreleased extension in Firefox anymore (except by manually installing it every time you start Firefox, via a “Debug Add-ons” menu).
    - skinnymuch 5y ago
      Wow did not know it’s that hard to sideload. Essentially not possible practically speaking if you want to do it all the time. That sucks.
smileforburrito 5y ago
That extension asks for permissions for all tabs, and for all data on all websites. Try https://addons.mozilla.org/en-US/firefox/addon/one-click-wayback/ https://addons.mozilla.org/en-US/firefox/addon/one-click-way... which asks for no permissions. A bookmarklet could work, too.
- pronoiac 5y ago
  I've been using a bookmarklet in Mobile Safari. Looking for its origin just now, it looks like archive.org uses a Wikipedia page as a reference: https://en.wikipedia.org/wiki/Help:Using_the_Wayback_Machine#JavaScript_bookmarklet https://en.wikipedia.org/wiki/Help:Using_the_Wayback_Machine...

generationP 5y ago

$ curl -sf https://gobinaries.com/wabarc/wayback/cmd/wayback https://gobinaries.com/wabarc/wayback/cmd/wayback | sh uname_os_check 'CYGWIN_NT-10.0' got converted to 'cygwin_nt-10.0' which is not a GOOS value. Please file bug at https://github.com/client9/shlib Hmm... let's add "cygwin*) os="windows" ;;" to the shell file. skraeling@moria ~ $ curl -sf https://gobinaries.com/wabarc/wayback/cmd/wayback https://gobinaries.com/wabarc/wayback/cmd/wayback > wayback skraeling@moria ~ $ nano ./wayback skraeling@moria ~ $ chmod 777 ./wayback skraeling@moria ~ $ ./wayback ==> Downloading github.com/wabarc/wayback/cmd/wayback@master ==> Resolved version master to v0.12.1 ==> Downloading binary for windows amd64 Error downloading, got 500 response from server Am I doing something wrong here?

wabarc 5y ago
I'm sorry about that install binary failure from https://gobinaries.com https://gobinaries.com. I've found that the GoBinaries's Go version still 1.15.x, but the wayback required 1.16.x, and it's waiting for the Pull requests https://github.com/tj/gobinaries/pull/39 https://github.com/tj/gobinaries/pull/39. Furthermore, I suggest that you could download it from the releases https://github.com/wabarc/wayback/releases/tag/v0.12.1 https://github.com/wabarc/wayback/releases/tag/v0.12.1.

kevincox 5y ago

I was curious about the IPFS archiving and it appears that it just inlines everything. This is a bit of a shame because with IPFS's content-addressing and deduplication it would be really cool if the assets were out-of-line so that they could be duplicated between webpages. Example of my blog. You can see that the images, CSS and JS are inlined: https://ipfs.io/ipfs/QmcEKGbtFsYrUoDLFMLv45yrZ1hP7ZyyX79x9Ts7TF5rkm https://ipfs.io/ipfs/QmcEKGbtFsYrUoDLFMLv45yrZ1hP7ZyyX79x9Ts... That being said there are problems with inlining IPFS URLs currently. Because few people support ipfs: URLs you essentially need to pick a gateway and embed that into the archived document which is unfortunate. That being said most people who run their own gateway intercept ipfs.io automatically so it is a sort of defacto IPFS scheme. You could probably post-process this yourself if you wanted to to "pop out" the inline blobs.

wabarc 5y ago
This sounds like a great idea about archiving assets out-of-line, and it will be one of the goals Wayback Archiver is pursuing.

kordlessagain 5y ago

Been messing about with this: https://mitta.us/https://github.com/wabarc/wayback https://mitta.us/https://github.com/wabarc/wayback