11 ms·
Firefox’s latest Test Pilot: No More 404s
- J_Darnley 10y agoDoes this mean it only took Firefox some 10 years to integrate (part of) the Resurrect Pages extension?
- Sylos 10y agoNot every possible feature has to be part of Firefox. And this is for now also just a suggestion that Mozilla is making, so that users can decide over it in the Test Pilot program.
- robert_tweed 10y agoNice idea. My only concern is that it could lead to an increase in overzealous removal requests, especially if the idea is copied by other vendors.
- J_Darnley 10y agoYou don't have to request anything. Just alter robots.txt and you can make the Wayback Machine memory hole your entire website.
- idsout 10y agoThe keyword here is 'your'.
- sp332 10y agoWho else would be sending takedown requests?
- Skunkleton 10y agoContent owners that already DCMA'd their content on some third parties website which the wayback machine has backed up.
- dingaling 10y agoOr someone else's website to which you now own the domain name
- minikites 10y agoIt's so infuriating when I come across a domain squatter that nuked the entire history of a domain in the Wayback Machine. I sort of get why they have to do that but it also defeats most of the point of the Wayback Machine.
- slrz 10y agoI sort of get why they have to do that I don't. Can you explain?
- btgeekboy 10y agoArchive.org doesn't know the domain changed hands, just that it used to be allowed to show the results but now no longer is.
- dragonwriter 10y agoDoesn't really explain why they have to nuke it, even if it is the current site owner. Respecting robots.txt is one thing, but that just means not spidering and archiving the content that is now there. Deleting already archived material based on later changes to robots.txt is a non-obvious behavior, given the usual understanding of the general meaning of robots.txt.
- jgruen 10y agoThe Test Pilot program is pretty small compared to the total traffic to the Wayback Machine. The Internet Archive guys were not worried about this, so I'm not either.
- speps 10y agoA huge amount of traffic to Wayback Machine, I hope they donated some money.
- fzzzy 10y agoIt is just a Test Pilot experiment, not on by default. People have to install Test Pilot and turn on this experiment.
- speps 10y agoHow many people are in Test Pilot?
- fzzzy 10y agoWell, the https://testpilot.firefox.com/experiments https://testpilot.firefox.com/experiments page shows the number of people participating in each experiment. Currently, Activity Stream has the most participants with 36,056; No More 404s is currently showing 2,468.
- jklinger410 10y agoAs someone who uses Wayback all the time for work I would not be a fan of this extra traffic slowing the tool down.
- XMPPwocky 10y agoHow much does your employer donate to the Internet Archive?
- deleted 10y ago[deleted]
- davidascher 10y agoThis was done in partnership with the Internet Archive folks -- they _want_ the traffic, in part because this helps people know about the Archive, but mostly because this is a public benefit feature, which is what non-profits strive to provide. As to the amount of traffic it generates, that's one reason why we're doing this through Test Pilot -- we can dial that up or down.
- kenrick95 10y agoTake note that it is an opt-in Test Pilot project.
- abstractbeliefs 10y ago"No More 404s" has been a long time coming, I remember it was one of the original projects announced with the launch of Test Pilot. You can find more upcoming projects and information on Test Pilot here: https://testpilot.firefox.com/ https://testpilot.firefox.com/
- swehner 10y agoSounds like better left for a browser plugin
- pbhjpbhj 10y agoMy first thought too. Firefox seems to be cramming stuff in as default recently that's really peripheral; strikes me we're approaching that point in the cycle where a new streamlined offering comes along and people start jumping ship to that. The place where Firefox started as a simple fast web browser without the bells and whistles.
- sp332 10y agoThere are many, many broken links on the web and it seems like this feature could get a lot of use.
- seba_dos1 10y agoAs TFA says, it is a browser plugin.
- Sylos 10y agoJust so this is clear: This is not a new feature in Firefox. They are just giving people a way to test it and to give feedback. Depending on the response, it might then become a feature, but for now it's only a proposition.
- jgruen 10y agoThis is an opt-in experiment in an opt in program.
- lmorchard 10y agoThe irony with your comment is that this is exactly one of the purposes of Test Pilot - to give new features a trial run rather than just "cramming stuff in as default". It's an opt-in experiment, and at the end it's possible to a) be rolled in as a feature, b) spun off as an optional add-on, c) scrapped entirely. Oh yeah, or d) taken back to the drawing board & re-engineered for another go-round.
- swehner 10y agoSounds like better left for a browser plugin
- sp332 10y agohttp://archive.org/donate/ http://archive.org/donate/ They're a non-profit, it's tax-deductible! The Archive is a lot bigger than just than the Wayback Machine. The collections include books, audio, video, and software.
- stephancoral 10y agoAre any of these copies being archived in physical media offsite (like thru Iron Mountain)?
- sp332 10y agoThe data is stored redundantly (3x) in California, and there are several datacenter locations around the world, but I'm not sure any of them are a full backup of the entire collection. There is an unofficial project by volunteers to make a geographically-distributed backup of the Archive. http://www.archiveteam.org/index.php?title=INTERNETARCHIVE.BAK http://www.archiveteam.org/index.php?title=INTERNETARCHIVE.B... The basic tech is functioning now, you can run it http://www.archiveteam.org/index.php?title=INTERNETARCHIVE.BAK/git-annex_implementation http://www.archiveteam.org/index.php?title=INTERNETARCHIVE.B... but we're looking for developers to make it more user-friendly. If you'd like to help out, stop by #internetarchive.bak on efnet. http://chat.efnet.org http://chat.efnet.org
- aroch 10y agoIf one had, say, 1-2PB in unused mag tape but no longer has the hardware to actually write to them -- is there some way to get them put to use?
- dublinben 10y agoYou should contact the archive team folks. It's likely that someone else might have the necessary drives to write them.
- ElijahLynn 10y agoOff topic but you seem like you may work for Archive.org... I tried uploading some Drupal meetup source videos a while ago but it took 1 a week to upload a 100GB file due to what appeared to be an upload cap, so I gave up that idea. I would still really like to upload the videos but I was trying this on a gigabit pipe and it was capped. I submitted feedback in the 'new uploader' thread but never got a response. Is there really an upload cap? How can I upload faster so it is more feasible? Thank you
- mynewtb 10y agoHow does this handle oblivious users entering their credentials to forms? I have seen that happen in the wild and it was really hard to explain. I don't think this is a great idea, it would make me worried about the Wayback Machine.
- throwanem 10y ago> it was really hard to explain No doubt; I'm not sure what you're describing here.
- mywittyname 10y agoI think his/her question is: what about POSTs from forms to 404 URLs. So, I User put my user/pass into a form field and hit submit. The browser makes a post to site.com/login with my information, but the server returns a 404. What happens in this case?
- eridius 10y agoI'm going to guess that it doesn't re-POST to the wayback machine, because that wouldn't make much sense and the wayback machine probably wouldn't even accept that.
- throwanem 10y agoThe same thing that happens in any other case: you're presented a banner that includes a link to a Wayback Machine search for the URL that 404ed. There's no magic here; it's just a link that produces a GET request when you click it. I feel like a lot of the concerns and objections raised in this comment thread originate in not having taken a minute to find out what this extension actually does.
- mynewtb 10y agoNope, I meant user getting 404 on a website they meant to log in to, getting redirected to Wayback copy and entering their credentials there.
- 10y ago
- zeveb 10y agoAn interesting idea, but I'm concerned about the privacy implications. Does it send all 404 URLs to the Wayback Machine to see if it has a copy? That reveals one's browsing habits without one affirmatively doing anything. Does it strip URL query parameters? These are commonly used for session IDs and auth tokens. Does it strip URL path parameters? Conceivably these might have private information in them as well. For that matter, paths themselves may contain private information. If all it does is offer a link to the Wayback Machine, then it sounds great. But I worry that folks might want it to do more, without realising what that might mean.
- deleted 10y ago[deleted]
- BoringCode 10y agoThe article (and the screenshot) seem to imply that it doesn't actually check with the Wayback Machine to see if the URL exists. It only displays a prompt telling you that you can try to view it there. So theoretically it would only send the URL to the Wayback Machine if you chose to use its service.
- jgruen 10y agoThe No More 404s experiment, like all of Test Pilot, is totally opt-in. The whole idea is to let us try things with Firefox, get feedback and iterate quickly. No More 404s Telemetry ping is only gathering data about how often the Add-on is fired and how often it is clicked. We (Mozilla) don't know anything about URLs. The Wayback Machine has it's own privacy protections in place that you can learn about here: https://blog.archive.org/2013/10/25/reader-privacy-at-the-internet-archive/ https://blog.archive.org/2013/10/25/reader-privacy-at-the-in... Also worth noting, every Test Pilot experiment comes with a brief explainer of all data collections. Here's the explainer for No More 404s: In addition to the data collected by all Test Pilot experiments, here are the key things you should know about what is happening when you use No More 404s: * We collect basic usage on how many times you encounter a Page Not Found error (code 404), how many times a cached version of that page exists from Archive.org, and how many times you choose to view the cached version. * To provide cached versions of pages, we send 404 error page URLs to Archive.org. Archive.org discloses its privacy policy here (https://archive.org/about/terms.php https://archive.org/about/terms.php). * We do not collect URLs of the pages you request or the URLs we send to Archive.org. * We may share survey results you submit to us and aggregated telemetry data related to this experiment with the Internet Archive.
- wvh 10y agoThe title makes it sound as if this is the default, a unconditional redirect, which appears not to be the case; it seems to offer an optional redirect, which sounds like a better idea. I don't think unconditionally redirecting users would be the way to go; it would be better for the end user to clearly understand the page they were trying to access is not there, to prevent security issues, spam or other abuse resulting from the confusion and automation. To offer the user clearly explained options after a clear 404 does sound like a good idea however, granted that the Wayback Machine can (and wants to) handle the additional load.
- awqrre 10y agoIt appears that you need to click a link to be redirected to the archived version, but it may still be a privacy issue if they check if an archived version of the page exist before showing the link: > The notification reads: "This page appears to be missing. View a saved version courtesy of the Wayback Machine". You may click on the link to open the Internet Archive website to read an archived snapshot of the page on the site.
- sctb 10y agoWe updated the title from “Firefox latest now directs users to the Wayback Machine on a 404” to that of the article.
- DonaldFisk 10y agoGood idea. One of the ideas in Ted Nelson's Xanadu project was unbreakable links. The problem on the WWW is that you don't always get a 404. Sometimes the original web site goes up for sale and you end up with something like this: http://www.strictlybowhunting.com/Anov01issue/crows.htm http://www.strictlybowhunting.com/Anov01issue/crows.htm which is not a 404. It would have to recognize that as being equivalent to a 404 and serve up the archive.org page anyway. Ideally, a URL should allow the option of including a date.
- slrz 10y agoMan, do I hate that crap. Oracle is another ugly example. After buying Sun, some genius there apparently thought that it's OK to break tons of links all over the web (and in Usenet archives, etc.), redirecting them to the fucking generic Oracle front page. Nothing spells out "fuck you" quite as clearly as breaking thousands of hyperlinks overnight.
- dublinben 10y agoEven better are content-addressable hashes, which will always point to the correct information, no matter what.
- ape4 10y agoJust change the error number, problem fixed :)
- Karunamon 10y agoNeat! Are there any plans to integrate, or allow integration of other services? If you could search for a dead URL on Wayback, Archive.is, Google Cache, and some others, you'd have a lot of coverage, and in the case of the last two, be immunte to IA's (frankly boneheaded) robots.txt policy.
- urda 10y agoWhy was this flagged?
- akkartik 10y agoLooks like the ghacks domain has really poor karma: https://news.ycombinator.com/from?site=ghacks.net https://news.ycombinator.com/from?site=ghacks.net
- merqurio 10y agoLet's see how long until someone claims to the Wayback Machine their right to be forgotten. I really like the idea of ephemerality on the internet. I kind of enjoy to see thing come and go; that nothing lasts forever.
- pYQAJ6Zm 10y agoIn the European Union, the “right to be forgotten” on the Internet is protected by law. I believe it is the right thing to do, but I recognize the issue rests on a delicate equilibrium, maybe yet to be found. On the one hand, we wouldn’t expect our daily interactions to be recorded and stored for long. Most of them, anyway – and certainly so for our most personal ones. On the other hand, the web is a medium where things tend to last a while, if not for very long thanks to efforts such as the Internet Archive’s. The delicacy of where to rest the equilibrium, I think, stems from the particularity of the web as a medium itself: a web page is kind of like ye olde book, but it adds the possibility of easy changes and user interactivity. In other words, if so used, a web will sit between traditional written media and face-to-face (or technologically mediated, but unrecorded) interaction. Perhaps by custom, perhaps by nature, we expect personal interaction to be ephemeral, and we have no problem with other media, such as books, newspapers, etc., to store their contents for long. It’s no harm to publish a book with our thoughts and have it found across libraries in the world (indeed, it is a very nice thing), but we would feel violated if our conversations at home where recorded without proper cause. The web stays somewhere in between. It’s a place for words to lie and last, but also for personal interaction. Personally, I favor ephemerality on the Internet, but don’t see it as characteristic of it. It’s not a given, and we should find a place for it, but only as far as personal interactions go. I want to be able to remove (or, at least, anonymize) whatever I post online in the manner of a casual conversation. But I want other kinds of content, such as manuals, news, etc., to be preserved from ephemerality. I’m not sure whether The Internet Archive stands within these bounds. I would like to see it more as an opt-in facility for website administrators, instead of requiring manual forbidding.
- perfectfire 10y agoI publish an open source Chrome plugin that does this for 404 or 503 errors. Except it tries the Google cache first and only hits the Wayback Machine if it gets a 404 from the Google cache. You can also turn it on and off manually if you don't want automatic redirection. When it finds a page in the Google cache or Wayback Machine it will change all links on that page to cache links, so if you're trying to browse a site that's down (rather than a single page) it's completely seamless. I started a Firefox extension and got the basic functionality working before getting bored reimplenting something I had already done. But now I've switched to Firefox for Android I'm thinking of reviving it, so I can have my plugin on mobile too. It's been pretty useful. I use it a couple times a week usually with about a 75% success rate at getting a cached or Wayback Machine page of a page that is either down or no longer has the content that used to be there. Edit: Also, it only turns on per-tab instead of browser-wide which I could see annoyed everybody that used similar plugins for Chrome that would turn the whole browser into "browse cache-mode" when activated.
- avarun 10y agoCan you link it? I would love to try it out.
- perfectfire 10y agoHere's the Chrome web store link: https://chrome.google.com/webstore/detail/google-cache-browser/kmjckhnhmegjfffjogbdefcbddmmchea https://chrome.google.com/webstore/detail/google-cache-brows... And here's the source: https://github.com/joelself/GoogleCacheBrowser https://github.com/joelself/GoogleCacheBrowser
- TheWorldIsFun 10y agoIf by chrome plugin you meant extension then you might not have to recreate your chrome extension from scratch for firefox. Look at firefox's webExtensions page, it's pretty much compatible with most of chrome's api and you probably won't have to change your code all that much.
- 10y ago
- makecheck 10y agoThe problem is, I don’t always want the URL that returns 404. In fact, I’d say almost every time I don’t want it; I’m seeing the page because I made a typo. When I have a mistake, the only thing I want is my UNALTERED original input, and a chance to fix it. The ire-inducing thing about most “helper” pages, including the ad-ridden search pages that ISPs like to return instead of 404, is that they complete rewrite my address bar. This means that instead of being able to fix a single character and try again, I most likely have to retype the whole damned thing. Especially on mobile, there’s nothing quite like entering "simplething.com/foo", accidentally typing "simplethin.com/foo", and being redirected to "godawfulisp.com/applications/helper.jsp?unnecessarycrap=1&adtracker=2&garbage=xyz&useextraobnoxiousads=true". At that point, my options are “go back” (returning a blank page) or trying to “select” the URL field (which is horrible on the iPhone) and reentering everything. All because I saw a page I didn’t even want. This made me so mad that I configured special blocker patterns to ensure that ISP pages could not even be loaded. In other words, please stop “helping” users if you don’t really understand what the problem is. Sometimes an error and the unaltered original input is exactly the right thing to show.