8 ms·
The Internet Archive is back online
- seestem 2y agoIt would be better if the Internet Archive was decentralized without a central point of failure, maybe run on something like bittorent.
- dusted 2y agoI kind of agree, but the way the internet is going, with everyone being behind carrier-grade nat, it's not much of a decentralized network of computers anymore, not to mention all the kids with their laptops and tablets not even hosting anything :(
- Kuinox 2y agoUPnP exists and allow devices to ask the router to open a port to them.
- cosarara 2y agoThat doesnt help with CGNAT.
- Dalewyn 2y agoUPnP is just automating the process of forwarding ports, CGNAT will still screw you sideways because you're behind a router you can't access or order around.
- Fidelix 2y agoUPnP is useless with CGNAT (Carrier Grade NAT), which is what the op is talking about. There are other ways to get seeding working, though, including IPV6, which is gaining adoption, so I don't agree with the OP.
- deleted 2y ago[deleted]
- nikisweeting 2y agoThere are ways around this, I've experimented with setting up a cluster of ArchiveBox instances that share snapshots over Tailscale. Tailscale lets users sign up for free accounts, and you can share machines between separate accounts. A (CGNAT-compatible) decentralized invite-only network could concievably spread that way.
- uniqueuid 2y agoThe problem is that it's hard to do this in a way that ensures good archival of ALL resources. Bittorrent works well for popular things but fails for marginal content (unless some really dedicated individuals step in.) What the internet archive provides is a way to have access to many many resources which you didn't know you needed in advance.
- maire 2y agoI don't know if bittorrent has improved - but 20 years ago I had a personal issue with it. At that time our son was using it for games. He goes away to college and came home for the first school break. I get a phone call from our internet provider asking if our son was home. I was so shocked and handed the phone to our son. Apparently at that time bittorrent was optimizing for the most efficient path to a host. Since we had relatively good connection, the mighty weight of the internet was funnelling through our tiny internet provider to our son's computer. The provider (without our knowing it) had made a deal with our son that he would only turn on bittorrent between midnight and 6 AM. I doubt other providers would be so generous. I have been sceptical of bittorrent since that day.
- jetrink 2y agoAll clients today (and probably back then) have options to limit bandwidth consumption including throttling, scheduling, and total data transfer caps. For serving mostly HTML and images, dedicating even 10% of a home broadband connection to serving content would allow many, many people per day to access archived pages.
- Cheer2171 2y agoYou say this as if it is an original idea. Of course the IA is working on this and have been for over 6 years. There already is a DWeb version. They have been advancing DWeb infrastructure. The IA hosts all kinds of DWeb developer events. But it is over 50 petabytes and the IA gets a huge amount of traffic through the regular web that they need to serve quickly and efficiently to their users. Guess what has happened over 6 years of decentralization of 50 TB? People only seed what they want or care about and there aren't enough seeders to host. They set all this up and nobody volunteers. You're a DWeb advocate and you haven't been seeding. That's a recipe for disaster if they rely on the goodness of volunteer seeders. The IA's mission is broader. DWeb will ever only compliment the IAs mission. https://blog.archive.org/2021/02/18/behind-the-scenes-of-the-decentralized-web-principles/ https://blog.archive.org/2021/02/18/behind-the-scenes-of-the... https://www.bleepingcomputer.com/news/technology/archiveorg-has-created-a-decentralized-or-dweb-version-of-their-site/ https://www.bleepingcomputer.com/news/technology/archiveorg-...
- mrtksn 2y agoThat would be an awful lot of replication or very shitty archive. Decentralization works when each node can serve all the functions and content alone or when you don't care about completeness. Unless I'm missing something, an archive is not something small or something that's just as good when part of it is missing.
- nikisweeting 2y agoI'm working on this, ArchiveBox v0.8 adds the beginnings of a content addressable store, with plans for bittorrent-backed instance-to-instance sharing in a later version. I think Archive.org should still exist too (and ArchiveBox donates + submits URLs to Archive.org too), but having a self-hosted option where you can archive personal stuff that requires a login, and do P2P sharing with with fine grained permissions is a gap that should be filled. Aiming to archive the entire internet is Archive.org's goal, aiming to archive the part of the internet YOU care about is our goal.
- sourcepluck 2y agoI'm hoping that Autonomi (formerly The Safe Network) is up for the job when (if) it makes it out into the real world one day https://forum.autonomi.community/t/the-internet-archive-a-perfect-partner/40479 https://forum.autonomi.community/t/the-internet-archive-a-pe... [I know that some percentage between 95 and 100 of crypto projects are a scam. I personally believe this one isn't, after much diligent reading. Whether it gets released or does what it claims it will do is another question, but please do spare me the kneejerk anti-crypto reactions, if you can. Just because they're almost all money-making scams, doesn't mean they're all money-making scams.]
- thrownaway561 2y agothat will never happen. no one is going to be able to seed the amount of data that IA has. The only thing they can hope for is that a company like Google or CF provides another data center for them.
- throwaway48476 2y agoWhen the internet archive censors a website is it deleted permanently or just not publicly available?
- chirau 2y agoI don't think they censor anything, strictly archiving. Do you know of any instance in which they censored a site?
- throwaway48476 2y agohttp://web.archive.org/web/20240000000000*/twitter.com/taylorlorenz http://web.archive.org/web/20240000000000*/twitter.com/taylo... For one. I'm just curious what their policy is.
- lukas099 2y agoI only know what I just read on wikipedia about her, but it seems like she has been heavily doxxed — I'm guessing she requested this information about herself be excluded? If so, I'm not sure I'd classify that as censorship.
- throwaway48476 2y agoIt's her own tweets, not dox.
- lukas099 2y agoNot dox but I was thinking there could be old materials in there that people were using to dox her. Idk, why else would they remove it?
- deleted 2y ago[deleted]
- 2y ago
- onetokeoverthe 2y agoStill down in my town.
- mananaysiempre 2y agoThe Internet Archive is not, in fact, completely online (as the article explains but the title doesn’t). The Wayback Machine, which is part of it, is kind of online but (in my experience) you are going to experience HTTP 504 timeouts from time to time on the first query for a given (URL, date) pair as it seemingly goes out to slower storage. (Long delays happened in the past occasionally as well but not to the point of a 504.)
- tiffanyh 2y agoI thought Cloudflare was going to provide "Always Online" access to Internet Archive https://blog.cloudflare.com/cloudflares-always-online-and-the-internet-archive-team-up-to-fight-origin-errors/ https://blog.cloudflare.com/cloudflares-always-online-and-th...
- adambb 2y agoOther way around! Cloudflare can optionally load your site from IA if it's down.
- imglorp 2y agoIf that's the case, I hope CF is making a big, periodic, donation to IA for the business value provided.
- diggan 2y agoIdeally, CF would keep track of exactly how many redirects they do to IA, and donate based on the usage. Would be more fair for everyone involved.
- tourmalinetaco 2y agoCloudflare willingly keeps CSAM and animal abuse sites online even when reported, the least they can do is cut IA a fat check every month.
- joenot443 2y agoMy intuition is that there's a mutually beneficial deal hammered out behind the scenes and that CF isn't just eating poor IA's lunch.
- TZubiri 2y agoDo you happen to remember how you learned this? I'm quite skeptical of it.
- jgrahamc 2y ago
- 0xedd 2y ago[dead]
- ChrisArchitect 2y ago[dupe] https://news.ycombinator.com/item?id=41856008 https://news.ycombinator.com/item?id=41856008
- LetsGetTechnicl 2y agoThere have been so many instances since it's been down that I tried to access IA resources and realized they were unavailable. I'm still bitter that of all the targets a hacker could've chose, it was the IA. Couldn't have happened to a better website. I plan on upping my monthly donation as soon as I can.
- Alifatisk 2y agoHope these hackers receive lots of negative reactions from their peers and people around them.
- 7402 2y agoSome source information about the group that has claimed responsibility: "A group known as SN_Blackmeta claimed responsibility for the attack, with a confusing antisemitic message that the archive “belongs to the USA” as if it were a government project." https://9to5mac.com/2024/10/15/internet-archive-data-breach-exposes-31m-users-under-ddos-attack/ https://9to5mac.com/2024/10/15/internet-archive-data-breach-... "Internet Archive Cyber Attacked by Pro-Palestinian Hackers" https://www.cybersecurityintelligence.com/blog/internet-archive-cyber-attacked-by-pro-palestinian-hackers-7998.html https://www.cybersecurityintelligence.com/blog/internet-arch... "Anti-Israel hacker group hacks 'Internet Archive', exposing 31 million users" https://www.ynetnews.com/business/article/bkird2rjke https://www.ynetnews.com/business/article/bkird2rjke
- Smithalicious 2y agoCan I say "false flag"?
- pcthrowaway 2y agoYeah, there's plenty of material which I've only been able to easily find on archive.org which does not look good for Israel. The dominant powers seem to be more effective at keeping their propaganda hosted than the resistance, so a takedown of the IA seems very much in line to me with Israeli capabilities and motives.
- binary132 2y agojust build your own 960 billion website archive
- idlewords 2y agoWorking on it.
- hersko 2y agoI wonder if it would be possible to identify and prosecute those responsible.
- serendipty01 2y agoDoes someone know where i can download MIT OCW videos ? As the videos are present on archive.org but it is down and i was unable to find them anywhere else online ? Also, yt-dlp is also not working: https://github.com/yt-dlp/yt-dlp/issues/10128 https://github.com/yt-dlp/yt-dlp/issues/10128 Example: https://ocw.mit.edu/courses/7-016-introductory-biology-fall-2018/download/ https://ocw.mit.edu/courses/7-016-introductory-biology-fall-...
- adamnew123456 2y agoI seed a torrent of the SICP lectures that originally came from IA, I'll have to see if that's still up and if there's some way of getting the other torrents from the tracker. If you're lucky there's other seeds around, and not just the IA web seeds which (I assume?) are down too.
- adamnew123456 2y agoNo such luck :( Both the bt1.archive.org and b2.archive.org trackers appear to be down. magnet:?xt=urn:btih:1814b8e2673e8a4547fd9c4f1a417b05860230b4&dn=MIT_Structure_of_Computer_Programs_1986&tr=http%3A%2F%2Fbt1.archive.org%3A6969%2Fannounce&tr=http%3A%2F%2Fbt2.archive.org%3A6969%2Fannounce&ws=https%3A%2F%2Farchive.org%2Fdownload%2F&ws=http%3A%2F%2Fia600204.us.archive.org%2F15%2Fitems%2F
- PeterCorless 2y agoAttacking the Internet Archive is like robbing from your own grandmother.
- twosdai 2y agoThank the internet.
- TruffleLabs 2y agoThere are other things still not available, like this! “Lisp lore : a guide to programming the Lisp machine” https://archive.org/details/lisploreguidetop0000brom https://archive.org/details/lisploreguidetop0000brom I discover this reference and boom the Internet Archive book is not available:( “Wayback Machine (provisional, read-only) service. Other Internet Archive services are temporarily offline. Please check our official accounts, including Twitter/X, Bluesky or Mastodon for the latest information. We apologize for the inconvenience.”
- sandwichmonger 2y agoI'm really disappointed with the Internet Archive's level of unprofessionalism when it comes to any form of downtime whatsoever (let alone the blog). The monolithic one stop shop "Internet Library" can not even bother to put updates on their downtime page and instead directs their user base to social media platforms. Call me a nut, but I feel the IA would work better if it was run by the Library of Congress, but then again that has it's own pitfalls.
- AceStar 2y agoThis article appears to be referring to just the Wayback Machine. The Internet Archive itself is still down.
- deleted 2y ago[deleted]
- jawshwa 2y agoWhat a goober