10 ms·

Internet Archive, decentralized

• krautsourced 8y ago
  Look, feel and content-wise this feels a lot like the internet of the 90s...
  • pmoriarty 8y ago
    Unfortunately, it's not 90's enough, as it still requires fucking javascript.
    • theoctopus 8y ago
      This is one of the (admittedly few) cases where javascript is actually required.
      • mitraardron 8y ago
        Of course it is, Javascript is what allows us to run significant code on the browsers without extensions or plugins or downloaded apps/peers, and its amazing how far its come as a language in the last few years. WebAssembly would probably work as well, but was far less developed when we started this project last year.
    • SquareWheel 8y ago
      Would you like to add more dependencies on server-side code for this decentralized project?
• ryanlol 8y ago
  Unsupported mediatype: data :(
  • mitraardron 8y ago
    Yes - we don't yet support some of the more unusual types on the Archive (including "data" and "software")
• CuriousSkeptic 8y ago
  When cloudflare screwed up and published everyones secrets there was a coordinated effort involving, amongst others, archive.org to try and scrub the internet of those secrets. Are there any mechanism available to allow similar efforts with this dweb version?
  • comboy 8y ago
    There should not be a mechanism for that. China mastered such mechanisms. We should rather optimize for not having that kind of points of failure.
    • deleted 8y ago
      [deleted]
    • burkemw3 8y ago
      In addition, optimize for dealing with those situations: secret rotation plans
      • pjc50 8y ago
        "Please change the name of your first pet, the school you went to, and any distinguishing ratios on your body"
        • vageli 8y ago
          People actually fill those questions in with real info? Those types of questions are easily compromised with social media.
        • crtasm 8y ago
          Tiddles. Ridgemont High. Golden.
        • newscracker 8y ago
          That's a funny take! At least the first two look like they need time machines, which would circumvent the solutions for the main issue in question here.
• marknadal 8y ago
  GUN author here (https://github.com/amark/gun https://github.com/amark/gun) happy to answer any questions. IA did this integration in 1 week, Mitra is awesome. Also, decentralized Reddit (https://notabug.io https://notabug.io) was built in 1 week on us, and pushed 0.5TB P2P traffic on 1st day. Note: I may not be awake for several hours, and might not be able to reply until Monday.
  • pandasun 8y ago
    Wow, your work (GUN) is really great. I hope you don't mind me asking how it works in layman's terms? Thank you for your work.
    • marknadal 8y ago
      Huge honor! Yes: - http://gun.js.org/distributed/matters.html http://gun.js.org/distributed/matters.html - https://youtu.be/EHZyaupYjYo?t=55m52s https://youtu.be/EHZyaupYjYo?t=55m52s - http://gun.js.org/explainers/data/security.html http://gun.js.org/explainers/data/security.html (not about GUN itself, directly, but SEA) Is at least 1 of those helpful? If not, have any ideas on how I can improve the explanation?
  • modernerd 8y ago
    GUN looks great! I love the quick-start tutorial too: https://gun.eco/think.html https://gun.eco/think.html How suitable is GUN for live multiplayer (non turn-based) web games? (Similar to https://airma.sh/. https://airma.sh/.) I see one game example at https://github.com/amark/gun/wiki/Awesome-GUN https://github.com/amark/gun/wiki/Awesome-GUN, although it's turn-based.
    • marknadal 8y ago
      Thank you!!! I need to fix the organization of the documentation (and update the docs, oye!). I'm impressed you found Awesome-GUN. https://github.com/amark/gun/blob/master/examples/game/space.html https://github.com/amark/gun/blob/master/examples/game/space... is probably what you were searching for. What would be better is if I made a blog/tutorial for ^ link. Not a priority for me, sadly, but maybe it is for somebody out there, that they could help? Thanks again!
• nanna 8y ago
  How does this work?
• krylon 8y ago
  This is really cool! A bit sluggish, but I love the idea! If you think back to, say, the library of Alexandria, to how much knowledge has been lost over the ages, it is so important to preserve as much as we can for future generations. And building a decentralized foundation for this archive is a big step going forward, congratulations!
  • pixelpoet 8y ago
    I recently went to the Internet Archive in SF, got a tour of their operations etc. Absolutely amazing place, very forward thinking, and they indeed are very serious about prevent something like the destruction of the Library of Alexandria from happening again.
    • krylon 8y ago
      There is a (great, IMHO) science fiction novel, "The Mote In God's Eye" by Larry Niven and Jerry Pournelle that describes humanity's first contact with another intelligent species. Due to their biology, this alien species experiences periods of massive population growth that eventually lead to all-out war and collapse of civilization. Over uncounted thousands, maybe millions of years, these aliens have accepted this vicious cycle as kismet, and deal with it by building "warehouses" filled their most advanced technology to jump start civilization after the next, inevitable, collapse. I hope humanity will never have to deal with such a collapse, but I given our collective tendency towards self-destructive behavior, maybe we should build such an archive as if it was meant for future cavemen to jump-start them into a new Anthropocene. Even if that collapse never happens (I am keeping my fingers crossed!), the resulting tome of knowledge would be a suitable monument to all the incredible things that humanity has accomplished, as well as an insurance policy in case we manage to mess up on a monumental scale. EDIT: A more positive perspective would be The Library from David Brin's uplift saga, a humongous collection of knowledge acquired by many, many species over millions, if not billions of years. EDIT: typo
• agumonkey 8y ago
  very nice
• Kagerjay 8y ago
  I didn't see any articles relating to how it was decentralized. Is internet archive going for something similar to a blockchain setup, because that would actually be one of the few cases it would make sense I am not familiar with this new dweb subdomain, what is unique about it? Archive has actually lost some of the archives I stored on it, which is weird, because I ran multitude of backups on it a few years back when the site got taken offline
  • justin_ 8y ago
    I wanted to submit an article about this but couldn't find anything either. My understanding is that this loads content from various protocols (listed at the top of the page), many of which support replicating data in a decentralized way. As far as I know, there's no blockchain involved in anything here yet. If you browse to the Community Video section and choose a video, you can see peer information as though downloading through WebTorrent. If I disable WebTorrent and look at a video, I don't see the peer information and it seems to fall back to HTTP. Pretty cool! It looks like almost everything is only seeded by the Internet Archive right now, but hopefully they want to encourage more people to participate.
    • Kagerjay 8y ago
      Hm that's interesting, I imagine the internet archive has lots of seedboxes/CDNs/datacenters distributed globally. I have no idea how much information the internet archive is currently backing up though, but its growing at an increasing rate. I would love to see a write up of how their infrastructure works though
      • toomuchtodo 8y ago
        My understanding is all of their digital content is stored and served from their location in SF, although it would be awesome if they started geographically distributing their storage nodes. http://archive.org/web/petabox.php http://archive.org/web/petabox.php
        • 8y ago
• drc0 8y ago
  on firefox it hangs past http.
  • mitraardron 8y ago
    Its still an experiment to see what was possible with a big site, in today's browsers using some of the emerging tools (IPFS, WebTorrent, GUN). We only test on absolutely current releases of Chrome & Firefox. If you run into problems on the site, there is a feedback button, otherwise you can report directly on https://docs.google.com/forms/d/e/1FAIpQLSe7pXiSLrmeLoKvlDi2wODcL3ro7D6LegPksb86jr5bCJa7Ig/viewform https://docs.google.com/forms/d/e/1FAIpQLSe7pXiSLrmeLoKvlDi2... or open an issue on https://github.com/internetarchive/dweb-archive/issues https://github.com/internetarchive/dweb-archive/issues
• nikisweeting 8y ago
  Aww I was going to add almost exactly the same thing to https://github.com/pirate/bookmark-archiver https://github.com/pirate/bookmark-archiver but IA beat me to the punch! I still hope to add decentralized storage and lookup mechanisms to BA eventually, but considering one BA's archive outputs is the Internet Archive, it's less pressing now.
• ezequiel-garzon 8y ago
  I strongly believe IA or any serious project working on permanent persistence must provide an option (opt-in if you will) to make the published material irremovable, à la arXiv [1]. [1] Red parts of https://arxiv.org/help/license https://arxiv.org/help/license
  • quickben 8y ago
    And that is your right. However most people would like to remove slander with a court order.
    • ezequiel-garzon 8y ago
      Pretty sure arXiv would remove slander with a court order as well. It's an unrelated issue.
      • crooked-v 8y ago
        If material can be removed, it's not irremovable.
        • mehrdadn 8y ago
          I think it's fair to assume "not-${VERB}able" means "as not-${VERB}able as legally possible".
          • ndr 8y ago
            This gets tricky with things like blockchains.
        • lapinot 8y ago
          We regularly confuse irrevocability of licenses and actual availability. The former is something we here probably agree cannot not make sense (once the beans are spilled there's no going back, you can't withdraw intellectual property: what you gave is now irrevocably tied to the recipient, they gained knowledge). The latter is clearly wrong: there's no such thing as universal availability. As humans behinds keyboards we pretty much decide in any way we want what part of (information about) our knowledge we share.
• bo1024 8y ago
  What is it? My browser (firefox) sees nothing* even after enabling javascript. *Except the sentence "The decentralized web is everywhere, but we have to find it." and a Name form that does nothing.
  • trumped 8y ago
    the problem is not firefox... it's probably addons or your internet connection that blocks content
    • superkuh 8y ago
      It is also nothing for me in a Firefox fork with almost all add-ons disabled. Error console says there's a bunch of "class is a reserved identifier" and syntax errors of missing semicolons (before main() can even run). My guess it's trying to use some bleeding edge emcascript stuff which borks the parsing in non-bleeding edge browsers.
      • trumped 8y ago
        oh... I forgot, I'm using the beta version of Firefox, that might be why.
        • mitraardron 8y ago
          If you run into problems, I'd love to know specific details - this is an experiment to see what was achievable with Dweb tools like IPFS, WebTorrent, Gun on browsers, with a big site like the Archive. It certainly pushes the edge of browsers - and does use the latest ES6 features without much attempt to support older browsers (unlike the production site at archive.org). We only currently test in the latest Chrome & Firefox, though it seems to work (mostly) on current Safari on iPad and iPhone and I heard its working on Android though I haven't seen it yet. If you can get on, there is a feedback button, if you can't feel free to go straight to the form at https://docs.google.com/forms/d/e/1FAIpQLSe7pXiSLrmeLoKvlDi2wODcL3ro7D6LegPksb86jr5bCJa7Ig/viewform https://docs.google.com/forms/d/e/1FAIpQLSe7pXiSLrmeLoKvlDi2... or open an issue on https://github.com/internetarchive/dweb-archive/issues https://github.com/internetarchive/dweb-archive/issues .