14 ms·
On Dat
- deanclatworthy 7y agoI tried beaker when it first featured on HN. My website isn't particularly heavy. It completely crashed the client trying to upload it. We're talking < 100mb of data. I believe they're still working on the issue [1]. [1] https://github.com/beakerbrowser/beaker/issues/952 https://github.com/beakerbrowser/beaker/issues/952
- pfraze 7y agoThe next major release of Beaker (0.9) is going to have a new version of Dat that improves scaling quite a bit. Mafintosh and his team has done some amazing work using trie indexes for much faster reads across the network. Beaker's also gotten much better at syncing with folders, which was the issue in your link.
- deanclatworthy 7y agoI believe the issue was with the client, not the protocol :)
- fiatjaf 7y agoBeaker didn't run on my Ubuntu. I opened an issue and was ignored. I guess Dat is too much JavaScript to be worth it anyway. The tooling and documentation are very poor, most of stuff is outdated. They claim to have a vibrant community but it's mostly two or three guys loving their Beaker browser and caring none about the rest of the world. Perfectly fine for a closed community, but not for a protocol that should be aiming widespread adoption.
- llao 7y agoHaving to stay on a website 24/7 to get new content sounds not very usable or user-friendly.
- ShorsHammer 7y agoGreat for engagement metrics though.
- kickscondor 7y agoHaha! :) Actually, as long as one peer is connected 24/7, everyone will see everything. Which seems reasonable.
- SturgeonsLaw 7y ago> The prolonged obsolescence of distributed protocols like Bitcoin and torrents means we’re maybe skeptical or jaded about any new protocols Funny choice of examples, personally I would have pointed to bitcoin and bittorrent as two examples of distributed protocols with runaway success
- ahje 7y agoYeah, Bittorrent is actually getting fairly old compared to many other technologies in use today; it's _older than_ Windows XP. :D
- cm2187 7y agoA newcommer compared to nntp, which arguably is a kind of p2p network!
- _Codemonkeyism 7y agoIsn't nntp more of a store and forward network?
- josteink 7y agonntp relied on central servers which clients connected to, so not fully decentralized. That the servers did inter-server replication and forwarding doesn’t really change that.
- ptman 7y agoFederation (email, nntp, matrix) is decentralization, but it's not p2p.
- cm2187 7y agoBut the servers are p2p between them, i.e. they propagate the content to the other servers. If you take down one of them, you can still connect to any other server to get the same data. Keeping in mind this is 1980s technology, it looks pretty close to p2p to me.
- tannhaeuser 7y agoWhile we're at it, could someone share their experience with Secure Scuttlebutt vs dat vs ipfs? My interest is to have the simplest thing that could possibly work to make existing static and moderately dynamic sites p2p-publishable. From what little I know, ipfs seems too heavyweight, requires/repurposes git with GPL license, etc. and isn't rooted in HTML/http concepts, while dat/beaker is at least targetting my use case directly. But I'm open to get convinced otherwise.
- langitbiru 7y agoDon't forget Swarm as well. https://swarm.ethereum.org/ https://swarm.ethereum.org/
- bennofs 7y agoI don't have a lot of experience with these, but I tested each of them a little bit and know mostly how they work. I believe that for small sites, the main aspect that differs between them is how peer discovery/name resolution is done. For this, IPFS has the most distributed approach (using a DHT and IPNS) but is also the most fragile and highest latency one. DAT has plans to switch to a DHT in the future (hyperswarm) but relies on centralised DNS discovery servers + local network multicast right now. Secure Scuttlebutt takes a more social approach and organises its network in terms of pubs. There is no central discovery, you can only find users that you "meet" in a pub. This means that DAT has the most website-like feel because their solution provides low latency (only DNS lookup) and is global. I also think that DAT is much more pragmatic compared to IPFS thus at the moment it is much more stable and its us ability is better (also because it has more focus on stable, core components compared to IPFS which is a bigger project trying to do lots of things at once). I like secure scuttlebutt's idea of focusing on communities but that is a different approach to how the web works right now.
- kickscondor 7y ago> Secure Scuttlebutt takes a more social approach and organises its network in terms of pubs. There is no central discovery, you can only find users that you "meet" in a pub. The caveat is: those users can’t meet _you_. I used SSB for a few months, joining pubs, commenting, trying to participate. No responses. (Not a huge surprise—you can get lost in the piles of people on Twitter, Reddit, etc.) However, eventually I discovered that no one could see my contributions unless they added me—and SSB (or Patchwork, in this case) gave me no way of advertising my presence. This was pretty self-defeating. So now I don’t have to just build a ‘presence’ inside the network, I also have to build a ‘presence’ outside the network to announce that I’m somewhere inside the network. The SSB tools also give you no inkling that this is the case. So just know to bring friends!
- hyperion2010 7y agoOn the question about how a search engine would work. The Dat team have thought (and implemented) quite a bit about building performant indexes for distributed file systems. I've wondered if we relaxed the time requirements for search whether there are other processes that we could use that would e more robust. "Who has 10.19.20.13?" seems like it might be a little bit too low level, but what if it was "Who has the most reliable data about the publishing industry between 1500 and 1600?" expiration in 30 minutes with an extension of up to 90 minutes for each preliminary result with more than two nodes responding. Could it work? What kinds of use cases could it support? How would a bot that had a good model of which forums to post things to do in comparison if it had 2 hours, 24 hours, 1 week? What about if we put _lower_ bounds on the amount of time that had to be spent looking (this seems like a fun challenge for proof of 'real' work). While this is at a much higher level than the current dat protocol, it is in a sense quite similar to the query "Who has 353904391670d2803b34990e37f4d2e96f49351998e162d0e335b16812daf592e0f71470af7bee31f6a1da03744d03bcde659d73a0ebf56fd4a9fc6ef67edf60 that is 5 bytes long?"
- saurik 7y ago> "Who has the most reliable data about the publishing industry between 1500 and 1600?" So, to verify I understand: the backend of this would be handled by grad students (working with some kind of 30/90 minute timer alarm), and would experience distributed consistency failures caused by differences in which departmental tradition they did their major area exams?
- hyperion2010 7y agoPretty much. Though for that question you might have to go all the way up to tenured faculty, which could incite a full on Byzantine war over which particular source was more reliable, however I think at that point it is clear that we have lucked out and found two really great sources of information instead of just one and potentially triggered the creation of a third :)
- pfraze 7y agoPart of the 0.9 release of beaker will be the https://unwalled.garden https://unwalled.garden spec, which is a bunch of file formats geared toward improving discovery and search (and general applications). Beaker 0.9 has some internal FTS indexes that let you query content published by people you follow. It's a kind of social search.
- AgentME 7y ago>IPFS is really cool—but how do I surf it? If a site has a domain that's set up with a IPFS DNSLink record, then you just go to its domain name and access it over HTTP(S). But if you have the IPFS companion extension, then your browser will access the site through IPFS peers instead of HTTP. So even if the site's webserver is down, you'll still be able to access the site if any other IPFS peers have its content pinned. You could then pin the site's content and help host it. This all would work through a normal web browser, and gracefully degrades to a plain HTTP connection to a web server for people without the IPFS companion extension. If you have a domain name and are able to keep your site content available in IPFS (on your own machines you keep up, on a pinning service, on a vps, on some friends' machines who you convince to pin it, etc), then you could have someone else host a ipfs web gateway for your domain scoped to your domain such as Cloudflare (https://www.cloudflare.com/distributed-web-gateway/ https://www.cloudflare.com/distributed-web-gateway/). Then you don't need to keep your own web servers up, and the only maintenance you have to do for the ipfs web gateway on your domain is to keep your IPFS DNSLink record correct. Regular users will access your content through Cloudflare (who fetch it from IPFS and then aggressively cache it), and IPFS companion users will access it directly through IPFS. There's also the concept of IPFS links (like ipfs://QmQB1L5PDwcEcMW6hWcLQrNMTKWY3wxX4aDumnKi385KPN/introduction/usage/), which you can access directly if you have the IPFS companion extension, or you can access through a public ipfs web gateway (like https://ipfs.io/ipfs/QmQB1L5PDwcEcMW6hWcLQrNMTKWY3wxX4aDumnKi385KPN/introduction/usage/ https://ipfs.io/ipfs/QmQB1L5PDwcEcMW6hWcLQrNMTKWY3wxX4aDumnK...). But you probably don't want to give links like either of those to your users unless you don't care for domain names. I think the IPFS documentation makes a mistake by emphasizing raw IPFS links so much as opposed to DNSLink records.
- xchaotic 7y agoToo complicated for very little gain.
- AgentME 7y agoAssuming Cloudflare is being used, then the effort on your side is to run "ipfs add -r directory_of_your_site_content" on a machine that stays up (or a pinning service), and then you copy the hash into a DNS record. The gain is that you don't manage any webservers, and other users can help host your site content and can keep your site alive after you stop hosting it on ipfs. I find the concept of being able to make a site that outlives my ability to host it (and hopefully outlives me) as long as people find it worth pinning is super interesting.
- hobofan 7y agoThe thing that always rubbed me the wrong way was how much Dat seems to embrace mutability. That the content behind every URL is mutable is one of the things I dislike the most about the current internet. A lot of the content and data I consume on a daily basis is rarely-changing, so it would be fine to be immutable. I feel that dragging along all the problems of mutability-by-default that plague the current internet when rebuilding it in a more decentralized way is a big mistake. Yes, I know that there are some fringe efforts to enable more immutability with "hypercore-strong-link" etc., but they look like an afterthought, that won't be supported by most of the Dat ecosystem if it finds major adoption.
- macawfish 7y agoInteresting! Immutability by default is what initially rubbed me the wrong was about IPFS and SSB. I've come to see now though that both have valid uses. This is a nuanced issue related to privacy and user safety.
- ComodoHacker 7y ago>A lot of the content and data I consume on a daily basis is rarely-changing, so it would be fine to be immutable Would you want to see typos corrected, facts checked and other improvements on that content?
- hobofan 7y agoWould you want to see code you depend upon arbitrarily change, content you linked to become invalid through a 404, etc.? I didn't say "mutability needs to be completely eradicated". It's mutable-only (what we currently have) vs mutable-by-default vs immutable-by-default. Each of them has their own trade-offs.
- pfraze 7y agoFWIW Dat is going to add "strong links" in the near future, which are links that include a version & content-hash for pinning to a specific version.
- Crinus 7y agoIf you need a custom browser for this to work then why not get rid of the entire HTML+JS+CSS+Wasm+whatever else complexity monstrosity and not build something much simpler based on almost three decades worth of hindsight?
- alex_duf 7y agoBecause I suspect whatever solution they would come up with will end up being as complex a monstrosity when trying to interoperate with other browsers, other OS, other organisations, other developers, various form factors and devices etc... Also, if I need to learn a whole new stack to use a whole new product that has no-one using it yet, chances are I won't bother. We may win on learning from the past and not having some of the tech debt the web has, but I've seen this industry repeating the same mistakes over and over again so I wouldn't bet on it too much.
- Crinus 7y agoWhat i meant is getting rid of html,css,js,etc and the entire stack and replacing it with something simpler. There isn't an issue with interoperability with other browsers as there wont be any other browsers, there is no need to interoperate with other organizations as there wont be any other organizations and any interoperability with operating systems can be addressed by simply using cross platform tech when building the browser. You may not bother but others will (and really you can say that about anything that tries to come up with something new, that is not a reason to stop trying to come with new stuff).
- pfraze 7y agoYesterday I was making a bunch of joke marketing lines and "Sorry we still use HTML" was one of them. It's really just a matter of constraining the novelty. We're trying to make some specific improvements on the applications & networking stack of the Web. Redoing the entire web platform is out of scope (for now).
- jhoechtl 7y agoSeriously, replace it with stgh. which is unfriendly to advertisment, yet still enables a freemium web experience. Ipfs with filecoin sounds resonable at first sight.
- m4r35n357 7y agoWhy do we need node?
- gioscarab 7y agoHas anyone looked into PJON? I see a lot of work done in the direction of decentralization, but is all at a really high level, shouldnt we re-work the layer 2, 3 and 4 to be compatible with a decentralized structure before thinking about content ids? See https://github.com/gioblu/PJON https://github.com/gioblu/PJON
- macawfish 7y agoSome of this high level stuff is really important for thinking about stuff like data structures and "asynchronous ux design". These problems don't strictly depend on the exact underlying transport or discovery mechanisms, although there is definitely work to do there as well! Thanks for sharing this interesting thing I'd never heard of!
- gioscarab 7y agoCiao macawfish absolutely :) the only issue is: until we don't rework the existing low level standards and we build our own private physical network infrastructure (each user takes care of his own router and provide others with connectivity for free without intermediation of any third-party or service provider) we will not have a truly "decentralized" network.
- 1023bytes 7y agoI just tried the Beaker browser and their own main site is currently unaccessible through it.
- pfraze 7y agoThanks for mentioning it, I just deployed a fix
- tekknolagi 7y agoI wrote a small essay about Dat for a course on distributed systems: https://bernsteinbear.com/dat-paper/ https://bernsteinbear.com/dat-paper/ In it I do some analysis of its strengths and weaknesses.
- _bxg1 7y agoThis blog has a really cool page layout
- marknadal 7y agoMulti-writer not working, and still requiring hacking workarounds? This bothers me. P2P Reddit ( https://notabug.io/ https://notabug.io/ ) has been doing this for over a 1+ year. All content there updates in realtime too, fully decentralized. (WebRTC or daisy-chained socket relays) I did not build NAB, but I work on the underlying P2P protocol that powers it ( https://github.com/amark/gun https://github.com/amark/gun ) which competes with DAT - but I've always recommended DAT to people because I thought it already had working multi-writer apps on it. Paul, isn't this already possible? You've shown me demos!
- ddrdrck_ 7y agoI had a look at https://notabug.io https://notabug.io, oh my ... this is a great reminder that full decentralization may not actually be the most desirable thing to pursue.