12 ms·
PeerTweet – Decentralized feeds using BitTorrent's DHT network
- jimktrains2 11y ago> "d": <unsigned int of minutes passed from epoch until when item was created>, I know it's a simple /60, but why change from the more-or-less standard seconds from epoch? To try to squeeze more time out of a 32bit unsigned integer? Since there is no discussion of what size this integer is, I'm expecting you're just defaulting to the json standard, which would support a 53-bit signed integer, iirc.
- sktrdie 11y agoYeah it's just a matter of saving space. Here's the actual code: https://github.com/lmatteis/peer-tweet/blob/master/app/components/Tweet.js#L77 https://github.com/lmatteis/peer-tweet/blob/master/app/compo... I'm really bad at binary arithmetic so I might have done something wrong, but essentially I calculate how many bytes I need to use for the integer, and I store it in a buffer. I might change this if the space required is the same for seconds. I just figured I'd save some space, and I only need minute precision really.
- dTal 11y agoMy personal intuition is that the tiny space savings isn't worth the creation of a new timestamp format that will undoubtedly cause frustration down the line, if this catches on.
- Franciscouzo 11y agoActually JSON doesn't mentions a limit on numbers http://stackoverflow.com/a/13502497/1192111 http://stackoverflow.com/a/13502497/1192111
- patrickaljord 11y agoOne of the top feature of twitter is search, trending topics, easy discoverability etc. In other words, all the goodness brought by centralization (which also comes with all the bad which is censorship, no open/free data etc). The first one to crack this thing (offer both great search and discoverability) and decentralization will win the internet. But it just seems super hard, people tend to concentrate and group no matter how decentralize they start, same thing is happening with bitcoin right now. Not saying it is impossible, but not coming in the near future I think. I sure hope I could be proven wrong though.
- deleted 11y ago[deleted]
- rasengan 11y agoI think a twitter-decentralized-blockchain could solve this?
- jimktrains2 11y agoWhich would also provide a signing key for messages as well.
- toyg 11y agoCheck out twister, it's exactly that: a blockchain-backed twitter, where direct messages are handled through DHT. It is very promising, although it had its birth pains: decentraluzation inevitably brings spammers and squatters...
- deleted 11y ago[deleted]
- pjc50 11y agoDoesn't that require everyone to keep a full copy of the firehose, including all the spam?
- EGreg 11y agoWhy do you need global search? Just have append-only merkle trees with vector clocks.
- patrickaljord 11y ago> Why do you need global search? You must not be a twitter power user, global search is one of its most powerful feature. You can know what's happening anywhere in the world about any topic in real time.
- metasean 11y agoDHT - Distributed Hash Table https://en.wikipedia.org/wiki/Distributed_hash_table https://en.wikipedia.org/wiki/Distributed_hash_table
- autoreleasepool 11y agoI think it's clear that, in light of what's been going on, we need a more diverse ecosystem of P2P networks in order to protect the integrity of our private communications. So I support any effort to experiment in this area
- supergreg 11y agoIs there any library/framework that helps build p2p things? That could help the ecosystem the same way web frameworks have helped build websites faster and more secure than ever.
- asp2insp 11y agoipfs just released their beta which includes a golang library for interacting with their DHT/p2p network
- cdata 11y agoHere is a link to go-libp2p, which I believe is what you are referring to: https://github.com/ipfs/go-libp2p https://github.com/ipfs/go-libp2p
- api 11y agoWe are working on a pluggable SDK version of ZeroTier with a tiny embedded IP stack that will allow p2p using standard unmodified network code, libraries, and protocols. Full interoperability with servers and desktops and other members of virtual networks is also nice. Basically the app is just a "device." An embeddable IP stack sounds fat but it's actually tiny. Whole ball of wax is similar in footprint to other p2p libraries that don't allow normal protocols and full code reuse. I'm a bit partial of course, but the approach is amazingly cool. It will be for Linux, Mac, iOS, Android, and eventually Windows. https://www.zerotier.com/product-netcon.shtml https://www.zerotier.com/product-netcon.shtml Currently GPL with dual licensing for proprietary closed use, which may or may not change later depending on how other things go. Linux and iOS are working. Android and Mac next. This gives you totally easy p2p transport but not things like a distributed database or identity, etc. But there are lots of other projects for that that become easier to use when you have transparent p2p.
- nyan4 11y agoWhat's the threat model here? Is the potential adversary able to do MITM? - The DHT does not anonymize the poster ipaddr, the post contents, posting time and other metadata - same for the reader ipaddr, what was reading, and when
- fiatmoney 11y agoBased on the timing, I would imagine the threat model is ad purveyors taking over a previously relatively open application provider and engaging in soft-to-hard censorship of controversial topics until the only acceptable content is tweeting about how happy you are about #brands and #socialjustice.
- IncRnd 11y agoThat isn't a threat model for this but speculation that this is a mitigation of a threat in twitter's threat model. It isn't clear what threats the design of this system guards against or that much thought was given to the types of attacks so much as to create something shiny and decentralized.
- ilostmykeys 11y agoFor security: this should be combined with something like CertCoin. HTTPS is Evil™ 8)
- rakoo 11y ago> "a": <utf8 http url of an image to render as your avatar>, HTTP, again ? Ah, if only there was a decentralized way to exchange content without the need for any central authority, all while being sure of the authenticity.
- hartator 11y agoYeah, they could have distributed via BT, but that's just an alpha and it's "just" the avatar, the actual tweet contents matter a bit more.
- tracker1 11y agoHmm.. could be a magnet url to a single jpeg image... anyone following someone could actively keep it alive.
- voltagex_ 11y agoYou'd want to put an upper size limit on it - how many Twitter followers do you have? Now x by 40 kilobytes (for example) and x by number of peers you'd be seeding the image to. Could be nasty, especially on mobile.
- tracker1 11y agoProbably... may need to have some tethering ability, so that a mobile app uses an application running on your home desktop, if available. Also, allowing tweaking based on mobile network vs wifi would be a good idea.
- voltagex_ 11y agoI can see a DHT test tool in my future - is there any code to simulate an entire DHT? It'd be worth playing around with something like that.
- rakoo 11y ago
- Humjob 11y agoSuper cool idea - I'll try this out today.
- PuercoPop 11y agohttp://twister.net.co/ http://twister.net.co/ A more mature similar open source implementation of the same idea.
- sktrdie 11y agoTotally appreciate Miguel's work and I've been following them for a long time. I think what they're doing is awesome and people should definitely check Twister out. I should probably have a related work section on the README. Anyway the approaches differ in some areas even though they seem to have the same goal in mind. First off Twister is its own network. They use libtorrent and therefore most of BitTorrent's tech (with some alterations). PeerTweet on the other hand actually uses BitTorrent's mainline DHT network. Then they also include ability to assign names to your feeds based on blockchain mining - again this is their own blockchain. PeerTweet doesn't do that and instead perhaps relies on others to build that functionality on top of it (hint namecoin). I'm not saying that having your own network for everything is bad, I'm just stating the differences :) Also I think we need different tools that try to do the same thing (ex. Twitter) using different methods.
- PuercoPop 11y agoCertainly, I posted the link as an apropos.
- synchronise 11y agoFrom what I've seen, some of the Namecoin devs have distanced themselves from the project because they want to introduce some sort of zero knowledge authentication such as Zerocash to improve anonymity, but because it may be incompatible with the current network they want to focus on either making a parallel or competing solution.
- yazriel 11y agoDoes this create a new DHT entry per tweet or per user ? I think the DHT itself can quickly become overloaded/slow - DHT does impose some b/w overhead for each node
- sktrdie 11y agoOne DHT entry per tweet. I do agree that it does create overload on the DHT. Initially in fact I was looking at implementing it as such: https://github.com/bittorrent/bittorrent.org/issues/19#issue-110508430 https://github.com/bittorrent/bittorrent.org/issues/19#issue... Problem is that you don't get the "hosting" effect that you get with the DHT so you always need a peer seeding your feed or else it won't be reachable. Anyway, I hope others take on the challenge of implementing alternatives looking at different methods.
- yazriel 11y agoActually - i have been playing around with building a feed on top of a single torrent - so it would be one torrent per user - and subscribers to the torrent would (hopefully ;)) get a feed of updates Could this interest u ?
- brokenmachine 11y agoI have always thought that it would be great if the original uploader of a torrent could (securely) modify the torrent so it could have multiple versions, ie if they found one file was corrupt they could update just that file, or add more files (say it was a TV series, they could just keep updating the original torrent with the new episodes). And when you connected to the DHT it could ensure you had the latest version. Of course it would add complexity and I'm not sure how torrent indexing sites would cope but it could be cool. Signed torrents with versioning.
- juanpabloaj 11y agoyou have some example about this?
- reitanqild 11y agoVery interesting, wanted to install and try until I read npm. I do not feel very dumb, but dealing with node takes more energy than I think I have right now. So summarized: still very interested but won't try unless someone explain how to like node.js.
- mrmondo 11y agoI see you've been down voted, however I felt the same way, I loved the idea - then saw that it was in nodejs and while I don't discount it's merits I can't bring myself to try it out if it requires nodejs & friends.
- reitanqild 11y agoThanks, I have no reason to hate Javascript, it's just that I find the Node JS hard and I fail to see the reason for using it for a lot of things. (Then again I found Java hard before I started and someone showed me the basics of eclipse so this is possibly more a cry for help : )
- lisowski 11y agoVery Cool! I've been thinking a lot about decentralized social networks and the value that they could bring. With companies I love like Soundcloud so far in the red, it is only time until they go under or change fundamentally. I think this idea of decentralized feeds fits very well with the current state of music, many different artists, collectives, and labels that basically only exist on the internet. If anyone wants to explore the idea with me I am very open to it!
- levemi 11y ago> PeerTweet Since it's not actually using Twitter and you're not turning into a little bird to share a song can we call writing short messages something other than "tweets"? You might also run into trademark issues with Twitter, but that doesn't bother me as much as grown adults referring to general status updates and conversating as "tweeting", especially when Twitter isn't even involved. We just like need to draw a line in the sand somewhere.
- Klathmon 11y agoDo we? Is it really that big of a problem? If someone says they read a "Tweet", I instantly know that it was a very short message most likely made with very little thought or planning behind it. It's a word, just like any other. This whole idea that we should stick to more vague words because they are "more adult" is silly to me.
- flashman 11y agoCall me cynical but a decentralized alternative to Twitter (or the domain name system for that matter) won't work unless: a) there's a mechanism for brands to get their associated nickname, and b) spammers can't create millions of accounts, or doing so is worthless. Even BitTorrent only really works because central authorities (torrent sites) confer some level of authority on the contents of particular torrents.
- sktrdie 11y agoBitTorrent's DHT is constantly being spammed and has been so since its inception several years ago. The whole point of DHT feeds is to actually filter out spam by only trusting specific feeds. Torrent site owners can publish their torrents on PeerTweet and avoid having to change their DNS every week or so (therefore having to rebuild their reputation).
- jorgecurio 11y agowhat's really holding back the explosion of trustless, decentralized, secure, p2p applications is the very people using the internet. Most people don't give a rats ass or even have the capacity to appreciate something like this and what's going on in the background and why it matters. All it takes is some teeny boppers like a snapchat demographic to adopt new technology. Facebook, twitter, all worked because we were all young once and so was everyone around us. Some applications by nature will not be possible such as Google but one can dream right? A decentralized, distributed Google that's as good as the real thing but without any of the inherent creepiness? Duckduckgo is a real market reaction to the ongoing battle between politics and technology, but it just isn't good enough as the real thing. I think that all in all, decentralizing applications mean a completely new paradigm change in how we value the end user. No longer is a browser fingerprint of some schmuck an exploitable piece of data by advertisers, government bodies. I believe decentralization movement is happening now things are moving far far faster than what I had expected, really underestimated how quickly adoption is measured on the internet...in months if not a year, people's habit drastically change, suddenly a new monster appears and everyone loses it.
- arahant7 11y agoEvery event like the lavabit shutdown, the sybil attack on Tor and even the current Apple issue will add further push to the decentralization movement. Hopefully in the next year or two we will see many completely surveillance proof systems come up.
- cm3 11y agoDo browsers prompt like for mic and camera before opening p2p connections? Nobody wants a letter from the MPAA because a random website's javascript starts downloading/seeding stuff in the background without the user's knowledge.
- uptown 11y agoP2P isn't always mic and camera. Here's an example: (WARNING: Don't open this link if you're on mobile, or on a capped data connection) https://webtorrent.io/ https://webtorrent.io/
- cm3 11y agoThat's what I'm saying. It should also prompt for permission if there's no audio/video involved. Visiting a random site can make you upload and download illegally at the same time. Therefore there's no justification for it to be enabled by default if just JS is allowed.
- im_dario 11y agoI did a decentralized Twitter in 2010 with a very similar design: https://github.com/imdario/qantiqa https://github.com/imdario/qantiqa
- TheAndruu 11y agoReminds me of https://github.com/TrsstProject/trsst https://github.com/TrsstProject/trsst and others like it before, like App.net. This isn't a new idea and I'm 100% behind a more decentralized and free interwebs. The question is what will it take to gain traction. Ease of use is helpful, but there must be something compelling for people to adopt it.