14 ms·
BitTorrent v2 (2020)
- pndy 5y agohttps://news.ycombinator.com/item?id=24401999 https://news.ycombinator.com/item?id=24401999 - 2020, 1227 points, 562 comments
- ikt 5y agoi do love that it immediately goes into yet another crypto bashing comment chain while not realising that a crypto company owns bittorrent: https://torrentfreak.com/bittorrent-inc-confirms-acquisition-by-tron-foundation-180724/ https://torrentfreak.com/bittorrent-inc-confirms-acquisition... and that after the creator of bittorrent left he created his own cryptocurrency: https://en.m.wikipedia.org/wiki/Chia_(cryptocurrency) https://en.m.wikipedia.org/wiki/Chia_(cryptocurrency)
- dncornholio 5y agoThat only says something about the company and creator instead.
- Semaphor 5y agoDoes Libtorrent have anything todo with Rainberry fka. Bittorrent Inc.? They seem to be the ones working on the protocol nowadays.
- cuteboy19 5y agoReading through the wikipedia page for Chia > The Chia Network was founded in 2017 by American computer programmer Bram Cohen, the author of the BitTorrent protocol.[4] In China stockpiling ahead of the May 2021 launch led to shortages and an increase in the price of hard disk drives (HDD) and solid-state drives (SSD).[5] Shortages were also reported in Vietnam.[6] Phison, a Taiwanese electronics manufacturing company, estimated that SSD prices will rise by 10% in 2021 as a result of Chia.[7] Hard drive manufacturer Seagate said in May 2021 that the company was experiencing strong orders and that staff were working to "adjust to market demand".[6] In May 2021 Gene Hoffman, the president of Chia Network, admitted that "we’ve kind of destroyed the short-term supply chain" for hard disks.[8] Concerns have also been raised about the mining process also potentially being harmful to drives' lifetime.[9] Thankfully the coin seems to have collapsed completely since May 2021 (https://www.coinbase.com/price/chia-network https://www.coinbase.com/price/chia-network) But you can see why people are skeptical
- maupin 5y agoAnd it wasn't completely original, as Burst (now Signum) was a cryptocurrency using storage space for mining back in 2014.
- ikt 5y ago> Thankfully the coin seems to have collapsed completely since May 2021 That doesn't appear to be the case: https://www.coingecko.com/en/coins/chia https://www.coingecko.com/en/coins/chia Currently rank #257 with a market cap of $247 million. The whole altcoin market has been in limbo/declining since May last year. Last update was also yesterday: https://www.chia.net/2022/01/10/understanding-the-changes-in-the-new-light-wallet.en.html https://www.chia.net/2022/01/10/understanding-the-changes-in...
- 300bps 5y agoThat “crypto bashing comment chain” is just as relevant today as it was when it was posted 16 months ago. People are still struggling to articulate a legitimate use case for cryptocurrencies. I made some money mining BTC way back in the day with a GPU. I bought a new computer for $1,900 last year and much more than paid for it mining ETH. I’ve spent years now listening to all the claims from BTC, BCH, BSV and a dozen more coins and web3 proponents. But I’ll be damned if I can articulate a legitimate use for it that is better than what we have today or that doesn’t fail to take human nature into consideration.
- ikt 5y ago> But I’ll be damned if I can articulate a legitimate use for it What's wrong with DeFi?
- hffftz 5y agoIt would help if HN would automatically link to previous discussions (next to the title?) if there are any... It seems wasteful having people do it manually
- digianarchist 5y agoThat’s what the “past” button does.
- hffftz 5y agoWhy are there always people linking to previous discussions then? Including dang himself...
- GordonS 5y agoWell, it's certainly handy when using a mobile client that doesn't have a "Past" button, like Materialistic :)
- jtbayly 5y agoBecause the past button doesn't actually do that. lol Edit: Oops. I didn't see there are two "past" buttons. I guess one of them does do that. TIL.
- iqanq 5y agoThey were doing it for you because you didn't know about that button. But now that you do, they will stop doing it.
- hffftz 5y agoWouldn't it be easier to create an automated comment for everyone else?
- Dylan16807 5y agoIt would need to change color when there are past discussions with more than a couple comments, or something. The "if there are any" part is more important than the actual links.
- kzrdude 5y agoThe wikipedia page doesn't say - do clients generally support this protocol now? https://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients https://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clien...
- inversetelecine 5y agoFirst bullet on qBittorent's changelog for v4.4.0 is that they do. So I'm sure it's rolling out quickly to others. v4.4.0 changelog: FEATURE: Support for v2 torrents along with libtorrent 2.0.x support (glassez, Chocobo1) - https://www.qbittorrent.org/news.php https://www.qbittorrent.org/news.php
- deleted 5y ago[deleted]
- viktorelofsson 5y agoBitTorrent v2 has been officially supported in PicoTorrent since the same date as the featured blog post from libtorrent :)
- alonmln 5y agoI just love PicoTorrent! It's exactly what I want from a Torrent client. Thanks!
- XzetaU8 5y agoBiglyBT was the first Torrent client with v2 support [1] https://torrentfreak.com/biglybt-is-the-first-torrent-client-to-support-the-bittorrent-v2-spec-201011/ https://torrentfreak.com/biglybt-is-the-first-torrent-client... [2] https://www.biglybt.com/ https://www.biglybt.com/
- viktorelofsson 5y agoNot really true - v0.20 of PicoTorrent [1] was released with BitTorrent v2 support on the same day as libtorrent released support for v2 torrents. I think this predates BiglyBT 2.5 with v2 [2] support with about two weeks. [1] https://github.com/picotorrent/picotorrent/releases/tag/v0.20.0 https://github.com/picotorrent/picotorrent/releases/tag/v0.2... [2] https://biglybt.tumblr.com/post/629947579144290304/2500-released https://biglybt.tumblr.com/post/629947579144290304/2500-rele...
- nodja 5y agoI assume this is being posted because qBittorrent recently released version 4.4 which includes libtorrent 2.x and therefore support for BitTorrent v2. Hopefully we'll start seeing hybrid torrents/magnets in trackers soon. Although these are mostly automated so I'm not sure how fast the tools will catchup. One thing I'd like qBittorrent and other torrent clients to do is to allow users to upgrade v1 torrents to hybrid torrents if they're fully seeded, this would allow cross polination for old torrents where peers are scattered between the original torrent and batch torrents, or where trackers add extra files causing the v1 hash to change.
- KennyBlanken 5y agoA bunch of open-source projects would like to use v2 torrents, but because Transmission is so popular and has zero support for v2, nobody can use them. Transmission's dev team hasn't said a word on the matter, ever, in four years: https://github.com/transmission/transmission/issues/458 https://github.com/transmission/transmission/issues/458 Elsewhere someone quipped: > That transmission bug is 4 years old and they've never announced plans to prioritize it (granted the effort is surprisingly high; they've never allowed empty dict keys in bencoding). I think either they either don't know or don't care that their market share makes them an adoption blocker.
- capableweb 5y agoIn the beginning I used µTorrent, until it got fucked by whatever company that bought it. It was easily the fastest and most resource-smart client I've ever used. After that I tried a bunch, Deluge, Transmission, Azure and plenty more, but only qBittorrent seems to be a suitable replacement (and is open source as well). Now it has BitTorrent V2 support too :) You should really give it a try.
- xd1936 5y agoIronically, the company that bought µTorrent was BitTorrent, Inc[1]. They're the geniuses also responsible for forking IPFS, renaming it to "BTFS", and not acknowledging IPFS as an upstream in any way[2]. 1. https://en.wikipedia.org/wiki/Rainberry,_Inc https://en.wikipedia.org/wiki/Rainberry,_Inc. 2. https://twitter.com/juanbenet/status/1250634833258143744 https://twitter.com/juanbenet/status/1250634833258143744
- dennis-tra 5y agoI‘m wondering if they could avoid yet another breaking protocol change if SHA256 proves to be insecure (at some point in the future) if they made use of Multiformats [0]. At least IPFS went this way (Multiformats grew out of the IPFS development). [0] https://multiformats.io/ https://multiformats.io/
- alex_duf 5y ago>if SHA256 proves to be insecure slight pedantism here, but "when" not "if"
- marcan_42 5y agoThat's not a given. As far as I know, there is no proof of the security or lack thereof of this kind of function. Increases in computing power alone can't render SHA256 insecure; that can be proven by thermodynamic/size-of-the-universe arguments. You need an actual cryptographic weakness.
- newby 5y ago> Increases in computing power alone can't render SHA256 insecure; that can be proven by thermodynamic/size-of-the-universe arguments. Fascinating. Could you please recommend a source where a curious layman could learn more about this?
- marcan_42 5y agoI posted a link and some further analysis in this comment: https://news.ycombinator.com/item?id=29883965 https://news.ycombinator.com/item?id=29883965
- hannob 5y agoThere's no indication whatsoever of this happening. In the time since SHA1 came under attack, a lot more research has happened on hash function security. If this has shown anything, it's that SHA256 is more secure than people used to think. People seem to have this widely believed idea that "all crypto will be broken eventually". But in pretty much all cases you can trace back that broken crypto was known to be weak for a long time. SHA256 is not known to be weak (with the slight caveat that it doesn't protect against length extension attacks, but this is a known property that matters only in very rare circumstances).
- deleted 5y ago[deleted]
- moolcool 5y agoBitTorrent slaps so hard as a protocol, and DHTs are absolutely brilliant. With all of the recent hype about decentralization with the rise of crypto, I'm surprised BitTorrent hasn't seen much hype or revitalization as well.
- nudpiedo 5y agoThe project it's still in pre-alpha, it even stopped paying rewards to its token stakers/seeders due some technical issues during the implementation. Perhaps it just rushed onto it too fast just to catch the hype.
- WithinReason 5y agoNow that I think about it, BitTorrent is the best example of how well decentralisation can work, and basically the only reason I believe that a decentralised web is possible.
- hiq 5y agoUnfortunately BitTorrent is still heavily centralized when it comes to discoverability. Without a website hosting .torrent files or magnet links, it's very limited.
- joconde 5y agoThese links are easy to mirror, they probably fit into a reasonably large SQLite file. Since the Web still allows decentralization, that doesn’t seem to be a problem. For example, the pirate bay is blocked in my country, yet I can still consult it on dozens of mirrors.
- account-5 5y agoThere's always tribler https://github.com/Tribler/tribler https://github.com/Tribler/tribler
- bondant 5y agoI'm surprised nobody has build a torrent catalog on bittorrent itself. I think that with the DHT mutable item one should be able to publish a torrent containing a list of torrent with comments and updating the list easily.
- einpoklum 5y agoCan someone explain how the Merkle-tree concept is useful? I didn't quite catch that part. On the one hand it is says: > all you need is the root hash of the tree. but on the other hand: > the .torrent file must still contain these piece hashes So what are we saving here? If a piece hash doesn't match the downloaded data, we re-download the piece.
- chasil 5y agoThe main use of a Merkle tree that I know of is the filesystem "scrub" operation in ZFS and BtrFS. https://en.wikipedia.org/wiki/Merkle_tree https://en.wikipedia.org/wiki/Merkle_tree
- cyphar 5y agoThe .torrent file is separate to the torrent metadata which you need to fetch for magnet links. This is mentioned a bit later in that section: > The .torrent file size is not smaller for a v2 torrent, since it still contains the piece hashes, but the info-dictionary is, which is the part needed for magnet links to start downloading. Also it seems that it'll be easier to detect peers which are sending bad data and correct it. I guess this is because previously you could fetch a piece from more than one peer but because the hashing was done at the piece level you didn't know which part of the block was wrong (meaning you'd have to redownload the whole thing). But now you can -- with a bit of tracking -- tell which block of a piece was sent from which peer and then redownload it from another (likely blocking the original peer), because you know which block hash failed to validate. (EDIT: This is correct -- [1] explains the issue and how torrent clients have worked around this problem in v1 torrents.) Merkle trees have some other benefits (you can cache parts of the tree in such a way that repeatedly checksumming data where lots of leaves are unchanged is cheaper, and you can efficiently prove that a given leaf was part of the hash), but I don't know if that's going to be useful for BitTorrent. [1]: https://blog.libtorrent.org/2011/11/smart-ban/ https://blog.libtorrent.org/2011/11/smart-ban/
- mixmastamyk 5y agoIs there a good book that explains modern bittorrent?
- WallyFunk 5y agoAnyone know if there's a performance increase in this spec? Couldn't find any reference to speed in the article.
- cletus 5y agoFrom a technical perspective, I love BitTorrent. I remember reading about it years ago and being amazed at the bandwidth efficiency and robustness it had. It still had some centralization (ie trackers) but they themselves could be duplicated (ie you could set up a tracker for an existing torrent). The part that surprises me from reading this is the issues raised from moving from SHA1 to SHA256, specifically because that's 20 vs 32 bytes respectively. This has created compatibility issues. The same thing happened with Git and it's a nightmare. I honestly don't understand how this happens. We've gone through this many times for many years. Replacing your hash function should be built in from day one and for any project to come along in the 21st century and not do this is really gross negligence. I really wonder how this happens. Like is it just hubris about how this hash function will be different? Or are engineers just so in love with the optimizations they get to make by assuming, say, 20 byte hashes? I really wish I knew. Anyway, I'm a little sad BitTorrent only really found traction for pirating media. Whether you approve of that or not, it's still an astounding technical achievement.
- tinaprice 5y agoI totally agree with this one, it just gained traction because of the pirating scene.
- throwaway2048 5y agoif you design a protocol to somehow have "upgradable" hashes, clients still need to be patched to support them, and old clients still will not work. Its the same mistake TLS and IPSEC made, supposedly extensible, but not really because to actually get new features, clients all need to be upgraded, and old incompatible clients still exist, and will never work with whatever new features you want to employ. You aren't really gaining anything, and you might as well just have new versions of the protocol, and save all the pointless ceremony, complexity and security holes. There is no way (beyond maybe embedding some kind of executable code in the protocol itself, which seems like an insane idea) to have support from "day one" for algorithms and functionality that don't even exist yet.
- r1ch 5y agoThe choice to use SHA256 is surprising to me given that there are a number of other hash functions such as the BLAKE family that are explicitly designed to have arbitrary output sizes.
- gkfasdfasdf 5y agoThe per file hash to assist in deduplication is pretty neat. Wonder if it could be taken a step further by chunking files into blocks based on a rolling hash, so that it can dedup portions of a file as well (a la lbfs https://pdos.csail.mit.edu/archive/lbfs/ https://pdos.csail.mit.edu/archive/lbfs/). Or does anyone know if it already does that?
- SethTro 5y agoIt is highly unlikely that the duplicate portions of the file will have an offset that's a multiple of 2^16 which would be required for chunks to have matching hashes. On the client side you could theoretically run lbfs over your files but on the swarm side this isn't going to happen
- Dylan16807 5y ago> It is highly unlikely that the duplicate portions of the file will have an offset that's a multiple of 2^16 which would be required for chunks to have matching hashes. That's exactly what chunking based on a rolling hash solves. You set the average size of chunks and the content controls the exact boundaries.
- gkfasdfasdf 5y agoRight, exactly. Chunk boundaries are not determined by fixed size chunks, but rather when the rolling hash matches some prefix, which means chunk sizes will vary but by controlling the prefix can set the average size of the prefix. Besides the lbfs paper, another nice writeup here: https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/ https://moinakg.wordpress.com/2013/06/22/high-performance-co...
- brokenmachine 5y agoRabin fingerprinting can do this I believe. "the idea is to select blocks not based on a specific offset but rather by some property of the block contents" https://en.wikipedia.org/wiki/Rabin_fingerprint#Applications https://en.wikipedia.org/wiki/Rabin_fingerprint#Applications
- 1970-01-01 5y agoThe next question is "How long will it take copyright lawyers find out about v2-only torrents?"
- ccbccccbbcccbb 5y ago> BitTorrent v2 not only uses a hash tree, but it forms a hash tree for every file in the torrent. Could this potentially make cracking down on illegal content easier?
- niij 5y agoMobile readable version: https://outline.com/ZAAtSk https://outline.com/ZAAtSk