11 ms·
Show HN: Bt – BitTorrent library in Java 8
- atomashpolskiy 9y agoThere are many great BitTorrent clients out there, yet I've made my own and willing to share and attract collaborators!
- niroze 9y agothank you!
- atomashpolskiy 9y agoGuys, I've added a code screenshot above the gif, is it better now?
- yodsanklai 9y agoJust curious, how many hours do you think you've spent on this project?
- atomashpolskiy 9y agoI'd say something in the range of 500-1000 hours.
- olalonde 9y agoThat's less than I was expecting. Did you study any existing library to get up to speed?
- atomashpolskiy 9y agoNo, after I had studied BEP-3, wiki and and a few papers (most importantly "BitTorrent economics"), I had a very clear understanding of what needs to be done.
- zitterbewegung 9y agoIt looks great but the top GIF makes it seem like you are offering a command line BT client. If I were you I would have it slowly scroll through the code of the client in the GIF or put a few frames of code of the client and or some compiling and then show the current top GIF.
- atomashpolskiy 9y agoOh, I didn't expect this :) Sounds good, but I'm not sure I have patience do this right, it's such a pain to record gifs - I always misspell or misclick something, and by take #47 go completely nuts
- zitterbewegung 9y agoHow about put a picture of source code above it then?
- deft 9y agoTry using asciinema. It records your terminal session and works so well. Of course typos and such are still an issue...
- maximumkek 9y agolol java
- jph 9y agoThank you, your work is terrific! Good docs, good code, and I especially like the ability to create custom code to write to alternative storage.
- atomashpolskiy 9y agoThanks!! Yeah, I was pursuing such design that would allow to replace/customize most of the parts, including the internals. As a bonus, it makes it easier to continue working on the library as the codebase grows (now nearing 25K lines of code)
- styfle 9y agoI'm curious, it looks like the screenshot shows the author is using a Mac. But Mac OS hasn't shipped with Java in several years. So why Java? Is it simply familiarity, or are there other reasons?
- pvg 9y agoMost OS's don't ship with a C compiler, go, python 3, rust, arc, etc, what does that have to do with anything?
- soperj 9y agoTechnically _most_ ship with a C compiler and a python compiler since pretty well every linux distribution & bsd distribution has those. But that really doesn't have anything to do with which language you should use.
- pvg 9y agoTechnically, that's not true.
- weberc2 9y agoYou're conflating developer tools with a language runtime. Every popular OS ships with everything necessary to run a C, Go, or Rust binary. Python 3 and Java both require a separate runtime to be installed, but I will say that every Java CLI I've used has had terrible startup performance.
- atomashpolskiy 9y agoI'm using Lanterna, a Java library for creating text-based terminal GUIs (https://github.com/mabe02/lanterna https://github.com/mabe02/lanterna), and the startup is 1-2 seconds. Can't recommend it enough
- weberc2 9y agoGood to know, but 1-2 seconds is still very slow compared to a native app. I have an 8MB Go CLI application that can startup and print its help in .05 seconds, and even that is probably slow compared to a C or Rust equivalent.
- gigatexal 9y agoMan this looks slick, going to give it a shot. Love the ability to be able to download a magnet link from the CLI and not worry about what some fancy GUI is doing in the background cuz I am paranoid.
- atomashpolskiy 9y agoThank you! Make sure to install http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html http://www.oracle.com/technetwork/java/javase/downloads/jce8... to allow 160-bit cryptography and use -e flag to be completely safe! I also have a simple shell script locally, that wraps the java command, so I have to just type `btmagnet AF0D9AA01A9AE123A73802CFA58CCAF355EB19F8` to download to a pre-determined location (hardcoded in the script). I wonder if I should share it on the CLI README page
- gigatexal 9y agodo it! that'd be my use case more or less.
- atomashpolskiy 9y agoDone: https://github.com/atomashpolskiy/bt/tree/master/bt-cli#shell-wrapper https://github.com/atomashpolskiy/bt/tree/master/bt-cli#shel...
- rena-anju 9y agoOr add something like this to your .bashrc function bt() { java -jar /path/to/bt-launcher.jar -m "magnet:xt=urn:btih:$1" -d /$HOME/Downloads }
- sarnowski 9y ago> Make sure to install http://www.oracle.com/technetwork/java/javase/downloads/jce8.. http://www.oracle.com/technetwork/java/javase/downloads/jce8.... to allow 160-bit cryptography Or don't use the Oracle proprietary distribution but the OpenJDK of your operating system which normally doesn't exclude proper crypto algorithms. (be aware that the JDKs downgrade silently if certain libraries are missing in your host)
- wodencafe 9y agoWow thanks man! This can be useful in a variety of circumstances!
- amzans 9y agoIt's great to see projects like this on HN. Congrats man, well done!
- yodsanklai 9y agoNice work. I'm wondering, how would you test/benchmark such an application. Besides manual testing on a variety of torrents, how would you do regression test there? same question for benchmarking. How to compare this work with another client?
- atomashpolskiy 9y agoI have several hundred unit tests and also a bunch of integration tests. While UTs usually test API of individual classes, each of the ITs creates a swarm of local peers and launches a torrenting session with certain conditions: seeders to leechers ratio, downloading from .torrent file vs using a magnet link, PEX enabled/disabled, encrypted vs raw message streams, etc. Benchmarks (like in libtorrent) is something I'm looking forward to, but the project is still in the early stage -- there's a lot to be done with regards to optimization, i.e. switching to using NIO selectors, adding a caching layer, etc. That's a lot of work, and that's why I'm looking for collaborators, before investing more time into performance-tuning.
- silotis 9y agoI would advise against implementing a custom cache for piece data. Caching file data in-app is counterproductive because it wastes memory duplicating data which the OS already holds in its page cache. Instead memory mapped I/O should be preferred.
- atomashpolskiy 9y agoIt makes sense, what would you advise in case of big files that don't fit into memory (or when it's undesirable to use big amounts of RAM, and speed can be sacrificed)? Maybe caching should be disabled by default but with the option to turn it on if the user wants it?
- the8472 9y agoyou can memory-map files in chunks and keep references to the mapped buffers or evict them once a certain limit is reached. note that mapped buffers are not necessarily backed by RAM, they act more like swap space, except it goes straight to the original file instead of a swap partition. the downside is that anything reading from the buffer can unpredictably incur IO overhead if there's memory pressure and the OS decides to not back them with memory. java doesn't have an api to check residency of buffers. a possible workaround is to first queue them up on a separate thread and forcing them in before using. if they're already there it should have low latency. if they aren't then only the victim thread eats the IO penalty.
- tonyrice 9y agoIt's amazing how this made it to the front page of HN and I had no idea until now. I actually discovered this by googling a couple hours ago :)
- segmondy 9y agoAmazing work, how long did it take to build? Where you working on it full-time or part-time?
- atomashpolskiy 9y agoThank you! Calendar year, in the evenings and on weekends, with sprints of several weeks and a month or two of rest/AFK after each sprint to keep sanity (I have a full-time job, a wife and a 3 year old kid). Happy that I had the persistence to keep working on the project
- segmondy 9y agoVery impressive! Thanks for sharing, it's encouraging to me and I hope to others to know that with proper time management we can put out amazing work.
- djhworld 9y agoReally like the approach around using Guice as a central point for hooking your own code into the system. Will look forward to reading through the code later!
- atomashpolskiy 9y agoThanks! God bless IoC, it saves my ass each time I make some silly design choice
- nunobrito 9y agoHello, thank you for writing the library. On my context it is useful for downloading large files without depending on a single point of failure or fixed location. Have a question in regards to the example syntax: client.startAsync(state -> { if (state.getPiecesRemaining() == 0) { client.stop(); } }, 1000).join(); This is a feature from newer Java syntax that I'm not yet familiar enough with lambdas to understand in full what is happening. Perhaps I can ask for an example that replaces the "->" into the older Java syntax? I know this would likely come at the cost of a few extra lines, it would certainly ease a lot for many developers looking at the example. Many thanks.
- train_robber 9y agoIn the older syntax, something like this: client.startAsync(new Consumer<TorrentSessionState> { void accept(TorrentSessionState state) { if (state.getPiecesRemaining() == 0) { client.stop(); } } }, 1000).join();
- nunobrito 9y agoThank you. Really useful.
- tokenizerrr 9y agoJust read the Java 8 docs.
- nunobrito 9y agoSome developers work on large code bases where adding lambdas or fresh new syntax features isn't really seen as an advantage for the rest of the teams, most often explicitly restricted. I can only thank those who wrote such obfuscated code in previous years, that eventually forced everyone else afterwards to write code as simple and clear as reasonably possible. That's the reason why I've asked for the simplified code, Might as well save the time for other developers on similar situation.
- 9y ago
- ikurei 9y agoNot very familiar with the Java world yet. I've heard Android now supports at least some features from Java 8, but would this library work in an Android app?
- atomashpolskiy 9y agoI guess not. There is a ticket for backporting this project to Java 7 or RxJava (https://github.com/atomashpolskiy/bt/issues/7 https://github.com/atomashpolskiy/bt/issues/7), but I personally think it's not worth it.
- intellix 9y agoI remember using Java based client Azureus and the memory usage was huge. Overnight it would leak to about 1Gb. Later uTorrent appeared and it used 2mb... Forever I guess nothing to do with this but was triggered by two combined words: Torrent and Java
- alexanderbisc 9y agoIt's possible in Azureus, but it might be a lot of effort to turn the functionality into a library.
- flarco 9y agoWonderful Lib!