5 ms·

Show HN: Synapse, a full featured BitTorrent client in Rust

195 points by Luminarys 9y ago

floatboth 9y ago
Unfortunate name clash (https://github.com/matrix-org/synapse https://github.com/matrix-org/synapse)
- kibwen 9y ago
  Someone ought to write a website that scrapes popular OSS hosting sites and package repositories and comes up with a list of English words that haven't been taken already. :P
  - djstein 9y ago
    waiting for kibwen to post a Show HN
  - xtagon 9y ago
    Make it, and then name this app "Dictionary" :P
- DennisP 9y ago
  More name clashes: https://en.wikipedia.org/wiki/Synapse_(disambiguation) https://en.wikipedia.org/wiki/Synapse_(disambiguation) Anyone using a common English word shouldn't expect to own it exclusively, especially if it's a word that has something to do with computation.
- culot 9y ago
  Proposal: Rusty Sintorrent
kibwen 9y ago
Not that I'm not always eager to see something get rewritten in Rust, but I'm curious whether web browsers have advanced to the point where a bittorrent client could exist as a webapp rather than requiring native installation. Could the bittorrent protocol operate over WebRTC?
- ronsor 9y ago
  http://webtorrent.io http://webtorrent.io
  - rapfaria 9y ago
    Does it seriously download a 129MB file just by opening the page?
    - CyberDildonics 9y ago
      Ha yeah. Everyone pointed that out the first time this came up and I guess he ignored the feedback.
      - QasimK 9y ago
        I actually find it seriously impressive. It demonstrates it works for you right now, and it works seamlessly!
        yjftsjthsd-h 9y ago
        It is impressive, and in a sense it's a very... effective... demo. The problem is that not all of us are in a position to spend that much bandwidth by accident.
        mook 9y ago
        It's impressive, but it doesn't necessarily actually download the file from webtorrent peers. It will happily do HTTP streaming, instead. It works fine for me even though my browser is configured to block third party traffic by default.
        QasimK 9y ago
        Ah, it's a bit misleading then
    - agumonkey 9y ago
josteink 9y ago
For automation software, transmission (BitTorrent-client written in python, slow) seems to be the standard. If it could offer a transmission-compatible API-layer, I bet it would receive much more interest. Edit: apparently not written in python. My bad.
- sanxiyn 9y ago
  Your wish is satisfied: https://github.com/SirCmpwn/broca https://github.com/SirCmpwn/broca
  - josteink 9y ago
    Nice. Appreciated. Now I clearly have a weekend project. (Also, what great irony this proxy is written in python, eh?)
    - Sir_Cmpwn 9y ago
      >(Also, what great irony this proxy is written in python, eh?) It's written by a different person (me). Also Transmission is written in C FYI
      - josteink 9y ago
        Yes. That was what was ironic since I (wrongly) complained about slow python based software.
        swinglock 9y ago
        The first implementation of BitTorrent was written in Python. I don't think it has happened once since, not as any serious attempt anyway. The clients that use Python today are not slow because they all use rasterbar-libtorrent as a C Python module that performs all of the heavy lifting.
  - abhishekjha 9y ago
    What do I use it for? The README isn't very explanatory.
black_puppydog 9y ago
So, if I understand this correctly, Rust would be a good fit for bittorrent, right? Handling lots of stuff simultaneously, peers, trackers etc, to me a bittorrent that actually does more than download and quit always seems like something quite heavy...
- Sir_Cmpwn 9y ago
  >a bittorrent that actually does more than download and quit always seems like something quite heavy... Don't forget to seed!
  - kibwen 9y ago
    Let's brainstorm ways to solve the seeders/leechers problem via cryptocurrency: how much would it cost to encode a complete feature-length film in the bitcoin blockchain? :P
    - Skunkleton 9y ago
      The user would torrent a scrambled file, once some conditions were met the user would receive the mechanism for unscrambling the file. While the user is seeding the scrambled file they would have an alternate metadata that describes that file. The version that is served could be based on consensus of the network.
    - blasdel 9y ago
      Bram Cohen built MojoNation which was the first real cryptotoken/smartcontract startup After that failed he designed BitTorrent to actually align the incentives of users, without any of the currency bullshit getting in the way
    - leetcrew 9y ago
      a solution already exists. it's called a private tracker.
    - UncleEntity 9y ago
      Seems like having every single full node storing multi-gigabyte files for the few who actually want them wouldn't go over too well.
atomashpolskiy 9y ago
Awesome work, congrats!
mitchtbaum 9y ago
> in Rust Saying what language a program uses only describes its grammar and runtime. Saying what dialect a program uses also describes its idioms. https://en.wikipedia.org/wiki/Programming_idiom https://en.wikipedia.org/wiki/Programming_idiom For (a simple) example, actix-web uses actix, a library that largely changes the nature of writing Rust code. This might be called Rust-a or Rust', or something. https://github.com/actix/actix-web https://github.com/actix/actix-web My point is that Rust is a really cool language where "the(/some) (sub)community(ies)" are developing and coalescing around great libraries outside of core std, making a very* robust language, or rather robust language dialect.
- chc 9y ago
  I'm not all that familiar with actix, but surely not every Rust program is written using a library that largely changes the nature of Rust code. Like, would you say there's a name for the dialect this program is written in?
  - mitchtbaum 9y ago
    Programming dialects don't seem to get names. I think a pattern of [language-name]-[dominant-principles] might make a good name scheme. Suffixes would then work across languages. Other suffix namespaces could coexist, though I'm not sure what they'd be. My point wasn't so much about any one program, so much as it was about an invisible programming ethnology. I like when Show HNs say what language they speak; I would like even more if they say how they speak it. Having name for program dialects would allow prospective users and contributors to gain deep insight about its intended structure, vision, scope, limits, etc, etc, etc, etc from a distance. It may have seemed narrow focused, but I didn't want my point to be limited to one set of principles, patterns, and libraries, even though I like what actix adds to Rust-std(?), especially since any code spoken in it can itself make, or through some elements factored out be made into, other libraries that then build on or toward another powerful dialect / speaker communities. #hashtag: ethnocodology(?) ..ethnokodikology?
stuaxo 9y ago
Interesting ! I tried integrating transmission for downloading content in a video wall app and it was a bit of a nightmare. Hopefully this will be better - please have a sane remote API. One thing that Azeurus did really well, that transmission really did not - You could can tell azeurus to re-check all the files that are there so it can find out how far it's got. In Azeurus I used this to reconstruct a large torrent by adding files that I found, then telling it to re-check. Transmission couldn't even tell if the files the torrent pointed at had been deleted from under it.
- StavrosK 9y ago
  Really? That's odd, that's a pretty standard feature in every client I've tried. I'm not positive, but I'm fairly sure transmission does it too.
- Sir_Cmpwn 9y ago
  >please have a sane remote API https://github.com/Luminarys/synapse/blob/master/doc/RPC https://github.com/Luminarys/synapse/blob/master/doc/RPC Pretty much all torrent clients can recheck the files.
- huntie 9y ago
  Transmission does this with the verify command.
- UncleEntity 9y ago
  The python API for transmission is pretty simple, I wrote a RSS feed reader that adds new torrents in something like 20 lines of code (not counting my custom rss parser module automagically generated from the schema).
Proven 9y ago
Doesn’t seem to solve any new problem
aw3c2 9y ago
It would be great to have a client that works well for seeding massive amounts of torrents (100,000+) on weak hardware (like a Raspberry). The main problem seems to be memory usage. How is the memory usage of this one? It says it is light-weight. What kind of light-weight is it and are there some metrics?
- nly 9y ago
  That's an interesting use case you have there... care to explain the motivation?
  - aw3c2 9y ago
    Wanting to make sure that lots of rare things stay accessible. ;)
- shadeslayer 9y ago
  Even if you solve the memory issues you will still be constrained by the NIC. Most of these SBC's run ethernet over USB, so the performance is still going to be shit.
  - andromeduck 9y ago
    Probably still good enough to saturate the typical American broadband connection though.
  - thisacctforreal 9y ago
    I recently selected an ODROID-C2 for a NAS because of its dedicated Ethernet controller. Cute device. My fingers are crossed for the performance test early this week.
  - aw3c2 9y ago
    If you want to just make sure that lots of content are accessible eventually that is neglible (and I am constrained by ISP anyways).
- Sir_Cmpwn 9y ago

195 points by Luminarys 9y ago

floatboth 9y ago

Unfortunate name clash (https://github.com/matrix-org/synapse https://github.com/matrix-org/synapse)

kibwen 9y ago
Someone ought to write a website that scrapes popular OSS hosting sites and package repositories and comes up with a list of English words that haven't been taken already. :P
- djstein 9y ago
  waiting for kibwen to post a Show HN
- xtagon 9y ago
  Make it, and then name this app "Dictionary" :P
DennisP 9y ago
More name clashes: https://en.wikipedia.org/wiki/Synapse_(disambiguation) https://en.wikipedia.org/wiki/Synapse_(disambiguation) Anyone using a common English word shouldn't expect to own it exclusively, especially if it's a word that has something to do with computation.
culot 9y ago
Proposal: Rusty Sintorrent

kibwen 9y ago

Not that I'm not always eager to see something get rewritten in Rust, but I'm curious whether web browsers have advanced to the point where a bittorrent client could exist as a webapp rather than requiring native installation. Could the bittorrent protocol operate over WebRTC?

ronsor 9y ago
http://webtorrent.io http://webtorrent.io
- rapfaria 9y ago
  Does it seriously download a 129MB file just by opening the page?
  - CyberDildonics 9y ago
    Ha yeah. Everyone pointed that out the first time this came up and I guess he ignored the feedback.
    - QasimK 9y ago
      I actually find it seriously impressive. It demonstrates it works for you right now, and it works seamlessly!
      - yjftsjthsd-h 9y ago
        It is impressive, and in a sense it's a very... effective... demo. The problem is that not all of us are in a position to spend that much bandwidth by accident.
      - mook 9y ago
        It's impressive, but it doesn't necessarily actually download the file from webtorrent peers. It will happily do HTTP streaming, instead. It works fine for me even though my browser is configured to block third party traffic by default.
        QasimK 9y ago
        Ah, it's a bit misleading then
  - agumonkey 9y ago

josteink 9y ago

For automation software, transmission (BitTorrent-client written in python, slow) seems to be the standard. If it could offer a transmission-compatible API-layer, I bet it would receive much more interest. Edit: apparently not written in python. My bad.

sanxiyn 9y ago
Your wish is satisfied: https://github.com/SirCmpwn/broca https://github.com/SirCmpwn/broca
- josteink 9y ago
  Nice. Appreciated. Now I clearly have a weekend project. (Also, what great irony this proxy is written in python, eh?)
  - Sir_Cmpwn 9y ago
    >(Also, what great irony this proxy is written in python, eh?) It's written by a different person (me). Also Transmission is written in C FYI
    - josteink 9y ago
      Yes. That was what was ironic since I (wrongly) complained about slow python based software.
      - swinglock 9y ago
        The first implementation of BitTorrent was written in Python. I don't think it has happened once since, not as any serious attempt anyway. The clients that use Python today are not slow because they all use rasterbar-libtorrent as a C Python module that performs all of the heavy lifting.
- abhishekjha 9y ago
  What do I use it for? The README isn't very explanatory.

black_puppydog 9y ago

So, if I understand this correctly, Rust would be a good fit for bittorrent, right? Handling lots of stuff simultaneously, peers, trackers etc, to me a bittorrent that actually does more than download and quit always seems like something quite heavy...

Sir_Cmpwn 9y ago
>a bittorrent that actually does more than download and quit always seems like something quite heavy... Don't forget to seed!
- kibwen 9y ago
  Let's brainstorm ways to solve the seeders/leechers problem via cryptocurrency: how much would it cost to encode a complete feature-length film in the bitcoin blockchain? :P
  - Skunkleton 9y ago
    The user would torrent a scrambled file, once some conditions were met the user would receive the mechanism for unscrambling the file. While the user is seeding the scrambled file they would have an alternate metadata that describes that file. The version that is served could be based on consensus of the network.
  - blasdel 9y ago
    Bram Cohen built MojoNation which was the first real cryptotoken/smartcontract startup After that failed he designed BitTorrent to actually align the incentives of users, without any of the currency bullshit getting in the way
  - leetcrew 9y ago
    a solution already exists. it's called a private tracker.
  - UncleEntity 9y ago
    Seems like having every single full node storing multi-gigabyte files for the few who actually want them wouldn't go over too well.

atomashpolskiy 9y ago

Awesome work, congrats!

mitchtbaum 9y ago

> in Rust Saying what language a program uses only describes its grammar and runtime. Saying what dialect a program uses also describes its idioms. https://en.wikipedia.org/wiki/Programming_idiom https://en.wikipedia.org/wiki/Programming_idiom For (a simple) example, actix-web uses actix, a library that largely changes the nature of writing Rust code. This might be called Rust-a or Rust', or something. https://github.com/actix/actix-web https://github.com/actix/actix-web My point is that Rust is a really cool language where "the(/some) (sub)community(ies)" are developing and coalescing around great libraries outside of core std, making a very* robust language, or rather robust language dialect.

chc 9y ago
I'm not all that familiar with actix, but surely not every Rust program is written using a library that largely changes the nature of Rust code. Like, would you say there's a name for the dialect this program is written in?
- mitchtbaum 9y ago
  Programming dialects don't seem to get names. I think a pattern of [language-name]-[dominant-principles] might make a good name scheme. Suffixes would then work across languages. Other suffix namespaces could coexist, though I'm not sure what they'd be. My point wasn't so much about any one program, so much as it was about an invisible programming ethnology. I like when Show HNs say what language they speak; I would like even more if they say how they speak it. Having name for program dialects would allow prospective users and contributors to gain deep insight about its intended structure, vision, scope, limits, etc, etc, etc, etc from a distance. It may have seemed narrow focused, but I didn't want my point to be limited to one set of principles, patterns, and libraries, even though I like what actix adds to Rust-std(?), especially since any code spoken in it can itself make, or through some elements factored out be made into, other libraries that then build on or toward another powerful dialect / speaker communities. #hashtag: ethnocodology(?) ..ethnokodikology?

stuaxo 9y ago

Interesting ! I tried integrating transmission for downloading content in a video wall app and it was a bit of a nightmare. Hopefully this will be better - please have a sane remote API. One thing that Azeurus did really well, that transmission really did not - You could can tell azeurus to re-check all the files that are there so it can find out how far it's got. In Azeurus I used this to reconstruct a large torrent by adding files that I found, then telling it to re-check. Transmission couldn't even tell if the files the torrent pointed at had been deleted from under it.

StavrosK 9y ago
Really? That's odd, that's a pretty standard feature in every client I've tried. I'm not positive, but I'm fairly sure transmission does it too.
Sir_Cmpwn 9y ago
>please have a sane remote API https://github.com/Luminarys/synapse/blob/master/doc/RPC https://github.com/Luminarys/synapse/blob/master/doc/RPC Pretty much all torrent clients can recheck the files.
huntie 9y ago
Transmission does this with the verify command.
UncleEntity 9y ago
The python API for transmission is pretty simple, I wrote a RSS feed reader that adds new torrents in something like 20 lines of code (not counting my custom rss parser module automagically generated from the schema).

Proven 9y ago

Doesn’t seem to solve any new problem

aw3c2 9y ago

It would be great to have a client that works well for seeding massive amounts of torrents (100,000+) on weak hardware (like a Raspberry). The main problem seems to be memory usage. How is the memory usage of this one? It says it is light-weight. What kind of light-weight is it and are there some metrics?

nly 9y ago
That's an interesting use case you have there... care to explain the motivation?
- aw3c2 9y ago
  Wanting to make sure that lots of rare things stay accessible. ;)
shadeslayer 9y ago
Even if you solve the memory issues you will still be constrained by the NIC. Most of these SBC's run ethernet over USB, so the performance is still going to be shit.
- andromeduck 9y ago
  Probably still good enough to saturate the typical American broadband connection though.
- thisacctforreal 9y ago
  I recently selected an ODROID-C2 for a NAS because of its dedicated Ethernet controller. Cute device. My fingers are crossed for the performance test early this week.
- aw3c2 9y ago
  If you want to just make sure that lots of content are accessible eventually that is neglible (and I am constrained by ISP anyways).
Sir_Cmpwn 9y ago