4 ms·
I'm building a decentralized ai marketplace and guess what? This piece of software is the backbone of our off-chain file transfer protocol. It just works; howev
by m00dy 4y ago
I'm building a decentralized ai marketplace and guess what? This piece of software is the backbone of our off-chain file transfer protocol. It just works; however, its seeder (BitTorrent-tracker) depends on node-webrtc, and it is not an official node package for web-rtc. It has some weird issues. For example, when you seed the torrent through BitTorrent-tracker, and on the client side (chrome browser), I used chrome://webrtc-internals to debug rtc connections. First time in my life, I coined the term connection leakage. A bug in the WebRTC handshake leads to too many connections (300+) for even 1kb torrents.
I hope it gets solved very soon.
I also attach my node profiling output. Looking for some advise if you have any..
[C++]:
ticks total nonlib name
5511 16.9% 43.2% epoll_pwait
692 2.1% 5.4% __pthread_cond_timedwait
682 2.1% 5.3% __lll_lock_wait
413 1.3% 3.2% __GI___pthread_mutex_lock
338 1.0% 2.6% __GI___pthread_mutex_unlock
128 0.4% 1.0% __write
84 0.3% 0.7% __pthread_cond_broadcast
83 0.3% 0.6% __lll_unlock_wake
62 0.2% 0.5% __mprotect
[Summary]:
ticks total nonlib name
153 0.5% 1.2% JavaScript
8206 25.2% 64.3% C++
116 0.4% 0.9% GC
19811 60.8% Shared libraries
4411 13.5% Unaccounted
[C++ entry points]:
ticks cpp total name
692 27.3% 2.1% __pthread_cond_timedwait
679 26.8% 2.1% __lll_lock_wait
413 16.3% 1.3% __GI___pthread_mutex_lock
337 13.3% 1.0% __GI___pthread_mutex_unlock
100 3.9% 0.3% __write
84 3.3% 0.3% __pthread_cond_broadcast
82 3.2% 0.3% __lll_unlock_wake
52 2.1% 0.2% __mprotect
- ThaUnknown 4y agohey, I'm one of the few remaining active WebTorrent devs and I've seen this issue myself, to the point where it's simply easier to use NWjs or Electron with node integration and use chromiums wrtc implementation instead of the wrtc package, if you ever manage to find a better replacement for wrtc for node tell me, and I'll make sure it's implemented ;p
- porsager 4y agoPerhaps node-datachannel could be a viable alternative? https://github.com/murat-dogan/node-datachannel https://github.com/murat-dogan/node-datachannel
- Sean-Der 4y agoI am a big fan of https://github.com/shinyoshiaki/werift-webrtc https://github.com/shinyoshiaki/werift-webrtc it is pure Typescript. Will be easier to debug/deploy. Lots of other great WebRTC implementations also exist. Check out https://github.com/sipsorcery/webrtc-echoes https://github.com/sipsorcery/webrtc-echoes for all the other implementations if you are open to non node.js
- PaulBGD_ 4y agoI originally went the same route as you, and found that https://github.com/pion/webrtc https://github.com/pion/webrtc is probably the best package out there for webrtc. I learned go just for it, and it paid off tenfold. Less memory, more connections, lower latency.
- Sean-Der 4y agoThank you so much PaulBGD! It means a lot that you have been consistently happy with Pion. We got super lucky that a company heavily depended on SCTP in the early days. One of their devs (enobufs) did an amazing job with it.
- Uehreka 4y agoSame (I tried Node-based WebRTC packages), same (I found pion to be head and shoulders above everything else), same (I learned Go just so I could work with it). Its ability to be simultaneously low-level and easy to work with is truly something else.
- anacrolix 4y agoI have this problem with WebRTC too. I'm operating from the native side with anacrolix/torrent and pion/webrtc. I wonder if it's not due to the JS implementation.
- Y_Y 4y agoCan I buy some of your stock?
- m00dy 4y agoyes, write me mod.moody@gmail
- greatest-ape 4y agoIf you run your tracker on Linux and an info hash whitelist approach would work for your use case, it might be worthwhile to have a look at aquatic_ws [0]. It relies on tungstenite [1] for websockets and achieves around 20x the throughput of the reference implementation when running with four threads. I don’t know of anybody who runs it in production, but its sister implementation (BitTorrent over UDP) handles 80k responses per second with lower load averages than opentorrent. [0] https://github.com/greatest-ape/aquatic https://github.com/greatest-ape/aquatic [1]: https://github.com/snapview/tungstenite-rs https://github.com/snapview/tungstenite-rs (I am the developer of aquatic)
- m00dy 4y agoOh very cool. It looks like it is compatible with WebTorrent. Very cool, thanks